# ADVANCES IN MUSCULOSKELETAL MODELING AND THEIR APPLICATION TO NEUROREHABILITATION

EDITED BY : Naser Mehrabi, John McPhee, Mohammad S. Shourijeh and Benjamin J. Fregly PUBLISHED IN : Frontiers in Neurorobotics

#### Frontiers eBook Copyright Statement

The copyright in the text of individual articles in this eBook is the property of their respective authors or their respective institutions or funders. The copyright in graphics and images within each article may be subject to copyright of other parties. In both cases this is subject to a license granted to Frontiers. The compilation of articles constituting this eBook is the property of Frontiers.

Each article within this eBook, and the eBook itself, are published under the most recent version of the Creative Commons CC-BY licence. The version current at the date of publication of this eBook is CC-BY 4.0. If the CC-BY licence is updated, the licence granted by Frontiers is automatically updated to the new version.

When exercising any right under the CC-BY licence, Frontiers must be attributed as the original publisher of the article or eBook, as applicable.

Authors have the responsibility of ensuring that any graphics or other materials which are the property of others may be included in the CC-BY licence, but this should be checked before relying on the CC-BY licence to reproduce those materials. Any copyright notices relating to those materials must be complied with.

Copyright and source acknowledgement notices may not be removed and must be displayed in any copy, derivative work or partial copy which includes the elements in question.

All copyright, and all rights therein, are protected by national and international copyright laws. The above represents a summary only. For further information please read Frontiers' Conditions for Website Use and Copyright Statement, and the applicable CC-BY licence.

ISSN 1664-8714 ISBN 978-2-88966-204-3 DOI 10.3389/978-2-88966-204-3

#### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

#### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

#### Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

#### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# ADVANCES IN MUSCULOSKELETAL MODELING AND THEIR APPLICATION TO NEUROREHABILITATION

Topic Editors:

Naser Mehrabi, General Motors (United States), United States John McPhee, University of Waterloo, Canada Mohammad S. Shourijeh, Rice University, United States Benjamin J. Fregly, Rice University, United States

Citation: Mehrabi, N., McPhee, J., Shourijeh, M. S., Fregly, B. J., eds. (2020). Advances in Musculoskeletal Modeling and their Application to Neurorehabilitation. Lausanne: Frontiers Media SA. doi: 10.3389/978-2-88966-204-3

# Table of Contents

*04 Editorial: Advances in Musculoskeletal Modeling and Their Application to Neurorehabilitation*

Mohammad S. Shourijeh, Naser Mehrabi, John J. McPhee and Benjamin J. Fregly

*08 An Optimization Method Tracking EMG, Ground Reactions Forces, and Marker Trajectories for Musculo-Tendon Forces Estimation in Equinus Gait*

Florent Moissenet, Colombe Bélaise, Elodie Piche, Benjamin Michaud and Mickaël Begon


Florian Michaud, Francisco Mouzo, Urbano Lugrís and Javier Cuadrado

*51 Design of the Cooperative Actuation in Hybrid Orthoses: A Theoretical Approach Based on Muscle Models*

Francisco Romero-Sánchez, Javier Bermejo-García, Jorge Barrios-Muriel and Francisco J. Alonso

*66 A Quick Turn of Foot: Rigid Foot-Ground Contact Models for Human Motion Prediction*

Matthew Millard and Katja Mombaur


*109 Kinematic Synergy of Multi-DoF Movement in Upper Limb and Its Application for Rehabilitation Exoskeleton Motion Planning* Shangjie Tang, Lin Chen, Michele Barsotti, Lintao Hu, Yongqiang Li, Xiaoying Wu, Long Bai, Antonio Frisoli and Wensheng Hou

*122 Neuromusculoskeletal Modeling-Based Prostheses for Recovery After Spinal Cord Injury*

Claudio Pizzolato, David J. Saxby, Dinesh Palipana, Laura E. Diamond, Rod S. Barrett, Yang D. Teng and David G. Lloyd

*132 Muscle Synergy Constraints Do Not Improve Estimates of Muscle Activity From Static Optimization During Gait for Unimpaired Children or Children With Cerebral Palsy*

Benjamin R. Shuman, Marije Goudriaan, Kaat Desloovere, Michael H. Schwartz and Katherine M. Steele

## Editorial: Advances in Musculoskeletal Modeling and Their Application to Neurorehabilitation

Mohammad S. Shourijeh<sup>1</sup> \*, Naser Mehrabi <sup>2</sup> , John J. McPhee<sup>3</sup> and Benjamin J. Fregly <sup>1</sup>

<sup>1</sup> Rice Computational Neuromechanics Laboratory, Department of Mechanical Engineering, Rice University, Houston, TX, United States, <sup>2</sup> General Motors Company, Canadian Technical Centre, Markham, ON, Canada, <sup>3</sup> Motion Research Group, Systems Design Engineering, University of Waterloo, Waterloo, ON, Canada

Keywords: musculoskeletal modeling, rehabilitation, neuromechanics, neurorehabilitation, neuromusculoskeletal modeling, assistive devices, predictive dynamic simulation, biomechatronics

**Editorial on the Research Topic**

**Advances in Musculoskeletal Modeling and Their Application to Neurorehabilitation**

### INTRODUCTION

Neuromusculoskeletal (NMS) modeling has the potential to serve as a valuable tool for informing the design of assistive devices or neurorehabilitation interventions. For these applications, an NMS model must be able to represent the relevant unique characteristics of the targeted subject, e.g., a person with a neurological disorder. Motion predictive simulations, by considering the task constraints and the neuromuscular requirements, can predict novel movements. Such simulation methods must ideally capture the features and functionalities of the motor control system to mimic human neuromechanical behavior. If a simulation performed with an NMS model fails to accurately predict how a person moves or how that person controls his or her muscles during movement, it will have limited utility for informing device and intervention design. Although, in the past few years, several studies have developed NMS models and techniques to anticipate responses to interventions, e.g., orthopedic surgery, orthoses, or neurorehabilitation, a well-established rigorous framework that can accurately predict neuromusculoskeletal dynamics of healthy and pathologic individuals is still lacking in the research community. Even after the published papers of this topic, there is a strong need for greater research effort in the area of NMS model personalization and personalized simulations of treatment design.

One of the emerging application areas for personalized NMS models is in the design of assistive devices for rehabilitation and augmentation purposes. Integrated human and device interaction models would save time and cost of multiple prototyping steps in both the design and evaluation phases of such devices. Bio-fidelic predictive NMS models could delineate crucial parameters of a design and elucidate the effect of a device conceptual design on an individual's kinematics and kinetics. Therefore, the design process will become human-centered in which a design optimization step includes all important factors, such as mechanical, electrical, material, as well as physiological. The perspective of this Research Topic was to cover the following topics by taking advantage of neuromusculoskeletal models:

Edited and reviewed by: Florian Röhrbein,

V-R-Robotics, Germany

\*Correspondence: Mohammad S. Shourijeh shourijeh@rice.edu

Received: 17 July 2020 Accepted: 11 August 2020 Published: 09 October 2020

#### Citation:

Shourijeh MS, Mehrabi N, McPhee JJ and Fregly BJ (2020) Editorial: Advances in Musculoskeletal Modeling and Their Application to Neurorehabilitation. Front. Neurorobot. 14:65. doi: 10.3389/fnbot.2020.00065


The 10 contributions to this Research Topic provide a wide range of perspectives and methodologies. These high-quality contributions highlight the present potential of a community of researchers interested in an interdisciplinary view of the interaction between neuromusculoskeletal modeling and rehabilitation science to compensate, augment, and restore human function.

## THE CONTRIBUTIONS

Many contributions to this topic combine experimental data with theoretical approaches that were directly related to neurorehabilitation (Pitto et al.; Romero-Sánchez et al.; Sauder et al.) or can provide a better understanding of human neuromechanics for future treatment design (Millard and Mombaur; Seth et al.; Shuman et al.; Moissenet et al.). Experimental human data spanned kinematics from surface markers, ground reaction forces from force plates or instrumented treadmills, and extensive fine wire and surface EMG for muscles. Theoretical techniques ranged from kinematic synergy analyses (Tang et al.) to highly complex predictive neuromusculoskeletal optimal control problems (Sauder et al.; Pitto et al.). Most of the contributions included a form of musculoskeletal modeling/simulation, either by developing a new method (Sauder et al.) or adapting a previously developed technique (.Moissenet et al.; Tang et al.). The assistive devices in the contributed studies ranged from passive orthoses (Michaud et al.) to hybrid active exoskeletons (Romero-Sánchez et al.). There was also a narrative review paper that presented interesting ideas and concepts for the next generation of neuroprostheses (Pizzolato et al.). Overall, the contributions to this Research Topic reveal the wide range of research being conducted in neuromusculoskeletal (NMS) modeling and the essential role it can play in the future of neurorehabilitation.

Below, an overview is provided of the scientific contributions to this Research Topic, which are listed within two main categories:

1) Development of new NMS modeling methods that contribute to the future of rehabilitation treatment design

2) Application of existing NMS modeling methods to rehabilitation treatment design.

## DEVELOPMENT OF NEW NMS MODELING METHODS THAT CONTRIBUTE TO THE FUTURE OF REHABILITATION TREATMENT DESIGN

There are two important challenges in using NMS models in the design process of neurorehabilitation. The first challenge is to personalize generic NMS models to the relevant unique characteristics of the patient, including personalization of the neuromechanical impairment. The second challenge is to simulate the effect of different treatment scenarios on movement performance. Two contributions to this topic (Sauder et al.; Pitto et al.) address these two challenges. Both studies used OpenSim as their musculoskeletal modeling package, modeled neural coordination through the concept of muscle synergies, used pre-treatment data for model personalization, and assumed that some components of the neural coordination stay the same after treatment. Sauder et al. improved a model personalization and treatment optimization pipeline for enhancing the efficiency of fast functional electrical stimulation. Optimal control simulations showed that optimizing muscle selection and stimulation timing results in a 23% improvement in the gait symmetry of an individual post-stroke compared to the simulated results from common clinical practice. Pitto et al. developed a simulation pipeline to explore the effect of treatment by changing muscle and geometry parameters in cerebral palsy children. A graphical user interface was also developed for modifying the musculoskeletal properties according to the surgical plan. Both studies are new doors to the use of predictive NMS modeling for the design and evaluation of treatment.

Foot-ground contact model is an essential element in predictive NMS models of gait. Although the foot-ground interaction is often modeled as a grid of viscoelastic elements, Millard and Mombaur developed and evaluated two rigid footground contact models that have potential advantages, e.g., model calibration is reduced to a geometric problem, and the numerical stiffness of the equations of motion is similar in both swing and stance phase. Both the ellipse-foot and the doublecircle-foot models were evaluated by analyzing ankle angle and the center-of-pressure (CoP) kinematics, accuracy of kinematics in a tracking gait problem, and changes in kinematics during a predicted gait problem. Each model showed pros and cons with the overall capability for use in predictive gait simulations.

To examine muscle function and to predict movement reliably, one must use physiologically plausible musculoskeletal models. The upper extremity is usually affected in neurological impairments such as stroke, and the shoulder is one of the key joints possessing a complex structure as well as a load-carrying role. Seth et al. developed a novel shoulder model and looked at the contribution of work and movement of muscles to shoulder movement. Previous models of the human shoulder had coupled scapula movement to humeral movement. To elucidate the roles of the thoracoscapular muscles, Seth et al. developed a shoulder model that represents the scapulothoracic joint accurately and includes scapular muscles. The authors also showed that the large thoracoscapular muscles do more work than glenohumeral muscles during arm-elevation tasks. Therefore, this model, which along with the experimental data is freely available on SimTK.org, is expected to be applied by researchers, including NMS modelers designing treatments for the shoulder joint.

One of the challenges in the design of active assistive devices is the harmony between the device actuators and the individual's desired natural movement. Romero-Sánchez et al. developed a theoretical framework and a computational model for a hybrid orthosis—also referred to as a rehabilitation exoskeleton, which consists of electromechanical actuation and functional electrical stimulation (FES). The main challenge in these devices is to coordinate and make transparent the two actuators governing each degree of freedom so that their actuation profiles lead to a natural motion, e.g., gait. Romero-Sánchez et al. included muscular dynamics to attain higher neurophysiological fidelity for the rehabilitation purpose for which these devices are designed. This integrated approach estimates the actuation profiles so that the reduction in peak muscle force due to FES-inducing fatigue is compensated for a hybrid hip-kneeankle-orthosis as the assistive device. It is hoped that the study's results and theoretical workflow will contribute to the design of more physiological hybrid exoskeletons that can maintain rehabilitative function of the device by improving its actuation control.

## APPLICATION OF EXISTING NMS MODELING METHODS TO REHABILITATION TREATMENT DESIGN

Moissenet et al. established a computational approach suitable for tracking simulations of healthy and pathological gait. The approach can track experimental data while also allowing for simulation of different variations in the model, e.g., pathology or treatment. A previously developed EMG-marker tracking optimization method was adapted to a lower extremity musculoskeletal model during equinus gait. This dynamic optimization approach tracked experimental marker trajectory, EMG signal, and ground reaction force data. Although this preliminary study was the authors' first step toward their treatment modeling framework, this approach showed its potential to be a candidate for experimentally and dynamically consistent musculoskeletal simulations.

One key aspect of neuromusculoskeletal modeling is muscle recruitment or force distribution problem. It has been hypothesized that the human central nervous system simplifies the construction of muscle excitations during dynamic tasks by constraining those excitations to weighted groups, referred to as muscle synergies. In this Research Topic, Pitto et al., Sauder et al., and Shuman et al. applied muscle synergies in different NMS simulations to address populations with different neurological disorders. While Sauder et al. targeted stroke, Pitto et al. and Shuman et al. looked at or cerebral palsy (CP). While Sauder et al. and Pitto et al. predicted gait post-treatment, Shuman et al. focused on estimating muscle activations and discussing whether estimates generated by imposing a muscle synergy structure improve the correlation with experimental muscle activities. Shuman et al. used a scaled generic musculoskeletal model and compared two cases: (1) static optimization (SO) with minimization of squared muscle activations, and (2) synergy SO (SynSO) with minimization of squared synergy activations and using synergy weights obtained by analysis of EMG data. In SynSO, synergy weights are decomposed from EMG data while the synergy activations are computed one time step at a time, similar to SO. The correlation with EMG data was not found to be higher in SynSO than in SO.

It is a kinematic requirement that rehabilitation exoskeletons move with a spatiotemporal motion that is synchronized with the kinematic structure of the upper-limb joint. Tang et al. analyzed the spatiotemporal kinematic synergies of arm reaching movements and investigated their potential usage in upper limb assistive exoskeleton motion planning. Kinematic synergies—coordination between shoulder and elbow joints—were extracted by running principal component analysis on experimental reaching trials of multiple subjects. Tang et al. concluded that kinematic synergies could be used for exoskeleton motion planning, and different principal components partly contributed to motion trajectory and end-point accuracy. Although this study did not include a kinetic and a neuromuscular model, the concept of kinematic synergies may provide simplified yet worthwhile strategies for initial motion planning and kinematic design of assistive devices to restore natural upper-limb motion and motor function.

Energy expenditure is one of the key criteria used for the design and evaluation of assistive devices in different populations. While total body metabolic energy rate can be measured by means of O2/CO<sup>2</sup> consumption/production rates experimentally, muscle metabolic models could be utilized for two main reasons: (1) It is nearly impossible today to measure the metabolic energy of a muscle experimentally and (2) There may be circumstances in which it is desirable to avoid in vivo measurement of subjects or activities. Michaud et al. adapted their previously proposed musculoskeletal modeling approach for estimating muscle forces/activations to a healthy subject and validated their total body metabolic energy rate estimates from two commonly used models with data from a portable gas analyzer. Then, both metabolic energy rate models estimated energetic efficiency using two types of assistive devices on an individual with spinal cord injury during crutch-assisted gait. The two assistive devices were a passive and an active knee-ankle foot orthosis (KAFO). The authors found that the active KAFO resulted in simultaneous improved gain pattern and reduced energy consumption.

Pizzolato et al. presented a narrative review of how NMS models combined with finite element models of musculoskeletal tissues can be integrated with models of assistive and robotic devices for neural restoration in the SCI population. The authors discussed how NMS models can be deployed in related real-time applications, such as muscle activity optimization, functional progress tracking, balance and safety monitoring, and augmented feedback during rehabilitation. Enhancement of current neurorehabilitation may be achieved via development of the next generation orthoses/prostheses incorporating personalized NMS models of patients.

## CONCLUSIONS

This Research Topic focuses on advances in neuromusculoskeletal modeling and applications to neurorehabilitation. Contributions to this topic utilized both experimental and computational approaches to address research questions involving neuromechanics and rehabilitation. The researchers contributed to the topic and the editors hope that this Research Topic will open new horizons to more direct clinical applications of neuromusculoskeletal modeling. Although research studies of high quality were presented under this Research Topic, we hope that these studies will only be the beginning and that momentum would continue to grow in this research area. We eagerly hope to observe more advances in neuromusculoskeletal model personalization and their application toward personalized interventions to restore, compensate, or augment motor function, such as via gait retraining or design of assistive devices, in neurologically impaired populations.

## AUTHOR CONTRIBUTIONS

All editors of this Research Topic, i.e., MSS, NM, JJM, and BJF, have made a significant contribution to this work and approved it for publication.

## FUNDING

This publication was supported by the Cancer Prevention Research Institute of Texas (CPRIT) under grant RR170026 (BJF and MSS) and the Canada Research Chair in Biomechatronic System Dynamics (JJM).

**Conflict of Interest:** NM was employed by the company General Motors.

The remaining authors declare that the research was conductedin the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2020 Shourijeh, Mehrabi, McPhee and Fregly. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

## An Optimization Method Tracking EMG, Ground Reactions Forces, and Marker Trajectories for Musculo-Tendon Forces Estimation in Equinus Gait

Florent Moissenet <sup>1</sup> \*, Colombe Bélaise<sup>2</sup> , Elodie Piche<sup>2</sup> , Benjamin Michaud2,3 and Mickaël Begon2,3

<sup>1</sup> Centre National de Rééducation Fonctionnelle et de Réadaptation–Rehazenter, Luxembourg, Luxembourg, <sup>2</sup> Laboratory of Simulation and Movement Modeling, School of Kinesiology and Exercise Sciences, Université de Montréal, Montreal, QC, Canada, <sup>3</sup> Sainte-Justine Hospital Research Center, Montreal, QC, Canada

In the context of neuro-orthopedic pathologies affecting walking and thus patients' quality of life, understanding the mechanisms of gait deviations and identifying the causal motor impairments is of primary importance. Beside other approaches, neuromusculoskeletal simulations may be used to provide insight into this matter. To the best of our knowledge, no computational framework exists in the literature that allows for predictive simulations featuring muscle co-contractions, and the introduction of various types of perturbations during both healthy and pathological gait types. The aim of this preliminary study was to adapt a recently proposed EMG-marker tracking optimization process to a lower limb musculoskeletal model during equinus gait, a multiphase problem with contact forces. The resulting optimization method tracking EMG, ground reactions forces, and marker trajectories allowed an accurate reproduction of joint kinematics (average error of 5.4 ± 3.3 mm for pelvis translations, and 1.9 ± 1.3◦ for pelvis rotation and joint angles) and ensured good temporal agreement in muscle activity (the concordance between estimated and measured excitations was 76.8 ± 5.3 %) in a relatively fast process (3.88 ± 1.04 h). We have also highlighted that the tracking of ground reaction forces was possible and accurate (average error of 17.3 ± 5.5 N), even without the use of a complex foot-ground contact model.

Keywords: musculoskeletal modeling, direct multiple shooting, co-contraction, musculo-tendon forces, neuromusculoskeletal simulations

#### INTRODUCTION

Walking is often considered to be the most important activity in daily living (Chiou et al., 1985). The ability to move without pain, fatigue, or major gait deviation is closely related to quality of life (Cuomo et al., 2007; van Schie, 2008). Many neuro-orthopedic pathologies (e.g., cerebral palsy, stroke) induce impairments (i.e., paresis, muscle overactivity, soft tissue contractures, and bone deformities) that compromise normal movement. Consequently, the goal of many therapeutic interventions is to minimize gait deviations in patients. In order to improve these

#### Edited by:

Benjamin J. Fregly, Rice University, United States

#### Reviewed by:

Farong Gao, Hangzhou Dianzi University, China Ning Sun, Nankai University, China

\*Correspondence: Florent Moissenet florent.moissenet@protonmail.com

> Received: 26 March 2019 Accepted: 24 June 2019 Published: 16 July 2019

#### Citation:

Moissenet F, Bélaise C, Piche E, Michaud B and Begon M (2019) An Optimization Method Tracking EMG, Ground Reactions Forces, and Marker Trajectories for Musculo-Tendon Forces Estimation in Equinus Gait. Front. Neurorobot. 13:48. doi: 10.3389/fnbot.2019.00048 interventions, understanding the possible mechanisms of these gait deviations, and being able to identify the causal motor impairments is of primary importance (Davids et al., 2004; Gough and Shortland, 2008; Wren et al., 2011). Currently, the relationship between motor impairments and gait deviations is unclear (Bonnefoy-Mazure et al., 2016), and there is a lack scientific evidence for these relationships due to the inherent complexity of the human neuromusculoskeletal system during dynamic tasks such as walking (Armand et al., 2017). Compared to existing approaches (e.g., pathologic models, experimental procedures with human subjects, robots with human-like gait), in silico neuromusculoskeletal simulations of normal and pathological gait could provide additional insight into gait deviations (Armand et al., 2017). The advantage of neuromuscular simulations as a method is that large numbers of simulations may be performed relatively quickly, and without the ethical issues involved with performing invasive and lengthy experiments in vulnerable patient groups such as those with neuromuscular deficits.

To date, simulations reported in the literature are often limited to the analysis of the consequences of isolated impairments on gait such as muscle weakness (van der Krogt et al., 2009; Thompson et al., 2013) or muscle spasticity (Jansen et al., 2014). Most of these studies only report possible muscular compensations (adaptations) that occur due to muscular redundancy–results are achieved by tracking the normal gait kinematics and then applying a perturbation to the model. Very few studies have been based on a numerical framework allowing kinematic adaptations in response to more varied perturbations. Within the TLEMsafe project (https:// www.tlemsafe.eu), Fluit et al. (2014b) combined an optimized inverse model and a ground reaction force predictive model to simulate lower limb kinematics after the removal of the rectus femoris and the vastus lateralis from the model. Santos et al. (2017) also proposed a numerical framework based on direct collocation and an optimal control package to simulate lower limb kinematics after the introduction of a weakening of the triceps surae and the tibialis anterior, or after increasing the ankle joint stiffness. These two approaches represent a first step toward the simulation of pathological gait. However, neither were able to reproduce muscle cocontractions. While this capacity was not necessarily needed in these studies, this feature is essential to establish a pathological gait simulator that would be able to reproduce physiological gait adaptations biofidelically.

From a methodological point of view, inverse dynamicsbased approaches (such as static optimization) are commonly used due to their computational efficiency (Erdemir et al., 2007), but are not appropriate for predictive simulations. Moreover, static optimization method underestimates or neglects antagonist co-contractions unless hybrid approaches are used (Brookham et al., 2011; Son et al., 2012). On the other hand, forward dynamics-based approaches are often criticized for being time-consuming–several studies report convergence times in the hundreds of hours range (Anderson and Pandy, 2001). Despite this disadvantage, these methods have the potential to predict new movements, such as an adaptation in response to a perturbation. For example, state-of-the-art algorithms used in conjunction with existing musculoskeletal models–like direct collocation (Santos et al., 2017) and direct multiple shooting (Bélaise et al., 2018a,b)—can be used to solve forward dynamics problems in a timely manner. Recently, Bélaise et al. (2018a,b) introduced an EMG-marker tracking optimization method to predict musculo-tendon forces in a co-contraction case. Based on simulated datasets of upper limb movements, the authors showed the importance of tracking both marker trajectories and EMG, in particular to reproduce muscle cocontractions. To the best of our knowledge, such an approach has never been applied on experimental gait records with muscle co-contractions.

The objective of our project is to establish a computational framework appropriate for predictive simulations of healthy and pathological gait, that is able to reproduce muscle cocontractions, and that allows for the introduction of various kind of perturbations on the model (e.g., therapy-related, surgeryrelated, pathology-related perturbations). This preliminary study represents a first step toward this project by adapting the computational framework proposed by Bélaise et al. (2018a,b) to a lower limb musculoskeletal model during gait. This framework has been tested for this purpose on a type of pathological gait known as equinus gait.

## METHODS

## Lower Limb Musculoskeletal Model

A generic three-dimensional musculoskeletal model of the lower limb [Lower Extremity Model, OpenSim (Delp et al., 1990)] was adapted for our study (**Figure 1**). This model consists of five rigid segments: the pelvis, right thigh, patella, shank, and foot. Twenty-six markers were associated with these segments by virtual palpation to reproduce the experimental marker locations (**Table 1**; see section Dataset). To simplify the dynamic optimizations to a two-dimensional motion in this preliminary study, the original degrees of freedom (DoF) were reduced to three DoFs for the pelvis-ground joint (vertical translation, translation in the direction of walking, pelvis tilt) and one DoF (flexion-extension modeled as a hinge joint) at the hip, knee, and ankle joints. Joints were actuated by the muscle torques resulting from 17 muscle lines of action (**Table 2**), and the pelvis DoFs were actuated by three generalized forces applied on the pelvis. The path, optimal length, maximal isometric force, tendon slack length, and pennation angle of each muscle lines of action were derived from the original model (Delp et al., 1990).

Segment lengths were scaled to the dataset used in this study (see section Dataset) using OpenSim 3.3 (Delp et al., 2007) by minimizing the distance between experimental and model markers placed on bony landmarks (**Table 1**). All components of the model that depend on bone lengths (e.g., muscle attachment points, optimal fiber length), segment masses, and inertial parameters were also scaled. The resulting scaled model was transferred to the bioRBD musculoskeletal modeling package (https://github.com/pyomeca/biorbd) based on the Rigid Body Dynamic Library (Felis, 2017).

This model was defined by 29 states (six generalized joint positions and their six related velocities, 17 muscle activations) and 20 controls corresponding to the 17 muscle neural excitations plus three generalized forces driving the three pelvis DoFs.

#### Equations of Motion and Activation Dynamics

The generalized accelerations **q**¨ of the rigid multibody system were computed using a forward dynamics approach for given generalized joint positions **q**, joint velocities **q**˙, and generalized forces τ:

$$\ddot{\mathbf{q}} = \mathbf{M}\left(\mathbf{q}\right)^{-1} \left(\mathbf{r}\left(\mathbf{q}, \dot{\mathbf{q}}, \mathbf{a}, \mathbf{e}\right) + \mathbf{C}\left(\mathbf{q}\right)^{\mathrm{T}} \mathbf{R} - \mathbf{N}\left(\mathbf{q}, \dot{\mathbf{q}}\right) \dot{\mathbf{q}} - \mathbf{G}\left(\mathbf{q}\right)\right)$$
 
$$\text{s.t. } \mathbf{C}\left(\mathbf{q}\right)\ddot{\mathbf{q}} + \dot{\mathbf{C}}\left(\mathbf{q}\right)\dot{\mathbf{q}} = \mathbf{0} \text{ and } \mathbf{C}\left(\mathbf{q}\right)\dot{\mathbf{q}} = \mathbf{0}$$

where **M** is the inertia matrix, **C** is the external contact Jacobian matrix, **R** is the Lagrange multipliers vector corresponding to the ground reaction forces (GRF), **N** is the non-linear effects (Coriolis and centrifugal forces) vector, and **G** is the gravity effects vector. It was assumed that contact points have a null acceleration and velocity throughout the entire contact phase. In line with equinus gait, one fixed contact point was defined on the forefoot for the entire contact phase (see section Dynamic Optimizations). Generalized forces were divided into τ**<sup>1</sup>** = [τ**<sup>11</sup>** τ**<sup>12</sup>** τ**13**] <sup>T</sup> driving the three pelvis DoFs, and τ**<sup>2</sup>** = ∂**Lmt** ∂**q Fmt** corresponding to the net joint torques due to the musculo-tendon TABLE 1 | List of the 26 markers used in this study.


Palpation details used to place experimental reflective cutaneous markers (as well as virtual markers by virtual palpation) and the related segment are also mentioned.

forces **Fmt**, where <sup>∂</sup>**Lmt** ∂**q** is the generalized muscular lever arms matrix and **Lmt** the vector of muscle line of action lengths. **Fmt** were computed from muscle activations **a** using a Hilltype muscle model with a generic force-length-velocity relation f (Zajac, 1989):

$$\mathbf{F\_{mt}}\left(\mathbf{q}, \dot{\mathbf{q}}, \mathbf{a}\right) = \mathbf{a}f\left(\mathbf{F\_{mt}^0}, \mathbf{l\_m}, \mathbf{v\_m}\right), \mathbf{a}$$

where **F 0 mt** is the maximal isometric forces vector, **l<sup>m</sup>** is the muscle fiber lengths vector, and **v<sup>m</sup>** is the muscle fiber velocities vector. Again, musculo-tendon forces were divided into **Fmt1** (with related activations **a<sup>1</sup>** and excitations **e1**), corresponding to the muscles for which electromyographic (EMG) records were available, and **Fmt2** (with related activations **a<sup>2</sup>** and excitations **e2**) where EMG measurements were unavailable. Muscle activation dynamics was implemented as a set of first-order differential equations (Buchanan et al., 2004):

$$\begin{array}{ll} \texttt{\dot{\bf a}}\left(t,\texttt{e}\left(t\right),\texttt{a}(t)\right) = \begin{cases} \frac{\left(\texttt{e}\left(t\right)-\ \texttt{a}(t)\right)}{t\_{\texttt{det}}\left(0.5+1.5\ \texttt{a}(t)\right)}, & \texttt{e}\left(t\right) > \texttt{a}(t) \\\frac{\texttt{e}\left(t\right)-\ \texttt{a}(t)}{t\_{\texttt{det}}}\left(0.5+1.5\ \texttt{a}(t)\right), & \texttt{e}\left(t\right) \le \texttt{a}(t) \end{cases} \end{array}$$

where **e**(t) is the muscle neural excitations at time t. Time constants tact and tdeact (for activation and deactivation) were set at 10 and 40 ms, respectively (Thelen et al., 2003).

#### Dynamic Optimizations

As proposed in Bélaise et al. (2018a), controls and state variables were simultaneously optimized using an EMG-marker tracking optimization process. Because the model has a reduced muscular redundancy and because a generic model was used as opposed to a subject-specific model, the optimal maximal isometric forces were also identified during this process.

The optimization consisted of the minimization of the differences between predicted **M<sup>p</sup>** and measured **M<sup>m</sup>** marker trajectories in the sagittal plane, and between predicted **e1p** and measured **e1m** (EMG envelop) muscle neural excitations (corresponding to the muscles for which EMG records are available). This tracking optimization was extended to the minimization of the differences between predicted **R<sup>p</sup>** and measured **R<sup>m</sup>** GRF in the sagittal plane. This tracking was necessary to impose physiologic generalized forces to the pelvis (τ**1**), i.e., generalized forces that compensate for the missing upper part of the body and contralateral lower limb.

To predict the activity of the muscles for which tracking was not possible (i.e., muscles for which EMG records were not available), the objective function J was written to find the


TABLE 2 | List of the 17 Hill-type muscle lines of action included in the model.

least squared muscle activations **a<sup>2</sup>** that produced the prescribed marker trajectories, muscle neural excitations, and GRF (during stance phase only):

$$\begin{split} J &= \sum\_{1}^{N\_i} \left( \left\| \mathbf{w\_{M}} \left\| \mathbf{M\_{p}} - \mathbf{M\_{m}} \right\| \right\|^2 + \left\| \mathbf{e\_{1p}} - \mathbf{e\_{1m}} \right\|^2 + \left\| \mathbf{R\_{p}} - \mathbf{R\_{m}} \right\|^2 \right) \\ &+ \nu\_{\rm L} \int\_0^{T\_i} \mathbf{a\_2} \left( t \right)^2 dt \\ &\text{where } \nu\_{\rm R} \left\| \mathbf{R\_{p}} - \mathbf{R\_{m}} \right\|^2 = 0 \text{ when } i = 2, \text{ i.e., the swing phase.} \end{split}$$

where wM, we, wR, and w<sup>L</sup> are weighting factors adjusted to the relative importance of each term, T<sup>i</sup> is the duration of the current stage (see section Simulations) and N<sup>i</sup> is the related number of time frames.

This objective function was minimized under three sets of constraints. First, boundary conditions were applied on the state and the control variables. In this study, the range of motion of each DoF and related velocities were set to physiologic values (**Table 3**), while activations and excitations were bounded between 0 and 1. Second, the velocity of the contact point was constrained to be null at the first frame and its acceleration to be null at each time frame (see section Equations of Motion and Activation Dynamics). Third, periodicity was ensured by constraining the first and last time point of the cycle to have similar values in terms of hip, knee, ankle joint angles, and velocities, pelvis velocities, muscle excitations, and GRF.

TABLE 3 | Boundaries constraints applied during the optimization process to each degree of freedom and related velocities.


The muscles abbreviations, the joint(s) they cross and the related electromyographic (EMG) signals (when available) are also mentioned.

## Simulations

Each dynamic optimization was solved using a direct multiple shooting algorithm with MUSCOD-II (Leineweber et al., 2003). Three phases were defined in the gait step: (1) the stance phase (with an external contact between foot and ground), (2) the swing phase (no external contact), and (3) the first frame of the next stance phase following the impact between the foot and the ground. These stages were divided into 25, 25, and 1 multiple shooting intervals, respectively. For the sake of simplicity, the first stage started just after the collision impact between the foot and the ground. The duration of each stage was fixed to the measured value.

The initial guess was set to the measured values for the joint positions and velocities, 1% for all activations and excitations, and 0 for the controls corresponding to the generalized forces related to the pelvis DoFs. Weighting factors were set to w<sup>M</sup> = 30 (except for the foot markers for which w<sup>M</sup> = 50 to ensure the correct position of the contact point), w<sup>e</sup> = 1, w<sup>R</sup> = 0.05, and w<sup>L</sup> = 1. These weighting factors were adjusted empirically to set values around 1, in order to ensure optimization convergence and produce simulation results close to the experimentally measured data.

## Dataset

The previously defined method was evaluated on a dataset of emulated equinus gait. All data were recorded on a healthy volunteer (male, 35 years old, 165 cm, 66 kg) without any neuroorthopedic conditions. This participant gave written informed consent prior to his inclusion and the protocol was conformed to the Declaration of Helsinki and approved by the National Research Ethics Committee of Luxembourg (201805/01).

The 3D trajectories of 26 reflective cutaneous markers (bilateral iliac anterior and posterior spines, right leg great trochanter, medial and lateral femoral epicondyles, peroneal head, tibial tuberosity, medial and lateral malleoli, 1st, 2nd, and 5th proximal and distal metatarsal heads, calcaneum, completed by a four-marker cluster on the thigh and on the shank) (**Figure 1**) were recorded using a 10-camera optoelectronic system (OQUS-4, Qualisys AB, Sweden) sampled at 200 Hz. Markers were placed by anatomical palpation (**Table 1**) following the recommendation of van Sint Jan (2007) by an experienced user. GRF and moments were recorded using two side-by-side force plates (OR6-5, AMTI, USA) sampled at 2,000 Hz. The EMG activity of nine right leg muscles (tibialis anterior, soleus, gastrocnemius medialis, vastus medialis, rectus femoris, semitendinosus, biceps femoris long head, gluteus medius, gluteus maximus) was collected with a wireless electromyographic system (DTS clinic, Noraxon, USA) sampled at 2,000 Hz. The EMG surface electrodes were placed following the recommended standard of the Surface EMG for a Non-Invasive Assessment of Muscles (SENIAM) project (Hermens et al., 2000).

All data were imported under Matlab (R2018a, The MathWorks, USA) using the ezc3d package (https://github. com/pyomeca/ezc3d). Marker trajectories were interpolated when necessary using a cubic spline and smoothed by a 4th order low-pass Butterworth filter with a cutoff frequency of 6 Hz. Generalized kinematics (q, q˙, q¨) were computed using an extended Kalman filter (Fohanno et al., 2014) following the segmental coordinate systems defined in the original generic three-dimensional lower limb musculoskeletal model [Lower Extremity Model, OpenSim (Delp et al., 1990)]. GRF were smoothed by a 4th order low-pass Butterworth filter with a cut-off frequency of 15 Hz. Raw EMG signals were band pass filtered (4th order) between 30 and 300 Hz, rectified, and EMG envelops were obtained by a 4th order low-pass Butterworth filter with a cut-off frequency of 25 Hz. EMG envelops were then normalized to their respective maximal voluntary activation (Gaudet et al., 2018).

The participant was asked to mimic an equinus gait by producing voluntarily controlled co-contractions of the muscles crossing the ankle joint to restrain ankle dorsiflexion. Eight trials were recorded and the related right steps were analyzed in this study.

## Analysis

In order to evaluate the capacity of the model to reproduce the measured gait pattern and muscle excitations under the mechanical properties and constraints imposed to the model, a set of goodness-of-fit parameters were employed. Root mean square error (RMSE) and coefficient of determination (R<sup>2</sup> ) were computed to assess the differences in intensity and shape, respectively, between measured and estimated excitations, joint angles and GRF.

Only estimated muscle excitations corresponding to the measured EMG envelops (**Table 2**) are presented in this analysis. The coefficient of determination (CC) (Giroux et al., 2013) was computed for the muscles for which EMG data was recorded. This method uses active/inactive state concordance between the estimated muscle excitations and normalized EMG envelopes to compute a coefficient of concordance defined as the percentage of concordance elements.

## RESULTS

The convergence time of the eight optimizations using MUSCOD-II was 3.88 ± 1.04 h on an Intel <sup>R</sup> CoreTM i5-3570 CPU @3.4 GHz. Estimated and measured muscle excitations, musculo-tendon forces, joint angles, and GRF are reported in **Figures 2**–**5**, respectively. Goodness-of-fit parameters (RMSE, R<sup>2</sup> and CC) are reported in **Table 4**.

Considering the tracked muscle excitations (i.e., muscles for which EMG records are available, see **Table 2**), the temporal muscle activity of the model was in good overall agreement with the experimental measurements, with an average CC of 76.8 ± 5.3%. RMSE values were generally low with an average value of 0.2 ± 0.1 (values were adimensioned between 0 and 1). However, RMSE was found to be higher for the gastrocnemius medialis (0.3 ± 0.1) and the tibialis anterior (0.4 ± 0.1). For all muscles, the correlation remained low with an average R<sup>2</sup> at 0.02 ± 0.52. Regarding all other model muscles, for which EMG was not tracked, the optimized muscle excitations were higher than those estimated for muscles for which EMG was tracked. This is the case for R\_GLUT\_MAX2 and R\_GLUT\_MAX3 compared

freedom are given in Table 3.

to R\_GLUT\_MAX1, R\_GLUT\_MED1, and R\_GLUT\_MED3 compared to R\_GLUT\_MED2, R\_SEMIMEM compared to R\_SEMITEN, R\_VAS\_INT, and R\_VAS\_LAT compared to R\_VAS\_MED, and for R\_GAS\_LAT compared to R\_GAS\_MED. The same results are observed on the estimated musculo-tendon forces. These forces are ranged between 0 and 2,000 N, with the highest peak forces obtained for R\_SEMITEN, R\_VAST\_INT, R\_VAST\_LAT, and R\_SOLEUS.

With regard to the pelvis position/orientation and joint angles, the model estimations were generally in agreement with the experimental measurements. Average RMSE were 5.4 ± 3.3 mm for the pelvic translations, and 1.9 ± 1.3◦ for pelvic rotations and joint angles. However, RMSE was found to be higher for the ankle joint (4.0 ± 0.9◦ ). Considering all degrees of freedom, the correlation remained very high with an average R<sup>2</sup> at 0.94 ± 0.09.

For the GRF, the model estimations were found to be in agreement with the experimental measurements (the average RMSE is 17.3 ± 5.5 N). For these forces, the correlation remained high with an average R<sup>2</sup> at 0.97 ± 0.03.

#### DISCUSSION

The main objective of this study was to adapt an EMG marker tracking optimization process to solve a forward dynamics

TABLE 4 | Root mean square error (RMSE), coefficient of determination (R2) computed to assess the differences in intensity and shape, respectively, between measured and estimated excitations (adimensioned), pelvis position/orientation, joint angles, vertical ground reaction force (R\_GRF\_V), and anterior/posterior ground reaction force (R\_GRF\_AP).


The analysis of muscle excitations is completed by the coefficient of concordance (CC). The results are reported for each gait cycle. Abbreviations of each muscle names and degree of freedom are given in Tables 1, 2, respectively. Bold values correspond to higher RMSE values and lower R2 values.

problem on a 3D musculoskeletal model of the lower limb during equinus gait. To the best of our knowledge, the use of a direct multiple shooting algorithm on a musculoskeletal model with the tracking of measured EMG, marker trajectories, and GRF has never been performed to date. As already shown by Bélaise et al. (2018a,b), this approach allows for an accurate reproduction of joint kinematics and ensures temporal fidelity in muscle activity with improved computational time compared to traditional forward dynamic approaches. We have also highlighted that the tracking of GRF could be performed accurately, even without the use of a complex foot/ground contact model.

#### Limitations

A primary limitation of this preliminary study is that it was based on a small number trials for a single task, performed by a single participant. As such, limited conclusions can be drawn from this paper.

A second limitation is that only one contact point was defined at the forefoot and it was only constrained to null velocity and acceleration during the whole contact phase. While this approach was in line with an equinus gait and was able to accurately reproduce the tracked GRF, this definition cannot be applied during normal gait trials, during which several contact points should be defined (Fluit et al., 2014a). Elastic contact elements (Peng et al., 2018), artificial muscle-like actuators (Fluit et al., 2014a) or distance and velocity-dependent force models (Jung et al., 2016) should be adapted to the present model to extend its use to normal gait.

The proposed musculoskeletal model also only consisted of the pelvis and the right lower limb. A forward dynamic approach was made possible by replacing the forces and moments produced by the opposite lower limb and the upper part of the body by a set of generalized forces acting on the pelvis. With such an approach, the individual contribution of the opposite lower limb and the upper part of the body to the muscle activity and joint contact forces of the right lower limb cannot be evaluated individually. It would thus be an important next step to complete the missing body segments of the present musculoskeletal model in order to obtain a full body musculoskeletal model. Several full body musculoskeletal models (Rajagopal et al., 2016; Bassani et al., 2017) have been proposed in the literature and could be transferred to the bioRBD musculoskeletal modeling package. These segments could be actuated by joint torques instead of muscles.

In addition to this, while we used a 3D musculoskeletal model, DoFs were reduced to only allow a two-dimensional motion in the sagittal plane. It is however, established that walking is a locomotion task that is performed in sagittal, coronal, and transversal planes (Perry and Burnfield, 2010). In particular, patients often develop compensatory movements in the coronal plane when pathological impairments result in reduced foot clearance capacity in the sagittal plane (Chantraine et al., 2016). Despite this simplification, the accuracy of kinematic tracking observed in the present results suggest that there is potential for the EMG marker tracking optimization process to perform 3D gait motion simulations.

Finally, most of the parameters of the Hill-type muscle model were kept with generic property definitions in this study. Only optimal maximal isometric forces were identified during dynamic optimizations, while muscle optimal lengths, tendon slack lengths, and maximal isometric forces are usually identified in similar studies (Sartori et al., 2014; Pizzolato et al., 2015). This may explain the high excitations observed in muscles contained in a group, for which tracking of excitation was applied to other muscles. The introduction of further muscle parameters could be implemented in future studies, and would only be expected to impact the convergence time of the simulations.

### Muscle Activity

The present study support the results of Bélaise et al. (2018a,b), which demonstrated that EMG tracking could be an efficient way to reproduce measured muscle excitations in simulations. While some amplitude differences appeared for certain tracked muscles, the temporal activity was generally reproduced with good fidelity. This outcome is crucial to ensure the ability of the model to produce muscle co-contractions. Similar approaches have already been proposed in the literature–EMG-driven musculoskeletal models have also been used to accurately reproduce muscle excitation patterns observed on EMG records (Shao et al., 2009; Sartori et al., 2012). However, these models are constrained to have as many muscle lines of action as EMG available in the dataset. To overcome this limitation, some authors have proposed hybrid approaches that combine EMG-driven and static optimization methods (Lloyd and Besier, 2003; Moissenet et al., 2014; Sartori et al., 2014). The drawback with this strategy is that by minimizing the difference between the motor joint moments computed by EMG-driven and inverse dynamics, kinematics may not be accurately reproduced. In that sense, the EMG-marker tracking algorithm proposed by Bélaise et al. (2018a,b) is a novelty. As this method tracks joint kinematics based on marker trajectories rather than joint moments as in a forward dynamic approach, the error diffusion is minimized and the simulation outputs reproduce the experimental kinematics more faithfully.

In our trials of emulated equinus gait, co-contractions of the ankle dorsiflexors, and plantarflexors can be observed during early stance to stabilize the joint in this specific posture. It is interesting to observe that, while the gastrocnemius medialis and the soleus (muscles for which EMG records were tracked) were contracted during this phase, the gastrocnemius lateralis (a muscle for which EMG records were not measured) was not in our simulations. Although use of EMG is somewhat limited to available hardware, muscle locations and the signal quality, measurement should be prioritized toward muscles presumed to be active during the task being investigated. In our case, focusing on a greater number of muscles crossing the ankle joint would have brought more relevant information to the model. A similar recommendation has already been proposed by Sartori et al. (2014); these authors suggested to prioritize EMG use on muscles "that reflect the patient's non-physiological muscular behavior."

Although the EMG-GRF-marker tracking algorithm was able to reproduce physiological muscle activity, two points must be considered. First, we observed that when EMG was not tracked, the optimized muscle excitations and musculo-tendon forces were higher than the ones estimated when muscle EMG was tracked. As pointed out in the limitations of the study, this over-estimation is perhaps related to the use of generic muscle model parameters. For example, if the parameters applied to a muscle group would tend to limit its capacity to produce a motor joint moment, a higher muscle excitation would be required to reproduce the experimental measurements. This effect is further exacerbated if the excitation of a muscle in this group is constrained to a low level in accordance with the experimental EMG tracking, as the excitation of the other muscles of the group will have to compensate for this reduced excitation in the other muscle. Second, due to equinus (results in increased plantarflexion), the capacity of the triceps surae to produce a plantarflexion moment is reduced (Delp et al., 1990), in particular during the push-off phase. Thus, hip flexor recruitment may be increased in this gait pattern to pull the leg forward (Romkes and Schweizer, 2015). However, in our model the primary hip flexors, i.e., the iliopsoas muscles, were not included. van der Krogt et al. (2012) showed that an increased activation of the rectus femoris may be developed to compensate for a weakness of the primary hip flexors. In our case, the absence of the iliopsoas muscles (equivalent to a complete reduction of strength in these muscles) may have induced the increased rectus femoris excitations observed during the simulations compared to the experimental measurements. Because the rectus femoris is a bi-articular muscle (i.e., hip flexor, knee extensor), an increased excitation of this muscle used to assist in hip flexion would simultaneously act to reduce knee flexion, which would have require compensation from knee flexors in order to maintain experimental kinematics (van der Krogt et al., 2012). This could explain the non-physiological activity of the triceps surae (ankle plantarflexors) during preswing and early swing observed in our simulations, and the increased tibialis anterior activity (ankle dorsiflexor) used to balance ankle flexion due to the increased activity of ankle plantarflexors. All these observations support the need for a more comprehensive, full body musculoskeletal model, as already discussed in section Limitations.

### Kinematics and Ground Reaction Forces

Unlike inverse dynamics-based optimization approaches (i.e., static optimization), where measured kinematics and calculated joint torques are the input constraints to the optimization problem, in a forward dynamics approach it is essential to assess the accuracy of reproduced kinematics. This is generally assessed by tracking experimental kinematics (Erdemir et al., 2007; Chèze et al., 2015). In the present study, marker trajectories were tracked rather than joint kinematics, as proposed by Bélaise et al. (2018a,b). This tracking was able to produce accurate marker trajectories as highlighted in Bélaise et al. (2018a) with a tracking residual of 0.31 ± 0.32 cm during elbow flexion. This approach, used in the present study gave accurate kinematics with a maximum RMSE obtained at the ankle joint <5 ◦ , a threshold recognized as critical for clinical interpretation (McGinley et al., 2009). The main errors appeared at the end of the stance phase in most of the DoFs. This issue may be associated with the non-physiological activity of the triceps surae and the rectus femoris during this phase, as discussed in section Muscle Activity. Our simulations tended to reduce the plantarflexion induced by the emulated equinus gait. By increasing ankle dorsiflexion, the triceps surae moment arm was increased (Delp et al., 1990) and a minimal ankle plantarflexion moment was kept.

Interestingly, GRF were estimated accurately without an advanced foot-ground contact model definition. The use of simple generalized forces applied on the pelvis, designed to compensate for the absence of the opposite lower limb and the upper part of the body in our model, acted as efficient reserve actuators (Modenese et al., 2013) to provide the forces required to track experimental GRF. This approach may thus present a valid means by which to manage external forces and moments in the dynamic equation when the interactions between the upper limb and/or the contralateral lower limb with the ipsilateral lower limb are not known. Otherwise, as already discussed in section Limitations, a full body musculoskeletal model would be recommended.

#### Clinical Perspectives

The simulation of equinus gait represents an important clinical issue in the context of toe walking, a common gait deviation observed in many pathologies such as cerebral palsy, myopathy, and neuropathy (Armand et al., 2007). Numerical simulations may present a useful tool in this context of identifying potential biomechanical causes of this deviation, such as: pre-tibial muscle weakness, inadequate ankle dorsiflexors activity, ankle plantarflexors contracture, and/or spasticity, excessive voluntary ankle plantarflexion in compensation for quadriceps weakness, knee flexor contracture caused by overactivity of the hamstring, combined spasticity of the hamstring and ankle plantarflexors, and leg length discrepancy (Armand et al., 2007; Perry and Burnfield, 2010). In more general terms, there is a need for a numerical framework allowing for the introduction of pathology (Santos et al., 2017), treatment, or surgical intervention-related (Fox et al., 2009) perturbations in a model, and the analysis of their impact on the structures of the musculoskeletal system during daily living activities. However, before clinical applications, models have to be evaluated and validated (Hicks et al., 2015). It will thus be necessary that we assess the capacity of our approach to produce physiological musculo-tendon forces and joint contact forces. Validation datasets, such as the ones made available by Bergmann et al. (2016) and Fregly et al. (2012), should thus be tested on our numerical framework in the future.

## REFERENCES


## CONCLUSION

In conclusion, we have improved the recent EMG-marker tracking optimization method to a multiphase cyclic movement with GRF. This numerical framework was successfully tested on a dataset of equinus gait for which our approach was able to estimate lower-limb kinematics, GRF and muscle activity with reasonable accuracy.

## DATA AVAILABILITY

The dataset used for this study is available from figshare (https://figshare.com/s/ddef4ef25acc2ea91c04, doi: 10.6084/ m9.figshare.7869293). They are all stored in c3d file format (https://www.c3d.org).

### ETHICS STATEMENT

The participant gave written informed consent prior to his inclusion and the protocol was conformed to the Declaration of Helsinki and approved by the National Research Ethics Committee of Luxembourg (201805/01).

## AUTHOR CONTRIBUTIONS

FM conceived of the presented idea and adapted the theory to the present project. MB, BM, and CB developed the theory. FM and EP performed the computations. MB and BM verified the methods. All authors discussed the results and contributed to the final manuscript.

## FUNDING

This project was supported by the Luxembourg National Research Fund (INTER/MOBILITY/17/11756737) and by the NSERC Discovery grant (RGPIN-2014-03912).

### ACKNOWLEDGMENTS

The authors gratefully acknowledge the Interdisciplinary Center for Scientific Computing of Heidelberg University (Germany) for the opportunity to work with MUSCOD-II.

in computing lumbar spine loads at L4L5 level. J. Biomech. 58, 89–96. doi: 10.1016/j.jbiomech.2017.04.025


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Moissenet, Bélaise, Piche, Michaud and Begon. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# SimCP: A Simulation Platform to Predict Gait Performance Following Orthopedic Intervention in Children With Cerebral Palsy

Lorenzo Pitto<sup>1</sup> \*, Hans Kainz <sup>1</sup> , Antoine Falisse<sup>1</sup> , Mariska Wesseling<sup>1</sup> , Sam Van Rossom<sup>1</sup> , Hoa Hoang<sup>1</sup> , Eirini Papageorgiou2,3, Ann Hallemans <sup>4</sup> , Kaat Desloovere2,3 , Guy Molenaers 5,6, Anja Van Campenhout 5,6, Friedl De Groote<sup>1</sup> and Ilse Jonkers <sup>1</sup>

*<sup>1</sup> Department of Movement Sciences, KU Leuven, Leuven, Belgium, <sup>2</sup> Department of Rehabilitation Sciences, Doctoral School of Biomedical Sciences, KU Leuven, Leuven, Belgium, <sup>3</sup> Clinical Motion Analysis Laboratory, University Hospitals Leuven, Leuven, Belgium, <sup>4</sup> Department of Rehabilitation Sciences and Physiotherapy, University of Antwerp, Antwerp, Belgium, <sup>5</sup> Department of Orthopedics, University Hospitals Leuven, Leuven, Belgium, <sup>6</sup> Department of Development and Regeneration, KU Leuven, Leuven, Belgium*

#### Edited by:

*Mohammad S. Shourijeh, Rice University, United States*

#### Reviewed by:

*Thomas Geijtenbeek, Delft University of Technology, Netherlands Matthew Millard, Heidelberg University, Germany*

> \*Correspondence: *Lorenzo Pitto lorenzo.pitto@kuleuven.be*

Received: *30 March 2019* Accepted: *03 July 2019* Published: *17 July 2019*

#### Citation:

*Pitto L, Kainz H, Falisse A, Wesseling M, Van Rossom S, Hoang H, Papageorgiou E, Hallemans A, Desloovere K, Molenaers G, Van Campenhout A, De Groote F and Jonkers I (2019) SimCP: A Simulation Platform to Predict Gait Performance Following Orthopedic Intervention in Children With Cerebral Palsy. Front. Neurorobot. 13:54. doi: 10.3389/fnbot.2019.00054* Gait deficits in cerebral palsy (CP) are often treated with a single-event multi-level surgery (SEMLS). Selecting the treatment options (combination of bony and soft tissue corrections) for a specific patient is a complex endeavor and very often treatment outcome is not satisfying. A deterioration in 22.8% of the parameters describing gait performance has been reported and there is need for additional surgery in 11% of the patients. Computational simulations based on musculoskeletal models that allow clinicians to test the effects of different treatment options before surgery have the potential to drastically improve treatment outcome. However, to date, no such simulation and modeling method is available. Two important challenges are the development of methods to include patient-specific neuromechanical impairments into the models and to simulate the effect of different surgical procedures on post-operative gait performance. Therefore, we developed the SimCP framework that allows the evaluation of the effect of different simulated surgeries on gait performance of a specific patient and includes a graphical user interface (GUI) that enables performing virtual surgery on the models. We demonstrated the potential of our framework for two case studies. Models reflecting the patient-specific musculoskeletal geometry and muscle properties are generated based solely on data collected before the treatment. The patient's motor control is described based on muscle synergies derived from pre-operative EMG. The GUI is then used to modify the musculoskeletal properties according to the surgical plan. Since SEMLS does not affect motor control, the same motor control model is used to define gait performance pre- and post-operative. We use the capability gap (CG), i.e., the difference between the joint moments needed to perform healthy walking and the joint moments the personalized model can generate, to quantify gait performance. In both cases, the CG was smaller post- then pre-operative and this was in accordance with the measured change in gait kinematics after treatment.

Keywords: cerebral palsy, muscle synergies, single event multilevel surgery, orthopedic interventions, capability gap, subject specific model

## INTRODUCTION

Cerebral Palsy (CP) is the most common cause of motor deficiency in young children with a prevalence of 2–3 cases per 1,000 live births (Fairhurst, 2012; Colver et al., 2014; Graham et al., 2016). Due to lesions in the developing brain, children with CP display motor disabilities that vary greatly in presentation and severity. While CP is not a progressive disease, with time, secondary symptoms might arise, such as bony deformities and muscle contractures. Alongside increasing pain and fatigue, these symptoms can pose severe limitations to the quality of life and independence of the patients (Hanna et al., 2009; Opheim et al., 2009). Nowadays, several orthopedic treatments, often in combination with physical therapy and orthoses, are available and aim at improving the functionality and therefore quality of life of these patients (Fairhurst, 2012; Narayanan, 2012; Fitoussi and Bachy, 2015; Strobl et al., 2015; Nieuwenhuys et al., 2016).

For ambulatory patients, orthopedic treatments usually aim at improving walking speed and stability, at reducing the need of walking aids and at mitigating or preventing fatigue and pain (Narayanan, 2012). The selection of the most appropriate surgical treatment is a complex endeavor that nowadays is mainly based on the clinical assessment of the patient, integrated 3D gait analysis and medical imaging (Molenaers et al., 2001; Strobl et al., 2015). The outcome, however, is not always as desired and studies reported a deterioration in 22.8% of the parameters used to describe gait performance after surgery (Filho et al., 2008). In 11% of the cases, additional surgeries are needed to improve the functional outcome, although, this can be as high as 32% when no gait analysis is used to support the decision-making process (Wren et al., 2009).

It is therefore of the utmost importance to identify the parameters that determine the success of an orthopedic intervention (Hersh et al., 2002; Arnold et al., 2006a; Niiler et al., 2007; Fox et al., 2009; Reinbolt et al., 2009; Hicks et al., 2011; Schwartz et al., 2013, 2016; Mansouri et al., 2016; Galarraga et al., 2017). This would allow making pre-operative predictions in order to guide the decision-making process toward the most effective treatments in terms of functional outcome. Several studies applying statistical approaches and more recently machine learning methods to explore these relationships (Hersh et al., 2002; Reinbolt et al., 2009; Hicks et al., 2011; Schwartz et al., 2013, 2016) have been quite successful in predicting the improvement or non-improvement of a few selected outcome indicators when dealing with selected surgeries. However, existing methods do not produce a comprehensive outcome prediction and do not account for combinations of different surgeries. Notably, Galarraga et al. (2017) developed a method based on dimension reduction and multiple linear regression to predict lower limb kinematics for a large number of surgical procedures. All these methods, however, have the drawback that they are black box methods and therefore do not allow investigating the mechanisms relating outcomes in motor function to the specific interventions (Halilaj et al., 2018). On the other hand, methods relying on musculoskeletal models and computational simulations are often suggested to have the potential to identify the causal relation between individual impairments, their interactions and the treatment outcome (Morrison et al., 2018).

To introduce simulation-based decision-supporting tools into clinical practice, a few obstacles have yet to be overcome. One of the major obstacles in this respect is the need for a representative translation of the neuromusculoskeletal dysfunctions of the patients (i.e., the altered musculoskeletal geometry, musculoskeletal parameters, and altered neural control) into the musculoskeletal models. The need to account for the musculoskeletal deformities of the individual CP patient and the bony deformities in particular, dictates the use of subject-specific musculoskeletal models when generating dynamic simulations of CP gait. In this respect, the added value of magnetic resonance imaging (MRI) based models has been extensively demonstrated (Scheys et al., 2011a,b; Bosmans et al., 2016).

The altered muscle parameters (i.e., muscle contracture and weakness) in patients with CP compared to a healthy population (Theis et al., 2016; Kruse et al., 2017; Kalkman et al., 2018) invalidates the use of scaled generic parameters. Appropriate parameter tuning capturing the patient-specific muscle properties is therefore needed. Several methods have been proposed for tuning and scaling musculoskeletal parameters (Van Campen et al., 2014; Modenese et al., 2016; Falisse et al., 2017). Nevertheless, most of these methods require an extensive amount of data collected based on a specific method (e.g., instrumented dynamometry) that is typically not available in the common clinical practice and might be difficult to apply in the case of neuromotor deficits.

The altered motor control of patients with CP is reflected in the use of aberrant coordination patterns of the muscles during gait compared to a healthy population (Steele et al., 2015). The concept of muscle synergies is an elegant way to summarize these coordination patterns and by comparing them between CP and typically developing (TD) children, altered motor control aspects have already been identified in terms of number of independent components and stride-by-stride variability (Steele et al., 2015; Kim et al., 2018). In addition, muscle synergies have already been used in the control of musculoskeletal models during dynamic simulations (Allen and Neptune, 2012; Sartori et al., 2013; Meyer et al., 2016). In healthy subjects, the muscle activations generating the observed muscle synergies are very similar to those generated when muscles are recruited independently according to an optimality criterion (De Groote et al., 2014). However, patients with CP exhibit different sets of muscle synergies with respect to a healthy population (Steele et al., 2015), thus highlighting the importance of including a subject-specific motor control model into the framework (Meyer et al., 2016; Sartori et al., 2017).

Literature results (Patikas et al., 2007) and a pilot study from our research group (Pitto et al., 2018), suggest that the same motor control model can be used to describe both the pre- and post-operative patient's condition. Therefore, the preoperative synergies may also be used for the simulations of the post-operative condition, as their composition remains mostly unchanged after a specific orthopedic treatment. The advantage of this approach is that it relies entirely on pre-operative data, thus making it suitable for the pre-operative decisionmaking process.

Apart from describing the patient-specific features in the modeling framework, also the specific therapeutic interventions (and multi-level surgeries in particular) need to be accounted for into the musculoskeletal model. Whereas the effect of muscle-tendon lengthening and muscle transfer on the momentgenerating capacity, lengths and velocities of the muscles have been described (Delp and Zajac, 1992; Arnold et al., 2006b), only few studies attempted a forward simulation aiming to predict the post-intervention outcome. For instance, two studies using musculoskeletal modeling and forward dynamics simulations (Fox et al., 2009; Mansouri et al., 2016) investigated the effect of a rectus femoris transfer surgery on the recovery of balance after a perturbation and on knee flexion in stiff knee gait in children with CP. While these studies represent a step forward in this direction, their scope remains quite restricted, accounting for only one kind of intervention and analyzing the effect on a single parameter.

Clinical use of simulation-based decision-supporting tools requires the definition of comprehensive parameters that relate to the functional improvement of the patient and therefore can be used as outcome measures to evaluate the effect of different interventions. In the field of assistive exoskeletons (Afschrift et al., 2014), the concept of the capability gap (CG) was introduced to represent the amount of support the exoskeleton had to provide in order to allow the patient to perform a given task. This concept can be translated to the estimation of the motor performance of the patient, before and after the simulated interventions, as a measure of "difficulty" in performing a desired motion, i.e., gait pattern of a TD child. By integrating the patientspecific impaired motor control, abnormal muscle properties, and/or altered musculoskeletal geometry, the changes in the CG after a simulated orthopedic treatment inform the clinician on how a specific intervention improves the ability of the child to adopt a TD gait pattern.

Within the SimCP project, a comprehensive simulationbased framework was developed to evaluate the functional effect of a therapeutic/surgical intervention in a specific patient with CP, thereby assisting the most appropriate treatment selection (**Figure 1**). This framework relies on the creation of a personalized neuro-musculoskeletal model of the patient. In this model, the musculoskeletal geometry is obtained from imaging data. The framework then provides the tools to personalize the muscle parameters according to information collected during gait analysis and clinical examination. Furthermore, the motor control is personalized using EMG data collected during the treatment-planning phase. Thereafter, a Graphical User Interface (GUI) allows clinicians to simulate combinations of different multi-level surgical procedures. Finally, the functional performance (i.e., walking ability) of the patient can be quantified for different simulated post-operative conditions by evaluating the change in the predicted capability gap with respect to the preoperative condition. These operations rely only on experimental data collected pre-operative. In this manner, it is feasible to compare the effectiveness of a set of candidate treatments in improving the gait performance of the patient, thus supporting the clinical decision-making process and optimizing individual treatment outcome.

Throughout this article, the different building blocks composing the framework are described and two representative cases studies are introduced to assess the methods and elucidate the several steps.

## METHODS

The goal of the SimCP framework is to predict gait performance following different candidate orthopedic treatments solely based on data collected before the treatment and the surgical plan. Pre-operative data includes gait analysis, clinical examination (e.g., documenting joint range of motion) and medical images. To this aim, the framework contains several building blocks (**Figure 1**). First, a personalized musculoskeletal model capturing the patient's musculoskeletal geometry (section Musculoskeletal Geometry) and muscle properties (section Muscle Parameters) is generated. Then, a description of the patient's motor control is added to this model (section Motor Control). Next, the orthopedic surgeries are simulated (section Surgery Simulation). Finally, the simulated post-operative gait performance is computed (section Capability Gap and Muscle Report). The framework also offers the possibility to investigate the contributions of the different impairments to gait performance (section Alternative Analyses). We illustrate the salient features of the SimCP framework using data from two representative patients (section Case Studies).

## Data Collection

In order to provide the information needed for the complete personalization of the models, inputs from several sources are required. First, imaging data, such as CT scans and MRI-images, allow defining the musculoskeletal geometry. Second, three-dimensional gait analysis data, including marker trajectories and ground reaction forces as well as EMG signals from the most important muscles in the lower limb, are needed for the personalization of the muscle parameters and definition of the motor control model. Third, clinical examination reporting the passive range of motion of the patient contains useful information to refine the personalization of the muscle parameters.

#### Musculoskeletal Geometry

The musculoskeletal models are based on a generic SIMM (Motion Analysis Corp., Santa Rosa, CA) model and are composed of 14 bodies and 21 degrees of freedom that are actuated by 86 muscles. The musculoskeletal geometry of this generic model is adapted to reflect the patient's musculoskeletal geometry. The model is then further personalized by tuning the muscle-tendon parameters (section Muscle Parameters) and by adding a model of the patient's motor control (section Motor Control).

Musculoskeletal geometry is derived from MRI-images. The workflow to create the MRI-based models has been published previously (Scheys et al., 2008, 2011b). In short, bones of the lower limbs and pelvis are segmented using Mimics (Materialize,

Leuven, Belgium). Afterwards, anatomical reference frames, joint axes and muscle origin and insertion points are defined and a patient-specific musculoskeletal OpenSim model is created using MuscleSegmenter (Leuven, Belgium) and customized Matlab (The Mathworks, Natick, MA) scripts. The models with personalized musculoskeletal geometry are then imported into the SimCP framework.

Performing virtual surgery alters the musculoskeletal geometry and hence generates a new model, which is linked to the pre-operative model and appears in the post-operative model list. In this manner, each pre-operative model can be linked to multiple post-operative models, allowing for the exploration of different treatment options (see also section Surgery Simulation).

#### Muscle Parameters

Using preoperative data, we tune the two parameters of the Hill-type muscle model that have the largest influence on the simulated muscle force: muscle optimal fiber length and tendon slack length (l moand l ts) (De Groote et al., 2010). Maximum isometric muscle forces are scaled based on the patient's body weight (Van Der Krogt et al., 2016; Kainz et al., 2018). Pennation angles are taken from the gait2392 model in OpenSim as they have a limited effect on simulated forces (Zajac, 1989).

Parameters are tuned using only the information from the pre-operative walking trials (marker trajectories, ground reaction forces and EMG) and, optionally, the clinical examination of the passive joint range of motion (**Table 1**). The underlying assumption of the procedure is that during gait the muscles generate the inverse dynamic moments with activations that are consistent with the measured EMGs and operate around their optimal fiber length. In other words, given the joint excursions during gait, muscles are not extremely short, as this would limit their force production given the muscle's force length relationship, nor too stretched, as this would induce excessive passive forces. In addition, it is assumed that at least part of the resistance encountered during the clinical examination at the extremes of the range of motion, is attributable to muscle passive force, i.e., muscles being at a length well above their optimal length.

To perform this tuning, we extended a static optimization problem since this allows us to optimize the fit between the computed activations and joint moments, and, respectively, the EMGs and inverse dynamic joint moments. Static optimization

#### TABLE 1 | Subjects demographics and data from clinical examination.


*(Continued)*

#### Pitto et al. SimCP: Simulation Platform for CP


#### TABLE 1 | Continued

*Information about the subjects included in the study. Data are collected from the pre- and post-operative clinical reports. For each entry, the first value is relative to the pre-operative condition, the second, in bold, is relative to the post-operative condition. "Mas" stands for manual Ashworth test (a score of 0 indicates no spasticity, a score of 4 maximal spasticity), "Tard" for Tardieu test (measured in degrees, defines the position where the spasticity limits the movement). In "Selectivity" a score of 2 indicates the maximum motor control selectivity. In "Strength" the maximum value is represented by a score of 5. "norm" stands for normal, defining full range of motion; "discr" stands for discrete, defining limited range of motion; "/" replaces values not present in the report. Reference values for TD subjects are obtained from Moon et al. (2017).*

computes muscle activations that produce the inverse dynamic joint moments underlying a measured movement while optimizing a performance criterion (e.g., minimizing sum of activations squared). Here, we allow optimal fiber lengths and tendon slack lengths to change during the optimization while imposing constraints on the allowable muscle lengths that represent the tuning criteria described above. In contrast to the typical static optimization approach that is solved for each time frame separately, here all time frames are coupled to obtain a single set of muscle-tendon parameters. It is important to note that static optimization neglects muscle dynamics by assuming that tendons are rigid but allows accounting for the muscle force-length-velocity relationship (De Groote et al., 2016). A static optimization approach was preferred over a dynamic approach that accounts for muscle dynamics (De Groote et al., 2016) to limit computation times. The problem was then solved using the fmincon function in Matlab.

To cope with the scarcity of input data (i.e., data from a limited number of movements and a limited number of EMG signals), we decided to tune the parameters only in a set of major muscles (M) (**Table 2**). In addition, we used a different level of detail when describing the force generated by these major and other muscles. For the major muscles, the forcelength relationship, derived from (De Groote et al., 2016), was taken into account (but not the force-velocity relationship). Hence, the generated force has an active component (f L ), depending on normalized muscle fiber length (˜ l) and muscle activation (a), and a passive component, (f P ) depending only on fiber length:

$${}^{m}F\_{i} = \,^{m}f\_{\circ} \left[ {}^{m}a\_{i} \,^{m}f\_{i}^{L} \left( {}^{m}\tilde{l}\_{i} \right) + {}^{m}f\_{i}^{P} \left( {}^{m}\tilde{l}\_{i} \right) \right], \,\forall m \in M,\tag{1}$$

Where f is the maximum isometric force the muscle can exert and the subscript i defines the instant in time. For the remaining muscles (N), the generated force is proportional to activation:

$$\,^m f\_i = ^m f\_\circ \,^m a\_i \,, \qquad \forall m \in N. \tag{2}$$

Hence, the resulting estimation problem has the following structure. Optimization variables consist of muscle activations and reserve moments (for each joint j) during the gait cycle ( <sup>m</sup>a<sup>i</sup> , j τ R i ), as well as the muscle parameters (ml mo , ml ts). Reserve moments are generated by ideal actuators and are added to the muscle moments to guarantee that the inverse dynamic joint moments can be matched even when the muscles are not sufficiently strong. Since they are not physiological, their use is heavily penalized in the cost function to keep their contribution to a minimum:

$$C^{SO} = \sum\_{i} \left( \sum\_{m} w\_1 \left( {}^{m} a\_i \right)^2 + \sum\_{j} w\_2 \left( {}^{j} \tau\_i \right)^2 \right), \tag{3}$$

Where w1−<sup>2</sup> are weighting coefficients that produced a proportional balance between muscle and residual activations (Hicks et al., 2015) in simple static optimization problems.

An additional penalty term is included in the cost function to ensure that the computed activations of the subset of muscles (ε) for which EMG was collected (see **Table 2**) reflect the pattern of the measured data:

$$C^{\epsilon} = \sum\_{i} \sum\_{m \in \epsilon} w\_3 \left( {}^{m} \sigma \, {}^{m} a\_i - {}^{m} \varepsilon\_i \right)^2,\tag{4}$$

Where ε represents the experimental EMG envelope. The scaling factor σ was introduced as an optimization variable to impose

#### TABLE 2 | List of muscle subsets.


*"Tuned muscles" reports the set of muscles whose parameters have been tuned; "EMG muscles" reports which muscles of the model were assigned to which EMG channels; "Passive Range of motion" reports the clinical examination measures included into the framework and which muscles were measured in each position.*

similarity between activations and EMG patterns irrespective of signal amplitude since, in the absence of maximum voluntary contraction tests, the relation between signal amplitude and muscle activation cannot be accurately derived.

The cost function was minimized subject to the following constraints. A first set of constraints describes that the muscles should produce the inverse dynamic joint moments:

$$\sum\_{m \in M} {}^{jm} r\_i \, {}^{m} F\_i \, + \sum\_{m \in N} {}^{jm} r\_i \, {}^{m} f\_i + {}^{j} \boldsymbol{\pi}\_i^R \, - {}^{j} \boldsymbol{\pi}\_i^{ID} = \mathbf{0},\tag{5}$$

Where <sup>j</sup> τ ID i are the desired joint moments from inverse dynamics and jmr i is the moment arm of muscle m with respect to joint j at time i.

A second set of constraints imposes bounds on the muscle fiber lengths during gait and the clinical exam of the range of motion. To ensure that normalized muscle fiber lengths during gait are within 0.4 and 1.5, we constrain the minimal and maximal fiber lengths. In addition, to ensure that muscles operate around their optimal length during gait, we constrain the maximal fiber length to be above and the minimal fiber length to be below optimal fiber length:

$$1 < \max\_{i} \left( {}^{m}\tilde{l}\_{i} \right) \le 1.5,\tag{6}$$

$$0.4 \le \min\_{i} \left( {}^{m} \tilde{l}\_{i} \right) < 1 \,, \quad \forall m \in M,\tag{7}$$

Maximal normalized fiber lengths during the clinical exam (˜ l R ) should be in the range where passive force is generated:

$$1 < \, ^m \hat{l}^R \le 1.5 \,, \quad \forall m \in R,\tag{8}$$

Where R defines the subset of muscles for which the length is computed using information from the clinical passive range of motion examination (**Table 2**).

To impose that muscles were stretched to a level where they generated considerable passive force during the clinical exam, we added a penalty term to the cost function:

$$C^{R} = \sum\_{m \in R} w\_4 \left( {}^{m}\tilde{l}^R - 1.5 \right)^2,\tag{9}$$

Within our formulation, the passive force exerted by a muscle stretched at 1.5 l mo is around 0.5 F. While a significant variation in passive force is present between muscles (Prado et al., 2005) and further variations are induced by CP (Kalkman et al., 2018), we made this simplifying assumption to allow for the selection of different sets of muscles without increasing the complexity of the tuning procedure.

Combining, (Equations 3, 4, and 9) and, the final cost function becomes

$$\text{C}^{\text{SO}} + \text{C}^{\epsilon} + \text{C}^{\text{R}},\tag{10}$$

Input data <sup>j</sup> τ ID i , jmr i , and muscle-tendon lengths <sup>m</sup>l mt i are computed based on the personalized musculoskeletal models using OpenSim's analysis tools, specifically Inverse Kinematics, Inverse Dynamics and Muscle Analysis using the gait data as input. Therefore, muscle moment arms are defined using the generalized force method (Sherman et al., 2014). Muscle fiber lengths (l) and normalized fiber length (˜ l) during gait were computed according to a Hill-type model assuming a rigid tendon:

$${}^{m}l\_{i} = \left( {}^{m}l\_{i}^{mt} - {}^{m}l\_{i}^{ts} \right) / \cos \left( \alpha \right),\tag{11}$$

$${}^{m}\tilde{l}\_{i} = \,^{m}l\_{i}/\,^{m}l^{mo} \,, \quad \forall m \in M,\tag{12}$$

Where α is the pennation angle. Joint positions during the clinical exam to test the passive range of motion are derived from the description of the test and are used to compute the maximum musculotendon lengths, muscle fiber lengths and normalized fiber lengths reached during the test for the subset of muscles R (l mtR , l R and ˜ l R , respectively).

The estimated parameters l moand l ts are incorporated into the subject's model and will define the force-length relationships of the muscles included in the set M during the following analyses.

#### Motor Control

In cerebral palsy, the ability to selectively recruit muscles is reduced. Therefore, we describe impaired motor control by imposing the pathological muscle activation patterns when computing gait performance. Muscle activation patterns are derived from pre-operative EMG data collected during walking using synergy analysis. For the motor control model to be useful in our simulations, it has to account for the activations of all the muscles. However, EMG data are collected from only a small subset of muscles (typically <10) while the musculoskeletal model contains many more muscles (typically more than 40). A multi-step procedure is proposed to derive muscle coordination for all muscles in the model based on EMG data from a limited number of muscles (cfr. Meyer et al., 2016).

First, muscle synergies are derived from the pre-operative EMG data acquired during three gait cycles to define the complexity of the patient's motor control by the number of synergies (N s ) using non-negative matrix factorization (NNMF) (Lee and Seung, 1999). The input matrix of EMG signals has dimensions N <sup>ǫ</sup> × N <sup>i</sup> where N ǫ is the number of muscles and N i is the number of time instants. The output of the synergy analysis consists of two matrices: a N <sup>s</sup> × N <sup>i</sup> matrix H containing the activation timing profiles of each synergy and a N <sup>ǫ</sup> × N <sup>s</sup> matrix W containing the weight vectors specifying how much an individual muscle is activated by each synergy. The matrices W and H are computed such that the product WH best approximates the original input matrix for a predefined number of synergies N s . We quantified N <sup>s</sup> using a bootstrapping procedure such that the percentage of the original signal explained by the synergies is above a predefined threshold (Cheung et al., 2009). Both the EMG signal and the H matrix were consistently resampled 500 times with replacements, using Matlab function datasample. The resampled matrices have the same dimension of the original ones and the same time instants of the original matrices can appear more than once in the resampled one. The variability accounted for (VAF) of these resampled signals by the synergies extracted from the original signal is computed. N s is the lowest number of synergies for which VAF is higher than 90% for at least 95% of the resampled signals. The number of synergies N s is used in the subsequent steps.

Next, an EMG-informed static optimization analysis is performed on the patient's pre-operative gait data, where the optimization variables are <sup>m</sup>a<sup>i</sup> , j τ R i , and <sup>m</sup>σ. The cost function is obtained by combining (Equations 3 and 4):

$$C^{\\$0} + C^{\epsilon},\tag{13}$$

The C ǫ term enables us to account for the pathological characteristics of muscle activations, such as antagonistic muscle co-contractions, which are very common in children with CP. This cost is minimized subject to constraints describing the equilibrium between inverse dynamic and muscle moments (Equation 5).

Finally, muscle synergies are extracted by performing a new NNMF on the muscle activations computed with static optimization. The number of synergies for this analysis is N <sup>s</sup> + 1. The extra synergy is included to take into account muscles for which no surface EMG was collected. For instance, Allen and Neptune (2012) found that a synergy including predominant contributions from the iliacus and psoas muscles is needed to control a 3D model during walking, and EMG from these muscles is typically not acquired. The result is a set of muscle synergies (Wpreand Hpre) that define the motor control model of the patient and describe the activations of all the muscles. This motor control model is later used in the computation of the gait performance of the patient (section Capability Gap and Muscle Report).

#### Surgery Simulation

We developed a set of virtual surgeries and a GUI that allow to directly manipulate the musculoskeletal models by leveraging the Matlab-OpenSim application programming interfaces. The surgeries implemented in the current version of the GUI are Extension and Derotation Osteotomy, Derotation Osteotomy, Muscle Transfer, Patella Advancement and Botulinum Toxin Injection. Only the Muscle Transfer and Botulinum Toxin Injections influence the muscle parameters.

In an Extension and Derotation Osteotomy (**Figure 2B**), two cutting planes define the bone wedge for removal (Lenhart et al., 2017). In the GUI, the pose of the cutting planes on the desired bone can be defined. The wedge is removed and the remaining bone segments are then reconnected by joining the cutting planes. The intra-segment rotation perpendicular to the cutting planes and the translation along the cutting plane can be specified by the user. Based on bony landmarks, important morphometric information, such as anteversion angle and neck

FIGURE 2 | Framework components. (A) Muscle transfer surgery. Used to change the path of a muscle. The lines of action of other muscles are visualized as guide to help during the transfer. (B) Extension and Derotation osteotomy surgery. Two sets of scrollbars define the pose of the two cutting planes (red and blue in the left side figures) that define the wedge of bone to be removed. After wedge removal, the two segments are brought in contact and it is possible to rotate and translate the distal part to correct for abnormal anteversion angles. (C) Patella advancement surgery. It is possible to define the new length of the patella ligament and/or move its insertion on the tibia, during the operation the ligament in the new configuration is shown as a red cylinder.

shaft angle of the simulated post-operative bone configuration are visualized in real-time to guide the user in performing the virtual surgery. The Derotation Osteotomy surgery is implemented in a manner similar to the Extension and Derotation Osteotomy but involves only a single cutting plane on the desired bone. After cutting the bone, the distal part of the bone can be rotated and translated with respect to the proximal part of the bone to correct for bony deformities. Within the Muscle Transfer tool (**Figure 2A**), we provide several options to modify the muscle geometry. These options include adding, removing, translating and changing the muscle attachment, insertion and via points. The resulting change in musculotendon length is translated into a change in tendon slack length. The underlying assumption is that muscle transfer surgeries do not directly affect the muscle fiber architecture, but change the length of the tendon either by removing part of the tendon, lengthening the tendon, or by transferring it to the tendon of another muscle. Patella Advancement (**Figure 2C**) can be performed by either changing the length of the patella ligament, or by transferring the attachment of the ligament on the tibia to a new position. Patella movement is defined as a function of knee flexion angle. After changing the ligament attachment or length, the new path of the patella is determined through an optimization procedure. This optimization procedure defines the rotation and the translations needed to represent the patella movement on the plane perpendicular to the knee joint axis. This optimization finds the patella movement that results in the most constant distance between two points on the patella, one proximal and one distal, and the femur surface throughout the motion while maintaining the patella ligament length constant. Botulinum Toxin Injections are modeled by a decrease in the injected muscle's maximum isometric force. This is a highly simplified representation of what botulinum toxin injections do to the muscle and further research is needed to refine this procedure in our model.

Any change applied to a given model results in a new postoperative model, which can be saved for future use.

#### Capability Gap and Muscle Report

As previously introduced, the main outcome of the present framework is the capability gap. The capability gap is the difference between the joint moments needed for performing a "desired" motion, i.e., TD walking, and the joint moments the personalized model of the patient (this can be either a pre- or post-operative model) can generate. We use a synergy-based static optimization approach to compute the capability gap. Here, the reserve moments appearing in the moment equilibrium function (Equation 5) represent the torque deficit and hence the capability gap. Subject-specific musculoskeletal geometry and muscle parameters are described in the musculoskeletal model. Impaired motor control is imposed through additional constraints on the activations based on the patient's muscle synergies.

The desired motion used for the computation is derived by scaling average TD walking data to the patient's dimensions (**Table 1**). First, TD kinematics are imposed to a generic model that was scaled to the patient and corresponding 3D marker trajectories are extracted. The magnitude of the ground reaction forces is scaled based on mass and their point of application, expressed in the foot reference frame, is scaled based on body height. Successively, using the marker trajectories and ground reaction forces, the joint moments required for the personalized model to perform the desired motion are computed by an inverse kinematics and inverse dynamics analysis. In addition, corresponding muscle moment arms and musculotendon lengths are computed. By tracking marker trajectories consistent with TD walking instead of imposing TD joint kinematics directly to the musculoskeletal model, we avoid that the presence of bony deformities leads to unrealistic gait patterns. As an example, if the femoral neck anteversion is 30◦ higher than normal, imposing TD kinematics to the personalized model would result in a gait pattern with the knee and foot pointing outwards by about 30◦ , whereas if we track the marker trajectories the knee and foot will point forwards.

Afterwards, the synergy constrained static optimization is performed. The cost function to be minimized is C SO as defined by Equation (3). The moment equilibrium (Equation 5) has to be satisfied. Inputs to Equation (5), <sup>j</sup> τ ID i , jmr<sup>i</sup> , and <sup>m</sup>˜ li , are computed based on the patient model and a TD walking pattern as described above. Instead of solving for independent muscle activation patterns, we now solve for synergy activation patterns Hopt. The optimization variables are hence H opt i , j τ R i and <sup>m</sup>W, which is a deviation from the pre-operative synergy weights (see below). Individual muscle activation patterns (including both subsets N and M) are then computed from the synergy activation patterns using the synergy weight vectors that describe the patient's motor control Wpre (see section Motor Control):

$${}^{m}a\_{i} = \left(\mathcal{W}^{pre} + \Delta^{m}W\right) \times {}^{opt}\_{i},\tag{14}$$

The same muscle co-contraction patterns (Wpre) are used to compute the pre- and post-operative CG, since we hypothesize that the orthopedic intervention does not alter muscle cocontraction patterns. Hence, we assume that the neural system will respond to the altered musculoskeletal geometry by changing the timing and magnitude of the pre-operative activation patterns H. A small deviation (W) from the original weights is allowed because the synergy matrices computed using NNMF typically do not capture 100% of the signal variability. In our formulation, W is normalized so that the maximum value in each vector equals one and W ≤ 0.05.

Because of the altered musculoskeletal geometry, muscle parameters and synergy-based constraints on muscle activations, it is likely that the muscles cannot generate the TD joint moments. The non-selective motor control imposed by the synergy weights might impose antagonistic co-contractions, hindering the moment generating capacity of a muscle. Alternatively, muscles could be excessively stretched when imposing TD gait kinematics and generate high passive forces. A considerable contribution from the residual actuators might thus be required to satisfy the moment equilibrium. The magnitude of the residuals moments required to match the desired moments defines the capability gap.

The CG is represented graphically as a function of the gait cycle for the different degrees of freedom (**Figure 3A**) and is quantified for each degree of freedom:

$$\:^j\text{CG} = \sum\_{i} \left| \:^j\pi\_i \right| / \sum\_{i} \left| \:^j\pi\_i \right|, \tag{15}$$

Furthermore, we provide a muscle report (**Figure 3B**) summarizing the intervals during which muscles operate with excessively short or long fiber lengths. Excessively stretched muscles are those whose passive force is >0.5 times their maximum isometric force (corresponding to ˜ l larger than 1.5 l mo). Muscles active at short lengths are those whose activations are >0.25 when operating at normalized fiber lengths smaller than 0.6, meaning that they are producing relatively little force.

#### Alternative Analyses

This framework is originally intended to work with the personalized neuro-musculoskeletal models as described above but it is also possible to exclude one or more of the personalization blocks. In fact, this procedure can be of value when assessing the importance of the different factors contributing to the impairment, as highlighted in section Case Studies. It is for instance possible to import a scaled, generic model, and perform the same analyses and surgeries, as envisaged for the personalized models, in order to evaluate the importance of the bony deformities in defining the impairment of the patient. Alternatively, it is possible to investigate alternative causes of the functional impairment by excluding the muscle force/length relationships or the constraints imposed by the motor control on the muscle activations when computing the gait performance (CG).

### Case Studies

We analyzed two representative patients with the proposed framework (**Table 1**). Both were diagnosed with diplegic CP and underwent SEMLS. For both patients, MRI images were acquired prior to the intervention (for details on the protocol, see Bosmans et al., 2014). A standardized clinical examination protocol (Desloovere et al., 2006) was conducted to evaluate the level of spasticity, strength, selectivity and range of motion. Three-dimensional gait analysis was performed before and after the intervention. Each participant was equipped with a set of reflective markers using the Vicon Plug-in-Gait marker set for lower limbs. Using a 10–15 camera motion capture system (Vicon Motion Systems, Oxford, UK) and two force plates (AMTI, Watertown, MA, USA), marker trajectories and ground reaction forces were collected during one static trial and at least three walking trials at self-selected walking speed. EMG signals were collected (Zerowire, Cometa, Italy) from eight major muscles per leg (rectus femoris, vastus lateralis, biceps femoris long, medial hamstrings, tibialis anterior, gastrocnemius, soleus, and gluteus medius). The local ethical committee approved all procedures, and written informed consent was obtained from the parents of the children prior to participation.

We created pre-operative personalized models using the experimental data (MRI, 3D gait analysis and clinical examination report) and post-operative personalized model by performing virtual surgeries according to the surgical plan of the actual intervention. For Patient 1 the following interventions were modeled: bilateral rectus femoris transfer, distal femur extension, and derotation osteotomy, patella advancement. Patient 1 also received a derotation of the tibia, but this was not modeled due to the fact that the MRI from which the musculoskeletal model was built did not include images of the feet and distal tibiae. For Patient 2 the following interventions were modeled: bilateral rectus femoris transfer, left distal femur derotation, right distal femur extension, and derotation osteotomy, patella advancement, right gastrocnemius and psoas release.

To simulate the Extension and Derotation Osteotomy intervention, the angle between the two cutting planes in the femur was modeled based on the knee extension deficit observed when testing the passive range of motion. For the femur Derotation Osteotomy, the anteversion angle was corrected to be equal to 0◦ in the simulated post-operative model in agreement with information provided by the orthopedic surgeons. Patella advancement was modeled by shortening the patella ligament by 2 cm, as reported in the surgical plan. Rectus Femoris Transfer was modeled in two steps. First, a via point in the femur reference frame was introduced in the middle of the muscle-tendon unit. Second, the insertion site was transferred to the semitendinosus tendon, while keeping the original length of the musculotendon unit unchanged. Our approach replicates the surgical procedure in which the rectus femoris is detached and reattached distally but left attached proximally, thus maintaining its function as a hip flexor. Muscle Release interventions were modeled by completely removing the muscle contribution from the generated moment, i.e., by setting the maximum isometric muscle force to zero. Although both patients additionally received botulinumtoxin injections, these were not included in the postoperative model given that their effect can be considered small given the time between the pre- and post-operative observations (10 and 13.5 months).

We compared the predicted motor performances of the patients in terms of the capability gap for the pre-and postoperative conditions (**Figure 3**). We tested the effect of different treatment options on the capability gap of the right leg of Patient 2 (**Figure 4**). This was done by creating different models with different angles of the cutting planes defining the extension osteotomy (20◦ and 25◦ ) as well as two different shapes of the wedge of bone (triangular and trapezoidal). We also investigated the effect of including/excluding the altered muscle parameters and motor control on the predicted gait performance (**Figure 5**) in the pre-operative condition. When excluding altered motor control from the analysis, muscle activations could vary independently for all muscles. Finally, we tested the ability of our framework to predict post-operative performance by comparing the predicted gait performance (CG) with the gait performance quantified based on the 3D gait analysis performed before and after the intervention (**Figure 6**). We analyzed the root mean square errors between patient and TD kinematics, corresponding to the Gait Variable Scores and Gait Profile Scores (Baker et al., 2009).

muscle release intervention. The continuous black lines represent the joint moments required for the model to reach the desired kinematics [i.e., typically developed (TD) gait cycle].

## RESULTS

#### Patient 1

The patient had near normal range of motion at the hip and ankle, but a bilateral knee extension deficit in both limbs, bilateral spasticity in most muscles, including rectus femoris, good strength in most muscles bilaterally but slightly lower strength in hip and knee extensors, as well as hip abductors, with overall good selectivity (**Table 1**). The pre-operative gait analysis (**Figure 6A**) indicates bilateral excessive knee flexion and ankle dorsiflexion, with incomplete hip extension at the end of stance. The right side hip presents excessive hip adduction.

Synergy analysis revealed that three synergies were sufficient to describe the pre-operative EMG signals in both legs. For comparison, previous work of the group found that, during walking, 57% of TD children use four synergies, whereas the remainder of the subjects uses three. Therefore, to take into account muscles for which no EMG were collected, four synergies were used for the CG computation.

An important CG was found bilaterally at the level of the knees and to a lesser extent hip adduction (**Figures 3A**, **6B**). The simulated interventions were able to reduce the calculated CG, especially at the knees, but also when averaged across all the joints. However, the effect on the CG at the level of the other joints was more variable. In particular, the left hip flexion and ankle dorsiflexion CG showed an increase after the surgery. The post-operative gait analysis shows that knee extension was restored successfully, whereas bilateral hip flexion increased after surgery. Ankle dorsiflexion was restored bilaterally, however, right ankle plantarflexion was still lacking.

The tuning of the muscle parameters was a necessary step to perform the aforementioned analyses. After applying the bony deformities to the model, most of the muscles would have operated at excessive values of ˜ l (**Figure 7**). Therefore, this model would have been unable to generate the required joint moments due to excessive passive forces generation, introducing excessive muscle activations to compensate these and resulting in high residual torques. For Patient 1, residual torques were as high as 45 Nm for the knee joint. Nevertheless, parameter tuning was able to bring the residual torque values below 1 Nm and to produce muscle activations closer to the measured EMG signals (**Figure 7**).

Within the framework, it is possible to evaluate the isolated effect of motor control deficit, by in- or excluding the motor control model when computing the post-operative CG. In **Figure 5**, the CG computed in both conditions is presented.

control.

For this patient, the CG gap is almost zero when the synergy constraints on muscle activations are not considered. The impaired motor control has thus a major contribution to the CG for this patient.

#### Patient 2

The patient's right side was more involved in terms of passive ROM, spasticity and muscle weakness (**Table 1**). More specific, hip adduction was slightly decreased whereas knee extension and dorsiflexion were more severely restricted in the right limb. Spasticity was present overall, although more pronounced in the distal compared to the proximal muscles. Overall, the patient had good muscle strength, with slightly lower values for the right proximal muscles and hip abductors and for the left hip extensors and plantarflexor muscles. The pre-operative gait analysis revealed bilateral increased knee flexion and hip adduction. Ankle dorsiflexion was increased on the left side, whereas on the right side a reversed second rocker was present. A reversed second rocker is defined by dorsiflexion during loading response and the first half of mid-stance followed by plantarflexion, whereas during normal gait, second rocker is characterized by plantarflexion followed by dorsiflexion. Insufficient hip extension in terminal stance was present at the left hip.

Synergy analysis revealed that four and three synergies could explain the pre-operative EMG signals for the left and right leg, respectively, thus leading to the use of five and four synergies in the CG computations. The CG in the preoperative condition reflected the reduced range of motion of the right leg. CG was higher for the right then for the left leg with muscles gastrocnemii, hamstrings, iliacus and psoas being excessively stretched (**Figure 3B**), leading to a large contribution of their passive forces to the CG. In addition, the comparison of the CG computed with and without the inclusion of the motor control (**Figure 5**) supports the interpretation that the vast majority of the CG is due to the aberrant musculoskeletal geometry and muscle properties, and not motor control.

The simulated treatment had a very different impact on the CG for the two legs. For the right leg, surgery massively reduced the CG. The simulated extension and derotation osteotomy in isolation reduced the CG generated by the hamstrings at the hip and knee at the beginning and end of the gait cycle. The use of a trapezoidal wedge, most commonly used

to correct large extension deficits, reduced the CG most by reducing excessive stretch in the muscles (**Figure 4**). The effect of the extension osteotomy was significant, but even after this treatment a large CG was present. This CG was most elevated in the stance phase for the ankle and knee joint and around toe-off for the hip joint. The muscle report indicated that the gastrocnemii, iliacus and psoas were markedly stretched during these intervals (**Figure 3B**). These muscles were targeted by the release procedure. The inclusion of these procedures within the framework further reduced the predicted CG as shown in **Figure 3A**. On the other hand, the pre-operative CG of the left leg was smaller and our framework predicted a slight increase in the CG after the intervention (**Figures 3A**, **6B**).The post-operative gait analysis confirms the positive outcome, with a bilateral marked improvement in the observed kinematics (**Figure 6**).

#### DISCUSSION

We introduced a novel and promising musculoskeletal modeling and simulation-based framework to assist clinicians in the treatment selection process to improve gait function in patients with CP. The salient feature of this framework is a comprehensive personalization of the models comprising subject-specific musculoskeletal geometry and muscle parameters as well as motor control. Furthermore, we introduced a GUI to simulate different orthopedic interventions and interactively modify the musculoskeletal models. As a result, the effect of several candidate orthopedic interventions on the gait performance, evaluated in terms of the patient's capability gap, can be evaluated. In comparison with a number of other studies that aimed to predict the outcome of orthopedic treatments (Hicks et al., 2011; Schwartz et al., 2016; Galarraga et al., 2017), our method differs by the fact that the predictions within our framework are not based on statistical methods. This allows the user to select different treatments or combinations thereof and to evaluate their combined or isolated effects.

In comparison with studies applying forward predictive simulations (Fox et al., 2009; Mansouri et al., 2016), our framework offers the possibility to include a variety of interventions and to fine-tune their parameters: it is for instance possible to combine a muscle transfer or patella advancement surgery with a femoral extension derotation osteotomy and to specify the amount of bony correction. A recently published paper (Lee et al., 2019) proposes a similar framework in which it is possible to predict the post-operative gait after an orthopedic surgery. However, several differences with respect to our work are worth noticing. The model of the motor control used by Lee et al. is obtained via a trajectory mimicking policy, which does not take into account the patient's coordination strategy, whereas we include this feature using EMG based muscle synergies.

While in Lee's work subject-specific muscle-tendon parameters are not included, we proposed a tuning based on detailed pre-operative information. In addition, there are differences in the proposed sets of interventions. Both studies implemented derotational osteotomies and muscle transfers, but Lee et al. included muscle-tendon lengthening while we included patella advancement and femoral extension osteotomy surgeries based on the interventions that are commonly used in CP treatment.

We demonstrated the potential of our framework based on two case studies. Indeed, we found a good agreement between the evolution of the predicted motor performance measured with the CG and the actual evolution of the patient kinematics. Using our framework, we were able to highlight the importance of taking into account the specific neurological and musculoskeletal impairments of the patients with CP when assessing gait dysfunction during the planning of an orthopedic intervention.

In Patient 1, virtual simulations of procedures correcting the bony deformities were able to reduce the CG. This agrees with the general trend of improved gait kinematics measured during the post-operative gait analysis. The CG computed without the inclusion of the motor control was almost negligible. Therefore, a hypothetical patient with the same musculoskeletal geometry and properties, but able to activate his/her muscles selectively, could be able to achieve a normal gait pattern. For this patient, the impaired motor control plays thus a major role in determining the altered gait pattern. It is interesting to note that, despite being mainly due to the impaired motor control, the CG of the patient is sensitive to the orthopedic intervention, suggesting an interaction between the motor control and musculoskeletal condition of the patient. In other words, motor control impairments might limit the compensation strategies that are available to patients with CP to counteract musculoskeletal deformities. In contrast, excluding the muscle synergies from the calculation of the CG for Patient 2 had little effect. This indicates that the abnormal muscle parameters, specifically the shortness of several muscles, are the main contributors to the altered gait pattern. This is evident in the muscle report, which indicates that many muscles, including gastrocnemius and psoas, operating at excessive lengths when the CG is high (**Figure 3B**). These findings support the need for additional muscular interventions on the right side, more specific the gastrocnemius and psoas release. The insights in the two case studies provided by our SimCP framework suggest that the underlying causes for the gait deviations might be very different in different patients, even when they present with similar gait patterns. The constraints imposed by the motor control and the musculoskeletal system should hence be taken into account during the clinical decision process.

Furthermore, we demonstrated that the implementation of a specific orthopedic intervention, more specific the choice of cutting planes and derotation magnitudes, might have a big influence on post-operative gait performance. By creating several models corresponding to different feasible variations in the surgical technique, it is possible to evaluate which variation has the highest potential to reduce the functional impairment of the patient. For example, increasing the angle of the bone wedge reduced the CG in Patient 2, whereas a trapezoid wedge reduced the CG even more. In the future, we plan to develop an optimization based procedure to automatically identify the combination of surgical parameters that minimizes the predicted CG.

The presented framework has still several limitations that will be addressed in future studies. First, there is a need for validation of the model prediction. Although we showed the potential of the framework here, many more cases are needed to demonstrate that the CG is a valid measure of gait performance. To this aim, we will compare the computed change in CG with the measured change in gait kinematics induced by the treatment in a large population of CP children. In addition, the different subcomponents of the framework need further validation. For example, we plan to use MRI-images collected post-operative to validate our implementation of the surgical interventions. Second, all analyses are based on static optimization with musculotendon units having rigid tendons. This approach was chosen to reduce computational time and for ease of implementation. We are currently developing a dynamic optimization implementation that takes into account muscle dynamics enabling the inclusion of a model of muscle spasticity (Falisse et al., 2018). These developments might further improve prediction accuracy. Third, the CG does not predict how the patient will walk in terms of the kinematics. The CG does not describe how a patient will move after treatment but how difficult it would be for him/her to achieve a normal gait pattern. In other words, the CG does not provide any insight in possible kinematic compensation strategies. However, the CG has the advantage of being fast and easy to compute, requiring only a few seconds per trial, and thus enabling the comparison of multiple treatment options. In the near future, we plan to include predictive simulations of gait kinematics in our framework building on the workflow for personalized modeling that we presented here.

A beta version of the developed GUI is freely available (https:// simtk.org/projects/simcp), which will enable the biomechanical community to create post-operative models in an easy way and therefore foster future research related to orthopedic interventions and pathological gait. Furthermore, the GUI as well as the concept of the CG can be applied to different populations (e.g., stroke) and research questions (e.g., strength training).

#### REFERENCES


To summarize, we conceptualized and developed a simulation-based framework that relies on highly personalized patient-specific models, including a description of the musculoskeletal geometry, the muscle parameters and the motor control. This framework is designed to assist clinicians in selecting the most promising treatment option for an individual patient solely based on pre-operative data. It is our aspiration that this in silico-informed clinical decision making framework will increase the number of positive treatment outcomes in ambulatory children with CP.

### DATA AVAILABILITY

The datasets generated for this study are available on request to the corresponding author.

#### ETHICS STATEMENT

This study was carried out in accordance with the recommendations of the local ethical committee [Commissie Medische Ethiek KU Leuven (Medical Ethics Committee UZ KU Leuven/Research)]. In accordance with the Declaration of Helsinki, written informed consent was obtained of the participants' parents prior to the experiment. The participants parents supervised the measurement session. The protocol was approved by the Medical Ethics Committee UZ KU Leuven/Research.

#### AUTHOR CONTRIBUTIONS

LP, IJ, FD, KD, and AH contributed conception and design of the study. EP collected the data. AV and GM provided clinical insights for the methods development. LP, HK, AF, MW, SV, and HH contributed in developing the methods. LP, IJ, and FD wrote sections of the manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.

#### FUNDING

Project was funded with IWT-TBM Grant Number: 140184. HK was funded by a H2020-MSCA individual fellowship (796120). AF was funded by the Research Foundation Flanders (FWO), Ph.D. grant 1S35416N.


during normal and pathological gait. J. Orthop. Res. 32, 1406–1415. doi: 10.1002/jor.22698


estimated muscle forces from musculoskeletal models of children with cerebral palsy. Gait Posture 65, 213–220. doi: 10.1016/j.gaitpost.2018.07.172


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Pitto, Kainz, Falisse, Wesseling, Van Rossom, Hoang, Papageorgiou, Hallemans, Desloovere, Molenaers, Van Campenhout, De Groote and Jonkers. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Energy Expenditure Estimation During Crutch-Orthosis-Assisted Gait of a Spinal-Cord-Injured Subject

Florian Michaud\*, Francisco Mouzo, Urbano Lugrís and Javier Cuadrado

Laboratory of Mechanical Engineering, University of La Coruña, Ferrol, Spain

Determination of muscle energy expenditure by computer modeling and analysis is of great interest to estimate the whole body energy consumption, while avoiding the complex character of in vivo experimental measurements for some subjects or activities. In previous papers, the authors presented optimization methods for estimating muscle forces in spinal-cord-injured (SCI) subjects performing crutch-assisted gait. Starting from those results, this work addresses the estimation of the whole body energy consumption of a SCI subject during crutch-assisted gait using the models of human muscle energy expenditure proposed by Umberger and Bhargava. First, the two methods were applied to the gait of a healthy subject, and experimentally validated by means of a portable gas analyzer in several 5-min tests. Then, both methods were used for a SCI subject during crutch-assisted gait wearing either a passive or an active knee-ankle foot orthosis (KAFO), in order to compare the energetic efficiency of both gait-assistive devices. Improved gait pattern and reduced energy consumption were the results of using the actuated gait device. Computer modeling and analysis can provide valuable indicators, as energy consumption, to assess the impact of assistive devices in patients without the need for long and uncomfortable experimental tests.

#### Edited by:

Benjamin J. Fregly, Rice University, United States

#### Reviewed by:

Moon Ki Jung, Imperial College London, United Kingdom Paweł Pławiak, Cracow University of Technology, Poland

#### \*Correspondence:

Florian Michaud florian.michaud@udc.es

Received: 29 March 2019 Accepted: 04 July 2019 Published: 18 July 2019

#### Citation:

Michaud F, Mouzo F, Lugrís U and Cuadrado J (2019) Energy Expenditure Estimation During Crutch-Orthosis-Assisted Gait of a Spinal-Cord-Injured Subject. Front. Neurorobot. 13:55. doi: 10.3389/fnbot.2019.00055

Keywords: energy expenditure, SCI subject, crutch-assisted gait, KAFO, human modeling and analysis, muscle recruitment problem

## INTRODUCTION

In the last decade, many mechanical and, more recently, electromechanical (or hybrid) devices have been developed to allow spinal-cord-injured (SCI) patients to stand and walk (White et al., 2014; Cuadrado et al., 2019). At the moment, the additional use of crutches is generally required for gait stability. Despite these technological advances, most SCI subjects prefer the wheelchair to move for energetic efficiency reasons (Merati et al., 2000). The gait efficiency can be defined as the percentage of energy input that is transformed into useful work. Use of a cane or a pair of crutches requires about 33% more energy than normal walking (Mcbeath et al., 1974). In addition, some devices (KAFO), don't allow some joints to move, which implies another gait pattern even less efficient. Moreover, since structures of the upper extremities are designed primarily for prehensile activities, not to walk, many patients suffer from shoulder and arm injuries (Lee and McMahon, 2002).

Energy cost in subjects using crutches was mainly studied by means of experimental measurements (Mcbeath et al., 1974; Waters and Mulroy, 1999; Merati et al., 2000), generally using a gas analyzer. IJzerman et al. (1998) proposed an alternative method to estimate the energy expenditure of paraplegic gait using measurements of heart rate and crutch forces. In all the previous methods, the patient must go through experimental tests lasting several minutes while wearing not only the assistive devices, but also measuring devices as the gas analyzer, which is rather uncomfortable. This may be too demanding for many patients. Conversely, the method proposed in this paper just requires the motion-force-EMG capture of a gait cycle, which is much more achievable for most patients. Using the captured data, a musculoskeletal model of the subject provides the joint efforts and muscle forces, activations and excitations and, then, models of human muscle energy expenditure proposed in the literature are applied to the results to estimate the energy cost of the measured gait.

Various Hill-based models can be found in the literature to calculate the human muscle energy expenditure (Minetti and Alexander, 1997; Umberger et al., 2003; Bhargava et al., 2004; Houdijk et al., 2006). Miller proposed a comparison of these models for the gait of healthy subjects (Miller, 2014). According to his recommendations, the models of Umberger and Bhargava have been implemented in this work to calculate the energy cost of SCI subjects during crutch gait. Since both muscle energy expenditure models are based on the Hill's muscle model, they require the knowledge of some muscular parameters. Such parameters had been obtained by the authors in a previous work (Michaud et al., 2017), using physiological static optimization (Ou, 2012), and a customized musculoskeletal model of the SCI subject.

The objective of this work is to estimate the energetic cost of the crutch-orthosis-assisted gait of a SCI subject so that comparison from the energetic point of view may be established between two assistive devices: a passive and an active KAFO. The latter was obtained by simply adding to the passive device a motor and gearbox at knee level and an inertial sensor at shank level, so that motion intention is detected, and knee flexion/extension is automatically produced during swing. First, the methods for energetic cost estimation were applied to the gait of a healthy subject, and experimentally validated by means of a portable gas analyzer on several 5-min tests. Then the same methods were applied to the SCI subject.

The motivation of the work comes from the fact that walking is essential for the general health state of SCI subjects, thus overcoming the sedentarism due to permanent use of the wheelchair. Orthotic devices enable some SCI subjects to walk, but sometimes the energetic cost of the resulting gait is so high that patients reject this option. Therefore, evaluation of the energetic cost of gait allows to assess, even since the early training period, whether a certain orthotic device is promising for actual use by the patient in the mid and long terms. Moreover, it can provide valuable data to track the training progress. However, experimental estimation of energetic cost through 5-min tests is not feasible in most cases and, then, the alternative of getting an acceptable estimation from a short motion/force/EMG capture appears as greatly interesting.

Contributions of the paper are: (i) the detailed description of Umberger's and Bhargava's methods for the estimation of energetic cost, providing all the necessary elements required for implementation of the methods; (ii) the experimental validation of both methods for healthy gait by comparison with the results obtained from 5-min tests; (iii) the application of both methods to crutch-orthosis-assisted gait for the cases of passive and active orthoses; (iv) the comparison, in terms of energetic cost, between assisted gait with passive and active orthoses.

The remaining of the paper is organized as follows: section Materials and Methods describes the experiments and models used in this work; section Results presents the two energy expenditure models implemented; and sections Discussion and Conclusion provide, respectively, the obtained results and their corresponding discussion.

### MATERIALS AND METHODS

#### Subjects

The SCI subject was a 49-years-old male of mass 82 kg and height 1.90 m, with injury corresponding to a Lower Extremity Muscle Score (LEMS) of 13/50. His injury allowed him a normal motion of the upper extremities and trunk, while partially limiting the actuation at the hips and right knee due to partial or no muscular innervation. Both motor and sensory functions at ankles and left knee were totally lost. Therefore, in order to walk he required the assistance of a passive KAFO at the left leg, a passive ankle-foot orthosis at the right leg and two forearm crutches. However, permanent left-knee extension, even during the swing phase, made gait become very uncomfortable as hip raising was required for swing, thus demanding high efforts which led to fatigue quickly. Consequently, in daily life he mainly used a wheelchair to move and resorted to the mentioned assisted gait only occasionally and during short periods of time.

To improve SCI subjects' mobility, a research prototype of a low-cost active KAFO was developed (Font-Llagunes et al., 2016). Starting from a conventional passive device, an electric motor (EC90 flat of 160 W) plus a Harmonic gearbox (CCD-P-20-100- C-I with a ratio of 100:1) was placed at knee level substituting the external original joint, so as to launch the swing cycle when motion intention was detected by an inertial sensor placed at shank level, in order to avoid foot-collision with the ground. After some training sessions, the subject was able to walk with the mentioned prototype, featuring an actuated left-knee flexion of 30 degrees.

In order to assess the subject's muscle activity at hip and knee levels, surface EMG measurements were taken during simple exercises.

The healthy subject was a 28-years-old male of mass 85 kg and height 1.87 m.

#### Instrumentation and Data Collection

Subjects walked over two embedded force plates (AMTI, AccuGait, sampling at 100 Hz), with the help of two instrumented crutches for ground reaction measurement in the case of the SCI subject (Lugris et al., 2013), while their motion was captured by 12 optical infrared cameras that computed the position of 37 optical markers attached to the subjects' body, and 3 more for each crutch in the case of the SCI subject. Moreover, 10 EMG signals were recorded (2 at the right leg, 3 at the trunk, 4 at the right arm, and 1 at the left arm) for the SCI subject and 10 for the healthy subject at the lower extremities (**Figures 1A**, **3A**). A complete gait cycle was captured of the SCI subject walking with (i) the passive orthosis owned by the subject; (ii) the

active orthosis with the motor locking the knee; (iii) the active orthosis with the motor moving the knee. He used the 2-point crutch-assisted gait cycle shown in **Figure 2**.

For the healthy subject, 21 complete gait cycles were recorded at seven different speeds (between the free selected speed and fast speed) for energetic cost calculation. The energy expenditure was also measured experimentally by means of a portable gas analyzer (Cortex MetaMax 3B) during two 5-min tests at free selected speed and fast speed (**Figure 3B**). This experimental method requires that the subject maintains a constant speed during at least 5 min. Since this was thought to be too demanding for SCI subjects, it was decided to carry out the experimental validation with a healthy subject.

Calculations were performed on an Intel <sup>R</sup> CoreTM i7–6,700 K, at 4.00 GHz with 16 Gb of RAM.

#### Model Description

For the healthy subject, the human 3D model consisted of 18 anatomical segments: pelvis, torso, neck, head, and two hind feet,

FIGURE 3 | Energy consumption for a healthy subject: (A) motion-force-EMG capture; (B) 5-min test with portable gas analyzer. (Written informed consent was obtained from the individual for the publication of these images. FlM, the main author, is on this picture).

forefeet, shanks, thighs, arms, forearms and hands. For the SCI subject (**Figure 1B**), the same model was used, but the hands were rigidly connected to the crutches, and the orthosis at the left leg was embedded in the corresponding body links (thigh, calve,

and foot). The segments were linked by ideal spherical joints, thus defining a model with 57◦ of freedom (6 of the base body plus 17 × 3 of the joints). The geometric and inertial parameters of the model were obtained, for the lower limbs, by applying correlation equations from a reduced set of measurements taken on the subject, following the procedures described in Vaughan et al. (1999). For the upper part of the body, data from standard tables (Ambrosio and Kecskemethy, 2007) was scaled according to the mass, and height of the subject. In order to adjust the total mass of the subject, a second scaling was applied to the inertial parameters of the upper part of the body. Assistive devices were taken into account by altering the inertia properties of hands (crutches) and thigh, calve and foot (orthosis). Mixed (natural and joint) coordinates along with matrix-R formulation (de Jalon and Bayo, 1994) were applied to obtain the joint drive torques along the motion using the in-house developed MBSLIM library (Dopico, 2016) programmed in FORTRAN language.

The musculoskeletal model was customized to the SCI subject according to his muscle activity (previously measured through EMG). The musculoskeletal model (**Figure 1C**) was composed of 112 muscles for the whole body: 28 at the right hip, 5 at the right knee, 21 at the left hip, 6 at the trunk, 15 at each shoulder, and 11 at each elbow. For the healthy subject, only the lower extremities were considered with their 92 muscles (43 muscles per leg plus 6 at trunk); the energy consumption of upper body muscles was considered into the basal energy consumption. Muscle properties were taken from Delp et al. (2007). The Hill's muscle model shown in **Figure 4** was employed, being considered both the tendon and the muscle, with its contractile (CE), and passive (PE) elements. The muscle recruitment problem was addressed by means of the physiological static optimization method (Ou, 2012) using in-house developed code programmed in Matlab, and calling to fmincon Matlab's function for optimization, thus getting the histories of muscle forces, activations and excitations. Energy expenditure calculations were also programmed in the same in-house code.

#### Energy Expenditure

Once muscular activity obtained as previously explained, results were validated with the experimental EMG measurements. The obtained activation, length, velocity and force of the muscles were used as input for the two models of energy expenditure. Both of them are based on the first law of thermodynamics. According to this law, the total rate of energy consumption **E**˙, is equal to the rate at which heat is liberated, **H**˙ , plus the rate at which work is done, **W**˙ :

$$
\dot{E} = \dot{H} + \dot{W} \tag{1}
$$

#### Umberger's Model

Umberger's muscle energy expenditure model (Umberger et al., 2003) considers the activation heat rate (**h**˙**A**), the maintenance heat rate (**h**˙**M**), the shortening/lengthening heat rate (**h**˙ **SL**), and the mechanical work rate of the contractile element of the muscle (**w**˙ **CE**), to determine the total rate of muscle energy expenditure (**E**˙). The relation is given by the sum of this four terms expressed in (2), where **E**˙ is calculated for each muscle in W.kg−<sup>1</sup> .

$$
\dot{E} = \dot{h}\_A + \dot{h}\_M + \dot{h}\_{SL} + \dot{\mathbf{w}}\_{CE} \tag{2}
$$

#### **Activation and maintenance heat rate**

A combined expression of the activation and maintenance heat rate is used for this first term,

$$
\hbar\_A + \hbar\_M = \hbar\_{AM} = 1.28 \times \%FT + 25\tag{3}
$$

where %FT represents the percentage of fast twitch found in Johnson et al. (1973).

#### **Shortening and lengthening heat rate**

During CE shortening (**VM**(**t**) ≤ 0) and lengthening (**VM**(t) > 0), the rate of heat production is modeled as the product of a coefficient α**<sup>S</sup>** and **VM**, the velocity of the muscular contractile element:

$$\dot{\boldsymbol{h}}\_{\rm SL}(t) = \begin{cases} -\boldsymbol{\mathfrak{a}}\_{\rm S(ST)} \tilde{\boldsymbol{V}}\_{\rm M}(t)(1 - \% \boldsymbol{FT}/100) - \boldsymbol{\mathfrak{a}}\_{\rm S(FT)} \tilde{\boldsymbol{V}}\_{\rm M}(t)(\% \boldsymbol{FT}/100) & \text{if } \boldsymbol{V}\_{\rm M}(t) \le 0 \\\\ \boldsymbol{\mathfrak{a}}\_{\rm L} \tilde{\boldsymbol{V}}\_{\rm M}(t) & \text{if } \boldsymbol{V}\_{\rm M}(t) > 0 \end{cases} \tag{4}$$

with the constant terms α**S**(**ST**) = 4×25 **V**˜ **<sup>M</sup>**(**MAX**−**ST**) , α**S**(**FT**) = 153 **V**˜ **<sup>M</sup>**(**MAX**−**FT**) , α**<sup>L</sup>** = 4α**S**(**ST**) , **<sup>V</sup>**˜ **<sup>M</sup>** <sup>=</sup> **VM l M** 0 , **V**˜ **<sup>M</sup>**(**MAX**−**FT**) = **V** M **MAX l M** 0 , **V**˜ **<sup>M</sup>**(**MAX**−**ST**) = **V**˜ **<sup>M</sup>**(**MAX**−**FT**)/2.5, and **V M MAX** = **l M** 0 /0.1 (**l M** 0 the optimal fiber length).

#### **Mechanical work rate**

The specific mechanical work rate is given by:

$$
\dot{\mathbf{w}}\_{\rm CE}(t) = -(F\_{\rm CE}^{\rm M}(t)V\_{\rm M}(t))/m\tag{5}
$$

so that this value is positive for concentric effort and negative for eccentric effort. **m** represents the mass of the muscle.

#### **Total energy expenditure scaled**

Equation (2) provides the energy expenditure of the muscle for the case of full activation and the contractile element length of the muscle (**l <sup>M</sup>**) is equal to the optimal muscular length (**l M** 0 ) of the contractile element. Scaling factors are needed to account for the length and activation dependence of **h**˙**AM** (dependence factor **AAM**) and **h**˙ **SL** (dependence factor **ASL**), and the dependence of the total heat rate on the metabolic working conditions (S = 1 for primarily anaerobic conditions and S = 1.5 for primarily aerobic conditions),

$$\dot{E}(t) = \begin{cases} \dot{h}\_{AM}A\_{AM}(t)\mathbf{S} + \dot{h}\_{SL}(t)A\_{SL}(t)\mathbf{S} + \dot{w}\_{\rm CE}(t) & \text{if } l^{\mathbf{M}}(t) \le l\_{\mathbf{0}}^{\mathbf{M}} \\\\ (\mathbf{0}.4 \times \dot{h}\_{AM} + \mathbf{0}.6 \times \dot{h}\_{AM} \times F\_{\mathbf{0}}^{\mathbf{M}})A\_{AM}(t)\mathbf{S} + \dot{h}\_{SL}(t)A\_{SL}(t)\mathbf{S} + \dot{w}\_{\rm CE}(t) & \text{if } l^{\mathbf{M}}(t) > l\_{\mathbf{0}}^{\mathbf{M}} \end{cases} \tag{6}$$

with **AAM**(**t**) = **A**(**t**) 0.6 , **ASL**(**t**) = **A**(**t**) **2** , and

$$A(t) = \begin{cases} u(t) & \text{if } u(t) \le a(t) \\ (u(t) + a(t))/2 & \text{if } u(t) > a(t) \end{cases} \tag{7}$$

where **u**(**t**) and **a**(**t**) represent the excitation and activation of the muscle, respectively.

#### Bhargava's Model

Bhargava's model presents some similarities with the previous one, since the general expression is similar to equation (2) with an additional a basal metabolic rate **h**˙ **B**:

$$
\dot{E} = \dot{h}\_A + \dot{h}\_M + \dot{h}\_{SL} + \dot{w}\_{CE} + \dot{h}\_B \tag{8}
$$

However, expressions of the components are slightly different.

#### **Activation heat rate**

$$
\dot{h}\_A = \phi f\_{FT} \dot{A}\_{FT} \mu\_{FT}(t) + \phi f\_{ST} \dot{A}\_{ST} \mu\_{ST}(t) \tag{9}
$$

$$\text{with} \qquad \phi = 0.06 + \exp(-t\_{\text{stim}}\mu(t)/\tau\_{\phi}), \tag{10}$$

$$\mu\_{FT}(t) = 1 - \cos(\frac{\pi}{2}\mu(t))\tag{11}$$

$$\text{and}\quad\mu\_{ST}(t) = \sin(\frac{\pi}{2}\mathfrak{u}(t)),\tag{12}$$

and the constant terms: **fFT** = %**FT**/100, **fST** = 1 − %**FT**/100, **A**˙ **FT**= 133 and **A**˙ **ST** = 40.

**Maintenance heat rate**

$$\dot{h}\_M(t) = L(\tilde{l}^M(t))f\_{FT}\dot{M}\_{FT}u\_{FT}(t) + L(\tilde{l}^M(t))f\_{ST}\dot{M}\_{ST}u\_{ST}(t) \tag{13}$$

where **L**( ˜**l <sup>M</sup>**(**t**)) is a function that models the dependence on muscle length:

$$L(\tilde{l}^{\mathcal{M}}(t)) = \begin{cases} 0.5 & \text{if } \quad \tilde{l}^{\mathcal{M}}(t) \le 0.5\\ \tilde{l}^{\mathcal{M}}(t) & \text{if } \quad 0.5 < \tilde{l}^{\mathcal{M}}(t) \le 1\\ -2(\tilde{l}^{\mathcal{M}}(t)) + 3 \text{ if } \quad 1 < \tilde{l}^{\mathcal{M}}(t) \le 1.5\\ 0 & \text{if } \quad \tilde{l}^{\mathcal{M}}(t) > 1.5 \end{cases} \tag{14}$$

with ˜**l <sup>M</sup>** = **l <sup>M</sup>**/**l M** 0 and the maintenance heat rate constants: **M**˙ **FT** = 111 and **M**˙ **ST** = 74.

#### **Shortening and lengthening heat rate**

During CE shortening and lengthening, the rate of heat production is modeled as the product of a coefficient α**<sup>S</sup>** and **VM**, as it happened in Umberger's model,

$$
\dot{h}\_{SL}(t) = -\alpha\_S(t)\tilde{V}\_M(t). \tag{15}
$$

However, expression of α**<sup>S</sup>** is different:

$$\mathfrak{a}\_{\mathcal{S}}(t) = \begin{cases} 0.16 F\_0^{\mathcal{M}} + 0.18 F\_{\text{CE}}^{\mathcal{M}}(t) & \text{if} \quad V\_{\mathcal{M}}(t) \le 0 \\ 0.157 F\_{\text{CE}}^{\mathcal{M}}(t) & \text{if} \quad V\_{\mathcal{M}}(t) > 0 \end{cases} \tag{16}$$

#### **Basal heat rate**

In addition, Bhargava's model proposes a basal metabolic rate calculated from a frog skeletal model at 0◦C and given by:

$$
\dot{h}\_{\text{B}} = 0.0225 \tag{17}
$$

#### **Mechanical work rate**

Both models consider the same expression for the mechanical work rate:

$$
\dot{\mathbf{w}}\_{\rm CE}(t) = -(F\_{\rm CE}^{\rm M}(t)V\_{\rm M}(t))/\mathfrak{m} \tag{18}
$$

#### Total Energy Consumption

Finally, the total energy consumption **E**˙ of the full body during a full stride was obtained for both models by:

$$\dot{E} = \left(\sum\_{i=1}^{n} \left(\frac{\int\_{t=0}^{t\_{cycle}} (\dot{E}\_i(t) \times m\_i) dt}{t\_{cycle}}\right) + k\_B \times m\_{residual}\right) / m\_{subject} \tag{19}$$

where **msubject** is to the mass of the subject, **n** the number of muscles, **tcycle** the time of a gait cycle, and

$$m\_{residual} = m\_{subject} - \sum\_{i=1}^{n} m\_i \tag{20}$$

Lastly, **k<sup>B</sup>** represents the basal added metabolic rate of 1.2 W.kg−<sup>1</sup> which corresponds to the energy consumption for upright quiet standing (Waters and Mulroy, 1999).

#### RESULTS

Before estimating energy consumption, the musculoskeletal model and the estimation of muscular activity were validated with the EMG measurements for the healthy subject. As there is no clear relationship between EMG amplitude and muscle force (Hof, 1997), the comparison was focused on the shape of the activity patterns, using normalized values. Good correlations between muscular activations and EMG measurements were obtained (**Figure 5**), with a mean R correlation over 0.70.

Then, both energy expenditure models were applied and experimentally validated for the healthy subject. Twenty one complete gait cycles were recorded at seven different speeds, ranging between his free selected speed (75 m/min) and his fast speed (90 m/min). As some variability was observed in the obtained values of energy cost for different tests at the same speed, a mean value is represented in **Figure 6**. Two experimental tests were done, at free selected speed and fast speed, respectively, to validate the models.

As it can be seen in **Figure 6**, a linear relation was obtained between gait speed and energy consumption, showing a good correlation with both experimental measurements and literature (Waters and Mulroy, 1999). Since a constant discrepancy of the results was observed with respect to the measured energy values, the model was calibrated with such a constant (0.12 W.kg−<sup>1</sup> for Umberger's model and 1.9 W.kg−<sup>1</sup> for Bhargava's model). This calibration can be considered as an adjustment of the whole-body basal metabolic rate kB.

On the other hand, three gait cycles were compared for the SCI subject, one with each assistive gait device presented before: (i) passive orthosis owned by the subject; (ii) active orthosis with motor locking the knee; (iii) active orthosis with motor moving the knee. After some few training sessions, the SCI subject was able to walk with confidence wearing the active KAFO, achieving the same self-selected speed of 33 m/min in the three cases.

While the walking speed was the same in the three cases, some kinematic differences could be observed (**Table 1**). First, the step length, of 45 and 66 cm for the right and left legs, respectively, using the original KAFO, changed to 58 cm for both sides when using the active KAFO with motor moving the knee. The initial circumduction of the left foot (KAFO's leg) of 11.5 cm with the original KAFO was reduced to 7.25 cm thanks to the actuated knee flexion. Pelvic maximum rotations

were reduced from −27.6 and 44.8 to −22.5 and 35.3◦ in the transverse plane, and from 19.18 to 15.23◦ in the frontal plane. Finally, the mediolateral center of mass (COM) displacement was significantly reduced from 13.48 to 11.54 cm, while the vertical displacement was almost the same in the three cases.

Instrumented crutch measurements did not show significant differences between devices. A mean load of 20% of the bodyweight was observed during the gait cycle, with peaks of 55% (left crutch), and 40% (right crutch) at swing start. Estimated joint reaction forces at shoulder were similar too, with peaks between 190 and 225% (left shoulder) of the bodyweight.

In order to check the validity of the inputs provided to the energetic cost calculations for the SCI subject, the muscle activations were compared with experimental EMG measurements. As it can be observed in **Figure 7**, acceptable correlations were obtained, with a mean R correlation of more than 0.55.

**Figure 8** and **Table 1** show the estimated energy consumptions yielded by both models. The energy cost obtained with the original KAFO was 3.49 W.kg−<sup>1</sup> for Umberger's model, and 3.11 W.kg−<sup>1</sup> for Bhargava's. Wearing the active KAFO with motor locking the knee, it was 3.56 and 3.13 W.kg−<sup>1</sup> . Finally, wearing the active KAFO with motor moving the knee, the energy cost was 3.28 and 3.02 W.kg−<sup>1</sup> .

#### DISCUSSION

The energy expenditure of a healthy male during gait was calculated, based on the muscular magnitudes obtained from a motion-force-EMG capture and a musculoskeletal model of the subject, through the application of two methods found in the literature (Umberger's and Bhargava's), and was validated by experimental measurements and references from literature for several gait velocities. Results showed that calibration of the methods is necessary to evaluate the whole-body basal metabolic rate. However, the slopes (energy cost vs. gait speed) obtained with both methods were coincident and agreed with those from experiments and literature, which is the essential point to


TABLE 1 | Comparison of obtained results with the three gait-assistive devices.

compare two activities performed by the same subject, and using the same model. Based on these findings, both methods were applied to a SCI subject walking with the help of crutches and wearing different gait-assistive devices.

The self-selected gait velocity achieved by the SCI subject with the three devices was of 33 m/min, which is higher than the velocity corresponding to his LEMS (20.2 m/min) according to Waters and Mulroy (1999). This discrepancy can be explained by the moderately strong linear relationship (R = 0.64) between walking speed and the LEMS, and by the fact that the subject was tall and athletic.

The SCI subject carried out few training sessions with the active KAFO, and probably needed more experience to show a significant evolution with respect to the passive device, as observed in Font-Llagunes et al. (2016). However, some improvements of the gait pattern thanks to the knee actuation

provided by the KAFO were already detected, as symmetry in the step lengths, reduced circumduction and reduced pelvic rotation. COM displacements are generally used as indicators of balance control to reflect the whole body motion during gait. While the vertical displacement was almost the same for the three cases and was close to that of healthy subjects [3.61 cm at 1 m/s (Orendurff et al., 2004)], the mediolateral displacement reflected differences in gait pattern and with respect to healthy subjects [5.96 cm at 1 m/s (Orendurff et al., 2004)].

Ground force reactions measured by the instrumented crutches did not highlight any differences between the devices used, likely because of the short training period with the new device. However, the obtained values showed the demanding use of the upper extremities, which are primarily not designed to walk and to put up with such loads.

Same observations can be done regarding the joint reaction force at shoulders, with estimated peaks >220% of the bodyweight. Westerhoff et al. (2012) reported maximum loads of up to 170% during in vivo measurement of shoulder loads during crutch-assisted walking, but subjects were not suffering from any lower limb disability. Highest peaks at the left arm were observed during the left-leg swing (leg wearing the KAFO), likely because the subject needed to compensate the instability of the left foot and the lack of force in the right leg, and to avoid the foot contact with the ground.

Correlations observed between EMG measurements and muscle activations for the SCI subject were acceptable and allow trusting in the input used to calculate the energy cost.

The estimated energy consumptions presented for the SCI subject were not calibrated because the 5-min tests carried out by the healthy subject were not possible for him. Bhargava's results were lower than Umberger's. However, the same order was maintained among the three devices. The active KAFO with locked knee showed the highest value, a bit more than the passive KAFO. This difference could be explained by the additional mass of the motor. The motor actuation reduced significantly (almost 8% for Umberger and 3.5% for Bhargava) the estimated energy consumption despite the short period of training with the device.

As a reference, at the speed developed by the SCI subject (33 m/min), a healthy subject should consume 2.385 W.kg−<sup>1</sup> (Waters and Mulroy, 1999). Continuing with Waters' references for SCI subjects, for a LEMS of 13, the subject should consume 149.8% more than a healthy subject at the same speed. This would correspond to an energy consumption of 3.57 W.kg−<sup>1</sup> , which is close to the values obtained with Umberger for the two first cases (3.49 and 3.56 W.kg−<sup>1</sup> ). In the third case the motor actuation produces the knee flexion/extension, so that the LEMS could be increased to 14. Then the corresponding energy consumption increase should be of 145.5% with respect to a healthy subject, thus leading to a consumption of 3.47 vs. 3.28 W.kg−<sup>1</sup> obtained with Umberger. While results obtained without calibration are closer to the mentioned references for Umberger's model, slopes (energy cost vs. LEMS) are closer (gradient of −0.1) using Bhargava's model (gradient of −0.11) than Umberger's (gradient of −0.28).

### CONCLUSION

A method to estimate the energetic cost of the gait of SCI subjects walking with the help of knee-ankle-foot orthosis and crutches has been proposed in this paper. The method just requires to make some motion-force-EMG captures of a subject's gait cycle and, using the generated data, perform an inversedynamics analysis, and muscle force sharing optimization on a musculoskeletal model of the subject, so that Umberger's or Bhargava's method can be applied to the obtained results in order to get an estimation of the energy consumption. Therefore, unlike experimental methods reported in the literature which require tests lasting several minutes, the method proposed here only needs that the subject walks during two or three gait cycles, so that one full gait cycle is captured in the gait analysis lab. This makes the method feasible even for the training period, and even for subjects who will not be capable of walking for several minutes after the training period has been completed. However, the advantage may also be a disadvantage, as lower accuracy in the estimation can be expected due to the short duration of exercise on which it is based.

Some limitations can be pointed out in this work. The first limitation is that one single SCI subject was considered in the study, but finding hip-flexion able SCI candidates for actively assisted gait is not easy and developing customized devices for them is expensive and time-consuming. A second limitation is that the SCI subject performed few training sessions with the active orthotic device; it would had been desirable to continue the study for a longer period and see the evolution of the energetic cost as the user became more acquainted with the device.

Future works could go in the direction of overcoming the limitations previously described. Repeating the study for more SCI subjects and spanning longer periods, from the initial training in the use of active orthoses to the stage when a strong skill is attained by the user, would allow to further confirm the validity of the method and its ability to provide a clue, already during the training period, on whether the particular orthotic device will be successful for the particular patient in the mid and long terms.

### DATA AVAILABILITY

The datasets generated for this study are available on request to the corresponding author.

#### ETHICS STATEMENT

This study was carried out in accordance with the recommendations of guidelines provided by the Committee of Ethics of the University of A Coruña with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the Committee of Ethics of the University of A Coruña.

#### AUTHOR CONTRIBUTIONS

FlM designed and performed the experiments with the supervision of UL, derived the models, and analyzed the data.

#### REFERENCES


FlM and JC wrote the manuscript in consultation with UL and FrM.

#### FUNDING

This work was funded by the Spanish MINECO under project DPI2015-65959-C3-1-R, co-financed by the EU through the EFRD program, and by the Galician Government under grant ED431B2016/031. Moreover, FlM would like to acknowledge the support of the Spanish MINECO by means of the doctoral research contract BES-2016-076901, co-financed by the EU through the ESF program.

#### ACKNOWLEDGMENTS

We like to acknowledge: J. Castro, from Efisaude S.L., for the experimental measurements of energy consumption; R. Garcia, the SCI subject, for his voluntary participation in this project; and B. Umberger, for his kindness in answering our questions about his model.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Michaud, Mouzo, Lugrís and Cuadrado. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Design of the Cooperative Actuation in Hybrid Orthoses: A Theoretical Approach Based on Muscle Models

Francisco Romero-Sánchez\*, Javier Bermejo-García, Jorge Barrios-Muriel and Francisco J. Alonso

Department of Mechanical Engineering, Energy and Materials, University of Extremadura, Badajoz, Spain

Hybrid orthoses or rehabilitation exoskeletons have proven to be a powerful tool for subjects with gait disabilities due to their combined use of electromechanical actuation to provide motion and support, and functional electrical stimulation (FES) to contract muscle tissue so as to improve the rehabilitation process. In these devices, each degree of freedom is governed by two actuators. The main issue arises in the design of the two actuation profiles for there to be natural or normative gait motion in which the two actuators are transparent to each other. Hybrid exoskeleton control solutions proposed in the literature have been based on tracking the desired kinematics and applying FES to maintain the desired motion rather than to attain the values expected for physiological movement. This work proposes a muscle-model approach involving inverse dynamics optimization for the design of combined actuation in hybrid orthoses. The FES profile calculated in this way has the neurophysiological meaningfulness for the device to be able to fulfill its rehabilitative purpose. A general scheme is proposed for a hybrid hip-knee-ankle-foot orthosis. The actuation profiles, when muscle tissue is fatigued due to FES actuation are analyzed, and an integrated approach is presented for estimating the actuation profiles so as to overcome muscle peak force reduction during stimulation. The objective is to provide a stimulation profile for each muscle individually that is compatible with the desired kinematics and actuation of the orthosis. The hope is that the results may contribute to the design of subject-specific rehabilitation routines with hybrid exoskeletons, improving the exoskeleton's actuation while maintaining its rehabilitative function.

#### Edited by:

Benjamin J. Fregly, Rice University, United States

#### Reviewed by:

Reza Sharif Razavian, Imperial College London, United Kingdom Borna Ghannadi, University of Waterloo, Canada

#### \*Correspondence:

Francisco Romero-Sánchez fromsan@unex.es

Received: 26 March 2019 Accepted: 11 July 2019 Published: 31 July 2019

#### Citation:

Romero-Sánchez F, Bermejo-García J, Barrios-Muriel J and Alonso FJ (2019) Design of the Cooperative Actuation in Hybrid Orthoses: A Theoretical Approach Based on Muscle Models. Front. Neurorobot. 13:58. doi: 10.3389/fnbot.2019.00058 Keywords: hybrid orthosis, functional electrical stimulation, inverse dynamics analysis, muscle model, biomechanics, rehabilitation, gait, fatigue

#### 1. INTRODUCTION

Spinal cord injury (SCI) and other neurological disorders impair the lower limbs' motor and sensory functions. The main treatment used to stop muscle atrophy is the use of robot assisted gait training devices. Currently there are many developments of active orthotics and exoskeletons under way, and some are already being commercialized. Several authors have reported on the state of the art of active orthoses and exoskeletons, see for example Aliman et al. (2017), on their control strategies (Jimenez-Fabian and Verlinden, 2012), or on the perspectives of their use (Herr, 2009; Young and Ferris, 2017). Assistance strategies for the most relevant active exoskeletons may be found in Yan et al. (2015).

Devices that combine functional electrical stimulation (FES) with active orthoses to assist gait, known as hybrid orthoses or FES-robot devices, have emerged as a promising technology in gait rehabilitation. The use of active orthoses in combination with FES is an effective strategy to optimize the outcomes of gait rehabilitation training. Using hybrid orthosis to stimulate the lower extremity muscles has proven to evoke muscle hypertrophy, increase strength, improve cardiopulmonary fitness, and reduce fatigue during gait, even in subjects with severe spasticity (Nightingale et al., 2007; Qiu and Taylor, 2016; Deley et al., 2017; Ekelem and Goldfarb, 2018; Lambach et al., 2018). Complete recent reviews of hybrid exoskeletons can be found in Stewart et al. (2017) for the upper limbs, and Anaya et al. (2018) for the lower limbs.

The correct use of FES in a hybrid orthosis presents various challenges. Examples are the prevention of the rapid onset of muscle fatigue, the design of the co-actuation scheme with the other actuators (electric motor drives), and the selection of the appropriate stimulation waveforms and the duration of the maximum stimulation current during gait. The effector redundancy produced in a hybrid actuation (FES and electric motors) complicates the system and makes it hard to control.

There have been different proposals in literature to overcome the aforementioned problems. To address actuator redundancy problems and the rapid onset of muscle fatigue, Ha et al. (2016) presented two control loops to assist the hybrid actuation so as to optimize the issue of the onset of muscle fatigue. The first is based on tracking the desired joint trajectories, and the second uses joint torque profiles already available from previous steps to improve the motor torque efficiency, by shaping the muscle stimulation profile for the subsequent step. These two control loops provide feedback on the joint torque produced by the motor and FES so as to minimize the motor torque contribution required for a joint angle trajectory. Although the work presents a real-time control solution, the applied FES is not based on muscle dynamics but is designed to track some desired kinematics. Also, while fatigue is measured indirectly based on variations of motor current, it is considered to be generalized, i.e., there is no control of which muscle is the earliest to fatigue.

In the same line, Kirsch et al. (2016) established a switching control strategy to change between combined actuation and electromechanical actuation only in accordance with the subject's previously calibrated fatigue state. The main limitation of this work is that actuation is switched between FES and electromechanical actuators, i.e., when muscle fatigue is detected due to electrical stimulation, the controller disconnects the FES actuation and connects motor actuators to facilitate muscle recovery. When the recovery time is up, this motor actuation is disconnected, and the FES actuator is again applied to the subject.

To overcome the electromechanical delay when applying FES and the change in muscle performance over time, Del-Ama et al. (2014) proposed a controller to balance muscle and robotic actuation during walking. The main limitation is that the FES profiles are those that maintain the kinematics, and, as is also the case with the aforementioned works, the controller's efficiency must be further investigated with regard to therapeutic applications.

Regarding the design of the FES profiles, Sharma et al. (2014) proposed a dynamic optimization approach to compute the stimulation profiles. Using a biomechanical model, a customized range of stimulations can also be obtained to determine the optimal step length and walking speed. Nevertheless, the model is so complex that in practice it requires several simplifications to reduce the computational cost. Furthermore, Anderson and Pandy (2001b) had already previously demonstrated that dynamic and static optimization solutions were practically the same, and therefore that simulations can be optimized. Ferrante et al. (2016) proposed a method to design a personalized multi-channel FES controller for gait training based on muscle synergies, but electromechanical actuation is not considered. Doll et al. (2018) proposed an off-line dynamic optimization method to determine the minimum number of pulses that would maintain a constant desired isometric contraction force. The main drawback of the method is that only isometric contractions are considered, not muscle behavior during dynamic contractions. Also, only knee extensor muscles are considered. Alibeji et al. (2018) presented a controller in which dynamic postural synergies between the electric motors and FES of the muscles were artificially generated by means of optimizations. The main limitations of their study were the electromechanical delay, muscle fatigue, and actuator dynamics.

There is still no clear strategy to overcome early fatigue due to FES actuation. As mentioned above, some authors argue for switching between FES and electromechanical actuation based on the subject's specific fatigue state (Del-Ama et al., 2014; Ha et al., 2016; Kirsch et al., 2016). Others propose a leading actuation of the motors to ensure the appropriate kinematics and a preconfigured low-amplitude stimulation to improve rehabilitation therapy (Obinata et al., 2007; Farris et al., 2009; Kobetic et al., 2009). While this ensures kinematic guidance, the stimulation may not be enough to produce functional contractions of muscle tissue, and therefore the rehabilitation process may be compromised. To the best of our knowledge, no subjectspecific design of stimulation profiles and the exoskeleton's electromechanical actuation has been proposed in the literature. Došen and Milovanovic (2009) ´ proposed a form of dynamic optimization to obtain FES profiles with which to track some desired kinematics, but the exoskeleton was not included.

The aforementioned works provide a methodological approach to controlling the hybrid exoskeleton so as to follow some desired kinematics. Nevertheless, the FES profiles applied are those that track the proposed kinematics, not the kinematics expected in a physiological contraction. Consequently, the rehabilitation process may lack neurophysiological feedback. The objective of the present work was twofold: first, to develop a method to simultaneously calculating the FES profiles and the electromechanical actuation of hybrid orthosis; and second, to design orthosis actuation and FES profiles that consider fatigue.

### 2. METHODS

The proposed algorithm to estimate the combined actuation of both the electromechanical and the FES actuators is presented in **Figure 1**. Briefly, the process starts with the inverse dynamics analysis of a normative gait. Alternatively, the normative motion can be obtained from gait databases (see e.g., Liu et al., 2008; Rajagopal et al., 2016). Once the net joint torques and joint reaction forces are known, the idea is to distribute them between the orthosis and FES systems. The orthosis torque profile can be directly applied in the electromechanical actuator. The FES torque, however, must be distributed among muscles. To do so, a load sharing problem must be solved. Once the distribution of forces among the muscles spanning any of the selected joints has been obtained, the artificially activated muscle model can be inverted to finally obtain the stimulation profiles that have to be applied to the selected muscles to provide the given FES torque. The following subsections will describe the different stages of the algorithm proposed to obtain the actuation profiles for the hybrid exoskeleton. The description of the method is based on the scheme depicted in **Figure 1**, and therefore we first present the data acquisition of the normative gait, then the inverse dynamics analysis (IDA), followed by the approach to the load sharing problem in order to estimate the contribution of each actuator, and lastly the inversion of the artificially activated model.

#### 2.1. Data Acquisition: Normative Gait

As one objective is to define the actuation profiles for the hybrid exoskeleton, we must first define the desired motion. Since the intended wearer of the exoskeleton is unable to perform normal motion, gait databases can be used (Liu et al., 2008; Rajagopal et al., 2016) and adapted to the anthropometric data of the specific subject. In this work, we used the database of Liu et al. (2008), and the model weighed 72.6 kg with no history of neurological disorders. To perform the inverse dynamics analysis, the latest version of OpenSim software was used (Delp et al., 2007; Seth et al., 2018) (OpenSim, RRID:SCR002683).

#### 2.2. Biomechanical Model

The biomechanical model used has 23 degrees of freedom and 92 actuators. It consists of 12 rigid bodies, and the joints are representative of the allowed motion, i.e., revolute joints at the knee or ankle, spherical joints at the hip, etc. The motion of the model is restricted to the sagittal plane, thus reducing the actuators to 15 in each leg (see **Table 1**). The actuators correspond to the model's flexor and extensor muscles. The rigid bodies are characterized by their mass, length, moment of inertia about the center of mass, and distance from the center of mass to the proximal joint. The equations of motion can be written as:

$$\begin{cases} \mathbf{M}\ddot{\mathbf{q}} + \Phi\_{\mathbf{q}}^{T}\lambda = \mathbf{Q} \\ \Phi(\mathbf{q}, t) = \mathbf{0} \end{cases} \tag{1}$$

where **M** is the system's mass matrix, **q**¨ the acceleration vector, 8<sup>T</sup> **<sup>q</sup>** λ the generalized forces associated with the Lagrange multipliers (λ), and **Q** the generalized force vector. 8 is the constraint equations vector, and 8**<sup>q</sup>** is the Jacobian matrix of the constraint equations. The net joint reaction forces and net driver (human-orthosis actuation) moments during some physical activity or motion can be estimated using kinematic and anthropometric data in Equation (1) together with information given by the force plates.

### 2.3. Co-actuation: Estimation of FES and Electromechanical Actuation Profiles

Since several muscles serve each joint of the skeletal system, muscle forces cannot be directly computed from joint moments. This is the well-known redundant actuator problem in biomechanics. In order to solve this problem, optimization procedures are used. Various methods (static optimization, dynamic optimization, augmented static optimization, largescale static optimization) and criteria (minimum metabolic cost of transport, minimum sum of muscle stresses, minimum hyperextension of the joints, time-integral cost of activations, torquetracking) for this optimization are available in the literature (Crowninshield and Brand, 1981; Nigg and Herzog, 1999; Anderson and Pandy, 2001a,b; Menegaldo et al., 2006; Ambrosio and Kecskemethy, 2007; Pipeleers et al., 2007; Rengifo et al., 2010). See Ojeda (2012) for a review of the optimization methods, and Ou (2011) for a review of the optimization criteria. The optimization assumes that the load sharing among the muscles follows certain rules during learned motor activities, and that the muscle recruitment strategy is governed by physiological criteria aimed at achieving functional efficiency. In order to quantify the simultaneous muscle and active orthosis contributions to the net joint torques of the human-orthosis



system, this analysis considered the aforementioned 15 muscle groups per leg and three external torques applied to the ankles, knees, and hips. The external actuation proposed in this theoretical approach is the complete case in the sense that it is an active hip-knee-ankle-foot orthosis (A-HKAFO) to provide hip, knee, and ankle joint moments that assist the pathological gait of impaired subjects. Furthermore, as many of the muscles considered are biarticular (spanning two rather than just one joint), the optimization problem should consider all the lower limbs joints simultaneously (Michaud et al., 2015). For an ankle-foot orthosis (AFO) or knee-anklefoot orthosis (KAFO), the problem is solved in the same way but neglects the contribution of the electromechanical actuation at the hip and knee joints in the first case, and the hip in the second case.

#### 2.4. Joint Torque Distribution

Inverse dynamics-based static optimization methods have been known for more than three decades. The net joint torques are calculated using the inverse dynamics approach, and then the muscle load sharing problem is solved at each time step by minimizing a cost function J (**F <sup>M</sup>**) that depends on muscle forces (e.g., the sum of muscle stresses). This optimization problem is subject to two constraints: that the sum of muscle moments must equal the net joint torque obtained by inverse dynamics, and that the maximum possible muscle force is limited by their maximum isometric force, **F M <sup>0</sup>** = [f M 0,1, ... , f M 0,n ] T (Crowninshield and Brand, 1981). The results are that the muscle forces provide the acquired motion. However, for impaired subjects or for cases in which the motion is provided by the exoskeleton, the net joint torque must also be distributed between the electromechanical actuator and the natural actuators (i.e., the muscles). The formal expression of this problem is given by Alonso et al. (2012):

$$\begin{array}{ll}\min & \mathcal{J}(\mathbf{F}^{\mathbf{M}}, \mathbf{T}\_{\mathbf{o}})\\ \text{s.t.} & \mathbf{R} \cdot \mathbf{F} = \mathbf{T} \\ & \mathbf{0} \le \mathbf{F}^{\mathbf{M}} \le \mathbf{F}\_{\mathbf{0}}^{\mathbf{M}} \\ & -\mathbf{T}\_{\mathbf{o}}^{\*} \le \mathbf{T}\_{\mathbf{o}} \le \mathbf{T}\_{\mathbf{o}}^{\*} \end{array} \tag{2}$$

where J is a cost function that depends on the muscle (**F <sup>M</sup>**) and orthosis actuation (**To**) vectors. In particular, **F** = [**F <sup>M</sup>**, **To**] <sup>T</sup> = [f M 1 , ... , f M n , To,1, ... , To,m] T is the muscle and orthosis actuation vector at each instant, n the number of muscle groups, m the number of joint actuators, **R** the matrix of equivalent moment arms of the different muscle groups and orthosis actuators, **T** the vector of net joint torques obtained from IDA, and **F M <sup>0</sup>** = [f M 0,1, ... , f M 0,N ] T the vector of maximum isometric forces that limits the maximum possible muscle actuation. Moment arms are defined as the distance between the muscle's line of action and the joint's axis of rotation. The muscle lengths and moment arms can be obtained from the OpenSim IDA results. The moment arms of each muscle with respect to ankle (ra), knee (r<sup>k</sup> ), and hip (rh) are considered to be variables of the motion. The orthosis actuation moment arm is taken to be 1 for the actuated joint and 0 for the rest. The third constraint above is to ensure that the orthosis actuation does not exceed the maximum available torque.

Static optimization (SO) is computationally more efficient than dynamic optimization since it does not require multiple integrations of the equations of motion. Nevertheless, it does not consider the activation and contraction dynamics of the muscle (see **Figure 2**, top), which can lead to physiologically inconsistent results. In the present work we therefore use the socalled physiological static optimization (PSO) approach (Alonso et al., 2012) which is a modification of the classical SO approach that considers muscle physiology. This scheme maintains the computational efficiency relative to dynamic optimization approaches while considering the muscle contraction dynamics, thus ensuring the physiological consistency of the solution obtained. The approach consists of two steps. In the first, the inverse contraction dynamics problem is solved, assuming that muscle activations are maximal at every instant, i.e., **A** = [a1, ... , aN] <sup>T</sup> = [1, ... , 1]<sup>T</sup> , where **A** is the activation vector and a<sup>j</sup> are the activation values for each muscle (j = 1, ... , N = 15). In particular, the contraction dynamics ordinary differential equation

$$\dot{f}^{\mathcal{M}}(t) = g(a(t), f^{\mathcal{M}}(t), l^{\mathcal{M}}(t), \dot{l}^{\mathcal{M}}(t)) \tag{3}$$

is integrated given a = 1 for all muscles once the values of l <sup>M</sup>(t) and ˙ l <sup>M</sup>(t) are known from the generalized coordinates of the multi-body model, i.e., from the OpenSim IDA results. The resulting muscle forces, expressed as f M,∗ (t) are the maximum achievable muscle forces compatible with the measured kinematics considering full activation of muscle tissue. For the sake of simplicity, the tendon is considered to be stiff (a rigid element). The result is the trajectory of the maximally achievable muscle force for each muscle.

In the second step, these force vectors are scaled to the real activations by solving an optimization scheme. Specifically, the activations compatible with the net joint torques obtained by inverse dynamics are calculated using a static optimization approach. The design variables are the activation vector, **A**, and the orthosis actuation vector, **To**. The cost function F is the sum of a function of muscle contribution to the net joint torque, J (**A**m), and a function of the electromechanical actuation, H(**T**o):

$$\begin{array}{cl}\underset{\begin{subarray}{c}\mathbf{M}\text{-}\\\mathbf{s}\text{-}\mathbf{s}\end{subarray}}{\mathcal{I}}\mathcal{F}=\mathcal{I}(\mathbf{A})+\mathcal{H}(\mathbf{T}\_{o})\\\text{subject to}\;\mathbf{R}\cdot(\mathbf{A}\_{m}\cdot\mathbf{F}^{\*})=\mathbf{T},\\\mathcal{I}(\mathbf{A})\cdot\mathcal{H}(\mathbf{T}\_{\mathbf{o}})\leq 0\\\ 0\leq a\_{j}\leq 1,\qquad\qquad j=1,\ldots,N=15,\\\ T\_{\min}\leq T\_{o,k}\leq T\_{\max},\qquad k=1,2,3\end{array} \tag{4}$$

where **F** <sup>∗</sup> = [**F M**,∗ , **T** ∗ **o** ] <sup>T</sup> = [f M,∗ 1 , ... , f M,∗ n , T ∗ <sup>o</sup>,1, ... T ∗ o,k ] T is the maximum muscle and orthosis actuation vector at each instant, **A**<sup>m</sup> = [**A**, **To**] <sup>T</sup> = [a1, ... , an, To,1, ... To,<sup>k</sup> ] T is the design variable vector with a<sup>j</sup> being the activations for each muscle and To,<sup>k</sup> the orthosis actuations at each joint. The first (equality) constraint ensures that the contribution of the two actuation profiles equals the net available torque, **T**. The second (nonlinear inequality) constraint ensures that the muscle and orthosis torques always assist each other. The third constraint bounds the values of the activation to between 0 and 1. And the fourth is used to allow both flexion and extension for the electromechanical actuation within the bounds set by the torque limits.

The cost function F may be expressed in different forms depending on the physiological criteria selected (Rasmussen et al., 2001; Ou, 2011). It has a strong influence on the result (Michaud et al., 2015). The term corresponding to the electromechanical actuation implies that one has to use either dimensionless cost functions, or the same units for both the electromechanical and the muscular actuations (provided by FES). A general expression for the cost function in the optimization problem can be written as:

$$\mathcal{F} = \mathcal{J}(\mathbf{A}) + \mathcal{H}(\mathbf{T}\_o) = \delta \cdot \sum\_{j=1}^{N\_m} \left( h\_1 \left( a\_j(t), f\_j^M(t) \right) \right)^n$$

$$+ (1 - \delta) \cdot \sum\_{k=1}^{N\_{\tilde{f}}} \left( h\_2 \left( T\_{o,k}(t) \right) \right)^n \tag{5}$$

where h<sup>1</sup> and h<sup>2</sup> are different functions (see the list below) and N<sup>m</sup> and N<sup>j</sup> are the numbers of muscles and joints, respectively. We tested four cost functions.

• CF1: Force and torque normalized to the j − th isometric muscle force (f M 0,j ) and the k − th maximum torque (T max o,k ):

$$\mathcal{F}\_1 = \delta \cdot \sum\_{j=1}^N \left( a\_j(t) \cdot \frac{f\_j^{M,\*}(t)}{f\_{0,j}^M} \right)^n + (1 - \delta) \cdot \sum\_{k=1}^3 \left( T\_{o,k}(t) \cdot \frac{1}{T\_{o,k}^{max}} \right)^n \text{(6)}$$

• CF2: Muscle and electromechanical power:

$$\begin{aligned} \mathcal{F}\_2 &= \delta \cdot \sum\_{j=1}^N \left( -a\_j(t) \cdot f\_j^{M,\*} (t) \cdot \nu\_j^M(t) \right)^n \\ &+ (1 - \delta) \cdot \sum\_{k=1}^3 \left( T\_{o,k}(t) \cdot \dot{\theta}\_k(t) \right)^n \end{aligned} \tag{7}$$

• CF3: Force and torque normalized to the j−th maximum value of the trajectory of maximally achievable muscle force (f max j ) in the cycle and the k − th maximum torque (T max o,k ), respectively:

$$\mathcal{F}\_3 = \delta \cdot \sum\_{j=1}^N \left( a\_j(t) \cdot \frac{f\_j^M(t)}{f\_j^{\max}} \right)^n + (1 - \delta) \cdot \sum\_{k=1}^3 \left( T\_{o,k}(t) \cdot \frac{1}{T\_{o,k}^{\max}} \right)^n \text{(8)}$$

• CF4: Largest relative muscle force and torque normalized to the k − th maximum torque (T max o,k ).

$$\mathcal{F}\_{4} = \max\left\{ a\_{1}(t) \cdot \frac{f\_{1}^{M,\*}(t)}{f\_{1}^{\max}}, \dots, a\_{N}(t) \cdot \frac{f\_{N\_{m}}^{M,\*}(t)}{f\_{N\_{m}}^{\max}}, \frac{T\_{o,1}(t)}{T\_{o,1}^{\max}}, \dots \right\}$$

$$\left\{\dots,\right\}, \frac{T\_{o,N\_f}(t)}{T\_{o,N\_f}^{\max}}\right\} \tag{9}$$

where n is usually set to 2 (Michaud et al., 2015), although different values have been applied in the literature Ou (2011). According to Yamaguchi (2005), contraction velocity v <sup>M</sup>(t) can be expressed as v <sup>M</sup>(t) = −˙ l <sup>M</sup>(t) in Equation (7). Rasmussen et al. (2001) proposed a method to deal with a min/max objective function (F4). Their procedure generates activation patterns consistent with contraction dynamics only if muscle force (f M) scales linearly with muscle activation. Although this is certainly not the case for standard Hill models, this assumption has been widely used in the literature, and is the basis of OpenSim's static optimization algorithm. In this first approach, we take the weighting factors to be the same for both actuators (δ = 0.5). The weighting factors can be associated with the level of assistance in a hybrid orthosis (Anaya et al., 2018). The parameter δ basically represents some priority given to the use of either the FES actuation or the motor actuators. A higher value of δ prioritizes muscle actuation through FES, and a lower value reduces FES actuations and increases the torque provided by the electromechanical actuation. Parameters accounting for atrophy should be considered in the computational model if present in the subject under analysis. For instance, various studies (Amankwah et al., 2004; McDonald et al., 2005) have shown that passive torque tends to be greater in pathological participants than in healthy ones, especially in the ankle and hip joints.

At the end of this step, two signals are available: the electromechanical actuation joint torque which can be applied directly to the exoskeleton, and the activation signal for each muscle that scales the maximum muscle force temporal histories. In the following subsection, these activation signals will be used as inputs in the artificial activation dynamics to calculate the FES profiles to apply to the subject so as to obtain the joint torque calculated in this step.

#### 2.5. Estimation of FES Profiles

The dynamic behavior of a muscle is modeled by means of two cascaded differential equations (Zajac, 1989): the excitationto-activation dynamics which describes the transformation of a neural signal into muscle recruitment levels, and the activationto-force dynamics which represents the transformation of an activation signal into muscle force (**Figure 2**, top). For an artificially stimulated muscle (**Figure 2**, bottom), the contraction process is considered to be the same as in the physiological case since the muscle parameters considered in the contraction dynamics do not vary significantly. Nevertheless, the excitationto-activation dynamics do change, since FES artificially induces a current in specific motor neurons, not in muscle tissue (Lynch and Popovic, 2008).

#### 2.5.1. Excitation-to-Activation Dynamics in Physiologically Activated Muscles

According to Nagano and Gerritsen (2001), the excitation-toactivation dynamics of a healthy, physiologically activated muscle (**Figure 2**, top) are described by:

$$\dot{a}(t) = (u(t) - a(t)) \cdot (t\_1 u(t) + t\_2) \tag{10}$$

where a(t) is the muscle activation, u(t) the excitation signal, and t<sup>2</sup> = 1/Tfall and t<sup>1</sup> = 1/(Trise − t2) parameters depending on time constants Trise and Tfall (Nagano and Gerritsen, 2001). This equation transforms an idealized muscle excitation signal, a dimensionless value between 0 and 1, into delayed muscle activation levels, also constrained to the same range of values.

#### 2.5.2. Excitation-to-Activation Dynamics in FES-Activated Muscles

In the case of a subject with gait disability (spinal cord injury, post-polio syndrome, knee extensor failure or weakness, etc.), the natural path of the neural signal to the muscles is interrupted in some way. It has been proven that the use of FES to induce muscle contractions under these circumstances has some benefits for the patient. The activation signal produced by FES depends on the stimulus's intensity and frequency, where the former can be controlled by the amplitude or pulse width of the stimulus signal. In the literature, there are mathematical models that describe excitation-to-activation dynamics in FES-induced contractions: Makssoud et al. (2004) presented an FES muscle model based on Huxley's cross-bridge theory, which was divided into activation and contraction parts, with the former accounting for stimulation intensity, pulse width, and frequency. Watanabe et al. (1999) presented a mathematical description of the frequency-force relationship which was completed by Gföhler et al. (2004) by including the effects of amplitude. In the present work, this last model is used to obtain an estimate of FES profiles in terms of intensity or of pulse width and/or frequency. The activation dynamics for this type of induced contraction can be represented by a nonlinear static block (related to stimulus frequency and intensity) coupled with a linear dynamics block represented by a second-order differential equation (relating FES and activation signals) by means of a two-block Hammerstein structure (Durfee and McLean, 1989).

The excitation signal, e(t), output of the first block, combines the influence of stimulus intensity, Ustim (in terms of amplitude or pulse width) and frequency, fstim, and can be expressed as:

$$e(t) = \mathbb{S}\_u \cdot \mathbb{S}\_f \tag{11}$$

where S<sup>u</sup> and S<sup>f</sup> are scaling factors for stimulus intensity and frequency, respectively. The first factor corresponds to an isometric recruitment curve divided into three regions. In the first, no muscle fibers are recruited below a threshold (Utr); in the third, all fibers are recruited above the saturation level (Usat); and, in the intermediate region, there is active recruitment between those limits (Gföhler et al., 2004):

$$S\_{\rm u} = \begin{cases} 0 & \text{for} \qquad U\_{\rm stim} < U\_{\rm tr} \\ \frac{U\_{\rm stim} - U\_{\rm tr}}{U\_{\rm sat} - U\_{\rm tr}} & \text{for} \quad U\_{\rm tr} \le U\_{\rm stim} \le U\_{\rm sat} \\ 1 & \text{for} \qquad U\_{\rm stim} > U\_{\rm sat} \end{cases} \tag{12}$$

As this expression represents a process of scaling between intensity threshold and saturation levels, either a pulse-width or an amplitude signal can be used as the input Ustim. The threshold and saturation values can be measured experimentally. The former corresponds to the amplitude of the input signal that produces the first effective contraction (minimal variation in the joint angle). The latter is the value of the amplitude beyond which no more motion is observed during muscle contraction. Both values depend strongly on the subject's morphology, muscle atrophy, treatment with botulinum toxin, and sensitivity, and they must be measured independently for each muscle. The values of Ustim may range from 10 to 50 mA. Greater values combined with higher stimulation frequencies or different pad sizes may cause skin burns or neuromuscular injuries (Martín, 2004). In the present work, we have assumed equal physiological actuators since we have no access to the database's subjects to perform any measurements. This assumption also maintains the simplicity of the calculations.

The second factor in Equation (11) has been defined as (Watanabe et al., 1999):

$$S\_f = \frac{k\_1 - k\_2}{1 + e^{(f\_{\rm stim} - f\_0)/R}} + k\_2 \tag{13}$$

where k1, k2, R, and f<sup>0</sup> are appropriate constants. The values of a<sup>1</sup> and f<sup>0</sup> can be obtained by assuming S<sup>f</sup> = 0 at f = 0, and S<sup>f</sup> = 1 at the critical fusion frequency (f = fCF):

$$k\_1 = -k\_2 e^{-\int 0 \, \text{R}} \tag{14}$$

$$f\_0 = R \cdot \ln\left[ (k\_2 - 1) \cdot e^{f\_{\text{CF}}/R} - k\_2 \right] \tag{15}$$

where k<sup>2</sup> is the ratio of the maximum force to the force at fCF, i.e., k<sup>2</sup> = Fmax/FCF, and can be determined experimentally. In the present work, we set the parameter R to 15, although it can also be measured on patients (Watanabe et al., 1999).

The second block of the Hammerstein structure can be represented as a second-order ordinary differential equation (Gföhler et al., 2004):

$$k\_1 \cdot \ddot{a}(t) + k\_2 \cdot \dot{a}(t) + a(t) = e(t) \tag{16}$$

where k<sup>1</sup> = T<sup>e</sup> · Trise/fall and k<sup>2</sup> = (Trise/fall + Te), with Trise and Tfall being time constants (Nagano and Gerritsen, 2001), and T<sup>e</sup> a time constant for the excitation of artificially stimulated muscles. These constants depend on the physiological cross-section area, muscle mass, and fast-twitch muscle fiber percentage (Gföhler et al., 2004). The model therefore takes into consideration atrophy in disabled patients, which is usually associated with those values.

As the activations of each muscle are known from the previous step, the excitation signal e(t) can be obtained directly from Equation (16) using backward differences and interpolating the last values with splines to avoid the loss of values during the process. If the stimulus frequency is fixed at typical values, i.e., 20–40 Hz, it is then possible to calculate S<sup>f</sup> from Equation (13) and then S<sup>u</sup> from Equation (11) to solve Ustim in Equation (12), and thus obtain the stimulation profile in terms of variable amplitude and constant frequency. Contrariwise, if the stimulus amplitude is fixed between typical values of 20–35 mA, then it is possible to calculate S<sup>u</sup> from Equation (12), then S<sup>f</sup> from

Equation (11), and last fstim from Equation (13) to obtain the stimulation profile in terms of variable frequency and constant amplitude.

#### 2.6. Fatigue in FES-Induced Contractions

One of the major drawbacks when dealing with artificial activation of muscle tissue is the lack of selectivity in muscle fibers. A characteristic tetanic contraction that produces movement in physiologically activated muscles is defined by the sequential stimulation of adjacent fibers at a frequency of 6-8 Hz. This sequential recruitment guarantees a value of fatigue in accordance with the activity. Contrariwise, in a FES-induced contraction, in which the system is stimulated at 20–40 Hz, the individual motor units are not stimulated sequentially. Instead, all types of fibers (type I, slow; type IIa, mid; and type IIb, fast) are stimulated at the same time with the consequent early onset of fatigue, since type IIb (fast) fibers present high levels of force production but also have poor fatigue resistance (Lynch and Popovic, 2008; Vromans and Faghri, 2018).

Since a hybrid exoskeleton must facilitate motion, fatigue effects should also be considered. In order to maintain the level of actuation, clinicians often increase stimulation intensity or frequency. Unfortunately, an increase of either of these parameters accelerates the onset of fatigue (Ding et al., 2003). This may be counter-productive during FES training. When using an exoskeleton, however, a variation of those parameters may contribute to prolonging the electromechanical actuation battery life.

There are several studies in the literature that address mathematical models of muscle fatigue under FES (Ding et al., 2003; Cai et al., 2010; Marion et al., 2013). They are mainly based on the physiological mechanism. These models are complicated to use in an IDA approach because of their large number of variables. In order to evaluate the evolution of the combined FES actuation and electromechanical actuation, we hold to the idea of calculating the co-actuation with the inclusion of a fatigue term. Tepavac and Schwirtlich (1997) proposed an exponential decay to describe the reduction in muscle force production under FES in the first 5 min of electrical stimulation. Chou and Binder-Macleod (2007) measured a 50% reduction in the peak force in the first 180 s. These variations must be considered when designing appropriate actuation profiles.

From this approach, there arises a new fatigue cost function (G) which accounts for the decrease in muscle actuation due to FES-induced fatigue:

$$\mathcal{G} = \mathcal{J}(\mathbf{A}) + \mathcal{H}(\mathbf{T}\_o) = \delta \cdot \sum\_{j=1}^{N} h\_1 \left( a\_j f\_j^{\mathcal{M}} \right) \cdot \psi\_P(t)$$

$$+ (1 - \delta) \cdot \sum\_{k=1}^{3} h\_2 \left( T\_{o,k} \right) \tag{17}$$

where h<sup>1</sup> and h<sup>2</sup> are the functions described in Equations (6–9), and ψp(t) represents a fatigue function that limits the actuation, where p describes the type of the function used (see Equations 18 and 19). We shall compare two different fatigue functions. The first represents an exponential decay that models the peak force reduction observed in the aforementioned works. It can be expressed as:

$$
\psi\_{\exp}(t) = e^{-C\_1 \cdot t - C\_2} + C\_3 \tag{18}
$$

where C1, C2, and C<sup>3</sup> are appropriate constants to model a decay of some 80% in peak force with a gentle slope. For the present work, we set these values to C<sup>1</sup> = 0.02, C<sup>2</sup> = 0.3, and C<sup>3</sup> = 0.2. The actuation profiles resulting from the optimization already consider muscle fatigue, and therefore, to compensate for the variation in muscle actuation, variation in electromechanical actuation is also considered.

The second fatigue function is that proposed by Riener et al. (1996):

$$\frac{d\psi\_R(t)}{dt} = \frac{\langle \psi\_R^{min} - \psi\_R(t) \rangle \cdot a(t)}{T\_{fat}} + \frac{\langle 1 - \psi\_R(t) \rangle \cdot \langle 1 - a(t) \rangle}{T\_{rec}} \tag{19}$$

where ψ min is the minimum value achievable when a muscle is fatigued, and Tfat and Trec are time constants representing fatigue and recovery times. In this work, we took the values proposed by Riener et al. (1996) for these constants. The use of these dynamics requires a slight modification in the optimization routine. At each step, the required activations for cooperative actuation are calculated by means of the proposed algorithm. These activations are then used to compute the fatigue function, which basically reduces muscle force capacity by scaling the activation. A second optimization is then computed to calculate simultaneously the muscle activations under fatigue conditions (and therefore FES profiles) and the compensated orthosis actuation profiles.

### 3. RESULTS AND DISCUSSION

The procedure described was implemented in Matlab (The MathWorks Inc.) running on an Intel(R) Core(TM) i5 CPU 650 at 3.20 GHz. We used the Matlab fmincon routine for CF1 to CF3, and fminimax for CF4. Simulation times (as obtained for a single cycle) were 28.35 s for CF1, 8.83 s for CF2, 18.87 s for CF3, and 35.69 s for CF4. The different cost functions evaluated yielded different actuation profiles. **Figure 3** shows the output of the optimization algorithm in terms of joint torque distribution. The "softest" results were obtained with cost functions CF1 and CF3. Normalizing the muscle force to the isometric muscle force f M 0,j (CF1) instead of to the maximum force in each cycle f max j (CF3) led to lower calculated joint torques related to muscle actuation. Although the results for the hip seem to be similar, there is a greater muscle contribution in CF3 than in CF1, concordant with the activations shown in **Figure 4**. One observes that the contrary is the case for these cost functions in terms of orthosis actuation. In general, the use of muscle force-based cost functions (CF1, CF3, and CF4) results in similar contributions for the hip, but not for the other joints. The use of CF2 results in muscle actuators having greater relevance than electromechanical actuators. The differences in joint torque contribution for these two actuators may be explained by the individual contributions of the muscles to the joint torque.

solid black line. CF4: Dotted green line.

FIGURE 5 | Comparison of the different FES profiles for the proposed optimization cost functions. CF1: Thick solid orange line. CF2: Dashed blue line. CF3: Thin solid black line. CF4: Dotted green line.

FIGURE 6 | Evolution of the FES profiles for the proposed muscles using exponential decay fatigue factor for function F1. Dashed orange line: Initial FES profile. Dotted yellow line: FES profile at 60 s. Solid green line: FES profile at 120 s. Dash-dotted blue line: FES profile at 180 s.

FIGURE 7 | Evolution of the FES profiles for the proposed muscles using the fatigue factor proposed by Riener et al. (1996) for function F1. Dashed orange line: Initial FES profile. Dotted yellow line: FES profile at 60 s. Solid green line: FES profile at 120 s. Dash-dotted blue line: FES profile at 180 s.

Dash-dotted blue line: joint torque at 180 s.

The muscle activations for the different cost functions are depicted in **Figure 4**. As noted above, the results for CF1 and CF3 are similar but the muscle activation values for the latter are greater, reflecting the greater contribution of muscles to torque production. There are marked spikes in the CF2 activations, with, at some points, tetanic contractions that may be inappropriate for rehabilitation purposes or for smooth control of the degree of freedom with the two actuators.

Once the artificial activation dynamics have been inverted, the FES profiles can be evaluated. For simplicity, we here assumed that all the muscles have the same threshold and saturation values. The results are shown in **Figure 5**. The process that leads from **Figures 4**, **5** (see section 2.5.2) is a temporal shift of the activation signal, followed by nonlinear scaling, and then normalization between the threshold and saturation levels. The main features of the activation profiles are preserved, i.e., the FES profiles obtained with CF2 are higher and at some points spiked, which, in terms of stimulation, may cause muscle tissue contractions that are hard to control. Moreover, a sustained FES-induced tetanic contraction, as in tibialis posterior or lateral gastrocnemius, may also result in early muscle fatigue.

If fatigue is included in the optimization process by using Equation (17), the FES and electromechanical actuation time profiles are expected to vary during time. Both fatigue functions proposed in Equations (18) and (19) decrease exponentially being the first one softer than the second one. Another difference is that second one allows the muscle to recover partially. As the activations in the cost function are limited by this fatigue factor, which is decreasing over time, the results of the optimization process lead to increasing values of the activation profiles, and therefore of the FES profiles (see **Figures 6**, **7**). As fatigue is compensated with increased values of muscle stimulation, the results in terms of joint torque are the same as in **Figure 3** for CF1, i.e., the torque profiles remain constant and distributed as in **Figure 3** for CF1 throughout the 180 s. These results are consistent with those of Del-Ama et al. (2014), in which the stimulation parameters are increased when fatigue appears to maintain a constant joint torque.

In terms of joint torque, if FES profiles of **Figures 8**, **9** are used, the contribution of FES and electromechanical actuation must be updated to reflect the effect of the fatigue factor. As the designed FES profiles increase over time, the contribution of the artificial contractions to joint torque decreases while the contribution of the electromechanical actuator must increase to compensate the effects of fatigue. These results are reflected in **Figures 6**, **7**. Both show a decrease in FES actuation while the motor actuation increases. The exponential decay limits the actuation and, from the beginning, leads to a major but steady decline in amplitude that is sustained over the 180 s of the tested cycle. On the contrary, the dynamics proposed by Riener et al. (1996) presents a sharp decline in the first 50 s that must be compensated by the motor actuation. This strong decline in muscle force production may be due to the values of the time constants which were determined from patients with complete thoracic spinal cord injury. This factor reflects the characteristics of the two fatigue dynamics: in the first case, the exponential decay function is adapted in accordance with the observed reduction in muscle force production over 180 s, whereas, in the second case, although fatigue and recovery dynamics are considered, the muscle force reduction is constantly updated in accordance with the current state of the muscle, so that fatigue appears earlier in muscles with higher levels of stimulation, as in medial gastrocnemius, and the same level of stimulation can be applied and maintained in muscles with a lower stimulation profile, as in adductor magnus or tibialis posterior.

The present results are not directly comparable with those of previous work in which the proposed controller either switches between actuators and there is no information about the combined actuation, or there is only information for one actuator. Nevertheless, the results shown in **Figure 3** are similar to those provided by Ha et al. (2016) for CF2 and CF3 at the knee level in which there was a reduction in the orthosis contribution to torque when FES was applied. Furthermore, the results obtained in this work explain, with a physiological model in the background, the use of bang-bang controllers to switch between orthosis and FES actuation when muscle fatigues or more complex controllers to switch between both actuators as in Ha et al. (2016) or Kirsch et al. (2016).

The proposed method is off-line. Nevertheless, it could be applied to improve current control algorithms using the designed

FES profiles which are physiologically consistent with the motion, instead of pre-defined ones that ensure kinematic guidance but may not have a rehabilitative function or result in delivering excessive electrical stimulation to the muscles causing either early fatigue or an exaggerated gait pattern (Anaya et al., 2018). For instance, Ha et al. (2016), Del-Ama et al. (2014), or Kirsch et al. (2016) do not design specific FES profiles for each muscle. In some cases, the FES profiles are already predefined, or the control algorithm switches between actuators to prevent fatigue. The method proposed here could be used to improve existing control algorithms, as in Ha et al. (2016), by including stimulation profiles that are physiologically consistent with the motion. Furthermore, according to Pizzolato et al. (2017), it might be possible to perform the inverse kinematics in real-time. By optimizing the programmed routines, cooperative control could be reached that is near real-time, or at most one step back, as in Ha et al. (2016), which may be enough for a cooperative controller that uses rehabilitative stimulation profiles. A possible solution for the control architecture of the cooperative controller is shown in **Figure 10**. A reference orthosis torque can be averaged (To,ref) from the reference kinematics and total torque, measured in the absence of muscle stimulation. Then the proposed method would be used to predict a distribution of the required net joint moments between the motors and the artificially activated (i.e., electrically stimulated) muscles to comply with the desired kinematics. The difference between the reference and the predicted orthosis torques provides an estimate of the FES torque (T ∗ FES). Then the difference between this value and the one predicted in the proposed scheme is used by a high-level controller to adapt the weightings (δ) in the optimization to reduce muscle fatigue. This scheme is similar to that proposed by Ha et al. (2016) but introduces muscle fatigue into the cooperative control of the hybrid orthosis by including the artificially activated muscle dynamics.

## 4. CONCLUSIONS

This work has described a method for the simultaneous design of the actuation provided by the electrical stimulator and the electromechanical actuators during gait assisted hybrid exoskeletons. The scheme ensures the physiological consistency of the results and is computationally efficient. There has been previous work (Ferrante et al., 2016; Ha et al., 2016; Alibeji et al., 2018) proposing methods for the control of such exoskeletons, but nothing regarding the design of the two actuation profiles at the same time. The present approach provided promising results for the definition of rehabilitation routines for hybrid exoskeletons or their control strategies. Furthermore, since fatigue was included in the model, estimates can be made of the rest intervals needed to improve muscle tissue recovery times. Despite the promising nature of the results, the following topics must be addressed to work toward a generalized solution:


## DATA AVAILABILITY

The datasets analyzed for this study are available on request from the corresponding author.

## AUTHOR CONTRIBUTIONS

All authors contributed to writing the manuscript and approved its final version for submission.

## FUNDING

The support given by the Consejería de Educación y Empleo, Junta de Extremadura and the European Regional Development Fund Una manera de hacer Europa, project GR18059, was gratefully acknowledged.

## REFERENCES


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Romero-Sánchez, Bermejo-García, Barrios-Muriel and Alonso. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# A Quick Turn of Foot: Rigid Foot-Ground Contact Models for Human Motion Prediction

#### Matthew Millard\* and Katja Mombaur

*Optimization in Robotics and Biomechanics, Institute of Computer Engineering, Heidelberg University, Heidelberg, Germany*

Computer simulation can be used to predict human walking motions as a tool of basic science, device design, and for surgical planning. One the challenges of predicting human walking is accurately synthesizing both the movements and ground forces of the stance foot. Though the foot is commonly modeled as a viscoelastic element, rigid foot-ground contact models offer some advantages: fitting is reduced to a geometric problem, and the numerical stiffness of the equations of motion is similar in both swing and stance. In this work, we evaluate two rigid-foot ground contact models: the ellipse-foot (a single-segment foot), and the double-circle foot (a two-segment foot). To evaluate the foot models we use three different comparisons to experimental data: first we compare how accurately the kinematics of the ankle frame fit those of the model when it is forced to track the measured center-of-pressure (CoP) kinematics; second, we compare how each foot affects how accuracy of a sagittal plane gait model that tracks a subjects walking motion; and third, we assess how each model affects a walking motion prediction. For the prediction problem we consider a unique cost function that includes terms related to both muscular effort and foot-ground impacts. Although the ellipse-foot is superior to the double-circle foot in terms of fit and the accuracy of the tracking OCP solution, the predictive simulation reveals that the ellipse-foot is capable of producing large force transients due to its geometry: when the ankle quickly traverses its u-shaped trajectory, the body is accelerated the body upwards, and large ground forces result. In contrast, the two-segment double-circle foot produces ground forces that are of a similar magnitude to the experimental subject because the additional forefoot segment plastically contacts the ground, arresting its motion, similar to a human foot.

#### Edited by:

*Benjamin J. Fregly, Rice University, United States*

#### Reviewed by:

*Shinya Aoi, Kyoto University, Japan Seungbum Koo, Korea Advanced Institute of Science and Technology (KAIST), South Korea*

\*Correspondence: *Matthew Millard matthew.millard@ziti.uni-heidelberg.de*

> Received: *31 March 2019* Accepted: *17 July 2019* Published: *07 August 2019*

#### Citation:

*Millard M and Mombaur K (2019) A Quick Turn of Foot: Rigid Foot-Ground Contact Models for Human Motion Prediction. Front. Neurorobot. 13:62. doi: 10.3389/fnbot.2019.00062* Keywords: foot contact, musculoskeletal model, motion prediction, optimal control, multibody dynamics

## 1. INTRODUCTION

Understanding the relationships between force and movement in the musculoskeletal system is key to correcting movement pathology. Though it is possible to study muscle movement in-vivo (Fukunaga et al., 2001; Ishikawa et al., 2003; Maganaris, 2003; Reeves and Narici, 2003) measuring muscle force in-vivo is not possible without invasive surgery. Currently the only way to study the motion and forces of the human musculoskeletal system is to use mathematical models and computer methods to predict quantities that cannot easily be measured.

The mathematics of optimal control can be used to predict the movements of a model (Ackermann and van den Bogert, 2010; Schultz and Mombaur, 2010; Mordatch et al., 2013). Casting human motion prediction as an optimal control problem (OCP) requires four components: a musculoskeletal model, a cost function, problem-specific constraints, and a method to solve for the vector of state and muscle force waveforms that simultaneously satisfy the equations of motion and minimize the cost function. While the underlying mathematics of multibody dynamics and optimal control is well-developed, many tissues and structures of the body are challenging to model. Though the human body contains many mechanically complex structures, it has proven particularly difficult to formulate models of foot ground contact that are both accurate and well-suited for the prediction of walking.

Inaccuracies in the model of foot ground contact affect the rest of the body because the foot forms the only boundary between the body and the ground during typical walking. The shape that a foot model makes during walking determines how ground forces are transformed into ankle torques and vice-versa. Though impressive movement predictions have been realized without an accurate foot shape (Van den Bogert et al., 2012; Mordatch et al., 2013; Koelewijn et al., 2018), differences in foot shape ultimately affect the ankle kinematics, and CoP progression. Accurately fitting the loaded shape of a foot model to experimental data is challenging because the optical markers placed on the skin of the foot move on the order of a centimeter with respect to the underlying bones (Fuller et al., 1997). Both the fitting and the simulation of viscoelastic foot models is made difficult by the widely varying stiffness of human foot pads (Aerts et al., 1995) which are compliant at initial contact (∼20 N/mm) and rapidly stiffen with load (1,445 N/mm at 1 body weight). Although the literature contains some excellent examples of fitted viscoelastic foot models (Halloran et al., 2010; Pàmies-Vilà et al., 2014; Shourijeh and McPhee, 2014, 2015; Millard and Kecskeméthy, 2015; Jackson et al., 2016; Brown and McPhee, 2018), rigid footground contact models are an attractive alternative: the fitting process is strictly dependent on geometry, and the numerical stiffness of the model does not change appreciably from swing to stance. A reduction in the numerical stiffness of the model is attractive because this makes the resulting optimization problem less sensitive and therefore easier to solve.

Although rigid-foot ground contact models are common in the passive dynamic walking literature (McGeer, 1990; Collins and Ruina, 2005) few rigid foot-ground models exist in the musculoskeletal modeling literature. Hansen et al. (2004) and Srinivasan et al. (2008) modeled the lower leg and foot in twodimensions (2D) as a single rigid body that rolls on the ground using a rigid cylinder-plane contact pair. While this approach can accurately replicate the motion of the entire lower stance leg, for many applications it is not acceptable to fix the ankle joint. Although the foot has been modeled using point contacts for sprinting motions (Kleesattel and Mombaur, 2018), point contacts do not capture the rolling motion of the foot during walking (García-Vallejo et al., 2016). The foot has been modeled as a single convex cam (Ren et al., 2010; Römer, 2018; Römer et al., 2018) which contacts the ground at a single point and rolls-without-slipping across the ground plane. Ren et al. (2010)'s planar foot model closely matched the ankle position of the 12 subjects they tested (≈1 cm on average), with the largest errors appearing during heel strike (≈2.5 cm) and toe-off (≈1.5 cm). It is worth noting that a certain amount of kinematic error is expected during heel-contact and toe-off since a rigid foot ground contact model does not capture the compression of heel (Gefen et al., 2001) and forefoot pads (Cavanagh, 1999). Felis and Mombaur (2016) developed a 3D rigid foot-ground contact model using a sphere and a planar triangle to represent the heel and forefoot, respectively, but did not fit the model to experimental data. Though there are good examples of rigid foot ground contact models that interact through the ground using a single curved shape (Ren et al., 2010; Römer, 2018; Römer et al., 2018), there are no examples of fitted rigid contact models that treat the hind and forefoot separately.

Unfortunately a foot model that fits kinematic data isolation does not necessarily translate into a walking prediction that produces human-like foot ground forces. Optimal walking solutions in the literature typically have ground force profiles that deviate from experimental data, sometimes dramatically, at heel contact (Ackermann and van den Bogert, 2010; Geyer and Herr, 2010; Dorn et al., 2015) where differences between two and three times body weight are typical. Large simulated heel contact forces often arise from pairing a musculoskeletal model with a viscoelastic foot and using a problem formulation that inherently does not adjust its walking pattern in response to large contact forces. Another common problem, in which an obvious solution is not clear, are ground forces that have an appropriate magnitude but a shape that differs from experimental data (Anderson and Pandy, 2001; Ren et al., 2007; Geyer and Herr, 2010; Sreenivasa et al., 2017). To make an improvement in prediction accuracy it is necessary to distinguish error that is caused by the model of the foot from error caused by other sources.

In this work we model and evaluate two planar foot-ground contact models using three different methods in an effort to identify differences with experimental data that are caused by the foot model. The first of the rigid foot-ground contact models we consider is similar to existing works in the literature because it interacts with the ground through a single curved segment (Ren et al., 2010; Römer, 2018; Römer et al., 2018). In contrast, the second foot-ground contact model has separate contact shapes for the heel and forefoot. As is typical, we evaluate how well the kinematics of each foot model track the stance kinematics of a subject's foot in isolation. In addition, we consider how well each foot model performs as part of a whole body gait model: first, when the gait model is used as part of an optimal-control problem (OCP) to track experimental data; and second, in an OCP that predicts motion. The musculoskeletal model and foot models are described in section 2 while the detailed evaluation procedure used to assess the foot models is described in section 3. The results of the work appear in section 4 and a discussion of the results in section 5.

## 2. MODEL

We model the human body as a planar floating base rigid-body system (**Figure 1A**) which interacts with the ground through one of two different rigid foot ground contact models (**Figure 1B**).

Foot ground interaction is modeled using contact and rolling constraints between an ellipse and a plane, and also between a pair of circles and a plane. The human body models used to test each foot are identical and have seven segments, nine degrees-offreedom (DoF), and are driven by six pairs of agonist-antagonist muscle-torque-generators (MTGs).

The differential-algebraic equations (DAEs) governing this system are described as

$$M(q)\ddot{q} + c(q, \dot{q}) = \text{ т } + G(q, \dot{q})^T \lambda \tag{1}$$

$$g\_V(q, \dot{q}) = 0 \tag{2}$$

where q, q˙, and q¨ are the generalized positions, velocities, and accelerations of the model; M(q) is the mass matrix, and c(q, q˙) is the vector of Coriolis, centripetal, and gravitational forces. The kinematic constraints between the foot and the ground are in the vector gV(q, q˙), while the generalized forces these constraints apply to the system are contained in the term G(q, q˙) <sup>T</sup>λ where G(q, q˙) is the Jacobian of the constraint equations gV(q, q˙) with respect to q˙ and λ is a vector of Lagrange multipliers. The foot ground constraints, gV(q, q˙), are described at the velocity level, index-reduced, and applied at the acceleration level.

Throughout this work we indicate position vectors using r, direction vectors with e, frames with K (which are composed of a position vector to the origin and a rotation matrix), points using letters, linear velocity with v, and angular velocities with ω. Forces are denoted using f while functions are indicated with f and appear with an argument. Subscripts are used with direction vectors and frames to provide additional information, while a more elaborate system is used with kinematic vectors: the origin of the vector appears in the left subscript, the termination in the right subscript, and the frame the vector is resolved into (if necessary) is indicated in the left superscript. Thus <sup>A</sup>r<sup>H</sup> means the position vector that begins at point A and terminates at point H but is not yet resolved into any particular frame since the left superscript is blank.

The constraints between the ellipse and the plane are applied at the point-of-closest approach (nS) and are described at the velocity-level using a contact constraint

$$(\nu\_{\rm A} + \omega\_{\rm A} \times (\_{\rm A}r\_{\rm E} + r\_{\rm S}(\phi)e\_{\phi}))^{\rm T} e\_{\rm Z} = 0 \tag{3}$$

and a rolling constraint

$$(\nu\_{\rm A} + \omega\_{\rm A} \times (\_{\rm A}r\_{\rm E} + r\_{\rm S}(\phi)\varrho\_{\phi}))^{\rm T} \varrho\_{\rm X} = 0 \tag{4}$$

where v<sup>A</sup> is the linear velocity of the origin of the ankle frame KA, ω<sup>A</sup> is angular velocity of KA, <sup>A</sup>r<sup>E</sup> is the vector from K<sup>A</sup> to the center of the ellipse, e<sup>φ</sup> the direction vector that points to nS, and rS(φ) is the radius of the ellipse at the polar angle φ. Since there is no closed form equation for the point of closest approach between an ellipse and a plane, we numerically solve for φ using first the bisection method, and finally Newton's method to polish the root to high accuracy. The parameters of the ellipse-foot (pE) are defined by the offset of the ellipse from the ankle (ArE), its relative orientation to the angle frame (AθE), and the lengths of the major and minor axes of the ellipse (r<sup>X</sup> and rY).

The double circle foot contact model uses one of three different sets of constraint equations depending on which circle is in contact with the ground. During heel contact the constraint equations are

$$(\nu\_{\rm A} + \omega\_{\rm A} \times (\_{\rm A}r\_{\rm H} - r\_{\rm H}e\_{\rm Z}))^{\rm T} e\_{\rm Z} = 0 \tag{5}$$

$$(\nu\_{\rm A} + \omega\_{\rm A} \times (\_{\rm A}r\_{\rm H} - r\_{\rm H}e\_{\rm Z}))^{\rm T} e\_{\rm X} = 0 \tag{6}$$

where r<sup>H</sup> is the radius of the heel circle. During forefoot contact the constraint equations are given by

$$(\nu\_{\rm A} + \omega\_{\rm A} \times (\_{\rm A}r\_{\rm F} - r\_{\rm F}e\_{\rm Z}))^{\rm T} e\_{\rm Z} = 0 \tag{7}$$

$$(\nu\_{\rm A} + \omega\_{\rm A} \times (\_{\rm A}r\_{\rm F} - r\_{\rm F}e\_{\rm Z}))^{\rm T} e\_{\rm X} = 0 \tag{8}$$

where r<sup>F</sup> is the radius of the forefoot contact. These equations are nearly identical to the constraints used for the ellipse, but without the extra computational expense incurred in computing the point of closest approach. To ensure that the foot is not over-constrained when both circles touch the ground, we apply

pair of circles and a plane (B).

the contact and rolling constraints of one circle, while the other is constrained with just a contact constraint. The parameters of the double-circle foot (pC) are defined by the offset of each circle from the ankle (Ar<sup>H</sup> and <sup>A</sup>rF), and the radius of each circle (r<sup>H</sup> and rF).

The model is actuated by six pairs of agonist-antagonist MTGs each of which model groups of extensors and flexors that cross the hip, knee, and ankle. The torque τ <sup>M</sup> developed by a single MTG resembles that of a rigid-tendon Hill-type muscle model (Zajac, 1988; Millard et al., 2013) and is given by

$$\boldsymbol{\tau}^{\mathcal{M}} = \pm \boldsymbol{\tau}\_{\boldsymbol{o}}^{\mathcal{M}}(\boldsymbol{a} \, \mathrm{f}^{\mathcal{A}}(\boldsymbol{\theta}) \mathrm{f}^{\mathcal{V}}(\boldsymbol{\alpha})) \tag{9}$$

where τ M o is the maximum active isometric torque of the MTG, a represents the chemical activation of the MTG, fA(θ) is the active-torque-angle characteristic, fV(ω) is the torque-angularvelocity curve, and the sign is set to be consistent with the anatomy of the muscle group and the generalized coordinates used to describe the model. The parameters (τ M o ) and curves [fA(θ) and fV(ω)] that define the flexors and extensors of the hip, knee, and ankle are fitted to the data of Anderson et al. (2007) and Jackson (2010). Please see Millard et al. (2017) for a more detailed description of the formulation and parameters of the MTGs. Since walking does not typically stretch the leg muscles appreciably (Arnold and Delp, 2011), we ignore the passive forces developed by the parallel element.

Although it is conventional to describe activation dynamics using an ordinary differential equation with a discontinuity, this formulation is not compatible with gradient-based optimal control methods which require C<sup>2</sup> continuity. Here we describe activation dynamics using a C<sup>2</sup> approximation

$$\dot{a} = \frac{e - a}{\frac{1}{2}(\tau\_\mathrm{A} + \tau\_\mathrm{D})} \tag{10}$$

where e is the excitation signal, a is the activation of the muscle. The activation τ<sup>A</sup> and deactivation τ<sup>D</sup> time constants are 15 and 50ms, respectively (Thelen, 2003).

At each leg joint the net torque is given by

$$
\tau\_i = \tau\_i^{\text{MF}} + \tau\_i^{\text{ME}} - \beta \alpha\_i. \tag{11}
$$

where F and E designate the joint's flexors and extensors, and β is light passive damping introduced by the musculature and tissue surrounding the joint. The damping coefficient is defined as

$$\beta = \eta \frac{\tau\_{\rm o}^{\rm MF} + \tau\_{\rm o}^{\rm ME}}{\omega\_{\rm max}^{\rm MF} + \omega\_{\rm max}^{\rm ME}} \tag{12}$$

so that the amount of damping is proportional to the strength the musculature, the scaling factor η, and inversely proportional to the maximum angular velocity of the musculature. We use a value of 2.0 for η which results in damping coefficients which range between 2.7 and 7.3 Nms/rad.

We use the open-source dynamics library Rigid Body Dynamics Library<sup>1</sup> (RBDL), an implementation of Featherstone's order-n dynamics methods (Featherstone, 2014), developed by Felis (2016), to solve the forward dynamics of our model. To simulate the MTGs, we use RBDL's muscle model library developed by Millard et al. (2017).

#### 3. EVALUATION PROCEDURE

As is typical, we first evaluate each foot model in isolation by considering how accurately each tracks the kinematics of the ankle and CoP of a subject's foot. Next, we pair each candidate foot model with a musculoskeletal model and solve a tracking OCP to determine if the experimental subject's gait is in the solution space of each foot model. Finally, we solve a prediction OCP to examine how well each foot performs when it is not guided by experimental data and is free to move.

#### 3.1. Experimental Data

The experimental data used in this study comes from a walking trial recorded in an experiment described in Millard et al. (2017). Briefly, the motions and ground forces of a 35 year old male subject (mass 81.7 kg and height of 1.72 m) wearing light hiking shoes were recorded during level walking. OptoTrack IRED markers clusters were used to track the 3 dimensional (3D) movements of 14 body segments (head, upper-torso, mid-back, pelvis, thighs, shanks, feet, upperarms, and lower arms) while Kistler force plates (Kistler GmbH, Germany) were used to measure ground forces. The recordings were conducted at Vrije Universiteit Amsterdam according to the guidelines of the Declaration of Helsinki 2013, approved by the ethics committee in Faculteit der Gedrags- en Bewegingswetenschappen (Faculty of Behavioral and Movement Sciences), and with written and informed consent from the subject. Mass and inertia properties were computed using Zatsiorsky's regression equations (Zatsiorsky, 2002) while the geometry of the human model is extracted using digitized bony landmarks from the experimental subject.

#### 3.2. Foot Model Fitting

As is typical in the literature, first we will fit the foot model in isolation before proceeding to use it with the whole body model. Since the force plates very accurately record the CoP trajectory, we have elected to fit the foot model by precisely matching the recorded CoP kinematics (experimental quantities are denoted with EXP) and then measuring the error between the position and orientation of the K<sup>A</sup> and the KEXP A frame (**Figure 2**). Prior to fitting the foot model, the data used for fitting was segmented to only include samples in which the normal contact force was >5% of the peak recorded ground force (928N). To fit the model foot, it was initially posed at the same orientation as the subject's foot had at toe-off (at time sample n) and offset so that the contact point of the model rP,n coincided with the recorded CoP r EXP P,n . Next, the model foot was rolled without slipping backwards until the foot contact point of the model matched the recorded CoP, a process which continued until the time of heel strike was reached (at time sample 1). We elected to pose the model in the toe-off position and roll it backwards because this made it easier for us to

<sup>1</sup>https://github.com/ORB-HD/rbdl-orb

so that it's contact point matches the subject's recorded CoP. This process is continued until heel strike. The geometric error of the foot model is the weighted sum of position and orientation errors between the model's ankle frame and the subject's ankle frame.

manually find a good set of initial parameters prior to beginning the optimization.

The geometric parameters of each foot model (p<sup>E</sup> and pC) are fitted by minimizing the cost function

$$\min \sum\_{i=n}^{1} \frac{(f\_{\rm P,i}^{\rm EXP})^{\rm T} \sigma\_{\rm Z}}{\max((f\_{\rm P}^{\rm EXP})^{\rm T} \sigma\_{\rm Z})} (\boldsymbol{\omega}\_{\rm r} (\boldsymbol{r}\_{\rm A,i} - \boldsymbol{r}\_{\rm A,i}^{\rm EXP})^2 + \boldsymbol{\omega}\_{a} (\boldsymbol{\alpha}\_{\rm A,i} - \boldsymbol{\alpha}\_{\rm A,i}^{\rm EXP})^2), \tag{13}$$

where the vectors r<sup>A</sup> and α<sup>A</sup> are the origin and orientation of the ankle frame, and f EXP P,i is ith experimentally recorded ground reaction force vector. The cost of each sample i, is weighted by the normalized magnitude of the contact force so that the final fit is closest when the ground forces are highest. The weights w<sup>r</sup> and w<sup>α</sup> are set to (1/0.01)<sup>2</sup> and (1/( 1 4 π))<sup>2</sup> so that distances on the order of 1 cm and <sup>1</sup> 4 π, which we consider to be large errors in this context, result in an error value of 1. We did not allow all parameters to vary but fixed the total length of the foot model to be 30.5 cm, which is 0.5 cm longer than the light hiking shoes worn by the subject. This extra constraint is added to prevent the optimization routine from converging on an unrealistically short foot. Though a shorter foot may fit the kinematics of this test best, it causes problems when the subject's CoP is followed in the tracking OCP problem (described in section 3.4).

The resulting prediction problem for the ellipse-foot has only four optimization variables (the center and orientation of the ellipse and the radius of the minor axis) while the double-circle foot has five parameters (the radius of both circles, the center location of the forefoot circle, and the height of the center of the hind-foot circle). In each case the least squares problem is initialized using manually and then solved using a Nelder-Mead simplex method (Nelder and Mead, 1965; Lagarias et al., 1998) to a tolerance of 10−<sup>6</sup> (MATLABR2018a). Each model is evaluated based on the how closely the ankle frame of the model tracks the subject's ankle movements.

#### 3.3. Walking as an Optimal Control Problem

In this work we use two different types of OCPs: a tracking problem which tries to follow experimental data, and a prediction problem that tries to minimize a cost function. Both of these OCPs define walking using the same mathematical framework, and differ only in the cost function used and a few constraints. Here we describe how walking is defined as optimal control problem in general before proceeding to describe the specific differences between the tracking and prediction OCPs.

An OCP has the objective of identifying the vector of state x(·) trajectories, control u(·) trajectories, and constant parameters p, that minimize the sum of the Lagrange φ L and Mayer 8<sup>M</sup> terms in the cost function across n<sup>p</sup> distinct phases

$$\min\_{\mathbf{x}(\cdot),\boldsymbol{\mu}(\cdot),\boldsymbol{\nu}} \sum\_{j=0}^{n\_{\boldsymbol{\rho}}-1} \left( \int\_{\boldsymbol{\nu}\_{j}}^{\boldsymbol{\nu}\_{j+1}} \phi\_{j}^{\rm L}(\mathbf{x}(t),\boldsymbol{\mu}(t),\boldsymbol{\rho}) dt \right) + \sum\_{j=0}^{n\_{\boldsymbol{\rho}}-1} \Phi\_{j}^{\rm M}(\mathbf{x}(t),\boldsymbol{\mu}(t),\boldsymbol{\rho}) \tag{14}$$

where j iterates sequentially across the phases that begin at time ν<sup>j</sup> and terminate at time νj+1. In addition to minimizing the cost function, the state trajectories must satisfy the state derivatives and impact state transitions

$$
\dot{\mathbf{x}}(t) = f\_{\mathbf{j}}(t, \mathbf{x}(t), \boldsymbol{\mu}(t), \boldsymbol{\rho}), \tag{15}
$$

$$\varkappa(t\_j^+) = c\_j(\varkappa(t\_j^-)), \quad \text{for } t \in [\nu\_{j-1}, \nu\_j], \tag{16}$$

$$j = 1, \dots, n\_{\rho}, \ v\_0 = 0, \ v\_{n\_{\rho}} = T.$$

which take the form of the DAEs in Equations (1, 2), and activation dynamics of the MTGs in Equation (10). In this work, state transitions from foot impacts cause discrete changes in the generalized velocities of the model but affect no other states.

The state vector, x = (q, q˙, a), of the musculoskeletal model has 30 states, 18 of which correspond to generalized positions and velocities while the additional 12 come from the vector of muscle activations. The vector of control signals, u(·), is composed of the twelve excitation signals that affect the activation dynamics of the MTGs as described in Equation (10). The vector of generalized forces has 6 non-zero elements, τ = (0, 0, 0, τ4, τ5, τ6, τ7, τ8, τ9), corresponding to torques that the MTGs and passive damping apply to each joint. The leading three entries in τ are zero because there are no generalized forces acting between the inertial frame and the pelvis. The number of kinematic constraints applied to the model ranges between 2 and 6 depending on the foot-model being simulated and the constraint set that is active.

We formulate walking as a multi-phase OCP that has four phases for the ellipse-foot (**Figure 3A**), and seven phases with the double-circle foot (**Figure 3B**). To distinguish between the

(with its separate hind and forefoot contacts) results in a 7 phase definition of walking (B). For brevity we refer to the ellipse model with an "e", the double circle model with a "c", double stance with (DS), and single-stance with (SS). Black chevrons indicate an impact occurs. An "\*" indicates that the phase is instantaneous.

various phases and foot model we introduce a number of short forms: ellipse-foot (e), double-circle foot (c), double-stance (DS), single-stance (SS), and instantaneous phases are marked with an "\*." Walking using the ellipse-foot consists of two continuous phases and two instantaneous phases:


The double-circle foot requires four continuous phases and three instantaneous phases to describe walking:


In both cases the foot-ground impulse are stored in the vector 1. Note that these specific phase descriptions match the experimental subject's gait, but there are many other possible phase descriptions for the double-circle foot.

We use continuous constraints

$$0 \le \gspace{0.0ex} \lnot (t, \varkappa(t), \mu(t), p) \quad \text{for } t \in [\nu\_{\hat{\jmath}-1}, \nu\_{\hat{\jmath}}] \tag{17}$$

on state and control bounds, as well as using phase specific constraints. The bounds on q are set to be at least ±1 m and (for the linear coordinates) and ±1 radian (for the angular coordinates) larger than the experimental measurements (except in knee extension where 0.1 radian of hyper-extension is permitted). Similarly the bounds on q˙ are set to be at least ±1 m/s and ±1 rad./s larger than the experimental measurements. The vectors a(·) and u(·) are constrained to be between zero and one. Phases which begin with an impact include equality constraints at the position level so that the respective foot-ground constraints begin on the constraint manifold. During swing phases an inequality constraint is used to ensure that the swing foot does not touch the ground. All ground forces and impulses are constrained act unilaterally and have tangential components that are limited by the coefficient of friction which we assume to be 0.8. To ensure that the final solution represents periodic and symmetric walking we apply periodicity constraints so that the joint angles, activations, and ground forces of the right leg (left leg) in the initial phase match the corresponding values of the left leg (right leg) in the final phase. At the velocity level we apply periodicity constraints to the linear and angular velocity of the pelvis.

#### 3.4. Least-Squares Tracking Problem

To determine if the subject's gait is within the solution space of the model we form a least-squares OCP to track the subject's data. As previously noted, not all of the experimental measurements are of equal accuracy: while the CoP and ground forces are very accuracy measured by the force plates, the kinematic data is subject to error on the order of a centimeter or two due to skin artifact (Fuller et al., 1997). To make the best use of the data, we have formulated a tracking (indicated by <sup>T</sup> where appropriate) problem which has a Lagrange φ L T and Mayer 8<sup>M</sup> T terms of

$$\begin{split} \phi\_{\rm T}^{\rm L} &= \sum\_{i=3}^{9} \left( \boldsymbol{\mu}\_{\theta,i} (\theta\_{i} - \theta\_{i}^{\rm EXP})^{2} + \boldsymbol{\mu}\_{\boldsymbol{\alpha},i} (\boldsymbol{\omega}\_{i} - \boldsymbol{\omega}\_{i}^{\rm EXP})^{2} \right) \\ &+ \boldsymbol{\nu}\_{\rm P} (r\_{\rm P} - r\_{\rm P}^{\rm EXP})^{2} + \boldsymbol{\nu}\_{\rm F} \{\boldsymbol{f}\_{\rm P} - f\_{\rm P}^{\rm EXP}\}^{2} + 10^{-3} \sum\_{i=0}^{n\_{\rm u}} \left( u\_{i}^{2} + a\_{i}^{2} \right) \end{split} \tag{18}$$

$$\Phi\_{\rm T}^{\rm M} = 10^{-5} \sum\_{i=0}^{n\_{\Delta}} \Delta\_i^2 \tag{19}$$

where n<sup>u</sup> and n<sup>1</sup> are the number of control signals and the number of impulses respectively. This cost function is applied across all phases of the problem. Note that the Lagrange term is an integrated quantity, as such all of the experimental data is interpolated as a function of time prior to evaluating and numerically integrating (Equation 18).

The Lagrange term is formulated so that the angles and angular velocities of the pelvis and leg joints (indices for θ<sup>3</sup> − θ<sup>9</sup> illustrated in **Figure 1**) are tracked along with the CoP , and ground forces. The weighting terms on the angles wθ,<sup>i</sup> , and angular velocities wω,<sup>i</sup> are set to <sup>1</sup> π/4 and 0.1 π/4 , respectively with the exception of the ankle joint which is set to <sup>1</sup> <sup>100</sup> of these nominal values: kinematic error that the foot introduces will be most readily observed at the ankle. The weighting terms w<sup>P</sup> and w<sup>F</sup> associated with the normal components are normalized with respect to maximum recorded contact forces. In addition, we have introduced three regularization terms: the sum of squared control signals u 2 and activations a 2 in the Lagrange term, and the sum of squared ground impulses 1<sup>2</sup> in the Mayer term. The coefficient on the regularization terms has been chosen so that the terms have a similar magnitude. Here we evaluate the Lagrange term only at the shooting nodes (making this a discrete least-squares problem).

#### 3.5. Minimization Prediction Problem

Inspired by the experimental work of Hoyt and Taylor (1981) and later Farley and Taylor (1991), we formulate a prediction (indicated with a <sup>P</sup>) cost function in with a Lagrange term on muscle activation

$$\phi\_{\mathbb{P}}^{\mathbb{L}} = \sum\_{i=0}^{n\_{\omega}} a\_i^2 \tag{20}$$

and a Mayer term that includes foot-ground impacts

$$\Phi\_{\rm P}^{\rm M} = \boldsymbol{w}\_{\Delta} \sum\_{i=0}^{n\_{\Delta}} \Delta\_i^2. \tag{21}$$

Here w<sup>1</sup> is set to 10−<sup>2</sup> , a value which found in our preliminary simulations to be sufficient to reduce the ground force discontinuities introduced by ground impacts to levels consistent with the experimental data. So that the physical demands placed on the foot models are comparable to the subject data, in addition, we introduce two constraints: that the average forward velocity of the solution is identical to the subject's (1.01 m/s), and that the step length of the model matches that of the subject (0.61 m). Note that this problem formulation, while useful for our purposes, cannot be used to predict human walking in general because we have explicitly included a desired forward velocity and step length.

#### 3.6. Numerical Solution Method

To solve the tracking and prediction OCPs specified we use a direct multiple shooting method described by Bock and Pitt (1984) and implemented in the software package MUSCOD-II developed by Leineweber et al. (2003). In a direct approach, the infinite-dimensional space of control functions u(·) is discretized in time using functions which provide only local support. Here we use piece-wise linear functions to describe the excitation signals to the MTGs. State parameterization is performed by the multiple shooting technique which transforms the OCP, together with the control discretization, from an infinite dimensional problem into a finite dimensional problem which is then solved iteratively using a sequential-quadraticprogramming (SQP) solver that has been tailored to exploit the structure of the problem.

We initialize the problem with a rough initial solution: positions and velocities are initialized using a linear interpolation of the experimental positions which are then polished to satisfy the foot ground constraints, activations are set to 0.01, control signals are set to 0.025. The initial solution does not satisfy the OCP constraints and is not a feasible motion. The OCPs using the ellipse and double circle foot models are discretized using 25 and 31 shooting nodes and control intervals, respectively. Each shooting interval is integrated using the Runge-Kutta-Fehlberg method with an absolute and relative tolerance of 10−<sup>8</sup> . Note that, in contrast to direct-collocation (Von Stryk, 1993), the dynamics of the system are simulated using a variablestep integrator over the entire duration of the simulation. To reduce the drift of the foot-ground constraints, we use Baumgarte stabilization (Baumgarte, 1972) applied to the contact constraint position and velocity errors, and to the rolling constraint velocity errors. The OCPs are run until the Karush-Kuhn-Tucker condition is satisfied to a tolerance of 10−<sup>5</sup> . Each problem required between 20 and 50 min of processing time on an Intel i7-3630QM CPU with a clock speed of 2.40 GHz.

### 4. RESULTS

When forced to track the recorded CoP, both the fitted ellipsefoot and the double-circle foot produce ankle trajectories that differ from the subject's on the order of one centimeter, but have maximum errors that exceed this desired limit as shown in **Table 1**. Though the fitting process restricted the length of the foot models to have a realistic length, the height of the TABLE 1 | The position (*r*A − *r* EXP A ) and orientation (αA − α EXP A ) errors between the ankle frame of each respective foot model and experimental data of a subject's foot during the stance phase.


*These differences are present when each foot model is constrained to have a CoP which is identical to that of the subject.*

TABLE 2 | The parameters of each foot model which best fit the subject's data are listed below.


*As noted in section 3.2, the heel segment is placed so that the length of the foot is 30.5 cm, while all other parameters are free to vary. Note that* <sup>A</sup>α<sup>E</sup> *is the rotation from* K<sup>A</sup> *to* KE*. Please see* Figure 1 *for a graphical depiction of the remaining parameters.*

foot model shapes does exceed the size of a shoe particularly at the forefoot of the double-circle foot as shown by the parameter value for r<sup>F</sup> shown in **Table 2**. The double-circle foot offers a slightly better tracking of the subject's ankle position while the ellipse-foot is slightly superior in its reproduction of the orientation of the ankle frame. The ankle trajectory (**Figures 4**, **5**) traced by the two different foot models show that the highest errors occur during heel contact: it is during this period that the rigid approximation to the foot is worst because the heel pad and shoe are compressing. Further, the ankle trajectory of the two models displays a characteristic difference: the ellipse foot produces a u-shaped ankle trajectory (marked with a "\*" in **Figure 4**) while the double circle produces a v-shaped trajectory due to the forefoot circle plastically contacting the ground.

The solution of the tracking OCP shows that the ellipsefoot is able to reproduce the orientation of the subject's foot (**Figure 6A**), and ankle angle (**Figure 6D**) with better accuracy than the double-circle foot as the summary statistics show in **Table 3**. Both tracking OCPs had difficulty reproducing the subject's knee angle (**Figure 6E**) between near 75% of the stance phase, because the foot models fail to capture the shape of the

FIGURE 4 | When the ellipse-foot is constrained to track the CoP (illustrated with the butterfly plots for reference) from the subject's data it is able to closely reproduce the subject's foot movements during mid-stance and for most of toe-off. The initial kinematics of the foot during heel contact are not well-captured. The continuous rolling motion of the foot forces the ankle frame through a smoothened cusp which is annotated with an "\*". Note that the dots which appear on each line would coincide if the foot model perfectly fit the subject's foot movements.

mid-stance both the heel and forefoot contacts touch the ground which fixes the ankle at the same location. Note that the dots which appear on each line would coincide if the foot model perfectly fit the subject's foot movements.

foot at the transition between mid-stance and toe-off. The hip angle is tracked with comparable accuracy by both foot models (**Figure 6F** and **Table 3**). Though the double-circle foot tracks the CoP more accurately than the ellipse-foot (**Figure 6B** and **Table 3**), the ground forces produced by the double-circle foot exhibit oscillations that are present to a lesser degree in the ellipse-foot (**Figure 6C**).

The solutions of the prediction OCP from each model deviates from the subject's data in general as shown in **Table 4**, but in

(double-circle foot in blue) of each plot. Note that the ankle angle (D) [and thus orientation of the foot (A)] is weakly tracked because kinematic errors at this joint are scaled to be <sup>1</sup> <sup>100</sup> the value of other tracked quantities. The ellipse-foot uniformly tracks the subject's data with a higher degree of accuracy than the double-circle foot. Both models have difficulty tracking the subject's knee angle at 75% stance (E), and the subject's ground forces near 25% and 75% of stance. During heel-only contact the double-circle foot is not able to track the subject's CoP movements (B), but closely follows the subject's CoP trajectory thereafter. Due to ground impacts both models have ground forces that begin the stance phase with finite values. The double-circle foot has an additional discontinuity in both the CoP profile (B) and ground force profile (C) when the forefoot strikes the ground. The forefoot strike completely arrests the angular velocity of the double-circle foot (see the flat line in phases 4c and 5c in A) while the ellipse-foot continuously rotates during stance. Both solutions are able to closely follow the subject's hip angle trajectory (F).

different ways as observed at the kinematics of the hip, knee and ankle (**Figures 7D–F**). These large differences underscore how influential the shape of the foot is on the gait of the model because everything else about the two problems is identical except for the foot model. Another difference of note is observed in the ground forces produced by the ellipse-foot model: the normal and tangential forces exhibit a transient that is about 13 ms in duration that departs from the recorded ground forces by 6582.7 N and 2137.0 N (marked with a "\*" **Figure 7C**), respectively. The nature of the transient is not numerical (the largest Baumgarte forces are 10.6 N), nor due to an impact, but due to an interaction between the motion of the model and the single curved foot segment: at precisely this moment the ankle of the ellipse-foot is at the bottom-most part of the u-shaped trajectory it traces (marked with a "\*" in **Figure 4**). The ellipse-foot rotates the ankle quickly (hitting the upper bound 14.6 rad s ) through the ushaped trajectory accelerating the ankle frame upwards. Since the

knee is nearly straight at this time the entire mass of the torso is also accelerated upwards. The brief, but rapid, acceleration of the ankle frame of the ellipse-foot results in a brief, but large, spike in the simulated ground reaction forces. To confirm this suspicion we re-ran the prediction OCP with but limited the angular velocity (from 14.6 rad s to 3.96 rad s ) of the ankle joint until the peak contact forces were comparable to those of the double-circle foot (1,444 N vs. 1,219 N). Though this extra constraint reduced the unrealistic ground forces, the constraint itself represents a departure from reality because the subject's ankle rotated at a greater velocity (4.41 rad <sup>s</sup> <sup>&</sup>gt; 3.96 rad s ) during the experiment. In contrast, the plastic impact of the forefoot circle arrests the motion of the double-circle foot effectively preventing the force transient produced by the ellipse-foot. Please see the accompanying **Supplementary Material** section for videos, additional plots, and code for both the models and the OCPs.



*Note that the ankle angle and foot orientation are free to vary, while all other quantities listed below are tracked.*

TABLE 4 | The difference between the results of the prediction OCP of each model's stance leg and that of the subject show that, when free to vary, the final gait is quite different from that of the subject.


*Although the kinematics of the stance ankle from the ellipse-foot model more closely follow the subject than the double-circle foot, the ellipse-foot has a large force transient (see section 4 for details) due to its mechanics. Though the ground forces created by the double-circle foot model differ from the subject, these errors are relatively small when compared to similar works in the literature.*

## 5. DISCUSSION

While there are many applications for computerized gait prediction, few applications are possible without an accurate model of foot-ground contact. Though much attention has been given to modeling the foot it has proven difficult to simultaneously achieve realistic foot-ground contact kinematics and simulated ground forces. In this work we evaluated two rigid foot-ground contact models in terms of how well each replicated the kinematics of the stance foot and how each affected a tracking OCP and a prediction OCP. The multiple layers of evaluation proved useful. Although the ellipse foot model is better able to fit the kinematics of the stance foot in the least squares OCP, the prediction OCP illustrates that, because this foot model continuously rolls at a single contact point, it is capable of producing enormous contact forces due to the curve it forces the ankle through. This result has a larger implication: the accuracy displayed by a foot-ground contact model during an isolated fitting, or a tracking OCP , does not necessarily generalize to a prediction OCP. In addition, to our knowledge, this is the first work in the literature which solves two prediction OCPs that are identical in every respect except for the model of foot ground contact. The differing predictions of the ellipse-foot and doublecircle foot models confirm a long held suspicion that the model of foot-ground contact has a large influence on the optimal motion of the model.

The transient present in the prediction OCP of the ellipse-foot indicates that foot models consisting of a single roll-over shape (Ren et al., 2007; Römer, 2018; Römer et al., 2018) should be treated with some caution. Although the transient we observed with the ellipse-foot does not appear in the work of Ren et al. (2007) there are a few reasons why this might be true. First, Ren et al. (2007) did not allow the feet to move freely during double stance, but constrained the CoP trajectory and ground forces under each foot to follow prescribed linear function. Constraining the movements reduces the magnitude of the simulated ground forces as clearly shown by the tracking solution (**Figure 6C**) and the prediction OCP with the constrained ankle angular velocity. While the constrained solutions produce more realistic results, this is an undesirable option: it is not clear what the constraint should be ahead of time. The second reason why Ren et al. (2007) may not have observed this transient is because they sampled system dynamics discretely during the solution process: the transient could have been skipped between grid points. Due to the brief nature of the transient, if the model is being simulated using a grid of time points (as is typical of direct-collocation) it is important that a final high-resolution integration be performed to ensure that the results have not been unduly affected.

The optimal control solutions of Römer (2018) also have ground forces which are free of transients (personal communication, ground forces are not reported in the thesis) likely because of differences in the problem formulation and solution method. We have used a forward-dynamics problem formulation which allows the optimization routine to manipulate the generalized forces but then integrates the dynamics of the system forward in time. Römer (2018) made use of a hybrid-zero-dynamic (HZD) approach developed by Westervelt et al. (2007) which uses a mixture of an inverse- and forwarddynamic problem formulation: all of the joints of the model are constrained to follow polynomial functions of the whole-body

(E), and hip (F) of the stance leg. In one regard the solutions of the ellipse-foot and double-circle foot are similar: in both cases the model keeps its weight on its heel until the last moments of the stance phase (B). Note that the double-circle foot produces ground forces that are similar in magnitude to the tracking problem while the ellipse-foot produces a large transient ground force (C) shortly after contact within the continuous phase 2e. This transient is not due to an impact, nor constraint stabilization, but instead due to an interaction between the walking motion and the ellipse-foot model. As with the tracking solution, the angular velocity of the double-circle foot is arrested between phases 4c and 5c by the forefoot contact while the ellipse-foot continuously rotates during stance (A). Finally note that phase 4c, while of brief duration in the tracking solution, is of zero duration in the prediction solution.

lean angle; the entire system is reduced to a single DoF which is integrated forward in time. The force transient we observed required a rapid change in the angular velocity of a foot, a rapid change which cannot be described using the polynomials employed by Römer (2018).

The inevitable discrepancies that arise between predicted motions and typical human movement can be illustrative of gaps between our understanding of the mechanics of the body, and how these structures coordinated during movement. Both models resulted in tracking OCP solutions in which the ankle angles which differed from the subject's at heel contact, and the knee angle departed from the subject's near 75% of the stance phase. The most likely explanation for both of these problems is that the shapes we used to represent the foot are a poor match at heel contact and near the transition from mid-stance to toe-off. The large increase in error between the prediction OCP and the experimental data show some obvious directions for improvement. In both cases, the model kept its weight close to its hind foot (**Figure 7B**) before rapidly pushing-off. This trajectory results in a large error between the simulations and the experimental data of the orientation of the foot (**Figure 7A**) and the CoP trajectory (**Figure 7B**). The departure in CoP trajectory is likely due to the fact that the MTGs we used in this work have rigid tendons which do not offer the cost savings that a elastic tendon can when it is loaded slowly and allowed to recoil rapidly. The rapid force oscillations present during the stance phase of the double-circle foot prediction OCP solution, while within the limits of the activation model, are not present in experimental recordings of human walking (**Figure 7C**). We suspect that these oscillations may be due to the fact that a Hill-model does a poor job of capturing the stiffness and damping properties of actively lengthened muscle (Kirsch et al., 1994). These force oscillations would appear larger with the double-circle foot during heel and forefoot contact because it is constrained from moving and thus perfectly transmits the wrench applied to the ankle to the ground. The ellipse-foot, in contrast, is always free to rotate about its contact point and would move, attenuating the observed ground force oscillation.

## 6. CONCLUSIONS

Single segment rigid foot ground contact models are an attractive means to model the foot but should be treated with caution: under the right circumstances these foot models can produce large transient forces if the ankle rapidly moves through a ushaped trajectory after heel contact. In contrast, we did not observe the same transient using the two-segment rigid foot model because the plastic impact of the forefoot arrests the motion of the ankle through its v-shaped trajectory. Although the two-segmented rigid foot model results in an OCP with substantially more phases than a single segment foot, the twosegmented foot has a benefit: it does not require special treatment and may be a closer mechanical analog to the human foot. Finally, though we treated the foot as a rigid object the ground forces of the prediction OCP are relatively smooth due to the inclusion of the impulses in the cost function. Though the inclusion of the impulse term improved our simulation results, the experimental work of Hoyt and Taylor (1981) and later Farley and Taylor (1991) suggests that terms for both muscular effort and ground contact terms should appear in cost functions used to predict legged locomotion.

### DATA AVAILABILITY

All datasets analyzed for this study are included in the manuscript and the **Supplementary Files**.

### ETHICS STATEMENT

The recordings were conducted according to the guidelines of the Declaration of Helsinki 2013 and approved by the ethics committee of Faculteit der Gedrags-en Bewegingswetenschappen (Faculty of Behavioral and Movement Sciences) at Vrije Universiteit.

## REFERENCES


## AUTHOR CONTRIBUTIONS

MM worked with KM to develop the proposal that funded this work. MM undertook the work and the writing. KM provided critical review during the preparation of the manuscript.

## FUNDING

We gratefully acknowledge the financial support from the Deutsche Forschungsgemeinschaft through grant no. MI 2109/1-1. In addition, we are grateful for the financial support for open access publishing provided by the Deutsche Forschungsgemeinschaft within the funding program Open Access Publishing, by the Baden-Württemberg Ministry of Science, Research and the Arts and by Ruprecht-Karls-Universität Heidelberg.

### ACKNOWLEDGMENTS

The authors would like to thank Dr. Gert Faber and Axel Koopman at the Vrije Universiteit for their help collecting the experimental data used in this work. We also wish thank the Simulation and Optimization research group of the IWR at Heidelberg University for allowing us to work with the optimal control code MUSCOD-II.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnbot. 2019.00062/full#supplementary-material

Please see the supplemental data for videos, data, and code. The data is accompanied by scripts that can be used to generate the tables and plots that appear in the results section. In addition, the scripts can also generate plots that show activation profiles, constraint error profiles, Baumgarte stabilization forces, and the phase of each OCP.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Millard and Mombaur. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

## Computational Design of FastFES Treatment to Improve Propulsive Force Symmetry During Post-stroke Gait: A Feasibility Study

Nathan R. Sauder <sup>1</sup> , Andrew J. Meyer <sup>1</sup> , Jessica L. Allen<sup>2</sup> , Lena H. Ting2,3, Trisha M. Kesar <sup>3</sup> and Benjamin J. Fregly <sup>4</sup> \*

*<sup>1</sup> Computational Biomechanics Laboratory, Department of Mechanical and Aerospace Engineering, University of Florida, Gainesville, FL, United States, <sup>2</sup> Neuromechanics Laboratory, Wallace H. Coulter Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, GA, United States, <sup>3</sup> Motion Analysis Laboratory, Department of Rehabilitation Medicine, Emory University School of Medicine, Atlanta, GA, United States, <sup>4</sup> Rice Computational Neuromechanics Laboratory, Department of Mechanical Engineering, Rice University, Houston, TX, United States*

#### Edited by:

*Hyung-Soon Park, Korea Advanced Institute of Science & Technology (KAIST), South Korea*

#### Reviewed by:

*Thomas C. Bulea, National Institutes of Health (NIH), United States Sarah A. Roelker, University of Texas at Austin, United States*

> \*Correspondence: *Benjamin J. Fregly fregly@rice.edu*

Received: *01 May 2019* Accepted: *10 September 2019* Published: *01 October 2019*

#### Citation:

*Sauder NR, Meyer AJ, Allen JL, Ting LH, Kesar TM and Fregly BJ (2019) Computational Design of FastFES Treatment to Improve Propulsive Force Symmetry During Post-stroke Gait: A Feasibility Study. Front. Neurorobot. 13:80. doi: 10.3389/fnbot.2019.00080* Stroke is a leading cause of long-term disability worldwide and often impairs walking ability. To improve recovery of walking function post-stroke, researchers have investigated the use of treatments such as fast functional electrical stimulation (FastFES). During FastFES treatments, individuals post-stroke walk on a treadmill at their fastest comfortable speed while electrical stimulation is delivered to two muscles of the paretic ankle, ideally to improve paretic leg propulsion and toe clearance. However, muscle selection and stimulation timing are currently standardized based on clinical intuition and a one-size-fits-all approach, which may explain in part why some patients respond to FastFES training while others do not. This study explores how personalized neuromusculoskeletal models could potentially be used to enable individual-specific selection of target muscles and stimulation timing to address unique functional limitations of individual patients post-stroke. Treadmill gait data, including EMG, surface marker positions, and ground reactions, were collected from an individual post-stroke who was a non-responder to FastFES treatment. The patient's gait data were used to personalize key aspects of a full-body neuromusculoskeletal walking model, including lower-body joint functional axes, lower-body muscle force generating properties, deformable foot-ground contact properties, and paretic and non-paretic leg neural control properties. The personalized model was utilized within a direct collocation optimal control framework to reproduce the patient's unstimulated treadmill gait data (verification problem) and to generate three stimulated walking predictions that sought to minimize inter-limb propulsive force asymmetry (prediction problems). The three predictions used: (1) Standard muscle selection (gastrocnemius and tibialis anterior) with standard stimulation timing, (2) Standard muscle selection with optimized stimulation timing, and (3) Optimized muscle selection (soleus and semimembranosus) with optimized stimulation timing. Relative to unstimulated walking, the optimal control problems predicted a 41% reduction in propulsive force asymmetry for scenario (1), a 45% reduction for scenario (2), and a 64% reduction for scenario (3), suggesting that non-standard muscle selection may be superior for this patient. Despite these predicted improvements, kinematic symmetry was not noticeably improved for any of the walking predictions. These results suggest that personalized neuromusculoskeletal models may be able to predict personalized FastFES training prescriptions that could improve propulsive force symmetry, though inclusion of kinematic requirements would be necessary to improve kinematic symmetry as well.

Keywords: fast treadmill training, functional electrical stimulation, neuromusculoskeletal modeling, computational modeling, direct collocation optimal control, paretic propulsion, stroke, muscle synergies

### INTRODUCTION

Approximately 15 million people experience a stroke each year (MacKay and Mensah, 2004), with walking dysfunction being one of the most common sequelae (Lloyd-Jones et al., 2010; Verma et al., 2012). Stroke-related walking disability has been associated with a host of co-morbidities including hypertension, heart disease, diabetes, and cognitive decline (Ostwald et al., 2006; Abellan van Kan et al., 2009; Mutikainen et al., 2011; Ostir et al., 2013; Garcia-Pinillos et al., 2016; Rosso et al., 2017; Savica et al., 2017) resulting in a decreased quality of life and increased risk of death (Nor Azlin et al., 2016). While stroke rehabilitation treatments often restore some level of walking function (Balaban et al., 2011), they rarely restore walking ability to a pre-stroke level (Bogey and Hornby, 2007). Strokeinduced walking deficits primarily affect one side of the body, resulting in a slow and asymmetric gait pattern (Verma et al., 2012) characterized by neural control changes (Lamontagne et al., 2007) and compensatory motion patterns. For example, post-stroke gait is often characterized by reduced paretic leg propulsion during stance phase (McGinley et al., 2006) and decreased paretic leg toe clearance from the ground during swing phase (Verma et al., 2012). These changes lead to inefficient compensatory strategies such as increased propulsive force generation on the non-paretic side and hip hiking to facilitate toe clearance on the paretic side.

Fast-speed treadmill training with functional electrical stimulation (FastFES) is a promising treatment for improving walking ability post-stroke. However, little data currently exist for determining the best way to customize the treatment to individual-specific gait deficits. The rationale for FastFES training is that fast speed walking by itself can improve gait biomechanics in individuals post-stroke, while task-specific electrical stimulation of selected muscles can provide feedback to the nervous system to promote motor learning of the appropriate timing and activation of the stimulated muscles (Kesar et al., 2011; Awad et al., 2014, 2016). The FastFES intervention has been shown to improve gait function and energy cost of gait in individuals with chronic post-stroke hemiparesis (Awad et al., 2014, 2016). This combination may help the damaged central nervous system adapt favorably to the new post-stroke reality. However, selection of muscles to stimulate, along with stimulation timing and amplitude, are currently standardized based on the normal activation profile of muscles during gait, clinical intuition, and the subject's tolerance to electrical stimulation. Furthermore, during FastFES, only two muscles are typically targeted for stimulation—tibialis anterior and gastrocnemius—due to technical limitations. Tibialis anterior is stimulated to improve paretic toe clearance during swing phase, and gastrocnemius is stimulated to improve paretic propulsion at the end of stance phase (Allen et al., 2018). Timing patterns of the stimulation are usually constrained to simple on/off cycles based on foot switch signals.

Though stroke affects each patient's neural control capabilities differently, current standardized FastFES treatment does not account for this reality. Patient specific coordination deficits suggest the need for patient-specific treatment prescriptions (Allen et al., 2018). Inter-individual variability in post-stroke sensorimotor impairments may explain why some patients respond to standardized treatment while others do not. Non-responders could potentially experience greater treatment efficacy if different muscles were stimulated, or if the standard muscles were stimulated with different timing. However, no method currently exists for predicting a priori which two muscles are the best targets for stimulation, and how they should be stimulated, to achieve the maximal improvement in walking function for any particular patient.

An emerging approach for addressing the treatment personalization problem is personalized neuromusculoskeletal modeling. If key parameter values in a neuromusculoskeletal model are personalized to the unique anatomical, physiological, and neurological characteristics of a specific patient, then the resulting personalized model could potentially be used to predict and even optimize an individual patient's functional outcome for different treatment scenarios under consideration (Fregly et al., 2012; Meyer et al., 2016). Several modeling studies have already analyzed or optimized various aspects of muscle electrical stimulation, including electrode shape in epiretinal stimulation (Cao et al., 2015), stimulation site selection for hand opening (De Marchis et al., 2016), stimulation pulse duration and polarity for de-enervated muscles (Pieber et al., 2015), stimulation profiles and lower limb trajectories to improve FES- and orthosis-based walking (Sharma et al., 2014), and stimulation timing for foot drop correction (Azevedo Coste et al., 2014). However, no study to date has used a personalized neuromusculoskeletal model to design a personalized FastFES treatment tailored to the functional limitations of a specific individual post-stroke.

This study evaluates the feasibility of designing a personalized FastFES treatment protocol using a personalized neuromusculoskeletal model coupled with direct collocation optimal control. The subject studied was a non-responder to the standard FastFES treatment protocol (Allen et al., 2018), making him an excellent candidate for computational exploration of alternative muscle stimulation protocols that theoretically could improve his treatment outcome. The treatment design problem was framed as a direct collocation optimal control problem that minimized propulsive force asymmetry between the two legs while making minimal changes to the subject's nonstimulated neural control strategy, which was modeled using subject-specific muscle synergies. Propulsive force symmetry was targeted for improvement since recent studies have shown that it is an important determinant of walking ability (Bowden et al., 2006; Schmid et al., 2007). The computational treatment design process involved personalizing key parameters in a full-body neuromusculoskeletal walking model to treadmill walking data collected from the subject, and then using the personalized model to solve a sequence of direct collocation optimal control problems. An initial optimal control problem verified that the personalized model could be used to predict the subject's unstimulated muscle activations, joint kinematics, and ground reactions. The remaining optimal control problems predicted how the subject would walk when two paretic leg muscles were stimulated in three ways: (1) Standard muscle selection with standard stimulation timing, (2) Standard muscle selection with optimized stimulation timing, and (3) Optimized muscle selection with optimized stimulation timing. The results demonstrate the feasibility of using this computational treatment design approach for identifying new avenues of clinical exploration.

### MATERIALS AND METHODS

### Experimental Data Collection

We collected treadmill gait data from an individual post-stroke (age >70 years, ∼8 years after stroke) who was a non-responder to the standard FastFES treatment protocol. The subject gave written informed consent, and the study was approved by the institutional review boards of Emory University and the University of Florida. Collected data included full-body video motion capture data (Vicon, Centennial, CO, USA), bilateral force plate data from a split-belt instrumented treadmill with belts tied (Bertec Corporation, Columbus, OH, USA), and surface EMG data from 14 muscles per leg (**Table 1**; 28 total signals) (Konigsberg Instruments, Pasadena, CA, USA). The subject had a slow self-selected walking speed of 0.3 m/s and a visually asymmetric gait pattern, with the paretic right leg exhibiting stereotypical hip hiking. The same subject was participant NR1 in a recently published FastFES clinical study (Allen et al., 2018), although the data used in the present study (which were much more extensive) were collected more than a year after completion of the clinical study. Notes taken during the clinical study indicated that further investigation into the causes of nonresponse and corresponding alterations to treatment design were needed for this subject.

Experimental data were collected for several types of trials. Static trial data were collected in which the subject stood upright with an anatomically neutral joint alignment for several seconds with feet pointing forward. Data from this static trial were used for scaling an initial generic musculoskeletal model and TABLE 1 | List of muscles present in each leg of the neuromusculoskeletal model, including muscles with measured EMG signals (Measured), muscles whose EMG signals were copied from neighboring muscles with similar anatomical function (Copied), and muscles whose EMG signals were predicted using synergy signals extracted from measured EMG signals (Predicted).


determining the locations of reflective surface markers on the model (see below). Six different isolated joint motion trials were performed to facilitate calibration of lower-body joint positions and orientations in the body segments. One isolated joint motion trial was performed for each hip, knee, and ankle, where each trial exercised all functional axes for the selected joint (Reinbolt et al., 2005, 2008). Gait trials were performed at the subject's fastest comfortable walking speed of 0.6 m/s without FES, which was the speed used for the subject's previous FastFES training. To help maintain balance during the treadmill gait trials, the subject rested his hands on a handlebar suspended from the ceiling. One representative gait trial was selected for subsequent computational modeling and optimization efforts. The selection process involved identifying gait cycles with clean surface marker, ground reaction, and EMG data, eliminating cycles near the start and end of the trial where transient conditions were present, and finally determining the one gait cycle whose period was closest to the mean.

### Computational Model Personalization

We personalized a generic full-body musculoskeletal model (Hamner et al., 2010) developed in OpenSim (Delp et al., 2007; Seth et al., 2018) to the unique anatomical, physiological, and neurological characteristics of the subject using the subject's experimental movement data. The generic model possessed 44 lower-body muscles, of which 36 were retained, and 37 degreesof-freedom (DOFs), including three DOF hip joints, one DOF knee joints, and two DOF ankle joints. As a preliminary task, the generic OpenSim model was scaled to the subject's dimensions using surface marker data from the static trial and OpenSim's Scale Model tool. Three mutually perpendicular forces and moments were added to each hand in the model to account for the subject's hands resting on a handlebar. In addition, a backpack was added to the torso of the model to account for EMG system hardware. Muscles controlled the hips, knees, and ankles of the model, while net torque actuators controlled the lower back joint and the two shoulder, elbow, and toes joints. Activation dynamics, Hill-type muscle models with rigid tendons (De Groote et al., 2016), surrogate musculoskeletal geometry models, and deformable foot-ground contact models were implemented in Matlab (the Mathworks, Natick, MA, USA) for use within the OpenSim skeletal model.

As described briefly below, model personalization involved a four-step calibration process performed in Matlab with calls to OpenSim analyses through Matlab Mex functions and OpenSim's C++ API. Traditional optimization problems used to calibrate model parameter values were solved with either the "lsqnonlin" or "fmincon" optimizer in Matlab, while direct collocation optimal control problems used to calibrate model parameter values and controls and to predict new gait motions were solved with GPOPS-II optimal control software for Matlab (Patterson and Rao, 2014) using the IPOPT optimizer (Wächter and Biegler, 2006). The four steps in the model personalization process were similar to the ones presented in a recent study (Meyer et al., 2016), which provides further details on the process.

#### Joint Model Personalization

The first step involved personalization of the model's lower-body functional axes using data from the isolated joint motion trials and selected gait trial combined with repeated OpenSim "Inverse Kinematics" analyses. Only pelvis and lower body marker motion data were needed for this step.

To perform this model personalization step, we formulated an optimization problem that sought to calibrate parameters defining the positions and orientations of the lower body joints (hips, knees, and ankles) in their respective body segments as well as parameters defining the positions and orientations of marker triads placed on the pelvis, thighs, shanks, and feet of the model. The cost function minimized the sum of squares of errors between experimental and model-predicted marker positions using all motion trials together. Each function evaluation performed an OpenSim "Inverse Kinematics" analysis to calculate the current marker location errors. Matlab's "lsqnonlin" non-linear least squares algorithm was used to perform the optimization. The calibrated joint and marker triad positions and orientations were applied to the model and used in a final OpenSim "Inverse Kinematics" analysis to determine joint position, velocity, and acceleration time histories for subsequent steps of the model personalization process.

#### Muscle-Tendon Model Personalization

The second step involved personalization of the model's EMGdriven muscle-tendon force and moment generating properties using data from 40 gait cycles and OpenSim "Inverse Dynamics" and "Muscle" analyses. The data needed for this step included joint kinematics found as in the first step along with ground reaction and EMG data. Three tasks were performed in preparation for this model personalization step. First, EMG data from 40 gait cycles were processed (high-pass filtered, demeaned, rectified, and low-pass filtered) as described in Meyer et al. (2016), resulting in envelopes of muscle excitation. Second, an OpenSim "Inverse Dynamics" analysis was performed to calculate the net hip, knee, and ankle joint moments to be matched by muscle forces estimated by this step of the model personalization process. Third, an OpenSim "Muscle" analysis was performed repeatedly to calculate muscle-tendon lengths and moment arms for each muscle over a wide range of sampled combinations of lower-body joint positions. The sampled quantities were fitted simultaneously as polynomial functions of joint positions as described in Meyer et al. (2016), thereby producing surrogate representations of the subject's musculoskeletal geometry for rapid calculation of muscle-tendon lengths, velocities, and moment arms.

Once these preparatory tasks were completed, we formulated an optimization problem that sought to calibrate Hill-type muscle-tendon model parameters (scale factors for EMG normalization, electromechanical delays, activation dynamics time constants, activation non-linearization shape factors, optimal muscle fiber lengths, and tendon slack lengths) along with parameters defining the surrogate musculoskeletal geometry. The cost function minimized the weighted sum of squares of errors between three types of lower-body quantities: (1) inverse dynamic and model-predicted total joint moments, (2) published experimentally measured (Silder et al., 2007) and model-predicted passive joint moments, and (3) initial and current model parameter values as regularization terms (i.e., terms that minimized changes in parameter values away from their initial guesses to make the optimal solution unique). Joint moment errors were calculated for hip flexion-extension and adduction-abduction, knee flexion-extension, and ankle plantarflexion-dorsiflexion and inversion-eversion. Twice as much weight was placed on hip moment errors as on knee and ankle moment errors to produce comparable magnitude errors at all three joints. No OpenSim analyses were needed for function evaluations. Matlab's "fmincon" sequential quadratic programming algorithm was used to perform the optimization. The calibrated muscle activations were required for the final step of the model personalization process.

Because fine-wire EMG data were not available for important deep muscles (i.e., iliopsoas, tibialis posterior, extensor digitorum longus, and flexor digitorum longus), we incorporated muscle synergy techniques into the optimization process to estimate the activations for the 4 muscles in each leg with missing EMG signals (Bianco et al., 2017). Synergy analysis of the subject's muscle activations revealed that only 2 or 3 synergies (depending on normalization method) were needed to achieve 95% total VAF for each leg. However, since our muscle synergies were not simply fitting EMG data but rather making the subject's personalized model walk in a dynamically consistent manner that closely matched all available experimental data, we wanted all muscle activations in the model to be reconstructed with at least 95% VAF. Consequently, we chose to control each leg with 5 synergies, which was the number required to surpass the 95% individual muscle VAF threshold and ensure enough flexibility for constructing the shapes of the missing muscle activations.

Once the number of synergies was selected, we used muscle synergy concepts to extend our EMG-driven model personalization process (Meyer et al., 2017) to the case where 4 important muscles per leg had missing EMG signals. The original personalization process was developed using a full set of EMG signals, where every muscle in the lower body model had either a measured EMG signal or an EMG signal that could be copied from a neighboring muscle with similar anatomical function (e.g., the semimembranosus EMG signal was copied from the semitendinosus EMG signal). Thus, no EMG signals needed to be predicted. To accommodate missing EMG signals, we added two new steps to the personalization process immediately after activation dynamics (**Figure 1**). The first new step performed muscle synergy analysis via non-negative matrix factorization (Lee and Seung, 1999; Tresch et al., 1999; Ting and Chvatal, 2010) on the 14 muscle activations per leg with associated measured EMG signals. This step produced 5 time-varying synergy activations that were assumed to apply to the 4 muscles with missing EMG signals (Bianco et al., 2017). The second new step performed muscle synergy reconstruction by multiplying the 5 time-varying synergy activations extracted in the previous step with the optimization's current guess for the 5 synergies × 4 unknown synergy vector weights/synergy = 20 synergy vector weights per leg for muscles with missing EMG signals. These synergy vector weights were new parameters added to the optimization problem formulation. The second step yielded 4 predicted muscle activations consistent with the optimization's current guess for excitation scale factors, activation parameters, and synergy vector weights for predicted muscles. The 4 muscle activations predicted for each leg were used in the current optimization iteration and updated in future iterations based on the latest values of the optimization parameters.

#### Ground Contact Model Personalization

The third step involved personalization of the model's footground contact properties using data from the selected gait trial combined with repeated OpenSim "Point Kinematics" and "Inverse Dynamics" analyses. The data needed for this step included inverse dynamic joint moments found in the second step along with marker motion and ground reaction data. Compressive non-linear spring-dampers were distributed over a rectangular grid on the bottom of each two-segment foot model (Neptune et al., 2000). Overall grid dimensions were defined using a foot outline generated with a marker pointer (Jackson et al., 2016). Contact elements whose center points were inside the foot outline were retained, while those whose center points were outside of the foot outline were discarded. Contact elements were split between the rear foot and toes segments according to the segment in which they resided. The normal force generated by each contact element was a function of the element's penetration and penetration rate into the floor, while the frictional force generated by each contact element was a function of the element's normal force and slip velocity relative to the floor (Meyer et al., 2016).

To perform this model personalization step, we formulated a direct collocation optimal control problem that sought to calibrate parameters defining the stiffness, damping, and frictional properties of the non-linear spring-dampers on the bottom of each foot (**Table 2**, Model Personalization Problem 1.1; Meyer et al., 2016). The cost function minimized the weighted sum of squares of errors between four types of quantities: (1) experimental and model-predicted marker positions, (2) experimental and model-predicted ground reaction forces and moments, (3) lower-body inverse dynamic and model-predicted joint moments (no muscles were used in this step), and (4) inverse kinematic and model-predicted toe joint angles. The last term was included since a small error in toe marker position could produce a large change in ground reactions, making the foot-ground contact model calibration process much more difficult. Three mutually perpendicular forces and moments were applied to each hand to approximate the loads applied to the hands by the handlebar. Path constraints were used to ensure that the full-body skeletal dynamic equations were satisfied to within the specified tolerance of 1e-6. Each function evaluation performed OpenSim "Point Kinematics" and "Inverse Dynamics" analyses to calculate the current errors in ground reactions, marker positions, and joint moments (cost function) and in skeletal dynamics (constraints). Since skeletal dynamics were evaluated in an inverse rather than forward sense, we added joint jerk controls to the problem formulation to provide explicit forward dynamic equations (e.g., joint jerk is the first time derivative of joint acceleration, etc.) as required by GPOPS-II. Thus, the state vector consisted of joint positions, joint velocities, and joint accelerations.

Using the model with calibrated foot-ground contact parameter values, we solved a subsequent direct collocation optimal control problem to generate a dynamically consistent walking motion that closely tracked experimental marker, ground reaction, and inverse dynamic joint moment data (**Table 2**, Model Personalization Problem 1.2). The problem formulation was identical to the previous one except that foot-ground contact model parameter values were fixed to their calibrated values, minimization of joint jerk controls and the three mutually perpendicular forces and moments applied to each hand were added to the cost function, and bounds were

TABLE 2 | Overview of direct collocation optimal control problem formulations for the neuromusculoskeletal model personalization and FastFES treatment optimization process.


*Model personalization required solving four separate optimal control problems, while treatment optimization involved solving five separate optimal control problems, each of which used a full-body walking model developed in OpenSim and Matlab.*

placed on allowable toe angle errors and on ground reaction and joint angle periodicity errors. The results of this problem were used as the starting point for the final model personalization step.

#### Neural Control Model Personalization

The final step involved personalization of the model's neural control properties using data from the selected gait trial combined with repeated OpenSim "Point Kinematics" and "Inverse Dynamics" analyses. The data needed for this step were the same as for the previous step except for the addition of muscle activations produced by the second step.

To perform this final model personalization step, we formulated a direct collocation optimal control problem that sought to calibrate parameters defining synergy vector weights and controls defining synergy activations (**Table 2**, Model Personalization Problem 1.3). Similar to Meyer et al. (2016), five muscle synergies were used to construct 36 muscle activations per leg. The cost function minimized the sum of squares of errors between four types of quantities: (1) inverse kinematic and model-predicted joint positions, (2) experimental and modelpredicted ground reaction forces and moments, (3) lower-body inverse dynamic and model-predicted joint moments, and (4) EMG-driven and synergy-constructed muscle activations. The cost function again included regularization terms that minimized joint jerk controls and the three mutually perpendicular forces and torques applied to each hand. Path constraints ensured that the full-body skeletal dynamic equations were satisfied, lower body joint moments calculated from inverse skeletal dynamics matched corresponding joint moments calculated from synergy activations, and the hands remained on the handlebars in their experimentally measured positions. Each function evaluation performed OpenSim "Point Kinematics" and "Inverse Dynamics" analyses to calculate the current errors in ground reactions and joint moments (cost function) and in skeletal dynamics and hand positions (constraints). This final calibration optimization yielded muscle synergy controls that closely reproduced not only the subject's muscle activations but also his experimental marker motion and ground reaction data while also producing a dynamically consistent full-body walking motion. The results of this optimization served as the starting point for all subsequent optimal control problems that explored different FastFES treatment scenarios.

#### Complete Model Verification

To gain confidence in the complete personalized model, we solved a verification optimal control problem to demonstrate that we could predict the subject's unstimulated walking motion, joint moments, ground reactions, and muscle activations without tracking any of these quantities in the cost function or bounding any of these quantities in the path constraints (**Table 2**, Model Personalization Problem 1.4). The problem formulation used path constraints to bound synergy activation changes and hand position errors and terminal constraints to enforce motion and ground reaction periodicity. Changes in synergy activations were limited by path constraints rather than tracking terms in the cost function to guarantee a solution with only small changes in synergy activations. No changes in calibrated synergy vector weights were permitted, and applied hand forces and torques were defined to match those found by neural control model personalization. Thus, the only term in the cost function was minimization of joint jerk controls. The verification problem predicted a walking motion that essentially represented the results of a forward dynamic simulation using the complete personalized model.

#### FastFES Treatment Optimizations

We used the subject's personalized neuromusculoskeletal model and direct collocation optimal control to predict the theoretically achievable improvement in anterior-posterior (AP) force symmetry for three FastFES treatment scenarios: (1) Standard muscle selection with standard stimulation timing, (2) Standard muscle selection with optimized stimulation timing, and (3) Optimized muscle selection with optimized stimulation timing. These three treatment scenarios built upon a baseline treatment optimization that predicted the theoretically achievable improvement in AP force symmetry under unstimulated conditions so that the effects of electrical stimulation could be isolated.

Each FastFES treatment optimization built upon a baseline treatment optimization with no electrical stimulation (see below) by adding simulated electrical stimulation to two selected paretic leg muscles. For each treatment optimization, simulated electrical stimulation was added on top of the subject's simulated unstimulated muscle activations. Electrical stimulation waveforms were assumed to be simple step functions defined by an on-time ton, off-time toff , and amplitude A. Since our optimal control prediction problems used controls related to muscle activation (i.e., the output of activation dynamics) rather than muscle excitation (i.e., the input to activation dynamics), we developed a closed-form equation that approximated the amplitude and shape of the activation output produced by activation dynamics (He et al., 1991) when given a step function excitation input:

$$\begin{aligned} a\_{\text{stim}}(t) &= \frac{A}{2} \left\{ \tanh \left( c\_1 \left( t - t\_{\text{on}} - t\_{\text{offset1}} \right) \right) \right. \\ &\quad - \tanh \left( c\_2 \left( t - t\_{\text{off}} - t\_{\text{offset2}} \right) \right) \right\} \\ &\quad + \frac{A}{2} \left\{ 1 - \tanh \left( c\_2 \left( t - t\_{\text{off}} - t\_{\text{offset2}} + t\_{\text{end}} \right) \right) \right\} \quad \text{(1)} \end{aligned}$$

In this equation, astim(t) is the amplitude of activation produced by electrical stimulation at the current time t, tend is the final time of the gait cycle, and c1, c2, toffset<sup>1</sup> , and toffset<sup>2</sup> are adjustable parameters. For any stimulated muscle, we used non-linear least squares optimization to calibrate the four parameters c1, c2, toffset<sup>1</sup> , and toffset<sup>2</sup> to match the output of the muscle's activation dynamics as closely as possible given a step function input of amplitude one. Thus, given A, ton, and toff for any muscle, Equation (1) with calibrated parameters was used to define the muscle's time-varying activation from electrical stimulation. The form of Equation (1) also allows for electrical stimulation to extend beyond the end of the gait cycle and wrap around into the start of the same gait cycle. The total activation of a stimulated muscle was assumed to be the sum of its activation from electrical stimulation and its activation from muscle synergies, where the sum was constrained to be less than one. Furthermore, the maximum activation from electrical stimulation was bounded to be ≤0.7 so that total activation would never exceed one.

#### Baseline Treatment Optimization With No Stimulation

As a starting point for treatment optimization, we formulated and solved a baseline optimal control problem to quantify AP force asymmetry in the absence of electrical stimulation but with minimization of AP force asymmetry added to the cost function (**Table 2**, Treatment Optimization Problem 2.1). The optimal control problem formulation was identical to that of the verification problem except for the addition of a cost function term that minimized the squared difference in AP force impulse between the two legs. The weight on the AP force asymmetry term was chosen to be as large as possible without visibly affecting the predicted motion. This problem formulation allowed AP force asymmetry to be reduced primarily through changes in initial conditions, which were the initial positions and velocities of the skeletal model generalized coordinates, rather than through changes in muscle activations. The AP force asymmetry produced by this baseline problem served as the reference for quantifying improvements produced by the three FastFES treatment scenarios.

#### FastFES Treatment Optimization With Standard Muscles and Standard Timing

The first treatment scenario used standard paretic leg muscle selection—tibialis anterior (TibAnt) and medial gastrocnemius (GasMed)—with standard stimulation timing (**Table 2**, Treatment Optimization Problem 2.2). This optimal control problem assessed how stimulation amplitude for the standard muscles could affect propulsive force asymmetry. The main optimization parameters were stimulation amplitude for both muscles. Stimulation on-time ton for both muscles was constrained to be within ±0.05 s of the standard experimental on-time, stimulation duration was fixed to the experimental duration, and stimulation off-time toff was set to on-time plus duration.

#### FastFES Treatment Optimization With Standard Muscles and Optimized Timing

The second FastFES treatment scenario used standard muscle selection with optimized stimulation timing (**Table 2**, Treatment Optimization Problem 2.3). This optimal control problem assessed how altered stimulation timing for the standard muscles could affect propulsive force asymmetry. The problem formulation was the same as for the first FastFES treatment scenario except that bounds on stimulation on-time were eliminated so that stimulation amplitude, on-time, and duration for both muscles became the main optimization parameters.

#### FastFES Treatment Optimization With Optimized Muscles and Optimized Timing

The third FastFES treatment scenario used optimized muscle selection with optimized stimulation timing. To predict the outcome of this treatment scenario, we followed a two-step process: First, we predicted which two muscles to stimulate, and second, we predicted when and how much they should be stimulated. For the first step, we solved an optimal control problem that identified which two paretic leg muscles should be stimulated to achieve the maximum reduction in AP force asymmetry (**Table 2**, Treatment Optimization Problem 2.4). Muscles without EMG data were not candidates for stimulation, since their unstimulated activations were not known with certainty. Muscles that shared EMG data between multiple heads used shared stimulation properties, leaving 25 muscles for the selection process. The stimulation properties of these 25 muscles were defined by 75 adjustable parameters that accounted for stimulation amplitude A, on-time ton, and duration, which fixed stimulation off-time toff . A terminal constraint was added to force the optimization to select only two muscles. Since gradientbased optimizations require continuous functions, a continuous approximation to the number of stimulated muscles n was constructed as a function of the stimulation amplitude A<sup>i</sup> of each muscle:

$$n = \sum\_{i=1}^{25} \left( 1 - e^{-4Ai} \right) \tag{2}$$

This approximation was constrained to be less than or equal to two plus a small tolerance to account for muscles with very low stimulation amplitude.

For the second step, we solved another optimal control problem that optimized stimulation amplitude and timing for these two new muscles (**Table 2**, Treatment Optimization Problem 2.5). The problem formulation was identical to that of the second FastFES treatment scenario except that the two stimulated muscles were changed, and thus stimulation amplitude, on-time, and duration for both muscles were again the main optimization parameters. This optimization was formulated to investigate whether stimulation of two different muscles in place of the standard ones might be a better choice for this particular patient.

#### RESULTS

### Neuromusculoskeletal Model Personalization

The neuromusculoskeletal model personalization process successfully calibrated model parameter values to closely reproduce the subject's marker motion, joint motion, joint moment, ground reaction, and muscle activation data. EMG-driven joint moments from muscle-tendon model personalization matched inverse dynamic joint moments with root-mean-square errors (RMSE) ranging from 2.5 to 6.4 Nm and mean absolute errors (MAE) between 2.0 and 4.7 Nm (**Table 3**). Ground reaction forces and moments from ground contact model personalization reproduced experimental ground reactions with RMS errors below 2.1 N for forces and 2.9 Nm for moments (**Table 4**). The verification optimal control problem using the complete personalized model produced a dynamically consistent full-body walking motion that matched lower body inverse kinematic joint angles to within 3.2 deg RMSE and 2.2 deg MAE, lower body inverse dynamic joint moments to within 4.8 Nm RMSE and 3.4 Nm MAE, measured ground reaction forces to within 31 N RMSE and 17 N MAE, and calibrated muscle activations to within 0.05 RMSE and 0.04 MAE (**Figure 2**; **Table 5**).

#### FastFES Treatment Optimization

The three FastFES treatment optimizations predicted progressively lower AP force asymmetry relative to the baseline



*Errors represent the difference between inverse dynamic joint moments calculated by OpenSim and net joint moments calculated by the calibrated EMG-driven model. Quantities represent averages between the two legs for 80 gait cycles (40 per leg) used in the model personalization process.*

optimization with no muscle stimulation (**Table 6**) along with visible changes in muscle activation patterns relative to baseline (**Figure 3**). Stimulation of standard muscles with standard timing decreased the difference in AP force impulse between the two legs by 41% relative to baseline, stimulation of standard muscles with optimal timing produced a 45% decrease relative to baseline, and stimulation of optimal muscles with optimal timing yielded a 64% decrease. When stimulation timing was allowed to change for standard muscle selection, TibAnt stimulation amplitude and timing remained relatively unchanged, while GasMed stimulation amplitude was decreased by 66% and stimulation duration was increased by 600% to cover a much larger portion of stance phase with greater similarity to healthy stimulation timing (**Table 7**). When the two stimulated muscles were allowed to change, the preliminary optimization selected soleus (Sol) and semimembranosus (Semimem) as the best two muscles to stimulate, and the subsequent treatment optimization predicted unique stimulation timings present only during stance phase (**Table 7**).

Predicted reductions in AP force asymmetry were accompanied by notable changes in predicted propulsive as

TABLE 4 | Root-mean-square error (RMSE), mean absolute error (MAE), maximum absolute error (MaxAE), and range in ground reaction forces and moments over the gait cycle from foot-ground contact model personalization.


*Errors represent the difference between ground reactions measured experimentally and ground reactions calculated by calibrated two-segment foot-ground contact models. Quantities represent averages between the two legs for representative gait cycle.*

well as braking force (**Figure 4**; **Table 8**). Experimentally, more propulsive force (positive peak and impulse) and less braking force (negative peak and impulse) were present on the non-paretic (left) side than on the paretic (right) side. Each of the three FastFES treatment optimizations decreased propulsive force and increased braking force on the non-paretic

TABLE 5 | Root-mean-square error (RMSE), mean absolute error (MAE), maximum absolute error (MaxAE), and range in joint angles, joint moments, ground reaction forces, and muscle activations from the verification optimal control problem.


*Errors represent the difference between quantities measured experimentally or calculated from experimental data and quantities predicted by the verification problem. None of the quantities included in this table was tracked in the verification cost function. Quantities represent averages between the two legs for the representative gait cycle.*

side while simultaneously increasing propulsive force and decreasing braking force on the paretic side. The one exception was standard muscles with optimal timing, which predicted a decreased propulsive force peak with an increased propulsive force impulse on the paretic side. Overall, the extent of predicted propulsive force changes tended to increase with each subsequent treatment optimization, though this general trend was not strictly followed. The AP force profile was the most similar between the two sides for optimal muscle selection with optimal stimulation timing.

While the predicted joint angles for the four treatment optimizations were similar to the experimental gait motion, some differences were still evident (**Figure 5**, see animations in **Supplementary Material**). For paretic leg hip flexion relative to the experimental trajectory, all four optimizations predicted

TABLE 6 | Difference in anterior-posterior (AP) force impulse between the two legs for the baseline optimization with no muscle stimulation and the three FastFES treatment optimizations, along with percent reduction in AP force impulse difference relative to baseline.


a decrease over most of stance phase, an increase during the first half of swing phase, and a decrease during the second half of swing phase. These changes made the paretic leg hip flexion trajectories more similar to the non-paretic leg hip flexion trajectories. For paretic leg knee flexion, the four optimizations predicted an increase over most of stance phase and a decrease at the end of swing phase. The increase during stance phase was most pronounced for the optimization that used optimal muscle selection. The decrease at the end of swing phase made the paretic leg knee flexion trajectories more similar to the non-paretic leg knee flexion trajectories. Maximum paretic leg knee flexion just after toe off increased only for the two optimizations that used standard muscle selection, and even then, the increases did not approach the corresponding peak values on the non-paretic side. For paretic leg ankle dorsiflexion,

#### TABLE 7 | Muscle stimulation parameters found by FastFES treatment optimizations.


*Muscle name abbreviations are defined in Table 1. A is stimulation amplitude, ton is stimulation on-time as percent of gait cycle, and toff is stimulation off-time as percent of gait cycle. Off-time is less than on-time if the stimulation wrapped around the end of the gait cycle to the start of the same gait cycle.*

FIGURE 3 | Experimental and predicted activation patterns for electrically stimulated muscles. Activation patterns for standard muscle selection involving stimulation of GasMed and TibAnt (top row) and optimal muscle selection involving stimulation of Sol and Semimem (bottom row) are presented for the paretic leg. Exp indicates experimental curves, Base indicates curves from the baseline treatment optimization with no muscle stimulation, Std/Std indicates curves from the FastFES treatment optimization using standard muscle selection with standard stimulation timing, Std/Opt indicates curves from the FastFES treatment optimization using standard muscle selection with optimized stimulation timing, and Opt/Opt indicates curves from the FastFES treatment optimization using optimized muscle selection with optimized stimulation timing.

FIGURE 4 | Experimental and predicted ground reaction forces over the gait cycle. Normal ground reaction force (top row) and propulsive ground reaction force (bottom row) are presented for the non-paretic leg (left column) and the paretic leg (right column). Thin vertical lines indicate locations of heel strike and toe off. Exp indicates experimental curves, Base indicates curves from the baseline treatment optimization with no muscle stimulation, Std/Std indicates curves from the FastFES treatment optimization using standard muscle selection with standard stimulation timing, Std/Opt indicates curves from the FastFES treatment optimization using standard muscle selection with optimized stimulation timing, and Opt/Opt indicates curves from the FastFES treatment optimization using optimized muscle selection with optimized stimulation timing. Note that the non-paretic leg is the left leg while the paretic leg is the right leg.



all four optimizations predicted an increase over the first half of stance phase. For paretic leg ankle inversion, the two optimizations that used standard muscle selection predicted an increase over the entire gait cycle, while the optimization that used optimal muscle selection predicted a decrease over most of stance phase. The same optimization predicted an increase in non-paretic leg ankle inversion over much of the gait cycle.

Predicted lower body joint moments also exhibited notable changes in response to the different simulated stimulation conditions (**Figure 6**). For all four optimizations, the hip extension and abduction moments were similar to experimental trajectories, with the largest deviation being an increased paretic leg hip abduction moment in the middle of stance phase. The knee extension moment on the paretic side showed little change from the experimental trajectory except for the optimization that used optimal muscle selection, which predicted a decreased knee extension moment at the start of stance phase and an increased moment from the middle to the end of stance phase. On the non-paretic side, the knee extension moment increased at the end of stance phase for all four optimizations. The paretic ankle plantarflexion moment exhibited the most prominent joint moment changes, with all four optimizations predicting an increase over stance phase relative to the experimental trajectory. The optimization that used optimal muscle selection predicted the largest increase, with the peak value reaching the corresponding peak on the non-paretic side. The paretic ankle eversion moment also exhibited prominent changes. The two

optimizations that used standard muscle selection predicted a decreased ankle eversion moment over most of stance phase, while the optimization that used optimal muscle selection predicted an increase over the same region.

### DISCUSSION

This study used a personalized neuromusculoskeletal walking model coupled with direct collocation optimal control to predict how FastFES treatments should be implemented to maximize propulsive force symmetry for an individual poststroke who was a non-responder to the standard FastFES training protocol. Though FastFES is a promising treatment for post-stroke gait neurorehabilitation, methods for customizing FastFES prescriptions to the unique needs of individual patients have yet to be developed. Using treadmill gait data collected from a non-responder to FastFES training, we personalized a full-body neuromusculoskeletal model and then used it to predict improvement in the subject's AP force symmetry for three FastFES treatment scenarios: (1) Standard muscle

the paretic leg (right column). The legend is the same as in Figure 2.

selection with standard stimulation timing, (2) Standard muscle selection with optimized stimulation timing, and (3) Optimized muscle selection with optimized stimulation timing. Overall, the more flexibility a FastFES treatment optimization was given, the more the subject's predicted AP force asymmetry was reduced. Our results suggest that for this particular subject, (1) Stimulation of standard muscles (i.e., TibAnt and GasMed) with standard timing should produce an acute improvement in the subject's propulsive force symmetry between the two legs, (2) A comparable improvement in propulsive force symmetry could potentially be achieved for this subject by stimulating TibAnt with standard settings and GasMed with decreased amplitude but increased duration, (3) A larger improvement in the subject's propulsive force symmetry could potentially be achieved by stimulating Sol and Semimem in place of TibAnt and GasMed, and (4) Large improvements in propulsive force symmetry may not guarantee large improvements in joint motion symmetry. Thus, future optimal control studies should explore adding kinematic symmetry terms to the optimization cost function so that improvements in both types of symmetry can be predicted simultaneously. The methodology developed in this study therefore provides only a first step toward computational design of personalized FastFES prescriptions that are customized to the unique functional limitations of the patient.

Since our subject was the non-responder in a recent FastFES clinical study (Allen et al., 2018), an important question is why our treatment optimization using standard muscle selection with standard stimulation timing predicted a large improvement in propulsive force symmetry. This apparent inconsistency can be explained by considering the differences between these two situations. During training in the laboratory with stimulation (the situation predicted by the model), AP force symmetry is improved due to an acute response to the electrical stimulation, often termed an orthotic effect. This orthotic effect demonstrates that the FES is able to augment force generation in the ankle muscles and generate greater paretic leg propulsion while the FES is on. In contrast, after multiple sessions of FastFES training, when gait performance is evaluated in the community or measured in the lab without stimulation, AP force symmetry is determined by the therapeutic or longterm retention effect of the treatment. The therapeutic effect may be influenced by multiple factors including the magnitude of favorable neuroplasticity induced through repeated training as well as improvements in muscle strength, cardiovascular endurance, and psychosocial factors. EMG data published in the previous FastFES clinical study (Allen et al., 2018) indicates that our subject exhibited statistically significant increases in unstimulated Sol and TibAnt activity pre- to post-training. However, increases in Sol activity were small in magnitude and thus potentially insignificant functionally, while the increase in TibAnt activity was in stance phase rather than swing phase as desired. Thus, our results suggest that for this subject, the level of acute improvement obtained due to stimulation during training may have been larger than the level of long-term improvement obtained due to neuroplasticity after the completion of training. The fact that this subject improved AP force symmetry during stimulation but did not retain the improvement afterward when tested without stimulation explains in part why he was classified as a non-responder to the intervention. A challenge for the future is finding a way to predict reliably which muscle excitations are the most amenable to long-term traininginduced neuroplasticity.

Three additional considerations may help explain this apparent inconsistency further. First, the subject did improve peak paretic propulsive force following FastFES training (Allen et al., 2018), but he started with extremely low peak propulsive force (as seen in **Figure 4**) and achieved only a small improvement following training. Second, our optimal control problems quantified improvements in propulsive force symmetry using the integral of AP force over the gait cycle, which accounts for not only peak propulsive force but also peak braking force using a mathematical function that is continuous and differentiable. In the clinical study, the change in the subject's peak paretic braking force was not reported, though our optimal control predictions suggest that standard muscle selection may not reduce peak paretic braking force as substantially as does optimal muscle selection (**Figure 4**; **Table 8**). Third, it is possible that the subject did not try to minimize propulsive force asymmetry when relearning to walk. A potentially insightful experiment would be to provide the subject with real-time feedback of his propulsive force asymmetry and instruct him to attempt to minimize it, similar to recent studies performed on healthy individuals (Schenck and Kesar, 2017) and individuals post-stroke (Genthe et al., 2018). Such an experiment could elucidate whether the subject's propulsive force asymmetry is primarily due to neural control limitations, biomechanical constraints, or a subconscious decision to optimize other quantities (e.g., metabolic cost Zarrugh et al., 1974; Bertram, 2005).

Our optimal control predictions suggest two alternate FastFES protocols that could potentially benefit this subject. The first alternate protocol would decrease GasMed's stimulation amplitude while prolonging its stimulation duration. This change is consistent with GasMed being stimulated to increase latestance paretic propulsion, whereas TibAnt is stimulated to prevent foot drop during swing phase (Hakansson et al., 2011; Kesar et al., 2011). This protocol has the potential benefits of reducing GasMed fatigue and stimulation discomfort while also reducing the sensitivity of the resulting motion to the selected stimulation on-time and off-time. The second alternate protocol would replace GasMed and TibAnt stimulation with Sol and Semimem stimulation. This protocol has never been investigated, so it is unknown whether stimulation of these alternate muscles would facilitate or hinder the subject's longterm neuroplasticity and motor learning. A potential benefit of the Sol and Semimem stimulation protocol is that it may produce a large decrease in braking force peak and impulse for the paretic leg (see **Figure 4**; **Table 8**). For any FastFES protocol change, implementation of predicted stimulation amplitudes would be a challenge. Some method would be needed to calibrate the relationship between model-predicted and experimentallyapplied stimulation amplitude (Kesar et al., 2008; Perumal et al., 2008). However, if the predicted relative stimulation amplitude between the two stimulated muscles was reliable, then it could be possible to constrain the two stimulation amplitudes to maintain the desired ratio. With this approach, only a single stimulation amplitude would need to be manipulated experimentally when exploring subject-specific stimulation settings, realizing that the maximum achievable stimulation will depend on the subject's tolerance of the discomfort caused by electrical stimulation.

It is interesting to consider whether the two muscles (Sol and Semimem) selected by our third treatment optimization would be logical choices based on interpretation of their biomechanical roles. While paretic (right) propulsive force generation was clearly inhibited in the subject's experimental gait pattern, braking force was also larger on the paretic side than on the non-paretic side, likely due to poor coordination. Thus, muscle stimulation that acts to decrease early-stance braking and/or increase late-stance propulsion would improve propulsion symmetry in this subject. Indeed, the optimization that selected the best two muscles to stimulate chose one muscle to increase propulsive force in late stance (Sol) and another muscle to decrease braking force in early stance (Semimem), consistent with minimization of AP force impulse asymmetry. Several studies have reported that Sol and GasMed contribute to forward acceleration of the trunk in mid to late stance (Neptune et al., 2001; Liu et al., 2006) and that both contribute to propulsive ground reaction force in late stance (Neptune et al., 2004; Allen and Neptune, 2012). Thus, selection of Sol as a replacement for GasMed is not surprising. Although stimulating Sol can improve propulsive force in late-stance, stimulation of Sol during mid-stance may actually contribute to increased braking forces (Neptune et al., 2004). In contrast, published studies have also reported that the hamstrings contribute to propulsive ground reaction force in early to mid-stance (Neptune et al., 2004; Allen and Neptune, 2012), potentially explaining the selection of Semimem to counteract increased braking from the Sol. However, these choices also resulted in increased knee flexion throughout most of stance phase, which may not be desirable from either a metabolic perspective or an aesthetic perspective. Finally, despite elimination of TibAnt stimulation, our optimal treatment still predicted that the paretic toe would clear the ground, potentially through increased knee flexion at the start of swing phase (**Figure 5**).

Though muscle synergy analysis is often used to quantify control complexity and inter-muscle coupling in experimentally measured EMG signals, our study used muscle synergy concepts for broader control-related reasons. First, we used a low-dimensional set of synergy activations rather than 36 independent muscle activations to control each leg since synergy activations have been shown to generate more accurate predictions of walking under new conditions (Meyer et al., 2016). Second, we used synergy rather than muscle activation controls to simplify the model's control structure, which significantly improves computational speed and convergence of optimal control walking predictions (Meyer et al., 2016). Third, we used synergy activation controls so that missing muscle activations could be predicted as linear combinations of the synergy activations extracted from muscle activations with associated EMG measurements (Bianco et al., 2017).

While our choice of five synergies to control each leg was based on achieving at least 95% VAF for each individual muscle activation in both legs, this choice was informed by three additional considerations. First, to closely match all available experimental data (i.e., joint angles, ground reactions, and muscle activations), more synergies are required than indicated by synergy analysis of muscle activation data alone. Because our walking predictions are dynamically consistent, the muscle activations controlling the model must be of high enough fidelity to reproduce the subject's experimental data closely. Based on our previous work (Meyer et al., 2016), the number of synergies found by synergy analysis of EMG data alone may not be enough to produce a simulated walking motion that tracks experimental data as closely as desired. The reason is that matching joint motion, ground reaction, and EMG data simultaneously with muscle synergy controls is a much more constrained situation than matching only EMG data with muscle synergy controls, thereby necessitating a larger number of synergies than would have been retained otherwise. Second, to predict missing muscle activations using synergy activations extracted from muscles with EMG data, more synergies are required than indicated by synergy analysis of EMG data alone (Bianco et al., 2017). As noted earlier, 95% total VAF does not guarantee a comparably high individual muscle %VAF. To minimize the risk of poor construction of missing activations, we chose the number of synergies so that all measured muscle activations were reconstructed with at least 95% VAF. Use of more than 5 synergies would likely not improve our ability to fit measured activations or predict missing activations, while use of fewer than 5 synergies would produce poorer fitting of some measured activations and likely poorer prediction of missing activations.

To evaluate theoretically whether stimulation of our two predicted muscles (Sol and Semimem) might be more effective for this subject than stimulation of the two standard muscles (GasMed and TibAnt), we examined the structure of the subject's synergy vectors (SVs) for 2 and 3 paretic leg synergies (**Figure 7**). We chose these low numbers of synergies since they achieved 95% total VAF for experimental muscle activations from both legs and provide the simplest perspective for interpretation purposes. For the 2-synergy solution, paretic GasMed did not appear predominantly in either SV, while paretic TibAnt appeared with moderate weight in the first SV. Thus, if training does not alter the composition of the SVs, then increased recruitment of GasMed may produce large unwarned increases in the recruitment of other muscles. In contrast, paretic Sol possessed the largest weight in the second SV, and paretic Semimem possessed a moderate weight in the same SV. Thus, if stimulating paretic Sol and Semimem during FastFES training resulted in enhanced recruitment of the second synergy, the activations of both muscles would increase together, which would be undesirable since no overlap exists in the predicted optimal stimulation timing for these two muscles. Not surprisingly, this interpretation changes for the 3-synergy solution, where paretic Sol and Semimem appear prominently in two different SVs the second synergy for Sol and the third synergy for Semimem. Since the optimal stimulation timing for these two muscles is non-overlapping, decoupling the recruitment of these two muscles could be beneficial. Increased recruitment of the second and third synergies would also increase recruitment of other muscles (e.g., gluteus maximus, peroneus longus, tensor fascia latae, and vastus medialis), which may or may not be beneficial. In contrast, neither GasMed nor TibAnt appears prominently in any of the three SVs, suggesting that it could be difficult to increase the activation of these muscles without creating undesirable activation increases in other muscles. If the number of synergies is increased further, GasMed and TibAnt remain weakly represented in the SVs, while Sol and Semimem become more dominant in separate SVs, which is consistent with the idea that the activation of these two muscles could potentially be trained independently. Thus, analysis of the subject's muscle synergies at least hints at the possibility that the two new muscles selected by the optimization may be worth considering for stimulation.

Interestingly, the three FastFES treatment optimizations predicted comparable values for peak paretic propulsive force during stance phase (**Figure 4**) as well as peak paretic knee flexion

during swing phase (**Figure 5**). In both cases, these peak values were substantially lower than the corresponding peak values predicted for the non-paretic leg. For our subject, a low value of peak paretic propulsive force resulted in a slow self-selected walking speed, while a low value of peak paretic knee flexion necessitated a compensatory hip hiking strategy to ensure that the paretic foot cleared the ground during swing phase (Chen et al., 2005). Since AP force asymmetry was minimized in the cost function, these observations suggest that the personalized model hit a "ceiling" on the increase in paretic propulsive force achievable using electrical stimulation of only two muscles. While the model may have also hit a "ceiling" on the achievable increase in paretic knee flexion, this conclusion is less clear since a knee flexion asymmetry term was not included in the cost function. If such a term were added, the resulting increase in paretic knee flexion would likely be accompanied by a corresponding decrease in paretic propulsive force. To improve peak paretic propulsive force and peak paretic knee flexion to the desired levels simultaneously, the optimal control problem would need to change the subject's paretic leg muscle synergies. A future optimal control study could therefore explore finding the smallest changes to a single paretic leg synergy activation that would bring the subject's peak paretic propulsive force and peak paretic knee flexion as close as possible to their desired levels. The predicted synergy changes could then inform complementary neurorehabilitation efforts or even electrical stimulation of the spinal cord to recruit the identified synergy (Wenger et al., 2016).

Our study possesses several important limitations that inform interpretation of our current results and suggest directions for future investigation. First, only a single subject was studied. We specifically selected a non-responder to FastFES training so as to maximize our chances of identifying alternate stimulation protocols that could potentially improve the subject's walking ability. Whether or not the same modeling approach would work for other subjects will require further investigation. Second, no measurements were available for the forces and torques exerted by the handlebar on each hand, plus the hand loads estimated during the model personalization process were applied to the model during the treatment optimization process. Experimental measurement of hand loads would greatly simplify model personalization. Allowing the hand loads to vary during treatment optimization could alter our propulsive force predictions. Third, experimental stimulation does not target individual muscles as directly as modeled in this study. In practice, medial and lateral gastrocnemius are often stimulated together by a single electrode. Even if GasMed and TibAnt were well-targeted for electrical stimulation, some stimulation would likely "bleed" into other plantarflexor and dorsiflexor muscles. Modeling of this "bleeding" phenomenon would impact our treatment predictions, though the extent to which the predictions would be changed is unknown. Fourth, no measure of kinematic asymmetry was included in the optimal control cost function. When initiating this study, we expected that improved propulsive force symmetry would naturally result in improved joint motion symmetry. This expectation proved to be incorrect, suggesting that some measure of kinematic asymmetry should be included in future optimal control studies of FastFES treatment design. However, no published study to date has presented an optimal control problem formulation that is capable of turning an asymmetric walking motion into a symmetric one. Thus, development of a problem formulation that enforces kinematic symmetry remains an important challenge for the neuromusculoskeletal modeling research community. Fifth, simulated electrical stimulation was explored for only two paretic leg muscles due to current technical limitations in the FastFES hardware and software. If stimulation of three or even four muscles was investigated, it is possible that substantial improvements in both propulsive force symmetry and kinematic symmetry could be predicted. However, stimulation of more than two muscles is not feasible with the electrical stimulation system used in this study. Sixth, our FastFES treatment optimizations assumed that the subject's neural control strategy remained largely unchanged by the application of FES. In reality, if a subject responds to FastFES training, one would expect his or her neural control strategy to change over the course of treatment as favorable neuroplasticity occurred (Allen et al., 2018). Modeling how a patient's neural controlstrategy changes over time with training would likely alter our treatment predictions.

Another important limitation of this study was the amount of time and effort required to perform the entire sequence of optimizations used for model personalization and treatment optimization. Generation of the results reported in this study required over 2 years of effort by a single Ph.D. student. Once the entire process was set up for a new subject performing a new task (walking with hands on handlebars), all model personalization optimizations could be completed in roughly 5 h of CPU time, while each treatment optimization required between 3 and 15 min of CPU time. Thus, the primary bottleneck was not computation time but rather the time required to learn the entire computational workflow, process the experimental data to get them into the correct format, and identify appropriate optimization problem formulations to get each step to work properly the first time. In the present study, the most challenging problem formulation issues were related to how to minimize propulsive force asymmetry, model the hands grasping the handlebar, and predict the best two muscles to stimulate. None of these issues had been explored in previous optimal control studies of human walking, and all of them required running hundreds of optimal control problems before appropriate problem formulations could be identified. We are continuing to refine our model personalization and treatment optimization workflow so that the steep learning curve currently required to become proficient in the entire process can be eliminated as a bottleneck.

One of the biggest limitations of this study was our inability to evaluate experimentally our optimal FastFES treatment prediction. Such an evaluation would have required completion of an additional training study for this subject, which unfortunately we were unable to perform. We hope to be able to apply our optimal treatment prediction to this subject, as well as explore the use of model-based optimal treatment predictions for other subjects, as part of a future FastFES training study.

In conclusion, this study explored the feasibility of using subject-specific neuromusculoskeletal models combined with direct collocation optimal control to predict novel FastFES treatment prescriptions that may improve a specified treatment target—in this case, inter-leg propulsive force symmetry. The ability to tailor neurorehabilitation treatments to the unique needs of individual patients would be an important next step in modern healthcare. In the case of stroke, walking deficits vary widely from patient to patient, highlighting the need for an objective and effective treatment customization process. In the current study, a computational approach to FastFES treatment customization predicted that changing the stimulation amplitude and timing of a typically stimulated muscle, or changing which two muscles are stimulated, may improve a specific subject's paretic propulsion significantly. While this computational approach is still too difficult and time consuming to be feasible on a large scale, future improvements in computational methodology and technology may eventually make it possible to perform this approach on a routine basis, potentially allowing treatment decisions to be based on objective predictions of a patient's posttreatment function.

#### DATA AVAILABILITY STATEMENT

The Matlab code and experimental data used to perform this study can be found on Simtk.org at https://simtk.org/projects/ predictfastfes.

#### ETHICS STATEMENT

The human subject was recruited at Emory University and gave informed written consent. The study was approved by the institutional review boards of Emory University where the data were collected and the University of Florida where the data were modeled.

### AUTHOR CONTRIBUTIONS

NS performed all model personalization and treatment optimization tasks, wrote the first draft of the manuscript, and assisted with preparation of figures and tables. AM assisted with development of the personalized neuromusculoskeletal model, modification of the optimal control problem formulations to accommodate the goals of the present study, and review of the manuscript draft. JA oversaw experimental data collection for the project, processed all experimental data, and provided feedback on the manuscript draft. LT and TK recruited the

### REFERENCES


experimental subject, organized the experimental data collection session, helped identify treatment scenarios to optimize, and provided feedback on the manuscript draft. BF coordinated the entire project, oversaw model personalization and treatment optimization tasks, edited the manuscript draft, and prepared the final figures and tables.

## FUNDING

This study was funded by National Science Foundation grant CBET 1404767 and by a National Center for Simulation in Rehabilitation Research pilot project award provided through NIH grant P2C HD065690.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnbot. 2019.00080/full#supplementary-material

Supplementary Video 1 | OpenSim animations comparing experimental walking motion (translucent skeleton) with predicted walking motions (opaque skeletons) for baseline treatment optimization.

Supplementary Video 2 | FastFES treatment optimizations using standard muscle selection with standard stimulation timing.

Supplementary Video 3 | Standard muscle selection with optimized stimulation timing.

Supplementary Video 4 | Optimized muscle selection with optimized stimulation timing.


**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Sauder, Meyer, Allen, Ting, Kesar and Fregly. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Muscle Contributions to Upper-Extremity Movement and Work From a Musculoskeletal Model of the Human Shoulder

Ajay Seth<sup>1</sup> \*, Meilin Dong<sup>1</sup> , Ricardo Matias 2,3 and Scott Delp<sup>1</sup>

<sup>1</sup> Neuromuscular Biomechanics Lab, Bioengineering and Mechanical Engineering Departments, Stanford University, Stanford, CA, United States, <sup>2</sup> Champalimaud Research and Clinical Centre, Champalimaud Centre for the Unknown, Lisbon, Portugal, <sup>3</sup> Human Movement Analysis Lab, Escola Superior Saúde—Instituto Politécnico de Setúbal, Setúbal, Portugal

Musculoskeletal models enable movement scientists to examine muscle function by computing the mechanical work done by muscles during motor tasks. To estimate muscle work accurately requires a model that is physiologically plausible. Previous models of the human shoulder have coupled scapula movement to humeral movement. While coupled movement produces a stereotypical scapulohumeral rhythm, it cannot model shrugging or independent movement of the scapula and humerus. The artificial coupling of humeral elevation to scapular rotation permits muscles that cross the glenohumeral joint, such as the rotator-cuff muscles and deltoids, to do implausible work to elevate and rotate the scapula. In reality, the motion of the scapula is controlled by thoracoscapular muscles, yet the roles of these muscles in shoulder function remains unclear. To elucidate the roles of the thoracoscapular muscles, we developed a shoulder model with an accurate scapulothoracic joint and includes scapular muscles to drive its motion. We used the model to compute the work done by the thoracoscapular muscles during shrugging and arm elevation. We found that the bulk of the work done in upper-extremity tasks is performed by the largest muscles of the shoulder: trapezius, deltoids, pectoralis major, and serratus-anterior. Trapezius and serratus anterior prove to be important synergists in performing upward-rotation of the scapula. We show that the large thoracoscapular muscles do more work than glenohumeral muscles during arm-elevation tasks. The model, experimental data and simulation results are freely available on SimTK.org to enable anyone to explore our results and to perform further studies in OpenSim 4.0.

Keywords: computational shoulder model, scapula mechanics, thoracoscapular muscle work, serratus anterior, trapezius, deltoids, rotator-cuff muscles

## INTRODUCTION

Abnormal scapular movement is indicative of shoulder dysfunction, such as subacromial impingement, rotator-cuff tears, and other injuries (Struyf et al., 2011). A symptom of shoulder dysfunction is scapular dyskinesia (Kibler et al., 2013), including scapular winging (Martin and Fish, 2008), in which the medial border of the scapula lifts off the thoracic surface. Before

#### Edited by:

Benjamin J. Fregly, Rice University, United States

#### Reviewed by:

Lauranne Sins, OSSimTech, Canada Fabien Dal Maso, Université de Montréal, Canada

> \*Correspondence: Ajay Seth aseth@stanford.edu

Received: 02 May 2019 Accepted: 14 October 2019 Published: 05 November 2019

#### Citation:

Seth A, Dong M, Matias R and Delp S (2019) Muscle Contributions to Upper-Extremity Movement and Work From a Musculoskeletal Model of the Human Shoulder. Front. Neurorobot. 13:90. doi: 10.3389/fnbot.2019.00090

**100**

researchers can investigate shoulder dysfunctions, we require biomechanical models with the degrees of freedom and musculature attached to the scapula, which is currently unavailable.

Models designed to understand glenohumeral injury and rehabilitation (Garner and Pandy, 2001; Holzbaur et al., 2005; Dickerson et al., 2007; Chadwick et al., 2009; Bolsterlee et al., 2013; Saul et al., 2015) ignore muscle actions of the largest thoracoscapular muscles: trapezius, rhomboids, and serratus-anterior (Rockwood, 2009). These muscles likely play important roles in human upper-extremity movements given their size and force-generation capacity. While Odle et al. (2019) included the rhomboids and serratus-anterior muscles in their model, they maintained the scapulohumeral coupling from the model reported by Saul et al. (2015), which does not need thoracoscapular muscles to move. We can only assume that coupling scapular kinematics to humeral rotation yields the perplexing results that the rotator-cuff muscles generate the largest forces during the recovery phase of wheel-chair propulsion, while the larger superior trapezius, rhomboids, anterior deltoid, and pectoralis major muscles produced virtually no force throughout the movement (Odle et al., 2019).

The model by van der Helm (1994a), was the first to include thoracoscapular muscles and enable realistic scapula kinematics by including scapular contact with the thoracic surface. While numerous models (van der Helm, 1994b; Garner and Pandy, 2001; Dickerson et al., 2007; Dubowsky et al., 2008; Odle et al., 2019) have computed thoracoscapular muscle forces for a variety of upper-extremity tasks, the work performed by these muscles during these tasks was not reported.

We have developed a musculoskeletal model of the shoulder that includes the large thoracoscapular muscles and the kinematically uncoupled movement of the scapula so that we may answer two fundamental questions about upperextremity muscle function. First, how much work is done by the thoracoscapular and glenohumeral muscles during shoulder shrugging and arm-elevation tasks? Second, what motions of the scapula are controlled by large thoracoscapular muscles such as trapezius and serratus anterior during these shoulder tasks?

## METHODS

## Model of the Human Shoulder

We developed a model of the human shoulder in OpenSim (Delp et al., 2007; Seth et al., 2018) (**Figure 1**) that combines a fast and accurate skeletal model of scapulothoracic kinematics (Seth et al., 2016) with muscle paths and architecture based on (Klein Breteler et al., 1999). To reduce complexity and improve computational performance of the model, muscle bundles from van der Helm (1994a) were aggregated and their parameters combined (**Table 1**). Muscle paths including wrapping surfaces and their geometry were adjusted to produce moment arms bounded by measurements from cadaver experiments (Ackland et al., 2008). Continuity of muscle moment arms were verified over the full range-of-motion of the model.

Model scaling and inverse kinematics were performed in OpenSim to compute model joint angles from experimental marker data (see below). Bones and corresponding joint locations and muscle attachment locations were scaled linearly based on marker-based distances between the subject and the base (generic) model. Muscle optimal fiber and tendon slack lengths were scaled to preserve their ratio over the muscle path length in the scaled model. The ellipsoid surface of the thorax in the scapulothoracic joint was scaled by optimizing the ellipsoid tilt and radii that minimized marker-tracking errors. The thorax muscle wrapping object was initially scaled according to the thorax scale factors, however this lead to serratus anterior insertions on the anterior scapula to enter the wrapping surface, which results in the wrapping path becoming undefined. The


TABLE 1 | Thoracoscapular shoulder model muscle parameters adapted from Klein Breteler et al. (1999) with aggregated bundles from by van der Helm (1994a).

wrapping ellipsoid surface was then hand adjusted by tilting the top of the ellipsoid toward the sternum until the path of the serratus anterior was well-defined for the complete scapula range of motion across all tasks. The greater freedom of the scapula also resulted in some muscles exceeding 150% of the optimal fiber-length and/or being too short (<50%) resulting in their inability to produce active force during the range-ofmotion of the anticipated tasks. In these situations, the muscle optimal fiber-length was incrementally increased (by 2%) and tendon slack-length reduced by the same length until muscle forces alone were sufficient to track desired task kinematics. See **Table 1** for the complete set of muscle parameters implemented in the shoulder model.

Computed muscle control (CMC) (Thelen et al., 2003) was used to generate muscle-driven simulations that tracked joint angles from inverse kinematics. All simulations were performed using OpenSim 4.0 (Seth et al., 2018) on a desktop computer with an Intel i7 3930K 3.2GHz processor and 32GB of RAM. All computations were evaluated running on a single CPU core.

## Experimental Data Collection and Comparison Methods

To test the shoulder model, we collected upper-extremity kinematics using Ascension 3D trakSTAR (Ascension Technology Corp, USA) and Motion Monitor software (Innovative Sports Training, Chicago, Illinois) to simultaneously and continuously track four miniaturize sensors (model 800) at a sampling rate of 120 Hz. Three sensors were fixed to the thorax, scapula and humerus, respectively. Prior to continuous collection, a fourth sensor was rigidly affixed to a stylus and used to digitize the locations of bony landmarks with respect to the corresponding sensors, while the subject was in a neutral pose. The thorax sensor was placed on the T1 spinous process; the scapula sensor was placed over the flat surface on the superior


Lower values are faster. Computation times evaluated for inverse kinematics (IK), computed muscle control (CMC) and forward dynamic (FD) simulations.

+Indicates the task with a 2 kg hand-held mass.

acromion. Both sensors were held in place with double-sided adhesive tape wrapped with EnduraSports tape (Endura-Tape). The arm sensor was fixed on a strap that was tightly adjusted around the lateral aspect of the most distal part of the humerus. The ISB shoulder protocol (Wu et al., 2005) implemented in the MotionMonitor software was used to collect data based on the recorded sensor and digitized landmark locations (Ludewig et al., 2009) and identified as markers in OpenSim.

Surface electromyography (EMG) electrodes were placed on the skin after preparation (Basmajian and de Luca, 1985) according to Cram (2010) with an interelectrode distance of 20 mm over the: superior, middle and inferior trapezius; serratus anterior; anterior, middle, and posterior deltoids; infraspinatus; teres major; pectoralis major (clavicular), and latissimus dorsi muscles. A reference electrode was placed on the contralateral acromion. We collected three trials of shoulder shrugging, forward flexion and abduction without and with a 2 kg handheld weight, for a total of 18 trials from the dominant shoulder (right) of a 26-year-old healthy female subject (height: 162 cm, weight: 52 kg). The experimental protocol was approved by the ethics committee of the Polytechnic Institute of Setúbal.

We processed the raw EMG by high pass filtering at 100 Hz, full-wave rectifying the resultant signal, and then low-pass filtering at 4 Hz to obtain EMG envelopes according to ISEK (Merletti, 1999). Processed EMG envelopes were normalized by maximum voluntary contractions obtained according to (Kendall et al., 2005).

We compared muscle computed activations to processed EMG waveforms by computing the mean-absolute error (MAE) over the shoulder task interval (de Zee et al., 2007; Dubowsky et al., 2008; Odle et al., 2019) for each muscle across all tasks. For serratus anterior, the average activation of the three muscle bundles in the model was used in the comparison.

To understand the contribution of individual muscles to shoulder movement in our subject, we calculated the work done by muscles by integrating the positive muscle power during scapular and humeral elevation. Muscle power was computed from the product of muscle-tendon unit force (from CMC) and shortening velocity, where concentric contractions yield positive power. The total positive muscle work during the elevation phase of the tasks was compared to the external work computed as the change in model potential energy due to elevating the arm (and added mass) against gravity. We expected the positive muscle work to be greater than external work due to negative work of lengthening muscles and the acceleration of limb segments relative to the center-of-mass.

## RESULTS

We generated muscle-driven simulations for all (18) experimental trials. Inverse kinematics accuracy for each trial was within 1 cm RMSE with respect to experimental marker locations and computed within 1.3 × of real-time. The average computation to real-time ratio for all CMC muscle-driven simulations was below 400 compute/real time. **Table 2**, presents the compute to real-time ratio for simulating our model for each task. For comparison, we obtained a 4–17 × speedup when executing CMC with our model vs. the model by (Saul et al., 2015) for the flexion and abduction tasks.

Muscle activations from muscle-driven simulations of the shoulder model were compared to the EMG for the same tasks, which yielded an average MAE of 0.06, with the vast majority of measured muscles below 0.1 (**Table 3**). The Pectoralis major muscle showed the worst agreement during the shrugging task (without a handheld weight) where EMG was relatively silent in the depression phase, while the model estimated low but consistent activation throughout the movement (**Figure 2B**).

The simulated shoulder shrug demonstrates that the model can elevate and rotate the scapula independent of humerus rotations (**Figure 2A**). Simulated muscle activity during shrugging indicates that levator scapulae elevates the scapula while superior trapezius may both elevate and upward rotate the scapula during shrugging (**Figure 2B**).

The MAE values for superior trapezius, deltoids and serratus anterior muscle activations when compared to EMG during shoulder flexion and abduction tasks (**Figure 3**) where 0.1 or below (**Table 2**) indicative of a high quantitative correlation (Morrow et al., 2010; Odle et al., 2019) between simulated and subject muscle activity.

Superior trapezius, serratus anterior, and deltoids showed the greatest muscle activity and did the most positive work during the elevation phase of each task (**Figure 4**). As expected, the total positive muscle work was consistently greater than the external work. For example, the total positive muscle work of 61.6J exceeded the total external work (49.5J) necessary to elevate the arm during abduction with a 2 kg handheld weight.

## DISCUSSION

We have developed a musculoskeletal shoulder model that reproduces the observed skeletal kinematics and muscle activity during shoulder shrugging and arm-elevation tasks. The model enabled us to compute the work done by upper-extremity muscles that drive the scapula and the glenohumeral joint. Prior to this study, scapulothoracic interaction was modeled either by forces of deformation using finite elements (van der Helm, 1994a) or by contact point constraints (Garner and Pandy, 1999) making use of these models challenging. The inherent model stiffness due to large (muscle and scapulothoracic contact) forces and a


TABLE 3 | Mean absolute error between subject EMG and model muscle activations across tasks.

A value below 0.1 corresponds to <10% difference between two signals. Values > 0.1 are in bold. +Indicates the task with a 2kg hand-held mass.

low mass scapula body has required custom system dynamics and contact formulations and the use of implicit integration (Chadwick et al., 2014) that are not widely accessible to the clinical and rehabilitation communities. Available models that couple humeral elevation to scapular rotation (Saul et al., 2015; Odle et al., 2019) are unable to accurately account for the muscle work required to move the scapula and the subsequent upper-extremity. We show that a model can capture scapular kinematics and account for muscles that drive the scapula, without a detriment to computational performance. In fact, the model computes 4–17× faster than a comparable model (Saul et al., 2015) without these capabilities while allowing researchers to study the function of thoracoscapular muscles.

We simulated shrugging, flexion, and abduction tasks with/out a 2 kg hand held weight using our shoulder model. We found agreement between the simulated model and subject measurements with marker tracking within 1 cm RMSE, and model activation compared to subject EMG with an average MAE below 0.1 for the most active muscles during the tasks we examined. While we did not directly measure muscle forces or velocities, the agreement of model kinematics and muscle activity give us confidence that the muscle work computed by the model is representative of the relative work done by the subject's shoulder muscles. One of the main benefits of complimenting experimental measures with a computational model, is that we can estimate quantities that are difficult to measure such as muscle force and work.

Our first aim was to evaluate how much work is done by the thoracoscapular and glenohumeral muscles during shoulder shrugging and arm-elevation tasks? To address this aim, we computed the work done by individual shoulder muscles during the simulated shrugging, flexion and abduction tasks (**Figure 4**) using the shoulder model. We found (superior) trapezius, serratus anterior, and rhomboids (i.e., the large thoracoscapular muscles) combined to exceed the work of the deltoids, rotator-cuff, and teres major (i.e., the glenohumeral muscles). While deltoids were the largest muscle contributor to humeral elevation during flexion tasks, trapezius and serratus anterior combined to do more work than deltoids for every task including flexion.

FIGURE 3 | Shoulder model muscle activations for primary muscles used to elevate the humerus during the (A) flexion and (B) abduction tasks with a 2 kg hand-held mass. Simulated muscle activations (red, ±1 SD shaded) compared to EMG (gray shaded).

performed <3% of the total muscle work.

Our second aim was to answer what motions of the scapula are controlled by large thoracoscapular muscles such as trapezius and serratus anterior during these shoulder tasks? We addressed this question by analyzing which thoracoscapular muscles perform work on the scapula during shoulder tasks. Our results show that levator scapulae elevates the scapula while trapezius and serratus anterior upward rotate the scapula during shrugging. As work demands increase due to a handheld weight, we found that superior trapezius and serratus anterior work together to form a powerful force-couple to upward-rotate the scapula during armelevation tasks. These results confirm the function of superior trapezius and serratus anterior muscles as described by anatomy textbooks (Stranding, 2016).

The implications of these results for human rehabilitation and neurorehabilitative robotics is significant. Examining the functional roles of the major upper-extremity muscles is key to understanding which muscles to assist and when to apply assistance.

The shoulder model provides unique opportunities to design and test rehabilitative strategies directly in a physics and physiologically consistent way. In the same way that simulation was used to test ideal-assistance in human running (Uchida et al., 2016) it can be applied to explore upper-extremity assistance strategies that enable a weakened model to reach target locations that minimizes device weight and power use. We can use the model to discover principles for upper-extremity assistance that enable patients to perform independently and effectively.

In patients with shoulder pathologies, for example due to brachial nerve palsy, the model enables us to test hypotheses about the causes and cures for scapula dyskinesia. There is mounting evidence that altered scapula kinematics is indicative of shoulder pathologies (Ludewig and Reynolds, 2009; Kibler et al., 2013), and scapula-focused treatments improve outcomes in patients with shoulder disorders (Struyf et al., 2013; Hotta et al., 2018). The biomechanics underlying these improvements, however, are poorly understood. Therefore, clinicians require both reliable measurements and accurate models to examine how muscles cause both healthy and pathological movements. We have shown that the thoracoscapular muscles play a major role in healthy upper-extremity movements.

While these results are promising, the shoulder model has its limitations. First, we presented comparisons for tasks performed by a single healthy subject. The inherent variability amongst individuals and particularly patients with varying pathologies calls for much more comprehensive testing. Second, scaling the model and particularly thoracic muscle paths was an arduous and time-consuming task. In some muscles, such as the rhomboids, the range of motion of the scapula resulted in fibers either being too short or too long to generate sufficient active force. In these cases, we had to increase optimal fiber length and to reduce tendon slack length for the muscles to generate force over the full range of motion. There is considerable work to be done to automate the scaling of the scapulothoracic joint and associated muscle paths and parameters. Third, the glenohumeral joint was modeled as a ball-and-socket joint, thereby ensuring the stability of the joint and reducing the need for rotatorcuff muscles. Nonetheless, rotator-cuff muscle forces required for joint stability (Cain et al., 1987; Lippitt and Matsen, 1993) are not expected to contribute significantly to the total muscle work reported in this study because: i) their contribution to reaction forces increases, but reactions do not perform work, and ii) their elevation/abduction moment-arms are small (Yanagawa et al., 2008). We recognize that stability of the glenohumeral joint remains necessary to accurately estimate rotator-cuff forces and glenohumeral reaction forces (Ameln et al., 2019).

### CONCLUSIONS

Diagnosing, treating and augmenting human performance requires deep understanding of the function of muscular and skeletal structures that produce healthy and pathological movements. The activity and work done by individual muscles provides insight into the actions of muscles. Since the pioneering model and analysis of the shoulder mechanism (van der Helm, 1994b), there has been little reported about shoulder muscle forces and work to move the scapula and the arm. We developed a model that includes both the musculature and degrees-of-freedom of the human shoulder, which we combined with experimental data to compute the work done by large thoracoscapular muscles. We showed that of these muscles, the trapezius and serratus anterior muscles combine to do the majority of the work of upward rotating the scapula and elevating the arm.

The shoulder model and simulation environment (OpenSim) are provided freely from SimTK.org (https://simtk.org/projects/ thoracoscapular). The model runs natively in OpenSim without third party dependencies. Clinicians, researchers and students can probe the model for muscle and joint reaction forces from the analysis of subject and patient motion capture data as we have demonstrated. The capability of running the model in a purely forward dynamics simulation also makes the model suitable to ask "what if?" questions. For example, in the case that serratus anterior is weakened, can external bracing prevent winging? If so, why might bracing outcomes vary widely (e.g., Vastamäki et al., 2015)? Or, can the model elevate the arm if serratus anterior is incapacitated? If not, what rehabilitation strategy or assistive device can support the role of serratus anterior to enable arm elevation? These and other questions can now be explored with our model.

## DATA AVAILABILITY STATEMENT

The model and simulation environment (OpenSim) are freely available, deployable, and modifiable for any research or commercial use without restrictions from SimTK.org at https:// simtk.org/projects/thoracoscapular and https://simtk.org/ projects/opensim, respectively. The shoulder model does not require additional third party environments or software. Scripts for batch processing the analyses in this study are provided as MATLAB files.

## ETHICS STATEMENT

This study was carried out in accordance with the recommendations of the Ethics Committee of the Polytechnic Institute of Setúbal with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the Ethics Committee of the Polytechnic Institute of Setúbal.

## AUTHOR CONTRIBUTIONS

AS and RM conceived of the shoulder model and study aims. AS supervised and performed data analysis and wrote first draft of the manuscript. MD extensively refined the muscle paths, performed the analyses, and generated the reported results. RM initiated the addition of thoracoscapular muscles and led the data collection effort. SD supported the study, contributed to study aims, and edited the manuscript.

## FUNDING

This work was supported by the National Institutes of Health (https://www.nih.gov/) through grants P2C HD065690, U54 EB020405.

#### REFERENCES


## ACKNOWLEDGMENTS

We thank David Delp and Thomas Uchida for contributing the figure of the scapulothoracic joint and Apoorva Rajagopal for making additional edits to the figure.

injury: the 2013 consensus statement from the 'Scapular Summit'. Br. J. Sports Med. 47, 877–885. doi: 10.1136/bjsports-2013-092425


Glenohumeral Joint Stability During Abduction. J. Biomech. Eng. 130, 021024. doi: 10.1115/1.2903422

**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Seth, Dong, Matias and Delp. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Kinematic Synergy of Multi-DoF Movement in Upper Limb and Its Application for Rehabilitation Exoskeleton Motion Planning

Shangjie Tang1†, Lin Chen1,2†, Michele Barsotti <sup>3</sup> , Lintao Hu1,4, Yongqiang Li 1,4 , Xiaoying Wu1,2, Long Bai 2,5, Antonio Frisoli <sup>3</sup> \* and Wensheng Hou1,2,4 \*

*<sup>1</sup> Key Laboratory of Biorheological Science and Technology, Ministry of Education, Bioengineering College, Chongqing University, Chongqing, China, <sup>2</sup> Chongqing Key Laboratory of Artificial Intelligence and Service Robot Control Technology, Chongqing, China, <sup>3</sup> PERCRO Laboratory, TeCIP Institute, Scuola Superiore Sant'Anna, Pisa, Italy, <sup>4</sup> Chongqing Engineering Research Center of Medical Electronics Technology, Chongqing, China, <sup>5</sup> College of Mechanical Engineering, Chongqing University, Chongqing, China*

#### Edited by:

*Naser Mehrabi, University of Washington, United States*

#### Reviewed by:

*Jing Jin, East China University of Science and Technology, China Mahdokht Ezati, University of Waterloo, Canada Borna Ghannadi, University of Waterloo, Canada*

#### \*Correspondence:

*Antonio Frisoli a.frisoli@sssup.it Wensheng Hou w.s.hou@cqu.edu.cn*

*†These authors have contributed equally to this work*

Received: *08 March 2019* Accepted: *14 November 2019* Published: *29 November 2019*

#### Citation:

*Tang S, Chen L, Barsotti M, Hu L, Li Y, Wu X, Bai L, Frisoli A and Hou W (2019) Kinematic Synergy of Multi-DoF Movement in Upper Limb and Its Application for Rehabilitation Exoskeleton Motion Planning. Front. Neurorobot. 13:99. doi: 10.3389/fnbot.2019.00099* It is important for rehabilitation exoskeletons to move with a spatiotemporal motion patterns that well match the upper-limb joint kinematic characteristics. However, few efforts have been made to manipulate the motion control based on human kinematic synergies. This work analyzed the spatiotemporal kinematic synergies of right arm reaching movement and investigated their potential usage in upper limb assistive exoskeleton motion planning. Ten right-handed subjects were asked to reach 10 target button locations placed on a cardboard in front. The kinematic data of right arm were tracked by a motion capture system. Angular velocities over time for shoulder flexion/extension, shoulder abduction/adduction, shoulder internal/external rotation, and elbow flexion/extension were computed. Principal component analysis (PCA) was used to derive kinematic synergies from the reaching task for each subject. We found that the first four synergies can explain more than 94% of the variance. Moreover, the joint coordination patterns were dynamically regulated over time as the number of kinematic synergy (PC) increased. The synergies with different order played different roles in reaching movement. Our results indicated that the low-order synergies represented the overall trend of motion patterns, while the high-order synergies described the fine motions at specific moving phases. A 4-DoF upper limb assistive exoskeleton was modeled in SolidWorks to simulate assistive exoskeleton movement pattern based on kinematic synergy. An exoskeleton Denavit-Hartenberg (D-H) model was established to estimate the exoskeleton moving pattern in reaching tasks. The results further confirmed that kinematic synergies could be used for exoskeleton motion planning, and different principal components contributed to the motion trajectory and end-point accuracy to some extent. The findings of this study may provide novel but simplified strategies for the development of rehabilitation and assistive robotic systems approximating the motion pattern of natural upper-limb motor function.

Keywords: kinematic synergies, upper limb movements, principal component analysis, motion planning, inter-joint coordination

## INTRODUCTION

Hemiplegia, or unilateral paresis, is currently reported to be one major cause of physical disability in the middle-aged and elderly (Paul et al., 2002; Gert et al., 2003). Patients often experience autonomic difficulties in daily life due to the unilateral motor dysfunction. Recent works on upper-limb robotic training experiments exhibited substantial improvements on joint motions in either the horizontal plane or threedimensional space for the patients of motor impairment with wearable exoskeletons (Kwakkel et al., 2008; Jarrassé et al., 2014). Therefore, exoskeleton technology has attracted extensive attentions for its noteworthy value of assisting multi-joint rehabilitation movement and daily life performance (such as reaching and grasping) (Frisoli et al., 2012). Pilot studies suggested that spatiotemporal motion patterns that well match the nature of upper-limb joint movement plays a key role in the long lasting effects in rehabilitation (Liu et al., 2018). A typical upper limb movement is fulfilled with spatiotemporal motor coordination of multiple joints, and kinematic synergy among limb joints have been widely explored as a control principle for motor function (Tomita et al., 2017). However, little is known how to effectively use kinematic synergies of upper limb movement in exoskeleton motion planning for motor assistive purpose.

Numerous robotic devices for upper limb rehabilitation have been developed with kinematic patterns mimicking actual human arm movements with multiple degree of freedoms (DoF). The kinematics of multi-joint coordination were extracted and implemented in exoskeleton to conduct activities of daily life (ADLs). Johnson et al. (2001) designed a 5-DoFs motorized assistive device with a 3-DoFs cable driven shoulder structure to enable patients with arm disorders to perform controlled movements, strengthening exercises, and continuous passive motion according to selective modes. Xu and Qiu (2013) proposed an alternative design of a flexible continuum exoskeleton apparatus to collaborate with human movements for the shoulder joint only. Most of these devices were programmed to certain trajectories for rehabilitation training without full considerations of actual continuous multi-joint movement modes, and may obstruct smooth performance of consecutive upper limb movements (Ding et al., 2014). To overcome the kinematic redundancy in reaching movements, Li et al. introduced an exponential method of human motor control strategy, which demonstrated accuracy improvements for real-time motion control of designed upper limb exoskeleton (Li et al., 2015). However, the regulation strategies in central nervous system (CNS) for multi-joint coordination in upper limb tasks were correlated to complex movement patterns (Scano et al., 2017). A simulated framework based on muscular and postural synergy indicated that a robotic arm with multi-DoF could generate arbitrary trajectories similar to human natural movements for the end-effector (Liu and Xiong, 2013), but also increased redundancy that deteriorated the motion control. Several studies attempted to simplify the motor control of assistive device for upper limb actuating planar or threedimensional motion performance, such as the impedance control (Hogan et al., 1992) and mirror-image movement enabler technique (Lum et al., 2004).

The temporal and spatial inter-limb coordination plays a key role in upper limb rehabilitation. The surface electromyography (sEMG) signal was frequently employed to control multi-DoF mechanical arm (Hang et al., 2014), however, the sEMG signal did not directly reflect those complex temporal multi-joint coordination across limbs (Merad et al., 2018). Alternative motion kinematics model has also derived via fitting a high-order polynomials based on sEMG analysis to estimate the multi-joint angles of upper limb movements (Ding et al., 2014). However, sEMG-based muscle activities were generally represented in high dimensional feature space. It was difficult to meet the robustness requirement of complicated multi-DOF arm motions (Ding et al., 2016). Other strategies have also been developed to optimize the near-natural trajectories of assistive equipment. For example, a mathematical model based on a criterion function to characterize planar two-joint arm movements was formulated by Flash and Hogan, and the predicted trajectories matched experimental observations of real human performance well in a horizontal plane (Flash and Hogan, 1987). Nevertheless, how to temporally represent motion patterns of the multi-joint coordination similar to actual trajectory of human arm movements still left an open question.

Kinematic synergy based on classical neuromechanic theories (Bernstein, 1967) is another important concept in motor coordination. Existing evidence suggests that the CNS generates motor command to co-activate multiple muscles working with specific extents, which was generally referred to as "synergy" (Turvey, 2007; Mukta et al., 2015). During motion behaviors, the neural control strategies were modulated to regulate synergistic patterns dynamically to meet the task requirements (Burns et al., 2017). Several pilot studies have employed kinematic synergies to evaluate the inter-joint movement patterns of different human motor approaches (Tresch et al., 2006; Chen and Xiong, 2013). Artificial neural network (Devi et al., 2013), linear discriminant analysis (LDA) (Ramana Vinjamuri et al., 2015), and several other algorithms have been introduced to derive synergies; however, principal component analysis (PCA) is most frequently employed for kinematic synergies analysis (Ramana et al., 2010; Chen and Xiong, 2013; Patel et al., 2016; Burns et al., 2017). Burns et al. (2017) successfully used PCA method to derive time-varying kinematic synergies of bilateral upper arm reaching movement. Bockemühl et al. (2010) analyzed the kinematics complexity of human catching movement, and found that the first three principal components captured more than 97% of variance. Further study confirmed that human upper limb trajectories can be reconstructed by a linear combination of few principal time-dependent functions (Averta et al., 2017). Also, researchers employed synergy analysis as inputs to control the rehabilitation mechanical system, such as upper limb exoskeleton rehabilitation robot (Liu and Xiong, 2013) or dexterous hand for grasping movements (Catalano et al., 2014). However, few efforts have been made to manipulate the motion control based on human kinematic synergies. This work analyzed the spatiotemporal kinematic synergies of right arm reaching movement and investigated their potential usage in upper limb

assistive exoskeleton motion planning (see **Figure 1**). Therefore, we employed PCA algorithm to analyze the spatiotemporal kinematic synergistic pattern of the shoulder and elbow joint in reaching activities and simulated the motion planning for upper limb assistive exoskeleton with kinematic synergies.

## SUBJECTS AND METHODS

This study was conducted at the Perceptual Robotics Laboratory (Pisa, Italy). Ten healthy subjects (eight males and two females, with the age of 26 ± 2) participated in the study after informed consent was obtained. All subjects were right handed and confirmed free of any arm neuromuscular disorder or previous joint injury. The experimental procedures were conducted in accordance with the Declaration of Helsinki and approved by the Ethical Review Board of Scuola Superiore Sant'Anna.

## Data Acquisition

An IMU-based motion capture system (Perception Neuron, Noitom Technology Ltd., Beijing, China) was used to record motion data of the subjects' upper limb during tasks. Four IMU sensors were placed on the subjects, as shown in **Figure 2C** (P1–P4): left shoulder, right shoulder, right elbow, right wrist. Then, the spatial position of shoulder, elbow, wrist, and their trajectory of upper limb motion could be recorded during test. Referred as an experimental report (Burns et al., 2017), frame calibrations were performed before the data acquisition as: (1) positive x extended backward away from the subject; (2) positive y extended to the right side of subject, and (3) positive z extended upwards toward the ceiling. The axis x, y, and z were illustrated in **Figure 2A**. Therefore, the data collected by this motion capture system can scale the right arm motion pattern with respect to subject's torso. The sampling frequency is 120 samples per second.

#### Experimental Procedure

The participants were asked to reach the target button placed on a cardboard in front of them. Three buttons (yellow, blue, and green) were designed as targets and a white button was assigned as the start point. A LED (light-emitting diode) was embedded in white button to cue a reaching task beginning, whereas a LED was embedded in yellow, blue or green button to show which target should be reached in a trial. Five linear tracks in different directions were defined by the angles from the central track (**Figures 2B,C**) as: −60, −45, 0, 45, and 60◦ . The central track (i.e., 0◦ ) was directly in front of the subject. The target buttons were placed on the selected 3 tracks. We assigned the yellow button to one of the contralateral two tracks (−60 or −45◦ ), blue button to the central track (0◦ ), and green button to one of the ipsilateral two tracks (45 or 60◦ ). On each track, 6 different locations with 5 cm interval were designed for the placement of target buttons, however, only 2 locations were used (the second and the fifth scales) in the experiments. Therefore, a total of 10 possible target locations, named in alphabetical order as "a" to "j," were involved (**Figure 2B**). The distance between the start and the target position was designed as about 80% of the maximum distance that the participant can reach.

The reaching task trial was conducted as: (1) pressing the start button (white); (2) pressing a button located at ipsilateral (green), central (blue) or contralateral (yellow) as the reaching task required; (3) returning to the start position (white). An acoustic trigger was used as the start cue for participants to reach and press the target button. Each target was pressed six times, but the target order was randomly assigned. The instructions for reaching movements were given with a randomized interval between 1 and 3 s. Each of these ten Participants were required to avoid any redundant movement during the whole procedure of task performance. A more detailed experimental procedure was described in a previous report (Tang et al., 2018).

#### Kinematics Measurement of Right Upper Limb

During our reaching task, the upper limb and joints performed spatial motions, which could be projected to the calibrated frame of axes. Then, the motion pattern of limb and joint can be evaluated through calculating the angular course at each time point. As illustrated in **Figure 2C**, the kinematics chain of arm was modeled with four segments passing through the five aforementioned joints to evaluate the following angles: shoulder flexion/extension (SFE), shoulder abduction/adduction (SAA), shoulder internal/external rotation (SIR), and elbow flexion/extension (EFE). The kinematics is defined by the following vectors:

$$p\_i = \begin{bmatrix} x\_i, y\_i, z\_i \end{bmatrix} \tag{1}$$

$$
\overline{s}\_{12} = p\_1 - p\_2 \tag{2}
$$

$$
\vec{s}\_{32} = p\_3 - p\_2 \tag{3}
$$

$$
\overline{s}\_{43} = p\_4 - p\_3 \tag{4}
$$

where s<sup>n</sup> is a normal vector to the sagittal plane, and pi(x<sup>i</sup> , yi , zi) is the spatial coordinates of each joint. The angle of SFE, SAA, SIR, and EFE are then calculated as θsf , θsa, θsi, and θef , respectively. sij<sup>x</sup> = 0 denotes the x coordinate in vector sij set to 0.

$$\theta\_{\mathfrak{g}'} = \cos^{-1} \frac{\overline{\mathfrak{z}}\_{32} \cdot \overline{\mathfrak{z}}\_{n}}{\| \ \overline{\mathfrak{z}}\_{32} \| \| \ \overline{\mathfrak{z}}\_{n} \|}, \ \overline{\mathfrak{z}}\_{32\_{\mathfrak{z}}} = 0 \tag{5}$$

$$\theta\_{sa} = \cos^{-1} \frac{\vec{s}\_{32} \cdot \vec{s}\_n}{\|\|\vec{s}\_{32}\|\|\|\vec{s}\_n\|\|}, \ \vec{s}\_{32\_x} = 0 \tag{6}$$

$$\theta\_{\rm sj} = \cos^{-1} \frac{\tilde{s}\_{12} \cdot \tilde{s}\_{32}}{\|\,\tilde{s}\_{12}\| \|\,\tilde{s}\_{32}\|}, \,\,\tilde{s}\_{12\_{\rm y}} = 0, \,\,\tilde{s}\_{32\_{\rm y}} = 0 \tag{7}$$

$$\theta\_{\mathfrak{e}f} = \cos^{-1} \frac{\tilde{s}\_{42} \cdot \tilde{s}\_{23}}{\|\,\tilde{s}\_{43}\,\|\,\|\,\tilde{s}\_{23}\,\|\,\|} \tag{8}$$

#### Synergy Derivation

The time-markers of action performance (button pressed/released and LED on/off states) were recorded through a Simulink model in MATLAB (the Mathworks, Natich, MA, USA). A complete recording set was segmented between the release and re-press of the white start button (**Figure 3**). The PT.F (forward-performance time) was used to segment the angular data of forward movement of reaching task trial by trial.

After data segmentation, the angular velocities were calculated by the derivative of the temporal joint angle profiles. For further processing, the angular velocity data of a reaching trial was resampled to 150 samples for each DOF. Then, the ith sample of a trial was used to construct a 6 × 4 sub-matrix as following:

$$W(i) = \begin{bmatrix} \alpha\_1^1(i) & \alpha\_2^1(i) & \alpha\_3^1(i) & \alpha\_4^1(i) \\ \alpha\_1^2(i) & \alpha\_2^2(i) & \alpha\_3^2(i) & \alpha\_4^2(i) \\ \vdots & \vdots & \vdots & \vdots \\ \alpha\_1^6(i) & \alpha\_2^6(i) & \alpha\_3^6(i) & \alpha\_4^6(i) \end{bmatrix} \quad (i = 1, 2, 3, \cdots, 150)$$

Where the subscript (row variable) and superscript (column variable) represent the joint DOFs and task repetitions, respectively. Then, a 6 × 600 matrix W composed of all 150 sub-matrixes was constructed to represent the six repetitions of a target reaching task to each subject.

$$W = \begin{bmatrix} W(1) \ W(2) \ \dots \ W(150) \end{bmatrix} \tag{10}$$

Thus, W was 6-by-600 matrix, which was then given as input to the PCA algorithm to extract spatiotemporal synergies. Then, singular value decomposition (SVD) algorithm was performed on angular velocity matrix W to derive kinematic synergies. Three-component matrices U, 6, and S were computed from W, as shown in:

$$W = U\Sigma S\tag{11}$$

U and S are orthogonal matrices, and 6 is a diagonal matrix. In this case, U is a 6-by-6 matrix, which has orthonormal

columns and S is a 600-by-600 matrix with orthogonal rows. The diagonal matrix 6 is of 6-by-600 and the diagonal elements of 6 correspond to the singular values (λi) of W.

$$
\Sigma = \begin{bmatrix}
\lambda\_1 & 0 & 0 & 0 & 0 & 0 & 0 & \cdots & 0 \\
0 & \lambda\_2 & 0 & 0 & 0 & 0 & 0 & \cdots & 0 \\
0 & 0 & \lambda\_3 & 0 & 0 & 0 & 0 & \cdots & 0 \\
0 & 0 & 0 & \lambda\_4 & 0 & 0 & 0 & \cdots & 0 \\
0 & 0 & 0 & 0 & \lambda\_5 & 0 & 0 & \cdots & 0 \\
0 & 0 & 0 & 0 & 0 & \lambda\_6 & 0 & \cdots & 0
\end{bmatrix}
\tag{12}
$$

S matrix is defined as in equation below:

$$S = \begin{bmatrix} s\_1^1(1) & \cdots & s\_4^1(1) & \cdots & s\_1^1(t\_{\text{max}}) & \cdots & s\_4^1(t\_{\text{max}}) \\ \vdots & & \vdots & & \vdots & & \vdots \\ s\_1^m(1) & \cdots & s\_4^m(1) & \cdots & s\_1^m(t\_{\text{max}}) & \cdots & s\_4^m(t\_{\text{max}}) \\ \vdots & & \vdots & & \vdots & & \vdots \\ \vdots & & \vdots & & \vdots & & \vdots \\ s\_1^M(1) & \cdots & s\_4^M(1) & \cdots & s\_1^M(t\_{\text{max}}) & \cdots & s\_4^M(t\_{\text{max}}) \end{bmatrix} \\ \tag{13}$$

The first m rows of S are called the first m principal components, or "synergies" and M was the maximal number of synergies. The approximation matrix W˜ can be composed by the first m columns of U, m-by-m of 6 (the other values are replaced by zeros), and m rows of matrix S. The product Umdiag{λ1, λ2, · · · , λm} is called the weight matrix.

$$\tilde{W} = U\_m \text{diag}\{\lambda\_1, \lambda\_2, \dots, \lambda\_m\} S\_m \tag{14}$$

The fraction of sum-squared variance can be calculated from the diagonal elements of 6 and this index was denoted as K.

$$K = \frac{\lambda\_1^2 + \lambda\_2^2 + \dots + \lambda\_k^2}{\lambda\_1^2 + \lambda\_2^2 + \dots + \lambda\_M^2} (k = 1, \ 2, \ 3, \ \dots, M) \tag{15}$$

K was used to determine how many principal components were sufficient to represent the whole data. The index threshold of 94% variance was used to determine the best number of synergies.

The angular velocity signal was reconstructed by selecting different numbers of synergies. The reconstruction error e between measured ωj(t) and reconstructed ω˜j(t) across joints (J = 4) was computed for each reaching movement. Oneway analysis of variance (ANOVA) was performed based on normalized reconstruction error according to different number of recruited synergies.

$$e = \frac{\sum\_{j=1}^{J} \sum\_{0}^{t} \left(\boldsymbol{\alpha}\_{j}(t) - \boldsymbol{\tilde{\alpha}}\_{j}(t)\right)^{2}}{\sum\_{j=1}^{J} \sum\_{0}^{t} \boldsymbol{\alpha}\_{j}(t)^{2}}\tag{16}$$

#### Motion Planning for Upper Limb Assistive Exoskeleton

To evaluate the impact of kinematic synergy on arm motion control, an upper limb rehabilitation exoskeleton model was built in SolidWorks (Dassault Systemes, Massachusetts, USA). The simulated exoskeleton was used to guide the joints of right

reaching movement (the end point of bracket 5).


*SIR was the arm rotating in transverse plane, and the upper arm moving to right side of sagittal plane was set as positive; SAA was the arm rotating in coronal plane, and the upper arm moving outward was set as positive; SFE was the arm rotating in sagittal plane, and the upper arm moving forward was set as positive.*

arm moving in 3-DoFs rotation for shoulder joint and 1-DoF flexion/extension for elbow joint. As illustrated in **Figure 4**, the exoskeleton was driven by four stepping motors within the controlled range of motion (ROM). The ROMs of EFE, SFE, SAA, and SIR joint for reaching movements were set as 0–135, 0–100, 0–90, and 0–105◦ , respectively. Motor I was assigned to drive the device rotating horizontally to achieve internal/external rotations of the shoulder joint. Motor II could enable the rotation and slide of bracket 2 on the chute 3 to perform the shoulder abduction/adduction. Motor III was used to control arm bracket 4 rotating around the bracket 2 to execute the flexion/extension. Motor IV rotated the forearm bracket 5 around the upper-arm bracket 4 to perform flexion/extension of the elbow joint. The description of motors in this model were detailed in **Table 1**.

In order to evaluate the effect of kinematic synergy on arm movement control, an exoskeleton Denavit-Hartenberg (D-H) model was established. This model consisted of five coordinate systems (see **Figure 4B**). The base coordinate system {0} was set at the shoulder joint center, whereas the coordinate system {1}, {2}, {3}, and {4} were specified with SIR, SAA, SFE, and EFE, respectively. In the Denavit-Hartenberg (D-H) model, the origins


*In our model, r* = *70 mm, the radius of the exoskeleton slip ring; l<sup>1</sup>* = *240 mm, the length upper arm in exoskeleton; l<sup>2</sup>* = *225 mm, the length of forearm in exoskeleton;* θ*si,* θ*sa,* θ*sf ,* θ*ef represented angles of SIR, SAA, SFE, EFE, respectively.*

of coordinate system {0}, {1}, and {2} were located at the center of the shoulder joint; the origin of coordinate system {3} was located at the position of motor for flexion and extension, and the origin of coordinate system {4} was located at the motor position of elbow joint. The origin of coordinate system {5} was located at the position of the end of the exoskeleton (see the red circle dot in **Figure 4A**).

(SFE), shoulder abduction/adduction (SAA), and shoulder internal/external rotation (SIR), respectively.

To quantify the effect of kinematic synergy on reaching movement control, we calculated the spatial position of the end-point of the exoskeleton model when angular variables was applied to D-H matrix. The Denavit-Hartenberg (D-H) matrix for our exoskeleton model was specified as

$$
\begin{bmatrix}
\cos\theta\_{\dot{i}} & -\sin\theta\_{\dot{i}} & 0 & a\_{(i-1)} \\
\sin\theta\_{\dot{i}}\cos\alpha\_{(i-1)} & \cos\theta\_{\dot{i}}\cos\alpha\_{(i-1)} & -\sin\alpha\_{(i-1)} & -\sin\alpha\_{(i-1)}d\_{\dot{i}} \\
\sin\theta\_{\dot{i}}\sin\alpha\_{(i-1)} & \cos\theta\_{\dot{i}}\sin\alpha\_{(i-1)} & \cos\alpha\_{(i-1)} & \cos\alpha\_{(i-1)}d\_{\dot{i}} \\
0 & 0 & 0 & 1
\end{bmatrix}
$$

$$
\sigma\_0^5 T = \,\_0^1 T\_1^2 T\_2^3 T\_3^4 T\_4^5 T \tag{18}
$$

Where ai−<sup>1</sup> was the distance between Zi−<sup>1</sup> to Z<sup>i</sup> along Xi−1, αi−<sup>1</sup> was the angle at which Zi−<sup>1</sup> to Z<sup>i</sup> rotated around Xi−1, d<sup>i</sup> was the distance measured from Xi−<sup>1</sup> to X<sup>i</sup> along Zi , θ<sup>i</sup> was the angle at which Xi−<sup>1</sup> to X<sup>i</sup> rotated around Zi . The Denavit-Hartenberg (D-H) parameters were listed in **Table 2**.

#### RESULTS

### Synergy Extraction Using PCA

All designed experiments have been successfully repeated 6 times at each target location in randomized order, and the kinematic data of right upper limb are collected during reaching task being conducted. Spatiotemporal kinematic synergies are derived using PCA subject by subject. **Figure 5** presents an example of angular velocity profile based on the first four synergies of subject 9 performing the task for reaching target "e." Here, the first synergy (column 1) involving shoulder flexion (SFE), abduction (SAA) and external rotation (SIR) clearly demonstrate a forward reaching movement. In the same time, an elbow extension (EFE) was conducted and guide his/her right arm moving to press a target. Synergy 2 (column 2) shows a similar shoulder kinematic trend, with higher amplitudes and longer duration time for the shoulder abduction (SAA) and flexion (SFE). Synergy 3 (column 3) consists of a shoulder extension (SFE), a combination of a shoulder adduction followed by an abduction (SAA), and a combination of internal rotation followed by a slight external rotation (SIR). The corresponding elbow angular velocity profile (EFE) illustrates the procedure of "flexion-extension-flexion." Synergy 4 (column 4) tends to behave similarly as synergy 2

(column 2), except that the elbow movement exhibits repeated flexion/extension with a shorter execution time (EFE).

**Figure 6A** presents the explained variance of PCs (or "synergies"). As shown in **Figure 6B**, four PCs are sufficient to capture more than 94% of the data's variance for all target positions (a-j). Moreover, the first three PCs account for at least 87% of the variance, and the first two PCs are also able to explained at least 64% of the variance. Specifically, the first two PCs at position "f " accounting for variance even achieve 78%.

Correlation analysis of the first PC (synergy 1) has been carried out for each DoF of upper limb across subjects (**Figure 6C**). Ipsilateral reaching tasks (target position at g–j) shows higher correlations than the central and contralateral tasks (a–f), which suggests that personalized kinematic parameters vary less in tasks of reaching the closer or more skilled positions. The high level of coefficient value for SFE (red dots) also reveals that the shoulder flexion/extension exhibits better stability across subjects than the other three DoFs (EFE, SAA, and SIR).

**Figure 7A** give an example of the angular velocity reconstruction based on different numbers of PCs (synergies) for shoulder abduction reaching target "a" (**Figure 2B**). As expected, the reconstructed kinematic profiles for upper limb joints get closer to the actual measurement as the number of synergies augmented. The reconstructed angular velocities with the first 2 synergies (PC1 and PC2 only) can represent similar kinematic trends, such as acceleration/deceleration phases of the real movements, while higher orders of synergies (PC3 and PC4) can supplement and refine the reconstruction to match the true measurements with higher accuracy.

FIGURE 7 | (A) Comparison between the measured (solid) and reconstructed (dashed) angular velocity for shoulder abduction of task "a" (Figure 2B) (upper), and the corresponding acceleration of movement (lower). The reconstructed angular velocities presented on the upper plots are calculated based on different numbers of PCs. (B) Reconstruction errors of angular velocities within stable and changing periods, respectively. (C) Relationships of normalized reconstruction errors against number of PCs (synergies) for angular velocity reconstruction. (D,E) Grouped comparison of the impact of synergy number and target position on reconstruction errors in "changing" or "stable" periods. (F,G) Grouped comparison of the impact of synergy number and shoulder DOFs on reconstruction errors in "changing" or "stable" periods (\*indicates *p* < 0.05).

We segment the movements into two specific periods (changing and stable) depending on the value of acceleration: the "changing period" refers to periods when the absolute acceleration value is higher than 20% of the maximum, while the rest period is considered as "stable period" (**Figure 7A**). We compare the difference of normalized reconstruction errors between the stable and changing period for reconstructed angular velocities (**Figure 7B**), and significant differences are observed (p < 0.05). The normalized errors tends to decrease as the number of PCs (synergies) augmented. In detail, the first 2 synergies yield a normalized reconstruction error of 0.38 in changing period and 0.28 in stable period, while the errors reduce to 0.18 in changing period and 0.05 in stable period when the first 4 PCs (synergies) were involved. Moreover, the normalized errors also vary with movement phases, as the error level for the first 3 PCs in changing period (0.29 ± 0.06) is similar to that for the first 2 PCs in the stable period (0.28 ± 0.05) (see the black dashed line in **Figure 7C**). In addition, **Figure 7C** also indicates that the impact of recruited synergy numbers on reconstruction error is greater in the changing period than that in the stable period. When comparing the impact of recruited synergies on reconstruction error among different reaching target positions, it showed similar trend; in other words, either for "changing" or "stable" period, the reconstruction error of upper limb moving decreased with the recruited synergy numbers (see **Figures 7D,E**). However, as illustrated in **Figure 7F**, the kinematic synergies showed different roles to reconstruct the moving pattern, and more synergies significantly reduced the reconstruction error in "changing" periods for DOF of shoulder abduction/adduction (SAA). In "stable" period, DOF of shoulder internal/external rotation (SIR) exhibited significant reconstruction error when only the first synergy or the first two synergies were recruited (**Figure 7G**).

The reaching task conducted by right arm are normalized as an angular trajectory. To simplify the simulation, the reaching movement is divided into 3 equal periods (0–33.3, 33.3–66.6, 66.7–100%T), the angular value for shoulder joints (EFE, SFE, and SAA) and elbow joint (SIR) are obtained through integrating angular velocity with time. Then, the angular values of the DOFs of EFE, SFE, SAA, and SIR are used to drive motor I, motor II, motor III, and motor IV, respectively. An example visualization of simulated postures at four specific time-points for subject 9 are presented in **Figure 8**. The postures are in accordance with the angular velocity profiles with different synergies as shown in **Figure 5**.

## Motion Simulation Based on Kinematic Synergies

As shown in **Figure 9A**, the movement profile of exoskeleton are simulated when applied to assist subject 5 reaching target "a" in SolidWorks. The first four columns demonstrate the postures of assistive exoskeleton when different order of kinematic synergies are employed to control exoskeleton movement at different time point of reaching task. The last column is the movement pattern of assistive exoskeleton when the original kinematic is

FIGURE 9 | (A) The upper limb assistive exoskeleton model driving by the first four PCs (synergies) at three specific time points of subject 5. (B) The calculated endpoint positions for subject 5 corresponds to reconstructions based on 1–4 PCs recruitment, with the true position of target "a" (location illustrated by Figure 2B) shown as ⋆. (C) Distance errors between calculated and actual target positions for all participants, and its changing trends as the recruited PCs (synergies) augmented.

used to drive the exoskeleton. It can be observed that, with the angular input of shoulder internal/external rotations, shoulder abduction/adduction, shoulder variable for the flexion/extension, and elbow flexion/extension, the assistive exoskeleton moves to target "a" step by step. **Figure 9B** visually illustrates the endpoint positions of assistive exoskeleton when different number of PCs are involved in D-H model for the subject 5. The distance deviations between simulated reaching points and actual target position are calculated, and the results also show that distance errors of exoskeleton endpoint decrease as the kinematic synergies augmented (**Figure 9C**).

### DISCUSSION

Motor control optimization is one of the most important issues in assistive arm device design. In our study, the timevarying kinematics synergies of shoulder and elbow joints during upper limb reaching tasks were analyzed using PCA algorithm. Our results showed that the first four principal components can sufficiently represent the dynamical profile of upper limb joint angles, and principal components with different scales exhibited different contributions to the multijoint motion behaviors. Specifically, the first two synergies (PC1 and PC2) could reflect the direction and motion range of the movement, while the higher order synergies (PC3 and PC4) could smooth the motion trajectory during acceleration period. The motion planning simulation of right arm assistive exoskeleton further confirmed that principal components with different scales played different roles in the motion trajectory and endpoint accuracy.

Synergy is an assemble of individual functional units performing motor behaviors in relatively independent DoFs (Turvey, 2007), and can be represented as multi-joint coordination in angular velocity space (Burns et al., 2017). Our results indicated that the first four PCs can be used as surrogates to describe synergistic characteristics since the four PCs were able to explain major proportion of variance. Moreover, we found that the level of variance explained by PCs was significantly reliant on the target position, and the joint coordination patterns were dynamically regulated over time as the number of kinematic synergy (PC) increased (**Figures 6A,B**). Bockemühl et al. (2010) suggested that the proportion of each principal component was modulated to compensate for the disadvantage of different catching positions. Verrel et al. (2013) proved that the proportion of variance explained by the first three PCs increased when more skilled upper limb movements were performed. In addition, Côté et al. (2002) reported that additional synergy modes were involved when adapting to motion task complexity. Therefore, our results were in accordance with these previous studies, and the correlation coefficient (**Figure 6C**) further revealed that the kinematic synergy of upper limb was associated with positions of reaching target.

Upper limb movement is conducted through coordination of multiple joints both in time and space (Tomita et al., 2017). As illustrated in **Figure 7B**, the reconstruction error varied in different motion periods (i.e., changing and stable) defined by angular velocity profiles. This result agreed to a previous finding, in which Ahmad et al. observed that hand motion stability depends on the motion period (Nadzri et al., 2014). Moreover, Mukta et al. reported temporal compensation during the motion starting period under different reach-tograsp conditions (Mukta et al., 2015). In our study, the error performance in **Figure 7C** indicated that the reconstruction results could significantly be improved as the number of recruited PCs (synergies) increased. Furthermore, as observed in **Figure 7A**, the low-order synergies appeared to show the overall change trend of motion, while the high-order synergies reflected the details at the special movement phase, which suggested potential implications for the level assessments of motor function and rehabilitation.

The multi-DoF synergistic pattern of upper limb movements can help to simplify the motion planning of assistive device for human rehabilitation. In fact, synergy has been considered as an effective method to improve motion smoothness of the rehabilitation device as human behavior (Burns et al., 2017). Therefore, implementing the time-varying principal component analysis into the upper limb prosthesis and rehabilitation device was potential to fulfill the requirements. Moreover, recent study (Tsai et al., 2018) investigated joint kinematics regulation of postural system, and confirmed the value of PCs in evaluating the contributions of individual joint. Our results also demonstrated that the low-order synergies or PCs, generally the first two PCs, represented major variance of original kinematics, and could fulfill common movements for rehabilitation devices. On the other hand, to improve the endpoint accuracy of reaching movement, additional high order components (such as the third and fourth PCs) should be involved in. Therefore, specific to daily life activities which generally require precise control, the highorder PCs (synergies) would have advantages on accurate control strategies for assistive device development. Altogether, the simulation results in our study implied the high potential of synergy being implemented for motion planning, and the corresponding precision of endpoints can be improved by synergy augments.

The present work has confirmed that the motion coordination of shoulder and elbow joint is manifested in the coordination of kinematic synergies. With the data from right arm reaching tasks, it can be concluded that different synergies have specific contributions to the upper limb movement, the low-order synergies represented the overall trend of motion patterns, while the high-order synergies described the fine motions at some moving phases. The results of exoskeleton movement simulation further confirmed that kinematic synergies could be used for exoskeleton motion planning, and different principal components contributed to the motion trajectory and endpoint accuracy with some specific extent. The findings of this study may provide novel but simplified strategies for the development of rehabilitation and assistive robotic systems approximating the motion pattern of natural upper-limb motor function.

## DATA AVAILABILITY STATEMENT

The raw data supporting the conclusions of this manuscript will be made available by the authors, without undue reservation, to any qualified researcher.

## ETHICS STATEMENT

The experimental procedures were conducted in accordance with the Declaration of Helsinki and approved by the Ethical Review Board of Scuola Superiore Sant'Anna.

## AUTHOR CONTRIBUTIONS

ST collected the data and analyzed the data. AF and WH designed the work. ST, LC, MB, and YL drafted the work. WH, XW, and LB interpreted the data. LH worked on the D-H model.

## FUNDING

This work was supported by the National Natural Science Foundation of China (31771069, 31800824, 31971287), Graduate Research and Innovation Foundation of Chongqing of China (Grant No. CYB17038), and the Chongqing Science and Technology Program (cstc2018jcyjAX0390).

## ACKNOWLEDGMENTS

The authors would like to thank the staff of the Perceptual Robotics Laboratory, Sant'Anna School of Advanced Studies, for their continued support of ongoing research and assistance in performing the experiment. The authors also would like to thank all the subjects who participated in this study.

#### REFERENCES


**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Tang, Chen, Barsotti, Hu, Li, Wu, Bai, Frisoli and Hou. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

## Neuromusculoskeletal Modeling-Based Prostheses for Recovery After Spinal Cord Injury

Claudio Pizzolato1,2 \*, David J. Saxby1,2, Dinesh Palipana2,3,4,5, Laura E. Diamond1,2 , Rod S. Barrett1,2, Yang D. Teng6,7 and David G. Lloyd1,2

<sup>1</sup> School of Allied Health Sciences, Griffith University, Gold Coast, QLD, Australia, <sup>2</sup> Griffith Centre for Biomedical and Rehabilitation Engineering, Menzies Health Institute Queensland, Griffith University, Gold Coast, QLD, Australia, <sup>3</sup> The Hopkins Centre, Menzies Health Institute Queensland, Griffith University, Gold Coast, QLD, Australia, <sup>4</sup> Gold Coast Hospital and Health Service, Gold Coast, QLD, Australia, <sup>5</sup> School of Medicine, Griffith University, Gold Coast, QLD, Australia, <sup>6</sup> Department of Physical Medicine and Rehabilitation, Spaulding Rehabilitation Hospital, Harvard Medical School, Charlestown, MA, United States, <sup>7</sup> Department of Neurosurgery, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, United States

Concurrent stimulation and reinforcement of motor and sensory pathways has been proposed as an effective approach to restoring function after developmental or acquired neurotrauma. This can be achieved by applying multimodal rehabilitation regimens, such as thought-controlled exoskeletons or epidural electrical stimulation to recover motor pattern generation in individuals with spinal cord injury (SCI). However, the human neuromusculoskeletal (NMS) system has often been oversimplified in designing rehabilitative and assistive devices. As a result, the neuromechanics of the muscles is seldom considered when modeling the relationship between electrical stimulation, mechanical assistance from exoskeletons, and final joint movement. A powerful way to enhance current neurorehabilitation is to develop the next generation prostheses incorporating personalized NMS models of patients. This strategy will enable an individual voluntary interfacing with multiple electromechanical rehabilitation devices targeting key afferent and efferent systems for functional improvement. This narrative review discusses how real-time NMS models can be integrated with finite element (FE) of musculoskeletal tissues and interface multiple assistive and robotic devices with individuals with SCI to promote neural restoration. In particular, the utility of NMS models for optimizing muscle stimulation patterns, tracking functional improvement, monitoring safety, and providing augmented feedback during exercisebased rehabilitation are discussed.

Keywords: spinal cord injury, neuromusculoskeletal modeling, neural restoration, functional electrical stimulation, brain-computer interface, real-time, digital twin, rehabilitation robotics

## INTRODUCTION

Spinal cord injury (SCI) partially or fully interrupts physiological connections between the brain, the spinal cord, and the muscles. Motor commands can still be generated but may not reach the muscles to produce movement. Similarly, sensory signals from below the injury site, such as signals indicating gravity, motion, touch, pain and/or temperature, cannot travel back to the spinal cord

#### Edited by:

Mohammad S. Shourijeh, Rice University, United States

#### Reviewed by:

Josep M. Font-Llagunes, Universitat Politecnica de Catalunya, Spain Moon Ki Jung, Imperial College London, United Kingdom

> \*Correspondence: Claudio Pizzolato c.pizzolato@griffith.edu.au

Received: 15 May 2019 Accepted: 05 November 2019 Published: 02 December 2019

#### Citation:

Pizzolato C, Saxby DJ, Palipana D, Diamond LE, Barrett RS, Teng YD and Lloyd DG (2019) Neuromusculoskeletal Modeling-Based Prostheses for Recovery After Spinal Cord Injury. Front. Neurorobot. 13:97. doi: 10.3389/fnbot.2019.00097

and brain for proprioceptive input to motor pattern generators, somatosensorimotor perception, and neocortical/conscious interpretation. This feedback loop must be reconnected if mobility, motor pattern generation, and sensation are to reoccur after SCI (Jackson and Zimmermann, 2012).

Recent advances in neural prosthetics and rehabilitation robotics have shown great promise for restoration of voluntary movement in individuals with SCI. These techniques have been shown to restore function in rats with a transected spinal cord (van den Brand et al., 2012). Direct electrical stimulation of the injured spinal cord with co-application of pharmacological agents, such as serotonergic receptor agonists, has also enabled individuals with SCI to regain some voluntary movement (Gerasimenko et al., 2015; Angeli et al., 2018; Gill et al., 2018; Sayenko et al., 2018). Researchers (Donati et al., 2016) recently combined motor-driven exoskeleton gait training and tactile feedback using simulated foot pressures (movement sensation) with advanced brain-computer interfaces (BCI) and virtual reality to restore the brain-muscle loop. Preliminary evidence suggests that concurrently engaging the central and peripheral nervous (e.g., propriospinal projection network, reticulospinal serotonergic neurotransmission) and muscular (e.g., proprioceptive receptors and neuromuscular junctions) systems may promote restoration of the central pattern generator of the spinal cord (i.e., neural restoration) (Ropper et al., 2017; Teng, 2019).

The abovementioned studies have raised expectations about the feasibility of designing non-invasive efficacious therapies for the SCI population, a prospect heretofore considered unattainable. However, application of these technologies often requires guesswork from clinicians or researchers to define the parameters associated with the amount of stimulation or support. For example, functional electrical stimulation (FES) requires users to specify frequency, duty cycle, current amplitude, and timing to achieve a predefined movement pattern. Parameters are usually predefined by the manufacturer, leaving the clinician to adapt the therapy to the patient based on trial-anderror approaches and clinical experience (Doucet et al., 2012). Similarly, motorized rehabilitation robotics require users to preselect gait kinematics and/or kinetics, which are then used to drive the patient during rehabilitation. These sets of parameters need to be maintained within safe limits to prevent injury (He et al., 2017; Angeli et al., 2018). Otherwise, applying electrical stimulation or powered rehabilitation robotics can result in excessive tissue strains and consequent tissue failure given the atrophied musculoskeletal tissues and low bone density present in individuals with SCI (He et al., 2017). However, these approaches to assisted therapy are currently often not personalized to the patient, which could potentially result in poor patient engagement, and consequently, sub-optimal interactionenhanced neural plasticity.

A neuromusculoskeletal (NMS) model is a physics-based functional representation of an individual's NMS anatomy and physiology, which can be used to estimate the internal states of musculoskeletal tissues non-observable via instruments external to the body. A NMS model may account for individualspecific musculoskeletal capabilities, and be used to quantify the difference between voluntary muscle activation and the external assistance required to perform a specific task. NMS models may also monitor musculoskeletal tissue stress/strain to prevent injury, and quantify improvement following rehabilitation, such as increases in voluntary force. Using NMS models, existing rehabilitation methods could be further expanded and improved upon developing a personalized therapy that reduces clinician guesswork by automatically stimulating the patient's muscles, adapting to the patient's recovering muscle activation patterns, challenging the patient in recovery to maximize engagement, and maintaining the amount of external assistance within safe limits.

Here we intended to focus on how NMS models can be used to integrate different neuromechanical prostheses to maximize potency of neurorehabilitation following SCI. The review comprises an overview of currently available neuromechanical prostheses and describes how real-time NMS models can be integrated with assistive devices to improve rehabilitation outcomes. We concluded with a summary of current limitations of the presented approach and suggestions for future research directions.

PubMed was searched for articles published in English from January 1980 to October 2020. Search terms included "FES," "BCI," "neural prosthesis," "exoskeleton," "rehabilitation robotics," "NMS modeling," "finite element (FE) modeling," and "digital twin." Abstracts were reviewed, and papers with a focus on applications in SCI were further analyzed in detail.

### NEUROMECHANICAL PROSTHESES FOR INDIVIDUALS WITH SPINAL CORD INJURY

A neuromechanical prosthesis can be defined as any device or combination of devices that support and/or replace any neural or mechanical function of an individual. In the context of SCI, neuromechanical prostheses to restore function include BCI, peripheral and spinal electrical stimulation, and rehabilitation robotics.

### Brain-Computer Interfaces

Brain-computer interface can capture the user's intention to perform a movement, which can be used to control computer simulations and/or external electromechanical devices (Pfurtscheller et al., 2003a; Silvoni et al., 2011). The patient's movement or force output is captured by the afferent pathways, which in turn affects the patient's brain activity. Motor imagery, the act of imagining performing a movement without producing mechanical output, can modify the neuronal activity of the sensorimotor cortex, similar to what occurs when performing the real movement (Pfurtscheller and Neuper, 2001). Electroencephalogram (EEG) recordings acquired synchronously with motor imagery of tasks such as cycling or walking can be used to train a machine learning classifier to discriminate between different brain states (Lotte et al., 2018). Using this approach, BCI can then be used in real-time to classify different motor intentions and control assistive devices, such as rehabilitation robotics (Barsotti et al., 2015) and FES

(Pfurtscheller et al., 2003b). BCI has also been used in combination with more sophisticated machine learning methods to predict kinematic of movement (Cheron et al., 2012) and control the gait of a virtual reality avatar in real-time (Luu et al., 2016). Practically, this means that EEG acquired by a BCI can be transformed into an output in the real world. Current evidence suggests rehabilitation using a BCI can induce neural plasticity and improve motor function in people with neurological conditions (Grosse-Wentrup et al., 2011).

## Peripheral and Spinal Electrical Stimulation

Electrical stimulation uses electrical current to stimulate the spine, peripheral nerves, or musculoneuronal junctions to artificially induce muscle contraction. Stimulation results in synchronous recruitment of motor neurons and force production at the level of muscle fibers. Force is then transmitted via tendons to the skeletal system producing final effector force and movement. Electrical stimulation of muscles, often referred to as FES, is non-invasive and can be performed transcutaneously via pairs of electrodes applied to each muscle or muscle group. FES during leg cycling is a popular rehabilitation modality (Ragnarsson, 2008), as the pedaling motion is a closed kinematic chain (i.e., constrained mechanical action) and accessible to people with tetraplegia. Even when implemented more than 20 years following SCI (Mohr et al., 1997), FES leg cycling continuously showed multiple clinical benefits, such as increased cardiorespiratory performance (Pollack et al., 1989) and endurance (Mohr et al., 1997), prevention of muscle atrophy (Baldi et al., 1998), and increased muscle mass (Mohr et al., 1997).

Epidural electrical stimulation of the spinal cord has been used to evoke rhythmic electromyograms (EMGs) from muscles of the lower limbs, resulting in individuals with complete SCI to independently walk again (Angeli et al., 2018; Gill et al., 2018). Less invasive transcutaneous electrical stimulation of the spinal cord also restored movement in the lower limbs of individuals with SCI, resulting in retention of some volitional movement control even in the absence of electrical stimulation (Gerasimenko et al., 2015). However, improper stimulation of the spinal cord may interfere with afferent neural pathways and disrupt proprioceptive information (Formento et al., 2018), which are proposed to be essential for restoration of neural function (Rushton, 2003). Moreover, there might be potential for adverse events when stimulation is applied to SCI patients using guesswork alone to define input parameters. Thus, technologies and methods for preventing excessive tissue loading in response to electrical stimulation are essential.

Treatments involving electrical stimulation necessitate a set of parameters to be defined, such as on/off timing of muscle stimulation, and stimulation frequency and amplitude. Clinically, these parameters are commonly set by the operator based on predefined values or via trial and error experiential approaches (Doucet et al., 2012). Automatic parameters selection can be achieved via closed-loop control strategies that automatically tune amplitude and/or frequency of stimulation to track predefined kinematics or kinetics targets (Hunt et al., 2004; Lynch and Popovic, 2008; Li et al., 2016). However, most control strategies presently available do not appropriately model the underlying NMS system of an individual, overly simplifying or ignoring the dynamics of muscle activation and contraction and their effects on joint movement (Sartori et al., 2016). These control strategies do not permit observation of the internal state of the musculoskeletal system, nor allow for planning optimal muscle coordination strategies when multiple degrees of freedom are involved. Moreover, even most recent approaches to controlling do not account for variations in an individual's anatomy, physiology, or neuromuscular system, nor do they automatically adapt to a patient's changing neural capabilities on any time scale. Collectively, these technologies have many limitations, that if addressed, could greatly enhance rehabilitation outcomes.

## Rehabilitation Robotics

Rehabilitation robotics involves any motorized electromechanical system, either wearable or stationary, that assists an individual to perform a target movement, such as exoskeletons (Jezernik et al., 2003) and motorized ergometers (Mekki et al., 2018). Robotic-assisted rehabilitation commonly involves securing the patient to the machine and the therapist defining what specific gait or cycling kinematics pattern the robot should provide. In most cases involving individuals with complete SCI, the patient is completely and passively guided by the robot and minimally engaged in the rehabilitation process, due to no need to deliver any kind of executive command from the brain. During this process, intact spinal loops and reflexes may be triggered, resulting in some amount of muscle contraction, force and movement generation, and may also contribute to maintaining overall musculoskeletal tissue health, but effectiveness of these therapies on walking function remains poor (Swinnen et al., 2010; Mehrholz et al., 2017). If these robotic assistive devices could be designed to maximally engage the individual's motor imagery, this would be an improvement over current use of this technology.

## Combined Use of Multiple Assistive Devices

Multiple assistive devices have been combined to maximize functional outcomes (Mekki et al., 2018). In a study involving BCI-controlled FES of wrist, continuous and sustained motor imagery throughout FES resulted in greater cortical activity when compared to lack of motor imagery during FES (Reynolds et al., 2015), potentially suggesting strengthening of corticospinal pathways (Pfurtscheller and Lopes da Silva, 1999). A similar strategy was also used to successfully restore wrist motor function in post-stroke individuals, wherein FES was significantly more effective when controlled by a BCI (Biasiucci et al., 2018). Combining FES with rehabilitation robotics has been proposed to enhance devices' performance (i.e., reduce the power of exoskeletons' motors) (Ha et al., 2012, 2016) and to prolong the length of the rehabilitation session (del-Ama et al., 2014). Finally, BCI has also been combined with virtual reality and/or assistive robotic devices for rehabilitation of individuals with SCI (Donati et al., 2016).

The fast pace at which research involving neuromechanical protheses is growing (Marchal-Crespo and Reinkensmeyer, 2009) reveals an increasing need to integrate multiple assistive devices that can collaborate with the individual to promote adaptive and patient-centered therapy (Holanda et al., 2017). However, current approaches based on classic control theory or machine learning often oversimplify the complex dynamics of the human NMS system, omitting the mechanism underlying the causal relationship between observed input and output data (Sartori et al., 2016). Physics-based NMS models are an alternative approach that enables natural control of neuromechanical prostheses and permits assessing the internal state of an individual's NMS system.

## REAL-TIME NMS MODELING TO INTEGRATE ASSISTIVE DEVICES

Electromyogram -informed NMS models used experimentally measured EMG to perform forward dynamics simulations of muscle dynamics and estimate musculoskeletal tissue states (Lloyd and Besier, 2003; Sartori et al., 2012; Pizzolato et al., 2015). Musculotendon units are modeled using a Hill-type structure, where an elastic tendon is in series with a contractile muscle fiber. Musculotendon units are connected to bones via insertion points and follow anatomically derived paths that wrap around bones, which are used to estimate musculotendon units' lengths and moment arms. The skeletal system is defined by multiple segments (i.e., bones) connected by three-dimensional joints mobilized accordingly to their anatomical function (Seth et al., 2018). EMG-informed NMS models have been used to successfully estimate muscle forces, joint contact forces, and joint stiffness in the lower and upper limbs of individuals with a variety of neuromuscular conditions (Sartori et al., 2015; Konrath et al., 2017; Hall et al., 2018; Hoang et al., 2018, 2019; Lenton et al., 2018; Kian et al., 2019). NMS models are the optimal platform for integration of multiple assistive devices, enabling physics-based sensor fusion, where input and output quantities are mechanistically and causally related. This is equivalent to the modern concept of a digital twin (Glaessgen and Stargel, 2012; Boschert and Rosen, 2016), but here applied to a person and their assistive device(s) (**Figure 1**). Although not explicitly addressed as such, digital twins based on NMS models are becoming a core technology for human-machine interaction (Sreenivasa et al., 2019) and personalized rehabilitation (Sartori et al., 2016; Pizzolato et al., 2017a, 2019), promising exciting technological advancement in prosthetic limb control (Sartori et al., 2019).

Calibration of NMS model parameters is an identification procedure whereby an optimization algorithm finds the optimal set of parameters that minimize the error between experimentally observed and model-predicted quantities (e.g., joint movement, joint moments, joint powers, and EMG). Importantly, calibration ensures that non-observable quantities predicted by the model (e.g., muscle forces, joint contact forces) are physiologically plausible (Gerus et al., 2013; Hoang et al., 2018). Prediction of muscle force has been shown to be particularly sensitive to optimal fiber length, tendon slack length, and maximum

isometric force (Scovil and Ronsky, 2006). Furthermore, parameters associated with muscle activation dynamics have also been shown to require calibration (Lloyd and Besier, 2003). Robotic devices (e.g., the motors of exoskeletons or cycling ergometers) with no stimulation can be used to acquire the joint moments generated by passive musculotendinous and ligamentous tissue (Yoon and Mansour, 1982), and to limit the range of motion of each joint within specific thresholds based on passive moments, both of which are parameters that can be directly input into a personalized musculoskeletal model of an individual with SCI. Similarly, electrically stimulating one muscle group at a time would isolate the effects of contractile properties and activation dynamics of specific muscles and their causal contribution to joint moments and movement. Thus, through performing a sequence of robotic movements and electrical stimulation of different muscle groups, it may be possible to create an automated protocol to identify critical neuromuscular parameters. Future research focused into these domains is required to establish robust protocols for model personalization in individual with SCI.

Following calibration, NMS models need to operate in realtime to appropriately control multiple assistive devices. This means that variables used for controlling or monitoring devices need to be calculated with minimum delay by the NMS model.

Recently, large scale real-time EMG-informed NMS model involving multiple degrees of freedom and musculotendon units have been used to estimate muscle forces and joint contact forces for the full lower limbs using experimental EMGs and motion capture (Pizzolato et al., 2017b,c; Durandau et al., 2018). This was enabled by an optimized multi-threaded software architecture, where a publisher-subscriber software pattern was used to independently handle the multiple input and output devices minimizing idle times (Pizzolato et al., 2017b). Another advantage of this architecture is the ability to modify input or output devices independently from the underlying NMS model, in a plug-and-play fashion. This allows, for example, the operator to easily use wearable sensors, or any other device in place of stereophotogrammetry motion capture systems to acquire human motion in real-time. Output devices may include audio, visual and/or haptic monitors to provide augmented somatosensory information, or control commands for external devices, such as neuromechanical prostheses. The multithreaded software architecture used in recent real-time NMS modeling enables compete decoupling between input and output devices whereby the NMS model acts as super-controller and interpreter between the human and the machine (Ceseracciu et al., 2015). This decoupling allows NMS models to be adapted for a multitude of neurological conditions and neuromechanical prostheses. Finally, these EMG-informed NMS models have the potential to be applied to individuals with SCI to (i) generate optimal muscle stimulation, (ii) improvement tracking and safety monitoring, and (iii) augment afferent feedback.

### Generating Optimal Muscle Stimulation

Successful rehabilitation for SCI patients will involve generating an appropriate set of muscle activation patterns that account for an individual's capabilities. From optimal activation patterns required to generate the desired kinematic or kinetic task, it is then possible to calculate the required amplitude and frequency of stimulation for FES. However, individuals with SCI have different levels of neuromuscular dysfunction, with varying ability to produce voluntary muscle activations (Kirshblum et al., 2011). To further promote neural restoration, it is necessary to engage the patient such that they actively participate in the rehabilitation process. Examples of this approach involve EMGgated FES, where muscle stimulation is provided only when concurrent voluntary contraction from the participant is present (Burridge and Ladouceur, 2001). This approach has been shown to produce better outcomes than standard non-EMG-gated FES (Dutta et al., 2009), though muscle stimulation occurs only if the patient is able to produce sufficiently large voluntary EMG, for which some patients are unable due to the severity of their injury. Furthermore, inter-individual anatomical differences in musculotendon lengths, moment arms, as well as differences in seating position and overall movement kinematics will result in different joint forces and moments for a given FES profile (Schutte et al., 1993). NMS models can appropriately account for these inter-individual differences, providing causal relationships between muscle stimulation and produced force.

A NMS model can be combined with a model of an electromechanical device to calculate an optimal set of muscle activation patterns required to perform a rehabilitation task. In a closed kinematic chain, such as in cycling, the hip, knee, and ankle joint angles are determined as a function of the crank and pedal angles. Thus, joint moments can be easily calculated when the desired values for average power output and cadence are provided (Farahani et al., 2014). NMS models are readily used in conjunction with mathematical optimization (e.g., static optimization) to estimate the optimal set of muscle activation pattern required to perform a predetermined movement. The difference between the voluntary muscle activations of the patient and the target muscle activations from the NMS model provides an objective basis for calculation of the FES compensation level (Yeom and Chang, 2010). Using NMS models, it is also possible to estimate the contribution of voluntary activation from each muscle of the patient to the final joint moments. Consequently, the ratio between the support provided to the patient via electrical stimulation of muscles and the support provided via mechanical assistance can be modulated, which introduces the possibility of creating advanced control strategies that reward patient engagement and voluntary muscle contractions.

A NMS model can be used as layer between a BCI and multiple assistive devices, transforming high level efferent neural commands into appropriate signals to control electrical stimulation and rehabilitation robotics. EEG acquired while an individual with SCI attempts to perform coordinated movement, such as cycling or walking, can be classified in real-time using machine learning approaches, and used to control a NMS model. A primitive but currently feasible solution would involve triggering the NMS model to perform a predetermined trajectory (e.g., cycling). Future approaches may explore BCI to extract basic spinal primitives that can be mapped to individual muscle to enable intuitive control of NMS models (Ubeda et al., 2018). This solution may be more advantageous compared to direct control of joint angles (Fitzsimmons et al., 2009) as it better reflects the current understanding of how the mammalian central nervous system organizes large groups of synergistic muscles during complex movement.

## Tracking of Improvements and Safety Monitoring

Globally, health-care system models are being redesigned to move from volume-based healthcare to value-based healthcare, which is organized around meeting a set of patient needs over the full care cycle (Porter et al., 2013). Although SCI patients consistently rate recovery of paralyzed limb function as their main priority (Anderson, 2004), clinical assessment alone is subjective and unable to quantify meaningful changes that precede major clinical and functional breakthroughs. Objective measurement of musculoskeletal states via NMS models can monitor a patient's state and identify required adjustments in therapy, such as muscle-specific functional improvements. These measurements also have an important strategic objective: to define trajectories of rehabilitation to better inform guidelines for best-clinical practice so therapists and service providers can tailor patient support during the most critical stages of care.

Critical variables that reveal patient improvement can be tracked during NMS model-based therapies, which include the amount of robotic assistance required to perform the task and calibrated neuromuscular parameters assessed at each session. The mechanisms underlying any reduction in robotic assistance, such as increased volitional muscle activation or force production, may also be tracked. Such improvements may be below the detectable thresholds of standard clinical assessments, especially if the improvement is not sufficiently large to generate observable changes in movement. Similarly, musculoskeletal parameters calibrated at each session, such as maximum isometric force, could provide longitudinal measures of neuromuscular adaptation.

Individuals with SCI experience tremendous tissue atrophy (Giangregorio and McCartney, 2006), often losing up to 55% of muscle cross-sectional after 6 months from initial injury (Castro et al., 1999). If, through therapy, inappropriate loads are applied to these weakened tissues, tissue failure may occur (Angeli et al., 2018). Powered exoskeletons and rehabilitation robotics that have been recently approved by the United States Food and Drug Administration (He et al., 2017) as medical devices are expected to be increasingly marked; however, appropriate risk mitigation strategies to prevent injury are lacking (He et al., 2017). Additionally, incorrectly applied magnitude and timing of electrical stimulation to muscles can trigger pain that adversely affects rehabilitation-induced functional recovery and quality of life (Turtle et al., 2018). These risks may partially be mitigated using NMS modeling approaches.

The FE method is a computational method that can be used to predict tissue damage or rupture by modeling the internal mechanics of tissue (i.e., localized stress and strain), as demonstrated by ex vivo studies (Shim et al., 2014, 2018). The geometry of FE models can be personalized to the individual via medical imaging (e.g., magnetic resonance imaging, x-ray computed tomography, or ultrasound) (Devaprakash et al., 2019), while material properties are typically applied from literature data or estimated experimentally (Hansen et al., 2017; Shim et al., 2019). Individual-specific boundary conditions calculated from NMS models (i.e., model pose and applied external forces) are supplied to FE models to estimate the internal stresses and strains of selected musculoskeletal tissues. However, FE analysis is computationally intensive and consequently, cannot be executed in real-time. Surrogate models of FE models have been developed for muscles (Fernandez et al., 2018), tendons (Shim et al., 2018), and bones (Ziaeipoor et al., 2019), enabling rapid evaluation of stress and strain patterns. Given a FE model of a tissue of interest, surrogate models are created in an offline process whereby a FE model is first solved for a complete set of physiologically plausible boundary conditions (e.g., known joint ranges of motion and applied muscle forces). Stress and strain data from all solutions are then used in conjunction with machine learning methods, such as partial least square regression, to create a surrogate model able to replicate the complete FE model with minimal computational complexity in real-time (Ziaeipoor et al., 2019). Surrogate FE models (Fernandez et al., 2018; Ziaeipoor et al., 2019) are currently being combined with real-time EMG-informed NMS models (Pizzolato et al., 2017c) to provide instantaneously estimates of tissue stresses and strains, as described in Pizzolato et al. (2017a, 2019). This is an exciting development, as it is now possible to combine NMS and FE models that are personalized to the individual to generate muscle activation patterns that ensure musculoskeletal tissues are loaded within safe limits. Furthermore, it is now feasible to objectively assess the effects of a rehabilitation exercise on the tissue-level signals that regulate the mechanobiology of musculoskeletal tissues (i.e., stress and strain), which could be coupled with tissue mechanobiology models (Mehdizadeh et al., 2017) to predict

FIGURE 2 | Schematic representation of closed-loop neuromechanical prostheses and their effect on movement and tissue adaptation. Neuromechanical prostheses interface with the central and peripheral nervous system bypass the spinal cord injury to modulate sensorimotor spinal loops, wherein activation of muscles and mobilization of joints result in limb movement and generation of sensory inflow via the somatosensory apparatus. Afferent signals synthetized by a digital twin of the person are redirected via alternative pathways to higher brain areas. At tissue level, the biomechanics (i.e., movement and muscle contraction) result in forces that are applied to the structure of tissues, generating a local mechanical environment (i.e., tissue strain) that modulates tissue biology and consequent tissue structural adaptation.

long term tissue adaptation following quantifiable mechanical stimulation (Pizzolato et al., 2017a; **Figure 2**).

#### Augmented Afferent Feedback

fnbot-13-00097 December 2, 2019 Time: 17:15 # 7

In people with SCI, afferent signaling from somatosensory receptors below the level of injury is hindered, abnormal or absent. To maximize likelihood of neural restoration, afferent signals need to be redirected to intact somatosensory areas for neocortical and conscious interpretation (Jackson and Zimmermann, 2012). It remains unclear what somatosensory signals are most critical to augment technologically, the minimum amount of afferent feedback required, or the preferred delivery modality. It has, however, been proposed that efferent and afferent stimulation need to be consistent and synchronized to enable plastic remodeling of the central nervous system (Rushton, 2003; Jackson and Zimmermann, 2012). Consistent with what has been shown in motor learning and biofeedback studies (Kannape and Blanke, 2013; Sigrist et al., 2013), the delay in delivering technology-augmented afferent feedback must be minimized in order for patients to associate their own movement with the augmented afferent feedback. Using this principle, a recent study employing robotically assisted and virtual reality gait retraining measured plantar pressure transformed into haptic feedback delivered via pads applied to the shoulder region to provide movement sensation (Donati et al., 2016). Further, visual feedback via virtual reality has been used to display patient's avatar limbs during BCI training (Donati et al., 2016), and augmented reality has been used to provide somatosensory feedback non-intrusively via peripheral vision (Clemente et al., 2017).

Neuromusculoskeletal models can be used to augment afferent somatosensory feedback and synthesize mechanoreceptors signals that are not externally observable (Pizzolato et al., 2017a,c). However, synthesized signals need to be integrated into a meaningful single or multi-modal feedback that can be easily interpreted by the patient (Sigrist et al., 2013). Peripheral nerve stimulation has been used to induce sensory feedback in amputees (Dhillon et al., 2004), and the same technique could be applied to redirect NMS model-synthesized afferent signals to intact sensorimotor areas for natural integration to mechanosensing feedback. Cortical interfaces are an alternative but invasive solution that could also be used to relieve burden from the visual system (Tomlinson and Miller, 2016). Cortical interfaces use an electrode microarray that is directly implanted into the sensorimotor cortex to provide electrical stimulation that mimics the natural cortex activity (Tomlinson and Miller, 2016), partially restoring proprioceptive function. Although still in its infancy, cortical interfaces have shown promise in animal studies, and could, in the future, be combined with multiple assistive devices to maximize neural restoration in individual with SCI.

### LIMITATIONS AND FUTURE DIRECTIONS

A variety of assistive technologies are currently available to aid the rehabilitation of individuals with SCI. However, significant improvements and recovery of motor function have been predominantly shown when these devices were combined rather than used in isolation. In this review we have proposed integrating these different technologies via computational NMS models. These models can be considered as a digital twin of the patient and their devices acting as an interpreter between human and machine, continuously monitoring internal tissue state, and tracking longitudinal changes throughout the rehabilitation journey. Nonetheless, several challenges will need to be addressed to achieve this goal, which will require the combined effort from multidisciplinary research engagements.

Current approaches to BCI based on motor imagery are not sufficiently robust and require retraining neural decoders at the beginning of each rehabilitation session (Lotte et al., 2018). Furthermore, only a few movements can be classified via this approach, limiting the use of BCI to simple motor tasks (Bamdad et al., 2015). Part of the problem resides in the poor signal to noise ratio, spatial resolution, and intersession variability of the EEG signals acquired at the scalp. More robust classification methods able to adapt to the user are currently being explored by the BCI research community, but an optimal classification method is yet to be established (Lotte et al., 2018). Recently developed minimally invasive implantable BCI have been able to acquire EEG for extended time periods with greater signal quality than superficial EEG (Oxley et al., 2016). In the future, this technology may enable superior classification of motor intention and seamless integration of humans with assistive devices.

Our proposed strategy involves using NMS models personalized to the individual; however, current personalization methods involve time consuming semi-automatic processing of medical imaging data (Valente et al., 2017). Machine learning methods to automatically segment tissue from medical imaging [e.g., neural networks (Zhou et al., 2018)] and to generate personalized models from population databases [e.g., statistical shape modeling (Suwarganda et al., 2019)] are emerging as promising technologies to personalize anatomy and function of NMS models. However, further efforts will be required to simplify the creation of these models via seamless processing pipelines (Zhang et al., 2014) in order to enable their routine clinical use.

The same NMS modeling-based approach described here for individuals with SCI can be applied for neurorehabilitation of other types of acquired neurological impairments, such as traumatic brain injury and stroke. NMS model-based neuromechanical prostheses are currently possible and within reach, but these assistive technologies will need to be codesigned with clinicians, care providers, and patients to develop devices that are fit for purpose and aligned with the expectations of the final users. If accepted by the clinical community, NMS modeling approaches to neurorehabilitation have the potential to reduce current clinical guesswork by automatically adapt to the individualized needs of each patient, enabling minimally supervised rehabilitation sessions, and reducing costs of care. Clearly, efficacy of NMS modeling-based neuromechanical prostheses will first need to be addressed in clinical trials to understand the effect of exercise dosage, afferent feedback modality, and pharmacological agents on rehabilitation outcomes. Overall, personalized NMS models have the potential to improve current assistive technologies and potentiate neural recovery after SCI.

#### AUTHOR CONTRIBUTIONS

fnbot-13-00097 December 2, 2019 Time: 17:15 # 8

CP contributed to conceptualize, draft, critically revise, and approve the final version of the manuscript. DS, DP, LD, RB,

#### REFERENCES


YT, and DL contributed to conceptualize, critically revise, and approve the final version of the manuscript.

#### FUNDING

This work was supported by the Menzies Health Institute Queensland Capacity Grant Scheme; School of Allied Health Sciences, Griffith University, Strategic Research Grant Scheme; Griffith University New Researcher Grant Scheme; Motor Accident Insurance Commission; and Perpetual IMPACT.



future is within reach. Br. J. Sports Med. 53, 11–12. doi: 10.1136/bjsports-2018- 099020


**Conflict of Interest:** CP, DS, DP, LD, and DL have applied for a patent regarding the use of neuromusculoskeletal-modeling-based controllers with robotic devices and brain-computer interfaces to synthesize efferent and afferent signals to restore function after spinal cord injury described in this review.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Pizzolato, Saxby, Palipana, Diamond, Barrett, Teng and Lloyd. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

## Muscle Synergy Constraints Do Not Improve Estimates of Muscle Activity From Static Optimization During Gait for Unimpaired Children or Children With Cerebral Palsy

#### Benjamin R. Shuman<sup>1</sup> , Marije Goudriaan2,3, Kaat Desloovere3,4, Michael H. Schwartz5,6 and Katherine M. Steele<sup>1</sup> \*

<sup>1</sup> Department of Mechanical Engineering, University of Washington, Seattle, WA, United States, <sup>2</sup> Department of Human Movement Sciences, Faculty of Behavioural and Movement Sciences, Vrije Universiteit Amsterdam, Amsterdam, Netherlands, <sup>3</sup> Department of Rehabilitation Sciences, KU Leuven, Leuven, Belgium, <sup>4</sup> Clinical Motion Analysis Laboratory, University Hospitals Leuven (Pellenberg), Lubbeek, Belgium, <sup>5</sup> James R. Gage Center for Gait and Motion Analysis, Gillette Children's Specialty Healthcare, Saint Paul, MN, United States, <sup>6</sup> Orthopaedic Surgery, Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN, United States

Neuromusculoskeletal simulation provides a promising platform to inform the design of assistive devices or inform rehabilitation. For these applications, a simulation must be able to accurately represent the person of interest, such as an individual with a neurologic injury. If a simulation fails to predict how an individual recruits and coordinates their muscles during movement, it will have limited utility for informing design or rehabilitation. While inverse dynamic simulations have previously been used to evaluate anticipated responses from interventions, like orthopedic surgery or orthoses, they frequently struggle to accurately estimate muscle activations, even for tasks like walking. The simulated muscle activity often fails to represent experimentally measured muscle activity from electromyographic (EMG) recordings. Research has theorized that the nervous system may simplify the range of possible activations used during dynamic tasks, by constraining activations to weighted groups of muscles, referred to as muscle synergies. Synergies are altered after neurological injury, such as stroke or cerebral palsy (CP), and may provide a method for improving subject-specific models of neuromuscular control. The aim of this study was to test whether constraining simulation to synergies could improve estimated muscle activations compared to EMG data. We evaluated modeled muscle activations during gait for six typically developing (TD) children and six children with CP. Muscle activations were estimated with: (1) static optimization (SO), minimizing muscle activations squared, and (2) synergy SO (SynSO), minimizing synergy activations squared using the weights identified from EMG data for two to five synergies. While SynSO caused changes in estimated activations compared to SO, the correlation to EMG data was not higher in SynSO than SO for either TD or CP groups. The correlations to EMG were higher in CP than TD for both SO (CP: 0.48, TD: 0.36) and SynSO (CP: 0.46, TD: 0.26 for five synergies). Constraining activations to SynSO caused

#### Edited by:

Benjamin J. Fregly, Rice University, United States

#### Reviewed by:

Farong Gao, Hangzhou Dianzi University, China Afshin Samani, Aalborg University, Denmark

> \*Correspondence: Katherine M. Steele kmsteele@uw.edu

Received: 13 June 2019 Accepted: 25 November 2019 Published: 17 December 2019

#### Citation:

Shuman BR, Goudriaan M, Desloovere K, Schwartz MH and Steele KM (2019) Muscle Synergy Constraints Do Not Improve Estimates of Muscle Activity From Static Optimization During Gait for Unimpaired Children or Children With Cerebral Palsy. Front. Neurorobot. 13:102. doi: 10.3389/fnbot.2019.00102 the simulated muscle stress to increase compared to SO for all individuals, causing a 157% increase with two synergies. These results suggest that constraining simulated activations in inverse dynamic simulations to subject-specific synergies alone may not improve estimation of muscle activations during gait for generic musculoskeletal models.

Keywords: electromyography, muscle synergies, musculoskeletal modeling, cerebral palsy, static optimization

### INTRODUCTION

fnbot-13-00102 December 12, 2019 Time: 19:31 # 2

Muscle synergies have been used as a method to describe how muscles are commonly activated during tasks such as walking, by identifying a low dimensional space of weighted muscle groupings (Bizzi and Cheung, 2013). These weighted groups of muscles have been shown to be altered among individuals with neurologic injuries, such as stroke or cerebral palsy (CP) (Cheung et al., 2009; Clark et al., 2010; Steele et al., 2015a; Tang et al., 2015; Shuman et al., 2016). Synergies appear to mature in stable patterns early in an individual's lifespan, making them a potential platform for quantifying and modeling an individual's motor control. In unimpaired children, synergies have been shown to be similar to adults after five years of age (Dominici et al., 2011; Rozumalski et al., 2017). For children with CP, synergies are altered but do not change over time, even after extensive surgical interventions with inpatient rehabilitation (Shuman et al., 2019). Although calculation of synergies has been used to describe muscle activation patterns in experimental data, these patterns have only begun to be applied to support musculoskeletal modeling. Using an individual's synergies calculated from experimental data to inform neuromusculoskeletal simulations may improve estimates of an individual's muscle coordination or response to interventions like assistive devices or rehabilitation.

Estimating muscle forces and activations are important for many questions asked with musculoskeletal modeling (Hicks et al., 2015). Examples include contributions of specific muscles to gait (Correa et al., 2011; Mcgowan et al., 2011; Steele et al., 2013; Mansouri et al., 2016) loads acting upon joints (Steele et al., 2012; Walter et al., 2014; Wesseling et al., 2015; Serrancolí et al., 2016), impacts of surgical interventions (Delp et al., 1996; Reinbolt et al., 2008; Fox et al., 2009), and use of orthotic devices (Hegarty et al., 2017; Rosenberg and Steele, 2017). However, when muscle activations are calculated using optimization-based methods, there are large variations in estimated muscle activations across studies (Tinler et al., 2018). Comparisons of modeled muscle activations to experimental data from electromyographic (EMG) recordings are frequently performed only qualitatively, broadly assessing timing and amplitudes (Hamner et al., 2010; Dorn et al., 2012; Hicks et al., 2015; Lerner et al., 2015; Wesseling et al., 2015; Krogt et al., 2016; Zuk et al., 2018b ˙ ). Prior quantitative assessments revealed only moderate correlations between experimental and modeled muscle activations for both typically developing (TD) individuals and individuals with neurologic injuries (Heintz and Gutierrez-Farewik, 2007; Blazkiewicz, 2013; Zuk et al., 2018b ˙ ; Veerkamp et al., 2019). A recent study found similar correlations between individuals with CP and TD individuals (Veerkamp et al., 2019). Prior research often used custom constraints, specifying when a muscle must be on and off, or other strategies to try to get better agreement between simulated activations and experimental measures from EMG data (Liu et al., 2008; Steele et al., 2012). Synergies may improve estimates of computed muscle activations by providing an alternate method to constrain which muscles are simultaneously activated based upon an individual's EMG data (Ting et al., 2012).

Static optimization (SO) is a common algorithm used to estimate muscle activity that minimizes an objective function, such as minimizing the sum of squared muscle activations, while satisfying the system's equations of motion. These optimization methods are theorized to reflect the strategies that unimpaired adults use to coordinate muscle activity. However, a recent study by Simpson et al. (2016) found that individual muscle activations could be adjusted to almost any level at any point in the gait cycle while still satisfying kinematic and kinetic constraints, suggesting that shapes of modeled activation patterns are driven predominantly by the choice of optimization function, rather than being required by the joint torques (Simpson et al., 2016). As high levels of co-contraction are a hallmark of gait in clinical populations like CP (Gage et al., 2009; Steele et al., 2017), other optimization criteria may be more appropriate when modeling pathologic gait (Steele et al., 2013; Sartori et al., 2017). If synergies reflect an individual's neuromuscular control strategy, constraining to individualized synergy structures may help capture subject-specific activations patterns.

Synergies have previously been used to constrain muscle activity for musculoskeletal simulations, most prominently in forward dynamic simulations (Neptune et al., 2009; Mcgowan et al., 2011; Allen and Neptune, 2012; Sartori et al., 2013; Gopalakrishnan et al., 2014; Gonzalez-vargas et al., 2015; Mehrabi et al., 2019). Two studies have employed synergies with musculoskeletal modeling in pathologic gait of adult stroke survivors (Allen et al., 2013; Meyer et al., 2016). Most of these simulation studies have focused on tracking ideal synergy activation patterns as part of the optimization (Allen et al., 2013; Sartori et al., 2013; Gonzalez-vargas et al., 2015; Meyer et al., 2016; Serrancolí et al., 2016), which has allowed for performance similar to EMG tracking simulations, while reducing the number of input parameters. These methods require extensive model calibration achieved by adjusting model parameters like muscle activation delays, EMG scale factors, and tendon slack lengths such that the models closely match experimental kinematics and kinetics. These procedures are time and computationally expensive. One study applied synergy controls without EMG tracking and found better calculation of joint loads than individual EMG alone; however, this model was also highly calibrated for a single individual (Walter et al., 2014). Another study in the upper limb used optimization of synergy activations

to model muscle activations during three-dimensional force generation and found that synergies better represented EMG data than independent muscle optimization (Borzelli et al., 2013).

The goal of this research was to evaluate whether constraining simulated muscle activations to an individual's synergies calculated from experimental EMG data can improve estimates of muscle coordination for both TD children and children with CP. We hypothesized that the similarity between EMG data and activations for traditional SO methods would be lower for children with CP than TD peers, due to altered motor control. By specifying and constraining muscle activations to an individual's synergies, we hypothesized that the similarity between EMG data and modeled activations would improve for both groups. This investigation examines whether synergy-based constraints alone, without changes to the model properties, can be used to improve fidelity of neuromusculoskeletal models to inform clinical or rehabilitation applications.

### MATERIALS AND METHODS

#### Participants

We retrospectively analyzed clinical motion analysis data collected at UZ Pellenberg, Belgium, for six children with CP (four males, age = 10.0 ± 3.3 years, mass = 33.2 ± 13.6 kg, height = 1366 ± 233 mm) and six TD children (three males, age = 8.9 ± 1.1 years, mass = 29.2 ± 1.8 kg, height = 1334 ± 39 mm). All children with CP were in Gross Motor Function Classification System (GMFCS) Levels I or II. Marker trajectories were tracked using a 10–15 camera VICON system (Nexus 1.8.4. Vicon-UK, Oxford, United Kingdom) sampled at 100 Hz. Each trial consisted of barefoot walking at a self-selected speed on a 10 m walkway. Ground reaction forces were collected using two AMTI force plates sampled at either 1000 or 1500 Hz. The number of over ground walking trials ranged between 4 and 8 for CP and 3 and 10 for TD.

#### Electromyography

Surface EMG data (Wave Wireless EMG, Cometa, Bareggio, Italy) were collected at either 1000 or 1500 Hz from eight muscles bilaterally (gluteus medius, rectus femoris, vastus lateralis, medial hamstrings, lateral hamstrings, tibialis anterior, gastrocnemius, and soleus) during clinical gait analysis. Because we were using retrospective clinical data, not all muscles were recorded for every trial and, for some individuals, a single muscle was missing from all trials (right vastus lateralis in CP03, CP04, and CP05, left tibialis anterior in TD02, and left rectus femoris in TD04). Raw EMG data were bandpass filtered between 20 and 500 Hz upon collection. We calculated a linear envelope for each muscle by high-pass filtering at 20 Hz, rectifying the data, and lowpass filtering at 6 Hz (Shuman et al., 2017). Prior to calculating synergies, we concatenated the middle 80% of EMG data for all available trials for each participant to maximize the amount of data for synergy analysis while removing periods of transient acceleration or deceleration near the beginning and end of each trial (Oliveira et al., 2014; Shuman et al., 2017). Each trial contained three to five strides of EMG data. The concatenated data were down-sampled to 100 Hz and scaled to a peak amplitude of one for each muscle.

## Synergy Analysis

For each individual, we calculated synergies with weighted non-negative matrix factorization (WNMF) using the Matrix Factorization Toolbox (Kim and Park, 2007; Li and Ngom, 2013) in Matlab (MathWorks, Inc., Natick, MA, United States) from the concatenated EMG data. We have previously used WMNF to accommodate clinical EMG data with poor or missing channels by assigning a weight of zero to those data points, allowing us to maximize data for synergy analysis (Shuman et al., 2018, 2019). Aside from the missing EMG channels noted above, all muscles were recorded in at least two trials within the concatenated session. WNMF numerically identifies a set of synergy weights (Wmxn) which are activated (Cnxt) such that the processed EMGmxt data are approximated, where m is the number of muscles (7 or 8), n is the number of synergies (2–5), and t is the number of time points in the concatenated EMG session:

$$\text{EMG} = \text{W} \ge \text{C} + \text{error}$$

Synergies were calculated for each side (unilaterally) using the following WNMF settings: 50 replicates, 1000 maximum iterations, 1 × 10−<sup>4</sup> minimum threshold for convergence, and 1 × 10−<sup>6</sup> threshold for completion. Synergy weights and activations were scaled such that the maximum weight in a synergy was one. Similar to prior research, reconstruction of the EMG data by n synergies accounted for more EMG variance in CP than to TD for all numbers of synergies (**Figure 1**) (Steele et al., 2015a).

#### Musculoskeletal Modeling

We used marker trajectories from an extended marker set, based upon the Plug-in-Gait (PiG) model, to scale a generic 19 degree-of-freedom and 92 musculotendon actuator model in OpenSim version 3.2 (Delp et al., 1990, 2007; Anderson and Pandy, 1999). We used inverse kinematics to calculate joint angles by minimizing the error between the experimental markers and virtual model markers. The average RMS marker error was 0.92 ± 0.19 cm and the average maximum marker error was 2.64 ± 0.97 cm (Hicks et al., 2015). The force plate data used in this study had a threshold applied upon collection where forces under 25 N were not recorded. Thus, to avoid dynamic inconsistencies in our models occurring near heel strike and toe off, we limited our investigation to single-limb stance. We performed a residual reduction analysis to improve dynamic consistency in our models by making small adjustments to joint angles and adjusting the position of the center of mass in the lumped head, arms, trunk, segment (Thelen and Anderson, 2006). A total of 88 simulations of single-limb stance phase (44 CP, 44 TD) were generated. Swing phase was evaluated for each limb when the opposite limb was in stance. The number of simulations per individual ranged between 3–13 for CP and 5–10 for TD.

We calculated simulated muscle activations using two methods (**Figure 2**). First, we used the standard SO algorithm

in OpenSim (Anderson and Pandy, 2001). SO estimates muscle forces that satisfy joint inverse dynamics at each point in time while accounting for muscle force-length properties. The cost function employed by SO minimizes muscle stress as the sum of squared muscle activations (Crowninshield and Brand, 1981; Kaufman et al., 1991; Anderson and Pandy, 2001). To evaluate whether constraining to synergies improved estimates of muscle activity, we used the synergy optimization (synSO) plug-in previously described by Steele et al. (2015b). SynSO allows the user to specify weighted groups of muscles to be commonly activated while minimizing the sum of squared synergy activations. For each synergy, the synergy weights we calculated from experimental EMG data were applied to the corresponding musculotendon actuators for each muscle. Note that the synergy weights were calculated from concatenated EMG data representing all parts of the gait cycle (not just single-limb stance). Thus, synergy weights for the gastrocnemius were applied to both the medial and lateral gastrocnemius actuators, weights for the medial hamstrings were applied to both semimembranosus and semitendinosus actuators, weights for the lateral hamstrings were applied to both biceps femoris long head and short head actuators, and weights for the rectus

femoris, vastus lateralis, tibialis anterior, and soleus were all applied to their individual musculotendon actuators. As only 26 of the model's 92 musculotendon actuators were accounted for with EMG data, the remaining 66 musculotendon actuators were independently activated as in SO. For each trial, we evaluated SynSO for sets of two, three, four, or five synergies for each participant. Synergies were calculated and applied independently for each individual and each leg (e.g., in a foursynergy simulation, we calculated and used four synergies for the right leg and four synergies for the left leg). Constraining activations through SynSO to fewer synergies reduced the number of successful simulations. For all numbers of synergies, more simulations were successful in CP than TD (e.g., 95 vs 86% for five synergies; 45 vs 34% for two synergies).

#### Outcome Measures

To determine whether constraining to synergies resulted in simulated muscle activations that were more similar to measured EMG data, we calculated the cosine similarity between the filtered EMG data and simulated muscle activations for SO and SynSO. We determined that the similarity due to chance was 0.55, which we calculated as the average cosine similarity across all individuals and EMG channels to 1000 random vectors with a truncated Laplacian distribution (Tresch et al., 2006). Thus, cosine similarity was normalized such that the similarity due to chance was given a value of zero (e.g., a similarity of 0.55 would have a normalized similarity of 0.0). We examined the similarity for each muscle by concatenating the simulated activations from all trials and calculating the cosine similarity to the corresponding measured EMG data. For muscles modeled with multiple musculotendon actuators, we averaged activations for comparison to EMG. We calculated the average similarity for each participant across all muscles. We compared the average normalized similarity of estimated activations to EMG data between SO and SynSO, and between single-limb stance and swing for each algorithm. We also computed the change in summed muscle stress (overall and by muscle) as the summed activation of each muscle and computed peak activation of muscles. Descriptive statistics (median and IQR) were used to compare normalized similarity, muscle stress, and peak simulated activations.

#### RESULTS

The similarity of estimated activations and experimental EMG data was similar between SO and SynSO, but generally poor for both algorithms. The normalized similarity across single-limb stance and swing between EMG and simulated muscle activations from SO was higher in CP [median (IQR): 0.48 (0.17)] than in TD [0.36 (0.18)] (**Figure 3**). Normalized similarity for SynSO in CP was less than SO when fewer synergies were used, with values of 0.37 (0.10), 0.38 (0.09), 0.48 (0.17), and 0.47 (0.19) for two to five synergies, respectively. In TD, similarity from SynSO was less than in CP and was lower than SO with a similarity of 0.24 (0.21), 0.19 (0.16), 0.27 (0.24), 0.26 (0.19) for two to five synergies. For SynSO, estimates of muscle activity were more similar to EMG

data during single-limb stance phase (CP: 0.59–0.65, TD: 0.48– 0.51) than swing (CP: 0.36–0.61, TD: 0.26–0.32). For SO, there was no difference in single-limb stance [CP: 0.39 (0.26), TD: 0.34 (0.16)] or swing [CP: 0.38 (0.26), TD: 0.33 (0.21)] for estimates of muscle activity compared to EMG data.

Similarity to EMG data was highly variable between individuals. The gastrocnemius, soleus, and tibialis anterior were the most similar muscles to EMG recordings using SO (**Figure 4**). SynSO tended to improve the similarity to EMG for the plantar flexors (CP: median soleus similarity increased from 0.65 for SO to 0.76 for three SynSO) and decreased the similarity to EMG for the tibialis anterior (TD: median similarity decreased from 0.57 for SO to –0.24 for three SynSO). The similarity of the gluteus medius to EMG was higher in CP [SO: 0.72 (0.22)] than TD [0.37 (0.30)] but did not become more similar when SynSO was

employed. The hamstrings and rectus femoris tended to become less similar to EMG when constrained to SynSO for the TD group and had only small changes in CP. The vastus lateralis tended to be only as similar as chance using SO and became more similar to EMG for all numbers of synergies in CP, but was still poorly represented in the TD group (SO: –0.16; five SynSO 0.19). Examination of the activation patterns between SO and SynSO showed that most muscles were recruited to higher amplitudes in single-limb stance (**Figure 5**).

Simulated muscle stress increased in SynSO relative to SO for all individuals and was highest for the two synergy solutions (**Figure 6**) with an increase of 157% (72%). Simulated muscle stress is a rough estimate of energetic cost, indicating that the constraints of SynSO find solutions requiring greater effort. Muscles that were constrained to a synergy increased muscle stress by 72% (61%) for two synergies and 64% (45%) for five synergies compared to SO. Muscles that were not constrained to a synergy in SynSO increased muscle stress by 323% (235%) for two synergies and 147% (98%) for five, suggesting that constraining to synergies for muscles with EMG data led to greater dependence on non-constrained musculotendon actuators in the model. Despite the large changes in summed muscle stress, changes in peak activation were less than 10% for over 60% of the muscles across all numbers of synergies in SynSO.

SynSO caused changes in simulated activations both for muscles included within synergies and muscles that were independently controlled (no EMG data, **Figure 7**). Muscles that were constrained to synergies demonstrated the largest changes, with a normalized similarity between SO activations and SynSO of 0.17 (0.22) for two synergies and 0.54 (0.16) for five synergies. Muscles that were independently controlled had smaller changes, with a normalized similarity between SO and SynSO activations of 0.52 (0.09) for two synergies and 0.60 (0.14) for five synergies.

## DISCUSSION

We investigated whether constraining musculoskeletal simulations to an individual's synergies calculated from experimental EMG data could improve estimations of muscle activation from inverse dynamic simulations of gait. Across all subjects, SynSO caused changes in estimated muscle activation patterns compared to traditional SO. Compared to SO, estimated muscle activations using SynSO tended to better match EMG data during single-limb stance for both TD and CP individuals. However, SynSO also tended to estimate activations that were less similar to EMG data during swing, such that overall SynSO did not better estimate EMG data than SO. These differences may indicate that synergies better represent coordination patterns in stance than swing. For both algorithms, the similarity to experimental EMG data for both CP and TD groups was generally poor, emphasizing the need for new methods to model muscle activity for analyses of human movement. As modeling methods are used to inform rehabilitation or assistive device design, identifying the changes in modeling and simulation methods required to accurately capture muscle coordination will be critical to ensure that predicted effects will be relevant for a specific individual.

The correlations found in this study between EMG and SO were similar to those previously reported (Heintz and Gutierrez-Farewik, 2007; Blazkiewicz, 2013; Zuk et al., 2018b ˙ ; Veerkamp et al., 2019). These four studies demonstrate variability both between individuals and across muscles, similar to our results. The best-represented muscles during gait were the plantar flexors, while the worst represented muscles were the knee extensors and hamstrings, consistent with our results for both CP and TD (Heintz and Gutierrez-Farewik, 2007; Blazkiewicz, 2013; Zuk et al., 2018b ˙ ).

Selection of an appropriate number of synergies is challenging for this type of problem. To avoid biasing our results based upon and ad hocthreshold, we computed our results over a range of two to five synergies. We chose to apply a minimum of two synergies,

as a representation of gross flexion and extension which has been previously been found to well represent data in infants and individuals with CP (Dominici et al., 2011; Steele et al., 2015a). As additional synergies were added, muscle activation patterns within each synergy became more independent (e.g., the tibialis anterior was largely independent in the five-synergy solution for CP and TD), more closely representing the conditions in SO (**Figure 1**). Constraining to a smaller number of synergies led to higher levels of overall muscle stress, indicating an overall less optimal solution. As SynSO did not tend to improve correlation

find an optimal number of synergies needed for either group. Our results using synergies to improve estimations of muscle activation during gait contrasts with previous studies (Borzelli et al., 2013; Walter et al., 2014; Meyer et al., 2016; Serrancolí et al., 2016) which found generally good estimation of EMG

with EMG data for any number of synergies, we were unable to

with synergies. The differences between the previous work and our results here broadly fit into three categories: the optimization criteria, the challenge of relating EMG amplitudes to neural excitations, and generic musculoskeletal properties. In the prior studies of Walter et al. (2014), Meyer et al. (2016), and Serrancolí et al. (2016), EMG shape tracking was used as part of the optimization algorithm. In this study, we sought to model muscle activations through a modified SO cost function which minimized synergy activations squared, consistent with the optimization previously implemented by McKay and Ting (2012) and Borzelli et al. (2013). This cost was motivated by the traditional physiologically motivated cost functions which seek to minimize fatigue or load in individual muscles (Crowninshield and Brand, 1981; Anderson and Pandy, 2001; Ackermann and Van Den Bogert, 2010), while constraining the space of allowable muscle activations to specified patterns

FIGURE 6 | Increased sum of muscle stresses for SynSO: Muscle stress measured as the muscle activations squared increased for both TD and CP across all number of synergies. Increases in muscle stress were highest for two synergies and lowest for five synergies. The "+" represents outlier points (greater than the 75th percentile + 1.5<sup>∗</sup> IQR or less than the 25th percentile – 1.5<sup>∗</sup> IQR.

equals perfectly similarity. The "+" represents outlier points (greater than the

75th percentile + 1.5<sup>∗</sup> IQR or less than the 25th percentile – 1.5<sup>∗</sup> IQR.

of coactivation. The constrained search space resulted in higher muscle stresses than what is found with independent actuation, consistent with prior studies (McKay and Ting, 2012; Borzelli et al., 2013). We note that minimizing synergy activations squared alters the cost function, removing the direct physiological relationship to muscle stress. A prior investigation by McKay and Ting (2012) in cats suggests that minimizing synergy activations squared, as performed in this study, results in higher muscle stress than minimizing muscle stresses squared subject to synergy constraints. A further difference is that, unlike the prior studies (McKay and Ting, 2012; Borzelli et al., 2013), we did not have EMG data for all muscles in the model and thus allowed the unmonitored muscles to have their activations optimized independently.

The challenges in directly comparing EMG to musculoskeletal modeling have been well documented (Farina et al., 2004; Sartori et al., 2017) and include scaling EMG to peak neural excitations (Ting et al., 2012; Borzelli et al., 2013; Kristiansen et al., 2015), electromechanical delays (Durandau et al., 2018), as well as interpretation of EMG stemming from inter-step variability, crosstalk, cancelation, measurement orientation, and pre-processing decisions (Farina et al., 2004; Shuman et al., 2017). In this study, we modeled activations using subjectspecific synergies whose weights were derived from EMG data normalized by peak measured EMG amplitude during walking. This choice was necessitated by our use of retrospective data and represents the simplest implementation of synergies into musculoskeletal modeling. To compensate for the uncertain scaling parameters between EMG and neural activation, previous forward dynamic simulations have tracked activation patterns but allowed the relative weights of the modeled muscles to vary either through a minimization of muscle stress with synergy activation tracking (Neptune et al., 2009; Allen and Neptune, 2012) or as part of the initial EMG tracking calibration process (Walter et al., 2014; Meyer et al., 2016; Serrancolí et al., 2016). Alternate methods of scaling synergies experimentally, such as by a maximum voluntary contraction or the use of force-to-EMG measurements (Borzelli et al., 2013), require the collection and integration of additional data, significantly complicating the implementation. Although the choice of amplitude scaling prior to calculating synergies can impact the relative weights of muscles within a given synergy (Shuman et al., 2017), a recent investigation by Kieliba et al. (2018) found nearly identical synergy structures between EMG data normalized by maximum voluntary contractions or peak activations in healthy adults. The consistency of these synergy structures suggests that the relative weights of muscles within a synergy scaled by experimental data may only have a small impact on our results.

A key limitation of our ability to model muscle activations using SynSO is the lack of any electromechanical delay, which neglects activation/deactivation dynamics. In those studies that used EMG shape tracking, the electromechanical delay was also uniquely scaled for each muscle (Walter et al., 2014; Meyer et al., 2016) or applied from the literature (Serrancolí et al., 2016). In a post hoc analysis, we evaluated the impacts of including a delay between EMG and modeled activations of 10–100 ms but found inconsistent impacts on similarity between phases of the gait cycle and number of synergies included in the optimization. For the previous studies which used an SO-based algorithm, the investigations were limited to examining muscle activity during a isometric force generation task across a variety of directions (McKay and Ting, 2012; Borzelli et al., 2013), negating the impact of activation dynamics. Conversely, for dynamic tasks such as walking, in which a gait cycle may take approximately

one second, even a 50 ms delay may have substantial impacts on the similarity between simulated muscle activity and experimental EMG. This is likely a defining difference in the higher accuracy of estimated muscle activity in previous studies incorporating synergies compared to the results of this research.

A significant limitation of this study was the use of generic musculoskeletal models. While generic models have the advantage of minimizing the amount of data that must be collected for any individual, they achieve this by including sample-based assumptions about geometry (e.g., muscle attachment points, and bone geometry) and muscle properties (e.g., activation delays, maximum muscle forces, and tendon lengths). These assumptions can have large impacts on estimated muscle activations (Correa et al., 2011; Ackland et al., 2012; Serrancolí et al., 2016; Roelker et al., 2017; Sartori et al., 2017; Zuk et al., 2018a ˙ ; Hegarty et al., 2019) and may not represent individual properties (Zajac, 1989), especially for individuals not well represented by the population used to develop the models, such as children or individuals with CP (Barber et al., 2012; Barrett and Barber, 2013; Mudge et al., 2014; Handsfield et al., 2016). We found variable results across participants and muscles in both TD and CP groups emphasizing the limitations of generic musculoskeletal models to capture heterogeneity in our populations. To address this, previous studies examining synergies in musculoskeletal modeling that use EMG shape tracking tune musculotendon properties as part of the model calibration (Walter et al., 2014; Meyer et al., 2016; Serrancolí et al., 2016), but these parameters are difficult to validate. Other musculoskeletal studies incorporate imaging data to personalize bone and muscle geometry (Barber et al., 2011; Scheys et al., 2011; Kohout et al., 2013; Handsfield et al., 2016; Modenese et al., 2016; Sartori et al., 2017). Incorporation of subject-specific geometry and muscle properties may influence the utility of synergies in modeling muscle activations, but the degree of personalization required remains unclear.

This study demonstrated that muscle activations estimated from SO using generic musculoskeletal modeling does not accurately predict EMG profiles for children with CP or TD peers. Constraining activation patterns to experimentally measured synergies increased estimated muscle stresses, but did not improve the estimation of muscle activations for either group.

#### REFERENCES


These findings suggest that when generic musculoskeletal models are used, constraining muscle activations to synergistic patterns alone may not improve estimation of muscle activations during gait. Additional methods, such as tuning of muscle-tendon model parameters, may be required to create neuromusculoskeletal simulations that can accurately represent muscle coordination for rehabilitation or assistive device applications.

#### DATA AVAILABILITY STATEMENT

The datasets generated for this study and source code for the plug-in (Steele et al., 2015b) are available on request to the corresponding author.

#### ETHICS STATEMENT

The studies involving human participants were reviewed and approved by Commissie Medische Ethiek (KU Leuven). Written informed consent from the participants' legal guardian/next of kin was not required to participate in this study in accordance with the national legislation and the institutional requirements.

## AUTHOR CONTRIBUTIONS

BS, KD, MS, and KS conceptualized the study. BS analyzed the data. BS, MG, KD, MS, and KS interpreted the data, and drafted and revised the manuscript. All authors read and approved the final manuscript.

#### FUNDING

Research reported in this publication was supported by the National Institute of Neurological Disorders and Stroke (NINDS) of the National Institutes of Health under Award Number R01NS091056, the Dutch Organization for Scientific Research (NWO) VIDI (Grant No. 016.156.346 FirSTeps), by the Flemish Research Foundation FWO (Grant No. T-GIAT T003116N), and by internal KU Leuven funding (Grant No. OT/12/100).



simulation muscle force results. J. Biomech. Eng. 139:139. doi: 10.1115/1. 4035472



**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Shuman, Goudriaan, Desloovere, Schwartz and Steele. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.