ASSESSING BIPEDAL LOCOMOTION: TOWARDS REPLICABLE BENCHMARKS FOR ROBOTIC AND ROBOT-ASSISTED LOCOMOTION

EDITED BY : Diego Torricelli, Jan Veneman, Jose Gonzalez-Vargas, Katja Mombaur and C. David Remy PUBLISHED IN : Frontiers in Neurorobotics, Frontiers in Robotics and AI, Frontiers in Bioengineering and Biotechnology and Frontiers in Human Neuroscience

#### Frontiers eBook Copyright Statement

The copyright in the text of individual articles in this eBook is the property of their respective authors or their respective institutions or funders. The copyright in graphics and images within each article may be subject to copyright of other parties. In both cases this is subject to a license granted to Frontiers. The compilation of articles constituting this eBook is the property of Frontiers.

Each article within this eBook, and the eBook itself, are published under the most recent version of the Creative Commons CC-BY licence. The version current at the date of publication of this eBook is CC-BY 4.0. If the CC-BY licence is updated, the licence granted by Frontiers is automatically updated to the new version.

When exercising any right under the CC-BY licence, Frontiers must be attributed as the original publisher of the article or eBook, as applicable.

Authors have the responsibility of ensuring that any graphics or other materials which are the property of others may be included in the CC-BY licence, but this should be checked before relying on the CC-BY licence to reproduce those materials. Any copyright notices relating to those materials must be complied with.

Copyright and source acknowledgement notices may not be removed and must be displayed in any copy, derivative work or partial copy which includes the elements in question.

All copyright, and all rights therein, are protected by national and international copyright laws. The above represents a summary only. For further information please read Frontiers' Conditions for Website Use and Copyright Statement, and the applicable CC-BY licence.

ISSN 1664-8714 ISBN 978-2-88963-270-1 DOI 10.3389/978-2-88963-270-1

### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

### Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

## ASSESSING BIPEDAL LOCOMOTION: TOWARDS REPLICABLE BENCHMARKS FOR ROBOTIC AND ROBOT-ASSISTED LOCOMOTION

Topic Editors:

Diego Torricelli, Spanish National Research Council (CSIC), Spain Jan Veneman, Hocoma (Switzerland), Switzerland Jose Gonzalez-Vargas, Otto Bock (Germany), Germany Katja Mombaur, Heidelberg University, Germany C. David Remy, University of Michigan, United States

Citation: Torricelli, D., Veneman, J., Gonzalez-Vargas, J., Mombaur, K., Remy, C. D., eds. (2019). Assessing Bipedal Locomotion: Towards Replicable Benchmarks for Robotic and Robot-Assisted Locomotion. Lausanne: Frontiers Media SA. doi: 10.3389/978-2-88963-270-1

# Table of Contents

*05 Editorial: Assessing Bipedal Locomotion: Towards Replicable Benchmarks for Robotic and Robot-Assisted Locomotion* Diego Torricelli, Jan Veneman, Jose Gonzalez-Vargas, Katja Mombaur and

C. David Remy

*08 Golden Gait: An Optimization Theory Perspective on Human and Humanoid Walking*

Marco Iosa, Giovanni Morone and Stefano Paolucci

*15 Comparing Gait With Multiple Physical Asymmetries Using Consolidated Metrics*

Tyagi Ramakrishnan, Christina-Anne Lahiff and Kyle B. Reed

*27 Benchmark Datasets for Bilateral Lower-Limb Neuromechanical Signals From Wearable Sensors During Unassisted Locomotion in Able-Bodied Individuals*

Blair Hu, Elliott Rouse and Levi Hargrove


Diego Torricelli, Camilo Cortés, Nerea Lete, Álvaro Bertelsen, Jose E. Gonzalez-Vargas, Antonio J. del-Ama, Iris Dimbwadyo, Juan C. Moreno, Julian Florez and Jose L. Pons


Mark Vlutters, Edwin H. F. Van Asseldonk and Herman van der Kooij

*85 Fusion of Bilateral Lower-Limb Neuromechanical Signals Improves Prediction of Locomotor Activities*

Blair Hu, Elliott Rouse and Levi Hargrove


Daniel F. N. Gordon, Graham Henderson and Sethu Vijayakumar

*130 Leg Force Control Through Biarticular Muscles for Human Walking Assistance*

Maziar A. Sharbafi, Hamid Barazesh, Majid Iranikhah and Andre Seyfarth


Khai-Long Ho Hoang, Sebastian I. Wolf and Katja Mombaur


# Editorial: Assessing Bipedal Locomotion: Towards Replicable Benchmarks for Robotic and Robot-Assisted Locomotion

Diego Torricelli <sup>1</sup> \*, Jan Veneman<sup>2</sup> , Jose Gonzalez-Vargas <sup>3</sup> , Katja Mombaur <sup>4</sup> and C. David Remy <sup>5</sup>

<sup>1</sup> Spanish National Research Council (CSIC), Madrid, Spain, <sup>2</sup> Hocoma AG, Volketswil, Switzerland, <sup>3</sup> Ottobock SE & KGaA, Duderstadt, Germany, <sup>4</sup> Optimization, Robotics & BIomechanics (ORB), Institute of Computer Engineering (ZITI), Heidelberg University, Heidelberg, Germany, <sup>5</sup> Institute of Nonlinear Mechanics, University of Stuttgart, Stuttgart, Germany

Keywords: benchmarking, wearable robots, humanoids, walking, balance, prosthetics and exoskeletons

**Editorial on the Research Topic**

**Assessing Bipedal Locomotion: Towards Replicable Benchmarks for Robotic and Robot-Assisted Locomotion**

## INTRODUCTION

#### Edited by:

Florian Röhrbein, Technical University of Munich, Germany

Reviewed by: Qining Wang, Peking University, China

\*Correspondence: Diego Torricelli diego.torricelli@csic.es

Received: 11 April 2019 Accepted: 03 October 2019 Published: 25 October 2019

#### Citation:

Torricelli D, Veneman J, Gonzalez-Vargas J, Mombaur K and Remy CD (2019) Editorial: Assessing Bipedal Locomotion: Towards Replicable Benchmarks for Robotic and Robot-Assisted Locomotion. Front. Neurorobot. 13:86. doi: 10.3389/fnbot.2019.00086 Human-centered bipedal robots, such as exoskeletons, powered prostheses and humanoids are demonstrating increasing levels of functionality, reliability, and safety, and are now breaching the barrier of surviving in a rapidly evolving market. In this crucial process, the lack of accepted standards to evaluate the different facets of robotic performance is hampering the efficient introduction of these new technologies into the different application domains, each of them characterized by the needs, requirements, and regulations of diverse users. A systematic benchmarking methodology to assess robotic systems on a quantitative and reproducible basis is eagerly awaited, as demonstrated by the numerous workshops and discussions on this topic in the most relevant international forums of the last 5–10 years. Benchmarks allow comparing systems to each other and against accepted references, and are therefore useful not only to end-users in search for the most suitable solution, but also to developers aiming to identify and solve the critical shortcomings of their machines. Solid benchmarks usually arise from research results, and are eventually converted into international standards after an iterative process that can last years. It can be argued that the more effort is done by the research community in identifying effective benchmarking methods, the easier it will be to arrive at a complete and relevant set of standards that will improve and accelerate the introduction of new robots in people's lives.

In the field of bipedal walking and posture, a huge body of literature focused on understanding or replicating human performance. Yet, just a few works specifically focused on proposing reproducible metrics that could be potentially converted into benchmarks for human-centered robotic devices. This Research Topic is a first step in this direction. We gathered a total of 16 original research articles from the fields of gait analysis, motor control, simulation, robotics and rehabilitation, covering together most of the aspects of bipedal skills, with a clear focus on identifying measurable metrics of performance. We believe and hope that the works presented in this collection can demonstrate how the process of identifying useful benchmarks of human and human-like robotic motion is possible, and encourage the scientific community to join the efforts in this ambitious goal.

## ASSESSING, UNDERSTANDING, AND REPLICATING BIPEDAL LOCOMOTION

In science, quantitative measures are necessary to understand the basic principles behind behavior and to validate or create scientific hypotheses. In robotics, performance measures are mainly used to achieve—and demonstrate—a certain expected level of performance. With the advent of human-inspired robotics, which aims to integrate biological principles into human-like or human-compatible machines, these two facets become strongly intertwined. Replicating human motion into real-life machines requires a deep understanding of the neuromechanical mechanisms at the basis of human behavior and, at the same time, the achievement of levels of performance comparable with those of the human counterpart. Assessing the performance of bipedal systems, being them humans, robots, or the combination of both (as happens in wearable robotics), is therefore a necessary component to bridging the two sides of the same coin: understanding and replicating humanlike locomotion. The articles collected in this Research Topic provide a remarkable example of how these three aspects are strongly enlaced, and how they can be translated into domainspecific metrics.

Three articles of this collection specifically focused on understanding and quantifying human balance, which is a crucial aspect for both walking and standing. Vlutters et al. aim to understand how humans deal with balance perturbations during walking, by experimentally testing reactions after pushes with different amplitudes, direction, and timing. Using simple metrics based on foot placement location, placement timing, and COM velocity, they demonstrate that human response is highly dependent on the timing of the perturbation, and potentially usable as a benchmark of human-like balance reactions. Mergner and Lippi present a comprehensive framework to test the posture control of human-like bipedal systems. The framework, based on a deep knowledge of the sensorimotor mechanisms of human balance, describes the experimental procedures to replicate the most relevant physical disturbances, and the metrics to quantify both performance and human likeness. Liu et al. investigate the causal relationship between spatiotemporal asymmetries and impaired reactive control of balance, by using two main metrics, i.e., whole-body angular momentum and Floquet analysis. These two metrics may be useful in grasping stability measure in impaired and robot-assisted gait.

Five articles focused on metrics and criteria aimed to evaluate or predict the effects of walking with prosthetic devices. Hu et al., based on the fact that bipedal walking requires interlimb coordination, demonstrate that neuromechanical information from both limbs is necessary to improve the prediction of transitions between level walking and stairs or slopes. This work, which has potential benefits for the design of more intuitive and transparent lower limb prostheses, is linked to a datasets of neuromechanical signals from wearable sensors, recorded bilaterally, which has been made available to the community for benchmarking purposes (Hu et al.). Ramasamy et al. propose a novel and subject-specific modeling tool to predict the interaction between a residual limb and a prosthetic device, providing relevant evidence to improve the prediction the potential sites of deep tissue injury generated by the socketstump interaction forces. Ramakrishnan et al. present a new metric, called combined gait asymmetry metric (CGAM), which provides a unified index of gait asymmetry that combines 11 spatio-temporal, kinematic, and kinetic parameters. This metric has promising potential in the prosthetic field to assess and categorize asymmetries due to different natures, such as leg length or added mass at the leg. Ho Hoang et al. investigated a number of benchmarking criteria to assess the subject-specific strategy of maintaining stability during unimpaired and impaired (i.e., transfemoral amputees) walking. By using optimal control to dynamically reconstruct recorded motions, they showed that human-like stability can be explained by two main indicators, i.e., foot placement and the Residual Orbital Energy.

Five articles proposed metrics and models to estimate and predict the effects of the interaction between humans and lower limb exoskeletons. Mummolo et al. propose an algorithmic framework to evaluate the balance stability of exoskeletons in crutch-less configurations during flat ground walking. They propose new metrics based on center of mass and joint-space dynamics and applied them to the lowerlimb exoskeleton Mina-2, demonstrating how this method has interesting potential for the assessment of the future generation of self-balancing exoskeletons. Torricelli et al. proposes a subject-specific musculoskeletal model able to infer the relative motion between the exoskeleton and the human limbs from the kinematic data recorded from the exoskeleton. This approach may contribute to improve predictive control strategies and/or mechanical designs for better humanrobot interaction. Gordon et al. applied a set of metrics based on dynamic stability and metabolic cost estimations to exoskeleton-assisted walking under varying conditions (speed, floor inclination, exoskeleton assistance), selecting some of them as potential control variables for adaptive exoskeletons. Gandolla et al. propose an automatic and patient-specific calibration procedure able to detect the best setting control parameters of the lower limb exoskeleton Ekso. The method is based on the maximization of the Gait Metric index, which quantifies the similarity of muscle activations to a reference normative set. Sharbafi et al. propose a control algorithm for exoskeletons based on the virtual pivot point (VPP) concept and simulate its effects on a detailed neuromuscular model of human walking. They showed that, with one cable-driven biarticular actuator, the exoskeleton can reduce subject muscle activation and metabolic cost, while ensuring balance stability during walking.

A group of three articles focused on experimental and theoretical methods for the evaluation of human-like performance of bipedal machines, in either real or simulated environments. Stasse et al. investigated the effect of temperatures variation on the walking performance of the HRP-2 robot, computing a set performance indicators previously defined by Torricelli et al. (2015) across different tasks, such as standing on tilting and horizontally moving surfaces, and walking during pushes, uneven or inclined surfaces, with the aim of testing the capability of a humanoid robot in real-life situations. Okajima et al. developed a control algorithm able to generate different human-like bipedal behaviors by adjusting a few signals in a symbolized control space, with the help of a tacit learning approach. This work has potential benefit not only in simplifying the control of a humanoid robot, but also in understanding the mechanisms at the basis of human-like walking and standing. Iosa et al. propose the golden ratio as the trade-off between advancement and equilibrium managed during walking, providing additional theoretical evidence in favor of the golden ratio as a descriptor of harmonic gait. This approach may be extended to assess the harmony in humanoid robotic walkers or neurorobots for rehabilitation.

## CONCLUSIONS AND FUTURE PERSPECTIVES

The large number of articles collected in this Research Topic are in itself proof of the substantial interest that different research communities have in the definition of reliable metrics of bipedal performance.

In the prosthetic field, we observed a clear need of metrics able to predict the interaction between the prosthetic device and human, as well as to measure the functional effects of walking in a real environment, in presence of transitions between different terrains. In the exoskeleton field, there is an increasing interest in identifying strategies for more adjustable/adaptable machines, under the perspective of control, actuation, or ergonomics. In the humanoids field, a promising trend is to test the performance in outdoor-like scenarios under a wide variety of perturbations, e.g., moving grounds or pushes.

The high heterogeneity of the approaches even further highlights the need for a unified framework in which these views are brought together and can be easily shared across the research community. Promising efforts in this direction are currently promoted by two European initiatives: the EUROBENCH project "European Robotic Framework for Bipedal Locomotion Benchmarking" (Torricelli and Pons, 2019, www.eurobench2020. eu) and the COST action CA16116 "Wearable Robots for Augmentation, Assistance or Substitution of Human Motor Functions" (https://www.cost.eu/actions/CA16116). Both initiatives aim to establish common methodologies and promote the discussion on standardized assessment of robotic systems, focused but not limited to bipedal locomotion.

## REFERENCES


This research field needs to address some important bottlenecks. First, the great majority of methods proposed in the literature are applied to individual devices or specific experimental settings. While many works claim a potential applicability of their metrics across different laboratory conditions or devices, very few of them actually have demonstrated such wider applicability. Nevertheless, this issue is critical for benchmarking, of which the principal goal is to compare different systems under reproducible conditions. This is particularly relevant considering the important trade-off between simplicity and reproducibility, validity of a benchmark and relevance of the outcome, considering the many subtle differences in applications, e.g., differences across industrial environments, or differences in pathologies across subjects.

We also observed an intrinsic difficulty in comparing bipedal robot performance with human performance. The concept of "human likeness," intuitively easy to grasp, becomes hard to define under a rigorous and systematic way, especially in robotics systems that have intrinsic differences (e.g., kinematic configuration) with humans, or when the human is not the golden standard for performance (similarly to the case of chess players). New challenges and questions (e.g., is it useful to compare robots with humans?) may be object of future research.

We believe that reaching an international consensus on these topics will be extremely beneficial to boost the process of finding reliable methods to test and compare different systems and identifying robust metrics to measure the Technology Readiness Level (TRL) of new robotic solutions.

## AUTHOR CONTRIBUTIONS

DT drafted the initial version of the manuscript. All authors have made substantial and iterative contribution to the work, and approved its final version for publication.

## ACKNOWLEDGMENTS

We thank all authors contributing with their work to this Research Topic. We also acknowledge all the European Projects that have supported past and current research in this topic: the FP7 Projects H2R (grant 600698), BALANCE (grant 601003), KoroiBot (grant 611909), WALK- MAN (grant 611832), BioMot (grant 611695), and the currently ongoing H2020 Project EUROBENCH (grant 779963), and COST Action CA16116.

**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Torricelli, Veneman, Gonzalez-Vargas, Mombaur and Remy. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Golden Gait: An Optimization Theory Perspective on Human and Humanoid Walking

### Marco Iosa\*, Giovanni Morone and Stefano Paolucci

Clinical Laboratory of Experimental Neurorehabilitation, IRCCS Fondazione Santa Lucia, Rome, Italy

Human walking is a complex task which includes hundreds of muscles, bones and joints working together to deliver harmonic movements with the need of finding equilibrium between moving forward and maintaining stability. Many different computational approaches have been used to explain human walking mechanisms, from pendular model to fractal approaches. A new perspective can be gained from using the principles developed in the field of Optimization theory and in particularly the branch of Game Theory. In particular we provide a new insight into human walking showing as the trade-off between advancement and equilibrium managed during walking has the same solution of the Ultimatum game, one of the most famous paradigms of game theory, and this solution is the golden ratio. The golden ratio is an irrational number that was found in many biological and natural systems self-organized in a harmonic, asymmetric, and fractal structure. Recently, the golden ratio has also been found as the equilibrium point between two players involved into the Ultimatum Game. It has been suggested that this result can be due to the fact that the golden ratio is perceived as the fairest asymmetric solution by the two players. The golden ratio is also the most common proportion between stance and swing phase of human walking. This approach may explain the importance of harmony in human walking, and provide new perspectives for developing quantitative assessment of human walking, efficient humanoid robotic walkers, and effective neurorobots for rehabilitation.

#### Edited by:

Diego Torricelli, Consejo Superior de Investigaciones Científicas (CSIC), Spain

#### Reviewed by:

Erik Prinsen, Roessingh Research and Development, Netherlands Stefan Schuster, Friedrich Schiller, Universität Jena, Germany

#### \*Correspondence:

Marco Iosa m.iosa@hsantalucia.it

Received: 25 September 2017 Accepted: 08 December 2017 Published: 19 December 2017

#### Citation:

Iosa M, Morone G and Paolucci S (2017) Golden Gait: An Optimization Theory Perspective on Human and Humanoid Walking. Front. Neurorobot. 11:69. doi: 10.3389/fnbot.2017.00069 Keywords: walking, neuroscience, anthropometry, rehabilitation, divine proportion, golden section, fractal, game theory

## INTRODUCTION

Optimization theory is a branch of mathematics aiming at identifying the best choice, from some set of available alternatives, that optimizes (maximizes or minimizes) a specific target function (Asghar Bhatti, 2000). A well-known application of optimization theory to human locomotion refers to the fact that the comfortable walking speed is, for healthy individuals, that minimizing the energy consumption (Ralston, 1958; Miller et al., 2012; Oh et al., 2012; Long and Srinivasan, 2013; Seethapathi and Srinivasan, 2015).

Game Theory is a branch of Optimization Theory in which there is not just one function to optimize, but there is the need to identify the best compromise among some entities involved into the problem (Kolokoltsov and Malafeyev, 2010). Game Theory has become a large and powerful theoretical framework providing mathematical models for predicting the choices of rational entities (usually called players) in conflict or in cooperation tasks (Rapoport, 1974; Sanfey, 2007). Mainly used in psychology, economy, political science, logic, computer science, Game Theory has also been enlarged to biology (Maynard Smith, 1982). Following this approach, game theoretical methods have been used in biochemistry and biophysics (Schuster et al., 2008), with some studies considering cells (Gatenby and Vincent, 2003) and even molecules (Bohl et al., 2004) as ''players'' working together or being in competition for the same objective.

The idea of applying game theory to human walking proposed in this article originates by the observation that current advances in these so different research fields reported the same solution for two apparently different problems. In fact, the same equilibrium point was found in human walking and Ultimatum game: this point coincides with the so-called golden ratio.

The golden ratio (φ) is the solution of the problem already reported by Euclid in III century B.C. to cut a given straight line so that the proportion between the shorter part to the longer one is the same as the longer part to the whole. It is an irrational number already found in many physical, biological fractal structures that are self-organized so that the larger-scale structure resembles the subunit structure (King et al., 2004; Yamagishi and Shimabukuro, 2008). In fact, it was found in structures of animal bodies (Livio, 2003) and plants' leaves (Okabe, 2011), in the solar systems (Lombardi and Lombardi, 1984), replicated in architecture (Hemenway, 2005) and in certain musical rhythms (Garland, 1995), as well as in financial market patterns (Agaian and Gill, 2017). In humans, harmonic proportions have been found in the physiological activity of the heart (Yetkin et al., 2013) and in anthropometry (Davis and Altevogt, 1979), as depicted in figurative arts (Hemenway, 2005). In general, the golden ratio has been found as the best choice for many biological processes (Bartl et al., 2010; Yetkin et al., 2013; Schuster et al., 2017).

## GAME THEORY: THE ULTIMATUM GAME EQUILIBRIUM

The Ultimatum game is a model game developed for analyzing fairness, and it is one of the most famous paradigms of Game Theory. In this game, two players must divide a sum of money: the proposer has to specify this division and the responder has the option of accepting or rejecting the offer (Sanfey, 2007). If the offer is accepted, the sum is divided as proposed, otherwise neither player receives anything. A rational intelligence should accept any offer and, knowing this, the proposer could offer the smallest nonzero amount (this choice is called Nash equilibrium). However, most of responders reject small proposals because judged unfair. Another point of equilibrium predicted by game theory is an offer split in 50% and 50% proportion (equipartition).

But most of the studies reported that the two solutions reported above are not the most common, and the average offer ranges around 62% for player 1 and 38% for player 2 (Oosterbeek et al., 2004; Henrich and Silk, 2013).

Some recent researchers (Schuster, 2017; Suleiman, 2017) noted that this division, chosen by subjects as the fairest one, is very close to the value of the golden ratio, and it was due to the fact that responders may tend to accept an amount that is in a proportion with that taken by proposer when this proportion is

FIGURE 1 | Schematic representation of the Ultimatum Game of Game Theory. Above the amount taken by player 1 (red line) and that taken by player 2 (blue line). The dash line is the Nash equilibrium (chosen by completely rational intelligence that accepts each offer higher than zero), the solid line the equipartition (fifty-fifty), the dot line the golden ratio solution (golden equilibrium, that observed in experimental data). Below, the offer and gain are expressed as proportions: red line is the offer ratio (expressed in percentage of whole amount) and the blue line is the gain of player 2 (with respect to that taken by player 1). The equilibrium is given by golden ratio solution (partition: about 38.2% for player 2 and 61.8% for player 1): player 2 tends to accept an offer of the minor fraction of φ (38.2%) because they accept a proportion of that taken by player 1 if this latter sum is in the same proportion with the whole amount.

the same between the latter part remaining at the proposer with respect to the whole amount.

**Figure 1** theoretically depicts the Ultimatum Game. This ''equality of fractions'' is conceivably perceived by both players as the fairest asymmetric division (Schuster, 2017).

The Golden Ratio plays a role also in other games analyzed in game theory (Camerer et al., 2004; Berg et al., 2015) and in the so-called justice evaluation function (Jasso, 2007). The Ultimatum Game is even used in economics (Güth and van Damme, 1998), and as its solution, the golden ratio has been reported as an example of ''Economic Harmony'' (Suleiman, 2017).

## CURRENT ADVANCES IN HUMAN WALKING: THE GOLDEN GAIT

Many different computational approaches have been used to explain human walking mechanisms, from inverted pendulum

model (Ivanenko et al., 2007) to fractal approaches (Hausdorff et al., 1995).

The inverted pendulum model refers to the pendular trajectory of center of mass and to the relevant transfer from potential to kinetic energy and its reverse (Ivanenko et al., 2007; McGrath et al., 2015). This trajectory can be seen in the above plot of **Figure 2** showing the stick diagram, a typical figure formed linking anatomical points and obtained using stereophotogrammetric systems (Ivanenko et al., 2007). At comfortable speed (about 1.4 m/s), locomotor system saves energy by exchanging forward kinetic energy and gravitational potential energy of the center of mass during the invertedpendulum oscillation of stance (Cavagna and Margaria, 1966), and by ballistically oscillating the limb as a compound-pendulum during swing (Mochon and McMahon, 1980). Thus muscle activity is only required to oppose gravity, maintain postural configurations in the face of interaction torques, and reintegrate energy losses during each cycle (Lacquaniti et al., 2012). Inverted pendulum is a simplified model in which a rigid rod represents the leg during stance phase, other more sophisticated models take into account also the stiffness and the elastic properties of the lower limbs (Lipfert et al., 2012).

The fractal approach refers to the idea that gait has a fractal structure, id est the larger-scale structure resembles the subunit structure (Hausdorff et al., 1995). The simplest examples of fractals are the structures based on the golden ratio. Recently, the Golden ratio has been found in gait cycle (GC) as the proportion between the stance duration (the longer part of a stride) and the swing duration (the shorter part). When the stance and swing are in the golden ratio, gait phases revealed a fractal structure based on the property of autosimilarity, with the same proportionality emerging between units and consecutive subunits of gait, resembling in each part the same whole structure (Iosa et al., 2016d) as follows:

$$\begin{aligned} \phi &= \frac{1+\sqrt{5}}{2} = \frac{\text{swing}}{\text{double support}} = \frac{\text{state}}{\text{swing}} \\ &= \frac{\text{stride}}{\text{state}} = \frac{\text{stride}+\text{state}}{\text{stride}} = \frac{2\text{striides}+\text{state}}{\text{stride}+\text{state}} = \dots \end{aligned}$$

With the expression ''golden gait'' we refer to a gait in which the above equations are respected. Recent researches showed as golden gait can be a bridge between pendular model and fractal approach of walking (Iosa et al., 2016b). This relationship is based on the strict intertwine between structure and functioning, been the anthropometric proportion between the stature and the distance of center of mass from the ground close to the golden ratio (Davis and Altevogt, 1979), as also represented in the ancient Greeks' sculptures (Di Dio et al., 2007).

However, the golden gait is not only related to the biomechanics of human body and its anthropometric proportions. In fact, the pendulum mechanism of walking should be considered as a forced oscillator for the need of rhythmic intervention of muscle activity and hence of nervous system (West and Scafetta, 2003). It has been hypothesized that a neural network formed by cerebellum, globus pallidum and central pattern generators of spinal cord is the generator of the golden ratio harmonic rhythm of walking (Iosa et al., 2016d). This hypothesis can be supported by clinical data showing as the golden ratio based rhythm is altered in patients with cerebellar ataxia (Serrao et al., 2017) and Parkinson's Disease (Iosa et al., 2016d), but not directly in presence of a damage of cortical areas due to stroke (Iosa et al., 2016a).

## APPLYING GAME THEORY TO HUMAN WALKING

During walking there is only one subject, the walking person, but he/she has to manage between the need of advancing and the need of maintaining stability without falling.

The relationships between the gait phases, expressed in percentage of the whole GC (100%), can be written as follows:

$$\begin{array}{rcl} \text{St} &=& 100 - \text{Sw} \\ \text{DS} &=& \text{St} - \text{Sw}' \end{array}$$

where St = stance phase; Sw = swing phase; Sw<sup>0</sup> = controlateral swing phase; DS = double support phase. For a symmetric walking Sw = Sw<sup>0</sup> and hence:

$$\text{DS} = \text{St} - \text{Sw} = 100 - 2 \text{Sw}$$

The advancement depends on the speed of walking, the efficiency of walking optimized around the speed self-selected by subjects as the comfortable one (Cotes and Meade, 1960). Walking speed is linearly proportional to swing duration (Hebenstreit et al., 2015). So the advancement depends on the ratio swing/stance (Sw/St), with the limit of Sw/St < 1, otherwise (Sw > St) walking becomes running, with the absence of any double support phase (Mann and Hagy, 1980).

Gait stability has been warranted by the fact that stance phase is longer than swing phase, that generates the presence of a double support phase (DS), with both feet on the ground (Perry, 1992). In fact, the maintenance of equilibrium is due to a dynamic stability in which the fall of center of mass has been controlled by the swinging leg arriving to touch the ground before subject falls (Mrozowski et al., 2007). This stability can be increased by prolonged double support phase (Perry, 1992). Hence, upright stability is strictly related to the duration of the double support (DS) with respect to swing duration (DS/Sw).

Given the above equations, it is possible to report the two ratios DS/Sw and Sw/St (related to stability and advancement, respectively) as function of the only swing phase (Sw) as follows:

$$\begin{array}{rcl} \frac{\text{DS}}{\text{Sw}} & = & \frac{100 - 2 \cdot \text{Sw}}{\text{Sw}} \\ \frac{\text{Sw}}{\text{St}} & = & \frac{\text{Sw}}{100 - \text{Sw}} \end{array}$$

 

These ratios can be seen as analogous to the offer of player 1 and the gain of player 2, respectively. Below plot of **Figure 2** shows the curves described by these two functions. This approach (as well as this plot) clearly replicates that reported for gain and offer ratios of the Ultimatum Game curves (**Figure 1**).

In the range of swing phase values in which DS/Sw > Sw/St, the stability is favored over advancement. On the contrary, if DS/Sw < Sw/St, the advancement is favored over stability.

In this subplot of **Figure 2** we also superimposed to the curves the experimental data extracted by a previous study on walking in healthy adults (Iosa et al., 2016a) and recorded using an optoelectronic system during comfortable walking (for details see Iosa et al., 2016a). Data recorded at slow and fast walking were not reported in that study, but collected in the same experimental session and reported here as original data that agree with those of literature (Perry, 1992).

Similarly to the responses observed for the Ultimatum Game, experimental data converged to this Golden equilibrium between advancement and stability.

Schuster (2017) suggested also an alternative approach for explaining the golden solution as the convergent equilibrium of the Ultimatum Game. This alternative way of explanation in terms of bargaining corresponds with the convergence of the following continued fraction that is another property of the golden ratio:

$$\varphi = 1 + \frac{1}{1 + \frac{1}{1 + \frac{1}{1 + \dotsb}}}$$

The results of this progressive division are: 3/2, 5/3, 8/5. . . All these numbers (1, 2, 3, 5, 8. . .) are consecutive Fibonacci numbers and the limit of the ratio between two consecutive Fibonacci numbers is the golden ratio.

Also this approach can be applied to human walking. In fact, the two limits are fixed by static posture (maximum stability, no advancement) and the limit of transition between walking and running (Sw = St = 50%). The following schema could be hence derived applying Fibonacci's numbers (1, 2, 3, 5, 8) to walking, using the ratio between the gait cycle (GC) and the stance (St):

$$\begin{aligned} \frac{\text{GC}}{\text{St}} &= 1 + \frac{1}{1 + \frac{1}{1 + \frac{1}{1 + \dotsb}}} \\ \text{Posture: } \frac{\text{GC}}{\text{St}} &= 1 \rightarrow \text{(state = 100\%)} \\ \text{Transition to Ranning: } \frac{\text{GC}}{\text{St}} &= 1 + 1 = 2 \\ &\rightarrow \text{(state = 50\%)} \\ \text{Slow walking: } \frac{\text{GC}}{\text{St}} &= 1 + \frac{1}{2} = \frac{3}{2} \rightarrow \text{(state = 66.6\%)} \\ \text{Fast walking: } \frac{\text{GC}}{\text{St}} &= 1 + \frac{1}{1 + \frac{1}{2}} = \frac{5}{3} \rightarrow \text{(state = 60\%)} \\ \text{Comforstable walking: } \frac{\text{GC}}{\text{St}} &= 1 + \frac{1}{1 + \frac{1}{1 + \frac{1}{2}}} = \frac{8}{5} \cong \rho \\ &\rightarrow \text{(state = 62.5\%)} \end{aligned}$$

The foot-off occurring at the minima located at 5/8 of the GC approximates the golden ratio with a difference of only 0.7% of stride duration.

The most important similarity between Ultimatum Game and the stance/swing trade-off in GC is that both need an asymmetric equilibrium point. In fact, in the Ultimatum Game, subjects usually assume that the proposer has some priority because he has the whole amount and he is allowed to choose the offer (Ichinose, 2012). Similarly, walking cycle is characterized by the presence of a double support phase, existing only if St > Sw.

## FUTURE DIRECTIONS: FROM HOMINIDS TO HUMANOID WALKING, PASSING THROUGH ROBOT-ASSISTED WALKING

The experience of million of years of evolution between extreme solutions (posture and running) in search for optimal bipedal walking, seems to have achieved the golden compromise between stability and advancement, muscle work and efficiency (Massaad et al., 2007). This is confirmed by studies on patients, showing as the golden gait minimizes the energy cost optimizing the transfer between potential and kinetic energy (Serrao et al., 2017), and on virtual model of walking, tested in different initial conditions, always converging to the golden ratio as an equilibrium point (Dzelaidini et al., 2014). Surprisingly, even chimpanzee during bipedal walking showed a proportion between stance and stride that is about 0.65 (Demes et al., 2015), so that the inverse proportion is close to the golden ratio. It has been even hypothesized that humans evolved from hominids in the actual antrophometric dimensions in golden proportion for favoring this harmonic golden gait (Iosa et al., 2016b).

It is well known that patients with neurological impairments have an alteration of the percentage duration of stance and swing (Perry, 1992), but only recently this alteration was put in relationship to a deviation from the golden ratio (Iosa et al., 2016a,d; Serrao et al., 2017).

For years, it has been suggested that rehabilitation should drive walking patterns of individuals with a gait impairment at resembling the patterns of healthy individuals as closely as possible. However more recent results revealed a more complex scenario. Hak et al. (2014) showed that step length asymmetry in individuals with a transtibial amputation are functional in terms of stability: a training aiming at recovering gait symmetry may affect dynamic balance during gait in these subjects, as theoretically suggested earlier by Merker et al. (2011). Our perspective study can contribute in clarifying that the optimization should take into account more target functions at the same time, as in Game Theory, not just one. Future research should focus on a more in-depth understanding of how gait impairments influence human locomotion and which target functions should be trained for a better recovery of gait autonomy finding a favorable trade-off that could even be different from physiological one.

A clear example is that of humanoid robots, in which the choice of bioengineers is guided by the cost of these robots. For most of humanoid robots, walking is quite different from that of humans because the distance between the center of mass and ground is usually maintained constant (Massaad et al., 2007). These humanoid robots use sophisticated motor control to walk smoothly while demonstrating appalling inefficiency with excessive energy cost: in this case the management between stability and advancement has been achieved favoring the former one for reducing the risk of fall and hence of damages. In humans, a walk with a flat trajectory of center of mass need muscles working in unfavorable conditions, wasting energy (Massaad et al., 2007). From this point of view, passive-dynamic mechanical walkers, moving up and down their center of mass, rival with humans in terms of efficiency (Collins et al., 2005). Recently, a bipedal robot has been developed using a golden ratio algorithm for reproducing golden gait obtaining a harmonious walking patterns similar to those of humans (Tez and Kus¸çu, 2017).

Robots for assisting human walking during rehabilitation have already been made for replicating a ratio between stance and swing close to the golden ratio: 60% and 40% (Hesse et al., 2000) or 62% and 38% (Volpini et al., 2017). However, this proportion was chosen just according to the principle of driving patients' gait patterns resembling the physiological patterns. It is worthy to investigate whether the golden ratio can be applied in robotic-aided therapy not just because it is a physiological pattern, but because it is the best harmonious trade-off between stability and advancement. The golden gait, in fact, seems to be a key of efficiency for locomotor control and stimulating the sensori-motor system at the frequency of the golden ratio may facilitate the restoring of the harmonic activity of neural circuits at the basis of walking.

Neurorobots already have the potential for accurately assessing motor functioning, therapy progresses and for providing patients with stimulation and real-time feedback on movement performance (Iosa et al., 2016c). This approach should take into account the golden ratio, using this proportion for stimulating and quantitatively assessing gait harmony (Torricelli et al., 2014; Saner et al., 2017), as suggested for assessing electrocardiographic patterns with respect to the deviation from the golden ratio of the cardiac cycle ratios (Ciucurel et al., 2018).

## CONCLUSION

Despite the golden ratio has often been reported as a ''magic number'', it is just the solution of a simple problem of a geometrical problem already reported by Euclid in III century B.C. and then also found as solution adopted by many biological and physical systems, including fractal structures and human physiology (Iosa, 2016). It has also been judged as the fairest proportion for geometrical figures (Green, 1995) and anthropomorphic sculptures (Di Dio et al., 2007) in psychological studies.

## REFERENCES


In the Game Theory, the golden ratio was found as the best equilibrium for the Ultimatum Game because judged as the fairest solution by the players. Following a similar approach, we showed that the golden ratio is also the best solution for managing advancement and stability during human walking optimizing bipedal efficiency.

As Suleiman (2014) noted and Schuster (2017) reported, the word ''fair'' has a double meaning, that of equitable and beautiful. For human walking, golden gait is also the most efficient. Once again, we can learn a lesson from Ancient Greek for whom a beautiful harmonic structure is strictly intertwined with an efficient virtuous functioning.

## ETHICS STATEMENT

This perspective article include some data already published (and here only re-analyzed) in Iosa et al. (2016b).

## AUTHOR CONTRIBUTIONS

MI developed the idea and the application of game theory to human walking. GM critically reviewed the original manuscript. SP coordinated and supervisioned the writing process. All the authors contributed to the discussion and the development of article structure.


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Iosa, Morone and Paolucci. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Comparing Gait with Multiple Physical Asymmetries Using Consolidated Metrics

### Tyagi Ramakrishnan, Christina-Anne Lahiff and Kyle B. Reed\*

Rehabilitation Engineering and Electromechanical Design Laboratory, Department of Mechanical Engineering, University of South Florida, Tampa, FL, United States

Physical changes such as leg length discrepancy, the addition of a mass at the distal end of the leg, the use of a prosthetic, and stroke frequently result in an asymmetric gait. This paper presents a metric that can potentially serve as a benchmark to categorize and differentiate between multiple asymmetric bipedal gaits. The combined gait asymmetry metric (CGAM) is based on modified Mahalanobis distances, and it utilizes the asymmetries of gait parameters obtained from motion capture and force data recorded during human walking. The gait parameters that were used in this analysis represent spatio-temporal, kinematic, and kinetic parameters. This form of a consolidated metric will help researchers identify overall gait asymmetry by showing them if the overall gait symmetry is improving and avoid the case where one parameter's symmetry is improving while another is getting worse. The CGAM metric successfully served as a measure for overall symmetry with eleven different gait parameters and successfully showed differences among gait with multiple physical asymmetries. The results showed that mass at the distal end had a larger magnitude on overall gait asymmetry compared to leg length discrepancy. It also showed that the combined effects are varied based on the cancelation effect between gait parameters. The metric was also successful in delineating the differences of prosthetic gait and able-bodied gait at three different walking velocities.

#### Edited by:

Diego Torricelli, Consejo Superior de Investigaciones Científicas (CSIC), Spain

### Reviewed by:

Rakesh Pilkar, Kessler Foundation, United States Zhijun Yang, Middlesex University, United Kingdom

> \*Correspondence: Kyle B. Reed kylereed@usf.edu

Received: 26 September 2017 Accepted: 17 January 2018 Published: 13 February 2018

### Citation:

Ramakrishnan T, Lahiff C-A and Reed KB (2018) Comparing Gait with Multiple Physical Asymmetries Using Consolidated Metrics. Front. Neurorobot. 12:2. doi: 10.3389/fnbot.2018.00002 Keywords: gait asymmetry, leg length discrepancy, distal mass, knee orthosis, prosthetic gait

## 1. INTRODUCTION

Human gait is a complex coordinated cyclic neuromuscular process that includes voluntary and involuntary aspects (Zijlstra et al., 1995). However, this cyclic process is frequently impaired following central nervous system damage, such as stroke, or physical changes, such as wearing a prosthetic. Physical and neurological changes often result in an asymmetric gait because the person's muscles and/or control actions becomes inherently asymmetric. Typically, human gait is represented by spatiotemporal, kinematic, and kinetic parameters obtained from analyzing motion capture and force plate data (Winter, 1995). The purpose of this study is to present a simple but versatile quantitative asymmetry metric that can be used to characterize the asymmetry of gait patterns as a whole.

Gait parameters offer quantitative data that can represent a person's gait. Using a quantitative data driven analysis offers an unbiased evaluation of the effects of multiple physical asymmetries that affect human gait. For this experimental study, the physical changes were selected based on the dynamic principles related to leg length and mass on a periodic system. Under appropriate conditions two dissimilar systems can be made to exhibit synchronized motion (Handzic et al., 2015). Handzic et al. demonstrated that two double pendulums with different masses at different locations and different lengths can exhibit symmetric motion. Human legs can be modeled as double pendulum systems, which allows for a simplified explanation of their synchronized dynamics (McGeer, 1990). Discrepancies in leg lengths and lower limb amputation disrupts natural propagation and dynamics that ultimately lead to asymmetric gait patterns. Further, asymmetric effects are also observed with changes in mass, such as the addition of external mass or a prosthesis. The study presented here also includes the effect of damping and stiffness at the knee to compare a larger range of physical changes that are not limited to altering the length and mass of limbs.

## 2. BACKGROUND

Previous research about asymmetric physical changes reveal a range of different effects on a person's gait. The literature review for this study looked at various physical changes such as leg length discrepancy (LLD), the addition of mass at the distal end of the leg, amputation, and stroke. It is important to remember that although these physical changes affect every person differently, they can all be characterized using the asymmetries of biomechanical gait parameters. It is not uncommon to find similar effects on gait asymmetry with different physical changes. To illustrate these differences and similarities, this literature review also focused on prior quantitative gait metrics and the algorithms used to discern between different types of gait.

## 2.1. Gait Patterns

Approximately 0.001% of people have some form of corrective gear due to LLD (Guichet et al., 1991). LLD may cause serious long-term consequences based on several variables such as the design of corrective devices, age, weight, posture, and level of activity (Gurney, 2002). An increase of 2 cm or 3.7% in leg length difference has dramatic overall gait asymmetry, especially in vertical reaction forces during push off and initial contact (Kaufman et al., 1996). Further, LLD causes abnormal changes in foot loading patterns and increases in joint torques/moments, which could lead to long-term effects (Perttunen et al., 2004). Finally, studies have also shown that LLD causes more overall strain on the body and leads to increased expenditure of energy (Gurney et al., 2001).

Limb mass, like limb length, plays an integral role in the dynamics of human walking. Adding mass on limbs, especially toward the distal end, brings about increases in metabolic activity and disrupts spatiotemporal symmetry (Browning et al., 2007). Adding mass at the distal end has been shown to force the user to change their walking posture by moving their arms in order to maintain balance (Donker et al., 2002). These effects may cause adverse changes in walking patterns in able-bodied symmetric individuals, but the addition of weight on the non-paretic limbs of stroke victims has shown improvement in walking speed, step length, cadence, and weight bearing in the paretic limb (Regnaux et al., 2008).

Studies show that prosthetic users exhibit less effcient and unnatural gait patterns (Gitter et al., 1995; Hoffman et al., 1997). This inefficiency is more evident in transfemoral amputees than transtibial amputees, which results in the users exerting a great deal of effort to compensate for unwanted motions (Huang et al., 1979). In some cases, simple solutions can correct irregular gait. When individuals with ataxia wore a 2 lb mass on their chest, unstable motions significantly decreased and the gait was more steady and efficient (Gibson-Horn, 2008). Since amputees are physically asymmetric, bringing about efficient and symmetric gait depends on multiple factors such as length, weight of prosthesis, type of socket, length of residual limb, etc. A study on unilateral transtibial prosthetic users shows that as the mass of the prosthetic gets closer to their intact shank weight, the subjects gait becomes more asymmetric (Mattes et al., 2000).

Stroke is one of the leading causes of disability among adults, affecting ambulation, performance of activities of daily living, communication, and cognition. Physical independence with respect to walking is characterized by improvement of walking function as defined by stroke survivors (Bohannon et al., 1991). However, only a minority of people (7–22%) are able to regain sufficient function to be considered independent community ambulators post stroke (Hill et al., 1997; Lord et al., 2004).

Gait retraining post-stroke typically focuses on two main outcome measures: velocity and symmetry. Walking velocity is used as an indicator of overall gait performance and can be used to differentiate the levels of disability among the stroke patient population (Perry et al., 1995; Lord et al., 2004). A gait speed of 0.8 m/s is considered the required minimum for community ambulation (Perry et al., 1995; Bowden et al., 2008), and typically people ambulate with a mean gait velocity of 1.14 m/s (Lord et al., 2004). Gait symmetry, in contrast, is used as a measure of gait quality (Dewar and Judge, 1990; Patterson et al., 2008). Normal gait measured among able-bodied individuals was found to be fairly symmetric in spatiotemporal, kinematic, and dynamic parameters with a range of up to 4–6% asymmetry between the limbs (Herzog et al., 1989; Titianova and Tarkka, 1995).

Gait after stroke becomes asymmetric (or hemiparetic) as a consequence of altered neuromuscular signals affecting leg motor areas, typically hyper extension at the knee and reduced flexion at the hip, knee, and ankle (Brandstater et al., 1983; Wall and Turnbull, 1986; Kelly-Hayes et al., 2003). Hemiparetic gait is characterized by a significant asymmetry in temporal (e.g., time spent in double-limb support) and spatial (e.g., step length) measures of interlimb coordination (Brandstater et al., 1983; Titianova and Tarkka, 1995; Balasubramanian et al., 2007). Propulsive force of the paretic limb is also reduced compared to the non-paretic limb, as are work and power of the paretic plantar flexors (Bowden et al., 2006; Balasubramanian et al., 2007). The significant decrease in propulsive force results in smaller overall step lengths, which in turn affects the patient's gait velocity. Finally, vertical ground reaction forces (GRFs) are decreased on the paretic limb relative to the non-paretic limb (Kim and Eng, 2003), reflecting Ramakrishnan et al. Comparing Asymmetries Using Consolidated Metrics

diminished weight bearing and balancing capabilities by the paretic limb.

When an individual with an asymmetric impairment walks with symmetric step lengths, other aspects of gait become asymmetric, such as the forces in the joints (Carpes et al., 2010; Handzic et al., 2015), the amount of time spent on each leg (Kim and Eng, 2003), and other temporal variables (Sadeghi et al., 2000; Highsmith et al., 2010), all of which can be detrimental to efficiency and long-term viability. Understanding how symmetry affects function could change the fundamental nature of clinical gait rehabilitation. The results from this research could also help tailor rehabilitation treatments to target each person's specific impairment. An overall analysis of multiple gait parameters can bring equilibrium to the different, and sometimes conflicting, requirements of gait. In order to distinguish and characterize the effects of multiple gait parameters, we use metrics that consolidate and quantify the overall change in gait. This paper demonstrates the effectiveness of these quantitative gait metrics in classifying multiple physical asymmetric changes.

## 2.2. Gait Metrics

Gait metrics have been in use clinically to evaluate a subject's progress throughout their rehabilitation process. These metrics can be classified based on the type of information required, which is of two types: qualitative (Steffen et al., 2002; McConvey and Bennett, 2005) and quantitative (Schutte et al., 2000; Schwartz and Rozumalski, 2008; Rozumalski and Schwartz, 2011). Most metrics focus either on kinetics or kinematics in order to categorize various walking patterns. However, there are some that can perform the analysis utilizing both kinetic and kinematic parameters (Chester et al., 2007; Hoerzer et al., 2015). Gait metrics have also employed statistical techniques such as principle component analysis (PCA) and singular value decomposition (SVD) to reduce the dimensionality of the biomechanical parameters (Muniz and Nadal, 2009). After processing the dataset, either the Euclidean or Mahalanobis distances (Muniz and Nadal, 2009) are found, which ultimately results in the score for the metric. Previous studies used Mahalanobis distances in conjunction with PCA to analyze kinematic and specific loading at knee joints. The precursor to this research study showed that the combined gait asymmetry metric (CGAM) used a symmetry index in conjunction with Mahalanobis distances. Without the restrictions of dimensionality reduction, CGAM served as a versatile gait asymmetry metric (Ramakrishnan et al., 2016).

## 3. METHODS

In order to analyze multiple asymmetric physical changes using gait metrics, two distinct datasets were collected from eleven different types of physical alterations. The physical alterations include a prosthesis with two different sockets on an amputee, healthy individuals with eight combinations of leg length and ankle masses fitted to the non-dominant leg, and a stroke simulator. The distinct datasets for the alterations were collected on amputee and non-amputee populations. The amputee data was collected while walking at three different speeds on two types


of sockets. The data collected on able-bodied subjects includes all of the perturbations.

## 3.1. Participants

The participants for this experiment consisted of 10 able-bodied individuals and a transfemoral prosthetic user who walked with two different sockets. **Table 1** describes the subject population. Both studies were conducted under approved University of South Florida IRB protocols. The subjects provided both informed and written consents to take part in the experiments. The transfemoral amputee was selected because the subject was a high functioning transfemoral prosthetic user and can walk at speeds that are comparable to able-bodied subjects. For the data analysis, we consider the prosthetic user to be two different subjects because the change in sockets alters the subject's gait to a large extent. The study involved the subject walking at 3 different speeds using 2 different sockets: the vacuum assisted suspension (VAS) brimless socket (Klute et al., 2011) and ischial ramus containment (IRC) (Kahle, 2013).

The able-bodied individuals had no prior injuries that would alter their walking patterns. The subject's walking speeds were determined by a 10 m walk test after which the height, weight, leg length, and age of each individual participant were recorded. The participants were put through a series of randomized increments of leg lengths, addition of masses at distal end, and a combination of both effects on the same leg, which was the left leg in all cases (Muratagic et al., 2017). **Table 2** shows the various perturbations of the experiment. Finally, the stroke simulator was fitted on their dominant side, which was the right leg for all participants.

## 3.2. Experimental Apparatus

The experimental data was collected in two separate trials, one on a single amputee and one on 10 able-bodied subjects. The motion capture and force plate data was collected using the Computer Assisted Rehabilitation Environment (CAREN), which was developed by Motek Medical, Netherlands, shown in **Figure 1**. The CAREN system incorporates a ten-camera Vicon (Edgewood, NY) motion capture system, 6◦ of freedom motion

#### TABLE 2 | Experimental procedure.


base developed by MOOG, immersive 180◦ panaromic screen for virtual reality environment, split belt treadmill, and continuous force plate systems developed by Bertec.

## 3.3. Experimental Procedure

The data for the prosthetic trial was collected using 30 reflective markers, which can be seen in **Figure 2**. This marker set was used to collect extensive data on the lower and upper body dynamics of the amputee as part of another study. In this study we only use the lower limb markers out of the 30 for the gait analysis while the others are used in another analysis. The unilateral right transfemoral amputee used two prostheses, shown in **Figures 3B,C**, with different socket types, and all other components were identical. The amputee walked on both sockets at three different speeds: 0.5, 0.9, and 1.3 m/s. This was done to have a range of cadences that can represent both prosthetic and able-bodied users. The able-bodied subjects also walked at these three speeds for direct comparison. The socket systems used in the prosthesis were the IRC and VAS (Kahle et al., 2016). The IRC socket is designed to reduce pistoning and increase stability, but compromises on comfort while the VAS is designed more for comfort and aims to be dynamically efficient.

The able-bodied subjects were put through a series of 9 asymmetric changes, shown in **Figures 3D–L**, and a baseline symmetric gait, shown in **Figure 3A**. The subject's height, weight, leg length, and walking speed were recorded before beginning the experiment. The walking speed of the subject is recorded using a 10 m walk test over ground. This walking speed was the constant velocity at which the subject walked for the duration of the trials, except for the three different speeds discussed above. An 18 marker setup was used to capture the motion capture data for the able-bodied subjects. The marker setup for the lower limb is shown in **Figure 2**. The asymmetric physical changes are combinations of leg length changes and the addition of mass at the ankle. There were two levels of leg length alteration: a small height change of L<sup>1</sup> = 27 mm and large increase of L<sup>2</sup> = 52 mm. The small and large mass added at the distal end weighed M<sup>1</sup> = 2.3 kg and M<sup>2</sup> = 4.6 kg. The leg length was chosen to reflect a larger than 2 cm change in leg length which is detrimental according to literature. We used a linear relationship x and 2x to select the larger leg length. Similarly the mass was chosen based on a previous PDW study that used a linear selection method (Handz˘ic and Reed, 2013 ´ ). In addition to these changes, the subject's normal walking pattern was recorded before and after all the perturbations. The leg length and mass changes were added to the non-dominant leg of the subject to compound the asymmetric effect (Muratagic et al., 2017).

Following this trial the subject was fitted with a variable stiffness and damping knee orthotic device, which is also known as the stroke simulator (SS) (Lahiff et al., 2016). The SS is used to simulate the damping and resistance at the knee joint felt by stroke patients. The knee joint of the stroke patient has a damping effect due to the imbalance in control of the anterior and posterior femoral muscles. Stroke victims also experience stiffness/resistance to flexion of the knee joint due to the over excitation of the rectus femoris and lack of control of the posterior femoral muscles that render the knee in a constant state of extension. The device is a modified knee orthosis with a rotary damper of ζ = 8,898 g-cm-s/◦ for the damping effect and a torsional spring of K = 0.457 kg/mm for the stiffness effect. The device was fit on the subject's dominant leg. This is because the dominant limb is less coordinated and hence, exhibits the maximum asymmetric change (Sadeghi et al., 2000). The subject then walked with the SS for 10 min to adapt to the device's dynamics. Then the device was removed and the subject walks for another 2 min to measure any after effects due to the asymmetric change applied at the subject's knee.

## 3.4. Data Analysis

The motion capture and force plate data gathered from the CAREN system is used to perform the gait analysis. The gait analysis was performed using a MATLAB script that evaluates the spatiotemporal, kinematic, and kinetic parameters from the raw coordinate and force data for each perturbation. Once the parameters are analyzed, their differences are evaluated for each step using the symmetric index formula (Herzog et al., 1989). This asymmetry data is then used to obtain the Combined Gait Asymmetry Metric (CGAM) (Ramakrishnan et al., 2016), which is a single number representing an Index/score for the level of asymmetry. The study further compares the CGAM to the machine learning grouping metric with the help of LibSVM library (Chang and Lin, 2011). **Figure 4** shows the complete setup for the development of the metrics.

The CGAM is a simple metric that uses the Mahalanobis distance from ideal symmetry to the data points obtained from gait analysis. Mahalanobis distances are calculated in multi dimensional datasets such as the calculations performed on the 11 gait parameters, shown in **Figure 4**. The formula for calculating the CGAM distance is shown in Equation (1). The equation presented in this article is modified from the previous version of CGAM (Ramakrishnan et al., 2016). This formulation provides more of a weighted means approach to decrease the

spine; PSIS, posterior superior iliac spine; FEM, femur; TIB, tibia; STRN, sternum; SACR, sacral wand marker; SHO, shoulder.

subjects are secured to the safety cage of the CAREN for their safety.

variability of results for the same gait parameters. This is achieved by dividing the original equation with the summation of the inverse covariance matrix. This method eliminates an extra step of dimensionality reduction that is carried out by algorithms such as Principle Component Analysis (PCA). Although PCA can help reduce the computational burden of multi dimensional datasets, it does so at the expense of losing information. CGAM's procedure analyzes the datasets without any loss in information and provides an overall perspective of the gait asymmetry based on biomechanical parameters. Further, the multiplication of the covariance matrix provides a weighted system that allows the metric to pick up on important changes in asymmetry among all the gait parameters.

$$CGAMDistance = \sqrt{\frac{(Data) \ast in\upsilon(\Sigma) \ast (Data)'}{\sum (in\upsilon(\Sigma))}} \qquad (1)$$


## 4. RESULTS

## 4.1. Calculating the CGAM Score

To further describe how the CGAM metric combines the gait parameters into one measure, the 11 gait parameters are shown in **Figure 5** with their respective CGAM score for four of the gait alterations. An important aspect for interpreting this metric is the covariance of the asymmetry matrix, which serves to weight the measures based on how much variability is present. From Equation (1) it is clear that the covariance of the data plays a major role in calculating the Mahalanobis distances from ideal symmetry. The measures that have more variability get weighted less and more consistent measures are weighted more heavily. These weights generally account for the variations in magnitudes across all the parameters. For example, pushoff and braking forces tend to show much higher magnitude asymmetry than other measures, but they also show more variability; scaling them based on their variability makes the influence comparable to the other measures.

Even though the Stroke Simulator in **Figure 5B** looks to have low asymmetry on many measures, the variability is high on those measures. The high variability means that some steps have large asymmetry. Specifically the stroke simulator data shows a large increase in the step length and hip moment asymmetry that are

FIGURE 5 | Comparing variation of mean, standard deviation, and CGAM metric among perturbations. (A) Normal walking without any alterations, (B) walking with SS or the variable stiffness and damping knee orthosis, (C) walking with big leg length and no weight addition at the ankles, and (D) walking with big leg length and weight. SL, step length; ST, step time; GRF, ground reaction forces; PF, push off forces; BF, braking forces; KA, knee angle; AA, ankle angle; HA, hip angle; AM, ankle moment; KM, knee moment, and HM, hip moment.

part of the resultant increase in the CGAM score magnitude. In contrast a large hip moment asymmetry seen in **Figure 5C**, with large leg length increase on the left leg, does not increase the CGAM score as much since the other parameters are in the nominal range. It is important to keep in mind that the CGAM scores are measured from perfect symmetry, so even normal walking with no alteration has some asymmetry, as shown in **Figure 5A**. **Figure 5D** shows the combined overall effect of a large mass at the distal end and a large increase in leg length, which results in a larger score compared to large leg length only. Thus, the overall CGAM score is higher than the normal walking shown in **Figure 5A**, even though some of the normal walking averages are fairly asymmetric.

## 4.2. Comparison of Alterations

**Figure 6A** illustrates the CGAM scores with the alterations applied to able-bodied subjects, and **Figure 6B** illustrates the comparison of the scores between able-bodied individuals and the transfemoral prosthetic user walking with two different sockets. It can be seen in **Figure 6A** that the addition of mass at the distal end makes gait overall more asymmetric than an increase in leg length. The effect of combining mass and LLD showed that a large mass at the distal end and a small LLD had the overall highest asymmetry. This demonstrates that the largest physical asymmetry, which in this case was the large mass at the distal end combined with a large LLD, may not necessarily lead to the largest deviation in overall gait asymmetry. On closer inspection of the individual gait parameters, it was revealed that there may be a cancellation effect with the large change in leg length and hence the overall CGAM value was lower. When the subjects walked with just the larger leg length, the step lengths were more asymmetric than the step times; however, the step times were more asymmetric than step lengths when only wearing a large mass. This kind of behavior is illustrated with the different perturbations and hence, these opposite effects tend to cancel each other out which results in a lower CGAM value. A previous study conducted by Muratagic et al. (2017) found that there were no significant effects observed due to the combination of LLD and distal. However, the study also observed some cancellation effects due to the combination of LLD and mass which showed that there are potential combinations that could result in a balanced gait pattern.

The changes related to prosthetics also had significant effects, as shown in **Figure 6B**. Wearing the SS affected the gait of all able-bodied subjects and caused a similar level of asymmetry in this metric compared to an amputee wearing a prosthetic. However, speed affected the gait asymmetry, and there was one speed on each of the prostheses that the subject was not comfortable with. Another observation from **Figure 6B** is that the IRC socket is more consistent in overall gait asymmetry but the subject felt less pain using the VAS socket, and gait with the VAS has a better overall gait at a high velocity (Kahle et al., 2016).

## 4.3. Statistical Analysis

A two-way repeated measures ANOVA analysis was performed with mass and leg length as independent variables and CGAM as the dependent variable. Mauchly's Test indicates that sphericity was not violated. The results of the ANOVA show that distal mass, F(2, 18) = 19.15, p < 0.005, and leg length, F(2, 18) = 5.72, p < 0.05, show statistically significance results in regards to CGAM. There was not a statistically significant interaction between the amount of mass added and amount of added leg length, F(4, 36) = 0.20, p = 0.49. This is similar to the effects observed in our lab's previous study (Muratagic et al., 2017). Further, the post-hoc comparisons for mass revealed significant difference between no mass and both small and large mass conditions. There was a statistically significant difference between no length and the large leg length condition. This analysis method matches our previous study, and the conclusions are similar. However, this analysis excludes the stroke simulator and different speeds, so an additional one-way ANOVA was performed.

A one-way repeated measures ANOVA analysis was performed with all 14 of the gait patterns shown in **Figure 6A** used as independent variables and CGAM as the dependent variable. This analysis was performed to examine the individual differences of all the gait patterns, unlike the two-way ANOVA

describe above that focused only on the added mass and height. Mauchly's Test indicates that sphericity was not violated. The results of the ANOVA show that there were a statistically significant differences in gait patterns, F(13, 117) = 10.21, p < 0.0001. The post-hoc test results are shown in **Figure 6A**. The normal gait pattern is statistically significant to the perturbation with large leg length and small mass and the gait pattern with the stroke simulator. Similarly, gait pattern with the subject walking at 1.3 m/s showed statistical significant difference between perturbation with large mass and gait with stroke simulator. This was to be expected since 1.3 m/s is close to the average self selected speed of all the subjects.

prosthetic user at three different speeds (0.5, 0.9, and 1.3 m/s).

## 4.4. Comparison to Machine Learning

Machine learning has been used in data driven industries to find patterns in large amounts of disparate datasets. The two datasets that were collected during this study represent gait with multiple asymmetric changes and hence, can be used to find patterns. For this study the LibSVM library (Chang and Lin, 2011) was used because it is easy to implement and it is widely used for research data. The machine is trained using labels and a training dataset. The labels are long vectors with a single number and the training datasets are ground truths. In the case of this study the labels were 0 and 1. Label 0 was used for the perfect symmetry which is a zero matrix with 11 columns and multiple rows. Label 1 was used for training asymmetry data. **Figure 7** shows the results of grouping predictions from 2 different asymmetry training datasets.

The pattern of the LibSVM grouping Index seen in **Figure 7A** is very similar to the pattern of the CGAM Mahalanobis distance in **Figure 6A**. Although the specific values cannot be compared directly because the modes by which they arrive at the results are inherently different, the trends highlight the differences between these two methods. The CGAM metric uses a simple Mahalanobis distance calculated from ideal symmetry while the more complex machine learning metric groups the data based on training datasets. LibSVM is not as reliable at this stage for being considered as a gait asymmetry metric because, based on the training datasets, the results vary substantially. This can be seen by comparing **Figure 7A,B** where the training datasets were different and the grouping predictions are completely different. This can be attributed to the different asymmetries present in the SS data and the weight/LL datasets. CGAM does not get affected by these differences and offers a more objective metric that can be used to classify the asymmetric changes. Another problem with Machine Learning as a metric is the requirement of large datasets.

## 5. DISCUSSION

This study demonstrates a simple metric that can help classify physical changes in human gait using the asymmetries of gait parameters. The results discussed above show that the metric is able to successfully categorize the extent of asymmetric changes caused by different perturbations. For example the CGAM scores for walking with the SS, which is designed to cause asymmetric gait, has a significantly larger value compared to the value that was gathered for gait immediately after the device was taken off. The after-effects of the SS are also more asymmetric than a normal gait pattern, which shows that the individuals adapted to the SS. Classification of gait based on overall symmetry will help clinicians keep track of a subject's progress, such as preand post- physical therapy regiments. The SS can be examined as an impeding exoskeleton. Hence, the gait wearing the SS and after removing the SS are both asymmetric overall. Conversely, in robot-assisted locomotion therapy, the outcomes are expected to be more symmetric (Lo et al., 2010). CGAM could provide researchers the tools to measure the overall change in gait asymmetry and modify their rehabilitation techniques to induce better gait patterns. This approach is different from prior research practices that limited their study to either spatio-temporal, kinematic, or kinetic data.

Another approach is analyzing an individual's gait parameters separately. This method could reveal insights on specific comparisons, but the complexity increases with the number of gait parameters. It is difficult to determine if the gait has improved when separately examining 11 parameters. The CGAM could make this evaluation easier since it can be used to represent a range of gait parameters, and it is not just limited to the 11 parameters that were used in this study. The subsets of the gait parameters can be made to fit the requirements of the clinicians such as reporting on improvements in only spatiotemporal parameters or only in kinematics. For example, in a prior study with CGAM, only 5 gait parameters were used to analyze the data (Ramakrishnan et al., 2016). The parameters were step length, step time, vertical forces, push off forces, and braking forces. Using these 5 asymmetry parameters, the CGAM was able to classify the different perturbations of leg length and addition of masses on separate legs. Although this metric used 11 gait parameters, the two-way ANOVA showed similar results to the analysis performed using five gait parameters in the study by Muratagic et al. (2017). This leads to one of the avenues for future research which involves determining the minimum gait parameters required to represent a gait pattern. CGAM is designed to be used for any number of gait parameter asymmetries representing multiple forms of data. However, many research studies typically do not come equipped with a CAREN or similar system to gather large amounts of data. One of the advantages of CGAM is that it can be potentially used on limited availability of quantitative asymmetric data. We are exploring the boundaries of this metric to be able to benchmark it for standard protocols for gait analysis.

Consolidated metrics such as CGAM and Machine Learning offer a unique and simplified perspective into categorizing gait data between multiple asymmetric datasets. CGAM has the potential of serving as a benchmark in representing overall gait asymmetry using multiple different parameters. The multidimensionality that CGAM offers makes it versatile and as shown in this article we can assess multiple gait patterns with different causations. These metrics have to be field tested in clinical trials in order to be formally proposed for clinical use. It is important to remember that these metrics could direct researchers to help patients achieve a well rounded gait. A well rounded gait can be characterized as a sustainable gait that an individual adopts that has the least overall asymmetry, not just a decrease in one parameter. Some parameters would remain asymmetric so that other parameters could become closer to symmetry. In case of a person who is physically asymmetric, this would mean adopting a gait and posture that will have a balance between all the gait parameters. This adaptation of a well rounded gait will help a physically impaired person to sustain a long-term gait that may not necessarily be as symmetric as an able-bodied gait, but it is subjectively beneficial to their specific physical asymmetry. A well rounded gait will alleviate long-term problems caused by asymmetric forces and moments acting on the person's body.

In this study the 11 parameters were chosen because they represent important gait parameter information and have clear symmetry values between each limb. With both metrics it is clearly seen that the addition of mass at the distal end has a larger effect on the overall symmetry than leg length discrepancies.

The combined effect of leg lengths and mass addition did not reveal a clear pattern but the results were as expected in most cases. For example, the combination of big leg length and mass had a slightly larger effect than small mass and leg length. However, the combination of a small leg length and big mass had a lot more deviation than big leg length and big mass. This is caused by the cancellation effects between gait parameters, which in turn resulted in a larger or smaller CGAM value.

There are some limitations associated with this method and study. There are many other gait patterns that were not discussed that arise from other gait impairments that should also be evaluated. Future studies can easily incorporate this metric in their analysis to compare the individual metrics to an overall picture of the gait. This will help in the generalization of this concept and also help to make the comparisons across different gait patterns more meaningful. This metric could also be optimized to find the most salient gait parameters to include; some of the ones used in this study may not be ideal and there may be others that are more beneficial to include.

The prosthetic gait at the three different speeds showed that the overall symmetry improves with increases in speed. It has been shown in literature that amputees achieve better spatiotemporal and kinematic symmetry at higher speeds, but at the expense of kinetic symmetry which can cause long-term degeneration effects (Nolan et al., 2003). We require a bigger patient population in order to gather all variations of prosthetic gait and leave that to future studies. The analysis provided in this study will improve further and can be more robust if a larger dataset from multiple patient population is used. The study presented in this article provides some proof into the efficacy of CGAM but it is limited by the small population size.

The CAREN is a versatile device that was used to collect all the data for this study and has been used in other similar studies (Ramakrishnan, 2014; Muratagic, 2015). To further understand the effects and dynamics of physical asymmetries, the split belt treadmill can be used to exaggerate asymmetries. Split belt treadmills are used to rehabilitate gait affected by hemiplegia by having the treads move at different velocities. This exaggeration of hemiplegic gait temporarily restores the person's gait closer to symmetry. However, successfully returning a person's gait to spatio-temporal symmetry does not necessarily guarantee an overall effective gait with a healthy ratio of symmetry between all gait parameters. To further explore how physical asymmetries combine, the split belt treadmill could be used in conjunction with an added mass and/or LLD.

## 6. CONCLUSION

Analyzing multiple physical asymmetries in one platform requires a special form of metric. This is because every perturbation of physical change that impairs an individual's gait has to be accounted for and kept track of following clinical procedures. The consolidated metrics such as CGAM and Machine learning can be quantitative data analysis tools that can help researchers keep track of a person's overall gait asymmetry. These metrics can be obtained using all gait asymmetry parameters such as spatio-temporal, kinematic, and kinetic or by using subsets and combinations of any or all of these parameters. This versatile platform allows researchers to have many options for generating metrics to represent the progress or regression of an individual over a period of training and time.

## AUTHOR CONTRIBUTIONS

TR designed and performed the experiments and analyzed the results. C-AL designed and tested the stroke simulator. C-AL also helped TR with the experiments and data analysis. KR advised C-AL and TR throughout the design, execution, and analysis. All authors participated in writing and revising the article.

## FUNDING

This material is based upon work supported by the National Science Foundation under Grant Number IIP-1602020 and the American Orthopedic and Prosthetic Association (AOPA).

## REFERENCES


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Ramakrishnan, Lahiff and Reed. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

## Benchmark Datasets for Bilateral Lower-Limb Neuromechanical Signals from Wearable Sensors during Unassisted Locomotion in able-Bodied Individuals

*Blair Hu1,2\*, Elliott Rouse3 and Levi Hargrove1,2,4*

INTRODUCTION

*1Center for Bionic Medicine, Shirley Ryan AbilityLab, Chicago, IL, United States, 2Department of Biomedical Engineering, Northwestern University, Evanston, IL, United States, 3Department of Mechanical Engineering, University of Michigan, Ann Arbor, MI, United States, 4Department of Physical Medicine and Rehabilitation, Northwestern University, Chicago, IL, United States*

Keywords: gait, locomotion, biomechanics, electromyography, benchmark

## *Edited by:*

*Diego Torricelli, Consejo Superior de Investigaciones Científicas (CSIC), Spain*

#### *Reviewed by:*

*Jan Veneman, Tecnalia, Spain Antonio J. del-Ama, National Hospital for Paraplegics, Spain Cristiano De Marchis, Università degli Studi Roma Tre, Italy*

> *\*Correspondence: Blair Hu blairhu@u.northwestern.edu*

#### *Specialty section:*

*This article was submitted to Bionics and Biomimetics, a section of the journal Frontiers in Robotics and AI*

> *Received: 25 September 2017 Accepted: 30 January 2018 Published: 19 February 2018*

#### *Citation:*

*Hu B, Rouse E and Hargrove L (2018) Benchmark Datasets for Bilateral Lower-Limb Neuromechanical Signals from Wearable Sensors during Unassisted Locomotion in Able-Bodied Individuals. Front. Robot. AI 5:14. doi: 10.3389/frobt.2018.00014*

The field of assistive robotics has experienced rapid growth in the number of and capabilities of wearable lower limb assistive devices, which include robotic exoskeletons, orthoses, and prostheses. These devices have shown promising potential to restore motor function to individuals with gait impairments by providing locomotion assistance. Although many devices have already demonstrated impressive performance in a variety of real-world conditions, comparing their performance objectively and improving their controllability remain challenging for several reasons. First, the outcome measures (e.g., joint kinematics, metabolic cost, clinical scores, and prediction accuracy) used by studies demonstrating improved walking ability with an assistive device are not consistent. Second, many studies only use treadmill walking or do not collect data from a variety of locomotor activities due to constraints in a device's mechatronic design and/or control system. Third, many devices are in the process of commercialization, so testing data are seldom shared with the research community. In addition, many devices implement their own unique control frameworks that are not generic enough to conveniently implement on other hardware. Therefore, we expect improving access to device-agnostic neuromechanical signals during walking-related activities (from which researchers could develop and test novel control strategies before implementation on hardware) will be valuable to the field of wearable lower limb assistive devices.

Meanwhile, many benchmarks for the biomechanics of able-bodied human locomotion without an assistive device have already been established, some of which are publicly available. The gold standard for high-resolution biomechanical gait analysis is marker-based optical motion capture with ground reaction force measurement. Decades ago, seminal work from Winter (1983) used these techniques to introduce an inter-subject biomechanical analysis of level ground walking (LW) at different speeds. Their normative gait dataset includes electromyography (EMG) and joint kinematic and kinetic patterns and has since been expanded by other researchers to include more subjects and strides (e.g., Kadaba et al., 1990; Kirtley, 2014). The steady-state biomechanics of other common locomotor activities such as ascending and descending stairs and sloped surfaces of different geometries have also been reported in separate studies using similar techniques but these data are not as accessible to researchers (e.g., McFadyen and Winter, 1988; Riener et al., 2002; Lay et al., 2006, 2007; Protopapadaki et al., 2007; Franz et al., 2012).

Human locomotion is most accurately quantified by joint kinematics, kinetics, and EMG using traditional laboratory-based instrumentation and techniques developed for biomechanical gait analysis. However, the exciting potential of wearable lower limb robotics lies in its promise to bring these devices closer to everyday life, where alternative techniques are required to more ubiquitously measure neuromechanical signals during walking-related activities. Methods to more freely measure human movement have been developed in the field of human activity recognition (HAR), which aims to use continuous streams of sensor data to recognize and monitor common activities of daily living such as sleeping, walking, exercising, and manipulating objects. As a result, HAR has produced an abundance of publicly available datasets. These repositories are valuable because they contain many different types of activity information from many subjects; however, they are not very suitable for more systematic characterization of normal locomotion. Sometimes, HAR datasets are collected from impaired populations or during more natural, but complex combined movements for which the ground truth activity is more ambiguous. Some datasets are collected using minimal instrumentation (e.g., smartphone only), which is convenient but incomplete. By contrast, others rely on non-portable instrumentation (e.g., optical motion capture or video), which is highly accurate but not representative of biomechanical signals accessible for controlling a device in a more ecological setting. Also, many only contain single modalities (e.g., kinematics but no EMG) and/or use lower sampling rates that may be insufficient for certain online control schemes.

To the best of our knowledge, there still does not exist a publicly available database of kinematic and EMG data simultaneously recorded from wearable sensors as able-bodied individuals freely transition between several distinct locomotor activities. To address some of these aforementioned limitations and provide relevant reference data for researchers in the field of wearable lower limb assistive devices, we introduce a device-agnostic benchmark dataset of bilateral neuromechanical signals called ENcyclopedia of Able-bodied Bilateral Lower Limb Locomotor Signals (ENABL3S). The dataset contains bilateral EMG and joint and limb kinematics recorded from wearable sensors for 10 able-bodied individuals as they freely transitioned between sitting, standing, and several walking-related activities [level ground, stair ascent (SA)/stair descent (SD), and ramp ascent (RA)/ramp descent (RD)]. Although these data are not intended to replace existing benchmarks for biomechanical gait analysis, we believe they still fill a gap between those benchmarks and HAR datasets by providing richer neuromechanical data collected from wearable sensors using a unified protocol for several distinct locomotor activities. In this data report, we summarize our methods for instrumenting subjects, collecting data, and post-processing for artifact removal and gait segmentation. We also present a summary of the types of locomotor activities and transitions captured by our protocol, validate our results, and conclude with suggestions for how other researchers in the field may benefit from this dataset.

## MATERIALS AND METHODS

## Instrumentation Setup

Ten healthy able-bodied subjects (seven male, three female; 25.5 ± 2 years; 174 ± 12 cm; 70 ± 14 kg) without any gait

impairments were recruited and completed the following protocol between January and February 2017. Before walking, subjects were instrumented with wearable sensors to measure bilateral lower limb muscle activity and joint and limb kinematics. EMG signals were recorded using bipolar surface electrodes (DE2.1; Delsys, Boston, MA, USA) from the same seven muscles in each leg: tibialis anterior (TA), medial gastrocnemius (MG), soleus (SOL), vastus lateralis (VL), rectus femoris (RF), biceps femoris (BF), and semitendinosus (ST). These muscles were chosen because they are in part responsible for hip and knee flexion/extension and ankle plantarflexion/dorsiflexion, movements that are commonly assisted by wearable devices. They are also relatively easy to target when facing the subject from in front and behind. The muscle sites were prepared by removing excess hair, and the skin was cleaned by mildly scrubbing with an alcohol wipe. Sensors were attached to the skin with a double-sided adhesive. Electrode placement was guided by palpation according to the Surface ElectroMyoGraphy for the Non-Invasive Assessment of Muscles standards and verified by having subjects perform maximum voluntary contractions (MVC). Subjects performed three repetitions of ankle dorsiflexion/plantarflexion and knee flexion/extension for both legs. EMG signals were amplified by 1,000×, hardware band-pass filtered between 20 and 450 Hz (Bagnoli 16, Delsys), and sampled at 1 kHz.

Joint kinematic signals (sagittal plane only) were recorded using electrogoniometers (SG150; Biometrics Ltd., Newport, UK) placed on the knee and ankle and sampled at 500 Hz. At the beginning of trials, the goniometers were zeroed while the subject was in the upright standing position. 6-DOF (tri-axial accelerometer and gyroscope) inertial measurement units (IMUs) were placed bilaterally on the subjects' thigh (below RF) and shank (adjacent to TA) and sampled at 500 Hz (MPU-9250; Invensense, San Jose, CA, USA). Goniometers and IMUs were secured to the subject using a combination of double-sided adhesive, elastic straps, and Coban self-adherent wrap. Another IMU was placed in a custom manufactured holster (tilted 20° from vertical) and worn around the waist with a belt. All signals were simultaneously recorded with a custom 16-bit data acquisition device that permitted multi-rate sampling. To facilitate integration with our custom data acquisition software, all wearable sensors were used in a tethered setup; as a drawback, fully instrumenting each leg took up to an hour. The full instrumentation setup with IMU orientations is shown for a representative subject in **Figure 1**.

## Data Collection Protocol

In an experimental session, each subject was barefoot and completed approximately 25 repetitions of a circuit consisting of sitting (S), standing (St), LW, ascending/descending a ramp with a 10° slope (RA/RD), and ascending/descending a four-step staircase (SA/SD) step-over-step. These activities were chosen because they encompass the different types of terrain likely encountered in community ambulation and were completed as a circuit in a 20 ft. × 30 ft. room for practicality and for increasing the number of repetitions. A platform (30″ tall) joined the staircase (7.75″ rise, 10″ run) and ramp (14 ft. long) to allow all possible transitions between these activities. Data from each circuit were divided into

two segments and recorded as separate trials. Odd-numbered trials consisted of S → St → LW → SA → LW → RD → LW → St → S. Even-numbered trials consisted of S → St → LW → RA → LW → SD → LW → St → S. The total distance walked for each continuous segment was approximately 45 ft. Trials during which sensors needed to be repositioned or the tether became tangled were excluded. Subjects were instructed to freely transition between locomotor activities at their self-selected speed, and breaks were routinely administered to avoid fatigue. The experimenter labeled the true locomotor intent of the subject using a key fob. Data collection took up to 2 h.

## Post-processing

Heel contact and toe off gait events for each leg were reliably identified by finding peaks in the mean-subtracted and low-pass filtered (first-order Butterworth, 6 Hz) sagittal plane angular velocity (GY) of the shank segment using a threshold-based method similar to Maqbool et al. (2016). Briefly, the largest peaks in angular velocity were first used to identify mid-swing events. Toe off events were identified by searching for peaks before each mid-swing event. Heel contact events were identified by searching for peaks after each preceding mid-swing event. Event switches were initially placed beneath the heel and first metatarsal of each foot, but they triggered many false negatives and positives in our setup perhaps due to mechanical wear and/or foot placement on the staircase. Therefore, they were only used for validating the IMU-based segmentation technique. Gait events corrupted by motion artifacts (i.e., pauses and trips) were excluded. EMG signals were high-pass filtered (sixth-order Butterworth) at 20 Hz, low-pass filtered (sixth-order Butterworth) at 350 Hz, and notch-filtered (sixth-order Butterworth, 6 Hz width) at 60, 180, and 300 Hz to attenuate motion artifact and ambient interference. Goniometer and IMU signals were low-pass filtered (sixth-order Butterworth) at 10 and 25 Hz, respectively. Joint velocities were indirectly computed by taking the central-difference numerical derivative of the joint position and added to the goniometer channels.

All signals were segmented into analysis windows beginning 300 ms before each identified heel contact or toe off gait event. Four additional 300 ms analysis windows near each identified gait event (delayed by 30, 60, 90, and 120 ms relative to each event) were used. For each window, we extracted features previously used in intent recognition for control of a powered knee-ankle prosthesis. Features for goniometer and IMU channels included the mean, SD, maximum, minimum, initial, and final values (Varol et al., 2010) (six features/channel). Features for EMG signals included the mean absolute value, waveform length, number of zero crossings and slope sign changes, and the coefficients of a sixth-order autoregressive model (Huang et al., 2005; Hargrove et al., 2008) (10 features/channel). There were a total of 23 sensors (14 EMG, 4 goniometer, 5 IMU), 52 channels (14 EMG, 8 goniometer, 30 IMU), and 368 features (140 EMG, 48 goniometer, 180 IMU).

## RESULTS

The data are saved in CSV format in subject-specific folders and are available to download from Figshare at https://doi.org/10. there is a metadata file, which catalogs the filenames, summary statistics (mean, SD, minimum, maximum) of each goniometer channel, and signal-to-noise ratios [ratio of maximum to baseline root-mean-square (RMS) voltage] of each EMG channel for each circuit. Subject-specific folders also include folders for the raw and processed data from individual circuits, a folder containing the processed EMG signals from all muscles during MVC trials, and a folder containing the features extracted from the five different 300 ms analysis windows (beginning 300, 270, 240, 210, and 180 ms before the gait events identified for each leg). Data from individual circuits also contain columns specifying the label of the true locomotor activity, the indices of heel contact and toe off gait events, and four-digit triggers denoting the outgoing and incoming locomotor activities and gait phases. The first row of each file is a header specifying the column order. The post-processed data from all trials are included for completeness although some trials 6084/m9.figshare.5362627. Within each subject-specific folder, include disturbances (e.g., pauses, trips, and missed transitions), which are noted in the metadata file. However, only gait events from disturbance-free segments of trials are reported and used for feature extraction. Feature data also contain columns specifying the corresponding leg-phase (1, right heel contact; 2, right toe off; 3, left heel contact; 4, left toe off) and the four-digit trigger. The first row of the feature data is a header specifying the column order. Ipsilateral refers to the side in which the gait event was detected (e.g., the right leg for right heel contact and right toe off events).

The overall composition of ENABL3S is shown in **Table 1**. For each subject, there were 530 ± 46 heel contact events and 536 ± 45 toe off events for each leg (mean ± SD) after excluding transitions to or from standing. Additional subject information and an explanation of nomenclature and numbering are also included on Figshare.

## DISCUSSION

ENcyclopedia of Able-bodied Bilateral Lower Limb Locomotor Signals represents a benchmark of bilateral lower limb neuromechanical signals recorded from able-bodied individuals using wearable sensors during unassisted locomotion. The purpose of introducing this dataset is not to replace existing benchmarks for biomechanical gait analysis of steady-state locomotion but to provide a publicly available set of rich biomechanical data from wearable sensors, representing a compromise between traditional techniques and methods from HAR. ENABL3S includes data from several distinct walking-related activities (with transitions), which we expect to be helpful for understanding patterns in normal locomotion and developing novel control strategies for wearable lower limb assistive devices.

In order to assess the validity of this dataset, we chose to compare our recordings (averaged across legs and all subjects) to previously reported biomechanical measurements of level

Table 1 | Characteristics of ENcyclopedia of Able-bodied Bilateral Lower Limb Locomotor Signals.


*The total number and proportion of gait events belonging to each type of locomotor activity are aggregated across all subjects.*

walking because these data are most accessible. Due to movement out of the sagittal plane and skin deformation/relative motion of the ankle goniometer, our measurements of ankle position were not considered biomechanically accurate signals. Nonetheless, these signals may still be useful for developing control strategies because many devices do not reproduce physiological motion and/or use embedded joint encoders to sense relative ankle position. However, our measurements of knee position were more accurate when compared to previously reported data recorded using optical motion capture (Winter, 1983; McClelland et al., 2011). The RMS error between ENABL3S and Winter (1983) was 5.9 and 9.5° for stance and swing phases, respectively. The *R*<sup>2</sup> values were 0.73 and 0.94 for stance and swing phases, respectively. Our reported values for knee range of motion (ROM) during stance and swing phases (flexion at initial contact: 10.9 ± 5.6°; stance ROM: 6.8 ± 5.2 to 26.9 ± 5.7°; swing ROM: 3.4 ± 5.4 to 58.0 ± 6.5°) were also comparable to reported values (McClelland et al., 2011). Errors in position can be attributed to a combination of differences in walking speed, minor misalignment of the sensor with the axis of rotation, and skin deformation/relative motion. The knee position could also be estimated (perhaps more accurately) by subtracting the orientations of the shank and thigh IMU sensors. The patterns of EMG activation for ankle plantarflexor/dorsiflexor and knee flexor/extensor muscles were also qualitatively similar to those previously reported for unassisted overground walking at self-selected speed (Winter, 1983; Sylos-Labini et al., 2014). Knee position and EMG from TA, MG, BF, and VL aggregated across legs for all steady-state level walking steps for all subjects can be found in a supplementary document on Figshare. By confirming the accuracy of our measured kinematic and EMG signals, we also validate the IMU-based method for gait segmentation.

Although these data are not as high resolution as optical motion capture, they strike a balance between resolution of signals, breadth of activities represented, feasibility for online control schemes, and contribution to existing publicly available datasets for human locomotion. These data can be used for developing novel control strategies such as intent recognition (i.e., predicting future states based on signals detected before movement completion) and more specifically investigating sensor fusion techniques and machine learning approaches for feature extraction and classification (e.g., deep learning). These data can also be interpreted as a simulation of able-bodied individuals walking with a completely massless and transparent (i.e., perfectly backdrivable) device and can be used to derive a device-agnostic upper bound on control strategies such as intent recognition. The raw data reported here may also be useful for comparing the performance of alternative control systems, assessing inter-subject variability, and comparing user-based biomechanical signals collected from an impaired population or a population walking with an assistive device (e.g., knee orthosis and ankle-foot prosthesis) to unassisted normal locomotion.

## ETHICS STATEMENT

This study was carried out in accordance with the recommendations of the Northwestern University Institutional Review Board with written informed consent from all the subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the Northwestern University Institutional Review Board.

## AUTHOR CONTRIBUTIONS

BH helped in conceiving the study concept, collecting, analyzing, and interpreting the data, and drafting the manuscript. LH and

## REFERENCES


ER helped in conceiving the study concept and interpreting the data, critically revising the manuscript for important intellectual content, obtaining funding, and supervising the study. All the authors read and approved the final manuscript.

## FUNDING

This research was supported by USSOCOM Contract No. H92222-16-C-0111.

total knee replacement. *Knee* 18, 151–155. doi:10.1016/j.knee.2010. 04.005


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2018 Hu, Rouse and Hargrove. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Corrigendum: Benchmark Datasets for Bilateral Lower-Limb Neuromechanical Signals from Wearable Sensors during Unassisted Locomotion in Able-Bodied Individuals

#### Blair Hu1,2 \*, Elliott Rouse<sup>3</sup> and Levi Hargrove1,2,4

**A Corrigendum on**

<sup>1</sup> Center for Bionic Medicine, Shirley Ryan AbilityLab, Chicago, IL, United States, <sup>2</sup> Department of Biomedical Engineering, Northwestern University, Evanston, IL, United States, <sup>3</sup> Department of Mechanical Engineering, University of Michigan, Ann Arbor, MI, United States, <sup>4</sup> Department of Physical Medicine and Rehabilitation, Northwestern University, Chicago, IL, United States

#### Keywords: gait, locomotion, biomechanics, electromyography, benchmark

#### Edited and reviewed by:

Diego Torricelli, Consejo Superior de Investigaciones Científicas (CSIC), Spain

### \*Correspondence:

Blair Hu blairhu@u.northwestern.edu

#### Specialty section:

This article was submitted to Biomedical Robotics, a section of the journal Frontiers in Robotics and AI

Received: 12 September 2018 Accepted: 26 October 2018 Published: 20 November 2018

#### Citation:

Hu B, Rouse E and Hargrove L (2018) Corrigendum: Benchmark Datasets for Bilateral Lower-Limb Neuromechanical Signals from Wearable Sensors during Unassisted Locomotion in Able-Bodied Individuals. Front. Robot. AI 5:127. doi: 10.3389/frobt.2018.00127

## **Benchmark Datasets for Bilateral Lower-Limb Neuromechanical Signals from Wearable Sensors during Unassisted Locomotion in Able-Bodied Individuals**

by Hu, B., Rouse, E., and Hargrove, L. (2018) Front. Robot. AI 5:14. doi: 10.3389/frobt.2018.00014

In the original article, there were two errors. In the text, the abbreviation for semitendinosus was omitted. In the text, the URL to the data repository available on Figshare was also incorrect.

Corrections have been made to Materials and Methods, Sub-section Instrumentation Setup, Paragraph one and Results, Paragraph one.

EMG signals were recorded using bipolar surface electrodes (DE2.1; Delsys, Boston, MA, USA) from the same seven muscles in each leg: tibialis anterior (TA), medial gastrocnemius (MG), soleus (SOL), vastus lateralis (VL), rectus femoris (RF), biceps femoris (BF), and semitendinosus (ST).

The data are saved in CSV format in subject-specific folders and are available to download from Figshare at https://doi.org/10.6084/m9.figshare.5362627.

The authors apologize for these errors and state that they do not change the scientific conclusions of the article in any way. The original article has been updated.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Hu, Rouse and Hargrove. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Automatic Setting Procedure for Exoskeleton-Assisted Overground Gait: Proof of Concept on Stroke Population

Marta Gandolla<sup>1</sup> \*, Eleonora Guanziroli <sup>2</sup> , Andrea D'Angelo<sup>1</sup> , Giovanni Cannaviello<sup>2</sup> , Franco Molteni <sup>2</sup> and Alessandra Pedrocchi <sup>3</sup>

<sup>1</sup> Nearlab@Lecco, Polo territoriale di Lecco, Politecnico di Milano, Lecco, Italy, <sup>2</sup> Villa Beretta Rehabilitation Center, Valduce Hospital, Costa Masnaga, Italy, <sup>3</sup> NearLab, Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milan, Italy

Stroke-related locomotor impairments are often associated with abnormal timing and intensity of recruitment of the affected and non-affected lower limb muscles. Restoring the proper lower limbs muscles activation is a key factor to facilitate recovery of gait capacity and performance, and to reduce maladaptive plasticity. Ekso is a wearable powered exoskeleton robot able to support over-ground gait training. The user controls the exoskeleton by triggering each single step during the gait cycle. The fine-tuning of the exoskeleton control system is crucial—it is set according to the residual functional abilities of the patient, and it needs to ensure lower limbs powered gait to be the most physiological as possible. This work focuses on the definition of an automatic calibration procedure able to detect the best Ekso setting for each patient. EMG activity has been recorded from Tibialis Anterior, Soleus, Rectus Femoris, and Semitendinosus muscles in a group of 7 healthy controls and 13 neurological patients. EMG signals have been processed so to obtain muscles activation patterns. The mean muscular activation pattern derived from the controls cohort has been set as reference. The developed automatic calibration procedure requires the patient to perform overground walking trials supported by the exoskeleton while changing parameters setting. The Gait Metric index is calculated for each trial, where the closer the performance is to the normative muscular activation pattern, in terms of both relative amplitude and timing, the higher the Gait Metric index is. The trial with the best Gait Metric index corresponds to the best parameters set. It has to be noted that the automatic computational calibration procedure is based on the same number of overground walking trials, and the same experimental set-up as in the current manual calibration procedure. The proposed approach allows supporting the rehabilitation team in the setting procedure. It has been demonstrated to be robust, and to be in agreement with the current gold standard (i.e., manual calibration performed by an expert engineer). The use of a graphical user interface is a promising tool for the effective use of an automatic procedure in a clinical context.

Keywords: lower-limb exoskeleton, electromyography, automatic calibration, neurorehabilitation, therapy personalization

#### Edited by:

Diego Torricelli, Consejo Superior de Investigaciones Científicas (CSIC), Spain

#### Reviewed by:

Giacomo Severini, University College Dublin, Ireland Iolanda Pisotta, Fondazione Santa Lucia (IRCCS), Italy

> \*Correspondence: Marta Gandolla marta.gandolla@polimi.it

Received: 29 September 2017 Accepted: 20 February 2018 Published: 19 March 2018

#### Citation:

Gandolla M, Guanziroli E, D'Angelo A, Cannaviello G, Molteni F and Pedrocchi A (2018) Automatic Setting Procedure for Exoskeleton-Assisted Overground Gait: Proof of Concept on Stroke Population. Front. Neurorobot. 12:10. doi: 10.3389/fnbot.2018.00010

## INTRODUCTION

Stroke is the leading cause of long-term disability in adults despite the advances achieved in the management of its acute phase (Heiss and Kidwell, 2014; Tacchino et al., 2017). Independent walking in particular has been associated to an increase in patients' ability to perform daily life activities and self-esteem. Although more than half of patients achieve an independent walking, this achievement may not be functional to carry out activities of daily living. Locomotion is defined as a cyclical lower limbs activity that results from intricate dynamic interactions between a central program (at brain and spinal cord level) and feedback mechanisms from muscles, tendons, and skin afferences, as well as vision, audition, and vestibular senses (Rossignol et al., 2006). The lower limb neuromuscular pattern should compensate body weight support, provide forward and lateral stability, and forward progression to ensure intra and inter-limb multi-joints coordination (Perry and Burnfield, 2010). Common stroke-related locomotor impairments (e.g., imbalance, gait asymmetry, poor inter-limb coordination) are often associated with abnormal timing and intensity of recruitment of the affected and non-affected lower limb muscles.

Timing and intensity of muscles recruitment influence kinematic and kinetic pattern of lower limbs and intra and inter-limb coordination (Mulroy et al., 2003; Den Otter et al., 2007). Restoring the coordination in muscles activation of lower limbs is a key factor to facilitate recovery of gait capacity and performance, and to reduce maladaptive plasticity in stroke patients. Evidence within the last 20 years has shown that an injured central nervous system has the ability to reorganize after damage (Nudo, 2013; Gandolla et al., 2016). The reorganization is dependent on motor activity executed during rehabilitative training, and is followed by functional improvements (Edgerton et al., 2004; Maier and Schwab, 2006; Gandolla et al., 2014, 2016). In order to achieve better outcomes in stroke survivors, gait rehabilitation should target impairments in coordination and allow to augment the number of repetitions during walking practice (Eng and Tang, 2007).

Nowadays, wearable lower limbs powered exoskeletons may be a valuable adjunctive rehabilitation therapy aiming at augmenting training dose with repeatable, task-oriented, and controlled movements, as suggested by the principles of motor learning (Dietz and Harkema, 2004). In fact, as a common approach implemented in lower limbs exoskeleton commercial devices, the devices include actuators that support patient's legs through the gait cycle in the sagittal plane (e.g., Lokomat, Hocoma; ReWalk, ReWalk Robotics). The robotic device guides the legs through pre-programmed physiological gait patterns this kind of therapeutic intervention is fairly new for stroke patients, however preliminary findings suggest that exoskeletal gait training is equivalent to traditional therapy for chronic stroke patients, while sub-acute patients may experience added benefit from exoskeletal gait training (Louie and Eng, 2016). Ekso is a wearable powered exoskeleton robot able to support stroke patients during over-ground gait training. The kinematic chain of the exoskeleton reproduces the human lower limbs walking pattern. In addition, Ekso actuators control patient's legs through the gait cycle in the sagittal plane. Ekso can be used as a therapeutic device in patients who must re-learn walking with a proper step pattern and functional weight shift by moving the patient's legs through a customizable predefined patient-tailored kinematic pattern. Ekso allows different setting for each patient in terms of swing velocity, step length, lateral shift. In this way, it is possible to control the walking pattern in terms of gait cycle timing (i.e., stance vs. swing phase duration), inter-limb and inter-joint coordination, lateral shift, trunk-lower limb angle, and timing to achieve appropriate limb loading.

The fine-tuning of the exoskeleton control system is crucial, and it is set according to the residual functional abilities of the patient. The interaction between exoskeleton and the patient can be seen under two different aspects: physical Human–Robot Interaction and cognitive Human–Robot Interaction (Pons, 2010; Lee et al., 2012). Physical Human–Robot Interaction includes the generation of supplementary forces to overcome human physical limits. In the case of the present study, the patient triggers each step, which however follows a predefined fully supported physiological trajectory. The interaction is therefore devoted to the generation of a proper gait cycle. Cognitive Human–Robot Interaction highlights the possibility to maintain the control of the robot from the human. In this study, the patient has the direct control on the trigger of each step though body lateral shift. Given the use of a commercial device, both aspects of Human-Robot Interaction depends on robotic device proper setting—the fine-tuning procedure is necessary to ensure the best power transfer between subject and robot. Surface ElectroMyoGraphy (sEMG) of the key muscles controlling multijoints coordination of lower limbs is an effective way to noninvasively define motor control during spontaneous over-ground gait.

This work focuses on the definition of an automatic calibration procedure able to detect the best Ekso setting for each patient. Ekso setting has been defined using the neuromuscular pattern of the lower limbs collected with the superficial EMG in hemiparetic stroke patients. The proposed approach for an automatic calibration procedure is based on the hypothesis that the best Ekso setting yields to be best muscular activation as detected from superficial EMG electrodes, and that muscular activation is as better as closer to healthy controls muscular activation patter, particularly in terms of muscular activation timing.

## MATERIALS AND METHODS

## Experimental Set-Up

Patient's overground locomotion has been supported by Ekso (Ekso Bionics, Richmond, CA, USA). Ekso is a wearable bionic suit: it enables individuals with lower limb disabilities and minimal forearm strength to stand, sit and walk over a flat hard surface with a full weight-bearing reciprocal gait under the supervision of a physical therapist. Ekso is intended for nonambulatory and ambulatory post-stroke patients, spinal cord complete, and incomplete injury patients with different etiology, and traumatic brain injury patients. It weighs 23 kg and can be used by individuals who weigh up to 100 kg and range in height from 160 to 190 cm. Patients must have a standing hip width at maximum of 43 cm. Ekso is equipped with four battery-powered motors at the hips and knees: these support or replace deficient neuromuscular function. There are four types of actuation for each patient step: (i) FirstStep, by which a physical therapist actuates steps with a button push; (ii) ActiveStep, by which the patient takes control of actuating steps via buttons on the crutches or walker; (iii) ProStep, by which the patient achieves the next step by moving body weight laterally and then forward; and (iv) ProStep Plus, by which steps are triggered by the user's lateral weight shift. The amount of power contribution to one or both legs during walking can be tuned with three types of assistance for each single step: (i) Bilateral Max Assist, in which Ekso provides full power to both legs and no strength is required from the patient; (ii) Adaptive Assist, in which patients with any amount of lower extremity strength contribute to their walking efforts and Ekso dynamically adjusts to produce a smooth, consistent gait; and (iii) Fixed Assist, where Ekso legs provide a fixed amount of pre-specified power to help patients to complete steps in a predefined amount of time. Within the present study, Ekso has been set with Prostep Plus, and Bilateral Max Assist. Ekso needs to be adjusted to fit patients' anthropometric data for a correct use of the device. In particular, it is necessary to collect hip width, length of right and left upper legs, and length of right and left lower legs.

The muscle activity has been recorded bilaterally with the FREEEMG wireless electromyograph (BTS Bioengineering, Garbagnate Milanese, Milano, Italy). Muscle groups considered for the analysis and placement of the electrodes has been selected accordingly to SENIAM guidelines (Hermens, 1999): tibialis anterior muscle (TA), soleus muscle (SOL), rectus femoris (RF), and semitendinosus muscle (SM). Lower limbs principal muscles have been selected for recording, and in particular, two couples of agonist/antagonist muscles in the proximal and distal compartment respectively, since they are more directly responsible for a correct walking-induced muscles activation profile, and EMG electrodes can be easily positioned without interfering with Ekso.

## Participants

Patients were recruited from the outpatient and inpatient services at the Villa Beretta Rehabilitation Centre (Costa Masnaga, LC, Italy). All patients had suffered from first-ever stroke, resulting in weakness of at least TA [to <4 on the Medical Research Council (MRC) scale Medical Research Council/Guarantors of Brain, 1986] and with a level of spasticity <2 as detected by Modified Ashworth Scale (Ansari et al., 2008) at hip, knee and ankle. Thirteen post-stroke patients were recruited [range: 29– 74 years, mean (standard deviation): 52 (14)], comprising 10 male and 3 female subjects. Patient's characteristics along with the degree of functional recovery at the time of recruitment are listed in **Table 1**. The control group was composed of healthy volunteers with no neurological or orthopedic impairment. The healthy control group was aged between 21 and 49 years [mean (standard deviation): 36 (10) years], comprising four male and three female subjects. Experiments were conducted with approval from the Villa Beretta Rehabilitation Centre Ethics Committee and all subjects gave informed written consent in accordance with the Declaration of Helsinki.

## Current Procedure for Manual Setting of Ekso

Current gold standard for Ekso parameters setting in clinical environment (i.e., manual calibration) consists on the patient performing a series of overground walking trials with the values of tunable parameters changed by the rehabilitation team so to identify the best setting for the current patient and condition. These parameters are set on the basis of EMG signal derived from analyzed muscles, and by looking at patient gait. EMG signals are not processed in this case, and they are displayed on a laptop screen. The information drawn from raw EMG signals is muscles activation timing. The best activation timing for both healthy and paretic muscles is defined according to typical activity of major muscle groups during the gait cycle. In particular, the standard procedure includes the setting of the three main setting parameters, i.e., (i) lateral shift (displacement of body weight under the patient's foot); (ii) swing time; and (iii) step length. Manual calibration starts with the first parameter to be set (i.e., lateral shift). A series of overground gait trials are performed, while setting the parameter to different values. The gait trials are minimum three, where the default value, and higher and lower settings are tested. By means of observation of the gait quality, and EMG signals acquired during walking, the expert Ekso user along with the rehabilitation team selects the best parameter setting. The first parameter is then fixed, and the next parameters are considered in a recursive procedure until Ekso is properly set (**Figure 1**).

## Computational Calibration Procedure

The proposed approach for automatic computational calibration procedure is based on the same number of overground walking trials, and the same experimental set-up as in the current manual calibration procedure, where the observation of the gait quality, and EMG signals by the expert Ekso user is substituted by EMG signal computational analysis (**Figure 1**). EMG signal computational analysis is based on the hypothesis that muscular activation profile is as better as closer to healthy control population pattern. This is the reason why data from a representative group of control subjects were also collected. The computational calibration procedure is applied to healthy controls, and the nonparetic side of neurological patients. In fact, it is known that the more natural is the step of the unimpaired side, the more physiological is the gait, and the more it is possible to state that the global ambulation is close to normative.

## Step Identification Procedure

Since it is currently impossible to autonomously extract data directly from Ekso sensors, it is not possible to synchronize Ekso with external systems (i.e., EMG). This limitation has been overcome by using a step identification procedure directly on the EMG signals. In particular, EMG signals coming from all muscular channels are pre-processed following a standard

#### TABLE 1 | Patients characteristics.


M, male; F, female; L, left; R, right; H, hemorrhagic stroke; I, ischemic stroke; FAC, functional ambulatory category (Mehrholz et al., 2007).

approach that includes high-pass filtering with a 6th order Butterworth filter at 20 Hz, rectification, and low-pass filtering with a 6th order Butterworth filter at 4 Hz (Solnik et al., 2008). Given that the computational calibration procedure should not include any additional workload to the rehabilitation team or to the patient, there are no footswitches or similar sensor available to give information about single steps. The proposed method is based on the hypothesis that the number of steps is proportional to the number of muscle activations. In order to satisfy this hypothesis, a mono-phasic muscle has been considered, so that only a single activation is expected throughout the step cycle. The Soleus muscle has been selected since is monophasic during the step (Pasinetti et al., 2013), i.e., it reaches only once the activation peak, characteristic which is preserved, as far as we observed, in our patients cohort. To this aim, Soleus EMG signal is further preprocessed to limit the bandwidth to frequencies where step cadence is located, i.e., 0–2 Hz (Pachi and Ji, 2005). Soleus deactivation is then identified through an algorithm based on a 20 samples sliding window and adaptive threshold derived from the integration of signal-to-noise ratio based adaptive threshold algorithm proposed by Sedghamiz, and Di Fabio and colleagues algorithm (Di Fabio, 1987; Sedghamiz, 2014). In particular, the algorithm is applied on the mean corrected EMG preprocessed signal, and the four variables—signal level, noise level, threshold,

and activation (binary on/off variable used to describe muscle activation/deactivation)—are null at the beginning. Variables levels are dynamically calculated sample by sample as detailed in **Figure 2**, considering the EMG portion included within the 20 samples sliding window. The signal portion included between two Soleus muscle deactivation corresponds to a step cycle, starting from the end of the push-off phase. EMG signal of all considered muscles is segmented accordingly.

#### Muscular Activation Pattern Definition

Muscular activation pattern for each muscle, both for control and for patients, is obtained by re-scaling each step to a 0– 100% scale in terms of step duration, and afterwards by averaging all steps, and by normalizing the muscular step template in terms of amplitude with respect to the peak value for each trial. To obtain a healthy controls muscular pattern, all averaged muscular patterns resulted from five different trials performed per participant have been averaged. The accuracy of the signal segmentation technique has been evaluated through qualitative inspection of the morphology of the muscle activation profiles and through a quantitative analysis of the inter-step variability (i.e., coefficient of variation) to verify consistency with the muscular dynamics reported in literature (Winter and Yack, 1987). Finally, using the same onset/offset detection algorithm described to detect Soleus muscle deactivation (**Figure 2**), for each muscle an activation/deactivation profile is determined, where the information "the muscle is active or inactive" can be derived with respect to the percentage of the gait cycle (i.e., 0–100%).

#### Performance Index Extraction

So to define the best parameters setting, the Gait Metric index (GM) has been extracted from the healthy controls, and the non-paretic side of neurological patients. GM is an analytical combination of amplitude and activation timing (Ricamato and Hidler, 2005), and quantifies the deviation of the muscular activation pattern from normal ranges defined within the healthy control group.

In particular, GM is composed by the arithmetic mean between an amplitude, and a phase component determined through the comparison of each muscle activation pattern, and the correspondent healthy controls activation pattern. The amplitude component (AC) is obtained by summing the EMG values where both the patient and the healthy controls patterns are over or under threshold. In other words, for each given sample (i.e., 0–100% of the gait cycle), AC is increased if patient muscle is active when also healthy controls muscle is, or is inactive when also healthy controls muscle is inactive (Equation 1).

$$AC = \sum\_{p=1}^{100} \text{(HCAP \(p\))} \text{(EMG \(p\) - threshold)} \tag{1}$$

Where p is the index representing gait cycle progression (i.e., 0– 100%); HCAP is the Healthy Controls Activation Profile which is 1 for the healthy controls pattern active portions, and −1 for the inactive portions; EMG(p) is the patient EMG profile sample value; and threshold is the activation threshold defined as described in section Graphical User Interface (GUI) For Clinical Use.

AC is then normalized to obtain a value between 0 and 1 (ACnorm) as follows (Equations 2–4).

$$\begin{array}{c} \text{AC}\_{\text{max}} = \left( (1 - threshold) \ast \text{#Active} \right) \\ \quad + \left( threshold \ast \text{#Active} \right) \end{array} \tag{2}$$

 threshold ∗ #Inactive (2)

ACmin = −1 ∗ (100 − ACmax) (3)

$$AC\_{norm} = \frac{AC - AC\_{min}}{AC\_{max} - AC\_{min}} \tag{4}$$

Where #Active is the number of active samples in the healthy controls activation pattern; and #Inactive is the number of inactive samples in the healthy controls activation pattern.

The Phase Component (PC) is determined for each given sample (i.e., 0–100% of the gait cycle) by summing 1 if patient muscle is active when also healthy controls muscle is, or is inactive when also healthy controls muscle is, and 0 otherwise. PC is then normalized dividing the obtained value by 100.

Once the GM has been obtained for each considered muscle, a Weighted GM (WGM) is obtained by weighting each GM with the standard deviation of the correspondent muscle obtained in the healthy control group as follows (Equations 5, 6).

$$\begin{aligned} \text{Normalized St.Dev.} &= \\ \left( \begin{bmatrix} 1 \ 1 \ 1 \ 1 \end{bmatrix} - \frac{\text{St.Dev.} \,\text{Healthy Sub.}}{\sum\_{i=1}^{n} \left( \text{St.Dec} \,\text{.} \,\text{Healthy Sub.} \right)\_{i}} \right) & \ast \frac{1}{n-1} . \end{aligned} \tag{5}$$
 
$$\text{WGM} = \sum\_{i=1}^{n} \text{GM}\_{i}^{\*} \left( \text{Normalized St.Dev.} \right)\_{i} . \tag{6}$$

Where n represents the considered muscles, St.Dev.HealthySub. is the vector containing GM standard deviation obtained in the healthy controls group; NormalizedSt.Dev. is the normalized vector of standard deviations considered for GM weighting (i.e., sum equals 1).

The higher the WGM, the closer the performance is to the normative muscular activation pattern, in terms of both relative amplitude, and timing. The trial with the best WGM would correspond to the best parameter set.

## Computational Calibration Procedure Validation

Repeatability of the automatic calibration procedure has been tested by running twice the algorithm for each participant (i.e., each neurological patient). The output parameters setting in the two runs have been compared through the Cohen's kappa for agreement between to evaluators (Cohen, 1960). The test of repeatability was important since the EMG signal portion selected to run the analysis is of free choice of the user, and therefore it cannot be taken for granted that

the same steps are considered for the analysis. Indeed the computational procedure should be robust with respect to steps selection.

Manual calibration is performed by means of observation of the gait quality, and EMG signals acquired during walking. However, EMG signal analysis is only performed by sight on a non-processed signal, and it is therefore not reliable. To test this hypothesis, three different raters selected Ekso parameters setting only by inspecting non-processed EMG signal, without seeing the patients. The agreement between the different raters has been evaluated through Fleiss' Kappa (Landis and Koch, 1977).

In addition, the agreement between computational calibration procedure parameters setting, and the gold standard procedure (i.e., parameters set by the expert clinical engineer and rehabilitation team during the effective calibration session) has been evaluated by Cohen's kappa (Cohen, 1960; Gandolla et al., 2015).

## Graphical User Interface (GUI) for Clinical Use

The computational calibration procedure has been implemented in a custom-made and guided software developed in MATLAB environment (**Figure 3**) to support the use of the proposed approach in clinical practice. The interface has three sections: the "Healthy Subjects" section dedicated to the analysis of healthy controls and the calculation of the normative muscular activation pattern; the "Patients" section for patient data analysis and searching for the best Ekso GT configuration; the "Common Tools" section where the user can perform an additional analysis of patients or healthy controls data.

## RESULTS

## Muscular Activation Pattern

The healthy controls muscular pattern is shown in **Figure 4**. The qualitative inspection of the morphology of the muscle activation profiles reflects what has been found in literature evidences (Winter and Yack, 1987; Tao et al., 2012), and in particular:


The mean value of the coefficient of variation across all subjects for the considered muscles are 0.230 (TA), 0.167 (SOL), 0.369 (RF), and 0.365 (ST), respectively.

Neurological patients muscular activation pattern is quite different among subjects, as expected (an example is shown

in **Figure 5**). For all patients, steps segmentation has been successfully performed on the non-paretic EMG signal during robotic-assisted gait trials, and the gait metric index has been calculated for all trials, and all patients.

## Gait Metric Behavior

GM has been calculated for each trial of the healthy controls cohort, and results are reported in **Table 2**, and are in accord with values reported in literature for a walking speed obtained during spontaneous walking, i.e., 1.3–1.6 m/s (Ricamato and Hidler, 2005). To check for WGM variability, WGM has been calculated for each control participant for the five walking trials obtaining standard deviations equal to 0.007, 0.014, 0.023, 0.031, 0.027, 0.005, and 0.026 respectively. **Table 3** shows the detailed WGM scores obtained by patients in the seven walking trials with different Ekso parameters settings. As it can be observed, in some cases, WGM differences between alternative parameters settings are not crucial (i.e., difference lower that healthy controls cohort mean WGM standard deviation−0.019). In these cases, Ekso parameters setting is therefore not crucial in terms of EMG activations obtained. WGM variability is higher within the first three trials, correspondent to the selection of first parameter, i.e., lateral shift, while decreases in the other trials. As a representative muscle, Tibialis Anterior EMG mean profile for each trial and each patient is represented in **Figure 6**. As it can be observed, some of the patients present quite substantial differences in terms of muscles activity, which can be easily detected by sight (e.g., PT01 or PT13). Other patients (e.g., PT07) shows muscles activity profiles almost superimposable among trials. Again, Ekso parameters setting is particularly crucial for patients who present substantially different muscles activation profiles.

## Reproducibility and Validity of Computational Calibration

The computational calibration procedure is robust with respect to steps selection, as shown by Cohen's kappa equals to 0.883, i.e., strong agreement (Sim and Wright, 2005). On the other side, the agreement between the three different raters who selected Ekso parameters setting only by inspecting non-processed EMG signal is equal to 0.296, i.e., mediocre agreement. Agreement between computational calibration procedure, and gold standard (i.e., expert engineer setting) shows substantial agreement, with Cohen's kappa equals to 0.648, while agreement between the three different raters who selected Ekso parameters setting only by inspecting non-processed EMG signal and gold standard is very weak with Cohen's kappa equal, respectively to 0.095, 0.058, and 0.045 (Sim and Wright, 2005). In particular, for the three Ekso parameters Cohen's kappa for the agreement between automatic procedure and gold standard selection are 0.614, 0.591, and 0.780, respectively.

## DISCUSSION

Gait recovery in post-stroke patients is one of the main goals of post-stroke rehabilitation (Molteni et al., 2017). Literature evidences demonstrated that central nervous system can reorganize after injury and that reorganization depends on motor activity performed during rehabilitative training (Edgerton et al., 2004; Maier and Schwab, 2006). Wearable robotic exoskeleton may be intended like an external environment acting with the patient—an extension of the body of the patient. Robotic devices induce patients' lower limbs to complete a pre-defined motor pattern according to a pre-programmed kinematic profile allowing subjects with gait dysfunctions to perform an overground gait training based on the principle of motor relearning.

There is a paucity of published data on powered robotic exoskeletons for gait rehabilitation in post-stroke patients. In a recent review (Louie and Eng, 2016) on the use of wearable powered exoskeletons in stroke patients, authors describe studies in which different robotic devices were used on a small number of stroke patients without general consensus on the results. Molteni et al. (2017) performed a pre-post study to analyse the effects of a wearable powered exoskeleton on 23 sub-acute and chronic stroke patients. Authors claimed that it is possible to modify clinical outcome measures in sub-acute and chronic post-stroke patients after 12 sessions of gait training with a


TA, tibialis anterior muscle; SOL, soleus muscle; RF, rectus femoris; SM, semitendinosus muscle; Std dev, standard deviation.

TABLE 3 | Weighted Gait Metric index values obtained by the patients cohort.


powered wearable robotic exoskeleton after fine-tuning of the kinematic gait cycle parameters. The fine-tuning of wearable robotic device parameters is therefore essential to produce the best neuromuscular pattern of the lower limbs enhancing shortterm neuromodulation. This may be a way to induce long-term potentiation of the mechanism controlling the gait pattern of non-affected and affected side (Kwakkel et al., 1999).

For this reason, the use of an automatic calibration procedure to identify the best settings for each patient is very important. This approach, which is repeatable, robust, and based on quantitative measures, may underline aspects hardly detectable only through direct observation, and may provide a valuable support to the rehabilitation team.

In this work, an automatic calibration procedure has been proposed for Ekso. The proposed approach is based on the hypothesis that the best Ekso setting yields to a muscular activation as close as possible to healthy controls muscular activation pattern, given that restoring a correct activation pattern is a key aspect of the rehabilitation program of neurological patients (Zhang et al., 2017). Although the sample size of healthy subjects is limited, the derived muscular activation patterns for all muscles agree with those reported in literature (Ricamato and Hidler, 2005). Coefficients of variations show the same relationship among muscles as described in literature (Winter and Yack, 1987). Distal muscles (i.e., TA and SOL muscles) present lower coefficients of variations with respect to proximal lower limbs muscles (i.e., RF and SM muscles). This is in line with the role of proximal muscles during gait, which is of support and equilibrium control. Given in fact the complexity of their functions, proximal muscles activation profile results to be more variable among successive steps (Winter and Yack, 1987). As a clinical recommendation for the computational calibration procedure everyday use, the authors suggest if possible to acquire the EMG signal from all four principal leg muscles or, as a possible alternative, to register distal muscles activity. In fact, as previously introduced, the function of proximal muscles during gait might be identified in the maintenance of balance, which is a complex motor task in post-stroke patient and the disability due to the paretic limb introduces compensatory mechanisms that affect its performance. Typically, if excessive cocontraction of distal muscles occurs, compensation is performed at the proximal level (Higginson et al., 2006). An analysis only based on the activity of the semitendinosus and rectus femoris muscles cannot guarantee the optimum performance of the motor task because a physiological activation of the proximal muscles may correspond to an abnormal activation of the distal muscles. Conversely, an analysis of both the soleus and Tibialis Anterior muscle seems to be more effective in defining the Ekso settings because a proper distal activation pattern more likely corresponds to a non-compensatory activation of proximal muscles.

The goal of the proposed approach is to equip the clinician with an instrument that could help clinician to identify the best Ekso setting, singularly for each patient. As far as we know, there are any quantitative data published in literature or indications provided by the fabricant so to evaluate the correctness of the setting. The gold standard procedure is manual regulation by expert operators, and, as it can be observed by the poor agreement revealed by Cohen's kappa between different operators in selecting best Ekso parameters, it lays on subjective evaluation, and it is not repeatable. The automatic procedure selection has been compared to the setting selection of an expert operator in the Villa Beretta Rehabilitation Center, as suggested by Ekso Company itself, and obtained substantial agreement, being at the same time robust for different steps selections. In this case, we are not claiming we are obtaining better results

with respect to manual calibration in terms of best setting, but we are claiming that the automatic procedure is robust and repeatable with respect to a gold standard, and can be used by any operator.

The proposed approach allows supporting the rehabilitation team in the setting procedure, and it has been demonstrated to be robust, and to be in agreement with the current gold standard. The use of a graphical user interface is a promising tool for the effective use of an automatic procedure in a clinical context. Indeed, the automatic calibration procedure does not imply any additional workload for the patient or the therapist with respect to the manual calibration procedure. The automatic calibration procedure has been validated with respect to the current gold standard, which is the selection of the expert Ekso user. However, the use of the automatic calibration procedure may allow a correct parameters setting from the very beginning of Ekso use, also when the rehabilitation team is still not well trained. The identification of an automatic procedure able to detect in an objective way the best devices setting, allows to plan a completely new individual tailored rehabilitation strategy.

## AUTHOR CONTRIBUTIONS

MG, EG, FM, and AP: Conceived the design of the work; EG and GC: Acquired the data; MG, EG, AD, and AP: Conceived the algorithm and analyzed the data; All authors contributed in interpreting data for the work. MG, EG, and AD: Drafted the manuscript; AP, GC, and FM: Revised it critically for important intellectual content. All authors read and approved the final version of the manuscript.

## REFERENCES


## ACKNOWLEDGMENTS

This work was made possible thanks to the patients that agreed to participate in the project and thanks to the physiotherapists, and technicians that gave their availability during data collection.

in hemiparetic patients after stroke. Arch. Phys. Med. Rehabil. 88, 1314–1319. doi: 10.1016/j.apmr.2007.06.764


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Gandolla, Guanziroli, D'Angelo, Cannaviello, Molteni and Pedrocchi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# A Subject-Specific Kinematic Model to Predict Human Motion in Exoskeleton-Assisted Gait

Diego Torricelli <sup>1</sup> \* † , Camilo Cortés 2†, Nerea Lete<sup>2</sup> , Álvaro Bertelsen<sup>2</sup> , Jose E. Gonzalez-Vargas <sup>1</sup> , Antonio J. del-Ama<sup>3</sup> , Iris Dimbwadyo<sup>4</sup> , Juan C. Moreno<sup>1</sup> , Julian Florez <sup>2</sup> and Jose L. Pons 1,5

<sup>1</sup> Cajal Institute, Spanish National Research Council (Consejo Superior de Investigaciones Científicas), Madrid, Spain, <sup>2</sup> eHealth and Biomedical Applications, Vicomtech, San Sebastián, Spain, <sup>3</sup> Biomechanics and Assistive Technology Unit, National Hospital for Paraplegics, Toledo, Spain, <sup>4</sup> Occupational Therapy Department, Occupational Thinks Research Group, Instituto de Neurociencias y Ciencias del Movimiento, Centro Superior de Estudios Universitarios La Salle, Universidad Autónoma de Madrid, Madrid, Spain, <sup>5</sup> Monterrey Institute of Technology, Monterrey, Mexico

The relative motion between human and exoskeleton is a crucial factor that has remarkable consequences on the efficiency, reliability and safety of human-robot interaction. Unfortunately, its quantitative assessment has been largely overlooked in the literature. Here, we present a methodology that allows predicting the motion of the human joints from the knowledge of the angular motion of the exoskeleton frame. Our method combines a subject-specific skeletal model with a kinematic model of a lower limb exoskeleton (H2, Technaid), imposing specific kinematic constraints between them. To calibrate the model and validate its ability to predict the relative motion in a subject-specific way, we performed experiments on seven healthy subjects during treadmill walking tasks. We demonstrate a prediction accuracy lower than 3.5◦ globally, and around 1.5◦ at the hip level, which represent an improvement up to 66% compared to the traditional approach assuming no relative motion between the user and the exoskeleton.

### Edited by:

Guang Chen, Tongji University, China

#### Reviewed by:

William Zev Rymer, Rehabilitation Institute of Chicago, United States Fan Gao, University of Kentucky, United States

#### \*Correspondence:

Diego Torricelli diego.torricelli@csic.es

†These authors have contributed equally to this work.

Received: 08 February 2018 Accepted: 10 April 2018 Published: 27 April 2018

#### Citation:

Torricelli D, Cortés C, Lete N, Bertelsen Á, Gonzalez-Vargas JE, del-Ama AJ, Dimbwadyo I, Moreno JC, Florez J and Pons JL (2018) A Subject-Specific Kinematic Model to Predict Human Motion in Exoskeleton-Assisted Gait. Front. Neurorobot. 12:18. doi: 10.3389/fnbot.2018.00018 Keywords: benchmarking, walking, wearable robot, rehabilitation, lower limb, skeletal modeling

## INTRODUCTION

The quantitative assessment of robotic performance is a critical issue in rehabilitation robotics (Torricelli et al., 2015b). The increasing number of wearable robots available in the market has triggered the strong need for reliable methods to compare the existing solutions on a common basis. In the field of lower limb exoskeletons, devices are usually tested according to self-defined procedures and metrics that cannot be easily replicated across different laboratories and/or users. The most relevant problems are related to the intrinsic differences between devices, in terms of degrees of freedom, actuation principles, mechanisms complexity, and materials, but are also due to the heterogeneous measurement systems and protocols available worldwide. Besides this, the close interaction between the user and the robot further challenges the assessment of robotic performance independently from the user (Torricelli et al., 2015a). As a results, performance indicators normally rely on global variables such as metabolic consumption (Mooney et al., 2014; Collins et al., 2015; Galle et al., 2017), joint kinematics (Sawicki et al., 2006; Van Asseldonk et al., 2008), or spatiotemporal parameters (Buesing et al., 2015; Arazpour et al., 2016). While these approaches are effective in grasping the overall behavior of a bipedal system, they do not provide any clues on the internal mechanisms that may be relevant to the global performance, e.g., humanmachine interaction (Torricelli et al., 2015a). In this respect, one aspect that has been particularly disregarded in the literature is the quantitative evaluation of human-machine kinematic compatibility. A wearable robot is, by definition, a machine that operates in constant physical contact with the human body, supporting its movement by applying forces on the subject's skin (Pons, 2008). Due to kinematic, dynamic, and morphological differences between the exoskeleton and the human body, a relative motion between them always exists. This motion is responsible for a number of disadvantages, such as energy losses during power transmission, inaccurate control of the human limbs, or discomfort and pain due to skin abrasion. During mechatronic design, the understanding of these factors is key for improving the device and its acceptance by the end user.

An accurate way to measure the relative motion between subject and exoskeleton is by means of marker-based motion capture (MOCAP) systems, which use reflective markers placed on both the exoskeleton frame and the subject limbs (Alvarez et al., 2017). This approach can produce very precise results, but requires a time-consuming experimental procedure for marker placement, post-processing, and fitting with human body models. In addition, current marker-based models are not usually compatible with the presence of an exoskeleton, leading to the need of custom-based protocols, which can be hardly replicated across different systems.

Motivated by these observations, we formulated the following question: "is it possible to predict human motion from exoskeleton motion?." A positive answer to it would support the feasibility of estimating both exoskeleton and human motion using only the exoskeleton sensors, overcoming most of the aforementioned drawbacks. Being independent from any external measurement system, this approach would also allow measuring human-exoskeleton interaction in realistic outdoor environments.

To address this research question, we propose a modelingexperimental approach that combines personalized skeletal models of human subject with a kinematic model of the exoskeleton. We previously addressed a similar problem in the context of exoskeletons for upper limb rehabilitation (Cortés et al., 2014, 2016). In that work, we formulated and assessed a computational method, denominated EIKPE (Extended Inverse Kinematics Posture Estimation), to estimate the joint angles of the human subject when the exoskeleton motion is known. In the original version of the EIKPE, the human limb and exoskeleton are modeled as a parallel kinematic chain in which the exoskeleton's cuff constraints impose motion constraints on the human limb. Then, for a given pose of the exoskeleton, the inverse kinematics (IK) of the parallel chain was computed to find the joint angles of the subject limb during the training of singlejoint (e.g., elbow flexion) or compound motions (e.g., reaching an object).

Here, we propose an extended version of the EIKPE, which adds skeletal (SK) modeling in order to improve the subjectspecific prediction ability of human limb motion given the absolute pose of the exoskeleton limb. To our best knowledge, no similar approaches have been proposed in the literature.

## MATERIALS AND METHODS

The process of creating, applying and estimating the accuracy of the EIKPE entails the following five steps:


## Ground-Truth Motion Recording

Seven healthy subjects (5 men, 2 women, age 29.7 ± 4.9) participated in the study. The experiments were performed in the Motion Analysis Laboratory of the Centro Superior de Estudios Universitarios La Salle, Universidad Autónoma de Madrid, Madrid, Spain. Subjects were asked to perform two different recording sessions. In the first session, each subject was asked to walk at 1 Km/h speed during 10 s. An additional trial was required to measure the subject in a static upright standing posture. In the second session, the subject repeated the previous trials while wearing a lower limb exoskeleton. The exoskeleton used in this experiment was the Exo-H2 (Technaid, Arganda del Rey, Spain; Bortole et al., 2015). The Exo-H2 has 6 degrees of freedom (DOFs), including hip, knee and ankle joints. Actuators are connected to each other by means of an aluminum frame with extensible plates that allow adjusting the inter-joint distance in order to adapt to a specific subject size. In this experiment, the exoskeleton was configured in "mechanically-transparent mode," i.e., with the motors physically decoupled from the joints. In this configuration, the exoskeleton was unable to apply any assistive or resistive force at the joint level. Several belts are used to attach the exoskeleton to the subject.

Subject and exoskeleton motion were measured by using a marker-based MOCAP system (BTS, Garbagnate Milanese, Italy) composed of eight infrared cameras. All walking trials were performed on a treadmill (LK6000 treadmill, BH fitness, Spain). A 2 cubic meter volume was previously calibrated to ensure accurate reconstruction of all markers during the experiment. Prior to the first experimental session, the subject was instrumented with 12 reflective markers of 10 mm diameter placed on different anatomical landmarks (**Figure 1**): three markers on the foot, placed on the calcaneus and on the fifth and first metatarsals; one marker on the malleolous; three markers on the shank; one marker on center-outside surface of the knee; three markers on the thigh; one marker on the trochanter.

After the first session, and before donning the exoskeleton, the markers on ankle, knee, and hip were removed, because the exoskeleton structure would impede their view from the cameras. The exoskeleton was equipped with five markers placed in the center-outside surfaces of the knee, ankle and hip motors, and in the midpoint of the shank and thigh bars. Motion data were recorded at 100 Hz and processed offline to obtain the labeled 3D trajectories of all markers.

## Skeletal Model Personalization

We generated a personalized skeletal model for each of the tested subjects by scaling a generic musculoskeletal lower limb model [model Gait2392 included in OpenSim (Yamaguchi and Zajac, 1989; Delp et al., 1990; Anderson and Pandy, 1999, 2001)], which includes 19 DOFs. Although we used a musculoskeletal model, in this work we refer to it as a skeletal model because the muscular components of the model were not used for the human kinematic estimation. The scaling and adjustment of markers were performed by using the Scale Tool of OpenSim. We have configured the Scale Tool to obtain the scaling factors along the longitudinal axis of the femur using markers Mk11 and Mk7, for the tibia using markers Mk7 and Mk3, and for the foot using markers Mk2 and Mk1 (**Figure 1**). For this scaling stage, we used only the static captures of the subject without the exoskeleton. Notice that with our marker protocol we were only able to scale the right lower limb. The rest of the model parts conserve the size of the original SK generic model. The results of the scaling were visually inspected. If the scaling was not coherent (e.g., the markers in the scaled model appear to be too low or high with respect to the segment they are attached to), then a manual scaling factor was applied.

## Human-Exoskeleton Model Generation

We modeled the fixation between the subject pelvis and the exoskeleton corset as rigid, and adjusted their relative translation such that the coordinates of the hip joint rotation centers (left and right) of the human and exoskeleton match along the anterior-posterior direction of the sagittal plane (**Figure 2**). Then, we adjusted the length of the exoskeleton links by using the static captures of the subjects wearing the exoskeleton, such that the axes of rotation of the joints of the hip, knee and ankle corresponded to the height indicated by markers Mk11, Mk7, and Mk3 respectively. Finally, the joint angles of the human hip (flexion, rotation, and adduction), knee (flexion) and ankle (flexion) were adjusted, using inverse kinematics, and manually revised, to match the leg posture in the static capture of the subject wearing the exoskeleton.

## Computation of the Ground-Truth Joint Angles

We computed the Ground-Truth (GT) joint angles of the human and the exoskeleton by using the Inverse Kinematics (IK) Tool of OpenSim (**Figure 3**). We have configured the IK tool to estimate the angles of the hip, knee and ankle for both human and exoskeleton and also the translations and rotations of the pelviscorset junction relative to the MOCAP coordinate system. We assumed that, during the gait using the exoskeleton, the hip rotation and ab-adduction angles are like those computed at the static posture. This assumption is realistic since the exoskeleton does not include neither adduction or rotation DOFs at the hip level. To estimate the exoskeleton ankle plantar-dorsiflexion, we created a new virtual marker (Mk14, see **Figure 3**) located in the mid-point between markers Mk0 and Mk1. Then, the exoskeleton ankle angle can be computed using IK from markers Mk3 and Mk14.

The GT joint angles computed correspond to the rotation around the Z axis of the joints. The neutral position of each joint is defined as the position in which the Y axis of two adjacent segments match. **Figure 4** shows the definition of the knee flexion-extension for the human and exoskeleton models, as an example of the mentioned convention. The coordinate systems of the human pelvis and exoskeleton corset are aligned between them.

## Human Joint Angles Estimation Using EIKPE

Our method to estimate the joint angles of the human lower limb during gait is based on a previous formulation of the EIKPE method (Cortés et al., 2014, 2016). In the EIKPE, the human limb and the exoskeleton are modeled as a single parallel kinematic chain connected by the fixations of the exoskeleton (**Figure 5**, left). For a given sequence of postures of the exoskeleton, described by its vector of joint angles v R (t), the human sequence of joint angles v <sup>H</sup>(t) is estimated.

Formally, the inputs to the problem are (**Figure 5**, right):

	- a. A 6-DOF constraint between the human pelvis and the exoskeleton corset.

FIGURE 1 | Marker placement and labeling (written informed consent was obtained from the individual for the publication of this image).


The goal of the implemented algorithm is to find the approximate angles of the joints of the patient limb v˜ <sup>H</sup>(t), such that the set of constraints C are met.

The application of the EIKPE to estimate v <sup>H</sup>(t) entails the following steps (**Figure 6**):


## Assessment of the Accuracy of EIKPE Estimations

To assess the prediction performance, we compared the joint profiles estimated by the EIKPE with the angle estimations obtained by the GT angles from captured data. In order to determine how the EIKPE compares to the traditional rigid method, which assumes no misalignment between exoskeleton and human joint axes and segments, we compared the joint angles of the rigid method against those of the GT. The similarity of the angle estimations provided by the EIKPE and the rigid method against the GT has been assessed in terms of Root Mean Squared Error (**RMSE**) and Range of Motion Error (**ROME**) according to the following equations:

$$\begin{aligned} \text{RMSE } &= \sqrt{\frac{\sum\_{i=1}^{n} \left(\mathbf{x}\_{i} - \mathbf{y}\_{i}\right)^{2}}{n}} \\ \text{ROME } &= \left[\max\left(\mathbf{x}\right) - \min\left(\mathbf{x}\right)\right] - \left[\max\left(\mathbf{y}\right) - \min\left(\mathbf{y}\right)\right] \end{aligned}$$

where **x** refers to the GT human joint angles and **y** are the joint angles obtained by either the EIKPE or the rigid model estimator.

The RMSE captures errors related to differences in the shape and offset of the estimations, whereas the ROME reflects the accuracy in the estimation of the maximum amplitude of the movement. This comparison has been performed on walking data, whereas static data have been used only for model building and calibration.

To check for statistical differences between the performance of the rigid model and the EIKPE, we conducted a Wilcoxon-Mann-Whitney test, as an alternative to the t-test given the low numbers of participants of this study. The test was applied on both metrics (RMSE and ROME) for each joint, and the significance was set to p = 0.05.

## RESULTS

**Figure 7** shows the results on human joint angle estimation from one representative subject. The three profiles represent human joint angles as obtained by GT captured data (in blue), the EIKPE (in red), and the rigid model (in green). Results are given for hip, knee and ankle DOF in the sagittal plane.

**Table 1** reports the RMSE and ROME values from all the test subjects. In addition, **Figure 8** presents the box-plots of the obtained values for the rigid model and the EIKPE across subjects, for each of the estimated joint angles.

When compared to the rigid model, the EIKPE showed lower errors with respect to the GT angles, in terms of both RMSE and ROME. Regarding the RMSE metric, the improvements produced by EIKPE vary between 27 and 44% while for the ROME metric vary between 15 and 66%. In particular, the hip flexion-extension showed the better estimation accuracy, with mean errors lower than 2◦ and a dispersion of 0.7◦ . For the knee and ankle, the estimation errors increased, with mean values below 3.5◦ for both metrics. The ankle estimations present the larger dispersion for both metrics among the studied joints (up to 2◦ for ROME metric). This is possibly due to cumulative errors which are amplified at the end of the human kinematic chain.

Concerning the estimations of the rigid model, the joint angle that is better estimated is the hip flexion-extension, as occurred with EIKPE, with mean errors around 2◦ and a dispersion near 1 ◦ . As opposed to the results obtained by the EIKPE, the knee angle presented the highest estimation errors. This result suggests that the larger misalignment between the kinematic models of the exoskeleton and human lower limb occurs at the level of the knee during gait.

**Table 2** presents the p-values obtained from the Wilcoxon-Mann-Whitney test for RMSE and ROME. According to this analysis, in terms of RMSE, the rigid model and the EIKPE result statistically different (p < 0.05) only at the knee and ankle. In contrast, in terms of the ROME, statistically relevant differences are found only at the hip.

**Figure 9** presents a qualitative comparison of the reconstructed poses of the human lower limb, at the knee level, during a particular phase of the gait cycle. The three figures TABLE 1 | Human joint angle estimation errors in terms of the ROME and RMSE metrics (mean ± sd) provided by the rigid method and the EIKPE with respect to the Ground-Truth angles.


\*Error reduction in the angle estimates provided by the EIKPE with respect to ones provided by the rigid model.

TABLE 2 | Results of the Wilcoxon-Mann-Whitney test's applied to the two population of errors obtained by the EIKPE and the rigid model.


The asterisk indicates a p-value lower than 0.05.

correspond to the reconstruction using the GT joint angles (left), the EIKPE (middle), and rigid model (right) respectively. It can be observed how the rotation axes of the human (ZHuman knee) and exoskeleton knee (ZExo knee) differ from each other in the case of the GT data. With the rigid model, their directions are parallel, recreating an idealized and inaccurate human exoskeleton relative pose. The EIKPE, on the contrary, is able to reconstruct the direction of ZHuman knee close to those estimated by the GT.

## DISCUSSION

Measuring the relative motion between human and exoskeleton is a challenging and increasingly relevant issue in wearable robotics. A recent sensitivity analysis showed that inaccurate joint angles estimates may led to inaccurate inverse dynamic estimations up to 232% during gait (Riemer et al., 2008). These sources of errors are particularly likely to happen in wearable robotics applications, where it is generally assumed that the exoskeleton and the human have negligible relative motion. This can lead to wrong estimates about power and force transmissions and therefore introduce important biases in the design of both the mechatronic components and the control paradigms of these devices. The EIKPE method here proposed showed an improvement on the accuracy of human motion prediction in the range of 15–66% over rigid model assumptions, leading to accuracies below 3◦ (RMSE between 1.6–2.3◦ ). In the context of gait analysis, these figures seem to be satisfactory, being close to those obtained with other motion capture systems. For example, the method in Seel et al. (2014), based on inertial sensors, reports a RMSE between 1 and 3◦ for the ankle and knee flexionextension movements, in a scenario where no exoskeletons were involved.

A major advantage of the EIKPE method is that no additional sensors apart from those embedded in the exoskeleton are required to obtain accurate estimations of the human joint angle throughout the gait cycle. This has important practical applications in clinical, industrial, and consumer domains, because they allow quick measurements in out-of-the-lab conditions. For instance, the EIKPE method can be used as a benchmark of adaptability of the exoskeleton to specific sizes of the subjects, either healthy or impaired, in the execution of real tasks. Such information can be even obtained prior to usability tests, and used to improve the design of the device, achieving better ergonomics and more efficient transmission of forces. In clinical diagnostic applications, such as during the assessment of the increment of the voluntary range of motion, the EIKPE method would enable the assessment of patient evolution during robotic intervention, and not by pre-post comparison, as currently done. The application of a method like the EIKPE becomes indispensable if the objective is the estimation of the inverse dynamics of the patient-exoskeleton system (e.g., voluntary torque applied by the patient to some joint) during the therapy. In this respect, real-time predictions of these quantities will be of invaluable help to those control strategies based on timely prediction of user intention/contribution, producing more effective assist-as-needed paradigms. Another advantage of the proposed method is to rely on purely geometrical algorithms. This means that the constraints between relative positions/orientations of exoskeleton and subject limbs are valid independently from the correct execution of walking movements. Therefore, the method is expected to be robust to different motor tasks or to the execution of abnormal movement patterns such as those experience in clinical cases. However, these aspects are still to be confirmed experimentally.

The model here proposed presents a number of simplifications that need to be considered when assessing the generalization of the results.

First, the personalization of our skeletal model is currently based on scaling factors obtained from the lengths of femur, tibia and foot of the subject. Therefore, the model is not able to account to subject-specific deformities (e.g., those experienced by cerebral palsy patients). In addition, this personalization process is not fully automatic. The intervention of a human operator is required to check and adjust, if necessary, the results of the inverse kinematics (IK), after joining the human skeletal model with the exoskeleton. If this initialization is poor, the EIKPE will produce estimations that do not match the shape of the angle vs. time curves of the GT, introducing a constant bias. This issue is due to the unavoidable absence of markers on the human hip, knee, and ankle joints when the subject is wearing the exoskeleton.

The second limitation is related to the kinematic constraints between the human and the exoskeleton. In the current implementation, these constraints are constant over time and do not account for the compliant behavior of soft biological tissues

or other non-rigid exoskeletons components, such as braces. These elements change their shapes under the effect of interaction forces, e.g., during changes in walking speed or level of robot assistance. In these cases, the relative movement between the exoskeleton and the human may diverge from the one estimated by the EIKPE.

The third limiting factor is represented by the limited sample size of the experiment performed, which included seven healthy people, and the motion considered, limited to treadmill walking. Larger experimental studies with higher number of people, including patients, and on different tasks are required to validate the suitability of our methodology for industrial (e.g., human capability enhancement) and clinical (e.g., neurorehabilitation) applications.

The aforementioned considerations, while showing the limits of our approach, also provide clear indications that, even in presence of strong simplifications, modeling the relative motion between the human and the robot produce significantly better results than conventional "rigid" approaches. This, in our opinion, represents important evidence that supports and motivates the following next research steps in this complex and emerging field: (a) the study the sensitivity of EIKPE to non-modeled dynamic behaviors, e.g., the elasticity of exoskeleton braces or soft tissues, (b) the generation of personalized musculoskeletal models from medical images (e.g., computerized tomography) to improve the prediction accuracy in presence of bone deformities present in specific populations, e.g., cerebral palsy, (c) the implementation of a real-time version of the EIKPE with musculoskeletal models that can be used in the control loop of the exoskeleton, (d) the inclusion and testing of new predictive models of interaction forces, including models of soft tissues and robotic compliant elements, (e) testing the accuracy of the EIKPE across different type of motor tasks, e.g., slopes, sit-to-stand, rough terrains.

## CONCLUSIONS

In this work, we presented a methodology, called EIKPE, that allows to generate subject-specific skeletal models to quantify the human-exoskeleton interaction at kinematic level. We have implemented a version of the EIKPE for the lower limb with the objective of testing whether such model allows to predict, with sufficient precision, the human joint motion starting from the knowledge of the exoskeleton motion. We have assessed, in terms of the RMSE and ROME metrics, the estimation errors of the EIKPE with respect to real motion of seven healthy subjects, and compared them with a traditional rigid model that assumed no relative motion between human and exoskeleton. Our results suggest that EIKPE can be used to predict human motion from exoskeleton motion, providing estimates close to the real joint angles calculated from motion capture data. Compared to the rigid model, the EIKPE demonstrated improvements in range of 15–66% in the RMSE and ROME, depending on the joint considered. This method has several potential applications in

## REFERENCES


real scenarios, e.g., assessing of the adaptability of a particular exoskeleton to specific subjects, monitoring the human-machine interaction in real-time, or improving assist-as-needed control strategies.

## ETHICS STATEMENT

This study was carried out in accordance with the recommendations of Research Committee of the National Hospital for Paraplegics. The protocol was approved by the Ethics Review Board of Complejo Hospitalario de Toledo. All subjects gave written informed consent in accordance with the Declaration of Helsinki.

## AUTHOR CONTRIBUTIONS

DT and CC: contributed equally to the development and writing of this work; ÁB, JG-V, Ad-A, JM, JF, and JP: contributed to the conception of the presented work, theory verification, and supervision of the project; DT and ID: contributed to the experimental setup, subject recording, and data review; CC, NL, and ÁB: contributed to the implementation of the methods, data processing, and article writing. All authors discussed the results and contributed to the final manuscript.

## FUNDING

This work was supported by the FP7 Project BioMot Smart Wearable Robots with Bioinspired Sensory-Motor Skills, grant agreement n◦ 611695, and the H2020 Project EUROBENCH European Robotic framework for bipedal locomotion benchmarking, grant agreement n◦ 779963.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Torricelli, Cortés, Lete, Bertelsen, Gonzalez-Vargas, del-Ama, Dimbwadyo, Moreno, Florez and Pons. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Posture Control—Human-Inspired Approaches for Humanoid Robot Benchmarking: Conceptualizing Tests, Protocols and Analyses

Thomas Mergner\* and Vittorio Lippi

Department of Neurology, University Clinics of Freiburg, Freiburg, Germany

Posture control is indispensable for both humans and humanoid robots, which becomes especially evident when performing sensorimotor tasks such as moving on compliant terrain or interacting with the environment. Posture control is therefore targeted in recent proposals of robot benchmarking in order to advance their development. This Methods article suggests corresponding robot tests of standing balance, drawing inspirations from the human sensorimotor system and presenting examples from robot experiments. To account for a considerable technical and algorithmic diversity among robots, we focus in our tests on basic posture control mechanisms, which provide humans with an impressive postural versatility and robustness. Specifically, we focus on the mechanically challenging balancing of the whole body above the feet in the sagittal plane around the ankle joints in concert with the upper body balancing around the hip joints. The suggested tests target three key issues of human balancing, which appear equally relevant for humanoid bipeds: (1) four basic physical disturbances (support surface (SS) tilt and translation, field and contact forces) may affect the balancing in any given degree of freedom (DoF). Targeting these disturbances allows us to abstract from the manifold of possible behavioral tasks. (2) Posture control interacts in a conflict-free way with the control of voluntary movements for undisturbed movement execution, both with "reactive" balancing of external disturbances and "proactive" balancing of self-produced disturbances from the voluntary movements. Our proposals therefore target both types of disturbances and their superposition. (3) Relevant for both versatility and robustness of the control, linkages between the posture control mechanisms across DoFs provide their functional cooperation and coordination at will and on functional demands. The suggested tests therefore include ankle-hip coordination. Suggested benchmarking criteria build on the evoked sway magnitude, normalized to robot weight and Center of mass (COM) height, in relation to reference ranges that remain to be established. The references may include human likeness features. The proposed benchmarking concept may in principle also be applied to wearable robots, where a human user may command movements, but may not be aware of the additionally required postural control, which then needs to be implemented into the robot.

Keywords: humanoid robots, sensorimotor system, posture control, human-like versatility and robustness, benchmarking

#### Edited by:

Jan Veneman, Tecnalia, Spain

#### Reviewed by:

Antonio J. del-Ama, National Hospital for Paraplegics, Spain Edwin Van Asseldonk, University of Twente, Netherlands Andrej Olensek, ˇ Univerzitetni Rehabilitacijski Institut, ˇ Slovenia

#### \*Correspondence:

Thomas Mergner thomas.mergner@uniklinikfreiburg.de

Received: 30 January 2018 Accepted: 18 April 2018 Published: 05 May 2018

#### Citation:

Mergner T and Lippi V (2018) Posture Control—Human-Inspired Approaches for Humanoid Robot Benchmarking: Conceptualizing Tests, Protocols and Analyses. Front. Neurorobot. 12:21. doi: 10.3389/fnbot.2018.00021

## INTRODUCTION

Considerable progress in the sensorimotor skills of humanoid robots has been made over the recent years, such as in bipedal walking (Vukobratovic and Borovac, 2004; Clever and Mombaur, 2017). Despite this progress, the human sensorimotor abilities still represent the ''gold standard'' for the humanoid robots (Nori et al., 2014; Torricelli et al., 2015, 2016). Currently, the robotics community is taking an important step towards developing objective standards for these skills<sup>1</sup> . An aim is to define benchmarks which allow for objective comparisons among robots and to thereby foster their progress. The benchmark tests may address in some form or other human-likeness of the robot's performance (Torricelli et al., 2016). Furthermore, they address posture control as an issue to be tested in addition to movement performance. Posture control is typically required, for example, when a robot tries to maintain balance while walking across rough terrain or when it needs to compensate the gravitational ankle torque during a voluntary body lean. Generally, the ability of posture control can also be tested separately from movement control by applying external disturbance such as a push against the body while standing. In this Methods article, we develop a concept for benchmark tests of posture control in humanoid robots. The concept addresses the generic principles underlying the human posture control features versatility and failsafe robustness. We hold that these features are based on simple basic mechanisms by which, during phylogenetic development over millions of years, even primitive animal species have learned to use and to combine the solutions needed to deal with the physics of the terrestrial environment. Elaborating on their functional basis, we try to make them testable in humanoid robots for benchmarking. Our hypothesis is that providing the robots with human-like versatility and fail-safe robustness in their sensorimotor control will help them to perform better in complex sensorimotor scenarios (compare<sup>2</sup> ).

To introduce our hypothesis, we will describe further below some basic sensorimotor mechanisms that we think are underlying the human versatility and fail-safe robustness abilities. We do not claim that these mechanisms are the only possible or ultimately best prerequisites for these abilities. Rather, our aim is to introduce some basic principles that shape the human sensorimotor and postural control and by this also the consequent human versatility and fail-safe robustness abilities. Versatility here is taken to mean that a standing human may involve in reaching with a hand, for example, also the torso and leg segments, thus involving either the hips joints or the ankle joints respectively, or some combination thereof. The choice for using hip and/or ankle joints provides some robustness in case that involvement of one or the other of these joints is not possible or falls short. This example shows one of several interrelations in the human sensorimotor system, which overall provide not only conflict-free interactions between its constituents, but also synergy effects and other benefits. Another example in the human sensorimotor systems is the causal chain of (a) the need to tolerate biological feedback times delays >100 ms, which (b) is achieved to a large degree by using a low loop gain for controlling the human actuation that, being force controlled, in turn shows (c) a soft mechanical compliance and in many situations (d) a low energy consumption (see Mergner and Peterka, 2017). Notably, each of the human solutions a-d taken alone may not reach optimality in terms of a specific cost function, but in view of their interrelation may represent ''good enough'' solutions (see also Loeb, 2012).

Interrelations also exist between posture control and voluntary movements in the form that posture control ''proactively'' compensates the self-produced disturbances arising from own motor activities such as the gravitational torque from a body lean in the ankle joints—this in addition to the ''reactive'' compensation of external disturbances, e.g., from an external push against the body that perturbs standing or walking balance (Mergner, 2010). Both disturbance compensations are needed to allow execution of poses and movements in the commanded (i.e., undisturbed) form. They involve posture control centers in the extrapyramidal system (EPS; comprising basal ganglia, cerebellum, brainstem centers and the cortical supplementary motor areas). The functionality of the EPS is closely linked to that of the movement commanding ''pyramidal system'' in the cortical centers with projections to the brainstem and spinal cord. EPS impairments tend to severely affect sensorimotor control, as witnessed by a variety of motor impairments in neurological patients (Bastian, 1997; Visser and Bloem, 2005). Proactive disturbance compensation is predictive (feed forward) and therefore considered as advantageous compared to reactive (sensory feedback) compensation in terms of lower noise and shorter time delays, which is supposed to yield improved control stability and motor performance (Wolpert and Flanagan, 2001). The following article addresses for the testing of humanoid robots both proactive and reactive scenarios as well as their superposition.

The testing of humanoid robots for human-like versatility and robustness is eased if the robots use torque-controlled actuation as humans do. This would facilitate direct robothuman comparisons. More importantly, torque-controlled robots represent the current state of the art for ''real world'' applications. Advantages of the torque control are, for example, reduced damage when falling or when interacting or colliding with the environment, and also a better acceptance by humans when directly (physically) interacting with them. The robot tests suggested in the following are in principle, however, not specific for force-controlled actuation in that they address human-like versatility and fail-safe robustness as a general functional rather than mechanical property of the human system.

With the focus on human-like versatility and fail-safe robustness, our approach differs from recent concepts on benchmarking of robot walking and posture control, which are more general and list numerous performance tests and metrics (e.g., Torricelli et al., 2015). Despite similarities in the details of the suggested tests, a relevant difference here is that we rest our focus on basic principles of controlling the posture of a segmented biped under the premise of versatility and robustness. But we conceive that the tests and metrics first

<sup>1</sup>http://www.benchmarkinglocomotion.org/workshop-at-humanoids-2017/ <sup>2</sup>www.bbc.com/news/technology-33045713

Mergner and Lippi Humanoid Posture Control Benchmarking

have to be discussed in this field, before certain concepts are finally favored and realized in a form outlined in Torricelli et al. (2015). We justify our approach with the problems that a robot benchmarking is facing when it is confronted, as expected, with many different control methods, mechanical constructions, and targeted applications of the robots, and with differences between testing procedures across the performing laboratories. Addressing basic control issues in our concept will help to abstract from these diversities. In our approach, we define as human-like postural versatility and fail-safe robustness the property and ability of the human sensorimotor system that enables humans to cope reactively with external disturbances and proactively with the self-produced disturbances in a flexible, yet efficient way by exploiting kinematic and kinetic inter-segmental interaction effects. Specifically, our approach considers the mechanically highly relevant interaction between whole body balancing in the ankle joints and upper body balancing in the hip joints.

With this goal, this article gives next a brief overview on basic issues of the human postural control system with a focus on the fundamental principles that are underlying the human versatility and fail-safe robustness. The following chapter then explains the basic methods we are suggesting for quantitatively describing human postural responses, before reporting then our attempts to implement human control principles in our bio-inspired humanoids and presenting examples of the suggested benchmark tests in our robots. In the following we consider quantification and metrics of benchmark results, before we finally discuss in the last chapter the usefulness of the human-inspired experiments and point out that testing human sensorimotor concepts in robots may provide beneficial impulses for both the human and robot posture control research.

## BASIC ASPECTS OF HUMAN POSTURE CONTROL

## Shaping of Postural Control by the Terrestrial Force Environment

Under terrestrial conditions, humans and humanoids are in a force environment where gravity plays an important role; in general, gravity compensation accounts for most of the joint torque produced during balancing tasks (see Zebenay et al., 2015 for a comparison in a humanoid biped with human-inspired posture control between the torques produced by different external effects). Therefore, it is not surprising that in humans the vestibular sensory system is one of the first to become behaviorally evident during human ontogenesis. Based mainly on vestibular signals, newborns soon learn to first bring the head and later the trunk upright for perceptual orientation in space and for interaction and communication with the world. Humanoids must also take gravity into account in most tasks. Other field forces need to be detected and counteracted as well, such as centrifugal forces in passive transport. Such forces must be compensated for in order to allow undisturbed execution of voluntary poses and movements of the body and its segments. This is schematically illustrated in **Figure 1** for a human subject

who tries to maintain the body upright or to move it into a desired orientation in space. Here, the subject is controlling the ankle joint torque in the body's sagittal plane (biomechanically, a single inverted pendulum, SIP, scenario) against the four relevant external disturbances. These are, in addition to the field forces, the contact forces (e.g., from a push against the body) and the dynamic joint impacts associated with support surface (SS) rotation and translation. Posture control refers in the following to the compensation of these joint impacts, independently of whether they are self-produced (such as the gravitational ankle torque with an active body lean) or external, and whether they are occurring during a held body pose (e.g., upright stance) or a movement. The disturbance compensation allows humans to maintain a pose or to execute a movement as it is commanded in **Figure 1** by the active torque Tact. For compensation, the disturbances are estimated on the basis of sensory information or by learned predictions of the sensory disturbance estimates.

## Sensory Estimations of the Four Basic Disturbances and Their Predictions

In addition to the vestibular signals, other sensory signals such as those for joint angle and angular velocity and joint torque are used for human posture control (more in Mergner and Peterka, 2017). Visual information may additionally be used to improve the other sensory signals with respect to noise and accuracy, and it may partially serve as a substitute for loss in some sensory functions. The four external disturbances in the SIP scenario of **Figure 1** can be estimated by combining several of these sensory signals (Mergner, 2010). Humans may learn from repeated presentations of external disturbances to predict them and to fuse predicted and sensor-derived disturbance estimates, as has been successfully mimicked in tests using a human-inspired robot (Mergner, 2010). In these tests, it was also assumed that commanding voluntary movements is associated with prediction of sensory disturbance estimates for the self-produced joint impacts. Dealing with disturbance estimates rather than with the underlying manifold of sensory transducer signals greatly reduces processing complexity for sensorimotor control, and it also appears parsimonious for the linkages between sensorimotor control, perception and cognition.

## Analytical Description of the Human Postural Control

A prominent feature of human postural control is the sensory re-weighting phenomenon. It consists of a basically stereotype and continuous adjustments of the postural responses to the environmental and test conditions (unless stance stability requires a step). These adjustments can be described as resulting from changes in the weighting factors of sensory feedback loops, as it has been analyzed in human data obtained for a SIP scenario using visual, vestibular, and joint angle sensory information in a simplified linear model of the human sway responses to moderate SS tilts in the body's sagittal plane (''independent sensory channel,'' IC, model; Peterka, 2002). Adjustments to a stepwise increase in surface rotation magnitude were captured using a family of gain curves in terms of a describing function. The identified time delays were in the order of 150–200 ms, and the loop gain was slightly above the minimum to resist gravity. Comparable system identification procedures were later applied to the biomechanically more complex scenario of lower and upper body responses to SS tilt in the frontal plane (Goodworth and Peterka, 2010). However, the number of model parameters needed to describe the control on the basis of the experimental data clearly increased, while confidence in parameter identification and in the attribution to specific physiological processes decreased, which limits the use of a corresponding model for clinical purposes.

## Heuristic Model of Human Posture Control

The four disturbances shown in **Figure 1** and their estimation and prediction mechanisms are the basis of the disturbance estimation and compensation (DEC) model (Mergner, 2010), which can be viewed as building upon the IC in that it is able to describe the same set of experimental data. However, the DEC model describes the data with one set of control parameters. In addition, it can also describe responses to other stimuli that are typically used in posture control experiments such as pushes that impact the body. The overall resulting increase in versatility comes, however, at the expense of an increase in model complexity, which includes more sensory signals (e.g., velocity signals in the disturbance estimation channels) and nonlinear operators (threshold elements in these channels). This model entails, in addition, a qualitative step forward in robustness, in that it allows for a conflict-free superposition of two or more of the four disturbances at a time, be these external or selfproduced. This improvement is made possible by combining the compensation of all four disturbances with the control of movement execution in one control mechanism (see Mergner, 2010). The control is realized as a servo loop controller for commanding the actuator to produce the force that is required to execute a desired movement or force. Superimposed on this force command are commands from the disturbance estimators for compensating the disturbance forces. Executed action then corresponds to the desired action to the extent to which disturbance compensation is complete (note that removing in **Figure 1** the four disturbing torques makes the total ankle torque T<sup>A</sup> equal to the commanded torque Tact). This control principle appears to apply to the majority of the human skeletal joints and their degrees of freedom (DoFs) and, after implementing it in the following in modular control architecture, provides the basis for our concept of human versatility and robustness.

## Modular Control Architecture of DEC Modules

Human-like versatility and robustness profits from combining, in a flexible way, several joints in a task performance. A well-known example in posture control research is the involvement of hip movements when the balancing of stance in the ankle joints tends to become insufficient (McCollum et al., 1985). The involved sensory signals are distributed by coordinate transformations across the joints of the body, as described for the vestibular signals that arise in the head and are used to sense motion of the support base (Mergner and Rosemeier, 1998). Such sensory interrelations between body segments inspired the concept of a modular control architecture consisting of a net of interconnected DEC control modules, one for each DoF in the three planes of the human body (sagittal, frontal, horizontal; Lippi et al., 2013). Proof-of-principle tests in human-inspired robots were positive and, in addition, revealed functional emergencies such as an inter-segmental hip-ankle coordination (Hettich et al., 2014; Lippi and Mergner, 2017). In this architecture, a shift of activity from ''ankle strategy'' to ''hip strategy'' (see below) or, for example, from a pain-blocked knee to the neighboring joints when walking, occurs per default. This is here thought to represent the basis for the human robustness, while it is attributed mostly to versatility when performed proactively.

## Mutual Inspirations Between Robotics and Human Posture Control Research

Modeling and simulating human experimental results per se may have limited value in face of the high complexity of the human posture control and the many unknown factors such as sensor and actuation noise and inaccuracies. Implementing and testing the DEC model in human-inspired robots were performed for proof of principle and demonstration of ''real world'' robustness of the modeling results (Mergner et al., 2009; Mergner, 2010; Hettich et al., 2014; Lippi and Mergner, 2017). From this approach, progress for posture control of both humans and robots can be expected, and the same applies to testing alternative posture control concepts in the same robot (Alexandrov et al., 2017) and to testing of a given control concept on different robots (Ott et al., 2016). Using here human-derived criteria for robot-human and robot-robot benchmark comparisons represents a further variant of this issue. Generally, promising linkages between humanoid robotics and neuroscience are well recognized in both the robotics and neuroscience fields (e.g., Cheng et al., 2007).

## CONCEPTUALIZING BENCHMARKS TESTS FOR POSTURE CONTROL IN ROBOTS

Many posture control tests used in humans could also potentially be used in humanoid robots. Posture control in humans is most often tested under medical perspectives, e.g., for evaluation of a child's development or of balance control deficits in diseases and with aging. These tests range from simple observations of standing balance with eyes closed vs. eyes open to sophisticated walking measures on a treadmill, under consideration of age, disease, etc. As for robots, their diversity with respect to control method, construction, etc. hampers attempts to globally apply the human benchmark ratings and the diagnostic criteria for malfunction. These considerations lead us here to mainly consider benchmark tests that challenge very basic postural skills such as the compensation of the four disturbances considered above (''Heuristic Model of Human Posture Control'' section and **Figure 1**). Testing these skills may provide not only benchmark ratings, but also ''diagnostic'' hints in case of malfunction. As mentioned above, the tests we are suggesting are not restricted to compensation of external disturbances, but also cover ''postural adjustments'' occurring in association with voluntary movements. In contrast, the postural stabilization by ''rescue steps'' or by multi-contact body configurations such as to support the balancing with the hands, for example, are not considered here. The supportive effect of vision on human posture control will be considered only briefly and preliminarily. The robots' adjustments to changes in body weight as they occur with lifting and carrying external loads are also left unaccounted for at present. Generally, we suggest performing all tests with the same set of control parameters to judge the robots' automatic adjustment to the changes in the test condition. Furthermore, we suggest mainly covering the normal range of posture disturbances in the tests, which includes supportive use of the hip joints when balancing in the ankles joints, whereas ''rescue reactions'' to extreme challenges should remain unconsidered, such as how the rapid hip movements that typically occur when standing on a narrow beam renders the balancing in the ankle joints ineffective.

The approach we suggest here for posture control benchmarking aims to finally judge the performance of a given robot under the viewpoint of versatility and robustness in the sense that the robot would be able to deal with the relevant different types of disturbances or even their overlap, be they external or self-produced. This does not exclude, however, developing a robot that provides high performance in only one or a few tasks and less so in the others.

## Conceptualized Scenarios and Tests

The test scenarios we are suggesting for reactive balancing of external disturbances primarily refer to the four basic disturbances of the human posture control (compare above ''Heuristic Model of Human Posture Control'' and **Figure 1**), which equally apply to humanoid robots. Their applications are shown schematically in **Figures 2A–D** for the sagittal body plane, where balancing tends to be performed mainly in the ankle and hip joints: (A) SS rotation about the ankle joints. When using small and slow tilts in humans, disturbance torque (evoking the passive torque Tpass in **Figure 1**) and disturbance compensation occur foremost in the form of whole body rotation around the ankle joints, drawing on both the SS tilt compensation and the gravitational torque compensation. Large and fast tilts tend to evoke additional rotation of the upper body around the hip joints (see above; for frontal plane, see Goodworth and Peterka, 2010). These inter-link effects draw on coordination of the body segments. (B) Support translation (evoking Tin). Similar as with tilt, responses to moderate translation stimuli involve mainly ankle joint responses, which are complemented by additional hip joint responses with rapid stimuli. The response draws specifically on the estimator of SS translation (and on the gravitational torque compensation for evoked body lean). (C) Contact force stimulus (evoking Text), which draws for the postural response on the contact force estimator. The example in **Figure 2C** shows a controlled pull on a body harness, which typically is compensated foremost in the ankle joints. (D) Compensation of the field force gravity (evoking Tgrav) is here tested in a condition called ''body sway referenced platform'' (BSRP); for this test, spontaneous body sway is measured and used to tilt the SS along with the body such that the ankle joint angle (and its proprioception) remains essentially constant and balancing of upright stance with eyes closed primarily depends on vestibular input (the hip is typically not involved).

DEC in humans for the four scenarios A–D is typically performed with eyes closed vs. eyes open. The comparison allows estimating to which extent visual spatial orientation cues are used to improve non-ideal vestibular and proprioceptive disturbance estimates. Note that with vision, the secondary task of stabilizing gaze (in terms of eye-in-head and head-in-space stabilization) tends to be added to the balancing task in humans. In future, also humanoid robots may use visual motion and orientation cues to improve standing and walking balance (e.g., by using visualvestibular fusion to improve the vestibular signals which tend to be rather noisy; compare Assländer et al., 2015). Then, a relevant test would be to evaluate standing balancing on stationary and level support in the presence of a moving scene (E). This test draws on the perceptual ability to suppress visual self-motion illusions that may result from visual surround motion (compare Mergner and Peterka, 2017).

Not depicted in **Figure 2** are scenarios of testing proactive balancing. For these, we suggest ''voluntary'' (commanded) whole body leans in the sagittal plane around the ankle joints (e.g., roughly sinusoidal of about 2–4◦ forward and back) and of the upper body around the hip joints (≈3–10◦ forward and back), noting corresponding compensatory counter-leans for balancing the center of mass (COM) in the hip and ankle joints, respectively (compare further below, **Figure 3F**; also **5A**). This allows drawing on the human-inspired ability to deal, in a conflict free way, specifically with superposition of self-produced and external disturbances. To this end, we suggest superimposing such voluntary movements on concurrent SS tilts with different waveforms (**Figure 2F**; compare example in **Figure 5B**). In principle, the suggested tests (**Figures 2A–C,E,F**)

can, in addition to the sagittal plane, also be performed in the frontal plane or some intermediate plane using the same testbed (compare Lippi and Mergner, 2017). However, interpretation is more difficult due to several factors such as the more efficient, yet also more complex body mechanics in these planes and a strong dependance on the legs' stand-width, as demonstrated for humans (Goodworth and Peterka, 2010).

The stimulus parameters such as stimulus magnitude and waveform may influence the usefulness of the suggested tests in robot-robot and robot-human comparisons. In human posture control, researchers nowadays typically use well-defined kinematic stimuli, most of which can equally be applied in the tests of the humanoid robot. Sinusoidal stimuli allow collecting responses that can be used to quantitatively characterize the response dynamics in the form of Bode diagrams. The experimental value in humans is somewhat limited, however, by the predictability of these stimuli (unless one uses sums of sine stimuli). This limits drawing conclusions about the involved sensory functions but would not be relevant in robots without implemented prediction. Humans consider prediction as difficult in the pseudo-random ternary sequence stimulus (PRTS), introduced by Peterka (2002; see **Figures 3A,B**, **4**, **5B**). It allows evaluation of gain, phase, and coherence of the disturbanceevoked body excursions over a defined frequency range (for data processing, see below). Transient stimuli may be applied with and without prediction (e.g., by onset announcement). A transient stimulus with ''raised cosine velocity'' (RC) profile is similar to a smoothed ramp and to the profile which humans use in many targeting movement tasks (the profile in the transient phase is given by v(t) = –A · f · cos(2πft) + A · f, where t is time, A is angular displacement, and f is dominant frequency). Applying this stimulus with standardized parameters may allow for a fast and simple estimation of static and dynamic postural response components.

For evaluation of the postural responses in humans it often suffices for a fast overview to test the SIP scenario, and to calculate the whole-body COM responses from measures of leg and trunk excursions (recorded for example using an optoelectronic device) and the body's anthropometrics. When the hip becomes involved, additional calculations are required

to obtain the COM of the HAT (head-arms-trunk) segment. In robots, one may also calculate whole-body and HAT COM motions using internal sensor signals (e.g., from IMU and joint angle sensors). Using the PRTS stimulus requires more extensive calculations, but these have the advantage that one can obtain frequency response functions (FRFs) over a broad frequency range for different peak-to-peak (pp) amplitudes and thus can better appreciate what the hip and ankle joints contribute to the

balancing in terms of dynamics (see Hettich et al., 2014). The proposed benchmarking tests are listed in **Table 1** with suggestions for stimulus magnitudes and waveforms, which we took from previous work on human balance control in our laboratory. A future aim would be to add to the table ranges for the performance measures, which are still to be established in human and robot experiments performed with a normalization for body weight and COM height (compare below). Also, considerable simplifications of the test performances may be developed and offered in future as alternatives. For example, the proposed BSRP test can be viewed as a ''soft terrain'' test and quantified by superimposing foam rubber layers. Overall, the suggested tests can be performed with relatively simple equipment such as a plate with an axis for tilting and a plate based on two or more roll axes for translation. Instead of the raised cosine, RC, velocity function, one may use a low-pass filtered smoothed ramp. The pull devices can be replaced by manual pulls, measuring the moment arm around the ankle joints and the pull force using a force (Newton) meter. Measures of COP require a force plate, while measures of whole-body and trunk COM requires recording of body and trunk angles respectively (for which sticks connected to potentiometers may suffice), given the body anthropometrics are known (compare Alexandrov et al., 2017). Changing weight of the body or its parts, which is known to affect human postural responses (Dietz et al., 1989), will affect also the robot's responses, whereas preexisting weight differences between human and robots hardly affect the responses (see below).

#### TABLE 1 | Suggested posture control benchmark tests.


Compare Figure 2 (SS, support surface; pp, peak-to-peak; COM, center of mass; PRTS, pseudorandom ternary sequence stimulus; RC, raised cosine velocity stimulus). The following "short list" likely suffices to estimate the potential value of a more comprehensive testing: (1) RC (0.2 Hz dominant frequency) rotation of SS (compare A) with 1◦ and, to estimate hip cooperation, 4◦ . (2) 0.2 Hz RC SS translation with pp 2 cm or 4 cm displacement (see B). (3) Manual pushes against front and back of robot (compare C). (4) Instead of BSRP (D) standing on compliant support (e.g. foam rubber layers) with and without moderate pushes. (5) Commanded 2◦ whole body forward and backward leans in the ankle joint with RC profile. (6) Repeated 4◦ RC trunk forward leans (should not lead to a considerable forward lean of the leg segment). (7) Repetition of previous test 6 on SS tilting with pp 2◦ . The tests are restricted to the sagittal plane (no hip fixation and vision absent) and evaluation is restricted to angular motion of trunk and leg segments. Successful smooth performance with one and the same set of control parameters may be taken as human-like criterion. A corresponding long list remains to be worked out in collaboration with several robotic labs. Concerning last column, Vision, compare Figure 2 and text.

FIGURE 4 | Frequency response functions (FRF) of sagittal COM sway responses for peak-peak 1◦ and peak-peak 4◦ PRTS stimuli of a human subject (A) and Posturob II (B). Gain, phase and coherence functions across frequency characterize the tilt-evoked sway. Gain gives the amplitude ratio between sway response amplitudes and tilt stimulus amplitudes, with a gain of unity indicating that the sway response amplitude equals the stimulus sway, while a gain of zero indicates that the stimulus did not evoke any sway. Phase characterizes the temporal relation between tilt stimulus and sway response. Coherence is a measure of the signal to noise ratio of the stimulus evoked sway. Responses to this stimulus lend themselves to evaluation of human-likeness (see text).

The suggested tests draw strongly on findings for human postural responses and in robots are testing thus implicitly human-like performance. As already pointed out, presented with moderate external stimuli in the body's sagittal plane, humans tend to primarily use the ankle joints for balancing stance, as if controlling an inverted pendulum (intrinsically

unstable; SIP scenario). When increasing stimulus amplitude, especially with SS tilt stimuli, humans tend to also involve the hip joints for COM stabilization in a gradual transition from a SIP into a double inverted pendulum (DIP) balancing that uses the hip in addition. The hip contribution is known to depend on a variety of factors such as the stability of the support base or the stimulus amplitude. If insufficient disturbance compensation by the ankle joints is predicted, posture control may even primarily use the hips (McCollum et al., 1985; compare Atkeson and Stephens, 2007). In the context of reactive balancing in robot benchmarking, we consider the volitional or task/situation-dependent involvement of the hip as a human-like versatility feature and its use as fail-safe backup as the robustness feature. Interestingly, a hip-ankle coordination planning is not always required. For example, commanding the ankle joints to maintain the body COM above the ankles in a robot automatically led during ''voluntary'' hip bending to the ''emergence'' of compensatory counter-leans of the legs segment (compare Hettich et al., 2014 and below).

An important question is how to normalize the suggested balancing tests in face of the considerable differences in height, weight, and number of DoFs of the robots. Similar as in modeling approaches of human standing balance (see ''Basic Aspects of Human Posture Control'' section) one may treat the robot as an inverted pendulum and measure the stimulus response in terms of angular body sway. In this approach, the control of the balancing is related to the mass of the whole body COM and its height above the actuating joints (here the ankle joints, but the approach is in principle applicable also to each body segment that is held upright such as the trunk and the head with respect to its supporting joints). A further advantage is that the envisage normalization would allow to standardize the parameters used for stimulation such as the amplitude of SS tilt, for example. An example of the suggested approach is given in the ''Examples of Robot Tests'' section and **Figure 6**.

(a) artificial vestibular sensor, see Mergner et al. (2009); (b and e) hip joints with angle/angular velocity sensors; c, pneumatic muscle and f, force sensors for actuation control; (d,e) ankle joints with angle/angular velocity sensors; (g) ground reaction force sensors under heels and forefeet; (h) hexapod platform for tilt, translation, and BSRP). In the 14 DoF Lucy Posturob (B), force-controlled actuation is using spindle drives; other technical features are analogous to those in Posturob II.

For addressing the versatility and robustness issue directly, one may inactivate the hip postural control in ankle joint tests (e.g., in the test shown in **Figure 3F**) and the ankle postural control in hip joint tests (test in **Figures 5A,B**). Finally, poor results for the here described benchmark tests may possibly predict failure in the balancing of walking, since this also involves postural control in the ankle and hip joints that respond proactively to the self-produced and reactively to unforeseen external disturbances. However, stabilizing walking balance is overall clearly more complex, involving control of body dynamics with foot placement adjustments.

## EXAMPLES OF ROBOT TESTS

Considering benchmarking of human-like versatility and robustness for humanoid robots meets already existing fruitful interrelations between the respective human and robotics research fields. For example, while robotics seeks inspiration from humans for robots (e.g., Pfeifer et al., 2007), researchers of human postural control have tested their concepts in robots for proof of principle and ''real world'' robustness in face of noisy and inaccurate sensors and actuation (e.g., Mergner et al., 2009; Mergner, 2010; Hettich et al., 2014; Lippi and Mergner, 2017). The tests suggested for benchmarking in **Figure 2** and **Table 1** have all been used in human posture control experiments and some of them also in robot experiments (see Appendix). For the envisioned benchmarking of robots, robust protocols must still be worked out to deal with the complex benchmarking issue and the large diversity of software and hardware of the robotics solutions (compare above and ''Discussion'' section). In the following, we present proof-of-principle examples of some of the envisioned tests. The tests were performed mainly in Posturob II, a human-inspired robot that has human-like anthropometrics, uses force controlled actuation and the DEC model (in a slightly modified form compared to Hettich et al., 2014) for postural control in two DoF (hip and ankle joints in the sagittal plane; **Figures 3A–F**, **4B**, **5A,B**, **6A**; see Hettich et al., 2014, for details of the robot and its control, the optoelectronic recording of its trunk and leg motions, calculation of the body COM excursions, and the testbed, a human posture control laboratory). In all of these tests, the same set of control parameters and adjustments were used for Posturob II, and visual input was not implemented. In **Figures 6A,B**, tilt responses of Posturob II were compared to those of another robot, the 14 DoF robot called Lucy Posturob (anthropometrics and technical solutions are different; also Lippi and Mergner, 2017). The main results of the robot tests are listed in the following text.

**Figure 3** presents responses of Posturob II as time series data for the disturbance scenarios suggested in **Figures 2A–D,F**. Shown are postural responses in terms of stimulus-evoked body COM sway. The examples also serve to show the three suggested stimulus waveforms (PRTS in A and B; sine waves in C and E; BSRP in D and RC for body lean in ankle joint in F). Notably, panel E shows a superposition of the pull responses (panel C) and the BSRP condition (D; push omitted in E). Note from this figure and those which follow that disturbance compensation tends to be suboptimal (considerably undershooting), which represents a typical human-likeness feature (compare ''Discussion and Is Human-Likeliness an Advantage?'' section).

**Figure 4** shows frequency response functions (FRFs) for body COM responses to SS tilt in terms of gain, phase and coherence plots of a human subject (panel A; unpublished material from previous experiments conducted with written informed consent and the study was approved by the Ethics Committee of the Freiburg University Clinics and was in accordance with the 1964 Helsinki Declaration). Panel B shows corresponding responses of PostuRob II (compare the responses to the six consecutive PRTS stimuli in **Figure 3A**). Coherence in **Figure 4** is a measure of the frequency-dependent signal-to-noise ratio (calculated by dividing the squared, absolute value of the averaged cross power spectrum by the product of the averaged input and output power spectra). Coherence is lower in the human responses than in the robot's responses, similarly as previously reported (e.g., Hettich et al., 2011), which may suggest that humans tend to show larger response variability, attributable to larger sensor and motor noise. Otherwise the main features of the robot's responses resemble those of the human subject. In particular, common to both is an ''amplitude non-linearity'' in terms of clearly larger gains of the evoked sway for the peakto-peak (pp) 1◦ PRTS stimulus than for the pp = 4◦ stimulus (attributed to detection thresholds in the disturbance estimates; see Maurer et al., 2006; Mergner, 2010). This similarity and those for the gain and coherence curves may be used to consider a robot's FRF more or less human-like (see below ''Discussion'' section).

**Figure 5** shows commanded (''voluntary'') sinusoidal forward lean movements of the trunk in the sagittal body plane of Posturob II. The leg segment shows corresponding counter leans, which support the balancing of the COM over the base of support, i.e. the area under and between the feet. The counter leans emerged from the intrinsic interaction between hip and ankle DEC control modules (the command for the ankle joints was to maintain the body COM vertical above the ankles; compare Hettich et al., 2014).

**Figure 6** shows a comparison between the postural responses of the robots Posturob II (A) and Lucy (B) to sinusoidal SS tilts. Body weight without feet (BW) and COM height (H) above the ankle joints of the two robots differed considerably (Posturob II: BW = 67 Kg, H = 167 cm; Lucy: BW = 17.5 kg, H = 139 cm). Note that their COM responses in terms of sway angle are, nevertheless, of similar magnitude. This owes to the fact that COM and it height are taken into account as parameters in the ankle and hip joint controllers in both robots in terms of mgh (where m gives the mass of the robot above the feet, h the COM height, and g is acceleration due to gravity; compare above, normalization across robots).

## DISCUSSION

## Empirical Benchmarking: Quantification of Experimental Results, Metrics and Human Likeliness Measure

The experiments discussed in this article provide several possibilities to characterize the robot responses to external and self-produced disturbances. Assessing and comparing a robot's performance require the definition of ''performance metrics'' on the basis of the experimental data. The first and most straightforward quantification consists in the ability to stand the imposed disturbances: for each scenario the maximum stimulus amplitude successfully tested may be used as a score. In general, however, it is not advisable to push the robots to failure. Rather, it would be desirable to obtain performance measures using basically ''safe'' moderate disturbances. The experiments may be then interpreted in terms of the sensitivity to the applied disturbances. For a sinusoidal stimulus, for example, gain as the ratio between the amplitude of the disturbance and the amplitude of the evoked body COM sway can be used; and, this in relation to a corresponding gain range in a human data set (to be established) may further be used as a measure of human likeness. However, this solution faces for the sinusoidal stimuli the aforementioned problem of their predictability. An alternative would be to use unpredicted ''transient'' stimuli with RC profile, where the uncompensated response of the system may be measured in terms of static gain (residual static lean response), overshoot and settling time. Such indices can be computed similarly for both commanded voluntary movements and external disturbances. The third possibility we consider is the PRTS stimulus. Its advantages are that humans consider it as unpredictable and that it allows for a description of the response in terms of a FRF. The total power of the FRF gives a measure of the sensitivity of the robot's response to the applied disturbance, which can be directly compared across robots. As mentioned above, it is notable that healthy human subjects typically do not show perfect compensation of the external stimuli, but always some sway, which has been explained by some threshold mechanisms in the control. This suggests that a total disturbance rejection may overall not be an optimal solution and it certainly does not represent a human-like feature. In fact, trying to stand for some time absolutely motionless is for humans extremely difficult and soon starts to be painful.

The FRF description contains further features that may be used to quantify the human-likeness of the robot response in comparison with human data. Experiments performed with human subjects provide a behavior description that can be used for comparison with robot responses. Human behavior can be defined in terms of several features of the gain values across each represented frequency, and also by the phase and coherence. As mentioned above in the description of **Figures 4A,B**, such features may reveal a human-like amplitude non-linearity or, on the basis of the phase characteristics, a robustness of the control's dynamics or, on the basis of a sufficiently high coherence, a hint on response reproducibility. It remains to be shown that such features of a robot's FRFs can be compared with the ones observed in humans using multivariate statistical techniques. Roboticists may find it more familiar, though, to train models such as neural networks or support vector machines on the available human response datasets and classify the robots' responses as belonging to the distributions of human responses (healthy or some particular class of patients). In both cases the idea would be to base the measure of human-likeness to reference sets of human data.

In general, we propose a set of test scenarios that relate to basic mechanisms of the human posture control system but are otherwise empirical and to a large extent independent from the particular robotic platform tested, in that the evaluation is based on measurable physical variables such as body sway and on normalizing across anthropometrics. While posture control can be considered a basic sensorimotor control skill, its efficiency also builds on lower level elements such as energy efficiency, compliance, and the actuators' dynamic performance. These aspects can also be tested, e.g., by measuring energy consumption directly. However, such low-level issues may be too specific for the hardware used and not easily relatable to human-inspired concepts. Nevertheless, the outcomes of such tests may provide useful insights about the implemented properties. For example, in Ott et al. (2016) it is shown how including passive stiffness and delays affect posture control and balance performance. Details of the performance of the suggested tests and the evaluation protocols as well as human reference data must still be worked out.

## General Robot Evaluation Issues

In the robotics literature, different robot evaluation principles have been proposed for different tasks. In general, robots may be compared with each other with respect to their ability to solve given problems or to perform given tasks. In O'Kane and LaValle (2008), for example, evaluation of robot capabilities is inspired by the principles used in computational theory to evaluate algorithms, i.e., by evaluating whether the robot can solve a problem or not and assessing how efficiently the problem is solved. In order to perform such an evaluation, it is crucial to formally define the problem and to find measures to evaluate the efficiency of its solution. From this, one should not be misled to postulate that a benchmark framework should be specific for each robot type and task. Rather, although particular with respect to the general case (i.e., not applicable for fixed-base and wheeled robots), biped balancing is a challenge to all humanoids, and its benchmarking can be based on a set of well-defined motor tasks. Furthermore, the benchmarking can use human postural skills as a reference, which is occasionally done in human-inspired robotics, typically by evaluating a set of human skills with various levels of difficulty. For example, in the field of developmental robotics (see Guerin and Rat-Fischer, 2014) a benchmarking framework based on the skills that are progressively acquired by humans in the period from birth to early childhood is applied. Correspondingly, one may evaluate to which extent the postural responses of a humanoid resemble those of humans. Generally, the assessment of selected challenges, which are often artificial and chosen to be easily replicable, should be empirical and independent of the specific hardware implementation. Such a premise is behind the design of the Turing test in the field of artificial intelligence and its variants. Furthermore, in view that artificial agents are not (yet) able to pass the Turing test in the general sense, it has been proposed to address sub-skills that must be solved in the context and may be tested separately in order to provide at least an insight into what may be still missing (Cohen, 2005). The tests described in this article for humanoid posture control can be seen as the implementation of these principles, i.e., (1) empirical evaluation; (2) applicability to different hardware; and (3) testing sub-skills that relate to the issue of robot interaction with the real world.

## From Human Experiments to Robot Evaluation Principles

Addressing here human-like versatility and robustness for benchmarking of posture control in robots, we focused on basic components of the human posture control that are neither task-specific nor hardware-specific. These components may therefore be used to rate, predict and possibly explain the shortcomings of the sensorimotor performance of robots, or they may identify limiting amplitude and frequency margins. We conceive that apparent discrepancies between impressive task performances in internet videos and failures in robot competitions with complex ''real world'' scenarios owe mainly to the problems of how to cope with superposition of two or several disturbances or of active movements with external disturbances. Also, we attribute versatility and robustness of sensorimotor control to the ability of humans to exploit the multi-segment body kinematics for distributing a demanding performance across two or more joints and to use the same ability for compensating for local flaws. These features of the human posture control have basically the same importance for humanoid robots. In addition to making task performance of robots more versatile and robust, they may facilitate human-robot collaborative interactions. Our focus on human-like posture control as a crucial precondition for many movement performances is supported by a recent outline concept on benchmarking humanoid locomotion, where a considerable part is devoted to posture control (Torricelli et al., 2015).

Posture control benchmarks may improve the interpretation of general sensorimotor performance benchmarks. Low ratings for such performances may be due to either insufficient postural disturbance compensation or to suboptimal action planning, commanding, and execution control. Furthermore, postural control tests may reveal and define restrictions for one or the other external disturbance, which can then be taken into account when planning performance tasks for a robot. The overall effort and time expense for the benchmarking is larger when posture control is separately tested, but this appears acceptable when one restricts the posture control benchmarking to the very basic aspects and to a relatively small number of meaningful tests.

One may object that the described external disturbances challenge a robot's performance especially during passive transport in a vehicle, for example. This challenge may represent an exceptional situation for which one could conceive some form of special solution, for example in terms of a passive fixation or by having the robot actively stabilizing its stance by holding with the hands. We contend, however, that challenging disturbances may have many reasons, mostly from mechanical interactions with the environment (e.g., collisions, work requiring interaction with machines or other robot agents). Furthermore, these disturbances tend also to occur during proactive movements such as walking, when body acceleration (or slowing, change in direction, etc.) produces force impacts on its buttress. The same applies to each moving body segment that is buttressing on a supporting body segment. Conceivably, focusing the suggested benchmarking tests on movements of the relatively heavy COM of the trunk and on the whole-body COM above the feet is clearly more relevant than considering head or arm movements.

The examples of robot tests shown above demonstrate that the robots' posture control can be characterized and evaluated in terms of both body COM dynamics and intersegmental coordination. The way in which humans exploit inter-segmental coordination depends on the functional context. For example, when a limited contact area with the SS does not allow for a sufficiently safe balancing of the body COM based on ankle torque, humans typically use additional hip movements to support the balancing, which can eventually fully take over the task. Such a ''hip-strategy'' emerges mainly in the presence of very intensive external perturbations (Atkeson and Stephens, 2007). In the scenarios described above, the balance behavior has not been pushed to such limits, however. In the extreme case, the hip strategy would require very forceful rapid movements that, at the current state of the art, are not implemented in robots. Such very strong perturbations may not safely be covered even by humans.

Our definition of human-like versatility and robustness and their interrelation refers in the present context to the human ability to distribute the performance of a sensorimotor task across two or more joints or to variably shift the performance from one joint to another. In the DEC concept, these mechanisms have a basis in the modular architecture of the control, including the emergence of inter-segmental movement coordination and a reduction of inter-segmental coupling forces (Hettich et al., 2014; Lippi and Mergner, 2015). Ultimately, however, the posture control mechanism as a whole builds on the ability to produce in each DoF of the skeletal system the compensation for the basic four disturbances (compare **Figure 1**) in a context-adequate way with respect to their overlap and timing. As already mentioned above, these responses are produced with one and the same set of control parameters and in conflict-free superposition with the movement execution control. Demonstrating this in the above robot experiments leads us to the suggestion to try the same in future robot benchmarking.

## Is Human-Likeliness an Advantage?

Referring robot benchmarking to human sensorimotor behavior and suggesting that robot performance be evaluated in terms of human-likeliness poses the general question about the value of the human-likeliness criterion. As already pointed out in ''Introduction'' section, human performance is still superior to that of robots and several human sensorimotor features are currently desirable for humanoids. Following this idea we proposed the evaluation of basic posture control features that allow humans to move and balance in a variety of different scenarios. Previous work (Ott et al., 2016) provides an example that using a human-inspired sensorimotor control, which includes passive joint stiffness and some form of feed-forward disturbance compensation, increases the tolerance for time delays in the control loop. Principally, the power of such a design lies in the option to use a relatively low gain in the active control; a safe choice considering the limited time margins imposed by the delay in the loop. The compliant behavior produced by the low gain, in turn, can be considered advantageous by itself for the interaction with the environment. This suggests that biological solutions may tend to address more than one problem and provide tradeoffs between different issues. On the other hand, humanoids and technical system may in principle face specific problems that are not relevant for humans (e.g., joint angles that should not hit their limits or actuators that are optimized for narrower ranges of torques and velocities compared to those of human muscles). Or, they may not be affected by the same limitations as humans (such as the long neural delays mentioned above). Because of this, the technological value of bio-inspired mechanisms depend on the specific features of the hardware involved. Nevertheless, the human-likeness feature may be of intrinsic value in human-robot interactions. Conceivably, humans would more likely perceive the motor behavior of a robot as intelligible and predictable when it is based on human-inspired control principles, which would offer safety benefits in tasks that require humans to directly collaborate with robots. This is especially important in the case of wearable robots that can partially impose motion pattern and posture control strategies to humans.

Wearable robot devices would be perceived as more transparent and reliable if they behaved in a way that reflects the natural motor schemas of the user. For this reason, we imagine that human-likeness should be evaluated specifically as one of the measures to be taken into account in a benchmarking framework for humanoids. Furthermore, several of the posture control mechanisms identified in humans may be of considerable relevance to robotic engineers. One example is given by the aforementioned solutions used by humans to deal with the relatively long sensory feedback time delays for disturbance estimates and the resulting challenges for control stability (compare above in ''Introduction'' section the interrelation of low loop gain with low mechanical resistance and low energy consumption). Another related example, which may be potentially interesting for robotics where processing time delays are considerable, is that the human responses to the postural disturbances occur in a cascade of three steps, the first being an instant resistance from passive muscle and connective tissue properties, which is followed by an automatic and stereotype short-latency proprioceptive reflex (latency 20–40 ms), after which the context-specific multisensory and voluntarily adjustable long-latency disturbance compensation develops (often referred to as ''long latency reflex''). It has been demonstrated in robots (Ott et al., 2016) that preceding the disturbance-specific counteraction by some early and fast response in equivalence to the human passive stiffness may improve control stability. A further means by which humans cope with the sensory feedback time delays is to learn external disturbances and then to predict the corresponding sensory estimates, and to also use prediction for self-produced disturbances (see ''Sensory Estimations of the Four Basic Disturbances and Their Predictions'' section). Preliminary robot experiments demonstrate an improvement in postural control when this includes prediction (Mergner, 2010).

Another and already previously considered point for humanlikeness benchmarking is the improvement of postural control, and of sensorimotor control in general, when humans can involve visual spatial orientation and motion cues (Torricelli et al., 2016). In the absence of vision, human arm-reaches fall short, and walking slows down and becomes insecure with an increased risk of falling. Particularly strong is the

## REFERENCES


beneficial effect of vision in humans with degraded vestibular function, which is consistent with the notion of a strong visualvestibular co-operation in sensorimotor control, self-motion perception, and spatial orientation. The improvement by vision in vestibular-able subjects is mainly attributed to a reduction of high vestibular noise by the visual-vestibular signal fusion (Mergner et al., 2009; van der Kooij and Peterka, 2011; Assländer et al., 2015). A basic problem in the use of visual cues is in the evaluation of whether optic flow is stemming from self-motion or from visual surround motion. Its solution in humans involves the interpretation of a manifold of visionderived motion and orientation cues, using cognition and learning, and it includes visual-vestibular fusion mechanisms that are still not completely understood to date (see, e.g., Mergner and Peterka, 2017). Overall, current knowledge of the human perception and sensorimotor systems is still rather limited. Studies on sensorimotor control in humans and humanoid robots will likely profit from each other and to some extent may proceed in parallel by using mutual inspirations from each other. Drawing on human-likeness inspirations for robot benchmarking to better understand, for example, the role of vision for sensorimotor skills remains a task for the future in both research fields.

## AUTHOR CONTRIBUTIONS

TM and VL performed the experiment, collected and analyzed the data and performed the computer simulations. Both authors contributed to the interpretation of the data, drafted and critically revised the manuscript and finally approved the manuscript for submission.

## FUNDING

Part of this research was supported by the European Commission (Seventh Framework Programme; FP7 Grant 610454 EMBalance).


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Mergner and Lippi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

## APPENDIX

An important further step towards benchmarking humanoids would be experiments that compare across different posture control methods in the same robot, or using the same control method across different robots (as in **Figure 6**). This important and necessary next step may start when a consensus is reached on the targets in the benchmarking approach. Qualitative and preliminary experiments in this direction have been performed in our laboratory. Corresponding films on posture control from our robots can be found in the internet:

(A) For Posturob I (see Mergner, 2010) under https://www.youtube.com/user/NeurologieFreiburg showing (1) Voluntary lean; (2) Balancing in response to pull stimuli when standing on foam rubber; (3) BSRP (body sway referenced platform); (4) Voluntary leans superimposed on support surface tilt (both with different frequencies); and (5) Pull stimuli applied during stance on BSRP.

(B) For similar tests using Posturob II, see https://www. youtube.com/user/neurozentrumukl Part 1: Containing balancing of tilts with (a) PRTS and (b) superimposed pushes, Part 2: Demonstrating and explaining control of voluntary movement superimposed on balancing sinusoidal tilts, Part 3: Demonstrating and explaining balancing on fixed support surface and BSRP during manual applied pushes, and Film 2: Superposition of robot and human trunk lean movements on support surface rotations.

(C) For tests in Lucy Posturob, see https://www. uniklinik-freiburg.de/neurologie/forschung/neurologische-

arbeitsgruppen/postural-control/video.html demonstrating balancing of stance while performing movements simultaneously in the frontal and sagittal planes and compensating superimposed pushes.

(D) For the robot Toro from DLR using the DEC control under https://www.youtube.com/watch?v=3ALCTMW3Ei4 (see also: films referred to in Ott et al., 2016).

(E) For balancing stance of Posturob II when it used instead of the DEC an Eigenmovement control method (complementary material in Alexandrov et al., 2017).

# Foot Placement Modulation Diminishes for Perturbations Near Foot Contact

### Mark Vlutters\*, Edwin H. F. Van Asseldonk\* and Herman van der Kooij

Department of Biomechanical Engineering, University of Twente, Enschede, Netherlands

Whenever a perturbation occurs during walking we have to maintain our balance using the recovery strategies that are available to us. Foot placement adjustment is often considered an important recovery strategy. However, because this strategy takes time it is likely a poor option if the foot is close to contact at the instant a perturbation occurs. The main goal of this study is to gain a better understanding of how humans deal with balance perturbations during walking if foot placement adjustments are constrained by time. Ten healthy subjects walked on an instrumented treadmill and received mediolateral and anteroposterior pelvis perturbations at various instances during the single support phase. The results show that foot placement modulation in the first recovery step following anteroposterior perturbations is fairly invariant of the perturbation magnitude and direction, regardless of the onset instance. For mediolateral perturbations, foot placement adjustments strongly modulate with the perturbation magnitude and direction, but these effects diminish when the perturbation onset is closer to the instant of foot contact. For most perturbations the first recovery step was consistent across subjects for all onset instances. However, in the second step various strategies arose that were not consistent across subjects, nor within subjects, especially for perturbations applied close to foot contact. Despite these different strategies, the COP location following foot contact strongly related to the COM velocity throughout these strategies. The results show that humans have various ways to compensate for limited availability of a foot placement strategy, with strategy selection highly dependent on the instant during the gait phase at which the perturbation is applied.

## Edited by:

Diego Torricelli, Consejo Superior de Investigaciones Científicas (CSIC), Spain

### Reviewed by:

Robert Peterka, Oregon Health & Science University, United States Lorenzo Masia, Nanyang Technological University, Singapore Thomas Mergner, Universitätsklinikum Freiburg, Germany

\*Correspondence:

Mark Vlutters m.vlutters@utwente.nl Edwin H. F. Van Asseldonk e.h.f.vanasseldonk@utwente.nl

#### Specialty section:

This article was submitted to Bionics and Biomimetics, a section of the journal Frontiers in Bioengineering and Biotechnology

Received: 18 January 2018 Accepted: 06 April 2018 Published: 08 May 2018

#### Citation:

Vlutters M, Van Asseldonk EHF and van der Kooij H (2018) Foot Placement Modulation Diminishes for Perturbations Near Foot Contact. Front. Bioeng. Biotechnol. 6:48. doi: 10.3389/fbioe.2018.00048 Keywords: perturbed human walking, balance control, foot placement, extrapolated center of mass, capture point

## INTRODUCTION

Human balance control is highly flexible, with a multitude of strategies that can be addressed to reject disturbances and allow continuation of walking. One example is the modulation of ankle joint moments to affect the movement of the body. Another are inertia-based strategies such as the hip strategy, in which changes in angular momentum are used to affect linear body motion. Furthermore, foot placement modulation can change the base of support area, allowing adjustments to be made to the walking cycle. This might be achieved by adjusting both the location and timing of foot placement. To gain more insight in human balance control and the preferred ways of balance recovery, it is helpful to understand how humans maintain balance when one or multiple strategies are restricted.

The way balance is controlled depends on physical capabilities and constraints. For example, experiments in standing balance show that humans no longer utilize an ankle strategy if the size of the support surface is decreased (Horak and Nashner, 1986). This makes the ankle strategy ineffective, and possibly even threatening to balance. In walking, we have previously shown that foot placement adjustments are elicited in response to anteroposterior (AP) perturbations after physically blocking the ankle joints to make an ankle strategy ineffective (Vlutters et al., 2018). Such adjustments in foot placement were not observed following AP perturbations in normal walking (Vlutters et al., 2016). Other changes to the physical capabilities of the body also modify balance control, such as increased body sway in unilateral amputees (Geurts et al., 1992), or enhanced lateral balance performance through the use of a powered ankle device (Kim and Collins, 2015).

Constraints on balance control can also be in the form of time. Especially for foot placement modulation, time is required to make adjustments to the swing leg (Hof et al., 2010). The instance of the gait cycle at which a disturbance occurs is therefore an important factor in determining how balance will be maintained. If a disturbance occurs shortly before foot contact, there is little time to make foot placement adjustments. As a result, adjustments might have to be postponed to the subsequent step, or other balance strategies have to be addressed to compensate. Especially mediolateral (ML) disturbances given briefly before foot contact are expected to be challenging, given the already limited availability of other strategies, such as ML ankle control.

In an attempt to make predictions of balance control strategies during gait, the center of mass (COM) velocity has previously been shown to relate to the center of pressure (COP) location following foot contact, in the first recovery step following both ML (Hof et al., 2007, 2010; Vlutters et al., 2016) and AP (Vlutters et al., 2016) perturbations. For AP perturbations, this COP shift was realized during the double support phase without the need to strongly adjust the location of the leading foot as compared to the unperturbed condition. For ML perturbations however, this COP shift was made possible mainly through foot placement adjustments. In addition, this COP shift was in line with the velocity-dependent extrapolated center of mass (XCOM) concept (Hof et al., 2005). This concept is also known as the capture point (Pratt et al., 2006), which can be derived from a linear inverted pendulum model. The XCOM can be represented as a point on the floor at a horizontal distance from the COM, equal to the COM velocity times a proportionality constant ω −1 0 . If the model's COM moves toward the COP while the COP coincides with the XCOM, the model will come to an upright movement stop. If human walking has similarities to the motion of an inverted pendulum, the ability to balance and to steer the COM might be reflected by the ability to locate the COP relative to the XCOM. If there would be insufficient time to adjust the base of support through foot placement adjustments, it might not be possible to displace the COP in accordance with the XCOM, because the COP is constrained to the base of support. The COM could move in an undesired direction as a result, and the COM velocity would lose its predictive value for the COP in that step. It is unclear if such predictions would hold for the subsequent second step, especially if subjects cannot counteract the disturbances during the double support phase after the first step.

The main goal of this study is to gain a better understanding of how humans deal with balance perturbations during walking if foot placement adjustments are constrained by time. Specifically, for perturbations with an onset increasingly close to the instant of foot contact we question (1) whether foot placement adjustments diminish when there is little time to use such adjustments as a recovery strategy, and (2) whether the COP will continue to modulate with the COM velocity, in line with the XCOM? Foot placement adjustments are expected to diminish in the first recovery step given the time required to move the foot. However, strategies other than foot placement adjustments might still facilitate COP modulation with the COM velocity. Furthermore, there will be more time to make foot placement adjustments in the second recovery step, which might allow for such modulation. We investigate these questions by applying both ML and AP perturbations to the pelvis of human subjects walking on a treadmill, at various instances within the single support phase, and capturing their kinematics. We analyze the foot placement locations and COP positions relative to the COM at specific instances following perturbation onset.

## MATERIALS AND METHODS

## Participants

Ten healthy young adults without known history of neurological, muscular, or orthopedic problems participated in the study (3 male, age: 21 ± 2 year, height: 1.76 ± 0.1 m, weight: 65 ± 9 kg). The local medical ethics committee (Medisch Ethische Toetsingscommissie Twente) approved the experimental setup and protocol. All participants gave written informed consent prior to the experiment, in accordance with the Declaration of Helsinki.

## Apparatus

Here only a brief description of the experimental setup is given. A more detailed description is provided elsewhere (Vlutters et al., 2016). A dual-belt instrumented treadmill (custom Y-mill, Motekforce Link, Culemborg, The Netherlands) and two motors (SMH60, Moog, Nieuw-Vennep, The Netherlands) adjacent to the treadmill were used to deliver ML and AP pelvis perturbations during walking in a controlled way, see **Figure 1**. Attached to each motor was a vertical lever arm, which in turn was connected to a horizontal rod through a ball-joint. The rod was connected to a hip brace (universal hip abduction brace, Distrac Wellcare, Hoegaarden, Belgium), also using a ball joint. The brace was worn by the subject. Control signals for the motors were generated using xPC-target (MathWorks, Natick, MA, USA) at 1,000 Hz.

## Data Collection

Subject kinematic data of the feet, lower legs, upper legs, pelvis, upper body, and head were captured using a 9-camera motion capture system (Visualeyez II, Phoenix Technologies, Burnaby,

Canada). To this purpose a three-LED cluster was attached to each of those body segments. Additional single LEDs were placed on both lateral malleoli, and both lateral epicondyle of the femur. The torque and angle of each motor in the perturbation device were collected over UDP using an Ethernet card (82558 Ethernet card, Intel, Santa Clara, CA, USA). Ground reaction force data of the treadmill were also collected at 1,000 Hz using an AD card (PCI-6229, National Instruments, Austin, TX, USA). Both cards were part of the xPC-target hardware. The AD card was also used to generate an analog signal to synchronize the motion captures system with the xPC-target hardware.

## Protocol

Before the start of the experiment, several kinematic measurements were performed during which the locations of the bilateral first and fifth metatarsal heads, calcaneus, medial and lateral malleoli, fibula heads, medial and lateral epicondyles of the femur, greater trochanter, anterior and posterior superior iliac spines, xiphoid process, jugular notch, 7th cervical vertebra, occiput, head vertex, and nasal sellion were indicated using an LED-based probe (Cappozzo et al., 1995), relative to the LED clusters on each body segment. Using these measurements and the measured global positions of the LED clusters, the indicated points can be reconstructed in global space throughout all measurements.

During the experiment, subjects walked on the treadmill with their arms crossed over the abdomen, to prevent balancing using the arms. The walking speed was 0.63 m s−<sup>1</sup> multiplied with the square root of the subject's leg length (Hof, 1996). Subjects walked two blocks of five trials each. The first trial of each block was a 2-min baseline trial in which no perturbations were applied. The first baseline trial was furthermore used to determine the single support duration during unperturbed walking. The remaining four trials were perturbation trials. A perturbation consisted of a sudden square-wave pulse with a duration of 150 ms. Perturbation onset occurred at right toeoff, at the start of the left single support phase (SS0), at one third of the left single support phase (SS1/3), and at two thirds of the left single support phase (SS2/3). The interval between subsequent perturbations was random, between 6 and 12 s. The delivered force magnitudes were equal to 8 and 16 percent of the subject's body weight. Perturbations were directed either inward (negative sign, leftward for right swing) and outward (positive sign, rightward for right swing), or forward (positive sign) and backward (negative sign). In one block only ML perturbations were applied, in the other block only AP perturbations. Block order was randomized across subjects. Within a block, all perturbations were randomized over onset instance, magnitude, and direction. Each condition was repeated 8 times, yielding 196 perturbations in total per subject. When no perturbation force was being delivered, the interaction force between subject and motor was regulated to (near) zero using admittance control, which allowed the subject to move freely. Subjects wore a safety harness at all times to prevent the body from hitting the treadmill in case of a fall.

## Data Processing

Data were processed using Matlab (R2016b, MathWorks, Natick, MA, USA). Marker data were filtered with a 4th order zerophase 20 Hz low-pass Butterworth filter. Landmark positions were subsequently reconstructed using the probe measurements through least squares estimation (Söderkvist and Wedin, 1993). Using a method comparable to that in Zeni et al. (2008), the calcaneus and first metatarsal head landmarks on both feet were used to detect gait phase events of toe-off right (TOR), heel strike right (HSR), toe-off left (TOL), and heel strike left (HSL). All landmark data was used to estimate the location of the COM of each segment, as well as that of the whole body COM (Dumas et al., 2007). The COM position was differentiated to find COM velocities.

The unperturbed walking data from the baseline trials was used to find the average Euclidean distance between the COM of the feet at heel strike. This value was used as a scaling factor (l0) to make all position and velocity data dimensionless following (Hof, 1996). Next, all position and velocity data were expressed relative to those of the whole body COM. The velocity of the whole-body COM itself was expressed relative to the treadmill belt. All data were sorted on perturbation magnitude, direction, and onset. The COM velocity data was cut into sequences using the gait phase events. Each sequence was resampled to 50 samples to allow averaging across repetitions and subjects. All data at gait events were averaged over repetitions to obtain average data per subject. These were used to obtain subject averages and standard deviations.

To investigate the predictive power of the COM velocity on the COP location, linear least squares fits were made to the distance between the COP and the COM at TOL as a function of the COM velocity at the preceding HSR, in line with our previous analysis (Vlutters et al., 2016). Such fits have previously shown to correspond well to the XCOM concept, with the COP-COM distance proportional to the COM velocity times a factor ω −1 <sup>0</sup> <sup>=</sup> √ (l/g), in which l is the subject's leg length, and g is the Earth's gravitational acceleration. A dimensionless XCOM proportionality constant (ω −1 0 ) was calculated for each subject, and averaged over all subjects for comparison with the fits. This constant was compared to the slope of the linear fits to the data. If both are similar, the COP modulates in a comparable way with the COM velocity as the XCOM does, for specific instances in the gait cycle.

Linear mixed models were used to assess the effect of the perturbation (fixed factor, with intercept) and the onset timing (fixed factor, with intercept) on the ML and AP distance between the COM and the COP at TOL, as well as on the duration of the single and double support phases during and after the perturbation. Subject effects were included as a random factor (intercept) to account for correlation effects from repeated measures within the same subject. A significance level of α = 0.05 was used and a Bonferroni correction was applied to correct for multiple comparisons during post hoc analysis. The perturbed conditions were only compared to the unperturbed condition and not mutually to reduce the number of comparisons. Finally, the analysis was performed separately for ML and AP perturbations. SPSS statistics 21 (IBM Corporation, Armonk, NY, USA) was used for the statistical analysis.

## RESULTS

Foot placement adjustments in terms of location and time were assessed following both ML and AP perturbations in walking subjects. All data are shown dimensionless. Subject-average scaling factors to make the data dimensionless were l<sup>0</sup> = 0.41 ± 0.03 m for distances, <sup>√</sup> (g ∗ <sup>l</sup>0) <sup>=</sup> 2.02 <sup>±</sup> 0.07 m s−<sup>1</sup> for velocities, and <sup>√</sup> (l<sup>0</sup> / g) = 0.21 ± 0.01 s for durations, where l<sup>0</sup> is the average Euclidean distance between the COM of both feet at unperturbed heel strike.

For various ML perturbations one subject showed stepping strategies that were not consistent with the other subjects. These special cases are shown separately in **Figures 4**–**6**, and were removed from the statistical analysis. For the 0.16 magnitude outward perturbation applied at SS2/<sup>3</sup> the responses were not consistent across subjects, nor within several subjects. As a result, the data cannot be pooled subject-wise to represent a specific strategy. Corresponding data were omitted from **Figures 4**, **6** to prevent image cluttering, and were also removed from the statistical analysis. However, all data were included when determining the relations of the COP data with the COM velocity. As we have previously demonstrated in Vlutters et al. (2016), the underlying COP might still modulate with the COM velocity, in line with the XCOM concept, throughout different balance strategies.

## Perturbation Effects on COM Velocity

Both the ML and AP perturbations affected the subject's COM velocity, see **Figures 2**, **3**. The velocity profiles following ML perturbations appear dependent on the onset timing. Deviations from the unperturbed case obviously start later for later perturbation onset, but the way the velocity progresses changes with the onset timing as well. The effects of various AP perturbations on the COM velocity appear less dependent on the onset timing. Though later onset leads to later deviations from the unperturbed case, the velocity profiles between the different onset conditions start to appear similar again at HSL, at the start of the second step.

The disturbances move the subjects in the direction of the perturbation, such that subjects had to return to the center of the treadmill during their recovery. This return to the center can be derived from the velocity data, corresponding to the instances where the perturbed velocity data crossed the unperturbed velocity data, see **Figure 2A**. The point at which this return occurs following ML perturbations shifts with the perturbation onset, becoming later for perturbations that are applied later. This is less the case for AP perturbations, see **Figure 3B**. Finally, the COM velocity perpendicular to the perturbation direction remains relatively unaffected by the perturbation itself, but may change through subject actions following HSR. This is mainly the case for ML perturbations, especially for those with SS2/<sup>3</sup> onset, see **Figure 2B**. Subjects speed up in the walking direction for inward perturbations, and slow down for outward perturbations.

## Foot Placement Location Following ML Perturbations

Subjects modulated their foot placement in terms of location and/or timing following the perturbations. Especially ML perturbations with SS<sup>0</sup> onset lead to adjustments in foot placement location in the first recovery step, see **Figure 4**. This might be expected given that these perturbations are perpendicular to the walking direction, while there is sufficient time to adjust the foot. Note that the locations of the feet in **Figure 4** are represented relative to the COM. The location of the leading foot relative to the COM results from the step. The location of the (mostly stationary) trailing foot can change relative to the COM because the COM itself moves as a result of the perturbation. In the first step, at HSR, subjects generally placed their foot in the direction of the perturbation at an increased ML distance from the COM with increasing ML perturbation magnitude. The ML distance between the COM and the leading foot was significantly affected by the ML perturbations [F(4, 126) = 114.410, p < 0.001], the onset timing [F(2, 126) = 7.605, p = 0.001], and their interaction [F(8, 126) = 126.000, p < 0.001]. However, the post-hoc analysis revealed that the leading foot was not placed significantly different from the unperturbed case for any ML perturbation with SS2/<sup>3</sup> onset (p = 1.000). The main effect of perturbation on the ML distance between leading foot and COM is therefore mainly caused by the SS<sup>0</sup> and SS1/<sup>3</sup> onset perturbations. Other exceptions were the −0.08 inward perturbations with SS<sup>0</sup> and

SS1/<sup>3</sup> onset, which also did not lead to significant changes in this distance (p ≥ 0.589).

for perturbed conditions to prevent image cluttering. Data is shown dimensionless.

For the second step, at HSL, modulation of the foot location occurred, but the changes in ML distance between the leading foot and the COM tend to diminish for perturbations with a later onset. A possible explanation is that part of the recovery occurred during the double support phase between the first and second step, and the single support phase prior to foot contact of the second step. This way, there is less need for adjustments in the location of the foot. Note that the 0.16 magnitude outward perturbations with SS2/<sup>3</sup> onset are disregarded here, for which lateral foot adjustments did occur. For perturbations with SS<sup>0</sup> and SS1/<sup>3</sup> onset, the ML distance to the leading foot in the second step deviates from the unperturbed condition for various reasons, such as uncrossing the legs following a cross-step, or to return to the center of the treadmill. The ML distance between the COM and the leading foot in the second step was significantly affected by the perturbations [F(4, 114.101) = 23.251, p < 0.001], the onset timing [F(2, 114.068) = 9.262, p < 0.001], and their interaction [F(7, 114.058) = 2.825, p = 0.009]. Visual inspection of **Figure 4** suggests that the modulation pattern as seen in the first step in response to ML perturbations with SS<sup>0</sup> onset do not clearly re-appear in the second step for ML perturbations with SS2/<sup>3</sup> onset, which suggests recovery occurs before foot contact of the second step, even if there was no foot adjustment in the first.

## Alternative Foot Locations Following ML Perturbations

The aforementioned statistical results do not include the alternative strategies performed by some subjects in the second step, deviating from the rest of the population, see **Figure 5**. For perturbations with SS<sup>0</sup> and SS1/<sup>3</sup> onset, one subject showed alternative stepping responses for a specific perturbation. Subject 4 consistently performed a double right step following the −0.16 inward perturbations with SS<sup>0</sup> onset, first crossing the legs in the first step like all other subjects, but then uncrossing the legs with a second right step (**Figure 5A**). Furthermore, subject 4 made a consistent short compensatory step with the left leg during the second step after the −0.16 inward perturbations with SS1/<sup>3</sup> onset, to further counteract the induced inward COM velocity (**Figure 5B**). This short step was also of shorter duration than that of the other subjects (**Figure 6**).

For perturbations with SS2/<sup>3</sup> onset more varying responses occurred, especially for the 0.16 outward perturbations. Subjects 1, 3, 4, and 10 performed left leg abduction, opposite of the perturbation direction, as well as foot pivoting. Moving the heel

laterally by pivoting on the forefoot, and subsequently moving the forefoot laterally by pivoting on the heel allows changes in the base of support using only a single foot, without actual stepping (**Figure 5E**). Subjects 7 and 9 performed a rear crossstep (**Figure 5D**), stepping behind the leading leg without the body fully toppling over the leading foot in the sagittal plane. Subjects 2, 5, and 8 performed both of these strategies, and subject 6 even performed three different strategies, including a cross-step using the left leg (**Figure 5C**). Furthermore, subject 6 was the only subject to perform such a cross-step for the 0.08 outward perturbations with SS2/<sup>3</sup> onset. Other subjects dealt with this perturbation through a relatively long lasting right single support phase during which the left leg was abducted, sometimes combined with foot pivoting as in **Figure 5E**. The leg abduction is not directly clear from the foot placement locations, but is in line with the long lasting single support duration of the second step following 0.08 outward perturbations with SS2/<sup>3</sup> onset in **Figure 6**.

## Foot Placement Timing Following ML Perturbations

Aside from adjustments in foot placement location, the gait phase durations following the perturbations were affected as well, see **Figure 6A**. For the first step, inward perturbations tend to increase the single support duration, whereas outward perturbations tend to decrease it. These effects diminish with later perturbation onset. In contrast, for the second step inward perturbations tend to decrease the single support duration, whereas outward perturbations tend to increase it. These effects become stronger with later perturbation onset. Specifically, major deviations occur for the −0.16 inward and the 0.08 outward perturbations with SS2/<sup>3</sup> onset. For the −0.16 inward perturbation a fast step with the left leg is used to correct in the second step. For the 0.08 outward perturbation the duration increases because of the earlier mentioned leg abduction strategy that occurs during this single support phase.

All gait phase durations were affected by the ML perturbations [F(4, 144.053) ≥ 5.248, p ≤ 0.001], but there was no main effect of the onset timing on any of the gait phases [F(2, 114.032) ≤ 1.585, p ≥ 0.209]. This is likely because the effects of inward and outward perturbations tend to cancel out in the average duration over all perturbations. A significant interaction effect was found only for the single support phases of the first and second step [F(7, 114.025) ≥ 9.069, p ≤ 0.001], but not for any of the double support phases [F(7, 114.127) ≤ 1.269, p ≥ 0.166]. For the single support phase of the first step, deviations in duration from the unperturbed case diminish with increasing perturbation onset delay. For the ML perturbations with SS2/<sup>3</sup> onset there were

perturbation direction. Ellipses represent the subject-standard deviation. Colors indicate the different perturbation magnitudes. Data is shown dimensionless.

no significant differences from the unperturbed case (p > 421). This is consistent with the findings for the foot locations in the first step following SS2/<sup>3</sup> onset perturbations (**Figure 4C**). It is likely that subjects cannot make major adjustments to their foot placement if the remaining time to the intended (unperturbed) foot contact is short.

## Foot Placement Location Following AP Perturbations

Changes in the AP distance between the COM and the leading foot during both the first and second step after the AP perturbations are generally small, see **Figure 7**. For forward perturbations there is a tendency for the first step to be longer and the second to be shorter, while the opposite is the case for backward perturbations. The leading-foot AP distance from the COM was significantly affected by the AP perturbations [**F**(4, 126) ≥ 9.252, p < 0.001] and the onset timing [F(2, 126) ≥ 7.432, p = 0.001], but not by their interaction [F(8, 126) ≤ 1.600, p ≥ 0.131], for both at HSR (step 1) and HSL (step 2). Unlike the results in Vlutters et al. (2016), here this AP distance was significantly affected by the perturbations. However the differences are generally small, with a typical mean difference of 1 cm between the various onset conditions, as well as between the various AP perturbations and the unperturbed condition. The foot locations in step 2 appear similar regardless of the onset timing. This suggest that the AP perturbations are mostly rejected during the double support phase following the disturbance. No alternative strategies were observed in response to AP perturbations.

## Foot Placement Timing Following AP Perturbations

For AP perturbations of any onset timing, the most prominent changes in gait phase duration seem to occur in the double support phases rather than in the single support phases, see **Figure 6B**. All gait phase durations were affected by the AP perturbations [F(4, 126) ≥ 5.168, p ≤ 0.001]. For the first double support phase, backward perturbation leads to increases in duration, and forward perturbations to decreases. The second double support phase has the tendency to show opposite effects. This is possibly related to the distance between the COM and the trailing foot at heel strike. If this distance is larger, the trailing foot will have to leave the floor earlier during the subsequent double support phase, making it of shorter duration. When considering the perturbation onset timing, only the single support durations were affected

step number in Figure 4. (A) Cross-uncross, making a double step with the right leg. (B) Near-cross step with the right leg, followed by a short left step to prevent the body from falling leftward. (C) Left cross-step. (D) Rear cross-step. (E) Foot pivoting, by first rotating about the toes, then shifting the COP back toward the heel and rotating about the heel. Accompanied by left leg abduction during the swing to prevent toppling over the right stance leg. Several subjects showed both (D,E) throughout the repetitions of the 0.16 perturbations at SS2/3.

[F(2, 126) ≥ 4.827, p ≤ 0.010] but not the double support durations [F(2, 126) ≤ 0.310, p ≥ 0.712]. Most gait phase durations were also affected by the interaction effect [F(8, 126) ≥ 2.112, p ≤ 0.039], with exception of the double support phase of step 2 (HSL-TOR). This indicates that the perturbation responses in this double support phase are independent of the onset

with an alternative strategy for step 2 are shown separately with a cross. Triangles show subject-averages and indicate the perturbation direction. Larger marker size corresponds with later perturbation onset. Error bars represent the subject-standard deviation. Open and filled markers correspond to the single and double support phases, respectively. Colors indicate the different perturbation magnitudes. Data is shown dimensionless.

timing of the perturbation. The effects of the perturbation onset must therefore have been negated in an earlier gait phase.

## Relations With the COM Velocity

In line with previous studies (Hof et al., 2010; Vlutters et al., 2016) we investigated the predictive value of the COM velocity for the

location of the COP after foot contact. Note that the relations presented here span two instances of the gait cycle: heel strike for the velocity, and the subsequent toe-off for the COP. We have previously found this to provide the best correspondence with the XCOM concept (Vlutters et al., 2016), which is also dependent on the COM velocity. The slopes, intercepts, and coefficients of determination of the fits to the data are presented in **Table 1**. If the data modulates with the same slope (ω −1 0 ) as that of the pink XCOM line in **Figures 8**, **9**, then the COP shifts to a constant offset from the XCOM on average over all perturbation magnitudes. If this is the case, the XCOM plus an offset might be used as predictor for the COP location. The subject-average dimensionless XCOM proportionality constant ω −1 <sup>0</sup> was 1.49 <sup>±</sup> 0.05, for comparison with the slopes in **Table 1**.

For ML perturbations, both COP and XCOM modulate in a similar way with the COM velocity within the first recovery step if the perturbation is given early, at SS0, see **Figure 8** and **Table 1**. Perturbations with a later onset diminish the similarities, in line with the reduced foot location modulation (**Figure 4**). The limited base of support adjustment prevents such COP modulation. In contrast, for the second step, similarities are highest if the perturbation is given late, at SS2/3. If the perturbation is given early instead (SS0), most of the balance recovery can occur in the first step, such that the second step can be used to return to the center of the treadmill. This might diminish the similarities with the XCOM. Furthermore, a major contributor to the fit to the data in the second step is the mixture of strategies for the 0.16 outward perturbations at SS2/3, see **Figure 5B**. Even though the standard deviation is large due to the various strategies, its variation is aligned with the XCOM line. Note that for the second step the data is located on the other side of the XCOM line, as the step is made with the other leg. For AP perturbations, both COP and XCOM modulate in a comparable way with the COM velocity within the first step for perturbations with SS<sup>0</sup> or SS2/<sup>3</sup> onset, see **Figure 9**. The similarity is less for perturbations with SS1/<sup>3</sup> onset (see **Table 1**). For the second step, too, the modulation in COP and XCOM is less similar compared to step 1, irrespective of the perturbation onset.

## DISCUSSION

The study aim was to investigate how healthy humans deal with balance perturbations if foot placement adjustments are increasingly constrained by time. Walking subjects received both AP and ML pelvis perturbations at various onset instances throughout the single support phase. For AP perturbations the first step is relatively insensitive to the onset timing. For ML perturbations, adjustments in foot placement location and time in the first recovery step clearly diminished with increasing perturbation onset delay. Most adjustments in foot placement were consistent across subjects, with several exceptions in the second step. Mainly the largest magnitude (0.16) outward TABLE 1 | Slope, intercept, and coefficient of determination of the linear least squares (LLSQ) fit made to the subject-average data at specific instances of the gait cycle after the perturbation.


Distance COP-COM is the independent variable, COM velocity the dependent variable. Underlined values correspond with fits of which the root mean square error is less than 5% of the range of the dependent variable.

perturbations with an onset at two-thirds of the left single support phase (SS2/<sup>3</sup> onset) resulted in inconsistent and varying responses across and within subjects during this second step.

## Balance Responses Are Gait Phase Dependent

Gait-phase-dependent responses arise because foot placement modulation takes time. Hof and colleagues (Hof et al., 2010) reported at least 0.28 s to be required for a "correct" lateral positioning of the foot to occur, though it is unclear how "correct" was defined. Here, the foot placement location relative to the COM did not significantly alter if ML perturbations were given at SS2/3, but did alter if the onset was at SS1/<sup>3</sup> or earlier. As a result, more than 0.15 s (0.73 dimensionless time units) are required for significant adjustments to be made, but less than 0.3 s.

Adjustments might also occur mechanically without active involvement of the subject. For −0.16 backward perturbations at SS2/<sup>3</sup> onset, the single support duration was significantly longer than that for the unperturbed condition. For these perturbations, the single support duration during the first step might increase because the body is pulled backward, which could postpone foot contact resulting from a forward fall due to gravity.

Because of the gait-phase dependency, balance responses should be carefully evaluated with respect to the gait and perturbation characteristics at hand. For example, the 0.16 outward perturbation might be additionally challenging if it is applied shortly before the weight transfers to the leading foot, near the end of the swing phase or start of the double support phase. As loading of the leading foot takes more time in slow walking compared to fast walking (Hebenstreit et al., 2015), it is possible that slow walking is more prone to this specific perturbation. This would contrast with previous indications that slow walking is generally more stable than fast walking, based on the analysis of kinematic variability in unperturbed gait (Dingwell and Marin, 2006; England and Granata, 2007). Because the onset instance within the gait phase can affect how threatening a specific perturbation is, and because the occurrence of gait phases alters with walking speed, one walking speed cannot be declared strictly more stable than another.

## Lack of Foot Placement Adjustments in the First Recovery Step Elicits Other Strategies

It is not always straightforward to group the balance responses into specific strategies. Though balance control is sometimes divided in ankle, hip, and foot placement strategies (Horak and Nashner, 1986; Maki and McIlroy, 1997), it is certainly not limited to these. For example, when adjustments to the foot location are not possible, foot pivoting allows one-legged

base of support corrections without making an actual step. Furthermore, changes in foot placement location and/or time are not necessarily the result of a foot placement strategy. Changes can also be the result of another strategy that occurred before foot contact. The increased single support duration in the second step following the 0.08 magnitude outward perturbations with SS2/<sup>3</sup> onset is an example of this. Subjects abducted their left swing leg possibly as an inertial strategy, or to provide a counter-weight and shift the whole-body COM in the direction of the perturbation. Because subjects spend time abducting their leg, the change in step time is not strictly because it was required for a specific foot placement adjustment. Hence, possible interaction with other strategies must not be disregarded.

It remains unclear why some subjects prefer one recovery strategy over another, and why alternative strategies mainly occur at higher magnitudes. In Hof et al. (2010) no different strategy uses between or within subjects were reported for a given instant of perturbation within the gait cycle. This is possibly due to the lower perturbation magnitudes. In general, subjects might have less experience dealing with large magnitude disturbances

compared to lower ones. Large magnitude perturbations occur less in daily life, such that differences are more likely to arise when the perturbation magnitude is high. The fact that different strategies occurred not only across subjects but also within subjects suggests it is not simply a matter of subject preference. Still, biomechanical constraints could provide some insight on the underlying causes. Whether a cross-step occurs behind or in front of the other leg following outward perturbations at SS2/<sup>3</sup> likely relates to the AP velocity relative to the ML velocity. That is, if the ML disturbance is so large that it would take too long to traverse over the stance foot in the AP direction to make a corrective cross-step, a backward cross-step might be a preferred option instead. Future experiments might point out if this is indeed the case, for example by combining ML and AP perturbations.

## Various Responses Contribute to the Same Relation

The relations between the COM velocity at heel strike and the COP distance from the COM at the subsequent toe-off, previously described in Vlutters et al. (2016), remained intact throughout various conditions. If the perturbations are given early, at SS0, the results are comparable with those in our previous study. However, the relation tend to diminish in the second step, possibly because the return to the center of the treadmill begins to play a role. If the perturbation is given late, at SS2/3, the relation did not occur in the first step for ML perturbations due to the lack of foot placement adjustments, but re-appeared in the second step, even throughout the varying strategies for the 0.16 outward perturbation. Only for the SS1/<sup>3</sup> perturbations the relation appears less in both the first and the second step. For these perturbations there might have been insufficient time to expand the base of support to realize the COP—COM velocity relation in the first step, and it might no longer be required for the second step, because recovery actions occur during single and double support phases before the foot contact as well.

## Effects of Treadmill Walking

Treadmill walking imposes various constraints on the subject that are not present during overground walking. The treadmill requires the subject to continue walking, the treadmill width is limited (∼1 m), and there is little optic flow. A comparison of joint kinematics and ground reaction forces between treadmill and overground walking conditions suggests that differences between the two conditions are within the normal variability of gait at a given speed (Riley et al., 2007). Furthermore, in a study by Zadravec et al. (2017), two similar perturbation devices were used to compare human stepping in response to pelvis perturbations during both treadmill and overground walking conditions. They concluded that the responses in both conditions were similar, such that a treadmill condition is generally preferred given the possibility to continuously measure ground reaction forces. We therefore expect the initial stepping responses to generalize to overground walking. However, the treadmill does impose an implicit "center of the road" on the subject, making them eventually return to the center of the treadmill. We expect this to have little effect on the first recovery step, but subsequent steps might generalize less well due to these effects. In addition, the result might not generalize to different walking velocities. Though responses to perturbations with SS<sup>0</sup> onset were mostly similar for walking speeds of 0.63 and 1.25 m s−<sup>1</sup> (Vlutters et al., 2016), slower speeds might show reduced effects of the onset timing. Due to lower limb excursion at lower speeds, the body configuration at perturbation onset would be more similar across the different onsets.

## CONCLUSIONS

First, we questioned whether foot placement adjustments diminish when there is little time to use such adjustments as a recovery strategy? Foot placement modulation takes time and therefore diminishes in the first recovery step if little adjustment time is available after a perturbation. Foot placement adjustments do occur for the second step, but the degree of modulation is dependent on the perturbation magnitude, direction, onset timing, and preceding actions. If foot placement modulation is not an option in the first step, actions during subsequent gait phases are addressed as an alternative. This can lead to peculiar balance strategies such as foot pivoting. These strategies can be inconsistent both across and within subjects. Though it remains unclear what causes the use of the various strategies, subject preferences are unlikely given that differences also occur within subjects. Second, we questioned whether the COP will continue to modulate with the COM velocity, in line with the XCOM? Despite the varying strategies, previously observed relations between the COM velocity and the COP location relative to the COM persist. This relation is in line with the XCOM concept (capture point), supporting its use in balance controllers for humanoid robotics. The relation might disappear in the first step and re-appear to the second step if the perturbation is given late in the preceding single support phase. These results suggest that foot placement, like any other balance strategy, is a way of achieving some underlying objective, possibly reflected in the COP location. Further probing human balance through perturbations might help reveal these objectives.

## DATA AVAILABILITY

The data supporting the conclusions of this manuscript will be made available by the authors upon request to any qualified researcher.

## AUTHOR CONTRIBUTIONS

MV: contributed to the design of the experimental protocol, to the collection, processing, analysis, and interpretation of the experimental data, and to drafting and revising the manuscript; EV and HvdK: contributed to the design of the experimental protocol, analysis, and interpretation of the experimental data, and to revising the manuscript.

## FUNDING

This work was supported by the BALANCE (Balance Augmentation in Locomotion, through Anticipative, Natural and Cooperative control of Exoskeletons) project, partially funded under grant number 601003 of the Seventh Framework Programme (FP7) of the European Commission (Information and Communication Technologies, ICT-2011.2.1). The funding parties had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

## ACKNOWLEDGMENTS

The authors would like to thank C. M. de Breij for her help in the data collection.

## REFERENCES


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Vlutters, Van Asseldonk and van der Kooij. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Fusion of Bilateral Lower-Limb Neuromechanical Signals Improves Prediction of Locomotor Activities

Blair Hu1,2 \*, Elliott Rouse<sup>3</sup> and Levi Hargrove1,2,4

*<sup>1</sup> Center for Bionic Medicine, Shirley Ryan AbilityLab, Chicago, IL, United States, <sup>2</sup> Department of Biomedical Engineering, Northwestern University, Evanston, IL, United States, <sup>3</sup> Department of Mechanical Engineering, University of Michigan, Ann Arbor, MI, United States, <sup>4</sup> Department of Physical Medicine and Rehabilitation, Northwestern University, Chicago, IL, United States*

Wearable lower-limb assistive devices have the potential to dramatically improve the walking ability of millions of individuals with gait impairments. However, most control systems for these devices do not enable smooth transitions between locomotor activities because they cannot continuously predict the user's intended movements. Intent recognition is an alternative control strategy that uses patterns of signals detected before movement completion to predict future states. This strategy has already enabled amputees to walk and transition seamlessly and intuitively between activities (e.g., level ground, stairs, ramps) using control signals from mechanical sensors embedded in the prosthesis and muscles of their residual limb. Walking requires interlimb coordination because the leading and trailing legs have distinct biomechanical functions. For unilaterally-impaired individuals, these differences tend to be amplified because they develop asymmetric gait patterns; however, state-of-the-art intent recognition approaches have not been systematically applied to bilateral neuromechanical control signals. The purpose of this study was to determine the effect of including contralateral side signals for control in an intent recognition framework. First, we conducted an offline analysis using signals from bilateral lower-limb electromyography (EMG) and joint and limb kinematics recorded from 10 able-bodied subjects as they freely transitioned between level ground, stairs, and ramps without an assistive device. We hypothesized that including information from the contralateral side would reduce classification errors. Compared to ipsilateral sensors only, bilateral sensor fusion significantly reduced error rates; moreover, only one additional sensor from the contralateral side was needed to achieve a significant reduction in error rates. To the best of our knowledge, this is the first study to systematically investigate using simultaneously recorded bilateral lower-limb neuromechanical signals for intent recognition. These results provide a device-agnostic benchmark for intent recognition with bilateral neuromechanical signals and suggest that bilateral sensor fusion can be a simple but effective modular strategy for enhancing the control of lower-limb assistive devices. Finally, we provide preliminary offline results from one above-knee amputee walking with a powered leg prosthesis as a proof-of-concept for the generalizability and benefit of using bilateral sensor fusion to control an assistive device for an impaired population.

Keywords: intent recognition, sensor fusion, assistive devices, bilateral, locomotion

#### Edited by:

*Diego Torricelli, Neural Rehabilitation Group (CSIC), Spain*

#### Reviewed by:

*Juan C. Moreno, Neural Rehabilitation Group (CSIC), Spain Filipe Oliveira Barroso, Neural Rehabilitation Group (CSIC), Spain*

> \*Correspondence: *Blair Hu blairhu@u.northwestern.edu*

#### Specialty section:

*This article was submitted to Bionics and Biomimetics, a section of the journal Frontiers in Robotics and AI*

Received: *25 September 2017* Accepted: *06 June 2018* Published: *26 June 2018*

#### Citation:

*Hu B, Rouse E and Hargrove L (2018) Fusion of Bilateral Lower-Limb Neuromechanical Signals Improves Prediction of Locomotor Activities. Front. Robot. AI 5:78. doi: 10.3389/frobt.2018.00078*

## INTRODUCTION

Worldwide, millions of individuals experience conditions such as stroke, spinal cord injury, and limb loss, which can cause severe and lasting gait impairments that limit functional independence and reduce quality of life (Verghese et al., 2006). Recent advances in mechatronic design and embedded systems have also led to the proliferation of wearable assistive devices that can provide locomotion assistance by actuating lower-limb joints. Such devices include robotic lower-limb prostheses, orthoses, and exoskeletons (e.g., Varol et al., 2010; Quintero et al., 2012; Mooney et al., 2014; Ottobock, 2015, 2016; Panizzolo et al., 2016; Young and Ferris, 2017). Compared to their mechanically passive counterparts, powered devices can be controlled to actively change their mechanical properties between different locomotor activities (e.g., level ground, stairs) and to inject energy into the system (e.g., powered plantarflexion in late stance). However, to maximize the potential benefits of powered assistance and to avoid disrupting the gait cycle, these devices must predict state changes before they occur. Currently, though, most control systems for these assistive devices require the user to explicitly indicate an intended transition with a key fob or an unnatural pre-programmed motion pattern (e.g., bouncing up and down on the Ottobock C-Leg) (Ottobock, 2015). Although the human-machine control interface varies among lower-limb assistive devices, their control systems share similar ideals. To restore normal walking ability, they should accurately infer and execute the user's locomotor intent in a manner that is automatic, seamless, and intuitive to the user.

To more intuitively infer the user's locomotor intent for control, intent recognition has been successfully developed for and primarily applied to powered lower-limb prostheses as an alternative strategy for predicting the appropriate assistance to provide. We define the intent recognition control framework as using information from the human, assistive device, and/or environment detected before movement completion (e.g., windows extracted before heel contact or toe off events) to predict the user's upcoming locomotor activity on a step-by-step basis (Varol et al., 2010). Several studies have already demonstrated the benefits of unilateral sensor fusion for controlling a prosthesis with intent recognition strategies which can operate in real-time on embedded systems. For example, neuromechanical sensor fusion of EMG from the residual limb and prosthesis load information from five above-knee amputees walking with a passive device significantly reduced error rates compared to either sensor set alone (Huang et al., 2011). Fusing above-knee EMG with a diverse set of mechanical sensors embedded in a powered knee-ankle prosthesis comprised of potentiometers and encoders at the knee and ankle, an axial load cell, and 6-degree-of-freedom (DOF) inertial measurement unit (IMU) on the shank also significantly reduced error rates (Young et al., 2014a). In subsequent work, the addition of a 6-DOF load cell and calculated thigh and shank inclination angles to the existing set of mechanical sensor information further reduced error rates; the control system also continued to benefit from fusion with EMG (Spanias et al., 2015). As an alternative to EMG, capacitive sensing has also been used for intent recognition with below-knee amputees (Zheng et al., 2014).

In addition to these unilateral sensor fusion strategies, other powered prosthesis-specific control system modifications (e.g., merging ramp ascent and level walking classes, using modespecific classifiers, and delaying predictions by 90 ms) have further reduced error rates (Hargrove et al., 2015; Simon et al., 2017; Spanias et al., 2018). Error rates during online sessions (i.e., the user interacts dynamically with the control system) using state-of-the-art intent recognition strategies have approached approximately 4% (Spanias et al., 2018); although impressive, they must be further reduced before intent recognition can be used to control a powered assistive device safely and reliably over long periods of time. Despite promising potential for controlling powered prostheses, intent recognition is still not commonly applied to controlling devices for individuals with impaired but intact limbs. Notably, powered orthoses and exoskeletons differ from prostheses because they assist by supplementing instead of substituting the movement of the instrumented limb(s). A few devices have used multimodal sensor fusion for control but they typically rely on pre-defined thresholds to switch between locomotor activities and are mostly limited to identifying transitions between sitting, standing, and level ground walking. For instance, the estimated location of the center of pressure controls switching between sitting, standing, and walking modes of a powered hip-knee orthosis for paraplegic individuals (Quintero et al., 2011, 2012). Ground reaction forces, posture, EMG, and electroencephalography (EEG) have also been used to infer user intent in order to synchronize robotic assistance with paraplegic subjects' movement during gait initiation/termination and level ground walking (Suzuki et al., 2007; Kilicarslan et al., 2013). Bilateral lower-limb neuromechanical signals have also been used to predict sitting, standing, and walking in one patient with multiple sclerosis using intent recognition (Zhang and Huang, 2013). Yet regardless of the devices or control signals used, incorrectly predicting locomotor activities still presents challenges for the long-term clinical viability of intent recognition.

Prediction errors can be categorized as steady-state or transitional, depending on whether the true locomotor activities before and after each gait event are the same (i.e., steady-state) or different (i.e., transitional). Whereas transitional errors can be especially destabilizing and more likely to result in injury (e.g., at the top of the stairs when descending) steady-state errors are harder to anticipate and more frustrating for users. Although intent recognition algorithms can produce seamless transitions during online use, transitional error rates for prosthesis control are still much higher than for steady-state steps (Hargrove et al., 2015; Spanias et al., 2015, 2018; Simon et al., 2017). Walking, especially transitions, requires bilateral coordination of the lower body. For example, the anticipatory lower-limb joint mechanics and EMG signals for able-bodied subjects differed for transitions from level ground walking to stair ascent and descent and for leading and trailing legs (Peng et al., 2016). The mechanical work performed by each leg also differs for both uphill and downhill walking (Franz et al., 2011). Although individuals with unilateral gait impairment typically develop new patterns of interlimb coordination, their non-affected limb generally remains anatomically and biomechanically intact (Chen et al., 2005; Segal et al., 2006; Ingraham et al., 2016). Yet, nearly all assistive devices that are commercially available and/or used in research settings do not incorporate information from both legs and it remains unknown whether contralateral side signals contain rich and robust enough information about the user's intent to justify their inclusion. For example, the powered Vanderbilt knee-ankle prosthesis (Varol et al., 2010) and hydraulic C-Leg knee prosthesis (Ottobock, 2015) and C-Brace knee ankle foot orthosis (Ottobock, 2016) are all controlled using only signals from the affected side; however, we hypothesized that information from the unaffected leg could improve controllability.

Previously, instrumenting the unaffected leg was impractical and considered a major barrier to clinical feasibility. Also, approaches for incorporating information from the contralateral limb have been limited beyond echo control (Grimes et al., 1977; Grimes, 1979; Joshi et al., 2010) and complementary limb motion estimation (Vallery et al., 2011). Echo control required cyclical activities, for which movement had to be initiated by the unaffected side, because the kinematic trajectory of the intact limb was simply "replayed" on the prosthesis side with a half-step delay. Complementary limb motion estimation, which uses residual body motion and interjoint couplings to infer an appropriate reference trajectory for the impaired limb(s), is a more intuitive and cooperative control strategy but has only been implemented for position control. Now, minimally invasive wearable sensors capturing neuromechanical signals are becoming more ubiquitous and can be more easily placed on the contralateral limb to supplement control information from sensors embedded in an assistive device. For instance, soft bio-electronics for physiological recording are already clinically viable (Liu et al., 2016). With these recent developments, minimally invasive bilateral instrumentation of the lower extremities is becoming more feasible. But to our knowledge, only a few studies have investigated bilateral sensor fusion for intent recognition. For example, able-bodied subjects wore bilateral pressure insoles and unilateral IMU sensors on the thigh, shank, and foot (Chen et al., 2014) or walked in a lower-limb exoskeleton with embedded sensors measuring bilateral ground reaction forces and shank/foot orientation to control knee assistance (Long et al., 2016). Although both studies achieved low error rates, their findings may not translate well to seamless, online control because they used prediction periods spanning the entire upcoming step instead of only the instant when the upcoming step begins.

Therefore, we still lack a clear understanding of both how bilateral sensor fusion across different modalities systematically affects intent recognition error rates and whether prosthesisderived intent recognition strategies perform well when generalized to non-prosthesis applications. In this study we present a proof-of-concept for an intent recognition control system using a broad set of bilateral lower-limb neuromechanical signals recorded from wearable sensors instrumented on able-bodied subjects freely walking without an assistive device. Our overall objective was to conduct an offline analysis to systematically compare and benchmark the performance of unilaterally and bilaterally -informed intent recognition control systems and to identify the most critical sensors. We confirmed our hypothesis that sensor fusion across different modalities and across legs would reduce steady-state and transitional error rates. We also report preliminary results from a separate offline analysis on one unilateral above-knee amputee walking with a powered knee-ankle prosthesis to demonstrate the benefit of incorporating kinematic information from the unimpaired leg to improve control of an assistive device with intent recognition. We expect our positive results to further the ongoing development of and broaden the scope of intent recognition strategies for controlling wearable lower-limb assistive devices.

## MATERIALS AND METHODS

## Experimental Protocol

This study was carried out in accordance with the recommendations of the Northwestern University Institutional Review Board with written informed consent from all subjects. Following IRB approval, 10 able-bodied subjects (7 male, 3 female; 23–29 years, 160–193 cm, 54–95 kg) completed the experiment. Before walking, subjects were instrumented bilaterally with wearable sensors to measure lower limb muscle activity and joint and limb kinematics. EMG signals were recorded using bipolar surface electrodes (DE2.1; Delsys, Boston, MA, USA) from the same seven muscles in each leg: tibialis anterior (TA), medial gastrocnemius (MG), soleus (SOL), vastus lateralis (VL), rectus femoris (RF), biceps femoris (BF), and semitendinosus (ST). These muscles were chosen because they are in part responsible for hip and knee flexion/extension and ankle plantarflexion/dorsiflexion, movements that are commonly assisted by wearable devices. They are also relatively easy to target when facing the subject from in front and behind and are similar to muscle sites used by Sylos-Labini et al. (2014). Electrode placement was guided by the Surface ElectroMyoGraphy for the Non-Invasive Assessment of Muscles (SENIAM, seniam.org) standards. We palpated to locate the muscle belly and oriented the electrode along the primary fiber direction (Kendall et al., 2005), and verified placement by having subjects perform maximum voluntary contractions. The muscle sites were prepared by removing excess hair and the skin was cleaned by mildly scrubbing with an alcohol wipe. Sensors were attached to the skin with a double-sided adhesive. Signals were amplified by 1000x, hardware band-pass filtered between 20 and 450 Hz (Bagnoli 16, Delsys), and sampled at 1 kHz.

Joint kinematic signals (sagittal plane only) were recorded using electrogoniometers (SG150; Biometrics Ltd, Newport, UK) placed along the knee and ankle and sampled at 500 Hz. 6- DOF (tri-axial accelerometer and gyroscope) IMU's were placed bilaterally on the subjects' thigh (below RF EMG electrode) and shank (adjacent to TA EMG electrode) and sampled at 500 Hz (MPU-9250; Invensense, San Jose, CA, USA). All signals were simultaneously recorded with a custom 16-bit data acquisition device that permits multi-rate sampling. We chose these wearable sensors because they are analogous to sensors commonly embedded in prostheses, orthoses, and exoskeletons (such as joint position encoders and shank/thigh IMU's) and are more easily integrated with existing device-based sensorization in a hypothetical hybrid system. Other force and interaction torque sensors such as load cells and strain gauges were excluded because they are not as relevant for our device-agnostic approach and are more difficult to integrate if not already embedded in the device.

To facilitate integration with our data acquisition software, all wearable sensors were used in a tethered setup; as a drawback, fully instrumenting one leg took up to an hour. The full instrumentation setup is shown for a representative subject in (**Figure 1** top, middle). Prior to data collection, the goniometers were calibrated while the subject was in the upright standing position. In an experimental session, each subject completed approximately 25 repetitions of a circuit consisting of walking on level ground (LW), ascending/descending a ramp with a 10◦ slope (RA/RD), and ascending/descending a fourstep staircase (SA/SD) step-over-step using a data collection procedure previously described in Young et al. (2014b). These activities were chosen because they encompass the main types of terrain likely encountered in community ambulation. Subjects were instructed to freely transition between activity modes at their self-selected speed while the experimenter labeled the true locomotor intent of the subject using a key fob.

## Signal Processing

Heel contact and toe off gait events for each leg were reliably identified by finding peaks in the low-pass filtered (1st order Butterworth, 6 Hz) sagittal plane angular velocity of the shank segment using a dual-minima method similar to (Jasiewicz et al., 2006; Maqbool et al., 2016). Briefly, the largest peaks in angular velocity were first used to identify mid-swing events. Toe off and heel contact events were identified by searching for peaks before and after each mid-swing event, respectively. Gait initiation and termination strides and trials during which the subject paused, stumbled, or tripped were excluded. For each subject, there were 530 ± 46 heel contact events and 536 ± 45 toe off events for each leg (mean ± standard deviation). EMG signals were first high-pass filtered (6th order Butterworth) at 20 Hz to attenuate motion artifact (De Luca et al., 2010). EMG signals were then notch-filtered (6th order Butterworth, 6 Hz width) at 60, 180, and 300 Hz (based on spectral analysis) to remove ambient interference. Goniometer and IMU signals were low-pass filtered (6th order Butterworth) at 10 and 25 Hz, respectively. Because we did not use foot-mounted IMU's, joint velocities could not be estimated using inertial signals only for the ankle. For consistency, we indirectly estimated joint velocities for both the knee and ankle by taking the centered-difference numerical derivative of the low-pass filtered joint position signals instead and added these velocities to the goniometer channels.

All signals were segmented into 300 ms analysis windows before each identified heel contact or toe off gait event (one window/event). For each window, we extracted features previously used in intent recognition for online control of a powered knee-ankle prosthesis. Features for goniometer and IMU signals included the mean, standard deviation, maximum, minimum, initial, and final values (6 features/channel)

FIGURE 1 | (labeled) were placed on each leg. The coordinate frame of IMU sensors is also shown. (Middle) A representative subject instrumented with all bilateral sensors in a tethered setup. The subject provided written informed consent for the publication of this image. (Bottom) Classifier comparisons for four modality groups and three laterality groups: EMG, goniometer (GONIO), IMU, fused (ALL), ipsilateral (I), contralateral (C), and bilateral (B). The number of sensors and extracted features (in parentheses) are shown next to each classifier configuration.

(Varol et al., 2010). Features for EMG signals included the mean absolute value (MAV), waveform length, number of zero crossings and slope sign changes, and the coefficients of a sixth-order autoregressive model (10 features/channel) (Huang et al., 2005; Hargrove et al., 2008). These heuristic features were chosen because they can be computed efficiently on an embedded system and concisely capture the general shape of mechanical signals and the frequency content of EMG signals. Bilaterally, there were a total of 22 sensors (14 EMG, 4 goniometer, 4 IMU) and 46 channels (14 EMG, 8 goniometer, 24 IMU). The feature dimensionality for all ipsilateral and bilateral signals was 166 and 332, respectively (**Figure 1**, bottom).

## Offline Classifier Evaluation

For each subject, we evaluated leg- and mode/phase-specific classifiers (e.g., right heel contact, left toe off) for several sensor sets to compare their offline error rates (**Figure 2**). We used a mode-specific classification scheme previously developed for powered leg prosthesis control (Young and Hargrove, 2016), which achieves lower error rates by encoding domain knowledge about the allowable transition(s) from each mode. Briefly, 20 total classifiers were trained to encompass all combinations of the four gait events (right/left heel contact or toe off) and five locomotor activities (level ground, ramp ascent/descent, stair ascent/descent). The total number of steps used to train each classifier (combined between legs for all subjects) is reported in **Table 1**. The appropriate classifier for each prediction was selected based on the activity just before the gait event (i.e., incoming activity based on the key fob label). The possible outputs for each classifier (i.e., predicted activities) only included remaining in the current locomotor activity or transitioning to another allowable mode (e.g., in stair ascent mode, remaining in stair ascent or transitioning back to level walking but excluding stair descent and ramp ascent/descent). The error rate was defined as the proportion of incorrectly classified gait events in the testing set and was computed by averaging across legs and gait events for each subject. Errors were also categorized as steadystate or transitional. We performed randomized 10-fold crossvalidation for each subject for 12 different sensor sets (**Figure 1**, bottom) using the steps collected from all circuits completed during the experimental session. The ipsilateral side was defined as the side on which the gait event was identified, which could have been either the leading or trailing leg. Sensors were divided into four modality groups: EMG only (EMG), goniometer only (GONIO), IMU only (IMU), or all combined (ALL). Sensors were also divided into three laterality groups: ipsilateral (I), contralateral (C), or bilateral (B).

FIGURE 2 | Mode-specific classification scheme. Ten classifiers were trained for each leg corresponding to all combinations of incoming activity [level walking (LW), ramp ascent (RA), ramp descent (RD), stair ascent (SA), and stair descent (SD)] and gait event [heel contact (HC) and toe off (TO)]. The possible predictions for each classifier are listed as steady-state (SS) or transitional (T) activities.

TABLE 1 | Number of training examples for each mode-specific classifier.


*The correct labels are listed in the header and instances are aggregated across subjects and between legs.*

Linear discriminant analysis (LDA) has emerged as a convenient a priori choice of classifier for intent recognition for the control of upper- and lower-limb prostheses because it provides a good compromise between classification accuracy and computational efficiency (Hargrove et al., 2007; Scheme and Englehart, 2011). Other commonly used classifiers include support vector machines (SVM) and artificial neural networks (ANN), which can represent more complex, non-linear decision boundaries and may be more appropriate for modeling transitions. The sensor set containing all bilateral signals contains more features than previously used in intent recognition for lower-limb prostheses so we also assessed the effect of classifier type on error rates for ipsilateral and bilateral sensor sets containing all modalities. For all classifiers, the feature data were normalized to have zero mean and unit variance. For LDA, the input dimensionality was further reduced using principal components analysis (PCA) to preserve 95% of the total variance and the prior for each classifier was set to be equiprobable. Hyperparameters for SVM (one-vs-one, linear kernel, C = 10 using the scikit-learn Python package) and ANN (one hidden layer with 10 units, hyperbolic tan activation function, stochastic gradient descent with momentum, adaptive learning rate initialized to 0.1 using the scikit-learn Python package) were chosen based on pilot data.

We performed repeated measures ANOVA for LDA classifiers with error rate as the response variable, modality and laterality (ipsilateral and bilateral only) as fixed within-subject factors, and subject as a random factor. We expected some modalities would benefit more from bilateral information so we included an interaction term. Post-hoc comparisons (paired t-test) with Bonferroni correction were conducted on statistically significant factors. We also used paired t-tests to compare ipsilateral and contralateral sensor sets. We performed repeated measures ANOVA for the combined sensor set with error rate as the response variable, laterality and classifier as fixed within-subject factors, and subject as a random factor. We expected more complex classifiers to perform worse for higher dimensionality data so we included an interaction term. Post-hoc comparisons (paired t-test) with Bonferroni correction were conducted on statistically significant factors.

## Optimal Sensor Selection

To determine the optimal number and type of sensors to instrument on the contralateral leg, we performed sequential forward selection for each subject to choose the sensors which minimized overall error rate with LDA classification (10-fold cross-validation), beginning with all ipsilateral sensors as the baseline and ending with all bilateral sensors. We chose not to identify the bilaterally optimal sensor combination (i.e., beginning with the empty set) because we were primarily interested in the effect of adding contralateral sensors. After each iteration, all features associated with the selected sensor were added to the existing feature set and the selected sensor was removed from the set of remaining sensors. The composition of the sensor set after each iteration was recorded. We performed repeated measures ANOVA with error rate as the response variable, iteration as a fixed within-subject factor, and subject as a random factor. We also performed post-hoc comparisons (paired t-test) between iterations using a Bonferroni correction.

## Preliminary Application to Controlling a Powered Leg Prosthesis

### Experimental Protocol

One individual with a left traumatic above-knee amputation (59 years, 48 years post-amputation, 83.9 kg, Medicare K3 functional level) gave written informed consent to participate in this study. The user was fitted to the Vanderbilt powered knee-ankle prosthesis (Varol et al., 2010) by a certified prosthetist and was experienced walking with the device (minimum of 5 h) (Simon et al., 2014). During data collection, the experimenter manually triggered the powered prosthesis into the correct mode as the user performed tasks including shuffling while standing and walking on level ground and ascending/descending stairs and ramps. To add variability to these movements, the subject was instructed to vary walking speed, include pauses, modify step length, use different angles of approach, and limit upper body support (e.g., only one hand on railing). The user always led with the sound side for stair ascent approaches, with the prosthesis side for stair descent approaches, and either side for ramp ascent and descent approaches. Data from 17 mechanical sensors embedded in the prosthesis were recorded at 500 Hz including knee and ankle joint position and velocity, motor currents, prosthesis acceleration and angular velocity, calculated thigh and shank inclination angles, and axial load. Two additional 6-DOF (tri-axial accelerometer and gyroscope) IMU's were worn by the subject on the non-prosthesis side thigh and shank (**Figure 3**) and sampled at 250 Hz (MPU-9250; Invensense, San Jose, CA, USA). The locomotor activity and state (i.e., phase of the gait cycle) of the prosthesis were also recorded to label the data.

## Signal Processing

The inclination angles of the non-prosthesis side shank and thigh were calculated using a complementary filter and added to the set of recorded signals to match inertial signals from the prosthesis side. All signals were segmented into 300 ms analysis windows around gait events (i.e., heel contact, mid-stance, toe off, mid-swing). Features including the mean, standard deviation, maximum, minimum, initial, and final values were extracted from each window (6 features/channel) (Varol et al., 2010).

FIGURE 3 | Bilateral sensorization for powered leg prosthesis control. Inertial measurement units were placed on the above-knee amputee subject's non-prosthesis side shank and thigh during offline data collection.

With the addition of both non-prosthesis side shank and thigh IMU's, the total number of channels was 31 and the feature dimensionality was 186. Feature data were normalized to have zero mean and unit variance. Consistent with previous studies (e.g., Simon et al., 2017), the dimensionality of the feature data was reduced to 50 using PCA to prevent overfitting.

### Offline Classifier Evaluation

To assess the offline classification accuracy, we implemented a state-of-the-art mode-specific classification scheme which uses delayed transitions (i.e., windows started 210 ms before the gait event and ended 90 ms after the gait event) to control a powered leg prosthesis using intent recognition (same as Simon et al., 2017). This baseline classifier also merged the level ground and ramp ascent data because those activities have similar device assistance settings and previous studies have shown that combining those activities is appropriate. We compared the baseline classifier to a more generic one which neither delays transitions nor merges level walking and ramp ascent activities. We used LDA for all eight mode-specific classifiers (Simon et al., 2017). Errors were categorized as steady-state or transitional and error rate was defined as the proportion of incorrectly classified gait events in the testing set after averaging across all classifiers. We performed leave-one-out cross-validation on all the steps recorded during the experimental session for four different sensor sets: prosthesis sensors only, prosthesis with contralateral shank or thigh IMU, and prosthesis with both contralateral IMU's.

## RESULTS

## Bilateral Neuromechanical Signals and Features

Subjects' bilateral neuromechanical signals were distinguishable based on locomotor activity and mostly consistent between legs and trials throughout the experimental session. Representative data depicting all sensors except SOL, RF, and BF (similar to MG,

circuit consisting of level walking (LW), ramp ascent/descent (RA/RD), and stair ascent/descent (SA/SD). Turquoise traces represent the right leg and purple traces represent the left leg. Circuits were recorded as two discontinuous trials (LW→SA→LA→RD→LW and LW→RA→LW→SD→LW) but are represented as continuous. VL, and ST, respectively) are shown in **Figure 4** (EMG), **Figure 5** (goniometer), and **Figure 6** (IMU) for one representative circuit. Qualitatively, unique patterns of activation in the feature space (**Figure 7**) of certain channels aligned closely with different activities and their associated transitions (only mean value features shown for overlapping 300 ms windows).

## Offline Classifier Evaluation

There was a significant interaction effect between modality and laterality (p = 5.53 × 10−<sup>7</sup> ). Simple main effects analysis showed that overall error rates for classifiers using bilateral sensors were significantly reduced compared to their unilateral counterparts for almost all modality groups and machine learning algorithms (**Figure 8**, **Table 2**). There was a significant effect of classifier (p = 2.22 × 10−16) for the combined sensor sets without interaction between classifier and laterality (p = 0.32). Steadystate and transitional error rates were also significantly reduced when using bilateral sensors for all modality groups and machine learning algorithms with the exception of steady-state errors with SVM and ANN for the fused sensor set (ALL) (**Table 2**). There was generally no significant difference between unilateral (ipsilateral versus contralateral) single modality sensor sets for overall, steady-state, or transitional error rates; however, contralateral sensors had significantly higher transitional error rates with LDA and overall and transitional error rates with ANN for the fused sensor set (**Table 2**). For unilateral sensor sets, the error rate of IMU was lower than EMG or GONIO and decreased with sensor fusion; the lowest average overall error rate (1.43 ± 0.24%) was achieved by the LDA classifier using all bilateral sensors. The random effect of subject was significant (p < 10−10) and the overall error rate using all bilateral sensors

represent the left leg. Circuits were recorded as two discontinuous trials (LW→SA→LA→RD→LW and LW→RA→LW→SD→LW) but are represented as continuous.

with LDA ranged from 0.52 to 2.78%. Interestingly, there was no significant difference between overall error rates for either GONIO(B) and ALL(I) (p = 0.08) or IMU(B) and ALL(I) sensor sets (p = 0.26). For ALL(I), there was no significant difference between classifiers; however for ALL(B), the overall error rate of LDA was significantly lower than ANN (p = 3.02 × 10−<sup>5</sup> ) but not different from SVM (p = 0.11).

## Contralateral Sensor Selection

The optimal collection of additional contralateral sensors after each iteration varied between subjects but on average goniometer and IMU sensors were preferentially selected before EMG sensors (**Figure 9**, top). One contralateral lower leg kinematic sensor (i.e., ankle goniometer or shank IMU) was selected within the first two iterations for every subject; additionally, at least one goniometer sensor (i.e., ankle or knee) was selected within the first two iterations for all but two subjects. Overall (p = 1.27 × 10−<sup>4</sup> ), steady-state (p = 2.35 × 10−<sup>3</sup> ), and transitional (p = 2.54 × 10−<sup>3</sup> ) error rates were significantly reduced from baseline after only one additional contralateral sensor (**Figure 9**, bottom). After four iterations, the error rate plateaued and even increased slightly when approaching all bilateral sensors.

## Preliminary Application to Controlling a Powered Leg Prosthesis

Adding kinematic information from the non-prosthesis side modularly and consistently reduced offline steady-state and transitional error rates for both the baseline delayed/merged classifier and the more generic classifier (dagger symbol) (**Table 3**). Sensor fusion with non-prosthesis side sensors yielded the greatest relative improvement for the heel contact level walking and ramp/stair descent classifiers. For the more generic classifier which used both additional IMU's (Row 7), transitional error rates were slightly better than baseline (Row 1) but overall error rates were still higher because the toe off classifier performed worse. By using prosthesis signals only for the toe off classifier only (Row 8), the performance of the more generic classifier improves and matches the baseline classifier (Row 1).

## DISCUSSION

In this study, we simultaneously recorded bilateral lower-limb neuromechanical (EMG, goniometer, IMU) signals from ablebodied subjects as they spontaneously transitioned between locomotor activities without wearing an assistive device. We applied a previously implemented mode-specific intent recognition framework (Young and Hargrove, 2016) to these signals for offline classification. Our primary objective was to determine the effect of including control information from the contralateral limb on offline classification accuracy. We found that using all bilateral signals achieved significantly lower overall (1.43 ± 0.24%, 32% reduction), steady-state (0.76 ± 0.14%, 39% reduction) and transitional (4.50 ± 0.76%, 24% reduction) error rates compared to all ipsilateral signals, which

were generally not different from contralateral signals. There was a large inter-subject range for the lowest achievable error rates but using bilateral information consistently and significantly reduced offline error rates even for single modality sensor sets. Sequential forward selection identified an optimal subset of contralateral sensors that performed as well as, if not better than all bilateral sensors. When compared to all ipsilateral sensors as the baseline, adding only one contralateral sensor (preferentially goniometer or IMU) significantly reduced overall, steady-state, and transitional error rates. We also demonstrated in a proof-of-concept offline analysis the potential for modularly incorporating kinematic information from the non-prosthesis side to improve an amputee's control of a powered leg prosthesis with intent recognition. Placing two additional IMU sensors on

right). Asterisks denote statistically significant differences.

the non-prosthesis side shank and thigh reduced overall, steadystate, and transitional error rates for the state-of-the-art classifier to 0.20% (62% reduction), 1.50% (60% reduction), and 0.35% (61% reduction), respectively. With bilateral sensor fusion, a more generic classifier (i.e., fewer control restrictions) can match the state-of-the-art.

## Related Work

Our protocol was nearly identical to Young et al. (2014a) and Spanias et al. (2015) but involved able-bodied subjects, no device, and wearable sensors. Using all ipsilateral sensors in this setup, we achieved average transitional error rates (5.94 ± 0.84%) that were roughly half those reported by Spanias et al. (2015) and Young et al. (2014a); steady-state error rates (1.25 ± 0.19%) were comparable. As expected, error rates were higher for transitions, which had fewer training examples and generally more variability. Although we used a more generic modespecific classification scheme compared to Simon et al. (2017) by neither merging level walking and ramp ascent classes nor adding special classifier configurations (e.g., predict transitions between stairs and level ground during mid-swing or midstance), we achieved error rates approaching the state-of-the-art for unilateral-informed intent recognition control of a powered leg prosthesis. Because the addition of only one contralateral sensor (preferentially kinematic) significantly reduced error rates, our results suggest that substantial improvements in controllability may be achievable with minimal instrumentation of the contralateral limb. Unexpectedly, the error rate increased slightly instead of plateauing after the addition of four sensors. Although these consistent increases were not statistically significant, they suggest that the extraneous sensors may not only be redundant but also detrimental for intent recognition because they contribute to model overfitting and/or introduce undesirable sensor drift. From a practical standpoint, the total time required for instrumenting the subject would also be substantially reduced by using an optimized subset of sensors instead of all sensors. Using delayed transitions (i.e., windows starting 210 ms before the gait event and ending 90 ms after the gait event) has significantly reduced intent recognition errors for prosthesis control because delayed windows span both the onset of and continuation of movement (Simon et al., 2017). However, although offline error rates would likely have decreased further we chose not to implement this delay for the sake of clarity and generalizability to non-prosthesis applications for which such delays may not be a desirable tradeoff for accuracy.

We showed that unilateral (ipsilateral or contralateral) sensor sets performed comparably for single modalities, which had previously never been demonstrated for intent recognition control strategies. Thus, our results suggest that using signals from the non-affected leg (which alternates between serving as the leading and the trailing leg) for control and perhaps signals from the affected side for gait segmentation could be suitable for assisting individuals with severe unilateral impairment given proper training on how to perform transitions. Therefore, we believe high performance intent recognition control systems could still be realizable for a range of assistive devices with simple sensorization by integrating with wearable sensors and


TABLE 2 | Complete offline classifier comparison.

*Error rates (mean [SEM]) are shown for all evaluated classifiers. Asterisks under contralateral denote statistically significant differences between contralateral and ipsilateral sensor sets (p-value). Asterisks under bilateral denote statistically significant differences between bilateral and ipsilateral sensor sets (p-value). The best-performing classifier for each type of error is bolded.*

modifying a generic intent recognition control architecture. Our findings also showed that linear discriminant analysis (LDA) can perform as well as, if not better than, more complex algorithms such as support vector machines (SVM) and artificial neural networks (ANN) on a feature set with higher dimensionality (up to 22 sensors, 332 features) than we have previously used in an intent recognition framework for lower-limb prosthesis control (Spanias et al., 2015). To our knowledge, no other studies have used simultaneously recorded bilateral lower-limb neuromechanical signals from able-bodied individuals in an intent recognition control framework; therefore, these results for predicting locomotor activities for unimpaired individuals freely walking without a device also help to establish classification accuracy benchmarks for this framework.

Our offline study with an above-knee amputee subject also lays a foundation for hybrid setups combining device-embedded and wearable sensors by demonstrating the feasibility and success of using bilateral sensor fusion for intent recognition control of an assistive device for a clinical population. Although bilateral sensor fusion achieved modest reductions in offline error rate for the state-of-the-art classifier, the clinical significance of these improvements remains unknown without an online analysis with more subjects walking in settings more representative of home use (longer sessions, more variability in terrain, no clinician to check alignment or supervise, etc.). We also demonstrated that a more generic classifier using bilateral sensor fusion could match the state-of-the-art classifier, thus showing that bilateral sensor fusion presents an alternative set of tradeoffs (instrumenting the non-prosthesis side versus delayed transitions and no unique assistance mode for ramp ascent) for reducing error rates which may be preferable for some subjects. Aside from intent recognition, other applications of neuromechanical sensor fusion have included volitional control of ankle position for below-knee amputees (Au et al., 2005, 2008; Kannape and Herr, 2014) and stumble detection and classification (Zhang et al., 2011a). Beside neuromechanical sensor fusion, vision-based environmental sensing has also shown potential to improve control in an intent recognition framework (Zhang et al., 2011b; Krausz et al., 2015). Although we have focused on bilateral neuromechanical sensor fusion for intent recognition, there are many other potential applications of bilateral sensor fusion including measuring balance, controlling stumble recovery mechanisms, modulating a reference trajectory (e.g., estimating slope of an incline based on the leading leg), and estimating walking speed. There is also

TABLE 3 | Offline error rates using bilateral sensor information to control a powered leg prosthesis with an intent recognition framework.


*Error rates are shown for different mode-specific classifier configurations with varying amounts of kinematic information from the non-prosthesis side. The total number of classifier decisions for each step type is also shown in the bottom row.*

*†Control system neither merges LW and RA classes nor includes 90 ms delay.*

<sup>H</sup>*Control system neither merges LW and RA classes nor includes 90 ms delay; toe off classifier uses prosthesis signals only.*

potential to use other modalities such as soft capacitive stretch sensors and vision in bilateral intent recognition systems.

## LIMITATIONS

The primary limitation of this study is that able-bodied subjects walked without an assistive device. Our setup represents the best case scenario of walking with a completely massless and transparent device; however, we believe this scenario is still valuable for establishing a device-agnostic upper bound for intent recognition. After showing the feasibility of intent recognition for a range spanning no gait impairment (able-bodied subjects without a device) to full impairment (amputee with a prosthesis), we believe that these strategies will reasonably generalize to individuals with an intermediate level of impairment who need some assistance from a powered orthosis or exoskeleton. The neuromechanical signals we collected came from wearable sensors only but capture information that aligns closely with control signals commonly accessible to wearable assistive devices. For example, the C-Leg and C-Brace typically use joint and/or limb kinematic information from sensors embedded in the device for control (Ottobock, 2015, 2016); in addition to these sources of information, the Vanderbilt leg has been controlled using signals from joint torque, load cell, and EMG sensors (Varol et al., 2010). Compared to sensors embedded in a device, wearable sensors are more susceptible to drift because they are not rigidly attached to the user; therefore, error rates are expected to decrease if the corresponding signals came from embedded sensors.

To be consistent with previous studies, our data collection used circuits consisting of level walking, ramps, and stairs only. This protocol is efficient but leads to the known issue of sparsity of transition examples, which were sometimes an order of magnitude less than steady-state steps for a given mode; however, we expect accuracy to increase with more data. Without load cell information, we chose an IMU-based segmentation approach which was not tuned for each subject's self-selected speed. Because the classifier relies on accurate and consistent detection of gait events, we confirmed that the segmentation algorithm's detection of gait events produced results that would have been similar to a thresholding approach based on axial load and joint kinematics as previously implemented on a powered knee-ankle prosthesis (Simon et al., 2014).

Another experimental limitation is that only relatively young, able-bodied subjects without any gait impairments participated in this study. We have included preliminary results from one above-knee amputee walking on a powered leg prosthesis but additional subjects (from different clinical populations and using different devices) are needed to establish the generalizability of a bilateral sensor fusion approach for intent recognition. Although the within-subject variability of control signals from an impaired population is expected to be higher, we still expect bilateral sensor sets to outperform their unilateral counterparts for individuals with unilateral impairment because the contralateral limb usually remains functional and likely a beneficial source of control information. Also, the results of evaluations done offline and with able-bodied subjects have generally been consistent with those from online testing (i.e., the user interacts with the device to control every step and can respond to errors) with amputee populations. Therefore, our promising results suggest that future

## REFERENCES


efforts should be directed toward online testing with unilaterallyimpaired individuals walking with an assistive device with some sensorization of the non-affected side to determine whether these significant improvements in offline accuracy translate to meaningful clinical benefit.

## CONCLUSION

We systematically demonstrated that using bilateral control signals consistently and significantly enhances offline accuracy for an intent recognition control system predicting locomotor activities. In particular, only one additional contralateral sensor was needed to provide significant benefit. Our work also establishes a benchmark for using bilateral lowerlimb neuromechanical signals in a device-agnostic intent recognition control framework. We also provided preliminary evidence from an offline analysis with one above-knee amputee subject walking with a powered leg prosthesis to demonstrate the feasibility and benefit of integrating wearable sensors on the non-affected side to improve control using intent recognition. Together, these promising results also suggest that the intent recognition framework is compatible with a wide variety of sensor configurations and has potential to improve the control of many types of assistive devices.

## AUTHOR CONTRIBUTIONS

BH helped in conceiving the study concept, collecting, analyzing, and interpreting the data, and drafting the manuscript. ER helped in conceiving the study concept and interpreting the data, critically revising the manuscript for important intellectual content, obtaining funding, and supervising the study. LH helped in conceiving the study concept and interpreting the data, critically revising the manuscript for important intellectual content, obtaining funding, and supervising the study. All authors read and approved the final manuscript.

## FUNDING

The research was supported by USSOCOM Contract No. H92222-16-C-0111 and the National Science Foundation NRI: Collaborative Research: Unified Feedback Control and Mechanical Design for Robotic, Prosthetic, and Exoskeleton Locomotion (Award Number 1526534).


of the IEEE Engineering in Medicine and Biology Society (EMBC) (Orlando, FL: IEEE), 5067–5070.


lower limb prostheses. IEEE Trans. Neural Syst. Rehabil. Eng. 24, 217–225. doi: 10.1109/TNSRE.2015.2412461


Annual International Conference of the IEEE Engineering in Medicine and Biology Society EMBC (Osaka: IEEE), 4957–4960.

Zheng, E., Wang, L., Wei, K., and Wang, Q. (2014). A noncontact capacitive sensing system for recognizing locomotion modes of transtibial amputees. IEEE Trans. Biomed. Eng. 61, 2911–2920. doi: 10.1109/TBME.2014.2334316

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewers JM and FOB and handling Editor declared their shared affiliation.

Copyright © 2018 Hu, Rouse and Hargrove. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

## Conservation of Reactive Stabilization Strategies in the Presence of Step Length Asymmetries During Walking

Chang Liu<sup>1</sup> , Lucas De Macedo1,2 and James M. Finley 1,3,4 \*

<sup>1</sup>Department of Biomedical Engineering, University of Southern California, Los Angeles, CA, United States, <sup>2</sup>Departamento de Engenharia Eletrica, Universidade de Brasília, Brasília, Brazil, <sup>3</sup>Division of Biokinesiology and Physical Therapy, University of Southern California, Los Angeles, CA, United States, <sup>4</sup>Neuroscience Graduate Program, University of Southern California, Los Angeles, CA, United States

#### Edited by:

Diego Torricelli, Consejo Superior de Investigaciones Científicas (CSIC), Spain

#### Reviewed by:

Jesse Craig Dean, Medical University of South Carolina, United States Noman Naseer, Air University, Pakistan Khai-Long Ho Hoang, Fakultät für Mathematik und Informatik, Universität Heidelberg, Germany

#### \*Correspondence:

James M. Finley jmfinley@usc.edu

Received: 02 December 2017 Accepted: 30 May 2018 Published: 27 June 2018

#### Citation:

Liu C, De Macedo L and Finley JM (2018) Conservation of Reactive Stabilization Strategies in the Presence of Step Length Asymmetries During Walking. Front. Hum. Neurosci. 12:251. doi: 10.3389/fnhum.2018.00251 The ability to maintain dynamic balance in response to unexpected perturbations during walking is largely mediated by reactive control strategies. Reactive control during perturbed walking can be characterized by multiple metrics such as measures of wholebody angular momentum (WBAM), which capture the rotational dynamics of the body, and through Floquet analysis which captures the orbital stability of a limit cycle attractor. Recent studies have demonstrated that people with spatiotemporal asymmetries during gait have impaired control of whole-body dynamics as evidenced by higher peak-topeak ranges of WBAM over the gait cycle. While this may suggest that spatiotemporal asymmetries could impair stability, no studies have quantified how direct modification of asymmetry influences reactive balance control. Here, we used a biofeedback paradigm that allows participants to systematically adopt different levels of step length asymmetry to test the hypothesis that walking asymmetrically impairs the reactive control of balance. In addition, we tested the hypothesis that perturbations to the non-dominant leg would cause less whole-body rotation due to its hypothesized role in weight support during walking. We characterized reactive control strategies in two ways. We first computed integrated angular momentum to characterize changes in whole-body configuration during multi-step responses to perturbations. We also computed the maximum Floquet multipliers (FMs) across the gait cycle, which represent the rate of convergence back to limit cycle behavior. Our results show that integrated angular momentum during the perturbation step and subsequent recovery steps, as well as the magnitude of maximum FMs over the gait cycle, do not change across levels of asymmetry. However, our results showed both limb-dependent and limb-independent responses to unexpected perturbations. Overall, our findings suggest that there is no causal relationship between step length asymmetry and impaired reactive control of balance in the absence of neuromotor impairments. Our approach could be used in future studies to determine if reducing asymmetries in populations with neuromotor impairments, such people post-stroke or amputees improves dynamic stability.

Keywords: stability, asymmetry, locomotion, reactive control, angular momentum

## INTRODUCTION

One of the primary challenges for human locomotion is to maintain balance when faced with internally generated or externally imposed perturbations. Two balance control strategies are generally used during locomotion: proactive and reactive control of balance (Patla, 1993). While proactive or feedforward control involves the use of predictions of impending perturbations to avoid falling, reactive control of balance involves the use of feedback about the body's state to generate balance correcting responses (Patla, 1993; Tang et al., 1998). One of the primary ways in which the reactive control of balance is studied is by applying perturbations during walking and characterizing the resulting perturbation recovery strategies.

Several metrics have been used to quantify balance during locomotion including measures of variability (Stergiou and Decker, 2011), measures derived from nonlinear dynamics such as the maximum Lyapunov exponent (Dingwell and Cusumano, 2000; Dingwell et al., 2001) and long-range correlations (Hausdorff et al., 1996), and biomechanical measures such as dynamic margins of stability (Hof et al., 2005; Hof, 2008). For a detailed review of metrics used to assess dynamic stability during gait, see Bruijn et al. (2013). While each of these methods is useful for characterizing features of control in the presence of instability, we are particularly interested in measures that directly capture whole-body dynamics. One such measure, whole-body angular momentum (WBAM), can be used to capture the body's response to perturbations and reflects the net contribution of all body segments to the body's rotation about a given axis. WBAM is highly regulated during normal human locomotion (Popovic et al., 2004; Herr and Popovic, 2008) as the peak-to-peak range of WBAM about the body's center of mass is much smaller than the angular momentum of single segments due to momentum cancellation between the limbs (Herr and Popovic, 2008). In a recent study, Martelli et al. (2013) used WBAM to characterize recovery strategies in response to multidirectional perturbations during walking in healthy individuals. They found that perturbations resulted in increased angular momentum and subsequent compensatory reactions.

Another metric used to characterize dynamic stability is the maximum Floquet multiplier (FM) which is commonly used to assess the rate of divergence/convergence from a fixed point, characterized by a kinematic state vector, in response to small perturbations (Hurmuzlu and Basdogan, 1994; Kuo, 1999; Dingwell and Kang, 2007). This measure is based on the fact that human walking is strongly periodic and can be characterized as a limit cycle attractor. Previous studies have established that the maximum FM remains below one during unperturbed walking (Hurmuzlu and Basdogan, 1994; Dingwell and Kang, 2007; Granata and Lockhart, 2008; Bruijn et al., 2009) which indicates that small perturbations always converge toward a limit cycle. The maximum FM increases when walking in destabilizing environments, but still remains below one as people are able to use proactive and reactive control to maintain balance (McAndrew et al., 2012). Both WBAM and the maximum FM capture a different aspect of reactive control during walking and together provide a detailed description of the control of dynamic balance.

The ability to successfully restore balance is vital for populations with gait asymmetries such as people post-stroke (Chen et al., 2005; Balasubramanian et al., 2007; Allen et al., 2011), unilateral amputees (Barth et al., 1992; Underwood et al., 2004; Zmitrewicz et al., 2006) and patients with ACL reconstruction (Winiarski and Czamara, 2012). However, these populations are known to have balance deficits during walking. For example, Lewek et al. (2014) examined the relationship between spatiotemporal gait asymmetry and balance in people post-stroke and showed that step length asymmetries were correlated with scores on the Berg Balance Scale, suggesting that gait asymmetries are associated with fall risk in these individuals. In addition, recent studies have demonstrated that people post-stroke have impaired control of whole-body dynamics as captured by higher peak-topeak ranges of WBAM (Nott et al., 2014; Vistamehr et al., 2016) and reductions in local and orbital stability (Kao et al., 2014). Likewise, unilateral amputees demonstrate a greater range of angular momentum during the half of the gait cycle from foot contact of the residual limb to contact of the intact limb and a smaller range of angular momentum during the second half of the gait cycle due to reduced leg propulsion in the sagittal plane (Silverman and Neptune, 2011). Although these studies have demonstrated an association between asymmetry and measures of stability, it remains to be seen if spatiotemporal asymmetry alone is causally associated with stability in the absence of neuromotor impairments.

In addition to an effect of asymmetry, the reactive control of stability may also be impacted by limb dominance. There is evidence suggesting that the dominant leg generates more propulsion during walking while the non-dominant leg preferentially provides support (Sadeghi et al., 1997). Õunpuu and Winter (1989) found that the normalized EMG amplitude of most plantar flexor muscles was greater in the dominant limb, which may reflect its preferential role in propulsion generation. Also, Martelli et al. (2013) demonstrated that recovery of WBAM about the roll-axis in response to a perturbation depends on the side of the perturbation. Specifically, the reactive responses to perturbations on the non-dominant side, as captured by the principal components of the segmental angular momenta, were more similar to pre-perturbation behavior than responses following perturbations to the dominant side. Thus, the nondominant leg may be better suited for maintaining stability in response to perturbations.

The objectives of this study are to quantify how direct modification of spatiotemporal asymmetry influences the reactive control of balance during walking and to determine whether the reactive control of balance is influenced by limb dominance. We hypothesized that: (1) modifications of step length asymmetry will impair the control of whole-body rotation and increase the maximum FM during unexpected perturbations which, together, would indicate that the reactive control of balance is compromised by asymmetry; and (2) that perturbations of the non-dominant leg would produce less whole-body rotation due to this limb's proposed role in providing stability during locomotion (Sadeghi et al., 1997). Here, we chose to use the maximum FM to quantify orbital stability because it allowed us to capture differences in stability throughout the gait cycle. These hypotheses were tested by using visual feedback to induce changes in step length asymmetry during walking and imposing slip-like perturbations on a dual-belt treadmill. Our findings may inform our understanding of how interventions aimed at improving symmetry in populations with neuromotor impairments may impact balance control.

## MATERIALS AND METHODS

## Participant Characteristics

A total of 19 healthy young individuals (10 M, 24 ± 4 years old) with no musculoskeletal or gait impairments participated in this study. Lower limb dominance was determined by asking participants which leg they would use to kick a ball. This study was carried out in accordance with the recommendations of the Declaration of Helsinki with written informed consent from all subjects. The protocol was approved by the Institutional Review Board of the University of Southern California.

## Experiment Protocol

The purpose of this study was to assess whether changes in step length asymmetry affect the reactive control of balance during walking. Participants completed six separate trials walking on an instrumented, dual-belt treadmill at 1.0 m/s (Bertec, Columbus, OH, USA) and reacted to unexpected accelerations of the treadmill belts throughout the experiment (**Figure 1A**). For the first trial, participants walked on the treadmill for 3-min (Baseline) to obtain their natural level of step length asymmetry. Then, for subsequent trials, visual feedback indicating the desired step lengths was provided to assist participants in actively modifying their asymmetry relative to their natural step length asymmetry. Participants completed a randomized sequence of five 6-min trials with target step length asymmetries (SLA, Eq. 1) of 0%, ±10% and ±15% where 0% represents each participant's baseline SLA.

$$\text{SLA} = 100 \ast \frac{\text{SL}\_{\text{left}} - \text{SL}\_{\text{right}}}{\text{SL}\_{\text{left}} + \text{SL}\_{\text{right}}} \tag{1}$$

Participants viewed the step length targets on a computer monitor attached to the treadmill post (**Figure 1B**). During each trial, participants first practiced walking at a given SLA with visual feedback for 1 min before experiencing any perturbations (**Figure 1C**). A ''success'' message would appear on the screen when the achieved step length was within the three standard deviations of the desired target length. The standard deviation for each target was determined on an individual basis from each participant's Baseline step length variability. Participants were encouraged to maintain the desired SLA and get as many success messages as possible.

Step length was estimated during the experiment as the anterior/posterior distance between the center of pressure on the left and right force plates at foot strike. Foot strike was defined as the point when the vertical ground reaction force became greater than 150 N. For the trials with visual feedback,

10 unexpected perturbations, where the treadmill accelerated to 1.5 m/s, were randomly applied to each side (right or left). Each perturbation was remotely triggered by preprogrammed Python code such that the participants could not anticipate when the perturbation would occur. During pilot testing, we found that there was approximately a 200 ms time delay between when the perturbation signal was sent to the treadmill and when the treadmill began accelerating. Thus, during the first minute of practice for each trial, we calculated each individual's average right and left step times. We then triggered the perturbations 200 ms before the predicted foot strike of the perturbed leg so that the acceleration of treadmill would coincide with foot strike. All perturbations were characterized by a trapezoidal speed profile in which the speed increased at foot strike to 1.5 m/s at an acceleration of 1.6 m/s<sup>2</sup> , was held for 0.3 s, and then decelerated back to 1.0 m/s at 1.6 m/s<sup>2</sup> during swing phase of the perturbed leg (**Figure 2A**). These parameters were selected based on results from a series of pilot tests which demonstrated that these perturbations were sufficient to elicit both changes in step length asymmetry and changes in WBAM. The belt speed was held for 0.3 s to ensure that the belt speed did not decelerate before the toe-off of the perturbed leg. The perturbations randomly occurred within a range of 20–30 steps after the previous perturbation to provide participants with enough time to reestablish their normal walking pattern.

## Data Acquisition

A 10-camera Qualysis motion capture system (Qualysis AB, Gothenburg, Sweden) recorded 3D marker kinematics at 100 Hz and ground reaction forces (**Figure 2B**) at 1000 Hz. A set of 19 mm spherical markers were placed on anatomical landmarks to create a 13-segment, full-body model (Song et al., 2012; Havens et al., 2018). Marker clusters were placed on the upper arms, forearms, thighs, shanks, and the back of heels. At the beginning of each trial, marker positions were calibrated during a 5-s standing trial. All joint markers were removed after the calibration.

## Data Processing

Kinematic and kinetic data were post-processed in Visual3D (C-Motion, Rockville, MD, USA) and Matlab 2017a (Mathworks, Natick, MA, USA) to compute variables of interest. Marker position data and ground reaction forces were low-pass filtered by 4th order Butterworth filters with cutoff frequencies of 6 Hz and 20 Hz respectively. The type of filter and cutoff frequency were selected based on previous literature (Reisman et al., 2009; Winter, 2009; Kurz et al., 2012). The timing of each perturbation relative to foot strike was reexamined in Matlab. If the treadmill belt did not accelerate during the 300 ms window around foot strike (from 150 ms before to 150 ms after foot strike), the perturbation was excluded from analysis. On average, approximately 4 of 20 perturbations were excluded for each trial.

In order to account for differences between the target and achieved SLA, we calculated achieved SLA as follows: first, we calculated the mean SLA of four strides before each perturbation and then distributed them into five equally spaced bins centered at −15%, −10%, 0, 10%, 15% with a bin width equal to 5%. We used this re-categorized SLA as the independent variable in our statistical analyses instead of target SLA.

### Whole-Body Angular Momentum

WBAM was computed to determine how the rotational behavior of the body changed in response to the treadmill perturbations. In order to calculate WBAM, a 13-segment full-body model was first created using Visual 3D (C-Motion, Rockville, MD, USA). The segments of the model included the head, trunk, pelvis, upper arms, forearms, thighs, shanks and feet. Each limb segment's mass was modeled based on anthropometric tables (Dempster, 1955). Segment geometry was modeled based on the description in (Hanavan, 1964). The trunk and pelvis were modeled as elliptical cylinders, the head was modeled as an ellipsoid, and all the other segments were modeled as circular cones. All segments had six degrees of freedom, and no connecting joints (i.e., constraints between segments) were defined. Segmental linear and angular velocity were

computed in Visual 3D using the filtered marker position data. WBAM (L) was then computed as the sum of all segmental angular momenta which were composed of segmental rotation about the body's center of mass and rotation of each segment about its own center of mass (Silverman and Neptune, 2011). Then, L was normalized by the participant's mass (M), treadmill velocity (V), and the participant's height (H) (Eq. 2).

$$\vec{L} = \frac{\sum\_{i} \left[ m\_i \left( \vec{r}\_{\text{CM}-i}^i \times \vec{v}\_{\text{CM}-i}^i \right) + I^i \alpha^i \right]}{\text{MVH}} \tag{2}$$

Here, m is segmental mass, r is the distance from segment to the body COM, I is the segmental moment of inertia, ω is segmental angular velocity, and the index i corresponds to individual limb segments. The coordinate system for analysis of angular momentum was defined as follows: the x-axis was the pitch axis and positive to the right, the y-axis was the roll axis and positive in the anterior direction, and the z-axis was the yaw axis and positive in the vertical direction. WBAM for each stride cycle (from the foot strike on one side to the subsequent foot strike on the same side) was normalized to 100 points. In addition, integrated WBAM (Lint) was computed as the area under the curve of the WBAM trajectory for each step cycle to quantify the degree to which the body rotates about its center of mass across a step cycle.

#### Orbital Stability

We used Floquet analysis (Hurmuzlu and Basdogan, 1994; Hurmuzlu et al., 1996; Kuo, 1999; Dingwell and Kang, 2007; Hobbelen and Wisse, 2007; Kurz et al., 2012; Bruijn et al., 2013) to determine how orbital stability was affected by walking with different levels of SLA. Two participants were excluded from this analysis as there was a break in one of their trials and therefore, we did not have continuous data for the analysis. For this analysis, we used WBAM data as computed in the ''Whole-Body Angular Momentum'' section. First, state vectors (S) at each time point in the normalized gait cycle were constructed from the WBAM signal and its first derivative (Eq. 3). Then, Poincare maps (Eq. 4) were defined at each section of the gait cycle.

$$\mathbf{S} = \begin{bmatrix} L\_{\mathbf{x}} & L\_{\mathbf{y}} & L\_{\mathbf{z}} & \hat{L}\_{\mathbf{x}} & \hat{L}\_{\mathbf{y}} & \hat{L}\_{\mathbf{z}} \end{bmatrix}^{T} \tag{3}$$

$$\mathcal{S}\_{k+1} = F(\mathcal{S}\_k) \tag{4}$$

Here, k is the stride number and S<sup>k</sup> are the state vectors of the system.

For each trial, we defined the fixed points (S ∗ ) (Eq. 5) at each Poincare section by averaging all sets of angular momentum trajectory during the four strides before each perturbation occurred (**Figure 3**). Strideto-stride fluctuations about the fixed point allowed us to examine the persistence of deviations from the mean trajectory.

$$\mathcal{S}^\* = F\left(\mathcal{S}^\*\right) \tag{5}$$

Orbital stability at each Poincare section was assessed by linearly approximating the effects of perturbations that caused deviations from the fixed point (Eq. 6). For all trials, we used 250 strides to compute FMs based on previous literature which established that at least 150 strides were necessary to precisely measure FM (Bruijn et al., 2009).

$$J\left[\mathcal{S}\_{k+1} - \mathcal{S}^\*\right] \approx J\left(\mathcal{S}^\*\right)\left[\mathcal{S}\_k - \mathcal{S}^\*\right] \tag{6}$$

Here, J is the Jacobian matrix estimated using the pseudo inverse at each Poincare section (Kurz et al., 2012). S<sup>k</sup> − S ∗ represents deviations from the fixed point. FM were calculated as the eigenvalues of Jacobian matrix (J(S ∗ )), and we selected the maximum value of the FM (FMmax) to assess orbital stability (Dingwell and Kang, 2007). If the magnitude of FMmax < 1, the system is orbitally stable, otherwise, the system is unstable (Hurmuzlu and Basdogan, 1994; Dingwell and Kang, 2007; Bruijn et al., 2013). We computed FMmax for each Poincare section (each % gait cycle) to determine how stability changes over a stride cycle. We also determined FMmax across the entire gait cycle for further statistical analysis as this value represents the most

unstable point during the gait cycle (Dingwell and Kang, 2007).

## Statistical Analysis

All statistical analyses were performed in Matlab R2017a (Mathworks, Natick, MA, USA). Repeated measures analysis of variance (RM-ANOVA) was used to determine if values of integrated angular momentum about each axis for the steps after the perturbation differed from values during baseline steps. Post hoc comparisons used the Tukey-Kramer correction for multiple comparisons.

Linear mixed-effect models were fit to examine the relationship between independent variables achieved SLA (Asym) and side of perturbation (Side) and dependent variables Lint about each axis to determine how the effect of perturbations varied with asymmetry and between limbs. This model included main effects for Asym and Side as well as an interaction between Asym and Side to determine whether the effect of asymmetry depended on the side of the perturbation. The linear mixedeffect models were fit for four consecutive steps (Baseline, Perturbation, Recovery 1 and Recovery 2) for each axis. The integrated angular momentum for the trial with an SLA of zero was selected as the reference level. We used a mixed effect model instead of a RM-ANOVA for this analysis because the number of observations at each level of achieved asymmetry was unequal.

Similarly, a linear mixed effect model was fit to represent the relationship between target asymmetry (independent variable) and the FMmax (dependent variable) in order to see if the orbital stability was associated with target asymmetry. For both sets of analyses, models including random intercepts and/or slopes were compared against a model with only fixed effects and the most parsimonious model was chosen based on the results of a likelihood ratio test.

## RESULTS

Participants were able to update and maintain their SLA for the full duration of each trial (∼6 min) using visual feedback and recover from deviations in SLA resulting from the perturbations (**Figure 4A**). If the perturbation occurred on the right leg, the left leg would step further forward to recover from the perturbation and SLA would increase based on Equation 1. On the other hand, if the perturbation was occurred on the left leg, SLA for the perturbation stride became more negative. The achieved SLA (**Figure 4B**) was calculated relative to baseline asymmetry of 1.7 ± 3%. There was considerable variability in the achieved asymmetry across participants, especially when the target SLA was large. The average residual between achieved SLA and target SLA (|SLAtarget − SLAachieved|) across all participants was 4.7 ± 2.9%, 2.9 ± 1.7%, 2.4 ± 1.6%, 4.3 ± 2.8%, 7.7 ± 3.5% for −15%, −10%, 0, 10%, 15% target SLA respectively. Our analysis used participants' achieved asymmetry rather than target asymmetry to better reflect their actual performance. The total number of perturbations in each step length asymmetry bin were as follows: 110 perturbations for −15%, 314 perturbations for −10%, 265 perturbations for −5%, 294 perturbations for 0%, 285 perturbations for 5%, 191 perturbations for 10% and 69 perturbations for 15% SLA.

## Modulation of Whole-Body Angular Momentum in Response to Treadmill Perturbations

Measures of WBAM varied systematically across trials. We measured the WBAM about three axes to better understand how participants react to the perturbations. The rapid acceleration of the belts caused consistent, immediate effects on WBAM and triggered multi-step balance recovery responses. The immediate effect was most obvious along the direction of perturbation (pitch axis, **Figure 2C**). During the perturbation step, angular momentum became more negative as the body rotated forward (−pitch). In order to compensate for the perturbation, participants increased the length of the subsequent step to generate positive angular momentum and initiate backward rotation (+pitch). Deviations in body rotation about the roll and yaw axes relative to unperturbed walking were less prominent (**Figures 2D,E**).

FIGURE 4 | (A) Raw step length asymmetry data for one representative participant. Each data point represents the step length asymmetry. The target asymmetries for this example followed the order of 10%, −10%, 0, 15%, −15%. Each target asymmetry is represented by a different color. BSL: baseline step; PTB: perturbation step; REC: recovery step. (B) Achieved step length asymmetry vs. target step length asymmetry for all participants (N = 19). Achieved step length asymmetry is calculated as the average of all pre-perturbation strides and tends to undershoot the target at 15% and −15%. The green dots represent individual data. Horizontal bars indicate the median across all participants.

To quantify the effects of the perturbations on whole-body configuration, we computed the integrated angular momentum across the step cycle (**Figure 5**). When walking symmetrically, the integrated angular momentum was relatively small during baseline walking and showed little step-to-step variability about the pitch axis. For the roll axis, positive and negative values of Lint correspond to transitions from the right to left leg and from the left to right leg, respectively. For the yaw axis, positive and negative values of Lint correspond to transitions from the left to the right leg and from the right to left leg, respectively. During the perturbation step, there was a significant increase in integrated angular momentum about the pitch axis which reflected the increase in the body's forward rotation. A repeated measures ANOVA was used to determine when the participants recovered from the perturbation when walking symmetrically. We found a main effect of step number on integrated angular momentum for the pitch axis (RM-ANOVA, F = 185.5, p < 0.001), and a significant interaction between step number and perturbation side for roll axis (F = 58.7, p < 0.001), and yaw axis (F = 434.76, p < 0.001). For the pitch axis, post hoc analysis revealed that Lint differed from baseline during the perturbation (PTB) step (p < 0.001 both sides), recovery (R) steps R1 (p < 0.001 both sides), R4 (p = 0.006 Dominant side, p = 0.002 Non-dominant side) and R6 (p = 0.013 Dominant side, p = 0.001 Non-dominant side). About the roll axis, significant differences in Lint were found during the PTB step (p = 0.001) and R3 (p = 0.001), but only on the dominant side. Lastly, about the yaw axis, a significant difference was found at perturbation step (p < 0.001) for both sides.

## Effects of Asymmetry on the Reactive Control of Whole-Body Angular Momentum

Next, we asked whether walking with asymmetric step lengths would have negative effects on participant's reactive control of balance (**Figure 6**). We fit a linear mixed effect model relating asymmetry to the integrated angular momentum at each step (Final Baseline step (B2), PTB, R1 and R2) and selected the simplest model based on a likelihood ratio test. An SLA of zero was selected as the reference level. The model

indicated that a random intercept was necessary to account for individual differences between participants (p < 0.001). There were no significant main effects found for Asymmetry, Side (pitch axis) or the interaction between Asymmetry and Side during baseline (B2) steps (**Table 1**). Similarly, we examined whether asymmetry influenced measures of integrated angular momentum at the PTB step and found no significant main effects for Asymmetry, Side or the interaction between Asymmetry and Side during any of these steps (**Table 1**). Lastly, we examined whether asymmetry influenced integrated angular momentum during the first or second recovery steps (R1 and R2). There were no significant main effects found for Asymmetry, Side or the interaction between Asymmetry and Side during first or second recovery steps (**Table 1**). The significance found for Side in roll and yaw axis was due to differences in the direction of body rotation at each step. Overall, these results indicate that imposed asymmetry does not have a systematic effect on the reactive control of balance as assessed by measures of WBAM.

## Orbital Stability

To further investigate how asymmetry impacts dynamic stability, we performed Floquet analysis to determine if asymmetry influenced measures of orbital stability during perturbed walking. Our results show that the FMMax from all five trials was less than one indicating that participants remained orbitally stable in spite of the perturbations that occurred while walking (**Figure 7**). The range of the FM computed across the stride cycle at 0% SLA was 0.41 ± 0.09 across all participants, which is similar, but slightly smaller than that reported by Dingwell and Kang (2007) (∼0.5) when participants walked on the treadmill in the absence of perturbations. Similar to our results for integrated angular momentum, there was no association between the SLA and measures of orbital stability (F(4,80) = 0.86, p = 0.5). The FMMax across all asymmetries were 0.61 ± 0.09, 0.63 ± 0.13, 0.57 ± 0.11, 0.62 ± 0.13 and 0.59 ± 0.09 for target asymmetries of −15%, −10%, 0, 10% and 15%, respectively.

## DISCUSSION

## Conservation of Reactive Response

This study asked the question of how step length asymmetry affects the reactive control of balance during walking. Previous studies have demonstrated that people with gait asymmetries have impairments in dynamic balance leading to the possibility that asymmetry itself is sub-optimal for balance control. Here, we hypothesized that asymmetry would impair the reactive control of balance, and we tested this hypothesis by imposing different levels of step length asymmetry and characterizing participants' response to perturbations. Although we consistently elicited reactive responses to regain balance, we rejected our primary hypothesis that asymmetry impairs the reactive control of balance as no significant difference in WBAM was found Step type Axis Factor DF1 DF2 F value p value B2 Pitch Asymmetry 6 210 0.10 0.99 Side 1 210 2.84 0.09 Asymmetry:Side 6 210 0.18 0.98 Roll Asymmetry 6 210 0.31 0.93 Side 1 210 76.10 <0.001 Asymmetry:Side 6 210 0.24 0.96 Yaw Asymmetry 6 210 0.72 0.63 Side 1 210 397.00 <0.001 Asymmetry:Side 6 210 0.96 0.45 PTB Pitch Asymmetry 6 211 0.25 0.96 Side 1 211 0.30 0.59 Asymmetry:Side 6 211 0.59 0.77 Roll Asymmetry 6 211 0.39 0.89 Side 1 211 85.00 <0.001 Asymmetry:Side 6 211 0.41 0.87 Yaw Asymmetry 6 211 0.38 0.89 Side 1 211 577.00 <0.001 Asymmetry:Side 6 211 1.33 0.25 R1 Pitch Asymmetry 6 211 0.51 0.80 Side 1 211 0.31 0.60 Asymmetry:Side 6 211 0.76 0.60 Roll Asymmetry 6 211 0.36 0.90 Side 1 211 2.85 0.09 Asymmetry:Side 6 211 0.39 0.89 Yaw Asymmetry 6 211 1.00 0.42 Side 1 211 491.00 <0.001 Asymmetry:Side 6 211 1.57 0.16 R2 Pitch Asymmetry 6 211 0.90 0.5 Side 1 211 2.70 0.1 Asymmetry:Side 6 211 0.21 0.97 Roll Asymmetry 6 211 0.65 0.69 Side 1 211 6.39 0.01 Asymmetry:Side 6 211 0.00 0.98 Yaw Asymmetry 6 211 0.47 0.83 Side 1 211 386.00 <0.001 Asymmetry:Side 6 211 0.92 0.48

TABLE 1 | Statistical results from the ANOVA examining the effects of asymmetry and perturbation side on integrated whole-body angular momentum (WBAM) for each step type.

B2, Baseline step; PTB, Perturbation Step; R1, First Recovery Step; R2, Second Recovery Step.

across levels of asymmetry. In addition, Floquet analysis revealed that orbital stability was well maintained and did not vary systematically with different levels of asymmetry. These results indicate that reactive control of stability may be well controlled by healthy people even when they change their preferred walking pattern to walk asymmetrically.

A potential explanation for the discrepancy between our hypothesis and the observed results is that participants may have chosen a more conservative strategy due to the novel study demands. In other words, participants may have taken more cautious, wider steps to increase their base of support (Woollacott and Tang, 1997) when asked to walk asymmetrically and subsequently improved the proactive control of stability during the task. However, we found no difference in step width between levels of target asymmetry. Thus, there does not seem to be strong evidence that participants' recovery strategies were biased by use of a more conservative pattern of movement.

Another possible reason why we did not observe an effect of asymmetry on the reactive control of stability is that reactive responses to unexpected perturbations may be mediated by neural pathways that generate stereotypical reactive responses that remain invariant across tasks. Previous work by Aprigliano et al. (2017) used principal component analysis (PCA) to analyze coordination between the shank, foot and thigh in response to slip-like perturbations and found that there was no difference in coordination between fall-prone elderly people and healthy young adults. Similarly, Martelli et al. (2013) used PCA of segmental angular momentum to show that the intersegmental coordination patterns observed during compensatory steps are highly correlated with the patterns observed during unperturbed walking. This conservation of whole-body momentum across asymmetries suggests that reactive responses may result from pre-programmed, stereotypical actions that are sufficient to restore the stability (Martelli et al., 2013).

## Effects of Limb-Dominance on Reactive Control of Balance

We also hypothesized that perturbations of the non-dominant leg would produce less whole-body rotation due to the non-dominant limb's role in maintaining balance. Previous work has shown that the non-dominant leg may preferentially be used to support body weight while the dominant leg may generate more propulsion (Sadeghi et al., 1997). Consistent with this idea, WBAM about the roll axis did not differ across strides for the non-dominant leg while, in contrast, we observed significant differences in Lint about the roll axis between unperturbed, perturbed, and recovery strides for the dominant leg. This suggests that the non-dominant leg may be better at maintaining medial-lateral balance. However, we also found that there was no significant effect of the side of the perturbation on our measures of integrated angular momentum. This presence of both limb-specific and limb-independent recovery responses requires further investigation to establish the effect of limb dominance on balance recovery.

## Orbital Stability During Asymmetric Walking

In general, populations with a high incidence of falls are shown to have increased orbital instability relative to unimpaired controls as characterized by a larger maximum FM (Hurmuzlu et al., 1996; Granata and Lockhart, 2008; Kurz et al., 2012). In our study, we were interested in determining whether walking with spatiotemporal asymmetry would modulate orbital stability. As the results demonstrated, all participants in our study had orbitally stable walking patterns, regardless of the level of SLA. Also, no difference in FMMax was found during asymmetrical walking compared with symmetric walking. This is in contrast to previous work which shown that voluntarily changing step length reduces orbital stability of human walking (McAndrew Young and Dingwell, 2012). One possible explanation for the discrepancy between the current study and previous work is that our study provided visual feedback for regulating SLA, which may decrease the step length variability compared to (McAndrew Young and Dingwell, 2012). In addition, we chose to maintain a consistent stride length while varying asymmetry whereas the McAndrew Young and Dingwell's (2012) study involved significant increases in stride length. As a result, Floquet analysis may not be sensitive enough to detect the destabilizing effects of SLA in the absence of changes in stride length. Our findings also contrast previous studies which showed that walking on a pseudo-randomly oscillating treadmill reduced orbital stability (McAndrew et al., 2010; Beurskens et al., 2014). A potential reason for this difference is that our study perturbed the participants with discrete mechanical perturbations at foot contact whereas previous work used continuous perturbations throughout the gait cycle. It is possible that the effect of these discrete perturbations dissipates quickly for healthy participants resulting in negligible changes in orbital stability due to imposed SLA.

Although this study showed that the whole-body reactive response was not affected by the presence of step length asymmetry in healthy participants, it is possible that this result was influenced by the fact that participants were instructed to walk on the treadmill with a fixed speed, which might alter the strategies participants use to generate the desired asymmetries. While Nagano et al. (2011) showed that there were no differences in step lengths when young adults walked on a treadmill or over ground at a given speed, temporal parameters such as double stance time and swing time did differ. In addition, people produce reduced dorsiflexor moments, reduced knee extensor moments, and greater hip extensor moments in the sagittal plane during treadmill walking (Lee and Hidler, 2008), which could also affect the strategies used to modify symmetry on the treadmill. As a result, it would be interesting for future studies to compare the effects of asymmetry on reactive control balance during over ground vs. treadmill walking.

Aside from reactive responses, proactive control also plays a part in maintaining balance on the treadmill in the presence of slip perturbations. Although participants voluntarily changed their SLA to match the visual feedback, modification of SLA did not impair their whole-body balance during unperturbed steps (**Figure 6**). This might reflect the fact that healthy individuals used novel proactive strategies to maintain a consistent range of WBAM in the presence of step length asymmetry. Another possibility is that after the initial exposure to the treadmill perturbations, participants may have adopted strategies to improve stability and to prepare themselves for the perturbations. Previous work has shown that the central nervous system can adjust its control strategy based on the prior experience to produce a more cautious gait and reduce the risk of balance loss by altering muscle activation and the resulting interaction between the foot and the surface (Heiden et al., 2006). In addition, previous studies have showed that people were able to reduce backward balance loss with exposure to multiple slip perturbations using proactive adjustments during sit-to-stance task and over ground walking (Pai et al., 2003; Bhatt et al., 2006). Analysis of interlimb coordination during baseline trials could be useful to reveal how participants were able to adopt an invariant control of WBAM despite the presence of marked step length asymmetries.

Lastly, although we found that symmetric walking did not necessarily bring benefits for reactive control of balance for healthy subjects, this does not necessarily mean it holds true for fall-prone populations with sensory or motor impairments. These impairments may significantly affect one's ability to recover from unexpected perturbation. Fall-prone populations such as people post-stroke may suffer from sensorimotor deficits,

## REFERENCES

Allen, J. L., Kautz, S. A., and Neptune, R. R. (2011). Step length asymmetry is representative of compensatory mechanisms used in post-stroke hemiparetic walking. Gait Posture 33, 538–543. doi: 10.1016/j.gaitpost.2011.01.004

which prevent them from appropriately sensing perturbations and planning and executing effective responses to regain balance. Although a number of recent studies have shown that it is possible to improve spatiotemporal symmetry in people post-stroke (Reisman et al., 2009; Awad et al., 2016), it remains to be seen if reductions in symmetry improve dynamic balance. The approaches used in this study may help separate the effects of asymmetry on balance from the effects of neuromotor deficits and lead to better informed locomotor training for people poststroke.

## Limitations

This study had a few limitations. First, there was inconsistency in performance such that the achieved step length tended to undershoot the target at the larger SLA. This is likely because larger asymmetries are energetically costly and may also reflect biomechanical constraints that prevent substantial extension of the hip beyond that observed during normal walking (Sánchez et al., 2017). Another factor that could have affected our analysis of reactive control is that participants were protected by a harness, which may have restricted forward trunk rotation. However, since the harness was slack during the full experiment, we think this is unlikely. Lastly, calculation of FMs requires a linear approximation to compute the effects of perturbations from stride to stride, but the large perturbations in our study may introduce some non-linearity. We think this effect is likely to be negligible as the FMs we computed were consistent with previous work that assessed stability during walking in destabilizing environments (McAndrew et al., 2012).

## AUTHOR CONTRIBUTIONS

CL designed the experiment, collected data, analyzed data and wrote the manuscript. LM designed the experiment, collected preliminary data and helped develop the procedure for data processing and analysis. JF conceived of the experiment, advised in data analyses and edited the manuscript.

## FUNDING

This work was supported by the University of Southern California.

## ACKNOWLEDGMENTS

We thank Natalia Sanchez, Ph.D. for her insights during the design of this experiment, Aram Kim for her assistance with the statistical analysis and Sungwoo Park for his help with Matlab coding.

Aprigliano, F., Martelli, D., Tropea, P., Pasquini, G., Micera, S., and Monaco, V. (2017). Ageing does not affect the intralimb coordination elicited by slip-like perturbation of different intensities. J. Neurophysiol. 118, 1739–1748. doi: 10.1152/jn.00844. 2016


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Liu, De Macedo and Finley. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Effectively Quantifying the Performance of Lower-Limb Exoskeletons Over a Range of Walking Conditions

#### *Daniel F. N. Gordon\*, Graham Henderson and Sethu Vijayakumar*

*Institute of Perception, Action, and Behaviour, School of Informatics, University of Edinburgh, Edinburgh, United Kingdom*

Exoskeletons and other wearable robotic devices have a wide range of potential applications, including assisting patients with walking pathologies, acting as tools for rehabilitation, and enhancing the capabilities of healthy humans. However, applying these devices effectively in a real-world setting can be challenging, as the optimal design features and control commands for an exoskeleton are highly dependent on the current user, task and environment. Consequently, robust metrics and methods for quantifying exoskeleton performance are required. This work presents an analysis of walking data collected for healthy subjects walking with an active pelvis exoskeleton over three assistance scenarios and five walking contexts. Spatial and temporal, kinematic, kinetic and other novel dynamic gait metrics were compared to identify which metrics exhibit desirable invariance properties, and so are good candidates for use as a stability metric over varying walking conditions. Additionally, using a model-based approach, the average metabolic power consumption was calculated for a subset of muscles crossing the hip, knee and ankle joints, and used to analyse how the energy-reducing properties of an exoskeleton are affected by changes in walking context. The results demonstrated that medio-lateral centre of pressure displacement and medio-lateral margin of stability exhibit strong invariance to changes in walking conditions. This suggests that these dynamic gait metrics are optimised in human gait and are potentially suitable metrics for optimising in an exoskeleton control paradigm. The effectiveness of the exoskeleton at reducing human energy expenditure was observed to increase when walking on an incline, where muscles aiding in hip flexion were assisted, but decrease when walking at a slow speed. These results underline the need for adaptive control algorithms for exoskeletons if they are to be used in varied environments.

Keywords: *exoskeletons*, gait metrics, stability, metabolic energy, control, musculoskeletal modelling

## 1. Introduction

Increasingly, exoskeletons are being used to great effect for the rehabilitation of people with lowerlimb pathologies (Dollar and Herr, 2008). Additionally, exoskeletons are being developed as assistive tools to reduce the metabolic cost of walking, with some recent advances in state-of-the-art soft exosuits (Panizzolo et al., 2016) demonstrating energy savings of more than 20% (Quinlivan et al.,

#### *Edited by:*

*C. David Remy, University of Michigan, United States*

#### *Reviewed by:*

*Priyanshu Agarwal, Rice University, United States Erwei Yin, China Astronaut Research and Training Center, China*

> *\*Correspondence: Daniel F. N. Gordon daniel.gordon@ed.ac.uk*

#### *Specialty section:*

*This article was submitted to Biomedical Robotics, a section of the journal Frontiers in Robotics and AI*

*Received: 19 February 2018 Accepted: 07 May 2018 Published: 27 June 2018*

#### *Citation:*

*Gordon DFN, Henderson G and Vijayakumar S (2018) Effectively Quantifying the Performance of Lower-Limb Exoskeletons Over a Range of Walking Conditions. Front. Robot. AI 5:61. doi: 10.3389/frobt.2018.00061*

**114**

2017). However, exoskeletons used for these purposes are largely restricted to supervised clinical or research settings, where time and care is taken to ensure that the behaviour of the exoskeleton and the nature of the rehabilitation or assistive regime is wellsuited to the subject in question (Wolff et al., 2014). The use of exoskeletons in a real-world setting, e.g. to assist the elderly in everyday life, is made difficult by the number of variables to consider as a human walks in an uncontrolled environment for example, walking speed, or whether the subject is walking on an incline. Each of these variables can affect the gait pattern of an individual, and therefore the optimal torques to be applied by an assistive device. If exoskeletons are to become widely used devices outside of a clinical setting it is important that a suitable control paradigm is developed that, either implicitly or explicitly, applies assistance that accounts for these variables.

Current control paradigms frequently use normalised kinematic trajectories (Riener et al., 2010), muscle amplification (Ferris and Lewis, 2009), or finite state controllers (Blaya and Herr, 2004). The respective issues with these paradigms are that the kinematic trajectory might not be appropriate for the user's task or their environment, the muscle firing patterns may be abnormal, and there are a large number of parameters to tune.

It is known that the human neuromuscular system optimises stability (Kuo and Donelan, 2010). By studying the effect of different walking contexts and constant perturbations (applied via an exoskeleton) on healthy walking, it is posited that there will be an underlying invariant metric that reflects the optimisation of the stability of human gait. Once identified, this metric can then be optimised as part of an exoskeleton control paradigm which provides assistance while maintaining balance, implicitly accounting for the effects of changing walking context and varying exoskeleton assistance. Previous work has been carried out to determine what effect walking speed (Winter, 1984; Stansfield et al., 2001; Orendurff et al., 2004), the environment (Lay et al., 2006; Franz and Kram, 2012), and exoskeleton forces have (Lewis and Ferris, 2011; Lenzi et al., 2012; Martelli et al., 2014) on a user's gait but these are constrained by using limited metrics and, for the work done on exoskeletons, limited walking contexts.

The human neuromuscular system also optimises energy efficiency (Kuo and Donelan, 2010). In a similar analysis to what is outlined above, the effect of different walking contexts and exoskeleton forces on healthy walking can be measured in terms of the metabolic energy consumed by the muscles of the subject. This relationship could be optimised as part of a model-based exoskeleton control paradigm, alongside a stability metric, where the aim is to reduce total human energy expenditure or, alternatively, target specific groups of muscles for rehabilitation or assistance. Once known, this relationship can be used to inform how exoskeleton controllers are implemented for use in real-world settings where steady, flat walking is not guaranteed.

In this study, a neuromuscular human and exoskeleton model is presented. Experimental data was collected using a unique setup, combining kinematic, kinetic, and exoskeleton angular and torque data. Using this data, stability metrics and metabolic energy consumption were compared between three walking scenarios: walking without an exoskeleton, walking with an exoskeleton in transparent mode, and walking with an exoskeleton in assistive mode. For each of these scenarios five different walking contexts were investigated: walking at baseline speed, walking up an incline, walking down an incline, fast walking, and slow walking. To carry out the analysis a range of spatial and temporal, kinematic, kinetic, and dynamic gait metrics (such as centre of mass displacement) were selected.. The selected metrics were compared to identify those which demonstrated the most invariance and therefore would be suitable for optimising in an exoskeleton control paradigm. In addition, metabolic energy consumption was calculated and is reported for a subset of muscles crossing the hip, knee and ankle joints, and the effect of variations in walking context and exoskeleton assistance level on these representative muscles is discussed.

## 2. Material and Methods

## 2.1. Model Development

The exoskeleton which we use to provide assistance is the Active Pelvis Orthosis (APO), a revised version of the device presented by Giovacchini et al. (Giovacchini et al., 2015) (see **Figure 1A**). This exoskeleton was developed at The BioRobotics Institute of Scuola Superiore Sant'Anna (Pisa, Italy); the technology is currently licensed to IUVO Srl (http://www.iuvo.company, Pontdera, Italy). The APO provides a force applied to the thighs of the user transmitted via two carbon fibre lateral arms which are actuated by series elastic actuation units.

The APO developers adapted work by Ronsse et al. (Ronsse et al., 2011) to construct a high-level assistive controller, which generates a zero-delay estimate of the hip angles during gait and calculates a desired torque which is proportional to the estimated change in hip angle. A constant virtual stiffness parameter is used to calculate the torque necessary to drive the user's joint positions towards their expected future values. The APO can also be operated in "transparent mode", where the system provides no assistance to the user and the joints are free of resistance.

We developed a unique model of a human subject wearing the APO (see **Figure 1B**) using the software OpenSim (Delp et al., 2007). We took a pre-existing OpenSim human model with 92 muscles and 23 degrees of freedom, known as the gait2392 model (Yamaguchi and Zajac, 1989; Delp et al., 1990a; Anderson and Pandy, 1999a; Anderson and Pandy, 2001), and constrained the APO to it with three weld constraints, located between the backpack and the pelvis and between the two links and the femur. The APO mass and inertia properties were imported from a CAD model provided by IUVO.

## 2.2. Experimental Protocol

Data was collected for each subject while they walked on a treadmill in a variety of walking contexts and exoskeleton assistance scenarios. Reflective markers were attached to each subject to accurately track their movements using a six camera motion capture system (Vicon, Oxford, UK). The marker set used was adapted from the Cleveland marker set and consisted of 33

FIGURE 1 | (A) The Active Pelvis Orthosis (APO). (B) An overview of the APO OpenSim model's constraints, bodies, and degrees of freedom: (1) APO backpack, (2) Weld constraint between APO backpack and pelvis, (3) right APO group body (houses the actuators), (4) APO free joint (6 DOF), (5) right APO link, (6) weld constraint between APO link and femur. (C) The experimental setup. This image is published with the written informed consent of the depicted individual. ©IEEE 2017

markers, 8 of which were solely used for the purpose of scaling the dynamic model. Ground reaction forces and moments were collected using a six axis, split belt instrumented treadmill (Motekforce Link, Amsterdam, Netherlands). The torques applied by the APO were measured directly from the device. **Figure 1C** demonstrates the experimental set up.

To capture data in different walking contexts, a script was implemented in the Motek D-Flow software to programmatically change the speed or incline of the treadmill appropriately. Each subject was made to walk in five different walking contexts, as follows: at baseline walking speed with no incline (BW), at baseline walking speed with an incline of 5 degrees (UW), at baseline walking speed with an incline of −5 degrees (DW), at a fast walking speed with no incline (FW), and at a slow walking speed with no incline (SW). The baseline walking speed used for the BW context was calculated using the principle of dynamic similarity as described by the Froude number (Vaughan and O'Malley, 2005):

$$\nu = \sqrt{F\_r \cdot \mathcal{g} \cdot L},\tag{1}$$

where *v* is the baseline speed, *Fr* is the Froude number (chosen to be 0.1), *g* is gravitational acceleration (9.81 m/s2), and *L* is leg length (as measured from the greater trochanter to the medial malleolus). The speeds for FW and SW were calculated by adding and subtracting 20% to the baseline speed respectively. Each context was timed to last 135 s, with data collection triggered to happen after 120 s to allow for the participant to become accustomed to the context. For time synchronisation, the D-flow script sent a command to a relay box which simultaneously triggered the Vicon system, Motek treadmill and APO to begin recording data while accounting for internal delays. The kinematics, ground reaction forces and moments, and APO data were captured at 100 Hz, 600 Hz, and 100 Hz respectively.

The contexts were repeated for 3 different assistance scenarios: one without wearing the APO (NE), one wearing the APO set in transparent mode (ET) and one wearing the APO set in assistive mode (EA) with the virtual stiffness set to 15 Nm/rad. For each subject, a static pose was collected in both the NE and ET assistance scenarios.

## 2.3. Post-Processing

Before the data could be analysed, several post-processing steps had to be undertaken. Any gaps in the raw recorded motion capture data were filled using Vicon's software Nexus. A combination of the built in algorithms were used including the spline fill, the pattern fill, and the cyclic fill. The MoNMS toolbox (Mantoan et al., 2015) was used for the remainder of the motion capture data processing. The marker trajectories were first low-pass filtered with a zero-lag 4th order Butterworth filter and then transformed from the Vicon axis system into the OpenSim axis system.

For the ground reaction forces and moments, custom scripts were written in MATLAB. The first step was to compensate for data collected when the treadmill was tilted, and therefore causing gravity to work in a different direction to the force plate sensors. The ground reaction forces were then filtered using a zero-lag 4th order Butterworth filter with a 6 Hz cut-off. For the next step a threshold filter was applied to the ground reaction forces and moments that set all values equal to zero when the vertical force was less than 40 N. This was implemented because the CoP values were noisy when the vertical ground reaction forces were low. Additionally, it filtered out any noise in the force measurements during the swing phase of the gait cycle when there should be no forces applied to the foot. After applying the threshold, the CoPs were calculated and the global force plate moments were converted into free moments around the foot. Finally, the D-flow axis system was transformed to the OpenSim axis system.

## 2.4. APO Joint Misalignment

The problem of joint misalignment is well known when dealing with physically coupled systems, e.g. humans wearing exoskeletons (Malosio et al., 2011; Jarrassé and Morel, 2012; Zanotto et al., 2015). If joints are perfectly aligned, exoskeleton forces can be modelled as equal and opposite torques applied to the relevant exoskeleton and human bodies. However, the presence of joint misalignment results in imperfect transmission of torque from the exoskeleton to the human user (Schiele and van der Helm, 2006), ultimately resulting in reduced torque about the human joint and the introduction of undesirable forces parallel to the human limb, which can cause discomfort or unintended changes to muscle activation patterns (Hidler and Wall, 2005). The design of the APO is such that the exoskeleton joints should closely align with those of the human (Giovacchini et al., 2015), however it is not realistic to expect perfect alignment. By fixing a reflective marker on the rear of the exoskeleton, it was possible to identify the offsets between the human and exoskeleton joints for each subject. The protocol for calculating the offsets is outlined below.


The above steps were undertaken for all subjects using the corresponding static pose data. The offsets for each subject are summarised in **Table 1**. Once the offsets between the subjects and the APO were known, a model was derived following a similar strategy to previous works to decompose the torque generated by the APO in to an assistive torque applied to the human hip joint and an undesired interaction force which is applied parallel to the thigh (Schiele, 2008; Malosio et al., 2011). A sample APO torque trajectory is provided in **Figure 2** which displays this decomposition for a single gait cycle.

## 2.5. APO Torque Transmission Models

Due to the presence of compliance in human-exoskeleton systems, largely due to flexible straps and soft biological tissues, power loss occurs between the torques generated by the exoskeleton and the

TABLE 1 | The static joint offsets between the hip joints of each subject and the APO joint centres.


*Note that the x axis is directed forward from the pelvis and the y axis is directed upwards.*

torques experienced by the human subject. A relatively limited number of previous studies have investigated these interface dynamics in more detail, using a mix of kinematic and load sensing measurement devices to estimate the visco-elastic properties of the human-exoskeleton system (Schiele, 2008) or the relative timing and magnitude of power loss (Yandell et al., 2017).

In subsequent metabolic analyses of the APO in assistive mode, two models were used for the transmission of exoskeleton torques. Both models account for human-exoskeleton joint misalignment. The first model, hereafter referred to as the **ideal** torque transmission model, assumes 100% transfer of torque from the exoskeleton to the human. The second model, referred to as the **compliant** torque transmission model, assumes that the torque transmission is subject to absorption-return dynamics as observed by a recent study in to the interface dynamics of a soft exosuit (Yandell et al., 2017).

The compliant torque transmission model partitions the APO torque signal in to phases categorised as **loading** while the APO torques are increasing in magnitude and **unloading** while the torques are decreasing in magnitude. A percentage *A* of the power generated by the APO during loading phases is absorbed by the soft tissue in the system, and a percentage of this absorbed power, *R*, is returned during the next unloading phase. This results in a temporal offset between the peak applied torque of the ideal and compliant models. In **Figure 2**, a comparison between the ideal and compliant torque models is displayed for a single gait cycle. The absorption and return rates were chosen to be *A* = 0.55 and *R* = 0.75, respectively, to match the observations of Yandell et al. and allow for qualitative discussion on the compliant human-APO interface.

## 2.6. OpenSim Analyses

The processed data for each subject was divided in to 10 gait cycles per combination of walking context and assistance scenario. A range of analyses were then carried out using OpenSim tools in combination with the gait2392 and human/APO musculoskeletal models.

The first step was to scale the generic versions of these models to fit each subject using the static pose data and the **Scaling Tool**, which matched experimental markers placed on bony landmarks to the corresponding virtual markers placed on the model. The

Figure 2 | A graphical representation of the APO force models for a single gait cycle. The ideal force model (With Offsets) and compliant force model (Compliant) are shown relative to the measured APO torques (No Offsets). Left: the torque applied to the human femur body. Note the delayed onset of the peaks in the compliant force model. Right: the introduction of undesired interaction forces directed parallel to the thigh. These forces are introduced due to the presence of joint offsets.

gait2392 model was scaled using static pose data from the NE assistance scenario, while the human/APO model was scaled using data from the ET assistance scenario. After scaling, the following sequence of analyses was carried out for each gait cycle:


Each of the dynamic analyses was performed twice for the EA assistance scenario; once using the ideal APO torque transmission model and again using the compliant model. The outputs of these analyses were used in the calculation of the stability metrics and metabolic energy consumption. A schematic of the overall data processing and analysis pipeline is provided in **Figure 3**.

## 2.7. Candidate Stability Metrics

A set of candidate stability metrics were chosen so as to cover a range of spatial, temporal and derived metrics. It is posited that those gait metrics which exhibit a strong invariance to changes in walking context or exoskeleton assistance scenario are good candidates for use as a measure of gait stability in variable walking conditions. The metrics and corresponding definitions were as follows:


$$\text{MoS} = |\mu\_{\text{max}} - \left(\chi + \frac{\nu}{\omega\_0}\right)|,\tag{2}$$

where *umax* is the boundary of the base of support, *x* is the centre of mass position, *v* is the centre of mass velocity, and *ω*0 is equal to:

$$
\omega\_0 = \sqrt{\frac{\mathbf{g}}{l}},\tag{3}
$$

with *g* denoting acceleration due to gravity and *l* being the distance from the pelvis ASIS to the lateral malleolus. The MoS was calculated in both the anterior-posterior and mediolaterial directions, resulting in two distinct metrics: MoS-AP and MoS-ML, respectively.

These metrics are summarised for reference in **Table 2**, which gives the names, units and notation for each metric.

Note that stability was considered for the system as a whole, and therefore the net kinematic and ground reaction force data was used for metric calculations within the EA assistance scenario. The APO force models were not used to distinguish


between the human and exoskeleton torque contributions in this case.

## 2.8. Modelling Metabolic Power Consumption

The calculation of metabolic power consumption followed a strategy used in other works (Uchida et al., 2016b; Dembia et al., 2017). The CMC algorithm (Thelen and Anderson, 2006) was run for each gait cycle, with a concurrent metabolics probe used to compute the instantaneous metabolic power consumption, *E* ( *t* ) , for each muscle in the model. This probe computes the metabolic power consumption according to a muscle energetics model implemented by the OpenSim developers (Uchida et al., 2016a), which in turn is based on a previous model (Umberger et al., 2003).

The gait2392 model and our adapted human/APO musculoskeletal model both use Hill-type muscle models implemented by the OpenSim developers, which represent muscles as musculo-tendon units (MTUs) consisting of a tendon in series with a contractile muscle (Thelen, 2003). Each muscle is characterised by its maximum isometric force, optimal muscle fiber length, tendon slack length, maximum contraction velocity, and pennation angle. The values of these parameters are informed by previous studies (Wickiewicz et al., 1983; Friederich and Brand, 1990; Delp et al., 1990b; Anderson and Pandy, 1999b).

Once calculated, the instantaneous metabolic power consumption of each muscle was integrated over the gait cycle



and divided by total subject mass (*m*) and gait cycle length (in seconds) to produce the normalised, average metabolic power consumption, as follows:

$$\bar{P}\_{\text{avg}} = \frac{1}{m \left(t\_1 - t\_0\right)} \int\_{t\_0}^{t\_1} E\left(t\right) \text{ d}t. \tag{4}$$

Normalised averaged metabolic power consumption was calculated both using the ideal APO torque transmission model and, separately, the compliant model.

## 2.9. Statistical Analysis

The mean and SD of each stability metric was averaged over all recorded gait cycles and all subjects, for each combination of walking context and assistance scenario. Therefore, the number of samples for each combination of stability metric, walking context and assistance scenario was 70 for most metrics (10 gait cycles *<sup>×</sup>* <sup>7</sup> subjects<sup>1</sup> ). An exception was the step width metric, which relies on pairs of adjacent gait cycles for its computation, and therefore had a sample size of 63 (9 pairs of adjacent gait cycles *×* 7 subjects).

The mean and SD of the normalised average metabolic power consumption was calculated in the same way as outlined above for each muscle in the musculoskeletal model, however only half of the recorded gait cycles were used for each subject to reduce the time taken to perform the simulations. Consequently, the number of samples for each combination of muscle, walking context and assistance scenario was 35 (5 gait cycles *×*7 subjects).

To investigate the effects of the exoskeleton assistance and the walking context on the stability metrics, a two-way ANOVA was used. For the post-hoc analysis, the MATLAB multiple comparison procedure "multcompare" was used with the comparison type based on Tukey's honestly significant difference criterion. The statistical significance level was set at *α* = 0.05.

For combinations of walking context and assistance scenario which demonstrated a significant difference in the mean of a metric, the effect size was measured by computing the absolute value2 of Cohen's *d*. These values were then averaged to produce a quantitative measure of invariance for each metric relative to changes in assistance level, changes in walking context, and overall. Qualitative analysis of the effect size of each metric was undertaken according to typical



cutoffs for the value of Cohen's *d* (Cohen, 1988; Sawilowsky, 2009), which are provided for reference in **Table 3**.

The two-way ANOVA and multiple comparison procedure was repeated with the same statistical significance level to investigate changes in normalised average metabolic power consumption for a subset of muscles crossing the hip, knee and ankle joints. The included muscles were as follows: the adductor brevis, adductor longus, adductor magnus, psoas, gluteus maximus, biceps femoris, rectus femoris, vastus medialis, medial gastrocnemius and soleus. For reference these muscles and their main actions are listed in **Table 4**.

In order to directly compare the effect of active exoskeleton assistance between different contexts, a one-way ANOVA was performed for each context over all assistance levels and for each muscle. Within each context, the muscles which had significantly different average metabolic power consumption when in active-ideal assistance mode (EA-I) or active-compliant assistive mode (EA-C) compared to transparent mode (ET) were identified. The relative change in metabolic power consumption going from transparent mode to active mode was then calculated as a percentage in order to quantify the effectiveness of the exoskeleton assistance.

## 3. Results

The anthropometric measurements and calculated walking velocities for each subject are presented in **Table 5**.

Running the RRA tool for all the data sets generated residual forces and moments, which are applied to the pelvis in simulation to account for the dynamic inconsistency between the dynamic model



<sup>1</sup>The data from one subject had to be discarded due to an issue with the force plate calibration.

<sup>2</sup>The decision to take the absolute value was motivated by an interest in the magnitude of an effect rather than its direction.


TABLE 6 | RRA residuals in OpenSim. ©IEEE 2017

and the recorded data. These residuals should be low to ensure accurate simulations. All of the average residual forces measured during our simulations were less than the thresholds specified by the OpenSim developers (see **Table 6**).

## 3.1. Gait Metric Invariance

For each metric and for every context and assistance scenario the percentage difference from the baseline condition (no exoskeleton assistance and walking at baseline speed) is shown in **Figures 4 and 5**. Additionally, the mean and SD values for every context and assistance scenario combination are detailed in the Table S1 (Data Sheet S1).

The effect sizes for each metric, averaged separately by walking context and by assistance scenario, are displayed in **Figure 6**. Comparing the effect sizes averaged over walking contexts, it is demonstrated that step width, CoP-MLdisp, and MoS-ML have the lowest average effect size. These metrics have Cohen's *d* values that indicate between small and medium effect sizes. The *θ*hip-RoM metric exhibits a Cohen's *d* of greater than 2.0, which implies a huge effect size. All other metrics show effect sizes of large or very large. This analysis implies that, relative to changes in walking context, the step width, CoP-MLdisp, and MoS-ML metrics demonstrate the highest level of invariance.

Comparing the effect sizes averaged over assistance scenario, the metrics which exhibit the lowest average effect size are CoP-APdisp, CoM-MLdisp and, similarly to the context results,

FIGURE 4 | Percentage difference from baseline, categorised by walking context and assistance scenario, for (A) step width, (B) step frequency, (C) *θ*hip-RoM, (D) *τ*hip-pp, (E) CoP-APdisp and (F) CoP-MLdisp. Black lines represent significant differences. ©IEEE 2017

MoS-ML. These metrics exhibit a Cohen's *d* of below 0.2, which indicates a very small effect size. All other metrics exhibit a small effect size. Compared to the results for walking context, the effect of changing assistance scenario is in general less than for changing walking context. This analysis implies that the CoP-APdisp, CoM-MLdisp and MoS-ML metrics demonstrate the highest level of invariance relative to changes in the assistance scenario.

FIGURE 6 | The effect sizes for each gait metric, averaged over (A) context and (B) assistance. Note that no significant differences due to changes in assistance level were observed for the MoS-ML metric. ©IEEE 2017

In **Figure 7**, the effect size is shown for each metric, averaged over both walking context and assistance level simultaneously. From these results, we see that step width, CoP-MLdisp and MoS-ML are the metrics which exhibit the lowest effect size, with values of Cohen's *d* between 0.2 and 0.5 corresponding to a medium effect. Analysing the remaining metrics, we see that the CoM-MLdisp metric exhibits a medium effect size, the *θ*hip-RoM metric exhibits a very large effect size, and all remaining metrics exhibit large effect sizes. From this analysis we conclude that overall, taking in to account both changes in walking context and assistance scenario, the three metrics which exhibit the most invariance are step width, CoP-MLdisp and MoS-ML.

## 3.2. Metabolic Power Consumption

For each muscle in **Table 4** the percentage difference in average metabolic power consumption from the baseline condition (no exoskeleton assistance and walking at baseline speed) as a function of walking context and assistance level is demonstrated in **Figures 8 and 9**. Additionally, the raw mean and SD values for every context and assistance scenario combination are detailed in Table S2 (Data Sheet S1). For analysis purposes the data from the two APO force models are presented as distinct assistance levels.

Given the cases where significant differences in the average metabolic power consumption were observed, the effect size was averaged along assistance and context. The results are presented in **Figure 10**. Notably, the change in assistance had no significant effect on four of the muscles, namely: the adductor longus, adductor magnus, gluteus maximus or biceps femoris long head. Changes in walking context introduced at least some significant differences to all muscles in the subset.

In general, the observed effect of assistance level on the average metabolic power consumption of the muscles was small, with all effect sizes lying within the range 0.2 *≤* 0.5 regardless of whether the ideal or compliant APO force model was used. In each of the muscles which demonstrated a significant effect, the compliant APO force model resulted in a slightly more pronounced or equivalent effect size when compared to the ideal force model.

In contrast, the muscles we considered experienced a wider range of effect sizes due to changing walking context. The psoas, gluteus maximus, biceps femoris long head, medial gastrocnemius and soleus muscles all exhibited large effect sizes, with a Cohen's *d* of greater than 0.8 but less than 1.2. The vastus medialis was the muscle which was most affected by changes in walking context, exhibiting a very large effect size. The adductor longus experienced a medium effect size and the adductor brevis, adductor magnus, and rectus femoris all experienced small effect sizes. Similar to the assistance-averaged case, use of the ideal or compliant force models did not alter the classification of the effect size for any muscle.

From **Figures 8 and 9** we can see that only one muscle from our subset shows a significant difference in metabolic energy consumption on average over all contexts due to the transition from transparent mode to active assistance, namely the adductor brevis muscle. Five muscles, the psoas, rectus femoris, vastus medialis, medial gastrocnmeius and soleus show significant differences between the NE and ET assistance levels, which implies the change in metabolic energy consumption of these muscles is attributable to the increased physical load of wearing the exoskeleton rather than the presence of active assistance itself. One muscle, the psoas, shows a significant difference between the APO force models.

The muscles from the full set which were identified via the context-specific one-way ANOVAS to experience a significant change in metabolic energy consumption between the ET and EA assistance levels are listed by context in **Table 7**. Note that significant differences were seen for the uphill walking, downhill walking and slow walking contexts for both force models. Significant differences were seen in the flat walking context, but only when the compliant model was used. No significant differences were seen in the fast walking context. The relative change in metabolic power consumption is shown for the muscles in these contexts in **Figure 11**. The overall effect, calculated from total significant metabolic energy change in each context, is presented in **Figure 12**. We see from these results that the active exoskeleton assistance has a positive effect in the BW scenario, but only when the compliant APO force model is used. There is a disagreement in the UW context, where the ideal APO force model predicts a net assistance of approximately 8%, whereas the compliant force model predicts that human metabolic energy consumption is increased. The ideal force model predicts that the iliacus and psoas muscles are assisted; the main actions of these muscles is to assist hip flexion. Meanwhile, both force models agree that the active exoskeleton assistance is detrimental in the SW and DW scenarios, where human metabolic energy consumption is seen to increase by between approximately 5% and 12% depending on the walking context and which force model is used.

## 4. Discussion

## 4.1. Stability Metrics

It is well known that walking speed is a cause of gait variability for kinematic, kinetic, and CoMdisp metrics (Winter, 1984; Stansfield et al., 2001; Orendurff et al., 2004) and the results from this study also demonstrate the same findings. This study

gluteus maximus and (F) biceps femoris long head muscles.

demonstrates that step frequency increases due to speed and anecdotally suggests that step length increases as well (step length was not included in the analysis due to unavailability of complete consecutive step data for some of the participants due to cross-talk on the force plates). The step width metric was demonstrated to be invariant with only one significant difference with a small effect size between walking downhill and walking at a fast speed. Walking speed causes variation for the CoP-APdisp and the CoP-MLdisp metrics, however, the effect on CoP-MLdisp is small and only between the fast and baseline walking speed contexts. In addition, walking speed affects the MoS-AP and

MoS-ML metrics, however there is only a small effect in the medio-lateral direction between the slow and baseline walking speed contexts. It is a logical result that walking speed has a greater effect on the metrics measured in the anterior posterior direction compared to the medial lateral direction because walking speed is a change of direction mainly in the anterior posterior direction.

The effect of walking up and down an incline has previously been demonstrated to have significant kinematic and kinetic changes (Lay et al., 2006) and the results from this study support this result. In addition to the effects on kinematics and

kinetics, the results from this study indicate that walking on an incline affects the CoM-Vdisp metric and that walking down an incline affects the step frequency, CoM-Vdisp, CoM-MLdisp , and MoS-AP metrics. The increase in step frequency is expected due to a shorter step length being taken. The effects on the CoM vertical displacement are also expected due to the change in height caused by the slope. The effect on the MoS-AP metric is expected because this measures the stability in the backwards direction and it is clear that when the torso is tilted forwards the MoS-AP values will increase. Neither



walking up or down an incline had a significant effect on the MoS-ML .

This study demonstrates there are significant differences between the *θ*hip-RoM metric caused by context and assistance scenario changes. This result aligns with the work by d'Elia et al. (d'Elia et al., 2017), and similarly their claim that the differences between the assistance scenario *θ*hip-RoM results are within the natural variation of gait is also applicable. There was a significant increase in the *τ*hip-pp metric from the ET to the EA scenario. This result suggests a disagreement with a study by Lewis and

FIGURE 12 | The overall metabolic effect of exoskeleton assistance in each walking condition, for both the ideal APO force model (blue) and the compliant APO force model (red). Absence of data denotes that no significant differences were observed in this case.

Ferris (Lewis and Ferris, 2011), which found that the net torques did not change between walking with an exoskeleton in passive mode and with it in assistive mode. The result from our study

assistance modes during (A) flat walking, (B) uphill walking, (C) downhill walking and (D) slow walking. Note that red bars correspond to the compliant force model whereas blue bars correspond to the ideal force model. Absence of data denotes that no significant differences were observed in this case.

suggests that torque from the exoskeleton is not entirely being transferred to the individual, which is quite probable due to the non-rigid attachments of the exoskeleton. There is a significant increase in the CoP-MLdisp metric from the NE and the ET and EA scenarios, which can be attributed to the extra weight of the exoskeleton laterally located to the participant. One consideration for the above findings is that the effect sizes for all the metrics for the assistance scenarios were between small and medium, which suggests the differences are small between the assistance scenarios. This is in contrast to the context effect sizes, where 6 out of the 10 metrics had large or greater effect sizes.

All gait metrics exhibited some significant differences due to the changes in walking context and assistance scenario. After factoring in the effect sizes the most invariant metrics were shown to be step width, CoP-MLdisp, and MoS-ML. All three metrics have been demonstrated to be associated with stability (Hof et al., 2005; Kuo and Donelan, 2010; Svoboda et al., 2016), and therefore it is intuitive that they remain fairly constant despite changes in walking context and the application of small constant perturbations. The MoS metric is most suitable for use in a control paradigm because it can be calculated at any given time during a gait cycle and therefore enables a significantly more responsive controller. As a source of further work, our investigation of stability metrics could be repeated with patients to determine how the invariance properties of the metrics are affected by gait pathologies.

## 4.2. Metabolic Power Consumption

In general, we observed that applying exoskeleton assistance had significantly less effect on metabolic energy consumption than changes in walking context. However, this analysis was limited by the fact that while there were three assistance scenarios, only one of these scenarios explored active assistance, and therefore comparison between different magnitudes of active assistance was not explored. A source of further work could be to collect data using a wider range of virtual stiffness levels, which would allow for an analysis of how the metabolic effect of active assistance varies with assistance magnitude. It should be noted that the motor torques commanded to the APO during active assistance trials were already close to the torque limitations in place on the device.

The relative effect of applying exoskeleton assistance was most pronounced in the flat walking, uphill walking, downhill walking and slow walking scenarios. In the latter two of these scenarios, both the ideal and compliant APO force models predicted increased metabolic cost. The compliant model predicted that flat walking benefited from assistance, while the ideal model predicted that uphill walking benefited from assistance. Anecdotally, this result agrees with feedback from subjects following data collection (e.g., the exoskeleton assistance was most beneficial when walking uphill). The negative effect of the exoskeleton when walking at slow speed may be a result of the choice of control algorithm. As discussed in Section 2.1, the control algorithm used is based on adaptive oscillators, which requires synchronisation to input joint angles. Therefore, the decrease in performance in the slow walking context may have been due to a suboptimal synchronisation. However, during walking trials, time was allowed both for subject familiarisation with the new context and for APO controller synchronisation.

A limiting factor of our study is that the adaptive oscillator control sceheme was the only exoskeleton controller tested. A source of further work could be to apply or develop additional control paradigms, so that their relative performance over different contexts can be analysed. Additionally, external measurement devices such as calorimetry systems, which have recently been used for investigations in to metabolic cost reductions of soft exosuits (Quinlivan et al., 2017), could be helpful in directly quantifying changes in net metabolic activity in future experiments, reinforcing our simulation framework.

It should be noted that the implementation of a compliant APO force model was intended largely as a point of qualitative comparison between the results from the ideal APO force model. Indeed, several differences between the experimental setup used for this work and the work on interface dynamics (Yandell et al., 2017) could lead to differing compliant behaviour in our case. However, these results do suggest qualitatively that including compliance in the system, as certainly is the case for contact between exoskeletons and soft straps or human tissue, can cause marked differences in the effectiveness of exoskeleton assistance. Further work is needed to implement control algorithms which can account for human-exoskeleton interface dynamics in real-time.

Overall, our results quantify the effect that varying walking context has on the effectiveness of active exoskeleton assistance. If exoskeletons are to be applied in real-world settings where subjects may frequently adapt their walking speed or incline, they must be able to rely on adaptive control algorithms which can account for these changes in walking context. Failure to do so can result in increased, rather than decreased, metabolic energy costs, as shown by our analysis. Techniques based on musculoskeletal modelling over various walking conditions can be useful and non-invasive tools for testing how well exoskeleton control algorithms perform in this regard.

## Ethics Statement

To carry out the experiments approval from the School of Informatics' Ethics Panel was received. Eight participants were recruited to undertake data collection after providing written informed consent.

## Author Contributions

DG and GH jointly created the human/APO model, devised the data collection protocol and carried out the experiments. DG implemented the batch OpenSim analysis pipeline and the metabolics calculations. GH implemented the post-processing pipeline and the gait metric calculations. SV provided continuous supervision and critical feedback on methods and results. All authors contributed to the drafting of the article.

## Funding

This research is supported by EU Horizon 2020, Project ID 780684 (MEMMO), and the Engineering and Physical Sciences Research Council (EPSRC), as part of the CDT in Robotics and Autonomous Systems at Heriot-Watt University and The University of Edinburgh.

## Acknowledgments

This article is an extension of a conference paper titled *Identifying invariant gait metrics for exoskeleton assistance* (Henderson et al.,

## References


2017), presented at IEEE ROBIO 2017, Macau, China, ©IEEE 2017. The model, data set and processing methodology are shared between these works. The simulations, results and analyses related to metabolic energy expenditure are original contributions for this paper.

## Supplementary Material

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/frobt.2018.00061/ full#supplementary-material

cerebral palsy based on magnetic resonance imaging. *J. Biomech.* 40 (3), 595– 602. doi: 10.1016/j.jbiomech.2006.02.003


**Conflict of Interest Statement:** The authors declare that this research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2018 Gordon, Henderson and Vijayakumar. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Leg Force Control Through Biarticular Muscles for Human Walking Assistance

Maziar A. Sharbafi1,2 \*, Hamid Barazesh<sup>1</sup> , Majid Iranikhah<sup>3</sup> and Andre Seyfarth<sup>2</sup>

<sup>1</sup> Control and Intelligent Processing Center of Excellence, School of Electrical and Computer Engineering, University of Tehran, Tehran, Iran, <sup>2</sup> Lauflabor Locomotion Laboratory, Institute of Sport Science, Centre for Cognitive Science, Technische Universität Darmstadt, Darmstadt, Germany, <sup>3</sup> Qazvin Branch, Mechatronics Research Lab, Center of Excellence in Robotics, Islamic Azad University, Qazvin, Iran

Assistive devices can be considered as one of the main applications of legged locomotion research in daily life. In order to develop an efficient and comfortable prosthesis or exoskeleton, biomechanical studies on human locomotion are very useful. In this paper, the applicability of the FMCH (force modulated compliant hip) model is investigated for control of lower limb wearable exoskeletons. This is a bioinspired method for posture control, which is based on the virtual pivot point (VPP) concept, found in human walking. By implementing the proposed method on a detailed neuromuscular model of human walking, we showed that using a biarticular actuator parallel to the hamstring muscle, activation in most of the leg muscles can be reduced. In addition, the total metabolic cost of motion is decreased up to 12%. The simple control rule of assistance is based on leg force feedback which is the only required sensory information.

#### Edited by:

Jose Gonzalez-Vargas, Otto Bock, Germany

#### Reviewed by:

Fan Gao, University of Kentucky, United States Dai Owaki, Tohoku University, Japan

#### \*Correspondence:

Maziar A. Sharbafi sharbafi@sport.tu-darmstadt.de; sharbafi@ut.ac.irr

Received: 10 February 2018 Accepted: 18 June 2018 Published: 11 July 2018

#### Citation:

Sharbafi MA, Barazesh H, Iranikhah M and Seyfarth A (2018) Leg Force Control Through Biarticular Muscles for Human Walking Assistance. Front. Neurorobot. 12:39. doi: 10.3389/fnbot.2018.00039 Keywords: exosuit, reflex-based control, neuromuscular models, walking assistance, biarticular actuation

## INTRODUCTION

Legged locomotion is a complex nonlinear hybrid problem. There are abstract models which simplify understanding such a complex problem that can explain basic characteristics of human walking to be used for design and control of the artificial legged systems. One of the most popular concepts for abstraction is the "Template and Anchor" concept (Full and Koditschek, 1999). In this method, simple conceptual (template) models are used to describe some basic features of legged locomotion than can be extended to more detailed (anchor) models to implement on robots. Another approach is using the locomotor sub-function concept (Sharbafi and Seyfarth, 2017a) which explains legged locomotion based on three locomotor sub-functions, which are intrinsically interrelated. As shown in **Figure 1**, these three sub-functions are "Stance" for redirecting the center of mass by exerting forces on the ground; "Swing" as rotational movement of the free leg (no contact with the ground) around hip joint and "Balance" for maintaining body posture. Splitting the legged locomotion as a complex problem to three sub-problems helps us simplify understanding human locomotion (Sharbafi and Seyfarth, 2017b) and improve design, and control of legged locomotor systems (Raibert, 1986). As a result, combination of the template-anchor and locomotor sub-function concepts provide a practical tool to benefit from biological locomotor systems in design and control of robots and assistive devices (Ahmad Sharbafi et al., 2017).

There is a variety of methods to control exoskeletons inspired by human locomotion and motor control such as a force controller that behaves similar to a biological hip torque profile (Yu et al., 2014), a proportional myoelectric controller (Ferris et al., 2006), impedance control (Sharbafi and Seyfarth, 2017a), central pattern generators (CPGs) (Sobrade et al., 2017) and recently, Hybrid Zero Dynamics (HZD) method (Agrawal et al., 2017). In order to produce a proportional control signal, the myoelectric controller makes use of electromyography (EMG) of the muscles of the lower limbs for the hip exoskeleton (Ferris et al., 2006). Impedance control regarding the dynamic interactions between the leg and the ground is also very popular in exoskeletons with rigid structure like LOPES (Veneman et al., 2007).

In this paper, we focus on balance control (the third locomotor sub-function), as it might be more challenging in assisting healthy humans via exoskeletons (Full and Koditschek, 1999), in which reducing cost of transport and robustness against perturbations are addressed. Postural control consists of complex interactions between a number of systems in the human body such as musculoskeletal components, neuro-muscular synergies and adaptive mechanisms to achieve gait stability.

In Maus et al. (2010), by analyzing human and animals locomotion experiments Maus et al. showed a pattern in ground reaction forces, introducing VPP (standing for virtual pivot point) for posture control. The VPP is a point on the upper body above the center of mass at which the GRFs are intersecting during the stance phase. This observation in human walking (and animal walking and running) can be also used for posture control in models and robots (Maus et al., 2010; Sharbafi et al., 2013). A new mechanical template model was developed in Sharbafi and Seyfarth (2015) to generate VPP using an adjustable hip spring. This model which is called FMCH (force modulated compliant hip) employs the leg force to adjust hip compliance. Here, this bioinspired control approach for balancing is utilized to design and control of an exoskeleton with one biarticular actuator. This method was inspired by neuromuscular models and reflex control while it benefits from biological feedback signal. For example, Geyer et al. demonstrated that reflex-based motor control (e.g., positive feedback of muscle) can generate efficient and reliable bouncing gaits instead of using central motor commands (Geyer et al., 2003). Other studies on reflex control show the important potential of this bioinspired method for developing human gait models (Geyer and Herr, 2010; Song and Geyer, 2015) and understanding human motor control (Haeufle et al., 2012). The neuromuscular model of Geyer and Herr (Geyer and Herr, 2010) is a well-accepted human walking model, which is extended to 3d in Song and Geyer (2015) and also for analyzing performance of prosthesis and exoskeletons in Thatte and Geyer (2016). This model can be utilized as a reference neuromuscular model for assessing human (bipedal) locomotion. We use it in our simulation studies to investigate the performance of the proposed design and control approach of exoskeletons in assisting human walking.

Exoskeletons are developed to enhance human's movement capabilities, e.g., to carry heavy loads or make up for physical disorders caused by deficiencies in the muscular nervous system.

Interaction between the robots and human beings can be improved if human body properties and motor control are better understood. Exoskeletons can be divided to rigid (Veneman et al., 2007; Esquenazi et al., 2012) and soft exoskeletons (Asbeck et al., 2014; Ding et al., 2017). The second group, namely exosuit, is matching better to human body properties while the rigid exoskeletons are more powerful and practical for impaired people. In exosuits, similar to the human body, the transfer of torques to the joints is performed through tensile forces parallel to the body muscles. The main application of exosuits is enhancement in performance of healthy subject needless to have powerful actuators to carry body weight. Applicability of FMCH for assisting a rigid exoskeleton (LOPES II) was shown in our previous study (Zhao et al., 2017). Here, we present advantages of implementing this method using a biarticular actuator (parallel to hamstrings, shown in **Figure 2A**) that can be easily implemented on soft exoskeletons.

## METHODS

One of the most useful applications of studying biomechanics of legged locomotion is design and development of assistive devices. The goal of this study is to investigate the applicability (assistance level) of a bioinspired template-based method for posture control regarding reduction in metabolic cost and muscle activation. In this paper, we use the reflex-based neuromuscular model of Geyer and Herr (2010) for human-like walking. In the following, biarticular thigh muscle.

first we explain bipedal walking models and then the proposed FMCH-based control approach and its implementation as a soft exoskeleton in the aforementioned neuromuscular model.

## Conceptual Modeling of Bipedal Walking

Bipedal walking can be described by repetition of two sequential phases: double support (DS) and single support (SS). In DS, both legs are in contact with the ground and the center of pressure (CoP) moves from hind foot to the front foot. When the hind leg leaves the ground (takeoff), SS starts and continues until its next contact with the ground (touchdown). In SS, one leg is in contact with the ground (called stance leg), while the swing leg moves to complete the step by touching the ground with a desired angle of attack.

In order to explain our control concept, a minimal model is required. To analyze and describe animal or human locomotion, simple conceptual models, called "templates," are useful. Templates provide a great deal of information, which can help explain the features of locomotion. Templates are also used as explicit control models. One of the most useful template models for walking and running is the SLIP (Spring-Loaded Inverted Pendulum) (Blickhan, 1989; Seyfarth et al., 2002; Geyer et al., 2006). In the SLIP model, the body mass is concentrated at the center of mass (CoM) on top of a massless spring representing the stance leg.

In order to address posture control, an upper body should be added to the SLIP model. The common way is extending the model by an additional rigid trunk, resulting in TSLIP (Trunk+SLIP) model (Sharbafi et al., 2013). The basic SLIP model was developed for hopping (Blickhan, 1989) and running (Seyfarth et al., 2002). For walking a second leg is required which together with the trunk results in BTSLIP (Bipedal + TSLIP) model (Sharbafi and Seyfarth, 2015). This model is used in section Control Method Description to describe the VPP and FMCH control concepts. In the next section, a brief overview of the neuromuscular model of walking is presented.

## Human Walking Model

All simulations are implemented on top of a basic 2D model named muscle-reflex model (Geyer and Herr, 2010) including 7 segments (1 upper body, 2 thighs, 2 shanks and 2 feet) and 7 muscle groups for each leg. **Figure 2A** shows the segmentation of one leg and different muscles, implemented in this leg. The model includes muscle dynamics and hypothesized reflex pathways to generate joint torques, to mimic human walking patterns regarding kinematics, kinetics, and muscle activation. With this model, a network of muscle reflexes is utilized as a practical tool to link complex, neural circuits of biological locomotor systems and abstraction in conceptual models. By dynamic interplay of the body and the ground besides internal neural circuitry, this model can generate human-like walking which is also robust against perturbations. Furthermore, this model can be employed for assessing robots' performance in assisting human locomotion (Thatte and Geyer, 2016).

The description of the human walking model in the following is basically borrowed from Geyer et al. (2003) and Geyer and Herr (2010), where the segmented model and the reflex-based neuromuscular control are presented.

In this model, MTC (muscle-tendon-complex) consists of a contractile element (CE) and a series elastic element (SEE) (**Figure 3**). The serial elastic element (SEE) which plays the tendon role in this model follows the nonlinear unidirectional spring, inspired by the model of van Ingen Schenau (1984):

$$F\_{\rm SEE}(l\_{\rm SEE}) = \begin{cases} \left(\frac{l\_{\rm SEE} - l\_{\rm test}}{l\_{\rm ref} - l\_{\rm test}}\right)^2 & \text{if } l\_{\rm SEE} > l\_{\rm rest} \\ 0 & \text{if } l\_{\rm SEE} \le l\_{\rm rest} \end{cases} \tag{1}$$

where lrest is the tendon's resting length and lref is the reference length.

The developed force of the CE is a function of the muscle activation level A(t), the maximum isometric force Fmax, the force-length function f<sup>l</sup> (lCE) (and the force-velocity relationship fv(vCE),

$$F\_{\rm CE} = A(t)F\_{\rm max}f\_{l}(l\_{\rm CE})f\_{\nu}(\nu\_{\rm CE})\tag{2}$$

in which lCE and vCE are the muscle length and contraction speed, respectively. The force-length function is given by the following equation (Aubert, 1956):

$$f\_l(l\_{\rm CE}) = e^{\left(c\left|\frac{l\_{\rm CE} - l\_{\rm opt}}{wl\_{\rm opt}}\right|^3\right)}\tag{3}$$

Where lopt , w and c are the optimum CE length, the width of the bell-shaped f<sup>l</sup> (lCE) curve and a constant value, respectively. The force-velocity relation is composed of the Hill model (Hill, 1938)

and Auber model (Aubert, 1956) for contraction and protraction, respectively (see Geyer et al., 2003, for details):

TABLE 1 | Muscle properties and reflex gains in the neuromuscular model of human walking, from Geyer and Herr (2010).

$$f\_{\nu} \left( \nu\_{\rm CE} \right) = \begin{cases} \frac{\nu\_{\rm max} - \nu\_{\rm CE}}{\nu\_{\rm max} + K \nu\_{\rm CE}} & , \qquad \nu\_{\rm CE} < 0\\ N + (N - 1) \frac{\nu\_{\rm max} + \nu\_{\rm CE}}{7.56 K \nu\_{\rm CE} - \nu\_{\rm max}} , & \nu\_{\rm CE} \ge 0 \end{cases} \tag{4}$$

in which vmax and N denote maximum contraction velocity and a constant value, respectively. The activation is resulted from stimulation signal, which uses reflex pathway as shown in **Figure 3**.

$$
\text{tr}A\left(t\right) = STIM\left(t\right) - \frac{d\left(A\left(t\right)\right)}{dt} \tag{5}
$$

$$STIM(t) = \begin{cases} \begin{array}{c} STIM0, \ \ \ t < \Delta\_{\mathcal{P}} \\ \end{array} \\ \begin{array}{c} STIM0, \ \ \ t \ge \Delta\_{\mathcal{P}} \end{array} \end{cases} \tag{6}$$

where STIM(t), A(t) , STIM<sup>0</sup> and are stimulation signal, activation signal, stimulation bias and signal propagation delay, respectively.

The feedback sensory signals P is given by a combination of three reflex pathways CE length (lCE), velocity (vCE), and the muscle force (FCE) which is equal to the MTC force and the FSEE. In this model, different combinations of reflex pathways are considered for different muscles and the gain values are optimized to achieve stable human-like walking.

In order to show the assistance level of the exosuit, we compare the metabolic cost (see section Optimization) for three different models. First, we consider the human walking model without assistance, borrowed from Geyer and Herr (2010). In our simulations, all the parameters of the neuromuscular model are set to values defined in Geyer and Herr (2010). **Table 1** shows the muscle properties and reflex gain factors and the model can be downloaded here.

In the second model, we added exosuit for assistance, neglecting the additional weight of the exo. The parameters of the exo are found using optimization techniques explained in section Optimization. This model can represent the level of assistance compared to the transparent mode. Another application of this


model is to represent walking of a human subject wearing a passive version of the assistive device (under construction) with low weigth to compare with normal walking. In addition, in most of the studies on exosuits, the weight of the actuation setup is neglected using the tethered actuation system in which the actuators and the corresponding electronics are mounted separately on a fixed frame and forces are transferred through cables connected to human body by ligth wearable parts (Asbeck et al., 2014; Ding et al., 2017). However, in the third model, we consider additional weight of the active exosuit including the motors, sensors, electronics and the box placed in the backpack besides the passive elements such as the cables, springs, wearable parts. In this model, we have added 4 kg to the mass of the upper body based on the parameters of our recently developed exosuit shown in section Discussion.

## Control Method Description

As the focus of this study is on assistance of healthy subjects to enhance their motion performance by exosuits, balance control is more crucial. For impaired people or elderly people who use crutches, stance and swing sub-functions have more importance. In this section, we explain the VPP and FMCH concepts for balance control and how to benefit from biarticular actuation to employ the FMCH on muscle-reflex model for walking assistance. Thus, first we explain VPP and FMCH concepts based on BTSLIP model that include prismatic (one-segment) leg, and then show how FMCH method can be implemented on musclereflex model that include segmented leg.

### Balance Control, VPP and FMCH

Humans walk with upright upper body as one of the main differences with other animals (Maus et al., 2010). Because of inherently unstable dynamics of the inverted pendulum, which is a popular model of balancing in bipeds, keeping upright posture is more challenging than body posture in multi-legged models. However, using external support, postural stability becomes less critical. Therefore, vertical body alignment, which has a key role in stabilizing human locomotion can be handled in an easy way. This external support can be considered as the core idea of virtual pivot point (VPP) concept (Maus et al., 2010).

As mentioned in section Introduction, it is observed that the ground reaction force (GRF) vectors of the stance leg in humans (and animals) walking intersect at a point on the upper body above CoM (Maus et al., 2010). This intersection point, which is called VPP can translate the balancing from an inverted pendulum model to a virtual pendulum (VP) as a point mass at CoM hanging from the VPP. This concept can be used for posture control of bipedal gaits (see **Figure 2B**). Using the BTSLIP model, a hip torque (τ ) between upper body and the virtual leg (connecting the hip to the CoP) can be found to redirect the GRF going through a determined VPP (Sharbafi and Seyfarth, 2015).

$$\tau = F\_{s}l\frac{r\_{h}\sin\psi + r\_{VP}\sin(\psi - \chi)}{l + r\_{h}\cos\psi + r\_{VP}\cos(\psi - \chi)}\tag{7}$$

where τ , F<sup>s</sup> , l, ψ , rVPPand r<sup>h</sup> are the hip torque, leg force, leg length, hip angle, the distance from CoM to VPP and from VPP to hip joint, respectively. The VPP angle is defined by γ as the angle between body axis and the vector from CoM to VPP as shown in **Figure 2B**. It is important that for posture control using VPP concept we do not need to measure the trunk angle with respect to ground and just the internal angle between upper body and the virtual leg is sufficient. Therefore, a hip spring between upper body and the leg in BTSLIP model can be used to measure this angle. In the next section, we depict how biarticular thigh muscles can be used to measure this angle. The second significant point is about modulation of the mentioned hip spring by the leg force (Fs) in Equation (7). Based on these two concepts the FMCH model was developed (Geyer et al., 2003) which simplifies the posture control as follows.

$$
\pi = k\_h F\_s(\psi\_0 - \psi) \tag{8}
$$

In which k<sup>h</sup> and ψ<sup>0</sup> are the normalized stiffness and the rest angle of the adjustable hip spring, respectively. It was shown that for a range of joint angles variations, used for human normal walking, FMCH presents a very precise approximation of VPP (Sharbafi and Seyfarth, 2015). As it is shown in **Figure 2**, the GRF vector can be decomposed to the perpendicular and axial directions. To control upper body posture using VPP method we can adjust Perpendicular GRF by hip torque, such that GRF direction crosses VPP. However, when we use neuromuscular model of Geyer and Herr (Geyer and Herr, 2010) with segmented legs, the model needs to be extended. For this we define the virtual leg from hip to ankle and the virtual hip torque between the upper body and the virtual leg. To control this virtual hip torque both hip and knee joints should be controlled in coordination. In the following, we demonstrate how biarticular muscles with appropriate lever arm ratios can be employed to provide access to control the virtual hip torque and then, the perpendicular term of the GRF with respect to the virtual leg.

### Segmentation and Biarticular Actuation

One of the important characteristics of the human body, which is beneficial for efficient locomotion, is the leg morphology. The zigzag configuration of the human leg is opposite to birds' leg curvature. Each of these configurations is optimized due to the body properties (e.g., position of CoM with respect to the hip, the ratio between different leg segments' lengths) during million years of evolution. For example, the segment lengths ratio in human leg correspond to the required highly loaded MTC in the human leg, determined by the stress-strain properties of the tendons (Seyfarth et al., 2000).

One of the most important properties of muscular systems in animals (including humans) is using biarticular muscles. The neuromuscular model of Geyer includes two biarticular muscles (HAM and GAS) and five monoarticular muscles GLU, HFL, VAS, TA and SOL as shown in **Figure 2A**. Several advantages of using biarticular muscles beside monoarticular ones were depicted such as coupling of joint movements, velocity contraction, passive energy transfer between and homogenous bending of the adjacent joints (van Ingen Schenau et al., 1990; Seyfarth et al., 2001). In our previous studies, we have shown the significant contributions of biarticular muscles to different locomotion sub-functions control (Sharbafi et al., 2016a). It was shown that with appropriate design of the thigh biarticular actuators in BioBiped3 robot, GRF direction can be controlled with minimum interference to GRF magnitude. In addition, minimizing the influence of GRF direction control on the axial leg function results in a decoupled control of stance and balance locomotor sub-functions. As posture control using the VPP concept is based on GRF direction control, we design our soft exo by a compliant adjustable biarticular actuator parallel to the human HAM muscle.

The ratio between lever arms of the two connected joints is a key design parameter in biarticular actuation. Due to similar size of shank and thigh in human leg, setting hip to knee lever arm ratio to 2:1 minimizes the crosstalk between changes in axial and perpendicular GRF (Sharbafi et al., 2016a). Therefore, here we attach a compliant actuator connecting the upper body to the shank, while the moment arm at hip is twice the moment arm at knee joint (see **Figure 2B**). Then, the FMCH control approach is employed to adjust stiffness in which the actuator is modeled by an adjustable spring with lengthl exo, rest length l exo 0 and normalized stiffnessch. Therefore, the actuator force F exois calculated as follows

$$F^{\rm exo} = c\_{\hbar} F\_{s} \max \left( l\_{0}^{\rm exo} - l^{\rm exo}, 0 \right) = c\_{\hbar} F\_{s} \max \left( \Delta l^{\rm exo}, 0 \right). \tag{9}$$

The max function in this equation shows that the actuator is unidirectional. Therefore, the actuator can pull the segments (like biological muscles or the SEA, shown in **Figure 2**). Considering the hip to knee muscle lever arm ratios of 2:1, the actuator length change will be proportional to the variation in the angle between the upper body and the virtual leg (ψ in **Figure 2B**).

$$
\Delta l^{\rm exo} = r\_{hip}^{\rm exo} \Delta \psi. \tag{10}
$$

in which r exo hip is the actuator lever arm at hip joint and . Derivation of this equation is presented in the Appendix.

As a result, the muscle force is given by

$$F^{\rm exo} = c\_{\hbar} F\_{s} r\_{hip}^{\rm exo} \max \left( \Delta \psi, 0 \right) = k\_{\hbar} F\_{s} m a \chi \left( \psi\_{0} - \psi, 0 \right) \tag{11}$$

This equation is similar to the FMCH control for BTSLIP model, explained in Equation (8). This shows that using biarticular actuator with hip to knee lever arm ratio 2:1, precise implementation of VPP is achieved through FMCH model. Based on this argumentation, we suggest designing an assistive device (e.g., in soft-exo) to generate forces almost parallel to the HAM muscle. This exo generates the following hip (τ exo hip )and knee (τ exo knee) torques

$$
\pi\_{hip}^{exo} = 2\pi\_{knee}^{exo} = r\_{hip}^{exo} F^{exo}.\tag{12}
$$

This method was implemented on LOPES II robot by emulating biarticular actuator using two monoarticular hip and knee actuators (Zhao et al., 2017).

### Optimization

In the proposed control approach (11), k<sup>h</sup> and ψ<sup>0</sup> are the two tuning parameters of the controller. First of all we need to define stable walking. Here we use step-to-fall approach to detect stable gaits. The model is initiated with a specific initial condition adopted from Geyer and Herr (2010) for normal walking without assistance. Then, the stability is verified if the model can take 50 steps. In order to minimize energy consumption in human body (metabolic cost), we define the following cost function (J) for optimization.

$$J = \frac{1}{M} \sum\_{i=N-M+1}^{N} \frac{E\_{mct}^{total}}{d}. \tag{13}$$

This metric shows the average of consumed energy for traveling 1 meter in the last M steps. In our simulations, Mand N are set to 30 and 50 determining the mean value of the metabolic cost for the last 30 steps of 50 steps defined for a stable solution. Here, we considered 20 steps to pass the transient behavior and to reach the steady state. In this equation, dis the travelled distance and E total met denotes total metabolic cost of human walking (E total met <sup>=</sup> P 7 i=1 E i met), consumed by the seven different muscle groups defined in the neuromuscular model (Geyer and Herr, 2010; see **Figure 2**). For each muscle the metabolic cost is calculated as follows

$$E\_{mct} = \int\_{t1}^{t2} P\_{mct} \tag{14}$$

where Pmet(t) is instantaneous metabolic power (Krishnaswamy et al., 2011). This value is computed for each muscle and their summation gives the total metabolic cost of the whole body motion (E total met ). At any time t, Pmet(t) is obtained as follows

$$P\_{met}(t) = p(\nu\_{\rm CE}/\nu\_{\rm max}) \times A(t) \times |F\_{\rm max} \times \nu\_{\rm max}| \tag{15}$$

in which p(x) is a function approximated based on empirical data (Alexander, 1997) by

$$p(\mathbf{x}) = \begin{cases} 0.01 - 0.11 \text{ ( $\mathbf{x}$ )} + 0.06 \exp \text{ ( $23\mathbf{x}$ )}, & \mathbf{x} < \mathbf{0} \\\ 0.23 - 0.16 \exp \text{ ( $-8\mathbf{x}$ )} & \mathbf{,} \mathbf{x} \ge \mathbf{0} \end{cases} \tag{16}$$

For this, we implemented an optimization procedure in the neuromuscular model (presented in section Human Walking Model). To find the optimal values of k<sup>h</sup> and ψ<sup>0</sup> that minimize the normalized metabolic cost [defined by (13], we searched in definable ranges of these parameters. These ranges are 0 to 10 for the normalized stiffness <sup>k</sup><sup>h</sup> and <sup>−</sup>25 to 25◦ for the rest angleψ0. Out of these ranges is not obtainable due to limitations in actuation mechanism. Here, we assume that the body control parameters including the reflex gains are fixed as presented in Geyer and Herr (2010). Hence, addition of the biarticular softexo can only affect the muscle force generation though changing the reflex signal (e.g., muscle forces). It is clear that optimizing the parameters of both soft-exo and reflex gains will result in higher reduction in metabolic cost as the fixed gains are one parameter set in the gain parameter space that may have other minima with lower metabolic costs. Hence, our method can result in higher assistance if we consider human adaptation to the assistive device.

## RESULTS

In this section, we explain the simulation results of applying the FMCH controller on a biarticular thigh actuator to assist human walking at normal walking speed (1.3 m/s). We compare muscle forces, activations and metabolic costs in the different 3 models. The first one is for human walking without assistance; the second one is with assistance in an ideal case without addition of the exoskeleton mass. This demonstrates the quality of the proposed method regardless the implementation issues. Finally, a 4 kg package is considered on the upper body to contain two actuators, electronics and processor. The force is transferred through a cable drive mechanism similar to the soft-exos (Ding et al., 2017). The kinematic behavior of the gait and the motion speed are not significantly different (less than 5%) in the three models (not shown).

The optimal control parameters for the stiffness and the rest angle of the exo based on Equation (11) are k<sup>h</sup> = 5.65 and ψ<sup>0</sup> = −7 ◦ . These numbers show that during the stance phase, the exosuit starts to pull after touchdown until ψ reaches −7 ◦ which is slightly after mid-stance. In the swing phase, the exo does not produce any force because the leg force is zero.

In the following the muscle force, activation and metabolic power are compared to demonstrate the advantages of the proposed method for design and control of the exoskeleton. Similar to previously explained approach for the cost function (section Optimization), here we consider the last 30 steps and the mean and standard deviation are shown in the following figures. First, in **Figures 4**, **5** we show the values for the Ham muscle, as it is parallel to the actuator. Then the mean values for one stride are shown for different muscles. In sectiion Energy Economy, the total metabolic power is used to realize the effectiveness of the proposed technique. Finally, the contribution of the exosuit design on posture control is analyzed using VPP.

## Effects on HAM Muscle

In this section, we show the activation and the force produced by HAM muscle in the three different cases. **Figure 4**. Illustrates the mean and standard deviation of the HAM muscle force in 30 steps, without assistance compared with these values with assistance. As expected, the force modulated compliant hip controller produces a significant part of the required force of HAM during stance phase. As a result, considerable decrease in HAM force is observed in the first 30% of the gait cycle. In spite of zero contribution of the exo in the swing phase, forces differ due to the effects from the stance phase and also the second leg, which is in stance phase. In the original model (without assistance) the variance among 30 steps is close to zero. Although, addition of the exosuit changes the force patterns slightly from step to step, the variance is still negligible.

**Figure 5** shows the activation signals for the HAM muscle. The activation patterns are similar except in the first 30% of the gait. In this period, the exosuit generates most of the required efforts resulting in activation reduction in HAM muscle. These results are in line with the observations in **Figure 4** for developed force. Addition of the exo mass does not have significant effects on the HAM muscle activation and force. This means that most of the required force in HAM to compensate the exo mass is provided by the actuator. Therefore, the person who wears the

exosuit will not suffer from the additional mass of the robot. In addition, variance in the activation signal increases by adding the exo, which is similar to the previous observations in **Figure 4**. The only difference appears in the beginning of the second step when the variance is larger in contrast to the force patterns. This might relate to pushoff in which HAM muscle activation is influenced by variations in other muscles at each step, but this is not significantly reflected in muscle force. Generally speaking, reduction in HAM muscle force and activation in the first 30% of the stride can be considered as the main effect of exosuit contribution to the stance leg assistance and even without adaptation of the reflex gains, a periodic motion (with low variance) can be obtained after addition of the assistive device. This second outcome might relate to the bioinspired control principle employed in design and control of our exosuit.

## Effects on the Whole Leg Neuromuscular Control

Based on the reflex control in the neuromuscular model, assisting hip biarticular muscles influences activation and force generation in the other muscles. In this section, we analyze these effects using the grand mean as the average of the mean values of the last 30 steps (gait cycles). Accordingly, **Figures 6**, **7** depict the grand mean of force (FGM) and activation signals (AGM) for different muscles during 30 steps. In addition, the standard deviations among different steps are shown in the same figures. The grand mean and standard deviation of the muscle forces are calculated as follows.

$$F\_{GM} = \frac{1}{30} \sum\_{i=21}^{50} \overbrace{\frac{1}{T\_i} \int\_0^{T\_i} F\_i(t)dt}^{\overline{F\_i}} \tag{17}$$

$$F\_{\text{Std}} = \sqrt{\frac{1}{\text{30}} \sum\_{i=21}^{50} \left( \overline{F\_i} - F\_{\text{GM}} \right)^2} \tag{18}$$

in which subindex i denotes the i th step. Similar equations are utilized to calculate the grand mean and standard deviation for activation signal of each muscle.

As expected, both force and activation are reduced in hip extensor muscles (HAM and GLU). Activation reduction in the other three muscles (GAS, HFL, and TA) does not significantly change the muscle force. This results in lower metabolic cost in these muscles while generating similar forces. Therefore, assisting the HAM muscle can reduce energy consumption in these muscles without significantly changing their developed forces. Although these muscles are more responsible for balancing, they also contribute to axial leg function. As a result, increases in monoarticular knee and ankle extensors (VAS and SOL) are observed. The additional mass of the exo-suit should be also handled by growth in SOL and VAS forces. As shown in these figures, the results are quite consistent for all muscles, as the standard deviations during 30 steps are very small. Adapting reflex gains after adding the exo (not performed in this study) may result in even smaller variance similar to the first case (blue bar) without assistance.

## Energy Economy

To investigate the effect of variations in activation and force of different muscles on the energy consumption, the normalized metabolic cost [calculated by (13)] of individual muscles are compared in **Figure 8**. In comparison between unassisted walking and the ideal assisted model (without additional mass), metabolic effort is just increased in one (VAS) muscle. In addition, the reduction in energy consumption of HAM, HFL, and GLU dominate the increment in VAS. Similar to the force and activation behavior, additional mass is mainly reflected in the growth of energy consumption in SOL and VAS. Still, decreased metabolic cost in other muscles is significantly higher than extra energy, required to support the additional mass.

To investigate the level of assistance at different moments of the gait cycle, the total metabolic power during a complete stride is drawn in **Figure 9**. This graph shows the mean and the total metabolic power of both legs, in a complete stride among 30 last steps. Considerable reduction in metabolic power is observed in the first 20% of the gait cycle meaning that the exosuit supports walking until shortly before the midstance. This is coincident with the time slot that the HAM muscle contributes the most. As the exo actuator is parallel to the HAM muscle, its contribution is low (or even zero) after midstance (more precisely when ψ < ψ<sup>0</sup> = −7 ◦ ). It is also observed that the three-hump pattern of the metabolic power in the unassisted case is changed to a single-hump resulting in significant reduction in metabolic power. Although an increase in metabolic power consumption is observed around midstance, it is compensated afterward (about 33% of the gait cycle). Roughly speaking, the total energy from midstance to touch down of the next leg is almost constant with and without assistance. Hence, after the exo contribution in first 20% of the gait cycle the metabolic power does not change significantly in the ideal model of assisted case, compared to the unassisted case while addition of the exo mass increases the metabolic power in this period. Interestingly, the human power consumption is barely affected by additional mass in the first 20% of the gait cycle (before midstance). It means that the exo compensate the required energy to support the additional mass. However, this cannot continue until the end of the gait cycle, because of the reduction in assistive device contribution. Therefore, the total reduction in metabolic cost will be lowered. Similar pattern is observed in the second step and the total metabolic energy is reduced with assistance.

Comparing the total energy consumption per stride, demonstrate advantages of walking assistance. As a result, the reductions in the model with and without exo mass are 6 and 12%, respectively. These numbers are 48 and 45% for the HAM muscle. After assistance, the motion performance is kept, except negligible reduction in the walking speed. Reduction in total metabolic cost using an exosuit with just one actuator for each leg that supports the body only in stance phase of the corresponding leg is considerable.

In order to validate the effectiveness of the proposed method, in **Figure 10**, we show the power consumption in the exosuit during one stride. Here, we have assumed that the FMCH is implemented by a rigid actuator, which mimics the adjustable compliance. Therefore, this is the maximum power required for

implementing the control concept. Obviously, benefitting from compliant structure (e.g., by designing a variable impedance actuator with parallel compliance) can significantly reduce the required power of the actuator. Nevertheless, it can be shown that there is a big advantage in the proposed design and control. The results shown in **Figure 10** support the previous observations in **Figure 9**. Note that in the proposed exo design and control, the assistive device contributes in half of the gait cycle in which the leg is in stance phase and the so called biarticular compliant element is stretched. Interestingly, the variance in power generation among 30 steps is reduced by adding the exo mass.

## Balance Control

As the core control concept of FMCH is the VPP model that is introduced regarding posture control, here we demonstrate the effects of walking assistance on balancing. In **Figure 11**, the GRF (ground reaction forces) are demonstrated in the coordinate frame centered at CoM (center of mass) with vertical coordinate aligned with upper body orientation. For regular walking without assistance, the VPP exists about 40 cm above CoM which is in line with finding in human walking at moderate speeds (Maus et al., 2010). By assistance of the exo, the GRF vectors are more focused which support balance control through VPP concept. In addition, the VPP becomes closer to CoM [smaller rVPP in Equation (7)] while the VPP angle (γ ) increases. This means more less oscillations in upper body and more performant posture control. This behavior is slightly deformed by adding the exo mass. In general, addition of the exosuit clearly improves posture control as expected.

## DISCUSSION

The bioinspired design and control of a soft exoskeleton was proposed in this paper. Concentrating on balancing as

FIGURE 11 | Ground reaction forces in body coordinate frame (dashed black lines) and the VPP (red circle) with and without assistance. The body coordinate frame is centered at CoM (green circle) with vertical axis aligned with the upper body orientation.

one locomotor sub-function, two basic design principles were employed: (i) As a control design principle, leg force is used as a sensory feedback signal to adjust hip compliance (FMCH) (Sharbafi and Seyfarth, 2015). This approach is motivated by the VPP concept for posture control (Maus et al., 2010). (ii) A second design principle is employing biarticular actuators to simplify GRF direction control and consequently, posture control. Using this actuator-skeletal design principle, we can benefit from synchronization between hip and knee joint and increasing efficiency by transferring energy between joints. Using these bioinspired principles we can improve interaction between human and the assistive device. Instead of regular position, force or impedance control of the end effector, in our approach we adjust the stiffness of a biarticular spring based on the leg force. Therefore, there is no need to track desired impedance with the controller. Instead, a sensory feedback circuit can be used to measure the leg force and modulate the hip actuator stiffness. The control loop will be closed at a higher level when this modulation finally influences the leg force. Hence, we do not use a desired signal for tracking. This is similar to reflex-based control in human locomotion that results in stable gaits without direct control of the target states (Geyer et al., 2003; Geyer and Herr, 2010).

In Zhao et al. (2017), we have implemented the FMCH control approach on LOPES II exoskeleton through separate control of the knee and hip torques using Equation (12). Since there was no physical spring in that rigid exoskeleton, the joint angles are measured to emulate virtual springs. With this approach, the activation of different muscles was decreased and more than 10% reduction in metabolic cost was achieved compared to transparent mode control. As there was no biarticular muscle in the LOPES II robot, we could not benefit from the other properties of biarticular actuation such as transferring energy and synchronization between the adjacent joints. Therefore, using biarticular thigh actuators in a soft suit may be beneficial to achieve higher performance with the assistive device using variable impedance actuators. One possible type of actuator for implementing the proposed method is a pneumatic air muscle (PAM).

In LOPES experiments, the results are compared with transparent mode in which the control target is zeroing the interaction force between the robot and human body. Therefore, there is no additional mass in the assisted mode with FMCHbased control compared to the transparent mode. In addition, the robot can generate force in both directions and not just pull as in exosuits. Hence, 13% reduction in metabolic cost using only one of the thigh biarticular muscles in our simulations is an achievement compared to 10% reduction with LOPES II via bidirectional actuators. This demonstrates the potential advantages of employing soft-suits and biarticular compliant actuators (e.g., PAMs). With biarticular actuators it is possible to avoid internal losses by transferring energy between two joints instead of positive work at one joint and negative at the adjacent joint. Furthermore, an adaptation of human motor control to the robot can even increase the efficiency of the proposed approach.

In addition to more aligned force direction and lower inertia of the soft suits, their lighter weights are of advantage compared to rigid exoskeletons (Panizzolo et al., 2016). In this paper, we showed that an additional weight of the exosuit (about 5% of the body weight) close to body CoM does not have substantial effects on assistance. It is of utmost significance that these wearable robots can appropriately interact with the body (Ding et al., 2017). In contrast to the exoskeletons with actuators paired with the biological joints, softsuits are merely capable of generating tensile forces, that prevents resistance against natural walking, and hence provides comfort and reduced metabolism (Asbeck et al., 2014). Based on our control method, the mechanism can be simply adjusted to individuals of different body constitutions and motor control properties. In Ding et al. (2018), Bayesian optimization was used to identify the peak and offset timing of hip extension assistance that minimizes the energy expenditure of walking with a wearable device. Similar learning based methods can be easily applied to our method to find the variable spring parameters (k<sup>h</sup> and ψ0) for each subject. It means that instead of time-based optimization, reflex-based control is employed and minimal parameter space is sufficient for finding the optimal controller. Using PAMs with adjustable compliance for actuation in the soft suit, the only required sensory information for control is the leg force. Therefore, the proposed mechanism for the soft suit can be implemented using minimal sensory measurements.

One drawback of the neuromuscular control model used in this study is the lack of rectus femoris. This muscle was neglected due to its minor contribution in normal walking. In order to investigate the idea of assisting human locomotion based on the proposed approach we have implemented the same mechanism using an OpenSim model of human walking. We achieved similar results in reduction of muscle activation and total metabolic costs with this model. One main issue with OpenSim models was that changing the structure of the model requires an adaptation of the control. This feature is provided in the reflex-based neuromuscular models, but in the OpenSim model, it is missing. This means that better results could be achieved by further tuning the model. Finally, we have built an exosuit (**Figure 12**), which works based on the biarticular actuation of thigh segment. In future we will implement the proposed controller on this system. The hardware properties of the exosuit (with mass) used in both models (Geyer model and OpenSim) are borrowed from this recently manufactured wearable robot. In this design, we focused on thigh biarticular actuators to validate the FMCHbased control methods on a soft-exo. This design principle makes this assistive device different from the other previously developed exosuits (Asbeck et al., 2014; Ding et al., 2017).

The bioinspired balance control based on the VPP concept is supporting efficient locomotion with reduced CoM accelerations and decelerations during the gait cycle. This control concept of using leg force feedback for control of the muscles could be extended for the ankle joint. However, due to the asymmetric function of the human foot during locomotion, the function of the individual ankle muscles on supporting body against gravity and in maintaining balance is still not well understood. Additional research will be required to better understand the interplay between hip and ankle strategies for stable locomotion while keeping the body aligned upright.

## AUTHOR CONTRIBUTIONS

MS is the main and corresponding author of the article. He was responsible for the conception and design of simulations and exosuit design, analysis and interpretation of results and writing of the manuscript. HB implemented the methods on neuromuscular model, performed simulation studies and provided the results. He has also a contribution in writing and revising the paper and discussions. MI was involved in discussions, writing the paper, design and manufacturing the exosuit. AS was involved in conception, description of the results, discussions, and revising the paper .

## FUNDING

This research is supported partially by INSF under grant No. 95849456 and partially by the German Research Foundation (DFG) under grants No. AH307/2-1 and SE1042/29-1.

## REFERENCES


## SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnbot. 2018.00039/full#supplementary-material


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Sharbafi, Barazesh, Iranikhah and Seyfarth. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Generation of Human-Like Movement from Symbolized Information

Shotaro Okajima1,2 \*, Maxime Tournier <sup>2</sup> , Fady S. Alnajjar 2,3, Mitsuhiro Hayashibe2,4 , Yasuhisa Hasegawa<sup>1</sup> and Shingo Shimoda<sup>2</sup>

<sup>1</sup> Department of Mechanical Science and Engineering, Graduate School of Engineering, Nagoya University, Nagoya, Japan, 2 Intelligent Behavior Control Unit (BTCC), Brain Science Institute (BSI), RIKEN, Nagoya, Japan, <sup>3</sup> College of IT, United Arab Emirates University, Al-Ain, United Arab Emirates, <sup>4</sup> Department of Robotics, Tohoku University, Sendai, Japan

An important function missing from current robotic systems is a human-like method for creating behavior from symbolized information. This function could be used to assess the extent to which robotic behavior is human-like because it distinguishes human motion from that of human-made machines created using currently available techniques. The purpose of this research is to clarify the mechanisms that generate automatic motor commands to achieve symbolized behavior. We design a controller with a learning method called tacit learning, which considers system–environment interactions, and a transfer method called mechanical resonance mode, which transfers the control signals into a mechanical resonance mode space (MRM-space). We conduct simulations and experiments that involve standing balance control against disturbances with a two-degree-of-freedom inverted pendulum and bipedal walking control with humanoid robots. In the simulations and experiments on standing balance control, the pendulum can become upright after a disturbance by adjusting a few signals in MRM-space with tacit learning. In the simulations and experiments on bipedal walking control, the robots realize a wide variety of walking by manually adjusting a few signals in MRM-space. The results show that transferring the signals to an appropriate control space is the key process for reducing the complexity of the signals from the environment and achieving diverse behavior.

#### Edited by:

Katja Mombaur, Universität Heidelberg, Germany

#### Reviewed by:

Eiji Uchibe, Advanced Telecommunications Research Institute International (ATR), Japan Manuel Giuliani, University of the West of England, United Kingdom

#### \*Correspondence:

Shotaro Okajima shotaro.okajima@riken.jp

Received: 01 February 2018 Accepted: 27 June 2018 Published: 17 July 2018

#### Citation:

Okajima S, Tournier M, Alnajjar FS, Hayashibe M, Hasegawa Y and Shimoda S (2018) Generation of Human-Like Movement from Symbolized Information. Front. Neurorobot. 12:43. doi: 10.3389/fnbot.2018.00043 Keywords: mechanical resonance mode, tacit learning, control structure, symbolized information, human-like movement

## 1. INTRODUCTION

Can robots be good neighbors? Despite much effort by many researchers to make robots be good partners, robotic systems remain limited to being merely useful tools in factories and houses<sup>1</sup> . This is the case even though mobility control for rough terrain<sup>2</sup> and artificial intelligence for understanding human speech3,4and behavior have improved drastically in recent years. What is the critical difference that distinguishes people from human-made machines?

<sup>1</sup>Roomba. iRobot Corporation. Available online at: https://www.irobot.com/

<sup>2</sup>Atlas. Boston Dynamics. Available online at: https://www.bostondynamics.com/

<sup>3</sup>Amazon Echo. Amazon.com, Inc. Available online at: https://www.amazon.com/dp/B00X4WHP5E

<sup>4</sup>Google Home. Google LLC. Available online at: https://store.google.com/product/google\_home

We think that an important function that is currently missing from robotic systems is a way to create behavior from symbolized information. For instance, when walking, we deliberately attend to symbolized behavioral purposes such as "walk faster" and "turn right" or more symbolic forms such as "go to the station." The detailed control signals that create such motion, such as joint trajectories and muscle activations, are then generated automatically.

It is said that these automatic control signals are created by the activities of local neural systems including the cerebellum and spinal cord. In our daily lives, we attend only to symbolized behavioral purposes that are highly specialized. The appropriate behavior and detailed control signals that achieve those purposes are then chosen according to the prevailing situation and the surrounding environment. If we could share such symbolized behavioral purposes with robots, and if the robots and we could create the appropriate behavior independently according to not only the surroundings but also the features of our respective functions, then we would feel that the robots are really our partners. Therefore, generating behavior from symbolized purpose could be an important way to assess the extent to which robots could be our partners with human-like behavior.

In this paper, we discuss the process of creating behavior from symbolized behavioral purpose, focusing on creating motor commands from simple behavioral targets through body– environment interactions. There have been various attempts to clarify the mechanisms that generate automatic motor commands from symbolized purposes, an important approach being a physiological one. Recently, Takei et al. (2017) reported the existence of neurons in the spinal cords of monkeys that commonly activate in association with various hand actions, suggesting that a small control signal, with dimensionality lower than the number of muscles, can encode complicated hand motion.

Model-based approaches provide the conceptual basis for the aforementioned physiological approach. A bow-tie structure (Csete and Doyle, 2004; Zhao et al., 2006) has been proposed to represent a biological control system whereby information acquired from the environment is gradually symbolized to reduce its dimensions, while control signals are created from this symbolized information to increase their dimension. The notions of muscle synergy (Bernstein, 1967; Tresch et al., 1999; d'Avella et al., 2003; Chvatal et al., 2011; Alnajjar et al., 2013; Barroso et al., 2014; Gonzalez-Vargas et al., 2015; Garcia et al., 2018; Kogami et al., 2018) and joint synergy (Schenkman et al., 1990; Latash, 2000; Yamasaki and Shimoda, 2016) that represent the output side of this bow-tie structure are prominent examples of estimating lower-dimensional signals from observable signals such as electromyographic signals. The sensor synergy representing the input side of the bow-tie structure has been discussed regarding estimating sensor signals from the environment (Ting, 2007; Latash, 2008; Alnajjar et al., 2015).

Another important approach to clarifying the mechanisms that generate automatic motor commands is the development of artificial controllers that have the same features as those of biological controllers. The autoencoders discussed in artificial intelligence (Hinton and Salakhutdinov, 2006; Hosseini-Asl et al., 2016) share the same idea as the bow-tie structure. Recently, there have been various discussions about using autoencoders to control robots (Noda et al., 2014; Finn et al., 2016; van Hoof et al., 2016; Kondo and Takahashi, 2017). KullbackLeibler control (Todorov, 2009) is an interesting task-dependent approach to control robot (Uchibe and Doya, 2014; Matsubara et al., 2015) with combination of control policies.

These computational approaches clarified that small control signals, with dimensionality lower than the number of motors, can represent behavioral features, suggesting that lower-dimensional control signals play the role of symbolized behavioral purposes. Shimoda et al. proposed a bio-mimetic behavior-adaptation architecture known as tacit learning (Shimoda and Kimura, 2010; Shimoda et al., 2013; Hayashibe and Shimoda, 2014) and have used it to generate bipedal walking from a roughly defined walking gait (Shimoda et al., 2013), to control the wrist joint of a forearm prosthesis in response to the wearer's shoulder movements (Oyama et al., 2016), and to control a lower-limb exoskeleton robot in response to the wearer's movements (Shimoda et al., 2015). Through experiments on this tacit learning adaptation, they established that two types of adaptation process could work simultaneously to adapt the behavior to an unorganized environment. One of these processes is selecting appropriate behavior and the other is adapting reactive behavior to unpredictable disturbances and small changes in body parameters and environment without changing the behavioral purpose.

Even though it has been established that it is important for these two processes to operate in parallel, the conditions of the controllers needed to realize such adaptation remain under discussion. Herein, we advance this discussion by using an artificial controller that can adapt the motor commands to real-time changes in the symbolized purpose, and we clarify the conditions for adapting in real time to both the environment and the symbolized purpose. We begin in section 2 by designing a controller with tacit learning and that transfers the control signals into a different control space know as mechanical resonance mode space (MRM-space). In MRMspace, the signals are used to control the mechanical resonance modes of the robot. This makes it easy to understand how the robot behavior changes when the control signal is changed in MRM-space. In sections 3 and 4, we propose an adaptation method in MRM-space using a two-degree-of-freedom (2DoF) inverted pendulum, 27DoF humanoid robot, and the NAO humanoid robot<sup>5</sup> , respectively. We show through simulation and experiment that this controller can adapt the motion to the environment. In section 5, we discuss the importance of body mechanisms in the process of simultaneous adaptation and how that process can be used to evaluate the human-like motion of a robot.

<sup>5</sup>NAO. SoftBank Robotics. Available online at: https://www.ald.softbankrobotics. com/en/robots/nao

## 2. METHODS USED TO DESIGN A CONTROL STRUCTURE

Transfer to MRM-space and behavior adaptation by tacit learning are the key analytical methods of the present study. Because both methods are discussed in detail elsewhere (mechanical resonance mode Kry et al., 2009; tacit learning Shimoda and Kimura, 2010; Shimoda et al., 2013; Hayashibe and Shimoda, 2014), we explain their essential points only briefly herein.

## 2.1. Mechanical Resonance Mode

A mechanical resonance mode is defined by the position and the condition of the robot joints. For instance, a 2DoF inverted pendulum has two mechanical resonance modes as shown in **Figures 1A,B**. A mechanical resonance mode is characterized mathematically by mode vectors and eigenvalues. Writing the equation of motion of a 2DoF inverted pendulum as

$$M\ddot{\theta} + \mathbf{K}\theta = \mathbf{0} \tag{1}$$

$$
\Leftrightarrow \quad \ddot{\theta} = -\mathbf{M}^{-1}\mathbf{K}\theta,\tag{2}
$$

where θ ∈ **R** 2 implies the angles of the joints, the mode vectors and eigenvalues are calculated by a singular-value decomposition (SVD) of **M**−1**K**:

$$\mathbf{M}^{-1}\mathbf{K} \implies \begin{cases} \nu\_1, \nu\_2 & \text{(mode vectors)}\\ \lambda\_1, \lambda\_2 & \text{(eigenvalues)} \end{cases},\tag{3}$$

where **M** ∈ **R** 2×2 is the inertia matrix of the pendulum linearized around θ = **0** and **K** ∈ **R** 2×2 is a stiffness matrix that has the spring coefficients of the joints on its main diagonal. The first mode (**v**1) corresponds to the smallest eigenvalue (λ1) and the second mode (**v**2) corresponds to the next-largest eigenvalue (λ2). The mode vector represents the shape of the pendulum oscillation.

The state variable θ of the pendulum can be represented as a superposition of the mode vectors as follows:

$$\boldsymbol{\theta} = \boldsymbol{\nu}\_1 \boldsymbol{\nu}\_1 + \boldsymbol{\nu}\_2 \boldsymbol{\nu}\_2 = \begin{bmatrix} \boldsymbol{\nu}\_1 \ \boldsymbol{\nu}\_2 \end{bmatrix} \begin{bmatrix} \boldsymbol{\nu}\_1 \ \boldsymbol{\nu}\_2 \end{bmatrix}^T$$

$$\boldsymbol{\varepsilon} \cdot \boldsymbol{\theta} = T \boldsymbol{\nu} \ (\boldsymbol{\Leftrightarrow} \ \boldsymbol{\nu} = T^{-1} \boldsymbol{\theta}),\tag{4}$$

where w1,w<sup>2</sup> represent the weights of each mode vector and **T** can be defined as a transfer matrix. We can define the weights of the mode vectors as symbolized state variables in MRM-space.

The adjustment of the symbolized state variables is reflected in the movement of individual joints by the transfer matrix. This is much like the top-down process in people, namely changing one's behavior by means of symbolized information without having to attend to the actions of individual joints.

## 2.2. Tacit Learning

Tacit learning is an adaptive learning method inspired by two features of living beings. First, living beings can perform adaptive behavior globally even though control is realized by only a summation of local nerve-cell firings. Second, adaptive learning

and behavioral control are calculated in parallel; this is unlike machine learning, whose calculation is divided into a learning phase and an action phase.

To apply these features to artificial controller, action targets and the concept of "reflex" are used in tacit learning. The reflex plays a role in directing the movement of the controlled system toward a state in situations in which the system does not receive many environmental stimuli from a global perspective. By enhancing the reflex by accumulating reflex commands, the system can acquire a state autonomously through system– environment interactions, where there are fewer environmental stimuli without having to distinguish between the learning phase and the action phase.

Other learning methods use behavioral functions or teaching signals to adjust the controlled system behavior and achieve adaptive behavior in a top-down manner. In that sense, tacit learning can be defined as a bottom-up learning process, adjusting the behavior through system–environment interactions. However, it can control the system to achieve adaptive behavior from a global perspective. Herein, we use tacit learning to develop a bio-mimetic adaptation process.

## 3. STANDING BALANCE CONTROL WITH 2DOF INVERTED PENDULUM

In this section, we introduce the controller with tacit learning in MRM-space and apply it to standing balance control of a 2DoF inverted pendulum. We show that the tacit learning controller can maintain balance against larger disturbances than the case without learning.

## 3.1. Model of 2DoF Inverted Pendulum

**Figure 2** shows the 2DoF inverted pendulum model used in this simulation. Its equation of motion is

$$\mathcal{M}(\theta)\ddot{\theta} + \mathcal{g}(\theta, \dot{\theta}) = \mathfrak{r},\tag{5}$$

where θ is a 2 × 1 vector consisting of the joint angles, τ is a 2 × 1 torque vector that affects each joint, and **M**(θ) is a 2 × 2 inertia matrix given by

$$\begin{aligned} \mathcal{M}(\theta) \\ = \begin{bmatrix} I\_1 + m\_1 a\_1^2 + l\_1^2 m\_2 + \eta + 2\xi \cos \theta\_2 & \eta + \xi \cos \theta\_2 \\ \eta + \xi \cos \theta\_2 & \eta \end{bmatrix}, \end{aligned} \tag{6}$$

where a<sup>1</sup> = l1/2, a<sup>2</sup> = l2/2, η = I<sup>2</sup> + m2a 2 2 , ξ = l1m2a2, and **g**(θ, ˙θ) is

$$\begin{aligned} & \mathbf{g}(\boldsymbol{\theta}, \dot{\boldsymbol{\theta}}) \\ &= \begin{bmatrix} -\xi (2\dot{\boldsymbol{\theta}}\_1 + \dot{\boldsymbol{\theta}}\_2)\dot{\boldsymbol{\theta}}\_2 \sin \theta\_2 - \boldsymbol{g}\_1 \sin \theta\_1 - \boldsymbol{g}\_2 \sin(\theta\_1 + \theta\_2) \\\ \xi \dot{\boldsymbol{\theta}}\_1^2 \sin \theta\_2 - \boldsymbol{g}\_2 \sin(\theta\_1 + \theta\_2) \end{bmatrix}, \end{aligned} \tag{7}$$

where g<sup>1</sup> = (m1a<sup>1</sup> + m2l1)g, g<sup>2</sup> = m2a2g, and g = 9.81 m/s<sup>2</sup> .

## 3.2. Standing Balance Control Structure

**Figure 3** shows a standing balance controller designed by using the transfer matrix **T** described in 4 and tacit learning. Terms θ<sup>i</sup> and τ<sup>i</sup> are the angle and torque, respectively, of joint i. The torque vector τ for each joint is

$$\pi = TAT^{-1} \Delta \theta + TBT^{-1} \Delta \dot{\theta} + T\xi\_m + \xi,\tag{8}$$

$$\mathbf{r} = \begin{bmatrix} \ v\_1 \ v\_2 \end{bmatrix}^T. \tag{9}$$

<sup>1</sup><sup>θ</sup> and <sup>1</sup> ˙<sup>θ</sup> are

$$
\Delta\theta = \left[\theta\_{r\text{cf}} - \theta\right], \Delta\dot{\theta} = -\dot{\theta}, \tag{10}
$$

FIGURE 2 | Model of 2DoF inverted pendulum: (A) pendulum model in simulator; (B) definitions of joint angles and system parameters. The pendulum sits on a board that is moved horizontally by a force f [N], thereby imparting disturbances to the pendulum.

where <sup>θ</sup> and ˙<sup>θ</sup> are state variables:

$$\boldsymbol{\theta} = \begin{bmatrix} \theta\_1 \ \theta\_2 \end{bmatrix}^T, \ \dot{\boldsymbol{\theta}} = \begin{bmatrix} \theta\_1 \ \theta\_2 \end{bmatrix}^T. \tag{11}$$

θref is a reference for each joint:

$$\boldsymbol{\theta}\_{r\!\!rf} = \begin{bmatrix} \boldsymbol{\theta}\_{r\!\!rf1} & \boldsymbol{\theta}\_{r\!\!rf2} \end{bmatrix}^T. \tag{12}$$

Terms **A** and **B** are diagonal matrices:

$$\mathcal{A} = \begin{bmatrix} k\_{p1} & 0 \\ 0 & k\_{p2} \end{bmatrix}, \ \mathcal{B} = \begin{bmatrix} k\_{d1} & 0 \\ 0 & k\_{d2} \end{bmatrix}, \tag{13}$$

where kp<sup>1</sup> and kp<sup>2</sup> are proportional (P) gains of the proportionalderivative (PD) controller in MRM-space, kd<sup>1</sup> and kd<sup>2</sup> are derivative gains of the PD controller in MRM-space, and **T** −1 is the transfer matrix from joint space to MRM-space.

Term ζ is a vector that consists of the integration of τ as follows:

$$\mathfrak{F} = \begin{bmatrix} \zeta\_1\\ \zeta\_2 \end{bmatrix} = \begin{bmatrix} k\_1 & 0\\ 0 & k\_2 \end{bmatrix} \begin{bmatrix} \int \mathfrak{r}\_1 dt\\ \int \mathfrak{r}\_2 dt \end{bmatrix},\tag{14}$$

where k<sup>1</sup> and k<sup>2</sup> are the coefficients of the integrators that accumulate the joint torques and output the integrated values. These accumulations correspond to tacit learning, and these integrators adjust the individual joint torques and work to keep the pendulum upright after disturbance through pendulum– environment interaction, as in Shimoda et al. (2013).

Term ζ <sup>m</sup> a vector that consists of the integration of ζ as follows:

$$\mathfrak{F}\_m = \begin{bmatrix} \zeta\_{m1} \\ \zeta\_{m2} \end{bmatrix} = \begin{bmatrix} 0 & 0 \\ k\_{m2} & 0 \end{bmatrix} \begin{bmatrix} \int \zeta\_1 dt \\ 0 \end{bmatrix},\tag{15}$$

where km<sup>2</sup> is the coefficient of the integrator in MRM-space that accumulates ζ<sup>1</sup> and outputs the integrated values. km<sup>2</sup> can change the level of learning in standing balance control. km<sup>2</sup> = 0 is defined as "without learning," and km<sup>2</sup> > 0 is defined as "with learning." ζ <sup>m</sup> adjusts the movement of the second mode, which we selected based on visual inspection of the movement of all modes. Because any disturbance has a pronounced effect on joint 1, the torque of the second mode is adjusted based on the torque of joint 1.

The whole system can be expressed by combining (5) and (8) as follows:

$$\begin{split} M(\theta)\ddot{\theta} + \mathbf{g}(\theta,\dot{\theta}) &= TAT^{-1}\Delta\theta + TBT^{-1}\Delta\dot{\theta} \\ &+ T\xi\_m + \xi. \end{split} \tag{16}$$

## 3.3. Standing Balance Control Simulation and Results

The two mode vectors **v**1, **v**<sup>2</sup> of the pendulum defined in **Figure 2** are given as

$$\mathbf{v}\_1 = \begin{bmatrix} -0.9\\ -0.3 \end{bmatrix}, \quad \mathbf{v}\_2 = \begin{bmatrix} 0.3\\ -0.9 \end{bmatrix}, \tag{17}$$

As shown in **Figures 3A,B**, the first mode **v**<sup>1</sup> represents samephase posture and the second mode **v**<sup>2</sup> represents anti-phase posture. The transfer matrix **T** is

$$T = \begin{bmatrix} -0.9 & 0.3 \\ -0.3 & -0.9 \end{bmatrix} . \tag{18}$$

Standing balance control simulations are conducted as follows.


We conduct simulations with each of f = 170, · · · , 210 N. The gains are kp<sup>1</sup> = 22.0, kd<sup>1</sup> = 21.0, kp<sup>2</sup> = 22.0, kd<sup>2</sup> = 21.0, k<sup>1</sup> = 1.0 × 10−<sup>3</sup> , and <sup>k</sup><sup>2</sup> <sup>=</sup> 1.0 <sup>×</sup> <sup>10</sup>−<sup>3</sup> . The references are θref <sup>1</sup> = θref <sup>2</sup> = 0.0.

**Table 1** gives the results of whether the pendulum falls down in the process of trying to maintain standing balance. The pendulum is clearly more stable with tacit learning in MRMspace than without tacit learning in MRM-space.

**Figure 4** shows an overview of standing balance control simulations without and with learning in MRM-space. The pendulum CoM falls lower in the process of regaining balance when tacit learning is applied in MRM-space. **Figure 5** shows


FIGURE 4 | Overview of standing balance control simulation: (A) without learning; (B) with learning. "Without learning" means that tacit learning is applied to only joint space, and "With learning" means that tacit learning is applied to joint space and MRM-space. The red dotted line is the general trajectory of the center of mass (CoM) of the pendulum in the process of regaining balance after a disturbance. The CoM falls lower while regaining balance with tacit learning in MRM-space.

the trajectories of joints 1 and 2 in the process of regaining balance. In the case without learning in MRM-space shown in **Figure 5A**, the pendulum becomes upright after the disturbance by moving joints 1 and 2 in phase. By contrast, in the case with learning in MRM-space shown in **Figure 5B**, joints 1 and 2 move in anti-phase.

**Figure 6** shows the relationship between the disturbance and the energy consumption E per unit time, which is calculated from

$$E = \frac{\sum\_{t=0}^{T} (\mathbf{r}\_1^2 + \mathbf{r}\_2^2)}{T},\tag{19}$$

where T is the time until the pendulum becomes upright. We also calculate E when the simulation is conducted with tacit learning

FIGURE 5 | Trajectories of joints 1 and 2 while regaining balance in the simulation: (A) without learning; (B) with learning. The gray areas are where the joints move in phase, the white areas are where they move in anti-phase. Joints 1 and 2 move in phase while regaining balance without tacit learning in MRM-space. Joints 1 and 2 move in anti-phase while regaining balance with tacit learning in MRM-space. The disturbance is f = 184 N. The gain of tacit learning in MRM-space is <sup>k</sup>m<sup>2</sup> <sup>=</sup> 5.0 <sup>×</sup> <sup>10</sup>−<sup>4</sup> .

in MRM-space by using different integral coefficients, namely <sup>k</sup>m<sup>2</sup> <sup>=</sup> 5.0 <sup>×</sup> <sup>10</sup>−<sup>6</sup> and 5.0 × 10−<sup>5</sup> .

The pendulum can become upright against a larger disturbance with learning in MRM-space than without learning in MRM-space. The size of disturbance that the pendulum can withstand without falling over increases with the integral coefficient km<sup>2</sup> for tacit learning. However, E also increases with km2.

## 3.4. Standing Balance Control Experiment and Results

We conducted an experiment on a real 2DoF inverted pendulum with the same block diagram as in the simulation. The transfer matrix was calculated based on the physical parameters of the pendulum, whereas the gains in the block diagram were determined by trial and error. The experimental conditions can be seen in the Supplementary Video. **Figure 7A** shows the actual 2DoF inverted pendulum. **Figure 7B** shows the trajectories of joints 1 and 2 in the process of regaining balance. The pendulum becomes upright by moving joints 1 and 2 in anti-phase, as in the simulation with tacit learning in MRM-space.

## 3.5. Discussion of Standing Balance Control

The pendulum can remain upright against larger disturbances as the coefficient of tacit learning in MRM-space is increased (see **Figure 6**). However, a problem is that the energy consumption per unit time in same disturbance also increases as the coefficient is increased. Another problem is that, although we did not analyze the stability of this system, too large an integral tacit learning coefficient makes the system unstable (see Shimoda et al., 2012). To regain balance efficiently after a disturbance, it is necessary to change the tacit learning coefficient according to the disturbance.

It is well-known that people change their standing balance strategies between ankle and hip strategies (Horak and Nashner, 1986; Runge et al., 1999; Robinovitch et al., 2002) according to the prevailing disturbances. Each strategy is shown in **Figure 8**. The ankle strategy is a standing balance control method in which the person mainly moves the ankle joints in response to a relatively small disturbance, whereas the hip strategy is a standing balance control method in which the person moves the hip and ankle joints in anti-phase in response to a larger disturbance. The movements involved in the hip strategy are similar to those of the 2DoF pendulum when tacit learning is applied in MRM-space. It is interesting that our method of adjusting signals in MRMspace with tacit learning has something in common with a human strategy.

## 4. BIPEDAL WALKING CONTROL ON FLAT PLANE WITH 27DOF HUMANOID ROBOT

In section 3, we discussed the use of standing balance control of allow a 2DoF pendulum to react to disturbances. In this section, we add a "top-down" signal to include intentional behavioral changes. We apply the same control strategies to a 27DoF humanoid robot walking control with the added top-down signal. The weight and the length of segments of the robot is decided based on the NASA biometric research (NASA). We show through simulation and experiment that the signal added to the controller in MRM-space plays the role of behavioral intentions to change walking direction while maintaining walking balance.

## 4.1. Bipedal Walking Control Structure

**Figure 9** shows a bipedal walking controller designed by using the transfer matrix **T** and tacit learning. Terms θ, ˙<sup>θ</sup> <sup>∈</sup> **<sup>R</sup>** 27 are vectors of state variables. Terms θref ∈ **R** <sup>27</sup> is a vector of angle references. The torque vector τ ∈ **R** <sup>27</sup> for each joint is represented as

$$\mathfrak{r} = T A T^{-1} \Delta \theta + T \mathcal{B} T^{-1} \Delta \dot{\theta} + T \mathfrak{c} + \mathfrak{f}, \tag{20}$$

$$\mathbf{r} = \begin{bmatrix} \mathbf{r}\_1 \ \cdots \ \mathbf{r}\_{27} \end{bmatrix}^T,\tag{21}$$

$$\begin{aligned} \xi &= \\ diag(\left[0 \stackrel{\textstyle 0}{\longrightarrow} \begin{array}{c} k\_{\mathit{ankle}} \ k\_{\mathit{rankle}} \ \stackrel{\textstyle 0}{\longrightarrow} \begin{array}{c} 0 \end{array} \right] ) \int \mathtt{r} \, dt, \end{aligned} \tag{22}$$

where ζ is a vector that consists of the integral values of tacit learning, and tacit learning is applied to only the left and right ankle joints. The balance in the sagittal plane is maintained by tacit leaning, as in Shimoda et al. (2013). Terms klankle and krankle are the integral coefficients for the left and right ankle joints, respectively. Terms <sup>1</sup><sup>θ</sup> and <sup>1</sup> ˙<sup>θ</sup> are

$$
\Delta\theta = \left[\,\theta\_{r\circ f} - \theta\,\right], \,\,\Delta\dot{\theta} = -\dot{\theta}. \tag{23}
$$

FIGURE 7 | (A) 2DoF inverted pendulum. (B) Trajectories of joints 1 and 2 in the process of regaining balance in the experiment. Each point is supplemented with a spline curve. The gray areas are where the joints move in phase, the white areas are where they move in anti-phase. Joints 1 and 2 move in anti-phase in the process of regaining balance, as in the simulation with tacit learning.

Terms **A** and **B** are diagonal matrices:

$$\begin{aligned} \mathbf{A} &= \operatorname{diag}(\begin{bmatrix} k\_{\rho1} & k\_{\rho2} & \cdots & k\_{\rho27} \end{bmatrix}), \\ \mathbf{B} &= \operatorname{diag}(\begin{bmatrix} k\_{d1} & k\_{d2} & \cdots & k\_{d27} \end{bmatrix}), \end{aligned} \tag{24}$$

where kp1, · · · , kp<sup>27</sup> are values obtained by multiplying the eigenvalues of each mode by 10, and kd<sup>1</sup> , · · · , kd<sup>27</sup> are values obtained by multiplying the eigenvalues of each mode by 0.1. The eigenvalues are given by the mechanical resonance mode of the robot.

Term **c** is a vector that consists of the constant value of the torque of the eighth mode and is given by

$$\mathcal{C} = \begin{bmatrix} 0 \ \cdots \ \xi\_8 \ \cdots \ \mathcal{O} \end{bmatrix}^T,\tag{25}$$

where ξ<sup>8</sup> is a constant that can be adjusted manually.

section 4.1 for details.

The transfer matrix **T** ∈ **R** <sup>27</sup>×<sup>27</sup> of the 27DoF robot can be calculated using the method given in section 2.1. The movement of all modes can be seen in the Supplementary Video. In controlling the robot behavior, we focus on two specific modes from all the modes, namely the first and eighth modes (see **Figures 1 C,D**). We expect to realize two specific types of walking. Adjusting only the signal of the first mode while changing the P gain kp<sup>1</sup> can make the robot change its walking velocity on a flat plane. Adjusting the signal of the eighth mode with the constant value **c** can make the robot turn left and right on a flat plane with fixing kp<sup>1</sup> for the robot to walk forward.

## 4.2. Bipedal Walking Simulation and Results

Bipedal walking is performed as shown in **Figure 10A**, with one cycle of walking consisting of eight steps. The integral coefficients for the left and right ankle joints are klankle = krankle = 1.0e − 4. The references for each joint at each step in the simulation are described in **Table 2**. After finishing one cycle, the reference returns to the beginning of the cycle.

References for joints are given in Table 2. (B) Overview of walking simulation for adjusting the eighth mode. By adding positive constant values to the eighth mode, the robot can turn left.



Each step shifts to the next step under specific conditions. When the robot raises a foot, that step shifts to the next step when a knee joint angle of the robot equals the reference of the knee joint angle. When the robot puts a foot down, that step shifts to the next step when the sole of the foot touches the ground. If the robot falls down while walking, the robot is moved to the initial position while holding the integral values of tacit learning.

### (i) Walking forward and backward

**Figure 11A** shows time series of the CoM position in the direction in which the robot walks and the P gain kp<sup>1</sup> used to adjust the movement of the first mode. In the early stage of walking, the P gain is set as kp<sup>1</sup> = 270.0. It can be seen that the robot walks forward and backward according to the adjustment of the P gain. An overview of the simulation can be seen in the Supplementary Video.

#### (ii) Turning left and right

**Figure 10B** shows an overview of the walking simulation adjust the movement of the eighth mode by adding positive constant values to the signal of the eighth mode with an appropriate P gain kp<sup>1</sup> to walk forward. The robot can be seen turning left.

**Figure 11B** shows the trajectories of the CoM of the robot from the top view; the robot walks from the left of the figure to the right with teh appropriate P gain kp1. From **Figure 11B**, it can be seen that the walking direction is changed depending on the constant value used to adjust the movement of the eighth mode. The robot turns more as the constant value is increased. An overview of the simulation can be seen in the Supplementary Video.

**Figure 11C** shows the trajectories of the CoM of the robot from the top view; the robot walks from the left of the figure to the right with a fixed constant value ξ<sup>8</sup> = 4.5e3. From **Figure 11C**, it can be seen that P gain kp<sup>1</sup> is a key factor that affects the velocity in turning behaviors, and the walking direction is changed.

## 4.3. Bipedal Walking Experiment and Results

We conducted not only simulations but also walking experiments by using an actual robot, namely the humanoid robot NAO made by Aldebaran. This has 25DoF, and we can control the angle of each of its joints. NAO is controlled by angle inputs rather than torque inputs. We designed a control structure that generates reference joint angles by using the transfer matrix **T**.

The reference joint angle for a wide variety of walking can be described as

$$\boldsymbol{\Phi}^{\prime} = \boldsymbol{T}^{\dagger} \left[ \begin{bmatrix} \boldsymbol{k}\_{p1} & & \mathbf{O} \\ & \mathbf{1}.0 & \\ & & \ddots \\ \mathbf{O} & & \mathbf{1}.0 \end{bmatrix} \boldsymbol{T}^{\dagger^{-1}} \boldsymbol{\phi} + \begin{bmatrix} \mathbf{0}.0 \\ \vdots \\ \boldsymbol{\alpha}\_{8} \\ \vdots \\ \mathbf{0}.0 \end{bmatrix} \right] \tag{26}$$

$$\text{if } k\_{\mathbb{P}^1} = 1.0 \text{ and } \alpha\_8 = 0.0, \,\phi' = \phi,$$

where φ ∈ **R** <sup>25</sup> is a vector that consists of joint-angle references for walking, and φ ′ ∈ **R** <sup>25</sup> is an adjusted vector. Term **T** † is a

FIGURE 11 | Trajectory and time series of CoM position in turning and walking forward and backward in simulation: (A) Time series of CoM position in the direction in which the robot walks and the P gain of the first mode in the simulation. In the early stage of walking, the P gain is set as kp<sup>1</sup> = 270.0. The robot walks forward and backward according to changes in the P gain of the first mode. (B) Trajectories of robot CoM in walking simulation with adjusting the movement of the eighth mode. The robot walks from the left to the right of the figure. The black line is a trajectory of the robot walking forward without adjusting the movement of the eighth mode. The walking direction is changed depending on the constant value. The robot turns more as the constant value is increased. (C) Directions of robot CoM in turning with different walking velocities. Solid lines represent walking directions. The constant value is set as ξ<sup>8</sup> = 4.5e3. Walking velocity can be changed in turning behavior with different P gain kp1, and the walking direction is affected by walking velocity.

TABLE 3 | Target set for bipedal walking control experiment.


transfer matrix modified from the transfer matrix **T** to fit the DoF of NAO. Term kp<sup>1</sup> works like the P gain in the simulation of walking forward and backward.Term α<sup>8</sup> works like the constant value in the simulation of turning. Walking balance is acquired by applying tacit learning to joint space in the same way as 2DoF inverted pendulum postural control.

Experiments are conducted using the same scheme as that shown in **Figure 10A**. The target joint angles at each step in the simulation are described in **Table 3**. Each step shifts to the next step under specific conditions. When NAO raises a foot, that step shifts to the next step when the knee joint angle of the robot equals the reference knee joint angle. When NAO puts a foot down, that step shifts to the next step when the sole of the foot touches the ground.

(i) Walking forward and backward

FIGURE 12 | Time series of left hip and left knee joint angles in an experiment involving walking forward and backward. The P gain kp1 for adjusting the movement of the first mode changes as follows: kp<sup>1</sup> = 1.0 for 0–150 s, kp<sup>1</sup> = 1.1 for 150–240 s, and kp<sup>1</sup> = 1.0 for 240–250 s. NAO walks forward for 0–150 s and backward for 150–240 s. When walking is switched from forward to backward by adjusting the movement of the first mode, the hip joint angle decreases and the knee joint angle increases. (A) Left hip angle. (B) Left knee angle.

NAO can walk forward and backward when we adjust only the movement of the first mode by adjusting its gain. An overview of the experiment can be seen in the Supplementary Video. **Figure 12** shows time series of the left hip joint and left knee joint angles while walking forward and backward. When

walking is switched from forward to backward by adjusting the movement of the first mode, the hip joint angle decreases and the knee joint angle increases (see around 150 s in **Figure 12**).

#### (ii) Turning right and left

**Figure 13A** shows NAO turning left adjust the movement of the eighth mode by adding a positive constant value to the signal of the eighth mode with an appropriate P gain kp<sup>1</sup> to walk forward. **Figures 13B,C** shows the trajectories of the CoM and feet when the movement of the eighth mode is adjusted with positive and negative constant values. **Figure 14** shows time series of the knee angles when NAO turns left and right.

NAO can turn left and right by adjusting the movement of the eighth mode, as in the simulation. As shown in **Figure 14A**, the amplitude of the left knee joint angle is bigger than that of the right knee joint angle in adjusting the movement of the eighth mode by the constant value α<sup>8</sup> = −25.0, whereupon NAO turns left. The converse holds in **Figure 14B**.

## 4.4. Discussion of Bipedal Walking Control

The robot NAO could turn left and right by adjusting the movement of the eighth mode, that is, adjusting the bending of the robot and NAO at the waist on the frontal plane by adjusting

value is negative, NAO turns left.

the constant value. Adjusting the movement of the eighth mode deflected the CoM to the left-hand or right-hand side of the body, whereupon one foot took a larger step than did the other foot. This phenomenon can be confirmed from the fact that the amplitude of the left knee joint angle was bigger than that of the right knee joint angle (see **Figure 14A**).

The robot and NAO could walk forward and backward by adjusting the movement of the first mode. The robot and NAO differed in how the gain was adjusted, but we consider this to be because the gains of the other modes differed between the robot and NAO. When the movement of the first mode is adjusted, the degree to which the legs are opened is adjusted, whereupon the position at which the foot touches the ground can be adjusted. This can be confirmed from the fact that the hip joint angle decreases and the knee joint angle increases when walking switches from forward to backward (see **Figure 12**).

It is normally difficult to make a robot walk in a wide variety of ways because that necessitates designing a plurality of controllers and preparing references concerning each combination of individual joints. Instead, our method realizes turning and walking forward and backward smoothly by adjusting a few symbolized parameters in MRM-space without having to care about each combination of 27 joints and switching controllers. This is because there is a pattern of movements according to the mode, and the pattern to adjust is easy to understand visually.

A person can change behavior while caring intentionally only about "turn left and right" or "forward and backward." Likewise, our method can adjust signals and realize behavior without caring

about each joint, which is important in considering human-like movement in the robot under consideration.

## 5. DISCUSSION

As discussed in section 1, the aim of this paper is to clarify the mechanisms of generating automatic motor commands to achieve a symbolized behavioral purpose. Our approach is to develop an artificial controller that embodies those mechanisms and derive the important features of that function to assess the extent to which the robot behavior is human-like.

We reasoned that the key problem in developing a controller with those mechanisms would be realizing biomimetic adaptation with two-way behavioral adaptation. The first way is selecting the appropriate behavior that progresses in a top-down manner, and the second is adjusting the behavior according to the environment, which is a bottom-up process that progresses through body–environment interactions.

In the preliminary study, we discussed the standing balance control of a 2DoF inverted pendulum to introduce our control strategies, focusing only on the bottom-up adaptation process. In our control strategy, the control signals to each joint are transferred to another space computed by using the mechanical resonance modes, whereby we can easily understand the behavioral pattern that each control signal creates. We applied tacit learning in MRM-space and developed the controller to maintain standing against various levels of disturbance. The simulation and experimental results showed that the standing balance control capability was increased when the 2DoF inverted pendulum was controlled using MRMspace, suggesting that the simple adaptation mechanism is enough to improve the behavioral performance of standing balance when behavior control is conducted in a space where we can set the direction of behavior by adjusting to the change of the disturbance. The similarities between the mechanical resonance modes used in this system and the hip and ankle motion strategies of people must be another indication of the importance of reacting to a wide range of disturbances in a space in which the body parameters are well represented.

In the bipedal walking study using NAO, we discussed twoway adaptation. To represent the top-down process in which the behavioral purpose is selected, we chose the appropriate resonance mode with which to adjust the behavior to the desired one. As described in **Figure 13**, when we adjust the parameter of the eighth mode, the robot begins to turn left and right while maintaining walking balance. Walking balance was maintained by tacit learning applying the method in Shimoda et al. (2013) to joint space in the same way as 2DoF inverted pendulum control as a bottom-up process. In the simulation and the experiments, we succeeded in changing the behavior to turn left/right and go forward/backward by stimulating different mechanical resonance modes.

We consider that the signals added to the specified mode control can be treated as symbolized behavioral purpose in our study. The detailed commands to the joint control were created in two processes, namely, transferring the control signal from MRM-space to joint space, and tacit learning for maintaining walking balance while reacting to environmental inputs. Therefore, these results suggest that one-dimensional signal change can create the complicated combinations of the 25DoF of NAO required to change the behavior when the appropriate mode is stimulated.

Joint control is much more complicated in people than it is in robots because complicated combinations of muscles are required in the former. The notion of muscle synergy introduced in section 1 shares the same features as those of the mechanical resonance mode used in our method for robot control because the muscle-synergy space represents the behavioral features of body mechanisms and contributes to adjusting behavior in our case by using lower-dimensional signals.

These results and discussion suggest that transferring the signals to the appropriate control space is the key process for reducing the complexity of the signals from the environment. Discussing muscle synergy in a human controller and the mechanical resonance mode is only one step to the final output. If such steps were to be accumulated to form a larger network as described in **Figure 15**, we could form various behaviors from the more-symbolized behavioral target such as "go to the station."

We reason that the extent to which a symbolized target that is represented by the lower-dimensional signals is used to create the robot behavior is the critical assessment for evaluating the extent to which the robot behavior is humanlike. The results in this paper are just one step up from pure motor-control signals, implying far from human-like behavior. Further discussion is required to elevate the proposed system to using more-symbolized behavioral targets such as "go to the station." A key problem is automatic creation of the mechanical resonance mode. Non-linear transfer for more complicated environment is another important problem. Even in the control of human behavior, the process of creating muscle synergy remains mysterious. We are now on the way to clarifying the

## REFERENCES


process to a morehierarchical system in both physiological and artificial ways.

## AUTHOR CONTRIBUTIONS

MT explained the concept of mechanical resonance mode of the robot to SO. MT made a model of robot control structure using the mechanical resonance mode. MH and SS explained the concept of a biological learning method called tacit learning and gave advices on how to use tacit learning to SO. To clarify the mechanisms that generate automatic motor commands to achieve symbolized behavior, SO made the concept of adaptive control method using mechanical resonance mode and a biological learning method called tacit learning, designed a control structure to realize symbolized behavior purpose in discussions with FA, MH, YH, MT, and SS. SO conducted standing balance control simulations and bipedal walking control simulations with a 2DoF inverted pendulum and 27DoF humanoid robot, respectively. In discussions with FA, MH, YH, and SS, SO conducted standing balance control experiments and bipedal walking control experiments with an actual 2DoF inverted pendulum and an actual humanoid robot, respectively. SS helped in using the actual pendulum and humanoid robot in experiments.

## ACKNOWLEDGMENTS

This study was funded by a grant from the European Commission within its Seventh Framework Programme (IFP7-ICT-2013-10- 611695: BioMot - Smart Wearable Robots with Bioinspired Sensory-Motor Skills).

## SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnbot. 2018.00043/full#supplementary-material


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Okajima, Tournier, Alnajjar, Hayashibe, Hasegawa and Shimoda. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# An Efficient Modelling-Simulation-Analysis Workflow to Investigate Stump-Socket Interaction Using Patient-Specific, Three-Dimensional, Continuum-Mechanical, Finite Element Residual Limb Models

Ellankavi Ramasamy <sup>1</sup> , Okan Avci <sup>1</sup> , Beate Dorow<sup>1</sup> , Sook-Yee Chong<sup>2</sup> , Leonardo Gizzi <sup>3</sup> , Günter Steidle<sup>2</sup> , Fritz Schick <sup>2</sup> and Oliver Röhrle1,2,4 \*

#### Edited by:

Jose Gonzalez-Vargas, Otto Bock, Germany

#### Reviewed by:

Alex Dickinson, University of Southampton, United Kingdom Zhen Yuan, University of Macau, Macau

#### \*Correspondence:

Oliver Röhrle roehrle@simtech.uni-stuttgart.de

#### Specialty section:

This article was submitted to Bionics and Biomimetics, a section of the journal Frontiers in Bioengineering and Biotechnology

Received: 18 January 2018 Accepted: 23 August 2018 Published: 19 September 2018

#### Citation:

Ramasamy E, Avci O, Dorow B, Chong S-Y, Gizzi L, Steidle G, Schick F and Röhrle O (2018) An Efficient Modelling-Simulation-Analysis Workflow to Investigate Stump-Socket Interaction Using Patient-Specific, Three-Dimensional, Continuum-Mechanical, Finite Element Residual Limb Models. Front. Bioeng. Biotechnol. 6:126. doi: 10.3389/fbioe.2018.00126 <sup>1</sup> Department of Biomechatronic Systems, Fraunhofer-Institut für Produktionstechnik und Automatisierung (Fraunhofer IPA), Stuttgart, Germany, <sup>2</sup> Diagnostische und Interventionelle Radiologie, Sektion für Experimentelle Radiologie, Department für Radiologie, Universitätsklinikum Tübingen, Tübingen, Germany, <sup>3</sup> Institut für Mechanik (Bauwesen), Universität Stuttgart, Stuttgart, Germany, <sup>4</sup> Stuttgart Centre for Simulation Sciences, Universität Stuttgart, Stuttgart, Germany

The lack of an efficient modelling-simulation-analysis workflow for creating and utilising detailed subject-specific computational models is one of the key reasons why simulation-based approaches for analysing socket-stump interaction have not yet been successfully established. Herein, we propose a novel and efficient modelling-simulation-analysis workflow that uses commercial software for generating a detailed subject-specific, three-dimensional finite element model of an entire residual limb from Diffusion Tensor MRI images in <20 min. Moreover, to complete the modelling-simulation-analysis workflow, the generated subject-specific residual limb model is used within an implicit dynamic FE simulation of bipedal stance to predict the potential sites of deep tissue injury. For this purpose, a nonlinear hyperelastic, transversely isotropic skeletal muscle constitutive law containing a deep tissue injury model was implemented in LS-DYNA. To demonstrate the feasibility of the entire modelling-simulation-analysis workflow and the fact that detailed, anatomically realistic, multi-muscle models are superior to state-of-the-art, fused-muscle models, an implicit dynamic FE analysis of 2-h bipedal stance is carried out. By analysing the potential volume of damaged muscle tissue after donning an optimally-fitted and a misfitted socket, i.e., a socket whose volume was isotropically shrunk by 10%, we were able to highlight the differences between the detailed individual- and fused-muscle models. The results of the bipedal stance simulation showed that peak stresses in the fused-muscle model were four times lower when compared to the multi-muscle model. The peak interface stress in the individual-muscle model, at the end of bipedal stance analysis, was 2.63 times lower than that in the deep tissues of the stump. At the end of the bipedal stance analysis using the misfitted socket, the fused-muscle model predicted that 7.65% of the residual limb volume was injured, while the detailed-model predicted 16.03%. The proposed approach is not only limited to modelling residual limbs but also has applications in predicting the impact of plastic surgery, for detailed forward-dynamics simulations of normal musculoskeletal systems.

Keywords: continuum-mechanics, diffusion tensor MRI, skeletal muscle, transfemoral amputation, injury

## 1. INTRODUCTION

When designing and fitting prosthetic devices, prosthetists try to cater to the unique needs and desires of a patient. Those prosthetic devices that are perceived as good prosthetic fits provide the ability to walk comfortably, which should be as close as possible to normal gait, i.e., someone with a healthy residual limb (Legro et al., 1999). However, studies indicate that prosthetic fit varies, not only from one prosthetist to another, but also within the same prosthetist (Boone et al., 2012; Kobayashi et al., 2015). This indicates that most fits are suboptimal, and thereby impede the acceptance of prosthetic devices, affect gait, and may lead to pain and injury (Klute et al., 2001; Van Velzen et al., 2005). Until 2008, both computational and experimental studies focussed their efforts towards reducing the socket-stump interface pressure, which was considered to be the most critical factor in deciding the comfort of a prosthesis. The contribution of internal tissue strains towards the evolution of deep tissue injuries was first investigated by Gefen et al. (2008) and Portnoy et al. (2008). These tissue injuries originate deep within the limb and propagate towards the outer skin, making their timely prognosis difficult and dangerous. On the other hand, there exist continuum-mechanical approaches and finite element (FE) simulations that have been extensively used to predict the response of various biological tissues to external stimuli without invasive experiments (e.g., Röhrle et al., 2012; Miga, 2016; Budday et al., 2017; Heidlauf et al., 2017). This makes FE analyses (FEA) a suitable tool to understand internal soft tissue strains in residual limbs. However, to succeed in using FE simulations for investigating and ultimately designing prosthetic devices, if deep tissue injuries can be strongly reduced or eliminated, one requires an efficient tool to construct patientspecific limb models, e.g., from MRI or CT scans and appropriate loading conditions such as the ones that occur during bipedal stance or locomotion.

Over the last two decades, medical scans have been used to generate patient-specific FE models (cf. Zachariah et al., 1996; Portnoy et al., 2008; Zhang et al., 2013; Sengeh et al., 2016; Cagle et al., 2018). In these models, the soft tissues were segmented at most into one or two layers, i.e., muscle-fat complex (Portnoy et al., 2008; Cagle et al., 2018), and fusedmuscle and skin (Sengeh et al., 2016), which we refer to in the following as fused-muscle models. The justification provided for such a segmentation was that the small volume of the residuum comprised mostly of a muscle lump. The experimentalnumerical validations of the proposed geometry were performed using interface pressure maps by Portnoy et al. (2008) and with indenters by Sengeh et al. (2016). However, it remains to be seen if such minimally-segmented models can also be validated in case of transfemoral residua, where the volume of soft tissues is high, and segmenting the stump into a two-layered model might lead to problems in fitting the material parameters. The necessity of highly detailed human models based on medicalimaging data have been emphasised by several authors. Scheys et al. (2006) and Blemker et al. (2007) emphasise that imagebased personalised biomechanical analyses are required to truly understand human movement, motion related disorders, or postoperative gait anomalies, and Fernandez and Pandy (2006) place importance on realistic subject-specific, three-dimensional, individual-muscle FE models. In spite of the emphases on the necessity of detailed muscle models, the review by Dickinson et al. (2017) showed that the proposed FE models were either generic or one- or two-layered segmentation of the limb from MRI and/or CT. Further, they ignore the muscle's anisotropic behaviour. These models were primarily used to study the effects of different loading conditions on the stresses at the socketresidual limb interface. This is despite the fact that recent studies involving patient-specific FE analyses are raising the importance of tissue anisotropy (Sengeh et al., 2016; Dickinson et al., 2017). Such information is, however, not considered to date although it can be obtained, e.g., by utilising diffusion tensor imaging (DT-MRI).

DT-MRI, which is widely used to study the fibrous white matter in the brain (cf. Alexander et al., 2007), can also be used to extract skeletal muscle fibres through a technique called tractography, and the muscle structure thereof (cf. Froeling et al., 2014; Oudeman et al., 2016). The fibre architecture is particularly important for skeletal muscles, as they strongly influence their mechanical behaviour. Without key mechanical properties such as the fibre direction, it would be difficult to study gait and the evolution of internal tissue stresses during locomotion with a prosthesis (cf. Shojaei et al., 2016). In the muscle-driven forward dynamic simulations to study normal and pathological gait (cf. Piazza, 2006), it was stated that knowledge of muscle activations would provide the aetiology of movement disorders, and lead to effective treatments. It therefore follows that patientspecific models that aim to distinguish individual muscles, i.e., resolve them individually, must include muscle fibre orientation if continuum muscle models and the FE method are utilised to perform FE-based biomechanical analyses. Moreover, Dickinson et al. draw in their latest review on the state of utilising FE simulations for analysing the mechanical behaviour of lower limb amputees consensus that computational models used in socket design and tissue viability studies require patient-specific geometry and material models to investigate the internal state of residual limb tissues under load.

To the authors' knowledge, no high-fidelity model of the lower residual limb, in which the muscles are modelled as individual objects exist (the so-called individual-muscle models). More importantly, for the transfemoral case, no conclusive evidence has been provided, which proves that a one- or twolayered segmentation similar to those of the transtibial case is sufficient. Hence, this research proposes a DT-MRI-based modelling-simulation-analysis workflow that requires minimal user intervention, generates highly detailed patient-specific FE models of a residual limb, and uses the detailed model to carry out simulations investigating the difference between individualmuscle and fused-muscle residual limb models. This study is a precursor to future forward dynamics studies involving muscle activations, and mechanical loading of the internal soft tissues during gait.

While this work focuses more on assessing the influence of a particular socket on deep tissue injury (DTI), the same modelling-simulation-analysis workflow can also be utilised for other research questions, e.g., for predicting the result of plastic surgery by simulating, for example, contraction of the planned muscle graft, for detailed forward-dynamics simulations predicting the forces acting within a musculoskeletal system during motion (e.g., as proposed in Röhrle et al., 2017; Valentin et al., 2018), or for analysing neural activity for investigating neural-driven control strategies for prostheses.

## 2. METHODS

Given the limitations of existing stump models, the primary objective of this research is to develop a modelling-simulationanalysis workflow to aid in the modelling of a highly accurate three-dimensional, patient-specific, FE residual limb model. The proposed workflow for generating the residual limb model is based on MATLAB to orchestrate the pre- and post-processing of medical image data, to enable data transfer between different software tools, to support the generation of FE mesh of the residuum, and to finally map the muscle fibre information present in the DT-MRI scans onto the generated FE mesh. To complete the modelling-simulation-analysis workflow, the generated, subject-specific residual limb model is used within an implicit dynamic FE simulation of bipedal stance to predict potential regions of the residuum that might be subjected to DTI. This requires appropriate constitutive material models characterising the deformation and injury of soft tissues and reasonable boundary conditions. Here, boundary conditions reflecting bipedal stance are considered. Further, to investigate the necessity of the proposed highly detailed FE model of the residual limb, the dynamic FE simulations of bipedal stance are used to compare the results of the FE analyses between the detailed models and the fused-muscle model of the same residual limb when the same loading conditions are imposed. This is done by comparing the resulting stresses and volumes of regions that are predicted to be affected by DTI.

## 2.1. Data Acquisition

The detailed patient-specific residual limb model that is required for the FE analyses shall be derived from a medical image set. For this purpose, DT-MRI scans of the entire residual stump of a left transfemoral amputee were performed at the University Hospital Tübingen, Germany. The MR Examinations were performed on a 3 T MRI whole body scanner (Magnetom Skyra, Siemens Healthcare, Erlangen, Germany), and the volunteer gave written informed consent prior to the examinations. Ethical approval was obtained beforehand (ref: 587/2014BO1).

For signal detection, body array coils of the manufacturer were employed. One coil was positioned below and the other above the thigh stump. A Magnetisation Prepared Rapid Acquisition Gradient-Echo sequence (MP-RAGE) sequence was used to obtain high-resolution 3D data sets of the residual limb, with the socket donned, and without the socket. The sequence parameters were: repetition time (TR) 2,300 ms, echo time (TE) 3.21 ms, inversion time (TI) 974 ms, readout bandwidth 200 Hz/px, flip angle (FA) 8◦ , in-plane resolution 1.1 × 1.1 mm<sup>2</sup> , matrix size 224 × 448, field of view (FOV) 245 x 491 mm<sup>2</sup> , slice thickness 1.1 mm, 192 sagittal slices and an acquisition time of 4 min and 42 s.

For the DT-MRI scans, a 2D Echo Planar Imaging (EPI) sequence with stimulated echo preparation and fat suppression was applied. The sequence parameters were TR 13,000 ms, TE 36 ms, mixing time (TM) 200 ms, Bandwidth 2,380 Hz/px, resolution 3.2 × 3.2 mm<sup>2</sup> , matrix size 76 × 150, FOV 245 × 491 mm<sup>2</sup> , fractional readout 6/8, slice thickness 3 mm, slice orientation sagittal, 42 slices, 12 diffusion directions, b-values 0 and 700 s/mm<sup>2</sup> , 4 acquisitions for each direction and 12 acquisitions for images with b = 0 s/mm<sup>2</sup> (b0 images) and an acquisition time of 13 min and 23 s.

## 2.2. Modelling the Residual Limb From DT-MRI Scans

DT-MRI involves scanning a volume along 6 or more predefined gradient directions (cf. Mori and Zhang, 2006). From the diffusion of water molecules in any given voxel of the scanned volume, the variation of the applied magnetic field gradient is measured and its three-dimensional diffusion characteristics determined. These characteristics can be represented by a tensor, the so-called diffusion tensor. In skeletal muscles, this diffusion occurs preferentially along the muscle fibres, thereby revealing the anatomy of the muscles. To obtain a computational subjectspecific model suitable for an FE analysis from such data, a workflow was created in MATLAB (R2016a, The Mathworks, Inc., Natick, Massachusetts, United States). The following sections describe the modelling process.

## 2.2.1. Data Preparation

The DT-MRI scans were obtained in mosaic format (see **Figure 1**), where each image contained the entire 3D scan volume. For this subject, a total of 60 mosaic images were obtained from 4 acquisitions, where each mosaic image had dimensions of 532×1, 050 px, and each 2D slice in the mosaic had dimensions of 76 × 150 px. By first traversing along the columns (k-axis) of the mosaic (cf. **Figure 1**), and stacking each 2D slice one behind another, a 3D representation of the residual limb was created. Each pixel in this mosaic can be accessed by providing two sets of coordinates. The first set of coordinates ([k, l]) selects a 2D slice, and the second set of coordinates ([u, v]) selects a pixel in this 2D slice. If i denotes the intensity of any given pixel in the mosaic, the voxel intensities in the 3D image stack, **S**, is given by

$$\begin{aligned} \mathbf{S} &= [i\_{\iota\nu}]\_{kl} = i\_{\iota\nu m} \\ \text{with} \quad m &= N(l-1) + k, \end{aligned} \tag{1}$$

where m can be thought of as the slice number (index) in the constructed 3D volume, N is the total number of rows or columns in the mosaic (here, N = 7), and indices k and l index a 2D slice in the mosaic (k = l = {1, . . . , 7}). The extents of each 2D slice within the mosaic are 76 × 150 px, i.e., u = {1, . . . , 76} and v = {1, . . . , 150}. In the given example (cf. **Figure 1**), the last row contains no images (m = {1, . . . , 42}).

To reduce noise in the mosaic images, the variance in pixel intensities was reduced by averaging the images over all 4 acquisitions, i.e., the average was taken over 60 images such that we had one image along each gradient direction, and one b0 image, which resulted in a total of 13 mosaics. The "Dicom to nifti converter, nifti tool and viewer" MATLAB toolbox was used to convert these 13 images into a NIfTI file. This NIfTI file was used in MedINRIA's DTI Track module (v1.9.2, 64-bit, ASCLEPIOS Research team, France) to track muscle fibres. The parameters, which were used for fibre tractography are fibre smoothness 10, minimum length of fibres 10, and both FA thresholds were 0. The resulting fibres were wavy due to inherent noise in the scans (cf. **Figure 2A**). It has been shown that this inherent noise in DT-MRI images is best described by a Rice distribution (cf. Basu et al., 2006). So, a technique using overcomplete local PCA was employed to remove this Rician noise from the NIfTI images (cf. Manjón et al., 2013). The resulting fibres are shown in **Figure 2B**. DICOM images (DT-MRI-DICOMs) were produced from these averaged and de-noised NIfTI images.

The fibres were segmented into various muscle fibre bundles using MedINRIA's fibre cropping tool. This tool allowed fibres passing through the region of interest, i.e., a cropping box, to be bundled together. Fibres of 11 muscles, which were distinctly visible, were bundled. These were Adductor Magnus, Biceps Femoris, Gluteus Maximus, Gluteus Minimus, Pectineus, Rectus Femoris, Sartorius, Semimembranosus, Semitendinosus, Vastus Lateralis, and Vastus Medialis. Segmentation of the voxels traversed by each of the above muscles was exported from MedINRIA, which were processed in MATLAB to create binary images of these muscles.

2.2.2. Image Segmentation Using Simpleware ScanIP The 3D residual limb model was created using Synopsys' Simpleware ScanIP (Synopsys, Mountain View, USA). The binary images that were created by MATLAB, were loaded as background image stacks in ScanIP. These binary image stacks that represent regions of interest (also called masks) are used to create 3D volumetric models. The geometric model for our analysis consisted of the femur, muscle tissues, fat, prosthetic liner and socket, whose construction are described below.

FIGURE 1 | Siemens mosaic file. A sample mosaic image is shown here. For the sake of illustration, the 2D slices that make up the mosaic are separated by thin lines. Any pixel in the mosaic can be accessed using two coordinate systems–the [k, l] coordinate that references a 2D slice in the mosaic, and the [u, v] coordinate that references a pixel in the 2D slice referenced by [k, l].

**Muscles.** The muscle masks that were created in the previous section were loaded in Simpleware ScanIP, whose segmentation resulted in volumetric muscle models.

**Femur and socket.** The highly diffused spongiosa, and the relatively low water content in the femur resulted in negligible or random diffusion of water in the DT-MRI scans. This made its segmentation difficult. Furthermore, the socket was also opaque in the DT-MRI scans due to absence of water content. For this reason, the femur and socket were segmented from T1 MP-RAGE sequences (cf. section 2.1).

To register the T1 MP-RAGE sequences with the DT-MRI scans, femurs from both these scans were segmented and exported as surface (STL) models. Using an iterative closest point technique, the transformation that correctly positioned and aligned the surface model of femur segmented from T1 scans (the T1-segmented femur) with that of DT-MRI (the DT-MRI-segmented femur), was determined. This transformation was then applied to the T1-segmented femur and socket, and were imported as CAD models into Simpleware +CAD module.

**Fat.** A mask of the complete residual limb was created from the DT-MRI-DICOMs. A boolean difference of this mask from the union of the muscle and femur masks resulted in the mask for fat tissues.

**Liner.** The subject's liner was 9 mm thick. This liner was modelled by uniformly extending the outer surface of the residual limb in the normal direction towards its exterior by 9 mm.

A Python script automated the process of loading the above masks into Simpleware ScanIP. Morphological and smoothing filters (dilation and recursive Gaussian) were applied on all masks to generate smooth surfaces. After post-processing, the maximum difference between the volumes of muscle masks before and after smoothing was <0.2%.

## 2.2.3. Generation of the FE Model

Using Simpleware ScanIP +FE, an FE mesh of the subject's residual limb was created for LS-DYNA (LSTC, Livermore, California). All parts were meshed with linear tetrahedral elements. The mesh parameters were: coarseness -10, number of quality optimisation cycles 10, mean Jacobian of 0.5 and minimum of 0.1. The parts were allowed to change during meshing, where the maximum distance that a surface node can be displaced off the original surface was limited to 0.2 mm. Contacts were not defined in Simpleware ScanIP, which resulted in common nodes between the tissues in the FE mesh.

As mentioned earlier in section 1, the muscle fibres were not only meant to automate the modelling process but also to enhance the FE mesh with directional fibre information, which are required in muscle activation studies. The fibres, that were in the NIfTI coordinate system (CS), were aligned with the FE mesh by transforming them back into the DICOM CS. These fibres were not physically modelled and embedded in the FE mesh but rather the fibre anisotropy within each element of the muscles was modelled using the "element\_solid\_ortho" card in LS-DYNA. To do so, the feature that fibres exported from MedINRIA were consecutively numbered was exploited in determining the number of fibre strands passing through an element. Three fibre strands passing through two elements of the FE mesh are illustrated in **Figure 3**. The effective fibre direction

in an element is the weighted sum of the trend lines fitted for each set of consecutively numbered fibre points in that element. In general, if an FE mesh consists of N<sup>e</sup> elements, where N i s fibre strands traverse the i th element (i ∈ Ne) with its centroid **x**<sup>i</sup> , and **g**<sup>s</sup> was the line of best fit for the strand s with weight w<sup>s</sup> , the effective fibre direction in that element, **f**(**x**i), was defined as

$$\begin{aligned} \mathbf{f}(\mathbf{x}\_i) &= \sum\_{s=1}^{N\_s^i} w\_s \mathbf{g}\_s, \\ \text{with} \quad w\_s &= \frac{n\_s^2}{\sum\_{s=1}^{N\_s^i} n\_s^2}, \end{aligned} \tag{2}$$

where n<sup>s</sup> was the number of fibre points in the fibre strand s. The effective fibre direction in all N<sup>e</sup> mesh elements was determined.

The estimation of effective fibre direction in some elements showed inconsistencies with the fibre directions in their immediate neighbourhood. In some other elements of the mesh, like near the boundary of Gluteus Maximus, no fibres were detected. This is attributed to the partial volume effects, and morphological and smoothing filters applied during image processing using Simpleware ScanIP that changed the muscle volume, resulting in missing some fibre information within the boundary elements. To correct the orientation of fibres, and to enhance the model, the fibre field was smoothed using radial basis interpolation with a Gaussian kernel. The new fibre direction in the i th element with its centroid at **x**<sup>i</sup> , ˜**f**(**x**i), was defined as

$$\begin{aligned} \tilde{\mathbf{f}}(\mathbf{x}\_{i}) &= \sum\_{\mathbf{c}=1}^{N\_{\mathbf{c}}} \phi \mathbf{f}(\mathbf{x}\_{\mathbf{c}}) \\ \text{with} \quad \phi &= \hat{\phi}(\mathbf{x}\_{i}, \mathbf{x}\_{\mathbf{c}}, p) = \exp\left(-p \frac{||\mathbf{x}\_{i} - \mathbf{x}\_{\mathbf{c}}||}{L}\right)^{2}, \end{aligned} \tag{3}$$

where **f**(**x**e) was the original effective fibre direction in the element with centroid at **x**<sup>e</sup> [cf. Equation (2)], φ was the Gaussian radial basis function kernel, the scalar parameter p set the number of neighbouring elements that influenced the fibre orientation in the current element (here, p = 20), and L was the length of the diagonal of the bounding box formed by mesh elements of the muscle. The above steps were performed for all 11 muscles in the residual limb. This resulted in the final FE mesh for our analyses. An overview of the entire workflow is shown in **Figure 4**.

## 2.3. Constitutive Model for Soft Tissues

Skeletal muscle contractions occur as a result of chemical interactions resulting in physical movement of muscle filaments. The grand sum of individual filament contractions results in the overall muscle contraction. These muscle contractions can be described in varying levels of detail (see Heidlauf and Röhrle, 2014; Bradley et al., 2018, for more details). The continuum-mechanical model for the soft tissues, within this work, was based on a 3D phenomenological approach, i.e., the constitutive model of the muscles is based on macroscopically observed muscle contractions as presented in Röhrle et al. (2017).

It is assumed that the force developed by the muscle fibres is additively split into an active and a passive part. Passive force is generated when fibres stretch, while an active force is related to muscle contraction stemming from chemo-electro-mechanical processes on the cellular level. In order to model these processes in a continuum-mechanical sense, these contributions were expressed as a function of the fibre's length or stretch. Time-dependent viscoelastic behaviour of soft tissues was ignored, and the soft tissues were modelled as hyperelastic, quasi-incompressible materials. Muscles are assumed to undergo large deformations, and therefore, the theory of finite elasticity is assumed. The material model for soft tissues implemented here is based on Röhrle et al. (2017). In the following, a brief overview of the theory and constitutive equations for the material model are given.

### 2.3.1. Constitutive Muscle Model

A continuum body B is assumed to be a collection of points P in space whose local deformation can be described by the deformation gradient **F**. The deformation gradient **F** is a map between the reference and the current configuration, and is defined as **F** = Grad **x** = ∂**<sup>X</sup> x**, where **X** is the position vector to a material point P at the reference configuration, and **x** is the position vector to the same point in the deformed configuration of the body B. In the theory of Finite Elasticity, the deformed state of the body B is formulated in terms of the right or the left Cauchy-Green tensor, which are defined as **C** = **F** <sup>T</sup>**F**, and **B** = **FF**<sup>T</sup> , respectively. The balance of linear momentum is given by

$$\mathbf{cliv}\,\mathbf{T} + \rho\mathbf{g} = \rho\ddot{\mathbf{x}}, \quad \text{with } \rho = \frac{1}{\mathbf{det}\,\mathbf{F}}\,\rho\_0,\tag{4}$$

where **T** is the Cauchy stress, ρ<sup>0</sup> is the material density in the reference configuration, ρ is the density in the current

configuration, **g** is the acceleration due to gravity, and **x**¨ is the acceleration of spatial points P ∈ B.

The stress developed in the soft tissues is additively split into one part stemming from the isotropic matrix and one part originating from anisotropic behaviour associated with the fibre direction. Likewise, the stress in the fibre direction is additively split into its active and passive contributions. Therefore, the second Piola-Kirchhoff stress tensor, **S**, can be expressed as

$$\mathbf{S} = \mathbf{S}\_{\text{iso}} + \mathbf{S}\_{\text{arriso}} = \mathbf{S}\_{\text{iso}} + (1 - \boldsymbol{\gamma})(\mathbf{S}\_{\text{pass}} + \alpha \mathbf{S}\_{\text{act}}), \quad \text{(5)}$$

where **S**iso and **S**aniso are the isotropic and anisotropic parts of the stress tensor. Likewise, the passive and active contributions to the anisotropic stress tensor are **S**pas and **S**act. As mentioned earlier (cf. section 2.3), the material model developed here was meant to be applicable for soft tissues, i.e., fat, skeletal muscles and tendons. For this reason, a binary variable γ was introduced to control the exclusion or inclusion of the stress contribution from fibres. For example, γ = 1 results in purely isotropic stress contribution, while γ = 0 allows one to consider the anisotropic contributions due to the fibres as well. Muscle activation is given by α, with 0 ≤ α ≤ 1.

The strain energy formulation for the compressible isotropic soft tissue matrix was taken from Crisfield (1997). The isotropic second Piola-Kirchhoff tensor is

$$\mathbf{S}\_{\mathbf{i}\mathbf{s}\mathbf{o}} = \left(B\_1 \mathbf{I} + B\_2 \mathbf{C} + B\_3 \mathbf{C}^{-1}\right) + k(\mathbf{det}\,\mathbf{F} - 1)\,I\_3^{1/2}\mathbf{C}^{-1},\tag{6}$$

where k is the bulk modulus of the soft tissue matrix, and

$$\begin{aligned} B\_1 &= 2C\_1 I\_3^{-1/3} + 2C\_2 I\_3^{-2/3} I\_1, \\ B\_2 &= -2C\_2 I\_3^{-2/3}, \quad \text{and} \\ B\_3 &= -2/3C\_1 I\_3^{-1/3} I\_1 - 4/3C\_2 I\_3^{-2/3} I\_2. \end{aligned} \tag{7}$$

Here, C<sup>1</sup> and C<sup>2</sup> are material parameters of the isotropic part of the strain energy function, and I1,I<sup>2</sup> and I<sup>3</sup> are the invariants of **C**. The fourth invariant of the transversely isotropic material is

$$\begin{aligned} I\_4(\mathbf{C}, \mathbf{M}) &= \mathbf{M} : \mathbf{C}, \\ \text{where} \quad \mathbf{M} &= \mathbf{a}\_0 \otimes \mathbf{a}\_0, \end{aligned} \tag{8}$$

where **M** is a structural tensor defined by means of the fibre direction, **a**<sup>0</sup> in the reference configuration. Then, one can define the fibre stretch 3 as 3 = √ I4.

The active and passive contributions of fibres to the second Piola-Kirchhoff stress (cf. Röhrle et al., 2017) are

$$\begin{aligned} \mathsf{S\_{\texttt{\tiny{\tiny{\tiny{\tiny{\tiny{\forall{\textrm{\tiny{\tiny{\tiny{\forall{\text{\tiny{\tiny{\forall{\text{\tiny{\text{\tiny{\text{\tiny{\text{\tiny{\text{\tiny{\text{\text{?}{\text{\text{?}}{\text{\text{?}}}}}}}}}}}}}}}}}} \,\text{\}} \\ \mathsf{S\_{\texttt{\tiny{\text{\tiny{\text{?}}{\text{\text{?}}{\text{?}}{\text{?}}{\text{?}}{\text{?}}{\text{?}}}}} \,\text{M} \,\text{M} \,\text{,} \quad \text{if } \,\Lambda \,\text{>} \,\text{M}} \,\text{M} \,\text{,} \\ \mathsf{S\_{\tiny{\text{?}}{\text{?}}{\text{?}}{\text{?}}{\text{?}}} \,\text{exp} \,\text{P} \,\text{(} - \left| \begin{array}{c} \Lambda/\Lambda \text{op}\,\text{t} - 1} \,\text{\/{\text{?}}{\text{?}}{\text{?}} \text{?} \text{?}} \\ \Lambda^{2} \,\text{W} \,\text{s} \,\text{s} \,\text{} \,\text{A} \,\text{"} \,\text{s} \end{array} \right. \end{aligned}} \end{aligned}$$

$$\mathsf{S\_{\texttt{\tiny{\text{add}}}} \,\mathsf{exp} \,\left( - \left| \begin{array}{c} \Lambda/\Lambda \text{op}\,\text{t} - 1} \\ \Lambda/\Lambda \text{op}\,\text{s} \,\text{} \,\text{A} \,\text{;} \,\text{end} \right. \end{aligned} \right)  $\mathsf{M}$ $$

Here, C<sup>3</sup> and C<sup>4</sup> are material parameters and Smax is the maximum stress that a muscle can produce at its optimal length (3opt). The parameters 1Wasc, νasc, 1Wdsc and νdsc affect the magnitude of the active part of the Piola-Kirchhoff stress. The material parameters chosen for the muscles are given in **Table 1**. The material tangent, C, that is required for implicit computations is a fourth-order tensor obtained from the secondorder Piola-Kirchhoff stress tensor as C = CMNOP = 2∂**<sup>C</sup> S**. It is noted that the implementation within LS-DYNA requires the definition of the spatial counterpart of C, which is denoted by

#### TABLE 1 | Material parameter table.


Parameters for the material model. Parameters for muscle and Skin/Fat were adapted from Röhrle et al. (2017).

B. But B is easily obtained by the push-forward of the material tangent C, i.e.,

$$\mathbb{B} = J^{-1} \chi\_\*(\mathbb{C}) = (\mathbb{F} \otimes \mathbb{F})^{\frac{23}{\mathsf{T}}} \mathbb{C} (\mathbb{F}^{\mathsf{T}} \otimes \mathbb{F}^{\mathsf{T}})^{\frac{23}{\mathsf{T}}}.\tag{10}$$

The superscript <sup>23</sup> above the transpose indicates that the transposition is defined by the exchange of second and third bases in the dyadic product. Hence, the spatial elasticity tensor is given in index notation by

$$\mathbb{E}\_{ijkl} = F\_{iM} F\_{jN} F\_{kO} F\_{IP} \mathbb{G}\_{MNOP}.\tag{11}$$

### 2.3.2. Tissue Injury Model

Exposing the residual limb to excessive stresses and strains leads to soft tissue damage through cell death (see Oomens et al., 2014). This is also associated with deep tissue injury, whose timely prognosis is difficult. Using patient-specific limb models, FE analyses can provide the internal stress state of soft tissues; the basis for predicting tissue damage.

The main idea of Gefen et al. (2008) is that cell death occurs if εeff > εcrit, where εeff is the effective strain in the cell, and εcrit is the critical strain limit beyond which cell death occurred. The effective and critical strains are defined as

$$\begin{aligned} \varepsilon\_{\text{eff}} &= \sqrt{\frac{2}{3} \varepsilon\_{ij} \varepsilon\_{ij}}, \quad \text{and} \\ \varepsilon\_{\text{crit}} &= \frac{K}{1 + \exp(\beta(t - t\_0))} + C, \end{aligned} \tag{12}$$

where ε is the symmetric Green strain. The values for the empirical constants in Equation (12) are K = 0.268, t<sup>0</sup> (time of midpoint of the sigmoid step) = 9.78 × 10<sup>6</sup> ms, β = 5.83 × 10−7ms-1 and C = 0.332 (see Gefen et al., 2008). The Boltzmann-type sigmoid curve seen in Equation (12) provides the evolution of critical strain based on which a cell was identified as either injured or healthy. This injury model was implemented in the material model.

The material parameters of the soft tissues, bone, liner and socket used in the FE simulations are provided in **Table 1**.

## 2.4. Loading and Boundary Conditions for FE Analysis of Bipedal Stance

The dynamic FE analysis of bipedal stance is performed in two stages: socket donning, followed by constant femoral load over a period of 2 h. The boundary conditions, loads and contact definitions applied in the analysis are detailed here.

For the generated FE mesh of the residual limb, no contact pairs were defined (cf. section 2.2.3). This resulted in common nodes between adjacent parts of the residual limb (muscles, bone, fat, liner) at their boundaries. A frictionless surface-to-surface mortar contact was defined between the outer surface of the liner and inner surface of the socket.

For the FE analysis of bipedal stance, the residual limb and prosthesis were only permitted to translate in the craniocaudal axis, while the other 5 degrees of freedom (DoF) were constrained. In the initial state, the socket and residual limbliner complex were separated by a distance of 130 mm in the craniocaudal direction. In the socket donning simulation, the femur was constrained in all DoFs and the socket was translated towards the limb (superior) by 130 mm along the craniocaudal axis in 10 s (see also **Figure 5A**). At the end of the socket donning simulation, the nodes of the liner that were in contact with the socket, were tied to the socket using tied-nodes-tosurface penalty-based contact in LS-DYNA. The translational constraint along the craniocaudal axis that was imposed on the femur during the socket donning simulation was removed. This was followed by the application of 400 N femoral load, which is approximately half the subject's body weight, acting in the positive inferior direction for 2 h (see also **Figure 5B**). This duration (2 h) was selected to be long enough to study strain-time dependent cell death.

The residual limb, socket and liner were meshed with 1-point, constant pressure, solid tetrahedral elements. The dynamic, implicit analysis was solved using Davidon-Fletcher-Powell (DFP) quasi-Newton iterative solver with LS-DYNA on a Linux workstation with 4 processors [Intel(R) Xeon(R) CPU E5-2687W, 3.10 GHz] using 1.12 GB RAM.

## 3. RESULTS

In the following, we present the outcome of the proposed modelling-simulation-analysis workflow, i.e., the result of the geometry and FE mesh generation of the residual limb, and how this model can be used to analyse and compare stresses and injured volumes between fused- and individual-muscle models in the socket-donned state, with the original and misfitted sockets, during bipedal stance.

FIGURE 5 | Boundary conditions for FE analysis. The two stages of the socket donning analysis are illustrated here. (A) In the first stage, the socket is donned over the residual limb through prescribed motion boundary condition. (B) In the next stage, the patient's body weight is applied on the femur in the socket-donned state as the prosthesis rests on the ground.

## 3.1. Geometry and FE Mesh of the Residuum

The results of applying the above described modelling workflow to the imaging data results in the discretised residual limb model, which is shown in **Figure 6**. **Figure 6A** depicts the geometries of the bone (residual femur) and the skeletal muscles within the stump by suppressing the opacity of the outer skin and liner while **Figure 6B** shows the stump including the fat/skin and liner geometry, and **Figure 6C** presents the model of the original socket worn by the subject. The FE meshes of these models, constructed with the mesh parameters listed in section 2.2.3, are shown in **Figure 7**. In **Figure 7A**, the FE mesh of muscles and bone is superimposed on the mesh of the skin. The linear Lagrange tetrahedral FE meshes of liner and socket are shown in **Figures 7B,C**. The mesh consisted of 5, 416 elements in the femur, 35, 257 elements in the muscles, 97, 503 elements in the skin/fat layer, 51, 791 elements in the liner, and 327, 865 elements in the socket.

The fibres that were mapped from MedINRIA into the FE mesh suffered from inconsistent fibre directions among neighbouring elements, and with some elements completely devoid of fibre information. This is shown in the case of Gluteus Maximus in **Figure 8A**. With the fibre-mapping algorithm described in section 2.2.3, these inconsistencies were corrected, which resulted in the fibre distribution in the Gluteus Maximus as shown in **Figure 8B**. The direction of muscle fibres in the complete residual limb is shown in **Figure 8C**.

The duration of the overall modelling process, from segmenting the DT-MRI scans to creating the FE model of the residual limb with tissue anisotropy, was about 20 min.

## 3.2. Mesh Convergence Studies

Prior to performing the bipedal stance analysis, a mesh convergence study with the boundary conditions described in section 2.4 was performed. Varying mesh densities of the residual limb model were obtained by adjusting the mesh coarseness slider in Simpleware ScanIP +FE between −35 and −5. The selection criteria was based on convergence of volume-normalised stresses in the residuum, with which the optimal mesh was selected. The volume-normalised stress σ is computed by

$$
\overline{\sigma} = \frac{1}{V} \sum\_{\varepsilon=1}^{N\_\varepsilon} \sigma\_\varepsilon \nu\_\varepsilon,\tag{13}
$$

where σ<sup>e</sup> is the von Mises stress in an element of volume ve, V is the total volume of residual limb and N<sup>e</sup> is the total number of elements in the residuum. The mesh convergence plot and the mesh densities of the residual limb model are shown in **Figure 9**. From the graph, one can see that the normalised stress is decreasing with increasing mesh density, in a negligible fashion, i.e., by <1.5%. As a result, the FE model created with coarseness factor of –35 resulting in a mesh with 517, 832 elements was chosen for this study.

## 3.3. Bipedal Stance Simulation

For highlighting the full capabilities of the modelling-simulationanalysis workflow and for emphasising the need of detailed, individual-muscle models, two models were created. These are (i) the fused-muscle model for the sake of comparison with the proposed individual-muscle model and (ii) a misfitting socket to predict tissue injury. To create the fused-muscle model, masks of individual muscles were fused. Using the same mesh parameters, the FE mesh of the fused-muscle model was generated. The resulting model and the FE mesh are shown in **Figure 10**. The number of linear tetrahedral elements in the fused muscle model were 477, 989. The misfitting socket was generated by isotropically shrinking the original socket by 10%. This scaling factor is not motivated by any particular application.

The bipedal stance simulation was performed using three different limb models, namely (i) the individual-muscle model

FIGURE 8 | Corrected fibre orientation. (A,B) show the fibres in Gluteus Maximus before and after interpolation using radial basis function, respectively. It can be seen that some boundary elements in (A) contained no fibre information. The inconsistent and missing fibres were corrected through the interpolation method proposed in section 2.2.3, and the resulting fibre distribution is shown in figure (B). Fibre orientation in the muscles are shown in (C) with femur, fat and liner suppressed for the sake of clarity.

On the right, the number of linear tetrahedral elements in the FE model is tabulated against the ScanIP coarseness factors. The graph depicts the decrease in normalised stress with increase of mesh density, i.e., decreasing the mesh coarsening factor from −35 to −5.

together with original socket, (ii) the individual-muscle model together with smaller, misfitting socket, and (iii) the fused-muscle model together with the misfitting socket. The von Mises stresses and tissue damage in the residual limb during the bipedal stance analysis are shown for each of the cases in **Figure 11**. A maximal von Mises stress of 63.5 kPa developed in the individual-muscle

model with the misfitting socket, i.e., in case (ii). The peak stresses for cases (i) and (iii) were 19 and 14.34 kPa, respectively.

**Figures 11D–F** depicts, for all three cases, the damaged muscle tissues in red. The magnitude of soft-tissue damage for the three cases is plotted in **Figure 13**. Here, the volume percent of damaged tissues with respect to the total, undamaged volume of the tissues is plotted over the duration of the entire stance phase. It is observed that with the original socket, the volume percent of damage is negligible. The maximum observed damage at the end of the simulation was 0.01%. When simulated with the misfitting socket, i.e., cases (ii) and (iii), this rose to 16.03 and 7.65%, respectively.

The interface stresses on the skin/fat layer in both the fusedand individual-muscle models after the 2-h bipedal stance when donned with the original sockets are shown in **Figure 12**. The mean and standard deviation of the interface stresses in the fused- and individual-muscle models were 5.4±1.6 and 4.9±1.1 kPa, respectively. The peak stresses in the two models were 16.3 and 24.1 kPa, respectively, which were observed in the proximal anterior and posterior regions of the residual limb.

## 4. DISCUSSION

In this study, we proposed an efficient modelling-simulationanalysis workflow to investigate stump-socket interaction during bipedal stance and hypothesised that detailed, individual-muscle models of residual limbs will provide more insight into the prosthesis-stump interaction than that provided by fused-muscle models. This study can be considered as a proof of concept for future work on automating and optimising socket design from imaging data.

## 4.1. DT-MRI-to-FE Mesh Workflow

The core of the modelling-simulation-analysis workflow of this research is the quasi-automatic generation of detailed patientspecific FE models of residual limbs from DT-MRI scans. The model generation relies on the fibres extracted from DT-MRI scans, i.e., the muscle and other soft tissue masks that are imported to Simpleware ScanIP, are created from the tracked fibres. But the process of extracting these fibres is done manually, and therefore subjected to some degree of human-related error. For example, the influence of parameters set in MedINRIA's DTI Track module, in the resulting fibre distribution, and in the muscle masks that were generated, is not known. It is expected that changes in these parameters might affect the volume of muscle masks but not the resulting fibre orientations. Another potential source of error is due to the manual grouping of muscle fibres into bundles, in MedINRIA. This can lead to some fibres being simultaneously grouped into two muscle bundles, and therefore result in overlapping muscle masks in Simpleware ScanIP, where the stacking order of these masks plays a vital role in determining the resulting model and FE mesh. In case of overlapping volumes, the masks at the top of the stack overwrite the volume of intersecting masks below them. Hence, different models can result from different mask hierarchies. Moreover, smoothing and morphological operations in ScanIP, which are essential to obtain a smooth and kink-free FE mesh, alter the resulting volume and geometry of the residual limb. However, utmost care was taken in performing these morphological and smoothing operations such that the total volume of the muscle masks did not vary by more than 0.2%.

As a result of the parameters chosen for fibre tractography, and the smoothing and morphological operations that alter the geometry of the residual limb, some elements of the FE mesh might either lack fibre information or contain inconsistent fibre orientations. Incorrectly bundled fibres might also lead to inconsistent fibre directions in the mesh. The implemented fibre-correction algorithm fills in the missing fibre information and also corrects the inconsistent fibre orientations within each muscle. This however, is done at the cost of smoothing the fibre orientation field over the entire muscle domain, which might lead to over-smoothing. The extent of any such oversmoothing needs to be verified by comparing the smoothed fibre field with that of the non-smoothed, original fibre field. The error associated with manual bundling of muscle fibres also affects the fibre orientation in the muscles. Incorrectly bundling fibres of neighbouring muscles might lead to non-realistic fibre orientation at the muscle boundaries. A potential scope for improving the fibre-correction algorithm is to include checks for mutual exclusion of muscle fibre bundles. Furthermore, the Gaussian kernel size p, which is assumed to be constant here, can be modelled as a function of mesh density. The value of p should be inversely proportional to the mesh density, i.e., a dense mesh will have a smaller value of p than a coarse mesh. For the proposed model, these errors are expected to play a very minor role since the majority of muscle fibres in the model tend to have distinct lines of action, where relatively negligible inconsistencies and error in fibre orientations do not contribute to large differences in the direction and magnitude of the generated muscle force.

Another source of error in modelling the residual limb is the fact that tendinous structures cannot be extracted directly from DT-MRI images. But the tendinous structures are responsible for transferring the forces generated by skeletal muscles to the skeleton. However, due to the inherent limitations of the DT-MRI scans, tendons cannot be tracked. To do so, scanners with large magnetic field strength, and extremely high pixel resolution are necessary (cf. Gupta et al., 2010; Karampinos et al., 2012). In Gupta et al. (2010), excised rabbit tendons were tracked using DT-MRI with in-plane pixel resolution of 200 µm, in a 11.74 T scanner. Scanning with such high resolution is not feasible for humans due to the extremely high scan duration. As a result, models created with the proposed workflow lack tendons, which is not a limitation of the proposed methodology but rather of the current scanner technology. Tendons may, however, be laboriously hand-segmented from T1- or T2-weighted MRI scans, and imported into the limb model.

Despite these drawbacks, having good knowledge in anatomy for grouping the muscle fibres is helpful in resolving most fibre-bundling-related problems in MedINRIA. It remains to be verified in future experiments, if higher resolution DT-MRI scans will mitigate partial volume effects to such an extent that the boundaries between muscle fibres are distinctly visible, leading to easier classification of mutually exclusive fibre bundles. If not, an alternative would be to overlay the muscle masks over T1- or T2-weighted MRI scans, and correct the mask boundaries before creating the model. It is common for surgical amputations to result in extensive modifications of the limb musculature, which results in a unique residual limb anatomy in each patient. In such cases, conventional segmentation of the residual limb with MRI or CT scans, with the intention of creating a detailed limb model, is rather difficult. With the proposed technique, the anatomy of this new musculature is evident from its fibre distribution with which individual muscles of the residual limb can be easily modelled. This ease of use of the proposed workflow is obvious given the relatively short time (about 20 min) that is required to model the highly detailed patient-specific FE mesh of the stump.

## 4.2. FE Mesh and Continuum-Mechanical Model

The use of realistic constitutive laws is, like for almost any other subject-specific computational model, a challenge and a source of error. Here, the soft tissue parameters of the continuum-mechanical model were adapted from Röhrle et al. (2017), where the skeletal muscle parameters were optimised for healthy upper limb muscles. Unlike healthy muscles, amputated muscles are incapable of generating normal muscular force owing primarily to reduced physiological cross-sectional area and altered insertion points of transected muscles, among other factors. Therefore, the soft tissue parameters of the stump were adapted from the healthy parameter set by simply weakening them. The correctness of such a set is not guaranteed here. Extensive experimental methods, which currently are active research topics (cf. Sengeh et al., 2016), are required to accurately determine the correct parameter set for the stump.

Further, the contact between soft tissues in the residual limb were idealised through common nodes at the interfaces between individual parts of the FE mesh. This assumption is comparable to the role of connective tissues that bind the soft tissues together, which leads to favourable conditions in the FE analyses. For example, through the use of common nodes, frictional or sliding contact between the individual parts, which is difficult to quantify, is eliminated. This results in a quicker computation. Another benefit of this contact-less model is that there are no nonsensical gaps in-between the soft tissues and also between the bone and surrounding soft tissues during the simulations. Similarly, the sticky contact between the inner surface of the prosthetic liner worn by the subject and the skin was idealised through common nodes at their surface.

## 4.3. Boundary Conditions and Results of Bipedal Stance

The models of residual limb and prosthetic socket were generated from DT-MRI and T1-MRI scans, respectively. This was due to the fact that the socket, which lacked water content, was transparent in the DT-MRI scans and therefore could not be segmented. The correct alignment between the residual limb and socket was achieved by manually segmenting and aligning the femur in both models. This manual segmentation of the femur in Simpleware ScanIP is trivial and quick due to large differences in the pixel intensities of the bone and the surrounding soft tissues. Following this alignment procedure, the initial position of the socket for the socket donning simulation was chosen such that the stump and the socket were not in initial contact. The duration of this simulation was assumed to be 10 s, which was not experimentally-motivated. The dynamic socket donning simulation pre-conditioned the limb for the bipedal stance simulation that followed. The duration for bipedal stance was chosen to be 2 h, which is motivated by the strain-based cell necrosis studies by Gefen et al. (2008), where tolerance of tissues to compressive strains decreased significantly between 1 and 3 h upon loading.

The fused muscles, which were generated for the sake of comparison with the individual-muscle model, were created by fusing the individual muscle masks that were created with the proposed workflow. This resulted in a bone-fat-muscle complex, which still contained an equivalent or more segmentation layers than the state-of-the-art fused-muscle models, where patientspecific stump has been either segmented into bone-muscle (cf. Portnoy et al., 2008; Cagle et al., 2018) or bone-muscle-skin layers (cf. Sengeh et al., 2016). However, in the above studies, no distinction was made between fat and muscles in the soft tissue complex despite their properties being largely dissimilar. And, in order to remain consistent with the above state-of-the-art models, tissue anisotropy was ignored in the fused-muscle model.

The differences between the individual- and fused-muscle models are immediately clear upon comparing the magnitude and distribution of stresses on the muscles. A 4-fold difference in the peak stresses is present between two models, with the peak stresses in the individual- and fused-muscle models being 63.5 and 14.34 kPa, respectively. This large difference may be attributed to the contribution of passive fibre stiffness to the global stiffness matrix in the individual-muscle model, which is absent in the fused-muscle model. With regard to

the stress distribution, the stresses in the fused-muscle model are easily spread over the single muscle volume, resulting in lower magnitude and larger spread of the stresses while those in the individual-muscle model are concentrated at the soft tissue interfaces. In contrary to the large stress differences in the deep tissues of the residual limb, the difference in the interface stresses between the fused- and individual-muscle models resulted in only a small difference, i.e., 5.4 kPa in the fused-muscle model and 4.9 kPa in the individual-muscle model. The magnitude of the mean interface stresses in both these models are similar to those obtained by Lacroix and Ramírez Patiño (2011). However, there is a marked difference between the peak interface and deep-tissue stresses within each model, e.g., in the individualmuscle model, the peak interface and deep-tissue stresses after the bipedal stance analysis were 24.1 and 63.5 kPa, respectively. Hence, the study of interface stresses, which is necessary to ensure good health of the skin, may not be a sufficient measure of the health of deep tissues in the stump. In the above results, the magnitude and distribution of stress in the soft tissues corresponded to those of strains, and therefore, only stresses

were reported here in order to be consistent with the rest of the literature. Since the DTI model was strain-based, which in this case corresponded to the stresses, the regions of skeletal muscle predicted to be affected by DTI was comparable to the stress distribution for all three cases. As expected, bipedal stance simulation of the residual limb model with original socket predicted negligible tissue damage. However, the volume of tissues in the fused-muscle model that was predicted to have been damaged was less than half of that predicted to be damaged in the individual-muscle model. The proposed hypothesis that individual-muscle models are advantageous and provide more insight in stump-prosthesis interaction studies is hereby verified.

## 5. CONCLUSION

The focus of this work was on developing an efficient modellingsimulation-analysis workflow to investigate stump-socket interaction during bipedal stance using patient-specific, threedimensional, continuum-mechanical, finite element residual limb model, in which the model is generated with minimal

user intervention. A nonlinear, hyperelastic, transversely isotropic continuum-mechanical model of the soft tissues was implemented in LS-DYNA to measure stresses in the residual limb. With the implemented FE model, potential sites of deep tissue injury were predicted. Comparison of the proposed individual-muscle model with the state-of-the-art residual fused-muscle/fat models reveals that the fusedmuscle models underestimate the stresses and volume of injured tissues. Tissue anisotropy contributes to both active and passive stresses, and must be considered when studying socket-stump interactions. Moreover, for forward dynamics simulations, the proposed model provides an efficient method to obtain FE models of skeletal muscles, including its fibre orientations. In the forthcoming studies, it is envisioned to extend this workflow to model tendons and to include the effects of muscle contractions for various dynamic boundary conditions.

## ETHICS STATEMENT

This study was carried out in accordance with the recommendations of ICH-GCP guidelines of, The ethics committee of the medical faculty of University of Tübingen and University Hospital Tübingen with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the the ethics committee of the medical faculty of University of Tübingen and University Hospital Tübingen.

## REFERENCES


## AUTHOR CONTRIBUTIONS

ER worked out most of the technical details, performed the numerical simulations and its analysis, and drafted the manuscript. ER and BD performed the image processing while OA was involved in implementing the constitutive laws. S-YC and LG were in charge of the data collection, and GS and FS were in charge of the MRI imaging. OR devised, conceptualised and supervised the project and drafted parts of the manuscript. All authors provided critical feedback, and agree with the content of the manuscript.

## FUNDING

The research leading to these results has received funding from the European Research Council under the European Union's Seventh Framework Programme (FP/2007–2013)/ERC Grant Agreement No. 306757 (LEAD), and through Fraunhofer as part of the Attract research grant Virtual Orthopedic Lab and the MAVO EMMA-CC.


ground walking compared to able-bodied individuals. Clin. Biomechan. 32, 157–163. doi: 10.1016/j.clinbiomech.2015.11.018


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer AD declared a past collaboration with two of the authors, ER and OR, to the handling Editor.

Copyright © 2018 Ramasamy, Avci, Dorow, Chong, Gizzi, Steidle, Schick and Röhrle. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Benchmarking Stability of Bipedal Locomotion Based on Individual Full Body Dynamics and Foot Placement Strategies–Application to Impaired and Unimpaired Walking

Khai-Long Ho Hoang<sup>1</sup> \*, Sebastian I. Wolf <sup>2</sup> and Katja Mombaur <sup>1</sup>

*<sup>1</sup> Optimization, Robotics and Biomechanics, Institute of Computer Engineering, Heidelberg University, Heidelberg, Germany, <sup>2</sup> Clinic for Orthopedics and Trauma Surgery, Heidelberg University Hospital, Heidelberg, Germany*

#### Edited by:

*Ramana Vinjamuri, Stevens Institute of Technology, United States*

#### Reviewed by:

*Marco Iosa, Fondazione Santa Lucia (IRCCS), Italy Alessandro Panarese, L'Istituto di BioRobotica, Scuola Sant'Anna di Studi Avanzati, Italy*

#### \*Correspondence:

*Khai-Long Ho Hoang KhaiLong.HoHoang@ ziti.uni-heidelberg.de*

#### Specialty section:

*This article was submitted to Bionics and Biomimetics, a section of the journal Frontiers in Robotics and AI*

Received: *15 March 2018* Accepted: *19 September 2018* Published: *12 October 2018*

#### Citation:

*Ho Hoang K-L, Wolf SI and Mombaur K (2018) Benchmarking Stability of Bipedal Locomotion Based on Individual Full Body Dynamics and Foot Placement Strategies–Application to Impaired and Unimpaired Walking. Front. Robot. AI 5:117. doi: 10.3389/frobt.2018.00117* The principles underlying smooth and effortless human walking while maintaining stability as well as the ability to quickly respond to unexpected perturbations result from a plethora of well-balanced parameters, most of them yet to be determined. In this paper, we investigate criteria that may be useful for benchmarking stability properties of human walking. We perform dynamic reconstructions of human walking motions of unimpaired subjects and subjects walking with transfemoral prostheses from motion capture recordings using optimal control. We aim at revealing subject-specific strategies in applying dynamics in order to maintain steady gait considering irregularities such as deviating gait patterns or asymmetric body segment properties. We identify foot placement with respect to the Instantaneous Capture Point as the strategy globally applied by the subjects to obtain steady gait and propose the Residual Orbital Energy as a measure allowing for benchmarking human-like gait toward confident vs. cautious gait.

Keywords: benchmarking, capture point, foot placement, multibody dynamics, optimal control, stability, transfemoral prosthesis, walking

## 1. INTRODUCTION

Human and human-like walking motions form an important and challenging class of motions with respect to dynamics and control. The development of efficient measures for benchmarking bipedal locomotion is an important topic for many fields of research, ranging from human motion studies in biomechanics or medical fields to the development and control of humanoid robots, exoskeletons, prostheses etc. Among others, benchmarking allows to define walking standards, measure progress of human walking during therapy, define training goals or to compare different robot platforms, alternative prostheses models or tunings for a patient, different exoskeleton technologies or control algorithms, etc. Good benchmarking measures allow to shift from a purely qualitative comparison of motions to a more useful quantitative one that precisely assesses the quantities in a motion that are considered relevant for quality, thus giving helpful insights for improvement.

There are many different types of benchmarking measures. In the KoroiBot project, which studied human locomotion with the intention to improve walking qualities of humanoid robots, we have distinguished three different groups of key performance indicators (KPIs) to benchmark locomotion (see Schubert et al., 2014). The first group concerns technical indicators for walking performance which are equally important for all the fields discussed above and which have also been discussed in Torricelli et al. (2015). The second group concerns computational properties of the algorithms used for generating and controlling locomotion, so it is a criterion which is mainly relevant for technical systems, i.e., robots or controlled technical assistive devices. The third group contains high level KPIs and aims to asses to which amount motions of robots are human-like, a concept that can also be extended to investigate the effect of impairments or assistive devices to human walking motions. This evaluation is, of course, to some extent related to the technical performance evaluation on the lower level.

Technical performance indicators include elementary characteristics of walking performance such as walking speed in different walking scenarios, step length and step width, different measures for energy consumption, efficiency and cost of transport, as well as measures related to the walking scenario, such as manageable stair height, slope inclination or roughness of terrain. These criteria are straightforward to define and in most cases also quite easy to measure on a given system and scenario. Technical performance indicators however also include measures of stability and robustness of motions which are less straightforward to define and, as we will review in the following paragraph, for which no uniform consensus exists yet.

Maintaining stability and being robust also to larger perturbations that might occur, is one of the major objectives during a locomotion task. Human walking is characterized by a repetitive sequence of well-coordinated motions of the upper and lower limbs which carry the human body into a desired direction. The stance leg serves as a body support while the swing leg is moved toward the next support location. As described in Perry and Burnfield (2010), the legs alternate their roles in a reciprocal manner until the subject intents to stop. According to Winter (1995), due to the elevated center of mass (COM) balancing over the small contact surface established by the feet, the human body is an inherently unstable system for which stability is maintained by a continuously acting control system as well as by exploiting the whole-body dynamics. Stability of human walking describes the ability to maintain the intended locomotion task without falling.

The focus of this paper will be to evaluate several criteria for benchmarking stability of bipedal locomotion. We are particularly interested in criteria that can be applied to all fields of application listed above. As we will outline in the following paragraph, several approaches exist for controlling stability of humanoid robots - which, however, are not based on a criterion relevant for humans - and on the other side for evaluating perturbation reaction in human walking a posteriori which are unsuitable to predict or control behavior of robots.

## 1.1. Related Research

An intuitive approach to bipedal locomotion defines any gait as stable as long as it does not lead to a fall. The set of all states of a walker which leads to stable gait according to this definition has been termed the viability kernel by Wieber (2002). Due to the vast computational effort required to compute this set as well as the lack of methods to generate control strategies based on this definition, the viability kernel has not yet found any practical application.

Stability in human walking has also been approached by Mombaur et al. (2001) by describing the human body as a hybrid dynamic system and examining its properties in terms of Lyapunov stability. This approach has led to some insights into the self-stabilization properties of human locomotion mechanics. However, to fully understand Lyapunov stability properties of human movement, fundamental knowledge about the feedback loops which are active during human locomotion would be required and would have to be included in the model, but it is not available yet. Modeling the human response to unpredictable changes in the environment in terms of a hybrid dynamic system has so far been an unsolved task (see Bruijn et al., 2013).

Other approaches which have been widely used in both clinical applications and the research community working on humanoid robotics derive control laws based on ground reference points which require a minimal amount of computational effort to be obtained and can be evaluated in real-time (Popovic et al., 2005). Other ground reference points which consider the velocities of the bipedal walker enable to explain foot placement and fall prevention as a response to sudden pushes Pratt and Tedrake (2006).

Maintaining the Ground Center of Mass (GCoM), i.e., the projection of the Center of Mass on the ground plane, within the borders of the Base of Support (BoS), i.e., the convex hull spanned by the contact points of the system with the ground, can be used as a very simple requirement for static stability of bipedal systems (Berns et al., 1994; Goswami, 1999). However, in our context, this approach is only applicable for static poses or quasi-static motions and is not feasible for describing the stability of dynamic bipedal motions such as human walking.

A very popular approach to stability in bipedal locomotion is based on the Zero Moment Point (ZMP), introduced by Vukobratovic and Branislav (2004), which is defined as the ground reference point in which the resulting horizontal moments from the inertial and gravitational forces of the bipedal system vanish. In case the bipedal system does not slip and no other external forces than the ground reaction forces act on it the ZMP coincides with the Center of Pressure (CoP). Maintaining the ZMP within the borders of a subset of the BoS has been used by various projects to control the walking motion of a humanoid robot (e.g., Sakagami et al., 2002; Wang et al., 2014). However, this approach leads to very conservative motions which do not resemble the dynamic appearance of human gait. In fact, human walking is characterized by ZMP locations very close to the borders of the actual BoS. In addition, the ZMP only reflects the current state of the system and does not provide any meaningful information to predict falls.

This work focuses on describing the human foot placement strategy in terms of the velocity-based Capture Point (CaP), introduced by Pratt et al. (2006), Koolen et al. (2012), and Hof et al. (2005), since it treats gait phenomena as future events and provides a versatile method to predict and evaluate the gait of individual subjects. The Capture Point indicates the foot location which should be anticipated after a push to come to complete stop. It considers the minimum time required to perform a step as well as the step's maximum reachable distance and has been implemented in the gait control of humanoid robots in Englsberger et al. (2015), Koolen et al. (2013), and Krause et al. (2012) and described as a recovery strategy applied by humans as a response to unexpected perturbations in everyday situation by Aftab (2012). Furthermore, based on the Capture Point, strategies to adapt temporo-spatial gait parameters to varying environmental conditions and asymmetric step lengths in transtibial prosthetic gait have been associated with functional compensation strategies in order to reduce the risk of falling backwards in Hak et al. (2013a,b, 2014). Disregarding the maximum step length and the minimum step duration, the Capture Point is referred to as the Instantaneous Capture Point (ICaP).

In addition, this work considers the Angular Momentum applied by the human walkers about the center of mass. It has been observed by Herr and Popovic (2008) that during straight and upright walking the average angular momentum about the principal axes remains close to zero. As discussed in Mombaur and Vallery (2018), the oscillations of the angular momentum around zero, even though small, are not small enough to be ignored and are contributing to the nature of walking, which is also in accordance with the observation of the virtual pivot point located above the center of mass (see Maus et al., 2010).

## 1.2. Contributions of This Paper

In this work, we discuss different benchmarking criteria for their applicability to quantify the stability of bipedal locomotion. We propose a combined assessment of bipedal gait based on an extension of the Capture Point as well as the full body angular momentum as a benchmarking tool for human walking. The application of this method is demonstrated by computing data for two unimpaired subjects and one subject walking with a prosthesis.

We hypothesize that, in order to maintain a steadily stable walking motion, the desired foot location approached by unimpaired humans while moving the swing leg forward is correlated to the Instantaneous Capture Point (Pratt et al., 2006; Koolen et al., 2012). Furthermore, we conjecture that humans with deviating habitual gait patterns, asymmetric body properties or limb replacements aim at a similar stability strategy by adjusting their gait dynamics according to the modified dynamic and actuatory properties. This implies that a symmetric foot placement strategy is maintained by applying asymmetric gait dynamics. In order to analyze human gait for these strategies we reconstruct the dynamics of the human walking motion from motion data obtained in a gait laboratory. Introducing the Residual Orbital Energy, we are able to simultaneously analyze the reconstructed motions for the underlying whole-body dynamics as well as the foot placement strategy with respect to the Instantaneous Capture Point.

The application of the proposed method results in distinguishable gait dynamics for impaired and unimpaired humans with symmetric and asymmetric body proportions leading to the common objective, namely to move the swing leg toward the Instantaneous Capture Point. In order to validate the method, however, walking motions of many more subjects need to be investigated.

## 2. METHODS

In this paper, we present a first study of the proposed benchmarking criteria. This includes their evaluation on existing walking data of a few subjects—two unimpaired subjects and one subject with a prosthesis—as a first indicator on the performance of the proposed method. A large statistical analysis of the benchmarking criteria is beyond the scope of this paper and will be subject of our future work.

The dynamics of the subjects' walking motion are reconstructed from recorded motion capture data using individualized multibody models of the subjects and optimal control methods in a least-squares sense. This approach ensures that the dynamics of the model are satisfied throughout the entire time horizon rather than only on discrete time steps. Based on the reconstructed motions, the whole-body dynamics as well as the temporo-spatial gait parameters are compared between the subjects. The analysis focuses on the behavior of the ICaP for each subject and, in particular, on the characteristics of each subject's foot placement with respect to the ICaP right at the heel strike event.

## 2.1. Motion Recordings

The recordings include a full stride beginning with the toe off of the left foot of three subjects: (A) an unimpaired female subject, (B) an unimpaired male subject as well as (C) a male subject walking unilaterally with a transfemoral prosthesis on the right side. Some characteristics of the subjects are included in **Table 1**.

The kinematic part of the walking motion of the subjects has been recorded using marker-based motion capturing. The recordings have been gathered in the Clinic for Orthopedics and Trauma Surgery at Heidelberg University Hospital<sup>1</sup> located in the Heidelberg University Orthopedic Hospital equipped with a Vicon Motion Systems Ltd.<sup>2</sup> MoCap system and three Kistler Instrumente GmbH<sup>3</sup> force plates. An extended version of the Plug-In Gait marker set provided by Vicon Motion Systems Ltd. (2010) has been chosen to enable the recording of the full body motion for all subjects, as shown in **Figure 1**.

The subjects' gait has been recorded at self-selected walking speeds. The gait appearance of the unimpaired subjects can be considered healthy, symmetric and without any physical limitations. The impaired subject has been individually fitted with a prosthetic knee which also includes a customized socket and appropriately selected prosthetic components. This subject has been provided on the right side with the Össur hf. (2017a) Rheo prosthetic knee and the Össur hf. (2017b) Vari-Flex foot. His gait appears smooth and symmetric.

<sup>1</sup>http://www.vicon.com

<sup>2</sup>http://www.kistler.com

<sup>3</sup>Heidelberg MotionLab, http://www.heidel-motionlab.de

#### TABLE 1 | Subject details.


## 2.2. Models

The human body is modeled as a 13-segment multibody model with 34 degrees of freedom (DoFs). The rigid bodies represent the body segments pelvis, left/right thigh, left/right shank, left/right foot, mid/upper trunk, left/right upper arm, left/right lower arm, left/right hand, and head, respectively.

The rigid bodies are connected by the 3-DoF joints right/left hip, right/left ankle, Lumbo-Sacral joint, right/left shoulder, and Cervicale as well as the 1-DoF joints right/left elbow and right/left knee, respectively. The absolute translation and orientation of the entire system with respect to the global frame in Euclidean space is defined by the six DoFs for the absolute translation and orientation of the pelvis segment.

The model is based on the 16-segment multibody model with 43 DoFs illustrated in **Figure 2**. During human walking, no significant motion occurs in the Xiphoid joint and the right/left wrists. Hence, zero DoFs are assumed between the middle and upper trunk as well as between the lower arms and the hands, respectively, and the model can be reduced to the model used in this study. The dynamic model parameters for all subjects were obtained using the regression equations provided by de Leva (1996). In addition, the dynamic model parameters for the prosthetic leg have been obtained by simple experiments involving scaling, balancing and oscillating the prosthesis. For Subject C the prosthetic leg's mass is approximately 35% the mass of his opposite leg. The model establishes ground contact with the feet which are represented by rigid triangular segments spanned by the three contact points heel, hallux and meta5 as shown in **Figure 3**.

## 2.3. Equations of Motion

Depending on the current gait phase, the mechanical system described above is subject to changing contact properties and can be described by a set of differential algebraic equations

$$
\dot{q} = \mathbf{v} \tag{1}
$$

$$
\dot{\nu} = a \tag{2}
$$

$$
\begin{pmatrix} H(q) & G(q)^T \\ G(q) & \mathbf{0} \end{pmatrix} \begin{pmatrix} a \\ -\lambda \end{pmatrix} = \begin{pmatrix} -C(q,\nu) + \tau(q,\nu) \\ \nu(q,\nu) \end{pmatrix} \tag{3}
$$

with the symmetric and positive-definite mass matrix **H**, the generalized non-linear effects **C**, the generalized internal forces τ , the differential variables **q** and **v** for the positions and the velocities, respectively, the algebraic variables **a** and λ for the accelerations and the contact forces, respectively, as well as the contact Hessian

$$\mathcal{Y}(q,\nu) = -\dot{\mathcal{G}}(q)\nu = -\nu^T \frac{\mathrm{d}\mathcal{G}(q)}{\mathrm{d}q}\nu. \tag{4}$$

with segment COM positions and local coordinate frames, (B) Full human body with transfemoral prosthesis (yellow).

and additional constraints on position and velocity level

$$\mathbf{g}(q) = \mathbf{0} \tag{5}$$

$$\mathbf{G}(q)\dot{q} = \mathbf{0} \tag{6}$$

For non-redundant constraints **g**(**q**) the contact Jacobian **G**(**q**) has full rank and (3) can be uniquely solved.

Whenever the model gains contact with the ground, the perfectly rigid foot model causes discontinuous transitions from the generalized velocity **v** <sup>−</sup> before the collision to the generalized velocity **v** <sup>+</sup> after the collision. The transition is determined by

$$
\begin{pmatrix} \mathbf{H}(\mathbf{q}) \ \mathbf{G}(\mathbf{q})^T \\ \mathbf{G}(\mathbf{q}) \ \mathbf{0} \end{pmatrix} \begin{pmatrix} \boldsymbol{\nu}^+ \\ -\mathbf{A} \end{pmatrix} = \begin{pmatrix} H(\mathbf{q})\boldsymbol{\nu}^- \\ \mathbf{0} \end{pmatrix} \tag{7}
$$

where the first line determines the change of the system's momentum caused by the collision and 3 denotes the contact impulse.

## 2.4. Identification of Walking Motions by Means of Optimal Control

In this paper, unimpaired and prosthetic human walking motions are reconstructed by fitting the motions of subjectspecific dynamic models to motion capture data by formulating and solving a large-scale multi-phase optimal control problem (OCP) in a least-squares (LSQ) sense. In order to minimize the dimension of the optimal control problem the motion is reasonably fitted along the generalized coordinates of the model instead of the Cartesian coordinates.

The reference motions for the optimal control problems are created by converting the measured motion capture data from marker trajectories in Cartesian space into trajectories in joint angle space. This can be performed by approximating the motion of a subject-specific multibody model such that the distance between virtual markers defined on the model and the appropriate measured marker positions are minimized for each time frame considered. The fit is performed in a leastsquares sense and considers the entire kinematic chain of the multibody model (see Sugihara, 2011; Felis, 2015). In this study, the reference motion has been fitted with an average matching error over all markers of 2.0 cm ± 1.1 (Female unimpaired), 2.0 cm ± 1.3 (Male unimpaired), and 1.6 cm ± 0.8 (Male impaired), respectively.

The optimal control problem is divided into 8 phases according to the phases of a whole gait cycle as well as four transitions to account for the discontinuities occuring when ground contact is established, see **Figure 4**. The gait phases can be distinguished by the different contact configurations between the human and the environment, expressed in the model as nonlinear point constraints, which determine the dynamics of the system.

The reference motions can be summarized in a set of time discrete postures expressed in terms of the generalized coordinates **q** ik <sup>j</sup> <sup>∈</sup> <sup>R</sup> ndof , j = 0, . . . , m at the time instances t0, . . . , t<sup>m</sup> ∈ R. The least-squares problem is then described for the nph = 8 model stages by

$$\min\_{\mathbf{x}(\cdot), \mathbf{u}(\cdot)} \sum\_{j=1}^{m} \frac{1}{2} ||\mathbf{q}\_j^{\mathrm{ik}} - \mathbf{q}(t\_j)||\_2^2 + \boldsymbol{\gamma}\_{\boldsymbol{\mu}} ||\mathbf{W}\boldsymbol{u}(t\_j)||\_2^2 \tag{8}$$

subject to:

$$
\dot{\mathbf{x}}(t) = f\_j(\mathbf{x}(t), \mathbf{u}(t)), \tag{9}
$$

$$\mathbf{x}(t\_i^+) = h\_i(\mathbf{x}(t\_i^-)),\tag{10}$$

FIGURE 4 | Gait phases for the optimal control problem. Transition phases (gray) are introduced to account for the discontinuities occurring to the velocities **q**˙ at ground collision when a new contact is gained.

$$0 \le \mathbf{g}\_i(t, \mathfrak{x}(t), \mathfrak{u}(t)),\tag{11}$$

$$0 = r^{\text{eq}}(\mathfrak{x}(0), \dots, \mathfrak{x}(T), \mathfrak{u}(0), \dots, \mathfrak{u}(T)),\tag{12}$$

$$0 \le r^{\text{ineq}}(\mathfrak{x}(0), \dots, \mathfrak{x}(T), \mathfrak{u}(0), \dots, \mathfrak{u}(T)), \tag{13}$$
 
$$\text{for } t \in [\mathfrak{r}\_{i-1}, \mathfrak{r}\_i], i = 1, \dots, 8, \mathfrak{r}\_0 = 0, \mathfrak{r}\_8 = T$$

$$\text{The first-order coupling between the two-dimensional } \mathcal{N} \text{-matrices is the only possible } \mathcal{N} \text{-matrices with } \mathcal{N} = \{0, 1, 2, \dots, N\} \text{ and } \mathcal{N} = \{0, 1, 2, \dots, N\}.$$

with the differential states

$$\mathbf{x}(t) = \begin{pmatrix} q(t) \\ \dot{q}(t) \end{pmatrix} \in \mathbb{R}^{2n\_{\text{dof}}} \tag{14}$$

and the model's generalized coordinates **q**(t) ∈ R <sup>n</sup>dof , generalized velocities **q**˙(t) ∈ R <sup>n</sup>dof and the controls **u**(t) ∈ R nact which represent the torques that act directly on the model's joints.

The first term of the objective function Equation (8) minimizes the sum of squared differences between the model joint angles **q**(t)<sup>j</sup> and the joint angles **q** ik j from the inversekinematics analysis of the motion capture recordings. The second term regularizes the problem and accounts for the different magnitudes of the joint forces weighted by the diagonal matrix **W** = diag(w<sup>l</sup> ),w<sup>l</sup> > 0, l = 1...nact and the factor γu. The ODEs Equation (9) describe the model dynamics in each phase where the right hand sides **f** i : R n<sup>x</sup> × R n<sup>u</sup> × R <sup>n</sup><sup>x</sup> are characterized by the different constraint properties. Discontinuities in the generalized velocities **q**˙(t) that occur due to the model specific perfectly rigid impact at ground collision in case of touchdown events are handled using the phase transition functions Equation (10). Upper and lower bounds for the differential states **x**(t) as well as the controls **u**(t) are covered by the path constraints Equation (11). The path constraints for the generalized coordinates **x**(t) are chosen to reflect the ranges for the typical walking motions. Additional constraints that, e.g., ensure physical feasibility such as unilateral ground contacts as well as switching conditions for phases are contained in the interior point constraints Equations (12, 13) which distinguish the several gait phases from each other.

Due to the hybrid dynamic character of the computed walking motions the optimal control problem is solved using the direct multiple-shooting method and a piecewise linear control discretization. The multiple-shooting state parameterization transforms the original boundary value problem into a set of initial value problems with corresponding continuity and boundary conditions. The multiple-shooting method is implemented in the software package MUSCOD-II (Bock and Plitt, 1983).

## 2.5. Angular Momentum

Human walking is characterized by the alternating swinging motions of the upper and lower limbs. Counter-rotating motions of the upper and lower body are applied to balance the whole body angular momentum around the longitudinal axis and enable the human to walk straight. Oscillating angular momenta are applied in the frontal plane to facilitate the transfer of the body weight from one leg to the other. We analyze the angular momentum applied by the subjects in the horizontal plane (i.e., with respect to the vertical axis) by the motions of the upper and lower body for their contribution to the full body angular momentum in order to reveal individual strategies to compensate for asymmetric dynamic properties of the body or unbalanced habitual gait patterns in steady walking.

## 2.6. Foot Placement

An intuitive approach to quantify and control stability in human walking is motivated as a response to an unexpected loss of balance, e.g., when a sudden perturbation occurs. Since walking can be considered a perpetual falling motion followed by a well-timed and well-placed step, we analyze the walking motion based on the Capturability concept introduced by Pratt et al. (2006), Hof et al. (2005) and Koolen et al. (2012) and the herein proposed location of the Instantaneous Capture Point (ICaP)

$$r\_{\rm icap} = r\_{\rm cop} + \frac{\dot{r}\_{\rm com}}{\alpha\_0}.\tag{15}$$

with the current position of the Center of Pressure **r**cop, the velocity **r**˙com of the pendulum's mass and its angular eigenfrequency ω<sup>0</sup> = p g/l with the pendulum's length l. In addition, we study the Orbital Energy Elip of the Linear Inverted Pendulum Model (LIPM) underlying the walking system which we normalize by dividing the quantities related to positions and lengths by each subject's leg length (hip to ankle) such that

$$E'\_{\rm lip} = \frac{1}{2} \dot{\mathbf{r}}'^2\_{\rm com} - \frac{1}{2} (\mathbf{r}'\_{\rm com} - \mathbf{r}'\_{\rm op})^2 \omega\_0^2. \tag{16}$$

We analyze the normalized orbital energy E ′ lip of the LIPM at a given time instance taking into account the current COM height and assuming that the velocity vector is horizontal. In particular, we are interested in E ′ lip at the time instances right after the heel strike and propose the expression Residual Orbital Energy E′ res. We use this parameter to characterize the specific gait pattern of the subject established by the self-selected combination of the step length, the step width and the magnitude of the ground collision impact at heel strike which causes a loss of kinetic energy and, thus, gait velocity. The link between the gait velocity and the Orbital Energy is established in the first term of Equation (16) and included into the model by Equation (10).

## 3. RESULTS AND DISCUSSION

## 3.1. Angular Momentum and Foot Placement

The results for the angular momentum applied by the subjects in the upper body, lower body and full body, respectively, in the frontal, sagittal, and horizontal planes are shown in **Figure 5**. **Figure 6** shows that the subjects choose their step locations such that the ICaP is approximately reached by the forefoot to midfoot of the anterior foot in sagittal direction. The subjects maintain a well-balanced gait in terms of the ICaP being smoothly moved between both feet in lateral direction to facilitate lateral oscillation from one stance leg to the other.

Subject A has balanced momenta in the frontal and horizontal plane during the left swing phase (**Figure 5A**). However, she diverges her momenta in both planes during the right swing phase. During that phase, the upper body has no significant contribution to the full body momentum to balance the momentum applied by the lower body. This causes Subject A to experience a strong tilt and slight turn to the left. She adjusts her step location accordingly to follow the ICaP by placing her swing foot further into the same direction, see **Figure 5A**.

The angular momentum of Subject B in the frontal plane is regular and symmetric mostly established by a balanced pendulum motion of the upper body (**Figure 5B**). The angular momentum in the horizontal plane can be considered strongly unbalanced in the right swing phase. This leads to a stronger external rotation of the right foot compared to the left foot rotation (**Figure 6B**). However, since the greater amount of angular momentum is applied by the upper body, balancing the full body momentum can be achieved by exploiting friction between the stance foot and the ground. Applying this strategy, Subject B achieves a perfectly straight walking path.

Subject C shows well-balanced angular momenta in the frontal and horizontal plane in the time instances right before the heel strike events of both feet (**Figure 5C**). Although walking with a transfemoral prosthesis on his right side, he adjusts his upper

FIGURE 6 | Footprints of the right (red) and left (green) feet as well as the trajectories of the ICaP (solid) and the ground projection of the COM (dashed) in the *xy*-plane. The ICaP positions (◦) are illustrated right at the heel strike of the right and left feet, respectively, along with the COM positions (▽) at the same time

TABLE 2 | Normalized residual orbital energy *E* ′ res [1/*s* 2 ] at left and right hell strike (HS) and the average value.


body motions to establish strongly asymmetric angular momenta which compensate for the equally asymmetric momenta from the lower body. Applying this strategy, Subject C achieves a well-balanced and straight walking path, see **Figure 6C**.

The angular momenta in the sagittal plane have typical profiles for all subjects. The whole body momentum is created by forward swinging motions of the arms and legs and is partially dissipated at heel strike.

## 3.2. Residual Orbital Energy

The normalized residual orbital energy E ′ res of each subject right at the heel strike events as well as their average values are summarized in **Table 2**. As can be seen in Equation (16), a greater value for E ′ res is caused either by greater gait velocities, greater CoP-ICaP distances, or both. In these cases, capturing from sudden disturbances become increasingly difficult. Accordingly, lower values for E ′ res are caused by lower gait velocities, shorter CoP-ICaP distances, or both and capturing from disturbances becomes less difficult.

Considering the Residual Orbital Energy E ′ res, the values in **Table 2** reveal a symmetric behavior for Subject A. In contrast, Subject B shows a strongly asymmetric behavior mostly caused by an asymmetric application of upper body angular momentum (**Figure 5B**) and, thus, an asymmetric gait velocity right at the heel strike events. The residual orbital energy E ′ res at the heel strike event of the right (prosthetic) foot of Subject C is slightly less than at left heel strike. We might assume that Subject C reduces his gait velocity and, thus, his impact at ground collision in order to prevent pain at the socket-stump interface.

Using the Residual Orbital Energy E ′ res in combination with the foot placement strategies with respect to the ICaP as a benchmarking tool, we consider the individual walking motions of the Subjects A and B (both unimpaired) as irregular gait and the walking motions of Subject C (walking with a prosthesis) as conscious gait.

## 4. CONCLUSION

In this work, we have investigated benchmarking criteria that help to quantitatively assess the stability of walking motions. As a first test, we have applied them to the walking motions of two unimpaired subjects and one subject walking with a transfemoral prosthesis which have been reconstructed from motion capture recordings using multibody dynamics and optimal control methods. The reconstructed walking motions have been analyzed for their dynamics and findings are gathered on how unbalanced habitual gait patterns can lead to irregular walking motions. On the other hand, the analysis provides insights into the individual strategies applied by the subject walking with the prosthesis to compensate for his asymmetric dynamic properties of the lower limbs and achieve a perfectly balanced walking motion.

For all subjects, the ICaP is shown to be consistently approached by the swing foot even if the walking paths deviate in lateral direction. The subjects choose their step locations such that the ICaP is located in anterior-medial direction of the foot and maintain a Residual Orbital Energy at heel strike >0 in order to facilitate the forward propulsion and lateral oscillation characteristic for human walking. The Residual Orbital Energy combines the subject's distance maintained between the CoP and the ICaP at each step with the gait velocity which, in human walking, is controlled to a great part by the rate of change of angular momentum.

In our proposed method we simultaneously interpret foot placement with respect to the ICaP as well as the Residual Orbital Energy. By deconstructing the walking motion into these parts we are able to reveal hidden phenomena in gait which superficially appears regular and symmetric. Regarding these criteria, the walking motion of both unimpaired subjects turn out to be irregular while the impaired subject's gait is wellcontrolled and well-balanced. We suspect that the unimpaired subjects' awareness of their physical capabilities provides them with enough confidence to allow for less conscious gait. On the other hand, we suspect that the impaired subject is aware of his limited ability to control his prosthetic leg and, therefore, follows a more cautious and conscious approach to walking.

The results are encouraging, but the criteria obviously remain to be further tested and validated on large sets of data. A thorough evaluation of these criteria based on existing whole-body walking data (i.e., including upper body and arms), e.g., from data bases, KoroiBot Motion Capture Database<sup>4</sup> , or newly collected wholebody data for unimpaired subjects and subjects walking with prostheses will be conducted. It remains to be determined how subject-specific properties have to be systematically taken into account to adjust the proposed criteria and allow for appropriate classifications.

## ETHICS STATEMENT

This study was carried out in accordance with the recommendations of the ethics guidelines 158/2006 issued by the ethics committee of the University of Heidelberg with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the ethics committee of the University of Heidelberg.

## AUTHOR CONTRIBUTIONS

K-LH developed the hypotheses, developed and executed the procedure for data processing and analysis, and wrote the manuscript. SW designed the experiment, collected data and

<sup>4</sup>KoroiBot Motion Capture Database, 2016.

advised in data analysis, KM advised in data processing and analysis and contributed to writing the manuscript.

## ACKNOWLEDGMENTS

The methods and computed results included in this work are based on the Ph.D. thesis Ho Hoang (2017). The Ph.D. thesis is

## REFERENCES


the only medium in which parts of this content have appeared. Its publication is in line with the author's university policy.

Furthermore, we acknowledge financial support by Deutsche Forschungsgemeinschaft within the funding programme Open Access Publishing, by the Baden-Württemberg Ministry of Science, Research and the Arts and by Ruprecht-Karls-Universität Heidelberg.

Mombaur, K., and Vallery, H. (2018). "Stability and robustness of bipedal walking," in Bio-Inspired Legged Locomotion: Models, Concepts, Control and Applications, eds M. Sharbafi and A. Seyfarth (Oxford: Butterworth-Heinemann), 139–163.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Ho Hoang, Wolf and Mombaur. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Stability of Mina v2 for Robot-Assisted Balance and Locomotion

Carlotta Mummolo<sup>1</sup> , William Z. Peng<sup>1</sup> , Shlok Agarwal <sup>2</sup> , Robert Griffin<sup>2</sup> , Peter D. Neuhaus <sup>2</sup> and Joo H. Kim<sup>1</sup> \*

*<sup>1</sup> Department of Mechanical and Aerospace Engineering, New York University, Brooklyn, NY, United States, <sup>2</sup> Florida Institute for Human and Machine Cognition, Pensacola, FL, United States*

The assessment of the risk of falling during robot-assisted locomotion is critical for gait control and operator safety, but has not yet been addressed through a systematic and quantitative approach. In this study, the balance stability of Mina v2, a recently developed powered lower-limb robotic exoskeleton, is evaluated using an algorithmic framework based on center of mass (COM)- and joint-space dynamics. The equivalent mechanical model of the combined human-exoskeleton system in the sagittal plane is established and used for balance stability analysis. The properties of the Linear Linkage Actuator, which is custom-designed for Mina v2, are analyzed to obtain mathematical models of torque-velocity limits, and are implemented as constraint functions in the optimization formulation. For given feet configurations of the robotic exoskeleton during flat ground walking, the algorithm evaluates the maximum allowable COM velocity perturbations along the fore-aft directions at each COM position of the system. The resulting velocity extrema form the contact-specific balance stability boundaries (BSBs) of the combined system in the COM state space, which represent the thresholds between balanced and unbalanced states for given contact configurations. The BSBs are obtained for the operation of Mina v2 without crutches, thus quantifying Mina v2's capability of maintaining balance through the support of the leg(s). Stability boundaries in single and double leg supports are used to analyze the robot's stability performance during flat ground walking experiments, and provide design and control implications for future development of crutch-less robotic exoskeletons.

#### Edited by:

*C. David Remy, University of Michigan, United States*

#### Reviewed by:

*Priyanshu Agarwal, Rice University, United States Ting Zhang, North Carolina State University, United States*

#### \*Correspondence:

*Joo H. Kim joo.h.kim@nyu.edu*

Received: *31 January 2018* Accepted: *10 September 2018* Published: *15 October 2018*

#### Citation:

*Mummolo C, Peng WZ, Agarwal S, Griffin R, Neuhaus PD and Kim JH (2018) Stability of Mina v2 for Robot-Assisted Balance and Locomotion. Front. Neurorobot. 12:62. doi: 10.3389/fnbot.2018.00062* Keywords: robotic exoskeleton, balance stability boundary, combined human-exoskeleton system, linear linkage actuator, Mina v2

## INTRODUCTION

Robotic exoskeletons have the potential to change the day-to-day life of countless individuals with mobility impairment. Commercial lower-limb exoskeletons, such as ReWalk (Esquenazi et al., 2012), Ekso (Ekso, 2018), and Indego (Parker, 2018) have made significant progress in restoring the mobility of individuals with spinal cord injury (SCI) (e.g., paraplegics or paraparetics). Current research addresses various aspects of exoskeleton functionality, such as providing mobility to patients who are confined to a wheelchair (Esquenazi et al., 2012) or are suffering from muscular weakness (e.g., the elderly and infirm; Sankai, 2010), improving rehabilitation (neurological or orthopedic) and recovery efficacy (Colombo et al., 2000; Veneman et al., 2007), and augmenting the performance of healthy individuals during heavy load carrying tasks (Guizzo and Goldstein, 2005; Walsh et al., 2007). Recent achievements in lower-limb exoskeleton assistance include successes in robot-assisted walking (Raj et al., 2011; Hassan et al., 2014; Sanz-Merodio et al., 2014; Griffin et al., 2017), stair ascent and descent (Xu et al., 2017), and sit-to-stand movements (Tsukahara et al., 2010). Additionally, other studies have focused on the reduction of exoskeleton's energy consumption during task performance through the use of elastic and dissipative elements (Wang et al., 2011; Kim et al., 2015).

In walking applications, the goal is to achieve stable robotassisted locomotion that is adaptable to various terrain and gait parameters, and at an increased range of speeds and larger step lengths, for which ankle actuation is essential. The robotic exoskeleton H2 was the first robotic exoskeleton for gait rehabilitation to include ankle actuation, and has been followed by the development of other designs employing active ankles (Bortole et al., 2015). The recently developed assistive device Mina v2 (Griffin et al., 2017) includes ankle, hip, and knee actuations with the intention of achieving more humanlike lower-limb motion during gait. Its powered plantar flexion allows the human operator to navigate various environments, such as stairs and ramps, as demonstrated during the 2016 Cybathlon competition, and to reliably achieve a conservative walking speed of 0.29 m/s (Griffin et al., 2017). These recent advancements could progress toward robot-assisted gait that requires reduced effort from the user (Griffin et al., 2017) and has desired dynamic walking characteristics, e.g., similar to normal or load-carrying human walking (Mummolo and Kim, 2013; Mummolo et al., 2013, 2016). The effort of translating human locomotion principles into robotic solutions requires quantitative benchmarks to evaluate the human-like performance of robotic assistive devices (Neuhaus et al., 2011). In existing studies, analyses of robot-assisted gait have been conducted with data collected from a sensorimotor wearable robotic system (Raj et al., 2011), a versatile instrumented cane together with body worn sensors (Hassan et al., 2014; Lancini et al., 2016), and motor encoders (Griffin et al., 2017). Given these data, several outcomes can be used to benchmark the exoskeleton-assisted gait of SCI individuals against normal gait (Torricelli et al., 2015).

While human-like dynamic walking is a desired performance goal in the design of exoskeletons for robot-assisted locomotion (Barbareschi et al., 2015; Li et al., 2015; Agrawal et al., 2017), user safety remains the primary concern. In addition to employing a structural design that guarantees the physical safety of humanrobot interactions (i.e., the user should not experience physical discomfort or injury by wearing and operating the robot), proper control design must also be implemented to stabilize the system so that the user is also protected from the risk of injury due to falls. To guarantee stable robot-assisted movements, a systematic and quantitative analysis of the balance stability of the human operator wearing the exoskeleton suit is required from its initial mechanical design to its final assessment. Currently, maintaining balance during robot-assisted gait remains a challenging problem and the operator often relies on the support of additional devices to improve balance. For example, one study (Slavnic et al., 2010) considered the use of a powered exoskeleton integrated with a wheeled mobile platform to provide balance during walking. In several cases, the operator relies on crutches or walkers in order to maintain balance (Acosta-Marquez and Bradley, 2005; Strausser and Kazerooni, 2011; Esquenazi et al., 2012; Farris et al., 2014; Stücheli et al., 2017). While real-time gait planning strategies using crutches as balancing aids have been implemented to produce stable and natural walking (Zhang et al., 2015a,b), they are far from ideal solutions. The use of crutches is often incompatible with the surrounding environment, restricts the operator's use of hands, limits the achievable walking speed, and requires a significant amount of upper limb strength during walking and standing, fatiguing the user (Griffin et al., 2017).

Researchers have recently begun to address the balance stability analysis for humans wearing exoskeletons in the absence of crutches. The design of a hybrid drive exoskeleton has been proposed (Hyon et al., 2011, 2013), in which a combination of pneumatic muscles and electric motors are used to provide sufficient torque and controllability in order to balance without crutches. In those studies, the analysis was focused on the robotic system alone, excluding its human component, and was based on a limited performance evaluation and validation. The design of a robotic exoskeleton with a balance stabilizer mechanism has been proposed and tested for use on SCI subjects (Li et al., 2015), which requires further improvement in order to manage significant shifts in body weight in the coronal plane. Control methods have been developed to provide active gait assistance in both sagittal and frontal planes (Wang et al., 2013, 2015) for the MINDWALKER exoskeleton, and its stable walking without crutches has been demonstrated for healthy subjects (but not yet for SCI paraplegics; Wang et al., 2015). Human and robot balance stability criteria, for instance, based on the capture point and extrapolated center of mass concepts, have also served as promising sources of inspiration for robot-assisted balance control (Huynh et al., 2016; Zhang et al., 2018) and balance recovery against slipping-like perturbation (Monaco et al., 2017); these studies have addressed healthy subjects or subjects with significant voluntary abilities retained. Very recently, ankle joints powered via variable stiffness actuators, which mimic the modulation of muscle impedance in the human ankle for balancing, were proposed to replace constant stiffness actuators (Ugurlu et al., 2016) in order to provide more favorable external disturbance dissipation. However, depending on the disturbance amplitude, the desired ankle stiffness may not be physically realizable with the variable stiffness actuator, and the system can still fail to maintain balance. In the absence of comprehensive human-exoskeleton combined models and a systematic balance stability analysis of lower-limb exoskeletons, control strategies will continue to rely heavily on additional balancing aids, such as crutches and ad hoc criteria.

There is no commonly applicable and comprehensive framework for the balance stability analysis of robot-assisted locomotion so far. The difficulty arises in part from the traditional challenges in determining balancing vs. falling conditions for general legged systems (Mummolo et al., 2017), but also from the modeling complexity of actuator and multibody dynamics of the combined human-exoskeleton system. When addressing the balance stability of robot-assisted gaits, it is essential to establish an accurate model that can describe the dynamics of the combined system representing the human operator wearing the robotic exoskeleton. While several models exist to describe the multibody dynamics of human and robotic biped systems separately, few studies take into account the combined system's dynamics. One study that proposed a hybrid zero dynamics controller for robot-assisted gait treated the human lower body and the exoskeleton as a lumped rigidbody system due to the lack of actuation from the legs (Agrawal et al., 2017). Another study used an improved humanexoskeleton model that introduced compliance at each joint by adding spring-mass-damper systems with parameters obtained through optimization using data from push recovery experiments (Schemschat et al., 2016).

In this study, the balance and locomotion stability characteristics of Mina v2 are systematically evaluated for its typical foot-ground contact configurations. An equivalent model representing the human-exoskeleton system dynamics is established, where the mechanical and actuation models of the human body and the robotic device are combined. A center of mass (COM)-state-based criterion is used to characterize the set of balanced states of the combined human-exoskeleton system in legged support without crutches. For a given foot-ground contact configuration of the combined system, the criterion determines the threshold between balanced and unbalanced states of the system with respect to that configuration. The balance stability criterion is applied to the equivalent model in single and double contact configurations to quantify the capability of Mina v2 to maintain balance through the support of the leg(s), for instance, during swing and transfer gait phases, respectively. The application of the balance stability criterion is demonstrated by using experimental data to characterize the state of balance of robot-assisted walking motions.

## ROBOTIC EXOSKELETON MINA V2

Mina v2 exoskeleton is a prototype paraplegic mobility assistance device designed and built by the authors at the Florida Institute for Human and Machine Cognition (IHMC), and is the third in a series of devices designed to provide upright mobility for people with lower extremity paralysis (**Figure 1**). Each of these devices provides sagittal plane motion of the legs while its upright balance is provided by the user with required forearm crutches (i.e., no balance controller currently implemented). These wearable devices rigidly constrain the operator's joint position and track a commanded joint profile from the walking controller.

The exoskeleton Mina v1 (Kwa et al., 2009; **Figure 1**) had four sagittal plane motors, at the hips and knees, and two passive compliant ankle joints. The actuators were brushless motors with a harmonic drive gear reduction and could be used with either stiff position control or torque control (Neuhaus et al., 2011). Similar to Mina v1, the exoskeleton X1 (**Figure 1**) had four sagittal plane motors, at the hips and knees, and passive compliant ankle joints. Driving each powered joint was a series elastic actuator that could allow for position or force control.

Mina v2 (**Figure 1**) has actuators at the hips and knees like its predecessors, and, in addition, includes an actuator for each ankle joint, resulting in full actuation in the sagittal plane. The powered ankle plantar flexion and dorsiflexion provide the exoskeleton system with stability and mobility, and is motivated by the analysis of human walking (Winter, 1990), which shows that the ankle plays an important role by injecting energy during the toe-off (terminal stance) phase of the trailing stance foot and allowing for dynamic walking (Torricelli et al., 2016). Moreover, the modulation of ankle torque can control the center of pressure displacement within the contact area during mid stance (Perry and Burnfield, 1992), which is a well-known fundamental strategy for balance control.

## Joint Actuator Design

The actuation of Mina v2 is modular in design. Each joint is powered by a custom Linear Linkage Actuator (LLA), allowing for ease of replacement, accessibility, and repair. The LLA was designed specifically for use with Mina v2, and features a frameless electric motor, integrated electronics, a load sensor, and an onboard motor amplifier and controller for distributed jointlevel control. The motor, via a linear ball screw transmission, drives a slider-crank linkage mechanism connected to the joint output (**Figure 2**). The frameless motor has no internal gearing, i.e., its rotor is mounted on the same shaft as the ball screw, hence the effective gear ratio from the motor shaft to actuator joint output is <sup>R</sup> <sup>=</sup> ω/θ˙A, where <sup>ω</sup> is the rotational speed of the motor shaft and <sup>θ</sup>˙<sup>A</sup> is the actuator joint output velocity. Mechanical power losses in the linear transmission are negligible, given that the majority of the loss from the motor to the joint output occurs at the ball screw, which is typically 98–99% efficient. As a result, the effective joint output torque achievable by the actuator is estimated as τ <sup>A</sup> = TR, where T is the motor torque.

The LLA exhibits a non-linear relationship between the motor position and the joint output position, resulting in an effective gear ratio R that varies with the stroke as a function of the output position θ <sup>A</sup> (**Figure 2**). The values of the LLA's effective gear ratio R are calculated from the geometry and then verified experimentally by varying the output position within its admissible mechanical range in actuator space θ <sup>A</sup> ∈ [−2.3, 0] (rad), where −1.25 rad corresponds approximately to mid-stroke configuration. As a result, the gear ratio varies between 41 and 53 as a function of the actuator output position, and is ∼46 around mid-stroke.

Based on the 48 VDC bus voltage, the motor can achieve a maximum no-load speed of 3,340 rpm (ωmax = 349.76 rad/s) and a maximum stall torque Tmax = 2.7 Nm due to thermal limitations. The motor limits and the effective gear ratio are used to obtain the joint output torque and velocity limits in actuator space as functions of joint position:

$$-\left.\alpha\_{\max}/R(\theta^A)\right| \le \theta^A \le \alpha\_{\max}/R(\theta^A) \tag{1}$$

$$-T\_{\text{max}}\mathcal{R}(\theta^A) \le \mathfrak{r}^A \le T\_{\text{max}}\mathcal{R}(\theta^A) \tag{2}$$

For a given actuator joint output position θ <sup>A</sup>, the above lower and upper bounds define a rectangular region of actuator joint output torque-velocity (τ <sup>A</sup>, <sup>θ</sup>˙A) limits (**Figure 3**). In the first and

third quadrants of this region, i.e., when the actuator performs positive work, the maximum and minimum velocities are also dependent on the torque, and are estimated as linear functions with intercepts at the peak no-load speed ωmax/R(θ <sup>A</sup>) and peak torque TmaxR(θ <sup>A</sup>) at the given joint output position. Hence, the feasible region of actuator joint output torque and velocity is additionally constrained by the following inequality:

$$-\left.\alpha\_{\max}\leq\dot{\theta}^{A}R(\theta^{A})+\frac{\alpha\_{\max}}{T\_{\max}R(\theta^{A})}\mathbf{r}^{A}\right.\tag{3}$$

When the actuator performs negative work (second and fourth quadrants), the speed of the motor is limited by the bus voltage, and the torque is limited by the rated current of the motor, and it is assumed that there is no additional relation between the speed and torque. Therefore, the four-quadrant torque-velocity feasible region in the actuator space takes the shape of a hexagon for a given output position and of a hexagonal-base volume for the entire range of joint output position (**Figure 3**).

## Exoskeleton Mechanical Model

The mechanical design of Mina v2 is illustrated in the frontal and sagittal planes (**Figure 4**). Since this study focuses on the sagittal plane mobility and balance stability, the planar model of Mina v2 is described by a seven-link kinematic chain in the (X, Y) plane, with the origin at the center of the leading stance foot. The exoskeleton's mechanical design includes lower body links (feet, shanks, and thighs) and actuators, a pelvic belt, and a backpack containing a lithium ion battery (2.3 kg), computer, power distribution system, and networking hardware. The total mass of the backpack including the battery is 11.2 kg. From its mechanical design, the total mass of the exoskeleton is ∼32 kg, while its actual mass including fasteners, wires, and pads (not included in the current model) may be slightly higher.

The local position of each robot link's COM (with mass m<sup>R</sup> i ) is indicated by the position vector <sup>i</sup> **r** R i , relative to the local frame {x<sup>i</sup> , yi} attached to each link i, for i = 1–7 (**Figure 4**). In the sagittal plane model, a pelvic link with negligible length connects the hip joints (Mummolo et al., 2013) and has a total mass m<sup>R</sup> 4 equal to

the sum of the pelvic belt and the backpack masses, combined into one point mass located at <sup>4</sup> **r** R 4 .

The first and last links of the robot connect each ankle joint to its respective foot plate, and their length corresponds to the operator's foot height. The length of the foot plate is 33.4 cm, which is approximately equal to the operator's foot length, including the shoe. The lengths a = 0.099 m and b = 0.235 m are the distances in the sagittal plane from the projection of the ankle joint onto the ground to the rear and front edges of the foot plate, respectively. When the system is in single foot contact, the contact surface length is a+b, while, during double contact, it may vary; the foot plate is not rigid and has stiffness properties similar to those of a shoe. In the general double contact configuration, the dimension c of the contact patch at the trailing stance foot depends on the operator/controller strategy to move forward during the transfer phase of walking, as described later.

Due to different orientations of the LLAs within the exoskeleton structure and the joint angle conventions used (**Figure 5**), the LLA output must be mapped from the actuator space into the anatomical joint space of the robotic device, as follows:

$$
\boldsymbol{\theta}\_{hip}^{R} = - (\boldsymbol{\theta}^{A} + 1.28); \; \boldsymbol{\theta}\_{knee}^{R} = - \boldsymbol{\theta}^{A}; \; \boldsymbol{\theta}\_{ankle}^{R} = \boldsymbol{\theta}^{A} + 1.05 \tag{4}
$$

$$
\tau\_{hip}^R = -\tau^A; \ \tau\_{kne\text{\textquotedblleft}}^R = -\tau^A; \ \tau\_{ankle}^R = \tau^A \quad \text{(5)}
$$

where superscripts A and R are used to indicate joint angles and torques in the actuator space and the robot (anatomical) joint space, respectively. Following the current lower body joint angle anatomical convention, positive angles θ R , velocities <sup>θ</sup>˙<sup>R</sup> , and torques τ R at the robot joints are used for hip extension, knee flexion, and ankle plantar flexion (**Figure 5**). The zero joint angles are the zero anatomical angles, which correspond to the angles observed in the upright standing pose on flat ground. In addition to the admissible mechanical range of the LLA output angle (θ <sup>A</sup> ∈ [−2.3, 0] rad), the joint angles of Mina v2 are further constrained by more conservative limits, which are based on the operator's joint limits, to protect the operator from any extreme joint flexion and extension.

## COMBINED HUMAN-EXOSKELETON SYSTEM: MODELS AND PARAMETERS

A seven-degree-of-freedom (DOF) model representing the combined human-exoskeleton system is established in the sagittal plane. The equivalent link, joint, and actuation parameters are derived by combining the planar models of the robotic exoskeleton and its human operator, both established in joint space.

## Models of the Exoskeleton's Operator

A seven-link model analogous to that used for the exoskeleton system describes the lower and upper body segments of Mina v2's human operator in the (X, Y) sagittal plane. The foot, shank, and thigh segments are modeled with three links for each leg. Lower body link lengths are directly measured from the human operator and used as a reference for modeling the exoskeleton's links such that Mina v2 and its pilot have identical link and foot lengths. The mass distribution of the human body is based on reference data from a biostereometric survey of six male subjects (Herron et al., 1976). The masses of human pelvis, torso, arm, and head segments are combined into one point mass located perpendicular to the pelvic link. Similarly to the exoskeleton model, the COM position of each link (with mass m<sup>H</sup> i ) is described with respect to the local frame {x<sup>i</sup> , yi} by the position vector <sup>i</sup> **r** H i , for i = 1–7.

In this study, the operator has no volitional motor control of the lower limbs and the passive ranges of motion were measured by moving the joints gently until the ligaments provided resistance. Note that this same procedure can be done on subjects capable of voluntary motion, whose passive (or externally driven) ranges of motion will usually be larger than their active (internally driven) ones. The resulting joint ranges of motion are used as references for the design of safe mechanical limits for the robot.

Depending on the type and level of impairment, an appropriate model for internal joint torque at the human lower limbs should be formulated and combined with the robot's actuation model. The exoskeleton pilot is paraplegic and is assumed to exert no active torque at the lower body joints. In addition, internal torques caused by neuromuscular reflexes are not considered in this model, since the pilot's experience operating the robot suggests that such reflexes at the lower limbs tend to disappear over time with acclimation to the device. Therefore, the only joint torques at the human lower body segments are, in this case, due to the passive contribution of elastic elements. Each internal torque at the anatomical ankle, knee, and hip joints is modeled as a non-linear function of joint angle (Anderson et al., 2007):

$$
\pi^H = B\_1 e^{k\_1 \theta^H} + B\_2 e^{k\_2 \theta^H} \tag{6}
$$

where the sign of θ <sup>H</sup> follows the same anatomical reference used for the robot joint space (**Figure 5**). The parameters B1, B2, k1, and k<sup>2</sup> for ankle, knee, and hip joints (**Table 1**) are obtained from a literature study (Anderson et al., 2007).

## Equivalent DH Model for the Combined Human-Exoskeleton System

Based on the above-mentioned planar models for the robotic exoskeleton and the human body, an equivalent model is developed to represent the kinematics and dynamics of the combined human-exoskeleton system. The equivalent model in the sagittal plane consists of a 7-DOF serial kinematic chain, and thus can be established according to the Denavit-Hartenberg (DH) convention (**Figure 6**). Joints 2–7 are the revolute joints of the lower body, while joint 1, which connects the leading stance foot to the global frame {X, Y} origin, is fixed and has zero range of motion.

In this study, the seven corresponding exoskeleton and human links are combined into seven equivalent rigid bodies, assuming that the relative motion between the two systems is negligible and, therefore, θ <sup>H</sup> = θ R for each lower body joint. Since the exoskeleton's links and joint ranges of motion are designed based on the operator's body and joint parameters, the equivalent DH model has the same link lengths and joint limits as those of the robot. The equivalent link mass m<sup>i</sup> of the combined system (**Figure 6**) is the sum of the ith link masses of the robot and human models, for i = 1–7. Point mass assumption is used to model the equivalent inertial parameters (COM location and inertia matrix) of each link, expressed with respect to the local frame {x<sup>i</sup> , yi} attached to link i. In particular, the local position of the equivalent point mass m<sup>i</sup> relative to frame i is <sup>i</sup> **r**<sup>i</sup> = (m<sup>R</sup> i i **r** R <sup>i</sup> <sup>+</sup> <sup>m</sup><sup>H</sup> i i **r** H i )/m<sup>i</sup> , from which the corresponding moments and products of inertia relative to frame i can be calculated.

The ankle, knee, and hip rotations of the equivalent model are described in joint space by the DH revolute joint variable θi , which is measured from positive xi−<sup>1</sup> to positive x<sup>i</sup> , counterclockwise by convention. The relationships between the DH joint variable θ<sup>i</sup> , for i = 2–7, and the lower body joint angles in the anatomical reference are given by:

$$\begin{aligned} \theta\_{ankle}^{R} &= \theta\_{ankle}^{H} = \theta\_{2} = -\theta\_{7}; \; \theta\_{kne}^{R} = \theta\_{kne}^{H} = \theta\_{3} = -\theta\_{6};\\ \theta\_{hip}^{R} &= \theta\_{hip}^{H} = \theta\_{4} + \pi/2 = -\theta\_{5} - \pi/2 \end{aligned} \tag{7}$$

Based on this transformation, the joint angle limits of the combined system, which are designed in the joint-space anatomical reference, can be expressed in the local reference of the equivalent model according to the DH representation.

TABLE 1 | Non-linear spring parameters for human passive joint torque models<sup>a</sup> .


*<sup>a</sup>The parameters shown represent the normative passive torque characteristics of healthy male subjects aged 18–25. More detailed subject- and impairment-specific characteristics of passive elements could be implemented in a similar manner, if such additional physiological data becomes available.*

In addition to the joint angle limits, the joint torque-velocity limits for the combined human-exoskeleton system must also be expressed in the joint space with respect to the DH joint variable θ<sup>i</sup> and torque τ<sup>i</sup> , by taking into account the actuation limits of the robotic device (defined in the actuator space) and the passive joint torques in the human body (defined in the anatomical joint space). Using Equations (4) and (7), the joint variable θ<sup>i</sup> is mapped into the actuator space through the relationships θ A = fi(θi), for revolute joints 2–7, where:

$$\begin{aligned} f\_2(\theta\_2) &= \theta\_2 - 1.05\\ f\_3(\theta\_3) &= -\theta\_3\\ f\_4(\theta\_4) &= -(\theta\_4 + \pi/2) - 1.28\\ f\_5(\theta\_5) &= \theta\_5 + \pi/2 - 1.28\\ f\_6(\theta\_6) &= \theta\_6\\ f\_7(\theta\_7) &= -\theta\_7 - 1.05 \end{aligned} \tag{8}$$

This mapping is used to model the robotic actuator's effective gear ratio in DH joint space, as a third-order polynomial function R(fi(θi)) = c<sup>0</sup> + c1fi(θi) + c2fi(θi) <sup>2</sup> + c3fi(θi) 3 , whose coefficients c<sup>0</sup> = 41.14, c<sup>1</sup> = −42.75, c<sup>2</sup> = −47.03, and c<sup>3</sup> = −12.84 are determined through curve fitting using the available R data (**Figure 2**). The joint velocity limits of the equivalent model as functions of the DH joint variable θ<sup>i</sup> and its time derivative are formulated as follows:

$$-\left.\omega\_{\text{max}}/R(f\_i(\theta\_i))\right| \le \frac{df\_i(\theta\_i)}{dt} \le \omega\_{\text{max}}/R(f\_i(\theta\_i)) \text{ for } i = 2-7 \tag{9}$$

At a given joint, the sum of the robotic torque τ R and the passive human torque τ <sup>H</sup> in the joint-space anatomical reference provides the total actuation of the combined system, which can be mapped into the DH joint torque τ<sup>i</sup> using the following relationships:

$$\begin{aligned} \boldsymbol{\tau}\_{ankle}^{R} + \boldsymbol{\tau}\_{ankle}^{H} &= \boldsymbol{\tau}\_{2} = -\boldsymbol{\tau}\_{7}; \ \boldsymbol{\tau}\_{kne}^{R} + \boldsymbol{\tau}\_{kne}^{H} &= \boldsymbol{\tau}\_{3} = -\boldsymbol{\tau}\_{6};\\ \boldsymbol{\tau}\_{hip}^{R} + \boldsymbol{\tau}\_{hip}^{H} &= \boldsymbol{\tau}\_{4} = -\boldsymbol{\tau}\_{5} \end{aligned} \tag{10}$$

where positive torques τ<sup>i</sup> in the DH local reference frames follow the right hand rule. Using Equations (5) and (10), the robotic torques in actuator space can be expressed as functions of the DH joint variables and torques through the mapping τ <sup>A</sup> = φi(τ<sup>i</sup> , θi), for i = 2–7, as follows:

$$\begin{array}{l} \phi\_{2}(\mathbf{r}\_{2},\theta\_{2}) = \mathbf{r}\_{2} - \mathbf{r}\_{\mathit{ankle}}^{H}(\theta\_{2})\\ \phi\_{3}(\mathbf{r}\_{3},\theta\_{3}) = -(\mathbf{r}\_{3} - \mathbf{r}\_{\mathit{knee}}^{H}(\theta\_{3}))\\ \phi\_{4}(\mathbf{r}\_{4},\theta\_{4}) = -(\mathbf{r}\_{4} - \mathbf{r}\_{\mathit{hip}}^{H}(\theta\_{4} + \pi/2))\\ \phi\_{5}(\mathbf{r}\_{5},\theta\_{5}) = \mathbf{r}\_{5} + \mathbf{r}\_{\mathit{hip}}^{H}(-\theta\_{5} - \pi/2)\\ \phi\_{6}(\mathbf{r}\_{6},\theta\_{6}) = \mathbf{r}\_{6} + \mathbf{r}\_{\mathit{knee}}^{H}(-\theta\_{6})\\ \phi\_{7}(\mathbf{r}\_{7},\theta\_{7}) = -\mathbf{r}\_{7} - \mathbf{r}\_{\mathit{ankle}}^{H}(-\theta\_{7}) \end{array} \tag{11}$$

where the human passive torques are written as functions of θ<sup>i</sup> using the transformations in Equation (7).

Based on the mappings fi(θi) and φi(τ<sup>i</sup> , θi), the LLA output torque limits can be rewritten as functions of the DH joint and torque variables θ<sup>i</sup> and τ<sup>i</sup> , for i = 2–7:

$$-\left.T\_{\max}R\langle f\_i(\theta\_i)\rangle \right| \le \phi\_i(\tau\_i, \theta\_i) \le \left.T\_{\max}R\langle f\_i(\theta\_i)\rangle \right| \tag{12}$$

FIGURE 6 | Planar model for the combined human-exoskeleton system in the sagittal plane. The links (thick solid lines) indicate the equivalent lower body segments (feet, shanks, and thighs) and the pelvis segment connecting the two hips. The local position of the equivalent COM of each link is described relative to the corresponding local frame {*xi* , *yi* }, for *i* = 1–7. The orientation of the backpack and upper body (dashed lines) is assumed to be perpendicular to link 4. Foot segments in single and double contact configurations are shown for reference, along with the corresponding contact area dimensions and resultant contact wrenches. In the double contact configuration, the distance between the front edges of the trailing and leading stance feet is equal to the step length (*ls*).

Lastly, the linear relationship in the first and third quadrants between actuator output velocity and torque at each joint can be rewritten as a function of DH joint variable (and its derivative) and torque, as follows:

$$1 - \omega\_{\max} \le \frac{df\_i(\theta\_i)}{dt} R(f\_i(\theta\_i)) + \frac{\omega\_{\max}}{T\_{\max} R(f\_i(\theta\_i))} \phi\_i(\tau\_i, \theta\_i) \le \omega\_{\max} \tag{13}$$

The equivalent DH model, along with the above link inertial parameters, joint transformations, and actuation model, is used to formulate the kinematics, dynamics, and the corresponding constraints of the combined human-exoskeleton system. In this study, the recursive Lagrangian dynamics is used to derive the joint-space equations of motion of the equivalent DH model.

## SYSTEM CONTROL AND EXPERIMENTS

The exoskeleton-assisted gait is generated using pre-defined reference joint angle trajectories for the hips, knees, and ankles, based on the upcoming footstep locations and the type of terrain to be traversed (flat ground, steps, or slopes). Within one step of the walking cycle, the exoskeleton's contact configurations with the ground are double contact (during the transfer phase) and single contact (during the swing phase), while the operator is always allowed to make additional contacts with the ground by placing the crutches.

During the transfer phase of walking (**Figure 7**), a toe-off movement is designed in order to exploit the presence of the powered ankle joints in Mina v2 (Griffin et al., 2017). In particular, a minimum jerk trajectory is planned for the trailing ankle joint, such that it reaches a given final plantar flexion angle at the end of transfer, while the body and the leading leg rotates about the leading ankle. This ankle plantar flexion during toe-off motion is powered by the ankle actuator and provides a forward force to the body and helps the push-off of the trailing stance foot prior to initiation of swing. As a result of the reference trajectories for continuous walking, the trailing stance foot during the transfer phase is mostly in toe contact, resulting in a contact region with an approximate dimension c = 8 cm due to the flexibility of the shoe and the foot plate (**Figure 6**).

During the swing phase, four Cartesian-space waypoints are defined for the swing foot: the starting position, the upcoming foothold at a distance equal to the stride length (2ls), and two midpoints positioned at fractions %ls,<sup>b</sup> and %ls,<sup>f</sup> of the nominal step length (ls) and at a fixed height (hs) (**Figure 7**). The parameters %ls,<sup>b</sup> , %ls,<sup>f</sup> , l<sup>s</sup> , and h<sup>s</sup> are tuned for a given step, while the joint angles at each waypoint are calculated using inverse kinematics. The reference joint trajectories are formulated as the minimum jerk trajectories passing through each of these waypoints. The combination of the flexible trajectory design during swing phase with the use of powered toeoff motion contributed significantly to the system's successful gait performance (Griffin et al., 2017).

For the exoskeleton to execute the generated walking trajectory, each actuator is operated in position control mode on the Elmo Twitter Gold embedded motor controller. This motor amplifier closes the position loop using current control to account for position error at a loop rate of ∼3 kHz. The resulting joint-level behavior produces the highest-achievable impedance actuation at each joint, tracking positions as best as possible. Using position control as the basis for the motion comes at the cost of low compliance between the device and the terrain.

The operator was not given specific performance instructions other than to execute a typical walking gait. While walking, the crutches are repositioned during every transfer phase, and are placed on the ground during the swing phase. The operator adopts a tripod-type gait during swing phase, balancing on one leg and two crutches, and always moves both crutches during the

transfer phase, standing stably on both legs (**Figure 8**). The phase changes of the controller were entirely governed by user-selected time, and did not, nor could, rely on any contact or force sensor.

The joint trajectories of the combined human-exoskeleton system were recorded by the exoskeleton's motor encoders for over 20 trials of flat-ground continuous walking experiments. The data was averaged over all the walking steps taken during the trials, representing 80 steps. The average robot joint kinematics (angular positions θ<sup>i</sup> R (t) and velocities <sup>θ</sup>˙<sup>R</sup> i (t)) measured across the walking trials is mapped into the DH kinematics (θi(t), <sup>θ</sup>˙ <sup>i</sup>(t)) and used to evaluate the forward COM kinematics of the equivalent DH model. In particular, the sagittal plane global position **<sup>r</sup>**(t) and velocity ˙**r**(t) of the combined system's COM are calculated at all times as functions of <sup>θ</sup>i(t) and <sup>θ</sup>˙ <sup>i</sup>(t), where the kinematic chain's global frame has its origin at the center of the leading stance foot, which is constrained to be flat on the ground during the entire step duration.

## CONTACT-DEPENDENT BALANCE STABILITY ANALYSIS

The state-based stability is evaluated for Mina v2's robot-assisted balance and locomotion. A numerical optimization framework is used to construct the balance stability boundaries (BSBs) of the combined human-exoskeleton system in single and double legged supports.

## Dynamic Model With Contact Constraints

The BSBs of the combined human-exoskeleton system are constructed by iteratively solving a series of constrained non-linear optimization problems, in which the jointspace constrained dynamics of the equivalent DH model is implemented. The joint-space equations of motion are recursively formulated for the equivalent DH model in its open- (single contact) and closed-loop (double contact) kinematic configurations, by taking into account the dynamics of the contact interactions between the system and its environment. The contact dynamics of the single and double contact configurations are the results of the kinematic and kinetic constraints at the feet imposed during the swing and transfer phase of walking, respectively.

Within one complete step cycle, the center of the leading stance foot is fixed at the origin of the global frame {X, Y} and its orientation is coincident with that of the ground plane at all instants in time. For both contact configurations, no relative motion between the contact surface of the stance feet and the ground is allowed. During the transfer phase, the front edge of the trailing stance foot is fixed to a point with X-coordinate equal to –(l<sup>s</sup> – b), consistent with the step length (**Figure 6**). The orientation of the trailing stance foot about the metatarsal joint [positioned at X-coordinate of –(l<sup>s</sup> – b + c)] is left free, as it rotates during the toe-off motion. During the swing phase, the stance foot remains in full contact with the ground, while the Ycoordinate of the swing foot and any other part of the system is constrained to be above the ground level.

The distributed reaction forces at the contact interface between the feet and the ground are modeled with one equivalent system of resultant contact force and moment (i.e., contact wrench) applied at each stance foot (**Figure 6**). The resultant contact wrench is null at the swing foot during the single contact configuration (e.g., swing phase), while the contact wrench at the support foot is uniquely determined for a given motion. Therefore, in single contact, the unknowns of the non-linear optimization problems for BSB construction are joint trajectories, while joint torques and the reactions at the fixed base are recursively determined from the inherent inverse dynamics scheme. In the double contact configuration (e.g., transfer phase), the distribution of contact wrenches between the two feet is indeterminate, and the contact wrenches, joint kinematics, and actuator torques must all be solved for within the given optimization problem. In this study, the unknowns of the optimization problems for the construction of the BSB in double contact are joint trajectories and contact wrenches at both feet, while joint torques are again determined from inverse dynamics. This formulation in double contact is based on the conjunction of joint- and COM-space dynamics of the given biped system [full details available in Mummolo et al. (2018)].


In addition, the kinetic constraints related to the ground reaction forces and moments applied at each foot are imposed. For each contact surface, the resultant contact force in the normal direction is subject to the unilateral constraint ensuring that the ground only exerts positive normal forces that push on the foot. The friction cone constraint is also imposed on the tangential component of the resultant reaction force to prevent sliding. The position of the center of pressure calculated for each contact wrench is constrained to be within the contact area dimension of the corresponding foot in the sagittal plane (i.e., c for the trailing stance foot and a + b for the leading stance foot) to ensure that any physically realizable pressure distribution (and the corresponding resultant contact wrench) does not cause the foot to tip over.

sequence shown corresponds to three complete steps (permission for image reproduction granted by the participant).

## Balance Stability Boundary Construction

The balancing capability of a legged system is a characteristic that is dependent on the system's current state (position and velocity) and its current contact configuration (Mummolo et al., 2017). In this study, the capability of the combined human-exoskeleton system to maintain balance solely through the support of the legs (i.e., without crutches) is quantified in the sagittal plane, thus isolating the role of crutches in assisting stability in the foreaft (+X and –X) directions. A state-based stability criterion for legged systems that was recently introduced (Mummolo et al., 2018) is used to evaluate the balancing capabilities of Mina v2 in its two main foot-ground (single and double) contact configurations. In the proposed criterion, the equivalent DH model's COM state as its global Cartesian position **r**(t0) and velocity ˙**r**(t0) at a given time <sup>t</sup><sup>0</sup> is used to determine whether the system is in a balanced state with respect to a specified contact configuration, according to the definitions in the authors' previous study (Mummolo et al., 2018). In other words, if the legged system can reach a static equilibrium from a current state (**r**(t0), ˙**r**(t0)) without ever altering its contact configuration, that state is balanced with respect to that contact configuration. Vice versa, if the current state (**r**(t0), ˙**r**(t0)) leads to an inevitable change in the system's current contact configuration, the state is unbalanced with respect to that contact configuration.

The implementation of the proposed COM state-based criterion consists in the numerical construction of the systemspecific and contact-specific BSB. The BSB is a partition of the COM state space that includes all possible balanced states of the given system in the specified contact configuration. A COM state outside of the BSB represents the sufficient condition for losing balance, from which a change in the system's contact is inevitable. For each sampled COM initial position **r**(t0) = **r** ∗ (t0), the optimal trajectories and actuator torques in the joint space are found, such that the component of the initial COM velocity ˙**r**(t0) along a desired direction is maximized while satisfying relevant constraints. From the joint-space solution of each optimization problem, the state (**r** ∗ (t0), ˙**<sup>r</sup>** ∗ (t0)) is calculated and stored as a point of the BSB, and represents the most extreme balanced state for the system at the given COM position and in the specified contact configuration.

At each iteration of this numerical construction algorithm, a new COM position is sampled within the system's contactspecific COM workspace, which is the region of all positions reachable by the legged system's COM subject to joint limits and the specified kinematic contact constraints. In this study, a rectangular grid with a uniform spacing is used to sample the entire workspace area, while any general discretization strategy can be used. In each optimization problem, the equivalent DH model is governed by the dynamics in joint and COM spaces, including the contact dynamics formulation, and is subject to the following constraints:


The resulting initial conditions (**r** ∗ (t0), ˙**<sup>r</sup>** ∗ (t0)), for all possible sampled initial COM positions, are the points of the BSB in the state space and identify the maximum allowable COM velocity perturbations along +X and –X directions for the given system such that static equilibrium can still be reached while remaining solely in the specified (single or double) contact configuration.

## RESULTS AND DISCUSSION

The established parameters are integrated into the combined human-exoskeleton system model. The stability boundaries constructed for the combined system in legged supports are analyzed in the system's COM state space. Then the states of balance of the robot-assisted walking motion are evaluated against the BSBs.

## Combined System Model and Walking Trajectories

Mina v2's design was customized for its pilot, and the joint positions and link lengths of the exoskeleton model closely match those of the human operator (**Table 2**). The human subject operating Mina v2 in the current experiments is a male, is 1.78 m tall, and has a total mass of 82.8 kg. This mass is similar to that of Subject 2 from a literature study (Herron et al., 1976), which is used to estimate the mass distribution of the operator's body segments. The link and mass parameters for the equivalent DH model of the combined human-exoskeleton system were calculated accordingly (**Table 2**). In particular, the equivalent link's COM local position <sup>i</sup> **r**i is calculated from <sup>i</sup> **r** R i and <sup>i</sup> **r** H i , for i = 1–7, using the proposed methods as described above.

The operator's (passive) joint ranges of motion were obtained as previously described and the robot joint limits are designed in the anatomical joint space to be less than or equal to the operator's joint range of motion, with a safety margin as a precaution. Based on these ranges of motion and the transformations in Equation (7), the lower and upper bounds of the joint variables in the equivalent DH model are expressed with respect to the local reference frames (**Table 3**).

The resulting dynamic models of the combined humanexoskeleton system are implemented into the optimization problems for the BSB construction. As a simple measure of model validation, the total normal component of the ground reaction force(s) is equal to 1,124.9 N at the final static equilibrium for all single and double contact solutions to the optimization problems, which accurately reflects the weight of the combined system with a total mass of 114.7 kg.

A forward kinematics algorithm processes the link parameters (m<sup>i</sup> and <sup>i</sup> **r**i ; **Table 2**), the DH joint angle limits (**Table 3**), and the kinematic contact constraints of the equivalent DH model, and evaluates the system's contact-specific COM workspace in the single and double contact configurations (**Figure 9**). The workspace area in single contact is larger than that in double contact. In particular, since the double contact configuration satisfies all kinematic constraints present in the single contact configuration plus the additional constraint of the trailing stance foot position, every joint configuration that satisfies the double contact kinematic constraints also satisfies those of the single contact configuration. As a result, the double contact workspace area is entirely included inside the single contact workspace area.

The experimental joint trajectories were averaged for one complete step (**Figure 10**) and correspond to the robot and human joint rotations in the sagittal plane. The nominal step length corresponding to the reference joint angle trajectories for each walking trial is l<sup>s</sup> = 0.4 m, with %ls,<sup>b</sup> = %ls,<sup>f</sup> = 30, h<sup>s</sup> = 0.1 m, and the swing and transfer time equal to 1.0 s and 0.4 s, respectively. The average walking speed of the trials was 0.29 m/s. The forward kinematics algorithm also processes the average joint trajectories [mapped into the DH kinematics (θi(t), <sup>θ</sup>˙ <sup>i</sup>(t))] for the calculation of the corresponding average COM trajectory of the combined human-exoskeleton system during the flat-ground walking experiments. The total COM trajectory in the sagittal plane during walking (plotted for one step in **Figure 9**) is included within and close to the workspace boundaries corresponding to the single and double contact configurations, with an average Y-coordinate of 1.00 m. In this study, the COM positions for the BSB construction were sampled within the workspace at the grid points nearest to the COM trajectory in the (X, Y) plane, in order to characterize the system's balance stability at a COM height similar to that of experimental walking trials. Therefore, the selected sample points for the COM initial position **r**(t0) = **r** ∗ (t0) have a Y-coordinate of 1.00 m and an X-coordinate within the corresponding workspace ranges of [−0.368, 0.4255] for single contact and [−0.162, 0.148] for double contact (in meters), with uniform spacing of 2 cm.

## Balance Stability Boundaries for Legged Support

The balance stability characteristics of the combined humanexoskeleton system in legged support are demonstrated through the calculation of the BSB for single and double contact configurations. The velocity extrema are found along the +X and –X directions to evaluate the stability characteristics of the combined system in the sagittal plane against positive and negative perturbations along the direction of forward walking TABLE 2 | Link parameters for the sagittal plane model of Mina v2, the operator, and the equivalent DH model for the combined human-exoskeleton system.


\**The robot's upper body consists of a pelvis link (2.953 kg) and a backpack (11.2 kg). The human's upper body includes the head, arms, torso, and pelvis.*

TABLE 3 | Joint ranges of motion (in degrees) of Mina v2, the operator, and the combined human-exoskeleton system.



progression. The BSB results corresponding to the selected grid points of interest are projected onto the X-state space (**Figure 11**).

The BSBs quantify of the state space regions within which the combined system can maintain balance using only the support of the leg(s) and without resorting to other balancing mechanisms, such as crutch placement. If the X component of any velocity perturbation of the combined system's COM are within the single contact BSB threshold (**Figure 11**, left), it indicates that balance can be maintained on a single foot and without crutches. If the X component of a velocity perturbation of the combined system's COM surpasses the single contact BSB threshold, then the human-exoskeleton system will not be able to stop unless the single contact is altered, for example, by placing the non-stance foot (i.e., stepping) or crutches on the ground. In this case, the current COM state is said to be an unbalanced state with respect to the specified single contact configuration, and will necessarily end up in a contact change. A similar statement can be made for the double contact BSB results (**Figure 11**, right). Note that the BSBs results are not associated to a specific motion, and their construction algorithm does not assume any specific controller design. Instead, the state space partitions identified by the BSBs are the result of the system properties (mechanical and actuation models) and the specified contacts with the environment.

The states of balance for each contact configuration can be analyzed with respect to the X limits of the COM workspace at Y = 1.00 m and the X dimensions of the base of support between the foot/feet and the ground (**Figure 11**). For the single contact, balanced states exist for the COM positions only within the range [−0.24, 0.26], in meters, which is smaller than its workspace range (**Figure 11**, left). While there are COM positions out of this range that are kinematically feasible within the single contact COM workspace, they cannot be balanced due to kinetic constraints. When the COM position lies sufficiently outside of the foot base of support, regardless of its velocity, restoring balance requires either the motion of the stance foot (sliding or tipping-over) relative to the ground or the presence of additional contacts (stepping or crutch placement); otherwise, falling is inevitable. For the double contact configuration, the BSB extends up to its workspace limits. Therefore, all COM positions within the double contact COM workspace can be balanced if their velocity perturbations in the fore-aft directions are within the double contact BSB. In addition, all COM positions within the double contact BSB are also statically stable, in other words, all

FIGURE 9 | Contact-specific COM workspaces, discretized using a rectangular grid for COM initial position sampling in the construction of the BSB. The COM trajectory in the sagittal plane is calculated from the average joint angle experimental data of one step walking cycle. The contact dimensions during single (*a* + *b*) and double (*a* + *b* and *c*) contacts are shown.

FIGURE 10 | Average joint angle trajectories during one step of the flat ground walking trials [adapted from Griffin et al. (2017)]. The desired trajectories are shown as solid (blue) and the actual trajectories are shown as dashed (red) lines. The horizontal axis (percent gait) is the time axis normalized by the duration of a step cycle. The shaded region represents the double contact transfer phase. The plots in the left column are the joint angles of the trailing stance leg that performs the swing motion, and those in the right column are for the leading stance leg, which is always in contact with the ground during a step. When the percent gait reaches 100%, the legs switch roles, such that a new step cycle begins.

their X-positions are within the base of support region. Note that a statically stable COM position can be inside or outside of the BSB (i.e., balanced or unbalanced) depending on its COM velocity.

The single contact BSB is much larger than and encloses the double contact BSB, and therefore has a greater balancing capability. In a single contact, the freedom of the system to employ its angular momentum enhances its balancing performance. For the double contact configuration, the balancing advantage of the additional contact wrench at the trailing stance foot is offset by the condition for double contact that both feet must be pinned to the ground. However, it should be noted that some unbalanced double contact states may result in being within the single contact BSB through a foot detachment, and thus can maintain balance without crutch placements.

## Balance Stability of Robot-Assisted Gait and Role of Crutches

The BSBs under the support of the leg(s) and in the absence of additional contacts, such as crutches, can serve as a basis for comparison when characterizing the balance stability of the current robot-assisted walking motion during transfer and swing phases. In particular, the balance stability of the average COM state space trajectory during one step obtained from the flat ground walking trials is analyzed with respect to the contactspecific BSB for the corresponding gait phases (**Figure 12**).

The COM state space trajectory from the experimental walking trials is the result of the gait planning and control implemented as described previously, which was formulated in the lower body joint space without any balance controller, and thus resulted in the use of crutches. It is observed that the COM state trajectory during transfer and swing mostly lies within the corresponding BSB (double and single contact, respectively). The transfer phase begins and ends well within the double contact BSB, indicating that each state of the prescribed transfer trajectory is balanced with respect to the double contact

configuration. This implies that a control strategy could be designed such that the same transfer motion can be performed stably (at least with respect to the sagittal plane) without using crutches. Moreover, the stability region in double contact indicates that the operator may safely reposition both crutches during transfer phase in preparation of the next swing phase, without losing balance in the sagittal plane. The COM trajectory is also contained within the single contact BSB during the swing phase, only briefly exiting at the very end when the legs switch roles in preparation for the next transfer phase. The balance stability of the given crutched walking trials in relation with the calculated crutch-less stability boundaries indicates that a future balance controller could be designed such that the role of crutches in the sagittal plane balancing could be reduced in transfer phase and most of the swing phase. This enhanced sagittal plane balancing capability in single and double contacts is in part due to the full (including ankle) joint actuation in the sagittal plane in Mina v2. On the other hand, the role of crutches may still be relevant at the very end of the swing phase for the sagittal plane stability of Mina v2, and also at the swing and transfer phases for the lateral stability, since the system is unactuated and has restricted joint rotation in the frontal plane.

The calculated BSBs in the single and double contact configurations predict the contact-specific state space regions within which the combined human-exoskeleton system has the physical capability to maintain balance using the support of the leg(s) during any generic task, without ever altering the respective contacts. Therefore, these regions represent a contactdependent system property that can be used as a reference for the design of task-specific controller domains in the state space, for which the contact-specific balanced and unbalanced regions are pre-computed (i.e., known a priori). As specific applications in robotic exoskeletons, the calculated balance stability regions would provide quantitative guidelines for the mechanical and control system design of robot-assisted locomotion. For instance, the BSBs can be used as reference maps for benchmarking human gait, improving the walking trajectory design at an early stage (before the trajectories' actual implementation and testing), and evaluating the role of crutches as balancing aids in multiple planes. The integration of the proposed balance stability criterion within novel human-robot interface technologies could provide the operator with quantitative feedbacks during training, hence providing assistance for the exploration of less conservative and more agile walking trajectories.

## ETHICS STATEMENT

This study was carried out in accordance with the recommendations of The Institutional Review Boards (IRBs)

## REFERENCES


of The Florida Institute for Human and Machine Cognition (IHMC) and New York University (NYU). The protocol was approved by the respective IRBs. All subjects gave written informed consent in accordance with the Declaration of Helsinki.

## AUTHOR CONTRIBUTIONS

CM developed the theory and the computational framework for balance stability, and led the modeling, coding, and analysis. WP developed the mechanical and actuation models of the equivalent DH system, and processed and analyzed the data. SA developed robot's mechanical model and the combined human-exoskeleton system. RG developed the control system and helped conduct the experiments. PN led the hardware design, performed all of the transformation and calculations for the actuator, and helped conduct the experiments. JK developed the theory, helped the modeling and analysis, and supervised the overall research. All the authors contributed to the discussion, writing, review, and editing of the article.

## FUNDING

This work was supported in part by the U.S. National Science Foundation (IIS-1427193 and IIS-1427213).

## ACKNOWLEDGMENTS

The authors would like to thank the pilot, Mark Daniel, for his participation and sharing his experiences and feedback regarding the use of the robotic exoskeleton, and Yunjiageng Chen for his help in processing the data.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Mummolo, Peng, Agarwal, Griffin, Neuhaus and Kim. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Benchmarking the HRP-2 Humanoid Robot During Locomotion

Olivier Stasse<sup>1</sup> \*, Kevin Giraud--Esclasse<sup>1</sup> , Edouard Brousse<sup>2</sup> , Maximilien Naveau<sup>3</sup> , Rémi Régnier <sup>2</sup> , Guillaume Avrin<sup>2</sup> and Philippe Souères <sup>1</sup>

<sup>1</sup> Laboratoire d'Analyse et d'Architecture des Systèmes, CNRS, Université de Toulouse, Toulouse, France, <sup>2</sup> Laboratoire Nationale de Métrologie et d'Essais, Paris, France, <sup>3</sup> Max-Planck Institute, Tuebingen, Germany

In this paper we report results on benchmarking a HRP-2 humanoid robot. The humanoid robots of this serie are known to be very robust. They have been successfully used by several research groups for the design of new motion generation algorithms. As such it is a reference in the category of electrically driven humanoid robot. As new humanoid robots are continuously built it is interesting to compare the performances of these new prototypes to those of HRP-2. This benchmarking study was realized through a campaign of measurements in an advanced equipped testing laboratory that provides a well adapted controlled environment. We have investigated the effect of temperatures variation on the robot walking capabilities. In order to benchmark various environmental conditions and algorithms we computed a set of performance indicators for bipedal locomotion. The scope of the algorithms for motion generation evaluated here ranges from analytical solution to numerical optimization approach, enabling real-time walking or multi-contacts motions.

Keywords: benchmarking, bipedal locomotion, humanoid robot HRP-2, controlled environment, numerical optimization, walking

## 1. INTRODUCTION

From the seminal work of Chestnutt (2010) to the recent methods proposed in the frame of the Darpa Robotics Challenge (DRC) (Radford et al., 2015; DeDonato et al., 2017; Johnson et al., 2017; Lim et al., 2017; Marion et al., 2017; Tsagarakis et al., 2017), humanoid robots use for moving a control architecture that roughly follows the general framework depicted in **Figure 1**. Based on an internal representation of the environment and the localization of the robot (rˆ<sup>b</sup> and <sup>θ</sup><sup>ˆ</sup> b being, respectively, the base position and orientation), the Motion Planner (MP) plans a sequence of reference end-effector contact positions (f ref ), or a reference center of mass linear velocity combined with a reference waist angular velocity (V ref). These references are then provided to a Model-Predictive Whole-Body Controller (MPWBC) which generates a motor command for each joint (joint torques (τ ref ), positions (q ref), velocities (q˙ ref) and accelerations (q¨ ref)). This block is critical in terms of safety as it maintains the dynamic feasibility of the control and the balance of the robot. The Model-Predictive Whole-Body Controller can be expressed as a unique optimal control problem but at the cost of efficiency in terms of computation time or solution quality. This is why this controller is usually organized in two stages. First, trajectories for the robot center of mass c ref and the positions of contacts with the environment f ref are found using a Centroidal Dynamics Pattern Generator (CDPG). Then, a Whole-Body Controller (WBC) computes an instantaneous controller enabling to track these trajectories. More details about the CDPG can be found in the

#### Edited by:

Katja Mombaur, Universität Heidelberg, Germany

#### Reviewed by:

John Nassour, Technische Universität Chemnitz, Germany Felix Aller, Universität Heidelberg, Germany

> \*Correspondence: Olivier Stasse ostasse@laas.fr

#### Specialty section:

This article was submitted to Humanoid Robotics, a section of the journal Frontiers in Robotics and AI

Received: 15 December 2017 Accepted: 02 October 2018 Published: 08 November 2018

#### Citation:

Stasse O, Giraud--Esclasse K, Brousse E, Naveau M, Régnier R, Avrin G and Souères P (2018) Benchmarking the HRP-2 Humanoid Robot During Locomotion. Front. Robot. AI 5:122. doi: 10.3389/frobt.2018.00122

next paragraph. The whole body reference is in turn sent to the Robot Hardware, which can be either the simulator or the real robot. The feedback terms are based upon the measurements of the different sensors. The encoders evaluate the joint position (q˜). The inertial measurement unit (IMU) measures the angular velocity (ω˜IMU) and the linear acceleration (a˜IMU) of the robot torso, which give information about the orientation of the robot with respect to the gravity field. Finally the interaction with the environment is provided by the force sensors classically located at the end-effectors (FEE ∈ {FRF, FLF, FRH, FLH}, where the subscripts have the following meaning: (EE): end-effector, (RF): right foot, (LF): left foot, (RH): right hand, (LH): left hand). All these information are treated in an Estimator to extract the needed values for the different algorithms. Finally the Localization block is used to locate as precisely as possible the robot in its 3D environment. Various implementations of this architecture have been proposed with various levels of success from the highly impressive Boston Dynamics System, to robots widely available such as Nao. An open question is the robustness and the repeatability of such a control system as well as its performance. In this paper our main contribution is to propose a benchmarking of the HRP-2 robot in various setups and provide performance indicators in scenarios which are possibly interesting for industrial applications. We hope this study will provide a quantitative comparison and will serve as a baseline for the elaboration of new algorithms. In addition we believe that this paper is one of the first attempt to apply the detailed performance indicators provided by Torricelli et al. (2015) to a human size humanoid robot. The paper is structured as follows: firstly, the section 2 presents the related work on control and benchmarking for humanoid robots. Then section 2.3 depicts our precedent contribution in the Koroibot project and how it relates to this work. To continue, section 3 lists the materials and different methods used to perform the benchmarking. In turn section 4 shows the experimental results using the indicators from section 3. Finally the conclusion in section 5 summaries the contributions and results of the study.

## 2. RELATED WORK

In this paragraph we present the work that has been done relative to the control and the benchmarking of the HRP2 humanoid robot.

## 2.1. Motion Generation for Humanoid Robots

The different benchmarks included in this paper are relative to the MPWBC sketched in **Figure 1**. This related work is presented in this first subsection. Several techniques are used to mathematically formulate this problem. For instance hybriddynamics formulations as proposed by Grizzle et al. (2010) or Westervelt et al. (2007) are efficient but difficult to generalize. The approaches used in this paper are based on mathematical optimization which is broadly used in the humanoid robotics community. More precisely, the locomotion problem can be described as an Optimal Control Problem (OCP). The robot generalized configuration (q ref) and velocity (q˙ ref) usually compose the state (**x** ∈ R n ). The future contact points can be precomputed by a Motion Planner or included in the state of the problem. The control of this system **u** ∈ R <sup>m</sup>, can be the robot generalized acceleration (q¨ ref), the contact wrench (φ<sup>k</sup> with k ∈ {0, . . . , Number of Contact}), or the motor torques (τ ref ). We denote by **x** and **u** the state and control trajectories. The following optimal control problem (OCP) represents a generic form of the locomotion problem (which can be for instance a direct multiple shooting problem):

$$\min\_{\substack{\underline{\mathbf{x}}, \underline{\mathbf{u}} \\ \underline{\mathbf{x}} = 1}} \sum\_{s=1}^{S} \int\_{t\_s}^{t\_s + \Delta t\_s} \ell\_s(\mathbf{x}, \mathbf{u}) \, \mathbf{d}\mathbf{t} \tag{1a}$$

$$\text{s.t. } \forall t \; \dot{\mathbf{x}} = d \mathcal{y}n(\mathbf{x}, \mathbf{u}) \tag{1b}$$

$$\forall \mathbf{t} \ \phi \in \mathbb{K} \tag{1c}$$

$$\forall t \; \mathbf{x} \in \mathcal{B}\_{\mathbf{x}} \subset \mathbb{R}^n \tag{1d}$$

$$\forall t \text{ } \mathbf{u} \in \mathcal{B}\_{\mathfrak{u}} \subset \mathbb{R}^{m} \tag{1e}$$

$$\mathbf{x}(0) = \mathbf{x}\_0 \tag{1f}$$

$$\mathbf{x}(T) \in \mathcal{X}\_\* \subset \mathbb{R}^n \tag{1g}$$

where ts+<sup>1</sup> = t<sup>s</sup> + 1t<sup>s</sup> is the starting time of the phase s (with t<sup>0</sup> = 0 and t<sup>S</sup> = T). In the direct multiple shooting problem a phase s corresponds to an interval where the system is simulated using constraint (1b) which makes sure that the motion is dynamically consistent. Phases are connected through the constraints (1d) and (1e) which impose bounds on the state and the control, they are lying, respectively, in admissible set of states B<sup>x</sup> and in admissible set of controls Bu. Constraint (1c) enforces balance with respect to the contact model. Breaking and adding contact are usually done at phases junctions because it changes the structure of the dynamics. Constraint (1f) imposes the trajectory to start from a given state (estimated by the sensor of the real robot). Constraint (1g) imposes the terminal state to be in the viable terminal states set X<sup>∗</sup> (Wieber, 2008). The cost (1a) is decoupled ℓs(**x**, **u**) = ℓx(**x**)+ℓu(**u**) and its parameters may vary depending on the phase. ℓ<sup>x</sup> is generally used to regularize and to smooth the state trajectory while ℓ<sup>u</sup> tends to minimize the forces. The resulting control is stable as soon as ℓ<sup>x</sup> comprehends the L<sup>2</sup> norm of the first order derivative of the robot center of mass (CoM), Wieber et al. (2015). Problem (1) is difficult to solve in its generic form. And specifically (1b) is a challenging constraint. Most of the time the shape of the problem varies from one solver to another one only by the formulation of this constraint. The difficulty is due to two main factors: (1) There is a large number of degrees of freedom (DoF). In practice we need to compute 36 DoF for the robot on a preview window with 320 iterations (1.6 s) to take into account the system inertia. (2) The dynamics of the system is nonlinear. **Figure 2** depicts the structure of the problem. To be able to solve the whole problem, represented by the full rectangle in **Figure 2** researchers often use nonlinear optimization. In this paper we evaluated a resolution of the MPWBC based on the formulation given by Equation (1). In this approach described in Koch et al. (2014), the authors computed a dynamical step-over motion with the HRP-2 robot, but this process can take several hours of computation. So simplifications are necessary, for example Tassa et al. (2014), Koenemann et al. (2015) use simplifications on the contact model. This method is very efficient but not suitable for complex contacts during walking. Seminal works (Kajita et al., 2003b; Orin et al., 2013) show that (1b) can be divided into two parts, the non-convex centroidal dynamics (top horizontal rectangle in **Figure 2**) (Orin et al., 2013) that includes few DoF, and the convex joint dynamics (vertical rectangle in **Figure 2**). Kuindersma et al. (2014) and Sherikov (2016) chose to deal the two aforementioned parts of **Figure 2** at once. They optimize for the centroidal momentum on a preview horizon and the next whole body control. Qiu et al. (2011), Rotella et al. (2015), and Perrin et al. (2015) decouple the two separated aforementioned rectangles in **Figure 2**. They solve first for the centroidal momentum and then for the whole body control. In general the centroidal momentum remains difficult to handle due to its non-convexity. Finally Kajita

et al. (2003a), Herdt et al. (2010), and Sherikov et al. (2014) linearize the centroidal momentum which provides a convex formulation of the locomotion problem. In Deits and Tedrake (2014), the problem was formulated has a mixed-integer program (i.e., having both continuous and discrete variables) in case of flat contact. In Mordatch et al. (2012), the same problem was handled using a dedicated solver relying on a continuation heuristic, and used to animate the motion of virtual avatars.

## 2.2. Benchmarking

Different methods exist to benchmark robot control architectures. In del Pobil et al. (2006) the authors argue that robotic challenges offer an efficient way to do so. For example, the results of the DARPA Robotics Challenge published in the Journal of Field Robotics special issues Iagnemma and Overholt (2015) and Spenko et al. (2017), show the different control architecture in a determined context. Each behavior successfully accomplished grants point to the team and the best team wins the challenge. This benchmarking was however costly as the robots had no system to support them in case of fall. In addition, as it is mostly application driven, the challenge provides an overall evaluation of the system integration but not of the independent sub-parts.

For the specific case of motion generation, it has been recently proposed by Brandao et al. (2017) to use a scenario called "Disaster Scenario Dataset." It allows benchmarking posture generation (solved by the WBC) and trajectory generation (MPWBC) using optimization. A set of problems is proposed by means of foot step locations (FRF, FLF). Using this approach, it is possible to compare algorithms realizing the two functionalities (WBC and MPWBC). The evaluation is realized in simulation using the Atlas robot and the ODE dynamic simulator. This first step is necessary but one step further is required to benchmark a real humanoid platform. For this paper we used a more systematic decomposition of the humanoid bipedal locomotion (Torricelli et al., 2015). Further description can be found in section 3.7. This paper focuses on evaluating the MPWBC and WBC on the Robot Hardware. The Estimator used in this context is important but it is reflected in the stabilization process. The Motion Planning is not evaluated here as the planned motion is always the same or solved at the MPWBC level. The Localization is provided by a motion capture system.

## 2.3. A Motivating Example: The Koroibot Project

The work presented in this paper takes its root in the context of the European project Koroibot (http://www.koroibot.eu/). The goal of the Koroibot project was to enhance the ability of humanoid robots to walk in a dynamic and versatile way, and to bring them closer to human capabilities. The Koroibot project partners had to study human motions and use this knowledge to control humanoid robots via optimal control methods. Human motions were recorded with motion capture systems and stored in an open source data base which can be found at https:// koroibot-motion-database.humanoids.kit.edu/. With these data several possibilities were exploited:


In order to evaluate the progress of the algorithms at the beginning and at the end of the project, a set of challenges focusing specifically on walking were designed (see **Figure 4**). **Figure 3** (right) shows all the robots hosted by the Koroibot partners. Each team owning a robot had to perform some of these challenges considering the current and the potential state of their robots and controllers.

## 2.4. The Key Performance Indicators (KPI)

In this context and in collaboration with the H2R project, a detailed set of key performance indicators (KPI) have been proposed (Torricelli et al., 2015). These KPI try to capture all the bipedal locomotion patterns. Specific sub-functions of the global motor behaviors were analyzed (see **Figure 5**, right). The results are expressed as two different sub-function sets. First, the sub-functions associated with the body posture task without locomotion. Second, the same sub-functions but including the robot body transport. The initial condition may vary depending on the experiment to perform. This is the idea of the intertrial variability. The sub-functions are also classified by taking into account the changes in the environment or not. Each of these functions can be evaluated for different robots using the criteria depicted in **Figure 5** (left). The performances are classified into two sub categories, quantitative performances and human likeness. In addition, information in the last two columns indicate whether the criteria is applicable on a standing task or on a locomotion task. Again, all the teams owning a robot had to perform an evaluation of these KPI, considering the current and potential state of their robots and controllers.

## 2.5. The Work Done in the Koroibot Context

In the Koroibot context the Gepetto team evaluated the KPI one the robot HRP-2 (second robot from the left in (**Figure 3**, right).


FIGURE 5 | (Left) Performances indicators, (Right) motor skills considered in the benchmarking scheme. This scheme is limited to bipedal locomotion skills. The concept of intertrial variability represents modifications of the environment between trials. (dashed) motor skills evaluated in Naveau (2016) (not dashed) motor skills evaluated in this paper.

Among the challenges presented in **Figure 4**, we considered the following ones:


They are depicted by red circles in **Figure 4**. In addition to these challenges we added the perturbation rejection. Considering the selected challenges we picked the following KPI:


while considering the following motor-skills:


All these choices are shown in **Figure 5** by red ellipses in the table. The mathematical details and results are presented below in section 3.7.

## 3. MATERIALS AND METHODS

The experimental setups used to compute each of the performance indicators given in section 3.7 are described in this section. The motor skills given in **Figure 5** and their implementation are also presented. In addition, the algorithms used to perform the different tests are depicted in section 3.8 and illustrated in **Figure 6**.

## 3.1. Different Temperatures

The LNE is equipped with temperature-varying rooms which allowed us to measure some of the performance indicators at various temperatures ranging from 5◦C to 45◦C. In this way, we evaluated the robustness and limits of our robot with respect to the performance indicators in different environmental conditions. It appeared that the robot behavior deteriorates at low temperatures. At 5◦C it is not possible to perform the calibration procedure as the robot could not move. At 10◦C the friction is sufficiently low such that the robot could move. Another phenomenon occurs above 40◦C after few motions due to internal temperature build up: thermal protection prevents the robot from moving if the temperature is too high. In this room, apart from these extreme cases, the motions and indicators measurements have been performed as expected on a flat ground or on the staircase testbed of the Koroibot project. This staircase is made of 4 15 cm high stairs and a top platform. The dimension of one stair case is 1 m × 0.25 m × 0.05 m.

## 3.2. Tilted Surfaces

In the context of the body skills in motion, we considered tilting surfaces. This was tested with the stabilizer commercially available with HRP-2. The setup is a platform which can be tilted upward and downward on one side with a hydraulic actuator. The surface was tilted continuously until the robot fell off. On the other hand, we tested walking algorithms with different angles (pointing up or down) until the robot fell down. Tests were realized with the robot pointing down, pointing up and across the slope. In **Figure 5** this test corresponds to Body Posture— Continuous Surface Tilts.

## 3.3. Horizontal Translations

We used a mobile plate controlled in the horizontal plane to perform continuous oscillating surface translations at various frequencies and various amplitudes. The platform was moved by a hydraulic actuator. The aim was to find the frequency and the amplitude that the controlled robot is able to sustain. In **Figure 5** this test corresponds to Body Posture—Continuous Surface translations.

## 3.4. Bearing

In order to test the robot capability to bear weights, we loaded it with additional masses (bags of 5–15 kgs) in such way that its balance is maintained. This approach is a bit limited as they are several ways to bear a weight. Indeed it can be done with a backpack, in collaboration with someone, or by holding the object against its chest. Each of this approach comes with its own specific constraint. In order to avoid such constraints, we decided to take the simplest choice and hang soft weights on the front and the back of the robot chest. In **Figure 5** this test corresponds to Body Transport—Bearing Constant Weight.

## 3.5. Pushes

This paragraph presents the pushes experiments. We tried to find the sufficient force to make the robot fall down. This was achieved by using a stick on top of which was fixed a force sensor displaying the maximum force measured during an experiment. The sensor used was a HBM 1000 N of type u3 together with a HBM Scout 55 amplifier. The experience was realized while the robot was standing and walking. The force was applied in the sagittal and frontal planes until making HRP-2 fall. The force was applied from behind the waist of the robot. This part of HRP-2 was made specifically soft to support impacts. The walking part is the most difficult in terms of repeatability as the robot might be in different foot support and therefore more or less stable depending on the configuration. In **Figure 5** this test corresponds to Body Posture—Pushes and Body Transport—Pushes.

## 3.6. Data

A CAD model of the staircase used is available on the github repository where all the log of the experiments are also present: https://github.com/laas/koroibot\_KPI. All the computations performed on the logs and implementing the key performance indicators are available here: https://github.com/ laas/EnergyComputation.

## 3.7. Key Performance Indicators (KPI)

In this section the performance indicators used to evaluate the humanoid robot HRP-2 are described. They are mostly based on the work proposed in Torricelli et al. (2015). In the Koroibot project we used key performance indicators (KPI) to analyze the behavior of the robot at the beginning and at the end of the project. These results lead us toward the improvements to be made. In 2013 the algorithm mostly used and implemented on HRP-2 in LAAS-CNRS where the walking pattern generators described in Morisawa et al. (2007) and Herdt et al. (2010). The performance indicators chosen were:

• The execution time T<sup>M</sup> = tend − tbegin, where tbegin is the time at which the sum of the norm of the motor axis velocities reaches 6 rad s−<sup>1</sup> for the first time in the log and tend is when the sum of the norm of the motor axis velocities passes below 0.5 rad s−<sup>1</sup> .

FIGURE 6 | Pictures of the experimental setup at LNE (A) the robot hang up to walk on a slope (B) the translational plate (C) the temperature-controlled chamber (end of the robot climbing 15 cm at 10◦C).


$$\begin{aligned} \textit{TrackingError}(t) &= \int\_{t}^{t+0.1} |q^{ref} - \widetilde{q}| dt/0.1\\ \textit{MaxTrackingError} &= \max\_{t} (\textit{TrackingError}(t)) \end{aligned}$$

with TrackingError being the average normed difference between the desired joint trajectory (q ref) and the joint pose measured from the encoder (q˜) during 0.1 s starting at time t. And MaxTrackingError being the maximum value of the TrackingError function.

• The mechanical energy consumed normalized over the walking distance D and the execution time TM.

$$E\_{mechanical} = \int\_{t\_{\text{bggin}}}^{t\_{\text{end}}} |\tau \phi| dt / \langle T\_M | D \rangle$$

with Emechanical being the integral over time of the mechanical power, τ being the torques applied at the robot joints and ω being the velocity of the robot joints.

• The electrical energy dissipated by the motor resistance normalized over the walking distance D and the execution time TM,

$$\begin{aligned} E\_{\text{motor resistance}} &= \int\_{t\_{\text{bj/in}}}^{t\_{\text{cnd}}} R \, i^2 dt / \langle T\_M \, D \rangle \\ &= \int\_{t\_{\text{bj/in}}}^{t\_{\text{cnd}}} R \, k\_c^2 \, \tau^2 dt / \langle T\_M \, D \rangle \end{aligned}$$

with Emotor resistance being the integral over time of the electric power dissipated, R being the motor resistances, k<sup>c</sup> being the electric motor torque constant and τ being again the torques applied at the robot joints.

• The total energy consumed during the walking distance D and the execution time TM,

$$E\_{\text{total}} = E\_{\text{mechanical}} + E\_{\text{motor resistance}} + E\_{\text{electrinos}}$$

with Etotal being the sum of the energy consumed by the system normalized over the walking distance D and the execution time TM, and Eelectronics being the energy consumed by the on-board electronic cards. Eelectronics is neglected in this study so:

$$E\_{total} = E\_{mechanical} + E\_{motor\ resistance}$$

• The mechanical cost of transport and the total cost of transport,

$$E\_{\text{mechanical cost transport}} = \int\_{t\_{\text{bogin}}}^{t\_{\text{end}}} |\tau \omega| dt / (m \text{ g } D)$$

$$E\_{\text{total cost transport}} = \left( \int\_{t\_{\text{bogin}}}^{t\_{\text{end}}} |\tau \omega| dt + \int\_{t\_{\text{bogin}}}^{t\_{\text{end}}} R \text{ } k\_{\text{d}}^2 \text{ } \tau^2 \text{ dt} \right) / (m \text{ g } D)$$

with Emechanical cost transport and Etotal cost transport being, respectively, the mechanical and total cost of transport, m being the total mass of the robot, and g = 9.81 ms−<sup>2</sup> the gravity constant.

• The Froude number,

$$F\_r = \frac{\nu}{\sqrt{gl}}$$

$$\nu = \frac{D}{T\_M}$$

where v is the robot center of mass mean velocity along the horizontal plane and l is the leg length. This number

represents the ratio between the kinetic energy and the potential energy. It can also be interpreted as an indicator on the stepping frequency.

The trajectories were generated off line and repeatedly played on the robot to analyze their robustness. Views of the experimental setups are given in **Figure 7**.

## 3.8. Motion Generation for Humanoid Robot Locomotion

This section explains the links between the motion generation architecture depicted in **Figure 1** and the Key Performance Indicators given in the section 3.7. The set of functions entitled body posture, depicted in **Figure 1** (right), represents the behavior which is provided by what is called a whole-body controller. It consists of two parts:


In this paper we have evaluated independently only one whole body motion controller. It is the stabilizer provided by Kawada Inc. We give detailed performances evaluation of this controller in the experimental part of this paper. It was described in various paper such as Kajita et al. (2007) and Kajita et al. (2001).

The set of function entitled body transport, depicted in **Figure 1** (right) in this paper, are four CDPG and one MPWBC. The four CDPG evaluated in this paper are the following ones: Carpentier et al. (2016), a multi-contact centroidal dynamic pattern generator used to climb stairs with given contact positions, Kajita et al. (2003a), the original walking pattern generator implemented by Shuuji Kajita with given foot steps, Morisawa et al. (2007), an analytical walking pattern generator allowing immediate foot step modifications, Naveau et al. (2017), a real time nonlinear pattern generator able to decide autonomously foot-steps positions. In each case the goal of the CDPG is to generate a center of mass trajectory and the foot-steps trajectories. For Kajita et al. (2003a), Naveau et al. (2017), and Morisawa et al. (2007) a dynamical filter is used to correct the center of mass trajectory to improve the dynamical consistency of the motion. In each case, a whole body motion generator (not to be confused with a whole body motion controller) is used without feedback to generate the reference position q ref , and the desired z ref which are then sent to the stabilizer. For Naveau et al. (2017) and Morisawa et al. (2007) we used the stack of tasks described in Mansard et al. (2009) as a Generalized Inverse Kinematics scheme. In Carpentier et al. (2016) a Generalized Inverse Dynamics was used to generate the reference value for q ref and c ref . The MPWBC provides the controls directly. The one used is from Koch et al. (2014) using the Muscod-II Diehl et al. (2001) nonlinear solver.

## 4. RESULTS

In this paragraph we present the numerical results obtained from the computation of the KPI explained in detail in section 3.7 for each set of experiments. As a reminder the list of the KPI is recalled:


A video displaying a mosaic of all the experiments is available at the following URL: https://www.youtube.com/ watch?v=djWGsb44JmY&feature=youtu.be or as a **Supplementary Material** on the editor site of this paper.

## 4.1. Climbing Stairs

## 4.1.1. Stairs of 10 cm

In this experiment, the humanoid robot HRP-2 is climbing stairs of 10 cm height without any handrail. The difficulty of this task is that the robot has to perform quite large steps and vertical motion. For this reason, the robot is climbing one stair at a time, which means that the robot puts successively one foot on the next stair and the other one on the same stair. This avoids a too large joint velocity that the robot could not track. Morisawa et al. (2007) CDPG was evaluated at the beginning of the project although the variation of height violates the assumption of the cart-table model. But thanks to the dynamical filter the motion generated was dynamically consistent so that the stabilizer could cope with the situation. Because this experiment was not performed at the LNE (it was done 3 years before) it was not possible to control carefully the room temperature but the test was performed at 20◦C. The KPI results can be seen in **Figure 11** (tool upstairs). The other test was performed at the end of the project using the CDPG (Carpentier et al., 2016). This time the CDPG took into account the center of mass height variation but not the whole body motion. The stabilizer should theoretically have less trouble to compensate for the simplifications made. For Carpentier et al. (2016) three different temperatures were tested: 10◦C, 20◦C, and 35◦C. The numerical results are depicted in **Figure 8**. Interestingly, the temperature level has a direct impact in terms of mechanical cost as it diminishes with the increase in temperature. It is reflected in the tracking error. This intertrial variation does not come from the change of reference trajectory as it is strictly the same for every trial. There is a level of adaptation due to the stabilizer, but each temperature has been tested at least 4 times. A possible explanation is the fact that the grease in the harmonic drives generates less friction at higher temperature. As the cost of transport is dimensionless it allows the two motions to be compared regardless of their duration. It is then interesting to see that the cost of transport in **Figure 11** (tool upstairs) and in **Figure 8** (10◦C) are very similar. And that, at the same temperature, the total cost of transport for Carpentier et al.

(2016) CDPG is 9.6% better (from 6.71 to 6.06). One explanation is that the motion from Carpentier et al. (2016) CDPG being more dynamically consistent, the stabilizer consumes less energy to compensate for the model simplifications.

## 4.1.2. Stairs of 15 cm

In this experiment, the humanoid robot HRP-2 is climbing stairs of 15 cm height using a handrail. In addition the robot is not using any stabilization algorithm, because there are non-coplanar contacts. In this setup the Morisawa et al. (2007) CDPG has to be used without handrail because of the model simplifications. Trials have therefore been done using a WBC (described in Mansard et al., 2009) without the handrail. The results show that the current demanded by the motors went up to 45 A. And because the HRP-2 batteries cannot provide more than 32 A, all trials failed. This is the reason why the results are not shown in this study. Nevertheless, tests using the handrail could be performed with Carpentier et al. (2016) CDPG. The corresponding results are depicted in **Figure 9**. It confirms that the energy is decreasing with the increase of temperature without the stabilizer. Note that the energy spent by the robot is clearly higher than for the experience on the 10 cm stairs, i.e., a 36% of increase of the energy for walking.

## 4.1.3. Stepping Stones

In this experience, the humanoid robot HRP-2 had to walk up and down on stairs made of red interlocking paving stones. Between each stair there is a height difference of ±5 cm. The CDPG described in Morisawa et al. (2007) was used. this test is slightly different from the previous experiments because the robot cannot put his two feet on a same level surface (contrary to a stair step). To cope with this, the generated trajectories had to always change the height of the next support foot. As the paving stones were always slightly moving due to the robot weight, the balance was difficult to obtain in a reliable way. As indicated in the graph depicted in **Figure 11**, despite a success rate of 1, the tracking error reaches a level (8e <sup>−</sup><sup>03</sup> rad). This tracking error is greater than the one obtained during the 10 cm climbing experiment at 10◦C but lower than the one obtained during the 15 cm climbing experiment at 35◦C (which is the lowest for this temperature and the CDPG). A possible explanation of why the energy consumption is greater than during the 10 cm climbing stairs might be the instability of the stones and the fact that in this experiment the robot climb the stairs in a human fashion, i.e., not one stair at a time.

## 4.2. Walking on a Beam

This experiment was realized using the CDPG Morisawa et al. (2007). In this experiment the humanoid robot HRP-2 is walking on a beam. Initially, the experiment success rate on a real beam was around 20%. This rate was improved to achieve a 90 % success rate, thanks a new implementation of the dynamical filter presented in Kajita et al. (2003a). It reduced the drift which is important as the beam length is 3 m long. This could probably be improved by a proper vision feed-back. However, in these experiments, the robot walked on a normal ground as if it was on a beam. The reason is the absence of a beam in the temperaturecontrolled room. Even though the foot step location is discarded,

FIGURE 11 | Multiple algorithms: Slopes at 5 degrees using Kajita algorithm (Skor), going up with a tool on a wooden pallet 10 cm (tool upstairs), going down on a wooden pallet 10 cm (down stairs), going over an obstacle solving an OCP approach (Muscod), stepping on a interlocking paving stones (stepping stones).

the balance problem is exactly the same. Here, the success rate is 1. The corresponding result is depicted in **Figure 10**.

To perform the motion on a limited bandwidth (beam), the robot has to execute faster motions with its legs in order to place its foot ahead the previous one. It is emphasized by the increase of the cost of transport compared to normal straight walking (see **Figure 12**). Though the robot's legs are moving faster, the step frequency is lowered compared to a normal walking in order to keep the joint velocities in the feasible domain. This is reflected by the fact that the Froude number is around 35% less than during a straight walking (see **Figure 12**).

## 4.3. Straight Walking on Flat Ground 4.3.1. Temperatures

In the temperature-controlled room the humanoid robot HRP-2 is performing a 2 m straight walking following the implementation of Kajita et al. (2003a). The corresponding result is depicted in **Figure 12**. Note that the energy with respect to the temperature is following the same trend as for the experiments on the stairs and on the beam. We also tested the algorithm (Naveau et al., 2017) at 10◦C. The total cost of transport is higher than the algorithm (Kajita et al., 2003a) at the same temperature but lower than the one used for walking over the beam. It is however strongly less than the total cost of transport for climbing stairs at 10◦C. The fact that the energy cost is higher for Naveau et al. (2017) than for Kajita et al. (2003a) at the same temperature is that Naveau et al. (2017) (illustrated in **Figure 13**) provides a higher range of motion but the generated motions are closer to the limit of the system, so the stabilizer spends more energy to compensate for this.

## 4.3.2. Bearing Weights

We made the humanoid robot HRP-2 walk while bearing weights at ambient temperature between 15◦ and 19◦ . The two algorithms Kajita et al. (2003a) and Naveau et al. (2017) were tested. The robot was able to walk while carrying up to 14 kg with the two algorithms. Note that, as expected, the effort to compensate for the additional weight reflects in the cost of transport.

## 4.3.3. Pushes

We performed pushes in the lateral direction and in the frontal direction while the robot was walking along a straight line. The two algorithms Kajita et al. (2003a) and Naveau et al. (2017) were again tested. In our case, the tested algorithm was not able to modify its foot-steps according to the pushes contrary to the impressive work by Takumi et al. (2017). For this specific set of experiments with push from the back, the robot was able to sustain forces from 31 N to 47 N. Pushes applied in the lateral plane were varying between 23 N and 40 N. For Kajita et al. (2003a), the cost of transport has a value of 3.31 similar to the one obtained when walking on the beam. It is lower than the cost of transport for climbing stairs. The cost of transport for Naveau et al. (2017) is of 4.08. For both algorithms pushes are among the most consuming behaviors. It is due to the stabilizer action to compensate for the perturbation.

## 4.3.4. Slopes

The robot walked on a straight line while being on a slope of various inclinations ([1◦ − 3.0◦ ]) -and with two possible directions (upward or downward). The two algorithms Kajita et al. (2003a) and Morisawa et al. (2007) were tested.

For Kajita et al. (2003a) the cost of transport is higher than for standard straight walking but far less than during the pushes. For Morisawa et al. (2007) the cost of transport is higher than when performing the pushes with Kajita et al. (2003a) approach and is at the same level than the beam test. It can be explained by the fact that when the experiment has been realized the dynamical filter was not used. Therefore the stabilizer had to compensate for the discrepancy between the motion dynamics and the reference given by the center of pressure. An algorithm able to estimate the ground slope and adapt the walking pattern to it would probably increase the efficiency of this motion.

## 4.3.5. Frictions

The robot walked on carpets with different textures including different friction coefficients. In this case, we did not see any consequences with the CDPG (Kajita et al., 2003a). This is probably due to the particular coating of HRP2 soles used, they might have avoided foot slippage. which is one way to affect the friction coefficient. A possible extension of this work would be to use more slippery ground. But a proper way to handle such case is to implement a slip observer such as it was done (Kaneko et al., 2005).

## 4.3.6. Uneven Terrain

The robot walked over gravels of calibrated size. We tested several diameters with the CDPG (Kajita et al., 2003a). The robot was able to walk on gravels of size up to 8 mm. Beyond this size, the robot was falling. Note that in **Figure 12** the cost of transport is slightly more expensive than for classical straight walking at nominal temperature, but not much than walking at 10◦C. It is far less expensive than climbing a slope or counteracting pushes. As expected it has no impact on the frequency of the footstep as can be reflected by the Froude Number.

## 4.3.7. Walking Over an Obstacle

We have computed the same performance indicators to achieve the task described in Koch et al. (2014) in the frame of the Koroibot project. This strategy is quite different from the others as it implements a MPWBC under the formulation of an Optimal Control Problem given by Equation (1). The solution of this problem was computed by the Muscod-II (Diehl et al., 2001) solver. As the solver is trying to maximize a solution which is not on a reduced space (the centroidal dynamics for the previous algorithms), but on the whole robot, the solution found is close to the limits of the robot in terms of joint position, velocity, acceleration and torques. This is reflected in the cost of transport which is very high, 10.15, almost as high as for climbing the stairs of 15 cm (see **Figure 11**, Muscod).

## 4.4. Stabilizer

The stabilizer described in Kajita et al. (2007) and Kajita et al. (2001) was extremely resilient during all the tests. A horizontal testbed platform was used to generate oscillations along the sagittal plane and the perpendicular plane at 1 and 2 Hz at various amplitude [10, 20, 30, 40, 48] in mm. Along the sagittal plane at 40 and 48 mm for both frequencies the feet of the robot were raising up. In the perpendicular plane at 40 and 48 mm for both frequencies the overall robot rotated of about 15◦

and 20◦ . It was also tried to increase the frequency for a given amplitude of 10 mm. In the sagittal plane, the robot was able to reach 7 Hz without falling. In the perpendicular plane at 7 Hz the robot was making violent oscillations (without falling) reaching mechanical resonance. The trial was subsequently

stopped. The results are depicted in **Figure 14**. We can clearly see that for the oscillation in the perpendicular plane the increase of total energy is following an exponential curve, compared to the same experience in the sagittal plane. This clearly shows that the resonance frequency of the system was reached as it can be seen in the video available at the following location https://www.youtube.com/watch?v=djWGsb44JmY&feature= youtu.be or as a **Supplementary Material** on the editor site of this paper.

## 5. DISCUSSION

Human performance in locomotion tasks is still unmatched by humanoid robots. Because of the lack of assessment methods shared and accepted by the entire robotics community, it is even difficult to estimate the level of maturity of existing technologies. A response to these evaluation needs should induce significant advances in robotics-research. Such an influence of evaluation on the progression of technology performance has been observed in the past, in particular for computer vision and NLP tasks (Martin, 2004).

The definition of evaluation protocols including testing scenarios, testing environments and KPIs or metrics is crucial for the definition of common standards for:


This study contributes to the definition of these performance evaluation standards by proposing reproducible experiments and evaluate repeatable performance measurements. These evaluation methods are intended to be passed on to the robotics research community and to standardization committees. In addition we proposed one of the first thorough evaluation of such performance indicators on a human size humanoid robot.

## 5.1. Summary and Major Outcomes

In this paper we presented a benchmarking for the control architecture described in **Figure 1** that was implemented on the HRP-2 robot owned by LAAS-CNRS. The performance indicator used in this paper are mostly based on Torricelli et al. (2015). Based on this work we computed the following set of KPI:


These KPI represent either the particular characteristics of the experiments or the performances of the control architecture used. The list of algorithms executed on the HRP-2 robot were:


The list of environmental conditions where the tests could successfully be performed is:


The list of motion performed in the environmental conditions :


From all these results and experiments few major results come out. First the temperature plays a role on the energy consumed during a motion. We observed that the colder the room is, the more mechanical and electrical energy is consumed. We also noticed that the more the motion is at the limit of stability the more the stabilizer has to inject energy into the system to compensate for potential drift. This creates a noticeable increase in energy consumption, e.g., when the robot walks on a beam, steps over obstacle, walks on stepping stones. However the most expensive motion is climbing stairs which is clearly a challenge for future potential applications in which stairs are involved. Finally, in terms of cost of transport, the algorithm proposed by Carpentier et al. (2016) seems to be the most efficient and the most versatile. Its main disadvantage during this campaign was the lack of on-line implementation compared to Morisawa et al. (2007) and Naveau et al. (2017).

## 5.2. Limits

The main limit in the approach proposed here is the difficulty to make the experiments to be more statistically significant. In its current form at least 3 people are needed to perform one experiment, which makes them error-prone and time consuming. Given the wide range of motions that a humanoid robot is able to perform, wear testing needs humanoid robots to be able to fall down and stand up again and restart their behavior. This is a current hot topic in humanoid robotics. The Atlas humanoid robot built by Boston Dynamics has recently demonstrated its capabilities to fall down without breaking and stand up. HRP-2 is an electric-based humanoid robot which is mechanically fragile due to its harmonic drive. Although several works (Fujiwara et al., 2006; Samy and Kheddar, 2015) have developed new approaches toward making such robot more resilient to falling, it is still difficult to implement them in practice due to the cost of failure. In the meantime, benchmarking will help to understand the repeatability and the robustness of the various algorithms implemented on humanoid robots. For very unstructured environments more tests will probably be needed, and a way to classify the environments necessary (using gravels, stairs, size of stairs, different shapes of stairs, or database of environments, forests). But so far such environments can be handled only by a small number of humanoids and the approach proposed in this paper is feasible for a large set of current humanoid robots.

## 5.3. Future Work

We could not properly compute the KPI when trying to vary the friction of the ground. A future work is then to implement a proper slip observer like the one in Kaneko et al. (2005). based on this observer, Based on this we should build a stabilizer that could be used in multi-contact motions in order to compensate for external perturbations and modeling assumption. Furthermore, the LAAS-CNRS has acquired a new humanoid robot Talos (Stasse et al., 2017). The future work consists in implementing all the algorithms presented in this paper and perform the benchmarking on this new robot.

## REFERENCES


## DATA AVAILABILITY STATEMENT

As a reminder, a CAD model of the staircase used is available on the github repository where all the log of the experiments are also present: https://github.com/laas/koroibot\_KPI. All the computation performed on the logs and implementing the key performance indicators are available here: https://github.com/ laas/EnergyComputation.

## AUTHOR CONTRIBUTIONS

OS, EB, and KG--E conducted the experiments on the temperature, climbing stairs, at the LNE. MN and OS conducted the experiments with Koroibot. OS, EB, and PS conceived the research idea. PS obtained funding for the project. OS, KG--E, MN, EB, RR, GA, and PS participated in the preparation of the manuscript.

## ACKNOWLEDGMENTS

The financial support provided by the European Commission within the H2020 project ROBOCOM++ (JTC2016-PILOTS, Projet-ANR-16-PILO-0001) is gratefully acknowledged.

## SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frobt. 2018.00122/full#supplementary-material


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer FA and handling Editor declared their shared affiliation.

Copyright © 2018 Stasse, Giraud--Esclasse, Brousse, Naveau, Régnier, Avrin and Souères. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.