# ADVANCES IN MECHATRONICS AND BIOMECHANICS TOWARDS EFFICIENT ROBOT ACTUATION

EDITED BY : Jörn Malzahn, Navvab Kashiri, Monica Daley and Nikos Tsagarakis PUBLISHED IN : Frontiers in Robotics and AI

#### Frontiers Copyright Statement

© Copyright 2007-2019 Frontiers Media SA. All rights reserved. All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.

The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.

Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.

Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.

As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.

All copyright, and all rights therein, are protected by national and international copyright laws.

The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use. ISSN 1664-8714 ISBN 978-2-88945-911-7 DOI 10.3389/978-2-88945-911-7

### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

## Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# ADVANCES IN MECHATRONICS AND BIOMECHANICS TOWARDS EFFICIENT ROBOT ACTUATION

Topic Editors:

Jörn Malzahn, Istituto Italiano di Tecnologia, Italy Navvab Kashiri, Istituto Italiano di Tecnologia, Italy Monica Daley, Royal Veterinary College, United Kingdom Nikos Tsagarakis, Istituto Italiano di Tecnologia, Italy

Image: sdecoret/Shutterstock.com

Citation: Malzahn, J., Kashiri, N., Daley, M., Tsagarakis, N., eds. (2019). Advances in Mechatronics and Biomechanics towards Efficient Robot Actuation. Lausanne: Frontiers Media. doi: 10.3389/978-2-88945-911-7

# Table of Contents

*05 Editorial: Advances in Mechatronics and Biomechanics Towards Efficient Robot Actuation*

Jörn Malzahn, Navvab Kashiri, Monica A. Daley and Nikos Tsagarakis


Svetlana Grosu, Carlos Rodriguez–Guerrero, Victor Grosu, Bram Vanderborght and Dirk Lefeber

*74 Efficiency and Power Limits of Electrical and Tendon-Sheath Transmissions for Surgical Robotics*

Christopher R. Wagner and Evangelos Emmanouil

*91 The Making of a 3D-Printed, Cable-Driven, Single-Model, Lightweight Humanoid Robotic Hand*

Li Tian, Nadia Magnenat Thalmann, Daniel Thalmann and Jianmin Zheng


Stefan S. Groothuis, Gerrit A. Folkertsma and Stefano Stramigioli

*137 Highly-Integrated Hydraulic Smart Actuators and Smart Manifolds for High-Bandwidth Force Control*

Victor Barasuol, Octavio A. Villarreal-Magaña, Dhinesh Sangiah, Marco Frigerio, Mike Baker, Robert Morgan, Gustavo A. Medrano-Cerda, Darwin Gordon Caldwell and Claudio Semini


# Editorial: Advances in Mechatronics and Biomechanics Towards Efficient Robot Actuation

Jörn Malzahn<sup>1</sup> \*, Navvab Kashiri <sup>1</sup> , Monica A. Daley <sup>2</sup> and Nikos Tsagarakis <sup>1</sup>

<sup>1</sup> Humanoids and Human Centred Mechatronics Lab, Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genova, Italy, <sup>2</sup> Structure and Motion Laboratory, Royal Veterinary College, London, United Kingdom

Keywords: energy efficient actuation, actuator design, actuator control, energetics of human locomotion, energetics of animal locomotion, bioinspired control, bioinspired design

**Editorial on the Research Topic**

### **Advances in Mechatronics and Biomechanics Towards Efficient Robot Actuation**

Biological systems such as animals and humans can be energetically autonomous for several days and some species endure even longer. Today's robotic systems aim to match the physical capabilities of their biological counterparts in terms of strength, agility, and dexterity, yet fail to display untethered operating times any longer than few hours. Changing this situation is challenging and likely requires fundamental shifts in the current paradigms for robot designs.

Today, new and promising paradigms have evolved from the recent advances at the interface of robotics and biomechanics. Robotics has become a prominent tool for validating and testing biomechanical models of animal locomotion, allowing direct testing of hypotheses about the mechanisms behind locomotion in nature. The physical implementation of the biomechanical models in robotic systems allows quantitative evaluation of theory and comparison between replicated movement and force patterns to those observed in biology. This approach has recently advanced our understanding of nature's locomotion principles, in particular for the integration of neural control and actuation for agile locomotion.

These new biomechanics insights are now being turned into novel mechatronic paradigms and solutions for dynamic and efficient walking robots and robust and lightweight robotic prostheses and orthoses. Conventional robot designs have been dominated by chains of rigid links with high transmission ratio non-backdrivable joints. The integration of intrinsically compliant components into robot designs has been researched for more than two decades now. However, we have only recently gained sufficient understanding to effectively exploit passive springs and intrinsically backdrivable low transmission ratio actuators for low inertia designs, with force sensing and control for efficient and impact resilient actuation. Impact resilience is especially important because impacts on the mechanism represent normal operating conditions for legged locomotion in unstructured environments and realistic terrain. Further insights on neural control in biological systems continues to spawn new paradigms for hierarchical and hybrid local/distributed control architectures. These bio-inpsired control advances help to tackle the task complexity involved in enabling robots to robustly perform in real-world scenarios outside the lab. Recent findings in this realm suggest that, in some cases, control loops need not run at high rates (> 1 kHz) to achieve robust performance. Sometimes closing a loop, with the associated delays and efforts to ensure stability, might not be advisable at all. Instead feedforward actions along with carefully shaped natural dynamics can lead to simpler, more robust and efficient solutions.

Edited and reviewed by: Giuseppe Carbone, University of Calabria, Italy

> \*Correspondence: Jörn Malzahn jorn.malzahn@iit.it

#### Specialty section:

This article was submitted to Humanoid Robotics, a section of the journal Frontiers in Robotics and AI

Received: 21 January 2019 Accepted: 04 March 2018 Published: 26 March 2019

#### Citation:

Malzahn J, Kashiri N, Daley MA and Tsagarakis N (2019) Editorial: Advances in Mechatronics and Biomechanics Towards Efficient Robot Actuation. Front. Robot. AI 6:19. doi: 10.3389/frobt.2019.00019

The present Research Topic joins research efforts from the robotics, control, and biomechanics communities toward novel robot design and control paradigms. The joint efforts included the organization of a physical work meeting in the form of a joint workshop on the energetic economy of robotics and biological systems during the International Conference on Intelligent Robots and Systems (IROS) on September 24, 2017<sup>1</sup> .

Given the interdisciplinary nature of the Research Topic, it is organized under the sections "Humanoid Robotics" and "Robotic Control Systems" within Frontiers in Robotics and AI as well as the "Bionics and Biomimetics" section with Frontiers in Bioengineering and Biotechnology. The collection of articles in this Research Topic captures a wide-angled perspective on the current state-of-the-art in robotic as well as biodynamic actuation modeling, analysis, design and control toward enhanced robot energy efficiency.

The wide-angled perspective is initialized with an overview article by Kashiri et al. on fundamental principles of locomotion inspired from natural biological systems, and principles for the design and control for efficient robot locomotion. As a foundation, Lee and Harris present and discuss a framework for quantitative evaluation of locomotion efficiency in animals and robots. The Research Topic further elaborates on some of the locomotor principles outlined in the overview with dedicated articles on minimally actuated walkers (Schroeder and Bertram) and articulated compliant parallel actuation arrangements (Roozing).

Nature demonstrates elegant solutions to minimize the inertia of moving limbs through remote tendon driven actuation. Several articles in the Research Topic are devoted to the potential and the challenges encountered during the technical implementation and application of such principles (Grosu et al.; Tian et al.; Wagner and Emmanouil).

Urbina-Meléndez et al. model and study how observability and state-estimation in balancing skills depends on the physical location of sensory organs. Nguyen et al. investigate the activation and response behavior of biological muscles. Engineers seeking to find suitable target functions for actuator designs or optimal sensor placements in their robots can greatly benefit from these insights, which emerge from millions of years of natural evolutionary design iterations.

Nature seems to effortlessly manage complexity, not only concerning growing and self-regenerating biomechanics, but also considering efficient and effective control of millions of cells, sensors and muscle fibres. In this Research Topic Groothuis et al. address this topic in the context of energy aware control across distributed control hierarchies as observed in biology and—with the aim to manage system complexity—also applied to technical systems (Barasuol et al.).

Apart from mobile and legged robots, the direct beneficiaries of the mutual insights in biodynamics and robotics engineering are humans. This Research Topic includes examples of robotic rehabilitation systems (Ghannadi et al.) as well as assistive robotic devices such as passive and active prostheses (Jeffers and Grabowski; Tahir et al.) to improve or recover the quality of life.

The editors hope that the results presented in this Research Topic will catalyze the advent of new robots and assistive robotic devices with enduring energetic autonomy, which will help us tackle the upcoming societal and economic challenges. At the same time the presented works report unique and invaluable experimental data that help verifying and refining theories as well as hypotheses striving to understand the nature of the species living on our planet (Nguyen et al.; Urbina-Mele'ndez et al).

## AUTHOR CONTRIBUTIONS

All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.

## ACKNOWLEDGMENTS

We gratefully acknowledge the kind support of the Frontiers Editorial Team during all stages of this Research Topic. Moreover the authors would like to thank all authors for submitting their high class works, which have truly rendered this Research Topic an all-inclusive image of the current state-of-the-art in efficient robotic actuation. Last but not least, we would like to express many thanks to all reviewers, who have contributed their time and effort to provide valuable suggestions and enhancements to the articles presented in this Research Topic.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Malzahn, Kashiri, Daley and Tsagarakis. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

<sup>1</sup>https://energeticeconomyrobotics.wordpress.com/

# An Overview on Principles for Energy Efficient Robot Locomotion

Navvab Kashiri <sup>1</sup> \*, Andy Abate<sup>2</sup> , Sabrina J. Abram<sup>3</sup> , Alin Albu-Schaffer <sup>4</sup> , Patrick J. Clary <sup>2</sup> , Monica Daley <sup>5</sup> , Salman Faraji <sup>6</sup> , Raphael Furnemont <sup>7</sup> , Manolo Garabini <sup>8</sup> , Hartmut Geyer <sup>9</sup> , Alena M. Grabowski <sup>10</sup>, Jonathan Hurst <sup>2</sup> , Jorn Malzahn<sup>1</sup> , Glenn Mathijssen<sup>7</sup> , David Remy <sup>11</sup>, Wesley Roozing<sup>1</sup> , Mohammad Shahbazi <sup>1</sup> , Surabhi N. Simha<sup>3</sup> , Jae-Bok Song<sup>12</sup>, Nils Smit-Anseeuw<sup>11</sup>, Stefano Stramigioli <sup>13</sup>, Bram Vanderborght <sup>7</sup> , Yevgeniy Yesilevskiy <sup>11</sup> and Nikos Tsagarakis <sup>1</sup>

<sup>1</sup> Humanoids and Human Centred Mechatronics Lab, Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genova, Italy, <sup>2</sup> Dynamic Robotics Laboratory, School of MIME, Oregon State University, Corvallis, OR, United States, <sup>3</sup> Department of Biomedical Physiology and Kinesiology, Simon Fraser University, Burnaby, BC, Canada, <sup>4</sup> Robotics and Mechatronics Center, German Aerospace Center, Oberpfaffenhofen, Germany, <sup>5</sup> Structure and Motion Laboratory, Royal Veterinary College, Hertfordshire, United Kingdom, <sup>6</sup> Biorobotics Laboratory, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland, <sup>7</sup> Robotics and Multibody Mechanics Research Group, Department of Mechanical Engineering, Vrije Universiteit Brussel and Flanders Make, Brussels, Belgium, <sup>8</sup> Centro di Ricerca "Enrico Piaggio", University of Pisa, Pisa, Italy, <sup>9</sup> Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, United States, <sup>10</sup> Applied Biomechanics Lab, Department of Integrative Physiology, University of Colorado, Boulder, CO, United States, <sup>11</sup> Robotics and Motion Laboratory, Department of Mechanical Engineering, University of Michigan, Ann Arbor, MI, United States, <sup>12</sup> Department of Mechanical Engineering, Korea University, Seoul, South Korea, <sup>13</sup> Control Engineering group, University of Twente, Enschede, Netherlands

#### Edited by:

Francesco Becchi, Danieli Telerobot Labs, Italy

#### Reviewed by:

Wiktor Sieklicki, Gdansk University of Technology, ´ Poland Giovanni Stellin, Danieli Telerobot Labs, Italy

> \*Correspondence: Navvab Kashiri navvab.kashiri@iit.it

#### Specialty section:

This article was submitted to Humanoid Robotics, a section of the journal Frontiers in Robotics and AI

Received: 18 March 2018 Accepted: 01 November 2018 Published: 11 December 2018

#### Citation:

Kashiri N, Abate A, Abram SJ, Albu-Schaffer A, Clary PJ, Daley M, Faraji S, Furnemont R, Garabini M, Geyer H, Grabowski AM, Hurst J, Malzahn J, Mathijssen G, Remy D, Roozing W, Shahbazi M, Simha SN, Song J-B, Smit-Anseeuw N, Stramigioli S, Vanderborght B, Yesilevskiy Y and Tsagarakis N (2018) An Overview on Principles for Energy Efficient Robot Locomotion. Front. Robot. AI 5:129. doi: 10.3389/frobt.2018.00129 Despite enhancements in the development of robotic systems, the energy economy of today's robots lags far behind that of biological systems. This is in particular critical for untethered legged robot locomotion. To elucidate the current stage of energy efficiency in legged robotic systems, this paper provides an overview on recent advancements in development of such platforms. The covered different perspectives include actuation, leg structure, control and locomotion principles. We review various robotic actuators exploiting compliance in series and in parallel with the drive-train to permit energy recycling during locomotion. We discuss the importance of limb segmentation under efficiency aspects and with respect to design, dynamics analysis and control of legged robots. This paper also reviews a number of control approaches allowing for energy efficient locomotion of robots by exploiting the natural dynamics of the system, and by utilizing optimal control approaches targeting locomotion expenditure. To this end, a set of locomotion principles elaborating on models for energetics, dynamics, and of the systems is studied.

Keywords: variable impedance actuators, energy efficiency, energetics, cost of transport, locomotion principles, bio-inspired motions

## 1. INTRODUCTION

The recent development in design and control of active or intrinsically controlled Variable Impedance Actuators (VIA) has demonstrated remarkable advancements in safety, robustness and peak power performance. However, despite the above, considerable performance improvements and progress made in the past 20 years in mechatronics and control, the motion/locomotion efficiency of even the most energy efficient robots still remains many times smaller than that of humans or animals. Due to these deficiencies there are several untethered applications (humanoids, manipulators, assistive and power augmentation exoskeletons, prostheses) where the limited power autonomy prevents their full practical exploitation. The advancement on robotic economy will therefore substantially impact the viable exploration of robotics in all applications requiring untethered operation. To enable this progress, new design principles and technologies are needed.

This paper<sup>1</sup> reviews the recent advancements in new robot design principles and control that target to reduce their energy consumption and lead to robots that are more efficient. The main objective is to present an overview of recent activities on the development of these new robot machines. Emerging topics to improve energetic performance of these robotic systems include novel joint-centralized variable impedance actuation, energy neutral intrinsic load cancellation and lock/release mechanisms, variable recruitment actuation principles, bioinspired distributed compliance actuation, embedded energy buffering and high efficiency power transmission concepts. The exploration of control principles for effective energy recycling and load cancellation during motion is also fundamental for improving the energy efficiency of these machines and will also be discussed with an emphasis on techniques for the exploitation of intrinsic resonance modes and energy efficient impedance regulators for under-actuated variable impedance or multi-articulated robots. Contributions on the biological side, particularly on the energy economy of humans and animals as well as on the biomechanics of locomotion efficiency, are complimentary and will provide the ground reference for today robot efficiency as well as set the energy efficiency goal of future robotic machines. In summary, we aim at demonstrating recent developments in mechatronics and control with a focus on the energy economy of robotic systems, to advance the understanding of actuation and control principles contributing to energetic economy in biological systems, humans and animals.

To clarify the energy efficiency of current robotic systems, we compare the Specific Resistance<sup>2</sup> (SR) of biological systems with that of robots. For instance, a horse trots with a SR of 0.2, and humans walk with a similar SR of 0.2. Based on the data reported by Tucker (1975) for Cost of Transport (CoT) of humans and animals, passive walkers (see the work by McGeer, 1990) are considerably more efficient than humans as shown by Collins et al. (2001), although they possess very limited flexibility in terms of functioning versatility and demand carefully dynamics tuning of operating conditions e.g., initial states. Humanoid robots that are capable of replicating human-like motions and executing human tasks, however, render fairly larger CoT/SR as compared to humans: Asimo presented by Sakagami et al. (2002) exhibits an SR of 2 (1.8 KW for 1.5 m/s, with a mass of 54 kg) and Durus introduced by Reher et al. (2016) targets an SR of 1. These are amongst the most efficient humanoids robot while the CoTs of them are ten and five times larger than human CoT, respectively. Similarly, the hydraulically actuated Big Dog operates with an SR of 15 that limits the autonomous operation time to about 30 minutes given the fuel capacity limitation (15 L of fuel for 20 Km, with a mass 110 kg). On the other hand, mammalian skeletal muscle exhibits a power density of 0.041 W/g and has about a 25% efficiency for concentric muscle action, while a motorized actuator can render higher values. An individual motor can possess a power density of 0.5 W/g and about 80% efficiency, although theses values may drop to 0.17 W/g and 40% when combined with a typical gearbox, respectively. Yet, the MIT cheetah actuators that exploit electrical energy regeneration/recycling demonstrate an SR of 0.5.

To understand efficient motion, we analyse human walking/running. Human locomotion comprises mostly unforced motion, where back-drivability significantly enhances the efficiency, and presents considerable energy storage due to recycling. Power consumption of the Walk-Man robot developed for performing disaster response tasks as the primary target, introduced by Tsagarakis and et al. (2017) requires about 387 W for electronics (45 W for perception system, 62 W for two processing units, and 280 for 36 motor driver electronics), and the total power consumption for standing is about 420 W. Slow walking (20 cm per second) requires a total power of 510–755 W in the most demanding condition. This shows that the maximum consumption includes an actuation power of 368 W; thereby representing an SR of 1.35. This describes an SR seven times higher than that of human walking only for actuation, while the total consumption expresses an SR of 2.8 which is 14 times larger than that of human walking. However, the lack of efficiency in comparison with humans is expected as the energy storage capacity of the system is limited to passive compliant elements with small deflection, that leads to large energy consumption for moving/accelerating joints. The other significant cause is the high gearing that renders large reflected inertia and results in a strict need for forced motions.

By incorporating the passive dynamics, as well as kinematic and actuator optimizations, the energetics performance of the robot can be significantly improved. Preliminary experiments on the bipedal robot CASSIE show that the 30 kg robot can walk at 1.0 m/s using a total of 200 Watts of power while performing different locomotion behaviors such as squatting, thereby rendering an SR of 0.7. This efficiency is owed to not only added compliance, but also to a leg design that (i) selects actuator/transmission through a joint-level actuator work minimization for performing walking tasks, see Rezazadeh and Hurst (2014); (ii) designs minimal toe inertia to reduce ground impacts, see Abate et al. (2015); (iii) utilizes a leg kinematics configuration which balances net task power among involved actuators (see Abate et al., 2016). Advancement in energy efficiency of robotic systems requires attention in various aspects of the robot operation problem, ranging from actuation and limb design to motion control. **Table 1**<sup>3</sup> presents a comparison between the energetics of biological systems and current robotic

<sup>1</sup>Contents of this paper are mainly based on the presentations of IROS 2017 workshop titled "On the Energetic Economy of Robotics and Biological Systems: a challenging handicap to overcome".

<sup>2</sup> Specific resistance is an index used to evaluate the energy efficiency of a mobile robot. It is defined as the ratio of the total energy consumption E for a travel of a distance d to the gravitational potential energy when the robot is lifted by the distance d, i.e., <sup>E</sup> Mgd with M and g representing the robot mass and gravitational acceleration; reported also by Kajita and Espiau (2008). This measure is equivalent to Cost of Transport (CoT).

<sup>3</sup>biological systems data is extracted from https://en.wikipedia.org/wiki/Muscle, Basalmetabolicrate,Humanbrain,Foodenergy


systems. This paper reviews recent advancements in design and control of robots, to elucidate the energetic state of current robotic and mechatronic systems in comparison with biological systems, and to derive insights and features for the development of more energy efficient robots. The rest of the paper is structured as follows: section 2 reports the variety of compliant actuators propounded for enhancing energy efficiency of robots, (see **Figure 1**). Section 3 discusses the importance of limb segmentation in design, dynamics and control, and how the robot design should rely on this information. Section 4 discusses a set of state-of-the-art energy efficient control methods on the basis of bio-inspired and optimal control principles. Section 5 describes a set of cutting-edge concepts initiating novel directions for robot locomotion. Finally, the paper summary is described in section 6.

## 2. COMPLIANT ACTUATION

Despite the potential of most electric motors for electrical energy regeneration, it is not often used in robotic systems, except for highly dynamic robots using direct drives e.g., cheetah robot presented by Seok et al. (2015). The lack of use is likely due to substantial losses in high gear reduction systems typically used in robotics requiring high torques, as discussed by Verstraten et al. (2015), thereby rendering significant copper losses (heat produced by electrical currents in motor windings) and leading to trivial energy regeneration. Furthermore, the actuation systems typically suffer from having the link and motor in a series chain. As a result, all torques pass from the motor, and even in stationary conditions, i.e., zero mechanical power, there is a non-negligible electrical energy consumption. Mechanical storage mechanisms/methods are therefore exploited in robotic machines, which often rely on integration of passive elastic elements into actuation units. Nevertheless, a suitable choice of a mechanical storage system cannot be based only on the mechanical energy domain. As discussed in Verstraten et al. (2016), the dynamics of reduction system and motor dynamics, as well as the operating position and range of motion, influence the optimal compliance and whether series and/or parallel

compliance is more beneficial. As shown in Beckerle et al. (2017), while the use of compliance in series with the motor can often be more energy efficient, compliance in parallel may become more beneficial when the operating point changes (therefore more static force).

As animal structure and motion have simultaneously evolved to be specifically designed to perform desired tasks efficiently and effectively, it is essential to account for both morphology and motion in robot design and control. Yesilevskiy et al. (2015, 2018a) discussed the effect of morphological variations in legged robots, and showed for a one-dimensional monoped hopper that is driven by a geared DC motor, with the correct choice of the transmission parameter, a hopper with Series Elastic Actuators (SEA) is more energetically efficient than one with Parallel Elastic Actuator (PEA). It is mostly due to the fact that, for a hopper with PEA, the motor inertia contributes to energetic losses due to the ground contact collisions. The energy saving capacity of this kind of compliance arrangement had been shown not only when used in legs, but also when used in torso structure as shown by Folkertsma et al. (2012). In all these cases, the natural mechanical dynamics connect the motion of a robot or animal to its morphology, thereby inherently coupling control and design of such systems.

## 2.1. Compliance in Series

The incorporation of built-in compliance into robot joints was primarily introduced to enhance the shock tolerance capacity via generation of inelastic collision force spikes, system responsivity due to higher force control bandwidth, and energy efficiency by cycling the energy flow; as well as amplifying output power. To address the limited storage capacity of the passive compliant component, various designs enabling larger deflection were proposed, e.g., by Tsagarakis et al. (2009). As presented by Verstraten et al. (2016), the inclusion of compliance in parallel or in series allows a decrease in both peak power and energy consumption provided that the stiffness of the elastic element is tuned properly, and substantiates the development of variable stiffness actuators, see Vanderborght et al. (2013).

Nevertheless, compliant systems exhibit unwanted vibrations that require inclusion of active, semi-active and passive damping. While active impedance control has been widely studied, e.g., by Ferretti et al. (2004); Kashiri et al. (2014b), the active dissipative action is often limited by the control-loop bandwidth, and suffers from feedback noise and phase-lag problems. The semi-active and passive damping therefore proved to be more effective as presented by Laffranchi et al. (2014) and Kashiri et al. (2017a). However, the inclusion of damping amplifies the complexity and total mass of the system. The former escalates the development and maintenance costs, and the latter reduces the energy efficiency. Moreover, semi-active solutions require additional controllers that intensify the complexity of the system, e.g., see works by Kashiri et al. (2014a), Kashiri et al. (2015), and Kashiri et al. (2016). As a result, despite the proven energy consumption reduction achieved by such systems, the utilization of variable impedance solutions, when considering the additional hardware/software complexity, requires careful attention to the application requirements.

## 2.2. Compliance in Parallel

## 2.2.1. Asymmetric Configuration

To develop a robotic actuator that combines high power and physical robustness with energetic efficiency, Tsagarakis et al. (2013) proposed an asymmetric compliant antagonistic joint concept, and developed a 1-DOF knee-actuated prototype leg. The novel design featured variable quasi-static load compensation, moderate gearing, large energy storage capacity, and controllable energy storage/release. It includes two branches working in parallel to each other: a high power branch, and an energy storage branch. The first branch embodies a series elastic actuation system, with an elastic element serving as a bi-directional coupling between the drive and the output link, protection of the drive unit, and torque sensing as demonstrated by Kashiri et al. (2017b). The second branch includes a low power motor coupled with a high reduction efficient linear transmission in series with a passive elastic transmission, and an elastic element with large energy storage capacity. To achieve this, it uses an elastic band similar to bungee cords, instead of metal springs. The coupling between the low power drive and the elastic element is done using a non-backdrivable transmission component, and a two-way overrunning clutch module, to remove the effort for maintaining pretension of the elastic element. On the basis of this concept, Tsagarakis et al. (2014) demonstrated the efficiency benefits of the large energy storage capacity in cyclic motion operations and in static load compensation in a unit with Series Parallel Elastic Actuator (SPEA). Roozing et al. (2016b) further expanded the work and proposed an integrated control strategy that actively utilized both branches, and experimentally demonstrated a 65% reduction in electrical power consumption when compared to conventional SEA.

Roozing et al. (2018) presented the development of a semi-anthropomorphic 3-DOF leg design, where series elastic actuation units are complemented with parallel high efficiency energy storage branches. Based on the earlier work in Roozing et al. (2016a), three exchangeable actuation configurations were described, where the energy efficiency achieved by parallel elasticity branches in mono- and bi-articulated configurations is demonstrated, as compared to a series-elastic-only configuration as a baseline. In biomechanical systems, biarticulated muscles span multiple joints and thereby allow the transfer of mechanical power between joints. Similarly, in the bi-articulated actuation configuration, the energy storage branch allows the transfer of mechanical power between the robotic knee joint and ankle joint. The multi-DOF energy storage branch system can thus provide a desired torque profile over the range of motion in an efficient manner to obtain minimal energy consumption and/or maximum gravity compensation. Additionally, it can increase peak torque output and assist in explosive motions such as pushoff during running. The leg presented in Roozing et al. (2018) performed a set of squat motion in three different configurations: without parallel elastic storage, and with parallel elasticity in mono-articulated and in bi-articulated configurations, with overall masses of 7.57, 9.14, and 9.21 kg, respectively, and exhibiting 33.1, 15.4, and 13.2 W electrical motor power when lifting 20 kg by 25.6 cm. On the other hand, humans<sup>4</sup> consume approximately 41, 53, and 54 W to perform a similar squat motion, for the overall mass of afore-mentioned configurations. If we account for efficiency of actuation in both cases (20% in muscles and 40% in robotic drives), the mechanical output power of the robotic leg is 13.2, 6.2, and 5.3 W, while that of humans is 8.2, 10.6, and 10.8 W; confirming the importance and effectiveness of parallel elasticity units.

## 2.2.2. Symmetric Configuration

Due to a strong demand for energy-efficient, yet low-cost, robotic arms, it is necessary to minimize the torque required to operate a robot while maintaining high performance. Considering robots developed for interaction purposes often operate at low speeds, the dominant torque is to carry the robot weight, especially when the payload to robot mass ratio is low. Gravity compensators can therefore save a significant amount of mechanical energy. It is therefore beneficial to counterbalance the gravitational torques resulting from robot mass, and accordingly employ the motor efforts for gravitational torque of varying payload and the inertial torques. Despite several advancements toward such efficient robotic manipulators using spring-based counterbalancing, the majority of such systems are often bulky and heavy, with a small range of rotation, and their utilization is limited to a one or two DOFs, see designs proposed by Koser (2009),

<sup>4</sup>We used the relation extracted in Robergs et al. (2007); however, we used twice the mass value, i.e., 55.14, 58.28, and 58.42 kg, as the formula was extracted from data corresponding to two legs (and significantly larger mass than the prototype), and then we divided the output power by two. Moreover, we subtracted a basic metabolic power of 80 W from the overall value as we compare with the prototype's electrical motor power.

Nakayama et al. (2009), and Lacasse et al. (2013). Passive gravity compensation using a counterbalance spring-based mechanism can rely upon various concepts, including (1) Wire-based systems, e.g., the service robotic arm proposed by Kim and Song (2014); (2) Gear-based devices; (3) Linkage-based mechanisms, such as the slider-crank based system developed by Kim et al. (2016). Given a multi-DOF manipulator, it is necessary to deal with continuously varying gravitational torques that depend on the robot configuration, thereby increasing the need for multi-DOF counterbalancing.

A possible solution to this problem is the development of a mechanism with a reference plane corresponding to each link, so that the spring counterbalancing the gravitational torques connects the link to this reference plane, and accordingly compensates for the gravitational torque of each link independently. Such a reference plane therefore is required to automatically align with the gravity direction, and enables proper linking with neighboring reference planes in a way that the compensated torque is transmitted to the ground, and the motion of one link does not affect others. On the basis of this concept, Korea University developed a series of spatial robots with multi-DOF counterbalancing features: including SCORA-H, SCORA-V, and KU-WAD. The latter employs a counterbalance mechanism based on two sections: (1) Spring and wire: to generate the compensation torque exhibiting zero counterbalance error; (2) Timing belt and pulley: to provide a suitable reference plane interconnection, and render a wide operation range without dead points. Experiments on SCORA-V (with six DOFs, 49 kg mass and 8 kg payload capacity) show significant mechanical and electrical energy savings thanks to the mechanical gravity compensation feature. Without using the gravity compensation system, the robot requires about 160 W for maintaining a high gravity posture and consumes 170 Wh for 1 h of operation (a repetitive task), while these values reduce by 34 and 30% by exploiting the counterbalancing mechanisms, respectively. Such a reduction in electrical energy consumption enables the use of smaller motors and gearing transmission systems in the design process, and saves a considerable amount of energy in the long-term use of a robot.

## 2.3. Compliance in Series/Parallel With Locking Mechanism

Another solution addressing actuation inefficiencies was presented by Plooij et al. (2016); Geeroms et al. (2017), that exploits Lockable Parallel Elastic Actuator (LPEA), as well as series elasticity. Experiments on a knee prosthesis powered by this actuator showed an energy consumption of 65 J/stride and peak power of 100 W, which are considerably lower than those generated by the same system powered by a direct drive (400 W and 156 J/stride) and SEA (200 W and 98 J/stride). Mathijssen et al. (2016) proposed another potential solution: the use of redundancy in actuation when a set of lockable SEAs (LSEAs) are set in parallel, to show a clear difference in mechanical energy consumption due to parallel springs that can be loaded with respect to any desired position. However, Verstraten et al. (2018) discussed the advantages of exploiting actuation redundancy in a more generic case, i.e., a multi-motor drive as compared to a single drive, to present smaller size and mass, and more importantly, a considerably higher efficiency for slower speeds, and larger static operation range; although it shows lower maximum efficiency, and slightly lower bandwidth. Mathijssen et al. (2017) discussed a similar concept for development of a discrete muscle-inspired actuator, presenting how this approach allows overpowering binary drive units (solenoids).

Chen et al. (2013) presented the employment of a locking mechanism in parallel with compliance in series, in order to store and release energy at the right time. Malzahn et al. (2018) discussed the employment of locking mechanisms (clutch) in series with the drive train possessing compliance in series, i.e., Lockable SEA (LSEA). The work is inspired by human muscle activity in different phases of running. As presented by Novacheck (1998), while the stance phase requires highly active motion of the joints to overcome gravitational torque, motion of the joints during the swing phase is passively driven by gravitational and inertial link torques. It is therefore more efficient to relax the muscles during the swing phase of running and exploit the passive dynamics of the system. However, the high transmission ratios used in conventional electric robot actuation do not permit such an operation. The transmission ratio trades the high speed of electric drives for increased output torques, which enable high torque density actuator designs. But, the transmission ratio steps up the motor as well as gearing friction torques, which render the actuation barely backdrivable by gravitational and inertial torques. Unlike in the human example, the motor therefore has to actively drive and overcome the intrinsic friction throughout any motion phase. As an alternative to direct drives, a clutch mechanism in series between the transmission and the link can be utilized so that the conventional mature high torque density drive unit can be partially/fully disengaged when the passive dynamics can partially/fully generate the required motion. The energy saving potential of the series clutch approach depends on the motion dynamics and the ratio between the link gravitational and motor friction torques. The theoretical savings range between 20 and 60 % of the mechanical energy required to perform the same motion compared to the identical drive without series clutch actuation.

## 2.4. Purely Passive Compliance

While robotic platforms often utilize active joints, with and/or without Passive Elasticity (PE) integrated as discussed above, fully passive compliant joints are not often used except for endeffectors such as feet. However, they are commonly used in leg prostheses, where metabolic and biomechanical effects of people with lower extremity amputations is of utmost importance. Herr and Grabowski (2012) presented a powered prosthesis (BiOM) that enables people with a transtibial amputation to achieve normative metabolic costs, preferred walking speeds, and step-to-step transition work while walking over level ground across a wide range of speeds. D'Andrea et al. (2014) showed that use of the BiOM enhances the regulation of whole-body angular momentum, and therefore reduces fall risk. Grabowski and D'Andrea (2013) found that use of the BiOM reduces unaffected leg knee loading and thus osteoarthritis (OA) risk. Jeffers et al. (2015) compared changes in metabolic power and mechanical power during step-to-step transitions while nonamputee subjects walked on a range of slopes at different velocities. They found that at faster velocities, metabolic power increased, the leading leg absorbed more power, and the trailing leg generated more power compared to slower velocities. Moreover, with increasing slopes, mechanical work of the leading leg became more negative while mechanical work of the trailing leg became more positive. Jeffers and Grabowski (2017) also found that use of the powered prosthesis (BiOM) compared to a passive-elastic energy storage and return (ESAR) prosthesis, improves biomechanics and metabolic cost on uphill slopes.

To discuss the effects of compliance and geometry of a class of passive-elastic running-specific prostheses in series with the leg, a set of this class of prostheses developed by manufacturers based on subjective stiffness categories was studied. Beck et al. (2016) found that the prosthetic stiffness values of these manufacturer recommended stiffness categories varied between prosthetic models. They also found that the force-displacement profiles of such prostheses are curvilinear, indicating that prosthetic stiffness varies with the magnitude of applied force. Beck et al. (2017) investigated the effects of running-specific prosthetic stiffness, height and speed on the biomechanics of a set of athletes with bilateral transtibial amputations. They found that with use of stiffer prostheses, athletes could apply greater peak and stance average vertical ground reaction forces, increase overall leg stiffness (inversely associated with running speed), decrease ground contact time and increase step frequency; although these effects were reduced at faster running speeds. The effects of ±2 cm changes in prosthetic height on biomechanics (inversely associated with step frequency) were unchanged. It was also shown that J-shaped running-specific prostheses often outperform C-shaped prostheses in terms of both metabolic CoT and maximum speed in athletes with transtibial amputations.

## 3. SEGMENTED LIMBS

Periodic motions such as crawling, walking, and running, are typical tasks in which energy storage and release frequently occur; where muscle and tendon elasticity plays the most important role in biology. Efficient generation of such motions requires realization and characterization of periodic oscillations. Eigenmodes of linear dynamics have been widely used for rendering such motions based on the classic generalized eigenvalue problem (see works by Blickhan, 1989; Geyer et al., 2006; Kashiri et al., 2017c).To approach the performance and efficiency of the biological archetype, it is crucial to employ physical elasticity in drive units. To exploit the full dynamics of segmented legs, Lakatos et al. (2017) described eigenmodes of non-linear dynamics. Such a complete modal model can enhance the system performance as the robot design targets match the segmented leg dynamics with a desired dynamics set based on template models, e.g., Spring-Loaded Inverted Pendulum (SLIP), and desired motion considerations. While the robot structure design relies primarily upon geometry specifications, statics requirements, and actuation principles; design of kinematics, elasticity and inertial parameters respecting the above-said matching dynamics initiates a mechanical system with an embodied modal task, as discussed by Duindam and Stramigioli (2005). Such a design can significantly facilitate the robot control; similar to humans who excite the resonance by means of timed and directed motions.

Geyer et al. (2006) discussed the significance of segmented leg dynamics, which are often ignored, thereby creating untapped potential for improved mechanical energetics and control. Exploring point mass models whose legs are reduced to force laws shows that increasing the number of leg segments helps to reduce the mechanical advantage of leg force with less burden on joint actuators, i.e., lower joint torques; however, it also increases the design complexity. In addition, multiple segments create internal degrees of freedom, which introduce joint buckling in elastic stance leg behavior. Seyfarth et al. (2001) showed different strategies to mitigate this destabilizing effect from segment length changes (non-equal lengths) to non-linear joint elasticities to biarticular actuation. Such remedies enlarge the design complexity. In addition, studies on human locomotion revealed the swing leg dynamics are double-pendulum like and appear nearly passive, as reported by Mochon and McMahon (1980). However, the motion of a double-pendulum is chaotic if not properly restrained. Potential field calculations of the foot point (using energy neutral coupling springs) show that the double pendulum can be suitably restrained by passive bi-articular coupling, resulting in natural and comparably robust swing leg behavior. Overall, these two examples show leg segmentation introduces significant challenges to locomotion dynamics, which if they are ignored, lead to increased control effort and actuator energy expenditure.

The knowledge gained from studying the effects of leg segmentation can benefit the control of robotic limbs. Desai and Geyer (2012, 2013) showed the influence of incorporating active bi-articular coupling into nonlinear swing leg control to generate natural swing leg motions without pre-recorded reference trajectories. Moreover, the resulting control approach can position the leg into a wide range of target postures with robustness to large swing disturbances. On the basis of this concept, Thatte and Geyer (2016) formulated a control policy for powered knee-ankle prostheses. Simulation results suggest this policy generates human-like leg behavior in steady walking, and responds to disturbances to the swing leg with experimentally observed elevation and lowering strategies. Furthermore, a comparison with the performance of an impedance controller shows the proposed policy enables a computer model of an amputee to walk over rough terrain and recover from larger disturbances.

## 4. ENERGY EFFICIENT CONTROL

Efficient motion of robotic systems can be based on various principles. A majority of studies in this area employ optimal control techniques and bio-inspired approaches. Another paradigm for the control of such systems is energy-aware robotics that targets energy flows, especially in interactions. This provides a basis to constructively tackle issues of stability during interaction and methods to analyse energy storage and consumption in robotics systems. Stramigioli (2015) and Folkertsma and Stramigioli (2017) elaborated the basis of this concept to present a universal framework that models drive and interacting robotic systems, as the basis for energy-limited control, so that the actuation controllers can expend an energy budget to execute a given task, without injecting more energy. Below, we discuss a set of state-of-the-art optimal/bio-inspired approaches.

## 4.1. Natural Motion

Development of systems capable of executing efficient cyclic motions requires the exploitation of the robot's natural dynamics (McGeer, 1990; Collins et al., 2005; Ferris et al., 2007). Mechanisms for efficient natural dynamics have often been inspired by studies of the agile and efficient dynamics of human and animal locomotion (McMahon, 1985; Ferris et al., 1998, 2007; Full and Koditschek, 1999; Daley and Biewener, 2006; Geyer et al., 2006). A bio-inspired Central Pattern Generator (CPG) approach is one commonly-used method for rendering cyclic motions, as explored by Ijspeert (2008). However, the method relies upon an isolated unit to generate a periodic motion pattern. The control structure is thus open-loop as the controlled system feedback is not considered in the control. Furthermore, the oscillatory dynamics of the elastic body (robotic or biological) system are not exploited. To identify/realize the resonance excitation mechanism of humans, Lakatos et al. (2014) carried out a set of psycho-physical experiments including a human in the control loop; when the human is asked to excite a simulated elastic limb to a limit cycle in minimum time or with minimum effort, and the force feedback device displays to the human the forces from simulation. To estimate human adaptability, the limb parameters are arbitrarily varied in simulation. The results revealed a bang-bang control law switching the position reference around an equilibrium point when the force feedback changes direction. Lakatos and Albu-Schäffer (2014b) exploited this switching control approach for generating limit cycle motions. In order to excite a multidimensional, non-linear elastic multi-body system, Lakatos and Albu-Schäffer (2014a) proposed to apply the bang-bang controller in the torque transformed direction, and compute the corresponding reference joint positions from the modal coordinate. Lakatos et al. (2014) showed the functionality of this approach in generating cyclic motions on a DLR variable stiffness arm, which can elaborate the modal coordinate transformation from a neuroscience perspective as "dynamics synergies", see the work of Stratmann et al. (2016).

## 4.2. Minimal Energetics

Legged locomotion in nature can be observed to happen in a variety of different gaits that can be characterized by their different footfall patterns and contact forces, as discussed, for example, by Hildebrand (1989). Biomechanical experiments have established a clear relationship between running speed, choice of gait, and energy consumption in humans by Minetti and Alexander (1997) and horses by Hoyt and Taylor (1981). These data suggest that animals may change gait as a function of locomotion speed based on metabolic CoT. To capture the effect of gait in robotic systems, Xi and Remy (2014); Xi et al. (2016) employed optimal control for motion generation of conceptual models of bipeds and quadrupeds. The approach generated motions that minimized positive mechanical work (normalized by distance traveled) while being subject to realistic robot dynamics and locomotion constraints such as foot non-penetration or actuator limits. By varying forward speed and contact sequence, the results show that changing gait as a function of locomotion speed can substantially increase mechanical economy. The optimal behavior in bipedal locomotion is to walk at slow speeds and run at high speeds and in quadrupedal robotic locomotion to walk at slow speeds, trot at intermediate speeds and gallop at high speeds. It is notable that there was only a small mechanical energetic difference between trotting and toelting, which may explain why the toelt is part of locomotion repertoire of some horses. In contrast to biological quadrupeds, galloping did not significantly outperform trotting in simulations. This might be attributed to the lack of an articulated spine in the original quadrupedal model Yesilevskiy et al. (2018b).

Smit-Anseeuw et al. (2017) extended this approach to discuss optimal motions for the bipedal robot RAMone, and investigated the results of comparing two different footfall sequences (a walking sequence with a double support phase and a running sequence with aerial phase) and two different orientations of the knee joints (pointing forwards and backwards). It showed the optimal gait switches from ballistic walking with an instantaneous double-support to spring-mass running with an extended aerial phase at a speed of around 1 m/s. That is, at slow speeds nearly no elastic energy is stored in the actuator springs, while at high speeds almost all of the mechanical energy fluctuations within the robot are conducted through the springs. Switching from ballistic walking to spring-mass running reduced metabolic energy consumption by up to 88%. This is comparable with studies on the metabolic cost of human walking.

Donelan et al. (2001) showed, when humans walk, they prefer a particular step width, and execute this preference with remarkably small variability. In arriving at this preference, the nervous system may seek to minimize an objective function composed of a weighted sum of objectives. One such objective may be metabolic cost. Toward understanding how the nervous systems of able-bodied people weight this objective in walking, Selinger et al. (2015) measured people preferred gait in different cost landscapes, defined as the relationship between metabolic cost and a given gait parameter, and demonstrated that people can continuously optimize step frequency to minimize metabolic energy. Abram et al. (2017) evaluated results on several ablebodied subjects and found that preferred step width in a new landscape was determined by continuous energy optimization. Using step frequency as that gait parameter, Selinger et al. (2016) found the key features that describe this energetic cost optimization process, which can also be partially reproduced using a simple reinforcement learning algorithm, as shown by Simha et al. (2017).

Human motor control relies on central loop control, synergies, learning and peripheral loop (reflexes) as core principles, while robot motion control for transport (locomotion) is typically based on a decentralized position/torque/impedance controller and motion generation via simple models such as cart-table and SLIP. Such classical methods cannot exploit the full-body robot dynamics to obtain efficient motions. A possible solution to address this problem can be to determine the robot trajectories based on whole-body dynamics using a trajectory optimization approach targeting minimum CoT. Gasparri et al. (2018) formulated an optimal control problem in a way that robot dynamic parameters such as joint impedance may also be optimized, in addition to typical state and control variables. Once the locomotion constraints defining periodic change of contact phases (single and double supports) are set, in addition to the robot dynamics and conventional constraints e.g., joint limits, the optimal control problem is solved using a direct method. It is, however, a computationally highly demanding problem that cannot be solved in real-time. To address this issue, a library of optimized trajectories is generated off-line, and then it is searched in real-time for the trajectory associated with the current robot states/conditions. Nevertheless, it is demanding to manage the trajectory library size for robotic hardware with a large number of DOFs. A feasible remedy to this problem is to decipher the trajectory library.

The method is applied to a six-Degree-Of-Freedom (-DOF) planar biped powered by compliant actuators. Based on Principal Component Analysis (PCA), 99% of the variance of 500 optimized trajectories can be explained by three principal components that can also be expressed/fitted by second order polynomials. The results of this implementation on various walking speeds render a CoT around 0.5, which is on average about five times more efficient than the CoT resulting from a Zero-Moment-Point- (ZMP-) based approach rendering the trajectory based on the cart-table model. Results show that the swing leg joint torques of the two techniques are comparable. However, the stance leg joint torques using the optimized locomotion are negligible while the ZMP-based locomotion render substantial joint torques due to the non-straight leg configuration. To realize the effect of compliance on CoT in walking/running, the optimization based approach is employed for two cases, when the robot is rigid and soft. The results show that the slow walking CoT in the two cases are similar, although the soft system exhibited considerably lower CoT for fast walking. In addition, the rigid system renders a CoT approximately twice as great as the soft system when running. Moreover, using the soft system reduces the walk-to-run transition speed, and increases the maximum feasible running speed.

## 5. LOCOMOTION PRINCIPLES

When comparing different animals in nature, as well as the above robotic optimization studies, it is remarkable that despite substantial differences in structure, legged systems of all kinds rely only on a small set of different gaits. One potential explanation could be that these gaits are a manifestation of the underlying mechanical natural dynamics of the legged system. Gan et al. (2016) explored this idea by reducing the models to be completely lossless. Even with such conservative models, all common bipedal and quadrupedal gaits can be represented as passive periodic orbits, suggesting that gaits are merely different dynamic modes of the same structural system. Gaits manifest themselves as different non-linear elastic oscillations that form distinct (yet connected) limit cycles that passively propel an animal or robot forward. It therefore implies that different phenomena observed in such systems should rely upon common models and principles. Below, we discuss a set of recent propositions on these concepts.

## 5.1. Metabolic Cost Model

Faraji et al. (2018) proposed a simple cost model to predict metabolic cost trends under general walking conditions. A 3D linear walking model (called 3LP) developed by Faraji and Ijspeert (2017) is used to predict swing and torso balance costs in the sagittal and frontal planes. The vertical collisional loss and recovery work missing in the 3LP model are included via a Center of Mass (CoM) velocity redirection cost. To account for walking with a non-zero amount of leg lift, a ground clearance cost is incorporated. A weight support cost is also added to account for the energy consumed by leg extensor muscles preventing the stance leg from collapsing under body weight. The resulting cost model is the sum of all four individual costs, scaled by a constant muscle efficiency to convert from positive mechanical work to metabolic input. To evaluate the model, a set of walking conditions from several studies are simulated, including variations in step frequency, step width, added mass, extra ground clearance, crouched walking and reduced gravity conditions. For example, while Donelan et al. (2001) found that the metabolic cost increases at a rate of 6.40 W/kg per meter squared of step width, a quadratic fit to the proposed model reveals a close rate of 5.21 W/kg per meter squared of step width. Overall, the proposed linear combination of four major costs can predict (within the data's 95% confidence interval) the metabolic cost of increasing step width and many other walking conditions. It also provides a detailed metabolic contribution of each component, which is valuable for improving or augmenting performance.

## 5.2. Bioinspired Insights

To realize principles of leg control for robust and economic locomotion over rough terrain, Daley (2017) focused on use of comparative biomechanics as a tool to derive insights into how mechanics and control are integrated to achieve agile, stable and economic locomotion. Birds serve as a useful bipedal animal model, because ground birds such as quail,fowl and ostriches use walking and running gaits that are similar in wholebody dynamics, limb trajectory and ground reaction forces to gaits of humans (Daley, 2018). To understand the principles of bipedal gait, it is essential to combine perspectives from biomechanics, sensorimotor control and engineering. Recent studies have focused on measuring movement biomechanics over simple terrain features, such as obstacle negotiation and single downward steps (Birn-Jeffery et al., 2014; Blum et al., 2014), gait transition dynamics of ostriches moving freely in an open field (Daley et al., 2016), and leg loading during kicking and locomotion in the snake-hunting secretary bird (Portugal et al., 2016). These studies provide insight into locomotor control strategies by comparing steady and transient movement tasks, and investigating potential trade-offs among factors such as speed, stability, robustness and economy.

Birn-Jeffery et al. (2014); Blum et al. (2014); Hubicki et al. (2015) compared bird running biomechanics to model predictions, using reduced-order models and trajectory optimization of bipedal locomotor dynamics, to directly test hypotheses about the priorities and mechanisms underlying bipedal locomotion control. Studies of obstacle and step negotiation revealed that running birds prioritize consistent leg loading (injury avoidance/safety), as the dominant control objective (Birn-Jeffery et al., 2014; Blum et al., 2014). These studies also revealed that the optimal leg trajectories to regulate leg loading (to maintain consistent forces) are similar to the optimal trajectories to minimize mechanical work. These findings highlight that control priorities for economy (minimal work) and safety (consistent forces) are closely aligned. Birds use a three-step recovery strategy over obstacles that reflects priority for economy and safety, but does not directly prioritize trajectory stabilization to maintain the nominal steady-state body center-of-mass dynamics. Running birds achieve stability through passive-dynamic mechanisms, so stability is not required as a direct target of actuation control, due to integration of passive-damping and (multi-articular) intrinsic compliance.These studies revealed that robustly stable and agile locomotion over uneven terrain can be achieved through a simple control strategy of prescribing a leg trajectory (and therefore foot landing conditions) to maintain desired leg loading as modeled by a simple point-mass spring-loaded inverted pendulum (SLIP) model. During stance, leg dynamics are asymmetric and consistent with work-minimizing control of an intrinsically damped leg model. These studies also found that the control strategy used by running birds was similar across many conditions, including unexpected (invisible) potholes, visible obstacles, single steps and multiple steps. Despite varying ability to anticipate the upcoming terrain, leg dynamics and control strategies remain consistent across contexts.

The principles from these studies have been implemented as control policies in the bipedal robot ATRIAS at Oregon State University (Hubicki et al., 2016b), resulting in robustly stable bipedal gaits that are dynamically similar to those of ostriches and humans. These studies have also provided insight into the similarities and differences between humans and birds as bipedal animals. Humans and birds share similar wholebody dynamics of walking and running gaits and similar ground reaction force patterns. However, humans and birds use different stride length and frequency characteristics that reflect their different leg morphology. Humans and birds also use different sensorimotor control strategies. Humans rely heavily on "cephalized" (brain-dominated) control, involving extensive learning and high reliance on predictive planning; however, these processes suffer from long control delays. In contrast, birds have specialized to more heavily rely on "spinalized" (spinal-cord dominated) control, primarily using spinal rhythm generation coupled to robustly stable intrinsic leg mechanics (Daley, 2018). Rehabilitation and control of prosthetic devices might benefit from bird-inspired control mechanisms to achieve robust stability using simple control algorithms and intrinsically stable leg mechanics.

## 5.3. Underlying Concept

Real-world applications require reasoning and decision-making higher-level control, for which complete perception data is necessary, in order to select suitable behavior and motion planning schemes. Patrick et al. (2018) discussed their research on planning for efficient reactive legged locomotion. Prior planning architectures for legged robots generally rely on either finding state trajectories with an on-line optimization process (see Feng et al., 2015; Kuindersma et al., 2016), or using a specific walking controller formula that allows for analytic approximations of the stance dynamics (see Arslan and Saranli, 2012; Englsberger et al., 2015). These reactive control methods are, however, poorly compatible with common robotic motion planning methods that rely on regulating the robot trajectory through its state space. To address this, Patrick et al. (2018) presented that an alternative way of planning legged locomotion is to plan through the action space of efficient reactive legged behaviors which is similar to the controllers shown by Hubicki et al. (2016a). The elements of this space consist of controllers for different periodic gaits and transient actions that can arbitrarily trade off efficiency and robustness. Motion planning using this action space makes the planner choose which behaviors the robot should execute at any given time. When disturbances occur, the behavior executing in the control layer takes immediate action to keep the robot from falling, and after some latency the planning layer can react by specifying a new plan that accounts for the new situation. As a result, the robot can be more robust to real-world disturbances, while also allowing the use of arbitrary energyoptimized gait controllers. However, to navigate through such a dynamic space, it is necessary to understand the underlying principles of locomotion.

Jonathan et al. (2017) adopted legged locomotion as a dynamical phenomenon, inspired by its analogy to a clock, to discuss the periodic attractor underlying most natural gaits.This expounds the dynamical phenomenon of legged locomotion including walking, running, skipping, hopping and jumping. However, it excludes decision-making and path-planning of the system, as well as balancing, gaits that maintain a center of pressure within a polygon of support, and slow one-foot-infront-of-the-other gaits. This concept can be described as a cycle of energy between internal potential, gravitational, and kinetic energy, given compliant interaction<sup>5</sup> renders bouncing due to discrete footsteps observed in bird experiments discussed above. The energy exchange cycle can therefore describe different

<sup>5</sup>Due to the compliance of the locomotor and the environment/world, foot contact is a process, rather than an event, in which the tangential foot forces cannot be applied at the contact instance. The robot interactions with terrain should then be compliant since compliant oscillations are insensitive to external disturbances and can eliminate inelastic collisions. Interactions can accordingly be robust with respect to ground height and/or impedance changes resulting from the lack or imperfection of world models. Additionally, prevention from rigid collisions avoids large impact forces and unnecessary energy consumption.

locomotion modes, e.g., a specific shift in the cycle defines walking and running, and one can find damped oscillation a key to stabilization. It can then suffice for dynamics of legged locomotion and just relies on on-board proprioception similar to blind walking/running for humans, which requires only an inertial measurement unit (IMU).

Inspired by the evaluation of bird running over uneven terrain (Birn-Jeffery et al., 2014; Blum et al., 2014; Hubicki et al., 2015), the fundamental locomotor functionality of a legged system can be independent of environmental conditions such as light, and recovery from small terrain variation need not require exteroceptive sensory information and/or large processing. It is then important to integrate compliance carefully into the system. In addition to passive compliance that was discussed earlier in section 2, the employment of active compliance (impedance control) enables the replication of different (linear or non-linear) compliant behavior, in addition to the adjustment of damping for variable energy dissipation, inspired by damping/energy dissipation regulation of birds for robust stability in uneven terrain. Moreover, integration of compliance improves the Markov Decision Process (MDP) of expected response, especially when set-point trajectory is given through feed-forward approaches.

## 6. SUMMARY

This paper reviewed recent progress in design and control of robotic systems from different perspectives, to indicate the current state of robotics in terms of energy efficiency, and to highlight solutions advancing this criterion. It includes a review of various robotic actuators exploiting compliance in series and in parallel for energy recycling, and discusses the importance of limb segmentation in design, dynamics analysis and control of dynamical systems. A set of energeticallyestablished control approaches are explored, and compared with human behaviors/controls. In addition, a set of cutting-edge topics initiating new directions in locomotion are reported. Overall, we can briefly conclude that:

## REFERENCES


## AUTHOR CONTRIBUTIONS

All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.

## FUNDING

The research leading to these results has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 644839 (CENTAURO) and 644727 (CogIMon).


disturbance rejection. PLoS ONE 9:e100399. doi: 10.1371/journal.pone.01 00399


efficiency in coupled compliant joints via PCA. Front. Neurorob. 10:2. doi: 10.3389/fnbot.2016.00002


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Kashiri, Abate, Abram, Albu-Schaffer, Clary, Daley, Faraji, Furnemont, Garabini, Geyer, Grabowski, Hurst, Malzahn, Mathijssen, Remy, Roozing, Shahbazi, Simha, Song, Smit-Anseeuw, Stramigioli, Vanderborght, Yesilevskiy and Tsagarakis. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Linking Gait Dynamics to Mechanical Cost of Legged Locomotion

David V. Lee<sup>1</sup> \* and Sarah L. Harris <sup>2</sup>

<sup>1</sup> School of Life Sciences, University of Nevada Las Vegas, Las Vegas, NV, United States, <sup>2</sup> Department of Electrical and Computer Engineering, University of Nevada Las Vegas, Las Vegas, NV, United States

For millenia, legged locomotion has been of central importance to humans for hunting, agriculture, transportation, sport, and warfare. Today, the same principal considerations of locomotor performance and economy apply to legged systems designed to serve, assist, or be worn by humans in urban and natural environments. Energy comes at a premium not only for animals, wherein suitably fast and economical gaits are selected through organic evolution, but also for legged robots that must carry sufficient energy in their batteries. Although a robot's energy is spent at many levels, from control systems to actuators, we suggest that the mechanical cost of transport is an integral energy expenditure for any legged system—and measuring this cost permits the most direct comparison between gaits of legged animals and robots. Although legged robots have matched or even improved upon total cost of transport of animals, this is typically achieved by choosing extremely slow speeds or by using regenerative mechanisms. Legged robots have not yet reached the low mechanical cost of transport achieved at speeds used by bipedal and quadrupedal animals. Here we consider approaches used to analyze gaits and discuss a framework, termed mechanical cost analysis, that can be used to evaluate the economy of legged systems. This method uses a point mass perspective to evaluate the entire stride as well as to identify individual events that accrue mechanical cost. The analysis of gait began at the turn of the last century with spatiotemporal analysis facilitated by the advent of cine film. These advances gave rise to the "gait diagram," which plots duty factors and phase separations between footfalls. This approach was supplanted in the following decades by methods using force platforms to determine forces and motions of the center of mass (CoM)—and analytical models that characterize gait according to fluctuations in potential and kinetic energy. Mechanical cost analysis draws from these approaches and provides a unified framework that interprets the spatiotemporal sequencing of leg contacts within the context of CoM dynamics to determine mechanical cost in every instance of the stride. Diverse gaits can be evaluated and compared in biological and engineered systems using mechanical cost analysis.

Keywords: biomechanics, energetics, economy, walking, running, bipedal, quadrupedal, comparative

## INTRODUCTION

As with any structural or functional animal feature, it is important to consider locomotion through the lens of organic evolution. In nature, the process of natural selection replaces the forward design used in engineering. As emphasized by Vogel (2013), instead of being optimized through a design process, biological designs emerge to be just good

#### Edited by:

Monica A. Daley, Royal Veterinary College, United Kingdom

#### Reviewed by:

Daniel Schmitt, Duke University, United States Peter Eckert, École Polytechnique Fédérale de Lausanne, Switzerland

> \*Correspondence: David V. Lee david.lee@unlv.edu

#### Specialty section:

This article was submitted to Bionics and Biomimetics, a section of the journal Frontiers in Robotics and AI

Received: 25 May 2018 Accepted: 05 September 2018 Published: 17 October 2018

#### Citation:

Lee DV and Harris SL (2018) Linking Gait Dynamics to Mechanical Cost of Legged Locomotion. Front. Robot. AI 5:111. doi: 10.3389/frobt.2018.00111

**20**

enough so that the organism can survive, compete, and reproduce in its ecological niche. In addition, natural selection is constrained by phylogenetic, constructional, and developmental factors. Leg number (Shubin et al., 1997), the composition of the skeleton (Currey, 2002) or cuticle (Hopkins and Kramer, 1992), as well as muscles and their innervations (Diogo and Abdala, 2010) are generally conserved within taxa. For example, when considering leg number, true crabs (Decapoda) have ten legs, insects (Hexapoda) have six, and terrestrial vertebrates (Tetrapoda) have four (except when lost in certain lineages). Crabs and insects (Arthropoda) also have an exoskeleton made of a chitin composite, whereas vertebrates have an endoskeleton made of bone. Such fundamental features are conserved within a given phylogenetic group and therefore place limits on the mechanical solutions that might emerge. Phylogenetic, constructional, and developmental features of an animal's biology, as well as body size, lifespan, and other characteristics, impose severe constraints on the structural and functional "design solutions" available. As extreme examples, biological constraints make it so that we cannot replace our bones with a jointed exoskeleton or develop longer sarcomeres for greater force production. Hence, the locomotion and neuromusculoskeletal function observed in any species should not be seen as "optimal" but simply as a competent solution emerging from the highly constrained process of natural selection. Here we review key features of nature's strategies for terrestrial locomotion, focusing on gait and center of mass dynamics with a view toward informing the design of legged robots.

## GAIT

Locomotion is fundamental for foraging, prey capture, predator evasion, securing territory, finding mates, and migration. Animals use various modes of locomotion called gaits as defined by their movement patterns. Gait is conventionally defined by temporal footfall patterns and, for terrestrial animals, the footfall sequence is the primary identifier of gait. Footfall sequence is quantified by the phase relationships of individual legs, expressing the time of foot contact as a fraction or percentage of stride period. For example, for walking, one foot lands at the beginning of the stride (0%) and the second foot lands at mid-stride (50%).

Yet a phase-based definition of gait is often incomplete and can fail to distinguish different gaits because they may have similar or even identical phase relationships. For example, bipedal walking and running show the same left-right-left sequencing of footfalls, with phases of initial contact of alternating feet at 0% and 50% of the stride period (**Figure 1A**). Hence, the distinction between walking and running traditionally relies upon the duty factor, which represents the duration of a given footfall as a fraction of stride period. On this basis, human running is distinguished from walking by its duty factor of <0.5, which specifies an aerial period in the case of bipedal running but not necessarily in quadrupedal or multilegged running, where footfalls with duty factors <0.5 may be sequenced such that there is no aerial phase. In contrast to humans, birds are bipedal striders that blur the distinction between walking and running. Using duty factors >0.5, they exhibit exhibit "grounded" running without an aerial phase. But how do we know that grounded running is indeed running as opposed to walking? Despite the convenience of separating running from walking on the basis of duty factor, it is clear that criteria beyond temporal footfall metrics are needed to distinguish the underlying physics of gait. This section focuses on the broad utility of temporal patterns in the definition of gait, and section Center of Mass Dynamics addresses similarities and differences in the underlying dynamics of various gaits.

## Symmetrical vs. Asymmetrical Gaits

Legged gaits are broadly characterized as symmetrical or asymmetrical according to the phase relationships of left-right pairs of legs. If the left and right legs of a pair are one-half stride cycle out of phase with one another, the gait is defined as symmetrical—if not, the gait is asymmetrical. This convention spans animals of different leg number. For example, bipedal running of humans, quadrupedal trotting of dogs, pacing of camels, and hexapedal trotting of cockroaches are all symmetrical gaits because left-right pairs of fore-, mid-, and hind-legs are onehalf cycle out of phase with each other. Regardless of leg number, gait symmetry is defined by this half-cycle phase relationship of the left and right legs at a given cranio-caudal position.

The number of legs limits the number of leg sequencing options, such that gait possibilities for bipeds are restricted to symmetrical striding gaits (walking and running) and asymmetrical hopping and skipping. Quadrupeds use five symmetrical gaits (lateral and diagonal sequence walking, trotting, pacing, and ambling) and six asymmetrical gaits (lope, transverse and rotary gallops, half-bound, bound, and pronk). These are broad definitions, and it is important to note that phase separations between foot contacts show substantial variation within gaits, as illustrated in (Hildebrand, 1965, 1968) plots for the gaits of horses and dogs.

## Bipedal Gaits

Bipedal striding gaits, including our own walking and running, are symmetrical by definition (**Figure 1A**). These gaits are used by birds and historically by theropod dinosaurs, which comprise the greatest diversity of bipedal striders. Humans and birds (except small songbirds) walk at slow speeds and run at fast speeds (Small songbirds typically use hopping rather than striding gaits). Some great apes and monkeys are facultative bipeds, occasionally walking or running for short distances. Lizards and cockroaches may run bipedally at their fastest speeds by pitching their body into a more upright attitude and straightening their legs as they transition from a quadrupedal (Irschick and Jayne, 1999) or hexapedal trot (Full and Tu, 1991). This seemingly odd behavior increases speed by increasing stride length. In general, bipedal striders achieve greater absolute stride lengths than quadrupeds of the same body mass (Reynolds, 1987), and this has been argued to be an advantage for endurance runners of our own genus—for example, in persistence hunting

of quadrupeds or aggressive scavenging in competition with quadrupeds (Carrier, 1984; Bramble and Lieberman, 2004).

Although the presence of only two legs allows relatively few sequencing options, two asymmetrical gaits—hopping and skipping—are used by bipeds (**Figure 1C**). Bipedal hopping gaits are common in mammals and birds, particularly at small body size. It may seem counterintuitive that the paired contacts of right and left hind limbs should indicate an asymmetrical gait, however, this deviates from the symmetry criterion that right and left contacts be one-half stride cycle out of phase. Bipedal hopping is found convergently in several groups of rodents, including springhares, kangaroo rats, and jerboas and the only large bipedal hoppers are the wallabies and kangaroos, with male red kangaroos equaling human body mass (reviewed by McGowan and Collins, 2018). Bipedal skipping, a gait occasionally used by children and sometimes in reducedgravity conditions, is the only asymmetrical gait used by humans (Minetti, 1998, 2001a,b; Ackermann and van den Bogert, 2012). Monkeys and apes with forelimbs specialized for brachiation in arboreal environments sometimes use side-ways bipedal skipping gaits on the ground, and this is typified by the fast, dance-like gait of the gibbon/siamang (Vereecke et al., 2006). Bipedal skipping gaits are thought to be akin to the asymmetrical galloping of quadrupeds and this is supported by similarities in relative phase of the hindlegs (**Figures 1C,D**), as well as by the analysis of CoM dynamics (Minetti, 1998).

## Quadrupedal Gaits

The symmetrical gaits of quadrupeds include the walk, trot, pace, and amble. Quadrupedal mammals, crocodiles, lizards, and salamanders normally walk at slow speeds and trot at intermediate speeds, while the pace and amble are used by just a few mammals. During walking, leg contact phases are typically separated by about 25% and the order of contact is typically a hind foot, followed by the ipsilateral forefoot, followed by the contralateral hind foot and forefoot (**Figure 1B**). This most commonly used sequence of footfalls is called a lateral sequence walk. Yet most primates walk with a diagonal sequence, which is thought to provide better roll stability when walking on branches (Hildebrand, 1980; Cartmill et al., 2002). When walking on branches, however, this advantage of diagonal support may be diminished by a relatively narrow stance compared to that of terrestrial locomotion. A full mechanistic understanding of this difference between primates and typical quadrupeds will likely require consideration of individual foot force and torque in addition to spatiotemporal considerations (e.g., Shapiro and Raichlen, 2005, 2007; Cartmill et al., 2007). Notwithstanding these distinctions, simple models have shown that the lateral sequence walking gait used by most quadrupeds provides a stable tripod of support, termed "static stability" (McGhee, 1976; Alexander, 1984), and a mathematical model of slow walking found this to be the most stable of the three possible gaits that can provide a continuous tripod of support (McGhee and Frank,

1968), while the other two solutions appear not be used by animals.

Quadrupeds use the remaining symmetrical gaits—the trot, pace, and amble—at intermediate speeds. Trotting is the symmetrical gait most widely used at intermediate speeds and its relative phases resemble those of bipedal striding (**Figure 1B**). During trotting, diagonal legs are in phase, providing a supporting leg on each side to resist rolling that may be induced by a sprawled posture. In contrast, ipsilateral legs are in phase during pacing, discarding roll stabilization but avoiding forehind interference in mammals with especially long legs, such as camels, as well as some dogs and horses (Hildebrand, 1968). The transition between walking and trotting occurs at a similar relative speed for all quadrupeds. However, the top trotting speed (i.e., trot-gallop transition speed) occurs at relatively faster speeds for larger quadrupeds. The gait referred to as an amble, tölt, or "running walk" is another symmetrical gait used by quadrupedal mammals including lemurs, monkeys, elephants, and some horses (reviewed by Schmitt et al., 2006). It is a fourbeat gait with phase relationships like those of walking but with speeds comparable to fast trotting. The amble reduces vertical oscillations of the center of mass and this relatively flat trajectory is thought to be advantageous for locomotion on branches (e.g., Cartmill et al., 2002) and as a fast gait that allows very large quadrupeds to avoid aerial periods (e.g., Hutchinson et al., 2003).

At intermediate to high speeds, quadrupeds use asymmetrical gaits, such as the gallop, half-bound, bound, and pronk. Sprawling quadrupeds like salamanders and lizards do not use asymmetrical gaits, although juvenile crocodiles are able to gallop with an upright posture (Zug, 1974; Renous et al., 2002). Asymmetrical quadrupedal gaits are linked with upright limb posture, where legs are retracted in the parasagittal plane. Because left-right pairs of fore- and hindlimbs may be kept more nearly in phase, asymmetrical gaits allow bending of the spine and pelvis to contribute to stride length, with the muscles of the trunk used to bilaterally dorsiflex and ventroflex the spine. The leftright phase separation during galloping is typically 10–35% in the forelimb pair and 10–25% in the hindlimb pair (Afelt et al., 1983; Alexander, 1984). During bounding, both left-right pairs are in phase, and, during half-bounding, the hind left-right pair are in phase while the fore left-right pair are somewhat out of phase. Deer, antelope, goats, and sheep use the bound and pronk for display or warning and in steep terrain. The pronk is a pronounced bouncing gait where all four feet strike the ground in unison. The half-bound is well-known in rabbits and is the fast gait most frequently used by rodents and other small mammals. Rodents tend to use the half-bound at speeds corresponding to the upper trotting range of larger quadrupeds.

The gallop is the most commonly used asymmetrical gait of quadrupeds larger than about five kilograms. Broadly defined, it includes loping or cantering—a slow, three-beat gait with one diagonal pair in-phase, being preceded by a single hindlimb and followed by a single forelimb. Wolves use the lope alternately with trotting when traveling long distances. Large mammals, that have longer forelimbs and a forward center of mass, such as hyenas and bison, have a reduced trotting range and switch to a lope at intermediate speeds where other quadrupeds would trot. The transverse gallop is a faster four-beat gait, introducing a phase separation in the lope's diagonal pair with the pair's forefoot following the hind foot. The rotary gallop is the fastest gait used by quadrupedal mammals and it changes the order of the forefoot contacts such that the contact sequence is circular rather than crossing in a figure-eight. Galloping typically has a single gathered suspension period but cheetahs and grayhounds add an extended suspension period between hind- and forelimb contacts that serves to increase stride length at the fastest speeds.

## Speed Effects

Animals choose a gait depending on their speed—they choose to walk at slow speeds and typically transition to symmetrical running or trotting at intermediate speeds and then asymmetrical half-bounding or galloping at fast speeds. The choice of symmetrical or asymmetrical gaits at faster speeds is related to leg posture (sprawling vs. upright), body mass, body mass distribution, rotational inertia of the body and appendages, leg length relative to body length, and fore-hind leg length differences. Gait choice and relative speeds at gait transitions may also change during uphill or downhill locomotion, or with varying substrate properties, or with uneven conditions in rugged terrain.

The analysis of phase relationships is generally effective for identifying and classifying gait, yet bipedal walking cannot be distinguished from running and quadrupedal walking from ambling based on foot sequencing alone. Thus, as introduced above, duty factor is typically used to distinguish these slower walking gaits from running and ambling. Greater duty factors are expected at slower speeds, and this provides a convenient method to discern gait differences using only temporal parameters. Despite its utility in gait analysis, duty factor cannot necessarily distinguish differences in gait dynamics. For example, walking and grounded running in birds have equal phase relationships and both have duty factors >0.5. Because duty factor decreases with speed, it can be associated with different gaits, as in the separation of walking from running of humans. Nonetheless, duty factor and footfall sequencing do not directly determine the dynamics underlying a particular gait and often lead to conflicting definitions of gait.

## CENTER OF MASS DYNAMICS

Identifying the temporal pattern of leg sequencing, as measured by relative phase and duty factor, is a critical first step in quantifying gait, yet it is often imprecise and is insufficient to reveal the physical basis of locomotion. The dynamics of a given gait can only be known by measuring the forces determining the motion of the center of mass (CoM). This section first describes the measurement of CoM dynamics to determine oscillations resulting from combined leg force. It then describes two conventional models of gait: the rigid inverted pendulum (RIP) model traditionally applied to walking; and the spring-loaded inverted pendulum (SLIP) model for running. We conclude the section with a discussion of mechanical cost analysis that evaluates CoM dynamics more objectively, without the a priori assumptions required by the RIP and SLIP models.

## CoM Measurements and Gait Patterns

Two approaches commonly used in the analysis of gait are a point mass model and a rigid-body model. The most basic measurement of gait dynamics treats the center of mass as a point mass, thus neglecting rotations about the center of mass (CoM). A point mass model of locomotion requires that forces exerted on all of the legs be summed and the instantaneous accelerations of the center of mass be determined throughout a complete stride (Cavagna, 1975). In contrast, a rigid-body model, requires separate measurements of forces exerted on each of the legs, as well as any torques, in order to consider not only the translation of, but also rotations about the center of mass. When more than one leg contacts the ground, as in the example of the diagonal limb pair during trotting, differential forces can produce force couples or "free moments," which act about the center of mass to resist rotation and contribute to balance (Gray, 1968; Murphy and Raibert, 1985; Lee et al., 1999). Torques exerted at the feet also influence rotation in a rigid body model but these are typically restricted to the yaw axis, unless the foot can adhere to or grasp the substrate. Here we focus on evaluating the dynamics of a point mass model of the center of mass, emphasizing the principal gaits of bipeds and quadrupeds.

During legged locomotion, forward progression is coupled with vertical oscillations of the CoM. This forward progression is achieved either by using one leg at a time, as during bipedal running with aerial phases, or by using more than one leg at a time, as during all other gaits of bipeds, quadrupeds, and multilegged animals. The net effect of the vertical and shear ground reaction forces summed across all legs, less the vertical force of body weight, is a cyclic redirection of the CoM. Hence, the summed vertical force always oscillates about body weight with equal variation above and below—otherwise, the CoM would have a net rise or fall during a stride. The summed foreaft forces exerted on the legs oscillate about zero with equal propulsive and braking impulse during steady-speed locomotion. At the beginning of stance, limbs are placed forward (protracted stance)—exerting force against the direction of travel, and at the end of stance, limbs are placed backward (retracted position) exerting force in the direction of travel. Thus, legs typically exert braking followed by propulsive force (Gray, 1968). When more than one leg is in contact with the ground, the summed vertical forces and the summed shear forces determine oscillations of the center or mass. The pattern of combined forces acting on the center of mass is influenced by the sequencing and duration of leg contacts—explaining in many cases the correlation of relative phase and duty factor with the dynamics of the center of mass.

For a given gait, the CoM will either oscillate once or twice per stride. Symmetrical gaits, such as bipedal walking and running, have two vertical oscillations per stride. These oscillations are achieved alternately by left and right legs of a pair, and each of these legs contributes to just one oscillation as long as each leg's duty factor is <0.5. During asymmetrical gaits, such as bipedal hopping and quadrupedal galloping, there is only one vertical oscillation of the CoM per stride. The legs of right-left pairs act either in unison or in a staggered sequence to achieve this single cycle of oscillation and an aerial phase separates the leg contacts of one stride from those of the next (Hildebrand, 1980). Exceptions to the rule of a single aerial phase include the fast gallop of grayhounds and cheetahs, as well as the fast halfbound of hares (Hildebrand, 1980; Bertram and Gutmann, 2008), which add a second aerial phase—an "extended suspension" between hind- and foreleg contacts—while preserving the typical "gathered suspension" between fore- and hind-leg contacts. Barring these exceptions, asymmetrical gaits include one vertical oscillation per stride and symmetrical gaits include two vertical oscillations per stride.

As already emphasized, total vertical force rises cyclically above and below body weight during the stride to achieve vertical oscillations of the CoM. This is true of both running and walking, although during bipedal walking the rise above body weight is achieved by simultaneous leg contacts during double support of the step-to-step transition (**Figure 2**), whereas a single leg contact achieves the rise above body weight during bipedal running. As a result of these differences between running and walking, vertical acceleration is downward during mid-stance of walking (**Figure 2**) and upward during mid-stance of running. Because acceleration is one-half cycle out of phase with its second derivative, position, the CoM reaches its lowest point during mid-stance of running and its highest point during midstance of walking. This traditional observation of the vertical

FIGURE 2 | Ground reaction forces of the left limb (gray) and right limb (black) during three steps of human walking (i.e., left-right-left)—showing vertical (solid lines) and fore-aft (dashed lines) components of force. This stride is defined from mid-stance of the left limb to the subsequent mid-stance of the left limb. Vertical ground reaction force (solid lines) is below body weight during mid-stance of single stance (SS) and peaks at values greater than body weight near the beginning and end of double stance (DS). Fore-aft ground reaction force (dashed lines) peaks during double stance (DS) for both positive (propulsive) force from the trailing limb and negative (braking) force from the leading limb.

position of the CoM at mid-stance is the basis for a long-standing dichotomy between walking and running (Cavagna et al., 1976, 1977).

## SLIP, RIP, and BSLIP Models of Locomotion

Running and walking have been characterized separately based on the phase relationships of their kinetic and potential energy oscillations which are typified by the spring-loaded inverted pendulum (SLIP) model and the rigid inverted pendulum (RIP) model. The view of kinetic and potential energy oscillations being either in phase, implying a SLIP model, or out of phase, implying a RIP model, remains the most influential construct in the field of gait analysis, including that of bipedal, quadrupedal, and multi-legged locomotion (reviewed by Dickinson et al., 2000). As explained in the previous section, patterns of vertical force dictate that the CoM reaches its lowest point (minimum potential energy) in mid-stance of running and its highest point (maximum potential energy) in mid-stance of walking. Despite this difference in potential energy at mid-stance, fluctuations in kinetic energy are determined primarily by changes in forward velocity, which follow the same pattern for running and walking. This is because braking occurs in the first half of a given leg's stance and propulsion in the second half. Thus, CoM velocity—and therefore kinetic energy—tends to reach a minimum near mid-stance in both walking and running.

In the case of running or other SLIP-like gaits, kinetic and potential energy both reach a minimum at mid stance making these energies more-or-less in phase. SLIP-like gaits include bipedal running and hopping, as well as quadrupedal and multilegged trotting, which are well-described as "bouncing" gaits because the greatest leg compression occurs at about the same time as the greatest vertical force (Blickhan, 1989; McMahon and Cheng, 1990). If physical leg springs are present, these spring-like dynamics provide some opportunity for energy savings by elastic storage and return of some portion of the absorptive and generative work done by muscles or robotic actuators. Current research hybridizes SLIP concepts with rigid body models and link-segment models (reviewed by Sharbafi and Seyfarth, 2017). For example, mass distribution away from the hip influences dynamics of the spring-loaded leg in a bipedal running model (Blickhan et al., 2015). This simulated biped is controlled using the virtual pivot point construct to direct the resultant force vector through a point above the CoM (Maus et al., 2008; Sharbafi and Seyfarth, 2017).

In contrast to SLIP-like gaits, potential energy tends to reach a maximum near mid-stance during walking—making kinetic and potential energies more-or-less out of phase. According to the conventional interpretation of "two basic mechanisms," this is sufficient to invoke a RIP model of walking dynamics. Nonetheless, experimental studies show that bipedal and quadrupedal walking dynamics (e.g., Lee and Farley, 1998; Griffin et al., 2004, Genin et al., 2010) do not match the RIP model. This is unsurprising given that an actual rigid inverted pendulum (i.e., a mass on a massless rod of fixed length) would show a peak vertical force instead of a minimum vertical force in the mid-stance position, as described by Geyer et al. (2006). The same authors propose an alternative compliant-legged model of walking called the bipedal SLIP or BSLIP, which reproduces the characteristic m-shaped force with a minimum at mid-stance by providing a spring-loaded leg that introduces compliance. The BSLIP also includes summation of leading and trailing leg forces during double support of the step-to-step transition. Although this revealing model is widely used and highly cited, the BSLIP has yet to upend the RIP model of walking in most textbook explanations. This may be partly because the BSLIP is more challenging to simulate and perhaps also because its conservative leg springs limit its ability to match the full range of human walking speeds (Lipfert et al., 2012). Nonetheless, a recent bipedal robot demonstrates SLIP-like running and BSLIPlike walking, using the same spring-loaded legs (Hubicki et al., 2018). In principle, the BSLIP model is more correct than the unrealistic impulsive step-to-step transition of the RIP model because it captures the m-shaped force profile and allows for double support.

## Mechanical Cost Analysis

Instead of focusing on phase relationships between kinetic and potential energies, mechanical cost analysis quantifies the mechanical cost of redirecting an animal's center of mass and reveals the underlying physics of that cost. This approach considers each instance of the stride. MCA reduces the observed center of mass dynamics to dimensionless parameters—the key amongst these being collision angle, which is equivalent to mechanical cost of transport (Lee et al., 2011). Using this approach, gaits are not constrained to a priori models that often invoke idealized gaits as ill-fitting approximations. Instead, mechanical cost analysis applies across gaits and species by focusing on the fundamental physics of the animal's interaction with the substrate. MCA provides further insight into not only the overall stride dynamics but also the individual events in the stride, indicating which phases of the stride are more (or less) costly.

The central concept of mechanical cost analysis is d'Alembert's (1743) "principle of orthogonal constraint," which shows that a mass can be redirected without mechanical work, as long as the constraint (force vector) is perpendicular to the path (velocity vector), such that their dot-product (mechanical power) is zero (**Figures 3D,E**). Considering the simple case of a flat trajectory, where the velocity vector is purely horizontal; the velocity of the center of mass would be increased by a forward-directed (propulsive) force vector or, decreased by a backward-directed (braking) force vector—these two conditions require generative or absorptive work, respectively, both incurring mechanical cost (**Figures 3A,B**). In contrast, a force vector (to resist gravity) does not change the velocity of the center of mass, thus requiring no work and moving with zero mechanical cost of transport (**Figure 3C**). In the latter example, the force and velocity vectors are kept perpendicular, matching the mechanics of an idealized wheel rolling on a flat surface. Although legged locomotion necessarily includes vertical oscillations, the benefit of moving without mechanical cost could be had as long as force and velocity vectors could be kept perpendicular throughout the vertical oscillations of the stride.

d'Alambert's theoretical redirection with zero work is not completely realized by legged systems because the ability to exert orthogonal forces is limited by a leg's position with respect to the CoM, its kinematic range of motion, and its force-torque capacity, amongst other factors. Nonetheless, certain gaits of bipeds, quadrupeds, and brachiators approximate orthogonal constraint. Deviations from orthogonal constraint require work. Specifically, work is required in any instance where force and velocity vectors deviate from a perpendicular relationship and the resulting mechanical power is non-zero (Ruina et al., 2005; Lee et al., 2011). Foundational studies in this area used mathematical models of discrete leg contacts to estimate the mechanical cost of redirecting the CoM for an entire stride of a given gait while considering leg number, step-to-step transition, or foot shape (McGeer, 1990; Kuo, 2001; Ruina et al., 2005). Mechanical cost analysis extends these previous models and captures the dynamics of the CoM not only for the entire stride but also at each instance of the stride by measuring the CoM velocity and ground reaction force on the CoM and then determining the instantaneous angle between these vectors. This analysis applies to both simulated models and experimental measurements.

Experimental data using mechanical cost analysis show that mechanical cost of transport is greater during running than during walking. Walking reduces mechanical cost by maintaining a more perpendicular relationship between force and velocity vectors throughout the stride. For example, during the stepto-step transition of human walking, also called double stance, the CoM (and thus, velocity) transitions from downward to upward while ground reaction force transitions from propulsive (forward) to braking (backward). Specifically during double stance, the force is first propulsive (dominated by the trailing leg) and then braking (dominated by the leading leg). In this way, force and velocity vectors are kept more nearly perpendicular during downward to upward redirection of the center of mass.

In contrast, SLIP-like gaits, such as running, cannot approach orthogonal constraint because they instead follow a brakingpropulsive pattern that simply aligns with the orientation of the support leg(s). Because bipedal running typically has only single-leg support, it cannot employ the propulsive and then braking pattern of the walking double stance. Instead, the SLIP-like pattern of a single leg's stance starts with braking (backward-directed force vector) with a large projection on the downward-directed velocity vector, followed by propulsion (forward-directed force vector) with a large projection on the upward-directed velocity vector (**Figures 3F,G**). In the SLIP construct, both early and late stance geometries deviate from orthogonal constraint, resulting in greater work during the downward to upward redirection of the CoM.

Mechanical cost analysis quantifies mechanical cost by measuring the collision angle φ , which is the deviation from perpendicular of the CoM force and velocity vectors. This angle is measured in each instance of the stride and is given by Lee et al. (2011):

$$\phi = \arcsin\left(|\mathbf{F} \cdot \mathbf{V}| / |\mathbf{F} \parallel \mathbf{V}|\right) \tag{1}$$

where F is the force vector and V is the velocity vector, either in two or three dimensions. The overall collision angle across the entire stride 8 is a weighted-average of φ , where the weights are the magnitudes of force and velocity vectors in each instance:

$$\Phi \sum |\mathbf{F} \parallel \mathbf{V} \mid \phi \rangle \sum |\mathbf{F} \parallel \mathbf{V} \mid \tag{2}$$

Substituting the small angle approximation of Equation (1) into Equation (2) then shows that stride collision angle 8 is approximately equal to the mechanical cost of transport:

$$CoT\_{mech} = \sum |\mathbf{F} \cdot \mathbf{V}| / n\overline{V\_{\mathcal{V}}} mg \implies \sum |\mathbf{F} \cdot \mathbf{V}| / \sum |\mathbf{F} \cdot \mathbf{V}| \tag{3}$$

where m is body mass, g is gravitational acceleration, V<sup>y</sup> is the mean forward velocity, and n is the number of sampled points in a given stride. Mechanical cost of transport was originally derived using dimensional analysis and it gives the work to move a unit body weight a unit distance during steady-speed locomotion. However, only mechanical cost analysis reveals the physical basis for this cost. Specifically, the collision angle provides a physical basis for the mechanical cost of transport as well as a way to quantify the cost of individual events or phases within the stride. In contrast, CoTmech, in its original form, depends upon average forward velocity and only applies to a full stride of locomotion, without providing information about the costs of individual events or phases.

From the point mass perspective, SLIP-like bouncing gaits incur greater mechanical cost of transport—i.e., require more mechanical work to move the body weight a unit distance than walking gaits that reduce required work by targeting orthogonal constraint. In humans, for example, the mechanical cost of transport during SLIP-like running is three-times that of walking (**Figure 4**, compiled by Lee et al., 2013). Again, this lower mechanical cost of walking is achieved by summed forces on the trailing and leading legs during double support, providing propulsion while the velocity is directed downward, then braking while the velocity is directed upward. This is the opposite pattern of SLIP-like gaits, such as running, and it allows human walking dynamics to more nearly approach orthogonal constraint, thereby reducing required work and achieving oscillations of the CoM with reduced mechanical cost.

Work is also mitigated by brachiating siamangs (Michilsens et al., 2012) in the same way but to an even greater extent than in terrestrial walking. Instead of negotiating substrate interactions with one or more foot, as discussed so far, these apes use the overhead support of branches to achieve fast, smooth locomotion through the canopy. Brachiators are able to reduce required work using a single handhold because they can either pull or push on the handhold to keep force and velocity vectors more perpendicular, whereas terrestrial animals can only load their legs in compression. Hence, brachiating animals can achieve propulsion while their velocity is directed downward and braking while their velocity is directed upward—i.e., the same strategy of orthogonal constraint that is approached using two legs during the step-to-step transition of human walking or by four legs during quadrupedal galloping (Bertram and Gutmann, 2008; Lee et al., 2011). The astonishingly smooth movement of brachiating gibbons, in fact, helped initiate a resurgence of theoretical studies of collision mechanics, which model generative and absorptive events during locomotion (Usherwood and Bertram, 2003; Ruina et al., 2005; Bertram, 2016). While suspensory locomotion and inverted quadrupedal walking generally show a propulsive-then-braking pattern, (Ishida et al., 1990; Chang et al., 2000; Granatosky, 2015; Granatosky et al., 2016; Granatosky and Schmitt, 2017), mechanical cost analysis has yet to be applied to these data.

Mechanical cost analysis of animals can also be evaluated in light of theoretical predictions of how work can be mitigated by legged systems. A key finding of this mathematical approach is that mechanical cost of transport is reduced in proportion to the number of contacts used to achieve a given redirection of the CoM (Ruina et al., 2005). In other words, the mechanical cost of redirecting the center of mass is theorized to be divided by the number of sequenced leg contacts. For canine walking, as compared with human walking (see leg contact sequences in **Figure 1**), this prediction is borne out by mechanical cost analysis of experimental data, which shows about one-half the mechanical cost incurred for four sequenced leg contacts of a

SLIP-like "bouncing" gaits, having two- to five-times greater CoTmech at a given dimensionless speed. An apparent boundary zone (gray shading) suggests a discrete separation between SLIP-like gaits and those targeting orthogonal constraint.

dog as for two sequenced leg contacts of a human (**Figure 4**). The gallop and amble are faster quadrupedal gaits that use sequenced leg contacts to reduce mechanical cost of transport but not quite to the extent of quadrupedal walking (**Figure 4**). The concept of reducing cost by increasing the number of sequenced ground contacts has been extended to human walking by modeling contacts of the heel and toe as discrete "leg" contacts (Ruina et al., 2005). Just as for sequenced contacts of separate legs, this model predicts that sequenced heel and toe contacts halve the mechanical cost of transport during walking.

Another important theoretical prediction is that work can be reduced by increasing the length of the foot. Simulations show that a foot that extends forward the full length of the step and that is just slightly rounded (convex relative to the ground), as opposed to cupped (concave), achieves work-free redirections from one leg contact to the next during walking this case is likened to a polygon with convex surfaces rolling smoothly between pendular support phases at its vertices (Ruina et al., 2005). However, effects of the foot's length and its radius of curvature cannot be easily separated because, during singleleg stance, a rounded foot centered on a rigid leg prescribes the path of the center of mass in addition to lengthening the foot in both the "toe" and "heel" directions. The question of foot shape has been addressed in passive dynamic walking robots and simulations (e.g., McGeer, 1990; Kuo, 2001), as well as in human studies with experimental boots of different lengths and radii (Adamczyk et al., 2006). In the latter experiment, mechanical cost was reduced by simultaneously increasing the length and radius of curvature, a result that is similar to theoretical predictions based on foot length alone (Ruina et al., 2005).

It stands to reason that hexapedal and octopedal animals would have even lower mechanical cost than quadrupeds by using more than four sequenced leg contacts, however, mechanical cost analysis has not yet been applied to arthropods. The ideal of orthogonal constraint could be approached more nearly by eight sequenced leg contacts; however, arthropods primarily use alternating tripods to achieve SLIP-like trotting gaits (reviewed by Holmes et al., 2006). It is plausible that work reduction might not be as strongly selected in small arthropods—perhaps due to scaling of energetics, terrain interactions, or constraints on motor control. Whatever the explanation, mechanical cost analysis holds potential to further explore nature's solutions to multi-legged gait.

## Comparing Legged Animals and Machines

Legged robots designed for economy are typically slower than animals of similar size (e.g., Collins et al., 2005), with a few exceptions, such as the MIT Cheetah quadruped, which uses regenerative motors (Seok et al., 2015). Humans walk slowly (1.0 ms−<sup>1</sup> ) with a mechanical cost of transport of ∼0.05 and a total cost of transport of ∼0.41 (Voloshina et al., 2013), so mechanical cost represents about one-eighth of the total cost. If we assume no co-contraction between antagonist muscle pairs and a muscle efficiency of 25%, the muscle energy needed for CoM oscillations would represent just one-half of the total cost of transport. Hence, the remaining half must be spent on muscle energy needed for co-contraction, work of the legs against one another, and work for swinging of the arms and legs. In comparison to humans, the world's most economical bipedal robot, Cornell Ranger, achieves a mechanical cost of transport of ∼0.08 (determined by doubling the cost of positive motor work) at a speed of 0.6 ms−<sup>1</sup> (Bhounsule et al., 2014). After subtracting the energy used by its control system, Cornell Ranger has a total (electrical) cost of transport ∼0.11, and all of this energy is accounted for by positive and negative motor work with an average motor efficiency of 65%. Walking at a little more than one-half the speed of a human walking slowly, Cornell Ranger would need improved dynamics to both reach typical human walking speeds and achieve a mechanical cost of transport as low as 0.08.

## Summary

Legged gaits were first identified by the relative phases of leg contacts, which also identifies the sequence of leg contacts. A gait is defined as symmetrical when the relative phases of corresponding left-right pairs are separated by one-half stride cycle, such as in a striding biped or a trotting quadruped. When a left-right pair is used in unison (such as in hopping) or has staggered contacts (as in quadrupedal galloping), the gait is considered asymmetrical.

Locomotion is more correctly evaluated by measuring the dynamics of the center of mass (CoM). Traditional models have compared the phases of the CoM's potential and kinetic energy, using a spring-loaded inverted pendulum (SLIP) model when these energies are relatively in phase and a rigid inverted pendulum (RIP) model when they are out of phase. More recent work has introduced mechanical cost analysis that quantifies the work a gait requires by using the geometry of force and velocity vectors of the CoM.

All legged gaits include cyclic oscillations of the CoM; symmetrical gaits use two vertical oscillations, and asymmetrical gaits typically consist of only a single vertical oscillation per stride. SLIP-like "bouncing" gaits may be symmetrical (e.g., running and trotting) or asymmetrical (e.g., hopping and bounding), but they always couple braking with downward vertical velocity and propulsion with upward vertical velocity, resulting in mechanical work performed by the animal. Other symmetrical (e.g., walking and ambling) and asymmetrical (e.g., galloping and half-bounding) gaits, however, couple propulsion with downward vertical velocity and braking with upward vertical velocity, reducing an animal's mechanical cost of transport. This latter pattern takes advantage of d'Alembert's principle of orthogonal constraint by keeping force and velocity vectors more nearly perpendicular in each instance of the stride—thus reducing the mechanical cost of transport, which is zero in the idealized case of orthogonal constraint. Slow symmetrical walking gaits and faster asymmetrical galloping gaits exploit this principle to reduce work, and thus reduce the mechanical cost of transport. This reduction of mechanical cost of transport is a key feature of several gaits including bipedal and quadrupedal walking, fast gaits with multiple sequenced leg contacts, and brachiation. Theoretical mechanics show that increasing the number of sequenced leg contacts, dividing single foot contacts into multiple contacts, and increasing foot length are all mechanisms that can reduce mechanical cost of transport, as measured by mechanical cost analysis. Together with an increasing body of data from comparative animal locomotion, these principles hold substantial promise for the design and control of legged robots and prostheses that can achieve economical locomotion by reducing the mechanical cost.

## REFERENCES


## AUTHOR CONTRIBUTIONS

DL wrote approximately 60% of this review paper. SH wrote approximately 40% of this review paper.

## ACKNOWLEDGMENTS

We wish to thank André Seyfarth for valuable discussions of locomotion and physics, including d'Alembert's early work. DL was supported by an Alexander von Humboldt Research Fellowship.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Lee and Harris. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Minimally Actuated Walking: Identifying Core Challenges to Economical Legged Locomotion Reveals Novel Solutions

#### *Ryan T Schroeder 1\* and John EA Bertram 2*

*1 Biomedical Engineering Graduate Program, University of Calgary, Calgary, AB, Canada, 2 Cumming School of Medicine, University of Calgary, Calgary, AB, Canada*

Terrestrial organisms adept at locomotion employ strut-like legs for economical and robust movement across the substrate. Although it is relatively easy to observe and analyze details of the solutions these organic systems have arrived at, it is not as easy to identify the problems these movement strategies have solved. As such, it is useful to investigate fundamental challenges that effective legged locomotion overcomes in order to understand why the mechanisms employed by biological systems provide viable solutions to these challenges. Such insight can inform the design and development of legged robots that may eventually match or exceed animal performance. In the context of human walking, we apply control optimization as a design strategy for simple bipedal walking machines with minimal actuation. This approach is used to discuss key facilitators of energetically efficient locomotion in simple bipedal walkers. Furthermore, we extrapolate the approach to a novel application—a theoretical exoskeleton attached to the trunk of a human walker—to demonstrate how coordinated efforts between bipedal actuation and a machine oscillator can potentially alleviate a meaningful portion of energetic exertion associated with leg function during human walking.

#### *Edited by:*

*Monica A. Daley, Royal Veterinary College, United Kingdom*

#### *Reviewed by:*

*Madhusudhan Venkadesan, Yale University, United States Christian Michael Hubicki, Georgia Institute of Technology, United States*

*\*Correspondence:*

*Ryan T Schroeder ryan.schroeder@ucalgary.ca*

#### *Specialty section:*

*This article was submitted to Bionics and Biomimetics, a section of the journal Frontiers in Robotics and AI*

> *Received: 22 December 2017 Accepted: 24 April 2018 Published: 22 May 2018*

#### *Citation:*

*Schroeder RT and Bertram JEA (2018) Minimally Actuated Walking: Identifying Core Challenges to Economical Legged Locomotion Reveals Novel Solutions. Front. Robot. AI 5:58. doi: 10.3389/frobt.2018.00058*

Keywords: bipedal locomotion, energetics, control optimization, dynamics modelling, work minimization

## 1. Introduction

The movement patterns of humans and other animals have been described in remarkable detail (Bregler et al., 2004; Winter, 2009). However, why any given movement pattern is used, and not some other, is currently not thoroughly understood. Some of the machinery of biological systems (aspects of their morphology and internal organization) is inherited (involving inevitable evolutionary inertia). As a result, it becomes a challenge to distinguish true adaptive design modifications that improve locomotory capability from adaptations that simply accommodate functionally neutral, or even detrimental, anachronistic features. This makes it very difficult to interpret the actions used in locomotion, regardless of the technical detail in which it is analyzed. It would be beneficial to put the actions observed in locomotion in the context of what they accomplish, and determine the advantages and limitations a particular strategy provides to the motor control system.

In this paper we describe our understanding of some key aspects regarding the dynamics of legged locomotion. This understanding has emerged largely from synthesizing the works of groups attempting to construct artificial walking machines. One advantage of trying to generate an original walking machine, rather than mimicking how humans or animals already move, is that it naturally identifies specific challenges and obstructions faced in legged locomotion without the biased expectations of an existing system. Identification of the challenges to be solved is one of the first steps in the design process and the discovery of new and different potential solutions.

There are two parts to this contribution. In the first part our intention is to describe an emerging perspective on legged locomotion dynamics. We use the context of human walking as a familiar example in which these ideas can be evaluated. The objective is to demonstrate how this perspective can aid in interpreting, and not just describing, observed movement strategies. In the second part of the contribution we explore the potential of simply actuated walking models to see how the identified challenges can be met most efficiently. Finally, we discuss the application of concepts on minimal actuation as an external environment (i.e., exoskeleton) for human locomotion by applying a theoretical oscillating impulse acting at the torso of a walking human.

In this contribution we discuss two hypotheses: (I) the action of the legs in human walking optimizes (or nearly optimizes) the interaction of the body mass with the external environment, and consequently specific movement strategies are selected based on taking advantage of energy saving opportunities while mitigating costlier alternatives; (II) external actuation applied directly to the center of mass (as opposed to at specific joints or in tandem with muscle groups along the body) can reduce the optimized leg work required in a reductionist bipedal optimal control model during walking. We advocate for a reductionist approach in our modelling in order to more clearly isolate features that contribute to effective actuation and control strategies. The proposal is that details of within leg function and other such physiology based features are secondary considerations relative to the more fundamental interaction between the body mass and its external environment that defines the task of locomotion.

## 2. Part I: Alternate Perspectives on the Task of Locomotion

One conventional definition of the task of locomotion might describe specific features observed in real-world examples (e.g., human walking can be distinguished from running because the latter has a non-contact phase during the stride cycle). However, in this case the solution to the problem is an observed feature without a clear definition of the problem being solved, so this approach mixes the task with the solutions implemented to accomplish the task. As such, it is nearly impossible to separate these two aspects of function, and this confuses the context of the observations and muddles our attempts to find and evaluate explanatory constructs.

Another common approach is to consider that locomotion simply seeks to transport the body from one location to another. However, this definition—fundamental though it may be does not provide any real insight into how such a task should be managed. Indeed, one could imagine an infinite number of solutions to this formulation of the problem. In order to deal with this issue, we have recently proposed a reformulation of the fundamental task of legged locomotion (Croft et al., 2017). Briefly, any form of locomotion ultimately requires an interaction

FIGURE 1 | A diagram of the contextual hierarchy of locomotion. The task involves the fundamental optimal dynamic interaction of the body mass of the individual with the external environment through which they move. The task fulfills the goal of transportation as specified (distance, direction, speed, etc.), and one of the defining features in biological systems appears to be a drive towards energetic minimization. Mechanism(s) of locomotion (e.g., leg actuation, generation of joint torques, adjustment of leg stiffness, etc.) manage the task. The mechanisms available are constructed from the machinery (physical structures/tissues such as motors/muscles/skeleton) and the control strategy implemented to the machinery. Mechanisms tend to attract the attention of observers since the kinematics of the limbs are often readily visible, but the implementation of those mechanisms are only understood in the context of the fundamental task they accomplish.

between the organism (more specifically, its mass) with its external environment. For example, steady level flight requires navigation of the body through the low density fluid of our atmosphere, while simultaneously balancing forces of lift and gravity, as well as thrust and drag. Given this fundamental task, there are a number of mechanisms potentially available to manage the body massenvironment interaction – fixed, rotary or flapping wings that can be powered by combustion engines, electric motors or muscles. In a similar manner, we contend that the fundamental task of legged locomotion should be considered the optimal dynamic interaction of the system mass with the external environment (e.g., in terrestrial locomotion, this is typically the substrate, gravitational force, etc.). An optimal (or near optimal) interaction allows for effective travel and must meet overarching goals determined by the priorities of the system (e.g., travelling some distance in a given amount of time, etc.) (Srinivasan and Ruina, 2006).

Similar to flight, terrestrial locomotion has its own set of mechanisms that constitute the locomotory apparatus, all of which can be used to mediate the mass-environment interaction. The available mechanisms are composed of the machinery of the system (supporting tissues and actuators, whether organic or artificial) and the control regime implemented on the machinery (**Figure 1**). Still, the phrase optimal dynamic interaction remains ill-defined. In the following, we describe the role of energy minimization and analyze some basics of the human walking system while drawing on this perspective.

## 2.1. The Energetic Basis for Gait Parameter Selection

In natural human walking there is a standard, repeatable relationship between overground speed and stride frequency (Grieve and Gear, 1966; Bonnard and Pailhous, 1993). In fact, this relationship is so standard that it is possible to determine the bounds of normal walking and use these to define abnormal locomotion (Schwartz et al., 2008; Lythgo et al., 2011; Dixon et al., 2014). A different, but equally consistent relationship exists for human running (Kurz et al., 2005; Perry and Burnfield, 2010; Hein et al., 2012; Floría et al., 2018). However, documenting the gait parameters used in a given circumstance does little to explain why these are the particular movement strategies (nearly) universally selected. Certainly it is physically possible to walk (or run) with an extremely broad range of speeds, stride lengths or stride frequencies – so why is one set of solutions selected over others?

A hint at the basis for gait parameter selection (in this example, the parameters of interest are speed, step frequency and step length) and the natural constraints that determine the advantages of one strategy over another, can be drawn from the observation that individuals tend to choose a preferred walking speed when unburdened from explicit time constraints (e.g., rushing to catch the light at a crosswalk). Preferred walking speed tends to coincide with the global minimum cost of transport (CoT), or energy per distance traveled (Holt et al., 1995), although this observation continues to be challenged, (Godsiff et al., 2018). The CoT also appears to have an important influence on the selection of gait parameters over a range of walking speeds (Bertram and Ruina, 2001; Bertram, 2005). Since speed (*v*) is the product of step length (*d*s ) and step frequency (*f* s ), it is theoretically possible to manage any speed with an infinite number of step frequency-step length combinations. However, healthy humans tend to employ a generally standard relationship (Kuo, 2001).

As with the selection of preferred speed (and its step frequencystep length combination), the systematic change in these parameters from preferred speed can also be explained based on CoT energetics. As speed changes, step parameters (*d*<sup>s</sup> and *f* s ) are chosen to match the minimum solution for speed constraints on the objective function of CoT (Kuo, 2001). Although it may be suggested that speed change is a natural requirement of walking control, this result suggests that the control strategy is treated as a constrained optimization, where the optimization approaches the minimum cost combination available on the CoT surface.

Similarly, because speed is the product of step length and step frequency, it is also possible to demonstrate the constrained optimization response for the other two parameters (*d*<sup>s</sup> , *f* s ) as well as for speed (*v*). When either step length or step frequency is constrained, the response of the other two parameters also tends to follow a minimum cost solution, but the solution differs based on the shape of the cost surface (Bertram, 2005). Optimizing the CoT surface as the objective function can explain a striking contrast in the speed-frequency relationship human subjects exhibit while walking with a constrained frequency (following a range of metronome beat frequencies), a constrained step length (walking in registry to a range of spaced floor markers) or a constrained speed (walking on a treadmill for a range of belt speeds) (**Figure 2**).

FIGURE 2 | Constrained optimization of gait parameters in walking. Light blue contours represent equivalent cost combinations (iso-cost contours), where each contour is energetically less costly than the one residing outside it (minimum cost is central at the point where the red, blue and green lines intersect). For any constraint of speed ( *v*), step frequency (*fs*), or step length (*ds*) the minimum cost solution features gait parameters where the constraint line is tangent to the cost contour (any other solution lies outside the contour, so is costlier). Constrained *v* relationship (red) is determined from horizontal tangents and the constrained *fs* relationship (blue) is determined from vertical tangents. The constrained *ds* relationship (green) is determined from sloped tangents (since *v* = *dsfs*).

This result shows that the selection of gait parameters in humans is not stereotyped but is actually quite plastic, and specifics of the gait are chosen (or at least highly influenced by a pressure) to minimize the cost of moving over the substrate. It should be acknowledged that other influences (e.g., obstacles to be avoided at the substrate, slippery surfaces, etc.) certainly play a role in the selection of gait parameters as well, and in fact, there is often an interdependence between other considerations and energy consumption (e.g., avoiding a slippery surface or else recovering from a fall has an energetic cost associated with it; Brandão et al., 2015). Regardless, in addition to human walking, energetic cost has also been shown to have a dominant influence on step width in human walking (Donelan et al., 2001), human running (Gutmann et al., 2006), walking in cats (Bertram et al., 2014) and for direct, acute manipulations of the objective function (the CoT surface; Selinger et al., 2015).

## 2.2. Actuator Performance

The evidence above indicates that minimizing energy expenditure is a key control factor in humans (and likely in other animals as well). So it might be useful to consider *how* energy can be minimized. One option is to seek more efficient actuators. However, even if ideal efficiencies are possible, this approach has a yield limited by the cost of the strategy. However, the strategy itself can be modulated (adjusting the control regime, **Figure 1**) and such modulation can have a substantial consequence for cost. Consider, for instance, that most high fidelity legged robots, such as Honda Asimo, have motors that are at least 3–5 times more efficient than mammalian muscle, yet their CoT for walking on legs can be well over 10 times greater than that of humans (Collins et al., 2005). Understanding the subtleties of human walking control may have large payoffs in robotics.

In many engineering circumstances inadequate energy or power capabilities can be addressed with the implementation of more sophisticated actuators and/or larger power supplies. However, state-of-the-art technology capable of maximizing performance potential is often very expensive. Furthermore, scaling up the power of actuator systems typically comes at a trade-off of increased volume and weight not particularly suitable for the mobility desired in locomotion systems. Thus, artificial design options may be informed by an understanding of how organic systems manage impressive performance despite efficiency limitations. In this, we contend that the goal of energy minimization directs attention to some important factors influencing general performance of legged locomotion systems and the effective movement strategies available to them.

## 2.3. Energy Transduction in Walking

The predominant conventional approach to analysis of walking gaits considers transduction of energy forms as it flows within the system (e.g., between potential and kinetic energy; Cavagna et al., 1977; Cavagna et al., 2002). However, we argue that a more comprehensive strategy should also track energy flow into and out of the system (Srinivasan and Ruina, 2007). This aspect is important because energy loss must be paid back in the form of mechanical work, and this imparts a metabolic cost on the organism, at least for the case of a steady state gait. Thus, assuming energy loss is undesirable, the manifestation of this loss must indicate either a limitation of the specific gait mechanism used and/or a constraint that restricts the strategy chosen. Understanding how the loss occurs (and why it occurs) allows for clear distinction of various strategies available to manage the interaction with the substrate. How does energy move through a legged walking system?

Walking is commonly described based on variations of an inverted pendulum model where potential (PE) and kinetic energy (KE) fluctuations are largely out of phase during the single stance portion of the stride. During this time, the center of mass (CoM) rises to a maximum (PE increases as KE decreases) and then begins to fall (PE decreases as KE increases), and this passive redirection is largely managed by the acceleration of gravity. Direct exchange of PE and KE during single stance implies a near constant total mechanical energy and minimal energetic losses from the system (i.e., single stance represents a low cost portion of the gait cycle; **Figure 3**). Typically, the inverted pendulum model only considers the stance phase described, and as such, energy losses from the system are often neglected, even though they do occur in realworld locomotion.

Specifically, redirection of the CoM (from down to up) incurs a cost that must be mediated by the action of the legs (Srinivasan and Ruina, 2007). This occurs during double stance in walking when the CoM reaches its lowest point in the gait cycle. Since this vertical redirection is largely active, it requires a high energetic cost (relative to the rest of the gait cycle; **Figure 3**), which manifests as a loss of energy that must be repaid through leg work (to maintain steady state gait). It is informative to look more closely at the mechanisms

through which this can occur in walking, and consider strategies implemented to minimize the energetic cost.

## 2.4. Collision Dynamics and Transition Loss

An important, and often overlooked, cause of energy loss originates with collision dynamics. A collision involves an abrupt change in the momentum of a body when it interacts with an impulsive force, and this results in a loss of energy. In terrestrial locomotion the legs contact the substrate and alter the trajectory of the individual's mass. Although in biological systems these interactions may not appear particularly impulsive in the classical dynamics sense, the trajectory change of the body mass from downward to upward during the step-to-step transition of walking can be viewed in terms of collision events (Kuo, 2002; Kuo et al., 2005; Ruina et al., 2005; Srinivasan and Ruina, 2006; Lee et al., 2013). The organism experiences a loss of kinetic energy as the ground reaction force does mechanical work on the CoM (in addition to the trunk, this also includes body segments with motion relative to the trunk). The energetic consequence on the organism can be quite meaningful and is quantified by the dot product of the ground reaction force (GRF) vector and the CoM velocity vector integrated over the duration of the impulse (Lee et al., 2011). The consequence of this relationship is such that a perpendicular vector orientation results in no work done by the impulse (no energy loss), since cosine of 90° (and 270°) equals zero. However, non-zero mechanical work is done with any other vector geometry.

Although in terrestrial locomotion the limbs act primarily as struts, the inherent compliance of the jointed limb means that force application is not purely impulsive but is instead distributed over the duration of the step. Nevertheless, the basic principles that govern redirection of colliding objects can be applied to the redirection of the CoM during locomotion. This results in an energy loss that forms the basis of legged locomotion costs in gaits such as walking and running. An alternative view is that at least some energy is retained and recovered by elastic structures in the leg. Elastic energy recovery is undoubtedly useful, but it is not essential to gait (Srinivasan and Ruina, 2006). The optimal CoM path appears to be identical whether the supporting legs have elasticity or not (Ruina et al., 2005). In reality, it is likely that collision mitigation and elastic energy recovery occur – with both being complimentary (Bertram and Hasaneini, 2013).

## 2.5. Minimizing Energy Loss at the Stepto-Step Transition

The reader may recall that the high cost portion of walking occurs when the CoM is redirected from moving downward to upward at the step-to-step transition during double stance (**Figure 3**) and forward momentum is maintained over the stride cycle. Since there are two legs contacting the substrate over the transition, various strategies exist to mitigate energy loss if the two limbs work together in a coordinated manner.

In fact, details of the step-to-step transition turn out to be critically important in determining the overall CoT of bipedal walking (Donelan et al., 2002a). One option is to use heel contact at the beginning of stance to redirect the CoM, where it is simply vaulted over the strut-like leg (**Figure 4A**). However, this vaulting action inevitably results in energy loss as the strut redirects the path of the CoM. This loss can be replaced by push-off work from the trailing (former) stance leg, which momentarily maintains ground contact during the transition period.

Although a strategy utilizing heel strike *before* push-off is a viable solution, it is not the most effective strategy for managing the stepto-step transition. Instead, it is highly advantageous to initiate heel strike just *after* push-off from the trailing leg (Kuo, 2002; Donelan et al., 2002b; Kuo et al., 2005). This particular sequencing allows the previous stance limb to begin redirecting the CoM with a forward and upward impulse (commonly referred to as preemptive pushoff) before the collision occurs. The preemptive push-off helps to orient the CoM velocity vector more perpendicular relative to the force vector resulting from heel strike (**Figure 4B**). Ultimately, this allows for substantial reduction of momentum (and energy) loss due to the collision (Ruina et al., 2005).

It is possible to eliminate collision loss at the step-to-step transition with a gait sometimes referred to as Groucho walking. To accomplish this, the substrate is contacted with a relatively straight leg that initially flexes and then extends over stance. This allows the CoM to maintain its vertical position as it passes over the contact point in a straight horizontal path. Although this can eliminate the collision-based loss, it turns out that the leg work required (extending and flexing under the load) is greater than

step-to-step transition (double stance) in walking. (A) Energy inefficient walking – the leading leg makes contact (heel strike) and the velocity vector magnitude decreases along a path parallel to the contact leg (collision loss). The trailing leg then applies a push-off force accelerating the velocity vector back to its original magnitude. The circular arc connecting the tips of the velocity vectors indicates a change in vector orientation without change in magnitude (constant kinetic energy). The area between the arc and the vector path during the step-to-step transition (shaded grey) is related to the work required for the transition. (B) Energy efficient walking – a preemptive push-off occurs from the trailing leg prior to heel strike of the leading leg. The push-off shifts the velocity vector to a more horizontal orientation making the interaction between the new stance leg and velocity vector much more favorable (less work required in the transition is indicated by the reduced size of the grey shaded area). Note leg and velocity vector angles are exaggerated for clarity.

the collision loss it prevents. This has been shown both analytically (Ruina et al., 2005; Gordon et al., 2009) and empirically (Ortega and Farley, 2005; Gordon et al., 2009).

Another feature of an energy effective step-to-step transition involves swing leg retraction. In swing leg retraction the impending next stance leg is accelerated opposite the direction of travel just prior to it contacting the ground (heel strike). Due to mechanical coupling of both legs at the pelvis, rearward acceleration of the leading leg results in a reaction force (at the hip) that accelerates the rest of the body forward, and this aids push-off of the trailing leg. As such, impulses from push-off can be partially down regulated. However, the relative magnitude and timing of stance leg preemptive push-off and swing leg retraction requires coordination to optimize energetic cost (Hasaneini et al., 2017).

In natural human gait, an optimal step-to-step transition strategy comprises a trade-off between collision loss reduction and leg work associated with flexion and extension at the joints. (Bertram and Hasaneini, 2013). It should be emphasized, however, that an effective step-to-step transition in walking requires coordination between both legs in the approach up to and during the transition. This coordination is indicated by the distinctive double hump vertical GRF of human walking. Whereas this pattern is generally interpreted with regard to function of each leg individually, it occurs largely because the second vertical maxima in stance is associated with the critical preemptive push-off while the first indicates the transfer of load to the new stance leg (i.e., heel strike). Each portion of the contact should be functionally interpreted with respect to its role in the transition, rather than as an aspect of the force sequence an individual leg generates over stance (Usherwood, 2016; Bertram, 2016c).

Given some insight into the subtle strategies available to manage the energetic cost of the step-to-step transition in human walking as described above, how can this be applied to alternative designs in legged robots? Passive dynamic walking machines (no actuators nor controllers, as the name implies) are equipped with legs that spontaneously swing in an appropriate manner to stabilize forward progress (McGeer, 1990) while moving down a slightly sloped ramp. With each step, a small amount of PE is converted to KE as the machine falls forward, however this extra energy is soon lost due to collision interactions with the ramp's surface at the stepto-step transition (Garcia et al., 1998). Ultimately, this allows for a near steady state gait pattern that qualitatively looks remarkably like human walking (Bertram, 2016a).

Variations on the passive dynamic walker incorporate simple actuators that can provide small impulses at each leg to allow for level surface walking (Collins et al., 2005). As discussed above, the preemptive push-off impulse of the actuator plays an important role in overcoming energetic losses due to collisions while redirecting the CoM from a downward trajectory to upward.

There is also a secondary role of the active (preemptive) pushoff in that it helps facilitate the leg's forward swing in order to set up the next step. Ankle plantar flexion just prior to heel strike has been associated with preparing the leg for the swing portion of the step (Winter and Robertson, 1978; Meinders et al., 1998). It is likely that the push-off does indeed fulfill this functional role, but the swing preparation and preemptive, collision mitigating push-off are not mutually exclusive, so it is likely that both roles are satisfied by this single action (Zelik and Adamczyk, 2016).

## 3. Part II: Simply Actuated Walking Models

In this part of the contribution, we outline various options for reductionist bipedal designs that rely (to varying degrees) on many of the concepts discussed in Part I. We begin with single actuator mechanisms and progress to multi-actuator mechanisms, in order to alleviate some of the restrictive dynamics inherent in simpler designs. Finally, we discuss an application of similar concepts to an exoskeleton strapped to the trunk of a walking human. For most of the models presented, control optimization software is used to determine energetically minimal solutions. These solutions are then analyzed post-hoc in order to isolate important features that either support or violate expectations of what economical locomotion should look like based on an understanding of established theory. This section is organized with specifically chosen models to invoke a discussion about important dynamic restrictions and the consequences of different actuation patterns on the energetics of effective locomotion during bipedal walking. A primary objective of the models is to explore the limit of reducing the number of actuators necessary to allow active bipedal locomotion (at least in the planar case).

## 3.1. Single Actuator Designs

## 3.1.1. Constant Force Single Actuator Inverted Pendulum

The placement of an actuator at each leg to power foot extension is one means by which to add work and replace energy lost from collisions and other inefficiencies (Collins et al., 2005). This may be considered a bioinspired design, but it is likely that much of the energetic benefit is achievable merely with a single actuator acting directly at the CoM. In fact, it is possible to mathematically replicate the constant gravitational forces acting on the passive dynamic walker on a sloped surface with a single actuator (constant orientation and force magnitude) acting directly on the CoM for a walker on a level surface (**Figure 5**). To solve for the actuator orientation and magnitude, the gravitational force (acting on a reference frame of an elevated slope, *γ* > 0*◦* ) is set equal to a constant actuator force plus a gravitational force (acting on a reference frame of no slope, *γ* = 0*◦* ). Two equations are formulated for the forces in the horizontal (Eq. [1]) and vertical (Eq. [2]) directions (left side of the equations: gravitational force acting on a sloped surface, right side of the equations: gravitational and actuator forces acting on a flat surface).

$$\mathcal{L}m\_{\mathcal{G}}\cos\left(1.5\pi+\gamma\right) = \mathbf{0} + F\_m \cos\left(\theta\_m\right) \tag{1}$$

$$m\_{\mathcal{L}}g\sin\left(1.5\pi+\gamma\right) = -m\_{\mathcal{L}}g + F\_m\sin\left(\theta\_m\right) \tag{2}$$

where *mc* is the body mass, *g* is gravitational acceleration (e.g. 9.81 *<sup>m</sup> <sup>s</sup>*<sup>2</sup>), *Fm* is a constant actuator force, *θm* is the angle of the actuator and *γ* is the angle of the ground's slope. When equations [1,2] are solved simultaneously, *Fm* and *θm* are analytically determined.

$$F\_m = m\_\text{cg} \sqrt{2 + 2 \sin \left( 1.5 \pi + \gamma \right)} \tag{3}$$

$$
\theta\_{\mathcal{W}} = \frac{\gamma}{2} \tag{4}
$$

The strategy of powering a walking machine purely with gravitational forces means that no batteries are necessary, and the work done by gravity is essentially free. Furthermore, only a very subtle slope is needed to overcome the energy losses due to collisions if the system is constructed properly. However, the constant-force actuator

alternative must do work to mitigate gravitational forces as well as overcome collision losses. Although this actuation strategy may exist as a viable solution, the constant force profile can likely be improved upon. For example, more sophisticated strategies might leverage dynamic force production as a means for reducing the mechanical work done by the actuator.

## 3.1.2. Optimized Single Actuator (Horizontal) Inverted Pendulum

Assuming that ideal actuation strategies are unknown a priori, control optimization procedures can be used to determine the actuator force profile that minimizes mechanical work over a step. Although a specific actuator angle (*θ<sup>m</sup>* = *<sup>γ</sup>* <sup>2</sup>) was necessary to replicate the gravitational forces acting on a passive-dynamic walker down a slope, this angle is not required for a non-constant actuator force profile. Instead, a fixed horizontal orientation (*θm* = 0*◦* ) was chosen somewhat arbitrarily (**Figure 6A**), although this configuration does allow for symmetrical force profiles mirrored about mid-stance (i.e., when the CoM is directly above the foot-ground contact). The equation of motion for a standard inverted pendulum model is expanded to reflect the influence of a fixed horizontal actuator.

$$m\_{\mathcal{L}}\ddot{\mathbf{x}}\_{t} = m\_{\mathcal{L}}\mathbf{g}\cos\left(1.5\pi-\theta\_{L}\right) + F\_{m}\cos\left(-\theta\_{L}\right)$$

$$-m\_{\mathcal{L}}\ddot{\theta}\_{L}L = m\_{\mathcal{L}}\mathbf{g}\cos\left(1.5\pi-\theta\_{L}\right) + F\_{m}\cos\left(-\theta\_{L}\right)\tag{5}$$

where ¨*xt* is the tangential acceleration of the CoM motion and *θ<sup>L</sup>* is the leg angle relative to vertical. Note, the actuator force, *Fm*, is not constant as in Eq. [3], however it is a control variable optimized in the control optimization process. The reaction force of the rigid leg is also shown for the inverted pendulum.

$$R\_{\varGamma} = m\_{\varGamma} \ddot{\upnu}\_{\varGamma} - m\_{\varGamma} \operatorname{g} \sin \left( 1.5 \pi - \theta\_L \right) - F\_m \sin \left( -\theta\_L \right)$$
 
$$R\_{\varGamma} = -m\_{\varGamma} \dot{\theta}\_L^2 L - m\_{\varGamma} \operatorname{g} \sin \left( 1.5 \pi - \theta\_L \right) - F\_m \sin \left( -\theta\_L \right) \quad \text{(6)}$$

where *L* is a constant leg length used in the model. Gait parameters such as average forward velocity (*v*), step frequency (*fs*) and step length (*ds*) are all pre-determined constraints in the model. Specifically, time is constrained from initial point *to* = 0 to final point *tf* <sup>=</sup> *Ts* where *Ts* <sup>=</sup> *ds <sup>v</sup>* . Step length was enforced by constraining CoM position at the initial point *xc* = 0 and at the final point *xc* = *ds*. A biologically realistic step length was chosen (Alexander, 1992) for an average forward velocity of *v* = 1*<sup>m</sup> s* .

$$d\_{\mathcal{S}} = 1.25 \left( \frac{L\_{\text{max}}^{0.7}}{\mathcal{S}^{0.3}} \right) \nu^{0.6} \tag{7}$$

A path constraint was applied to the optimization in order to ensure that only solutions requiring reaction forces greater than or equal to zero (i.e. *Rr ≥* 0) throughout the step were considered (tension leg forces were not allowed since this would require the foot to actively stick to the ground). Endpoint constraints were also applied such that only periodic force profiles and CoM kinematics (i.e. steady state patterns) were considered.

Finally, the objective function, or cost function, was chosen to minimize the summation of a mechanical work-based cost and a force-rate-squared term (scaled by an arbitrarily small number, *ϵ*1). The force-rate-squared term was employed in order to avoid extreme impulsive actuator forces (a theoretical, but unrealistic optimum). This allows for smoother force profiles and a quicker optimization with more reliable results. The cost function is explicitly stated.

$$\mathbf{C} = \int\_{t\_0}^{t\_f} \left( \dot{W}\_m^+ - \dot{W}\_m^- + \epsilon\_1 \dot{F}\_m^2 \right) dt \tag{8}$$

where *W*˙ <sup>+</sup> *<sup>m</sup>* is the positive mechanical power of the actuator, *W*˙ *<sup>−</sup> m* is the negative mechanical power of the actuator and *F*˙*m* is a time-rate of the actuator force. Mechanical power is calculated.

$$
\dot{W}\_m = -F\_m \dot{\mathbf{x}}\_l \cos \left( -\theta\_L \right) = \dot{W}\_m^+ - \dot{W}\_m^-
$$

$$
\dot{W}\_m = F\_m \dot{\theta}\_L \mathcal{L} \cos \left( -\theta\_L \right) = \dot{W}\_m^+ - \dot{W}\_m^- \tag{9}
$$

optimal solution of the inverted pendulum model with a horizontal actuator (multiple force rate scaling constants are shown: (*ϵ*<sup>1</sup> = 3*x*10*−*6, *ϵ*<sup>2</sup> = 9*x*10*−*6, *ϵ*<sup>3</sup> = 3*x*10*−*<sup>5</sup> *and ϵ*<sup>4</sup> = 9*x*10*−*<sup>5</sup> ). (D) The Groucho walker with a vertical actuator has zero vertical GRF and a constant *Fm* that defaults to supporting body weight.

Orthogonality of *W*˙ <sup>+</sup> *<sup>m</sup>* and *W*˙ *<sup>−</sup> m* was ensured by augmenting the cost function with an additional cost term scaled by a small number: *ϵoW*˙ <sup>+</sup> *mW*˙ *<sup>−</sup> m*. The cost of this term was always driven to zero in all optimizations, and therefore it did not contribute to the overall cost of the solution. However, its implementation ensures that the actuator can never produce both positive and negative work simultaneously.

A sparse nonlinear optimizer program (SNOPT) (Gill et al., 2005) was used to solve for the optimization problem and the MATLAB (The MathWorks Inc. in Natick, Massachusetts) software GPOPS-II (Patterson and Rao, 2014) was used for problem discretization and setup. A duel part optimization process was employed. In the first part, multiple solutions (*n* = 15) were determined with random initial guesses in order to reduce the likelihood of settling at a local optimum in the cost function. The lowest cost solution of the 15 random initial guesses (i.e., seed) was then put through a perturbation phase where initial guesses were supplied by the seed solution plus random noise scaled to 12.5%, or one eighth, of each variable's overall range. Multiple perturbation solutions (*n* = 15) were determined, and the seed solution was only considered optimal if its cost remained lower than the outcome of all perturbation iterations. In the case that a perturbation iteration resulted in a lower cost solution, it was chosen as the new seed, and an additional round of perturbation iterations was conducted. This process was reiterated until the seed's cost was found to be lower than all perturbation solutions. The perturbation phase was conducted in order to fine tune the optimal solution.

The solution resulting from the optimization is characterized by an actuation strategy similar to what optimal control theorists often refer to as bang-coast-bang (Athans and Falb, 1969). Specifically, near impulsive forces mark the beginning and end of the step, with a quiet period of inactivation toward mid-stance (*to* = 0 is associated with the beginning of stance, essentially heel strike). The first bang (impulse), toward the beginning of the step, is positive (i.e., in the direction of travel) and accelerates the body's tangential motion from rest. The second bang, toward the end of the step, is negative (i.e., opposite the direction of travel) and decelerates the body's tangential motion back to rest (**Figure 6C**). It should be noted that a true bang-coast-bang pattern more commonly exhibits instantaneous discontinuities of state, however this kind of solution is penalized with the force-rate-squared term. Nevertheless, the near impulsive forces (high magnitude, short duration) can still be considered an approximation of a more literal bang-coast-bang pattern. To illustrate the smoothing effect of the force rate cost, the optimization was run with force rate scaling constants over a broad range of values (*ϵ*<sup>1</sup> = 3*x*10*<sup>−</sup>*6, *<sup>ϵ</sup>*<sup>2</sup> = 9*x*10*<sup>−</sup>*6, *<sup>ϵ</sup>*<sup>3</sup> = 3*x*10*−*<sup>5</sup> *and <sup>ϵ</sup>*<sup>4</sup> = 9*x*10*−*<sup>5</sup> ) As the scaling constant increases, the force magnitudes decrease and are spread out over a longer period of time in order to achieve the impulse required by the solution (**Figure 6C**).

One can compare the dynamic function of the optimization's near impulsive forces to similar actions in human walking: push-off and heel strike, respectively. In efficient bipedal locomotion, the preemptive push-off earns its name by initiating the impulse just before heel strike. As a result of adding energy into the system first, the CoM velocity vector is redirected upwards (and forwards). This serves to orient the angle relating force and velocity vectors more perpendicularly, and ultimately results in a reduction of collision losses imparted by the heel strike impulse (**Figure 4B**).

However, the current walker utilizes a reversed strategy with a heel strike-like impulse toward the end of the step to slow to a stop and then a push-off-like impulse toward the beginning of the next step to accelerate back to speed again. This strategy is particularly expensive and re-emphasizes the benefit of optimal sequencing of leg forces during human walking. The reason the walker cannot utilize the alternate beneficial sequencing is because it must satisfy constraints of periodicity. The result is that the CoM is required to begin and end with zero velocity at the stepping transition, as a direct result of the inverted pendulum beginning with a rising arc and ending with a falling arc. As such, a unique continuous periodic solution exists where the CoM begins and ends with zero velocity (note the option of a collisional impulse at the transition is excluded since it creates a discontinuity in the CoM trajectory).

## 3.1.3. Single Actuator Groucho Walker

An alternative system which allows for radial deviations in the CoM (e.g., telescopic legs) could potentially achieve continuous periodic gaits. Such a system might rely on a vertically oriented actuator in order to effectively support the weight of the body, since the legs are not actuated and cannot bear load (**Figure 6B,D**). In this case, it is easy to imagine that a trivial solution would be optimal. Specifically, the solution could utilize a constant force actuator to consistently support body weight along a straight path. Further, because no vertical oscillation is necessary, zero mechanical work is required of the actuator.

It should be noted that the analogous gait in human walking referred to earlier in Part I as Groucho walking (Bertram et al., 2002)—imparts a much greater cost on the person relative to natural walking (Ortega and Farley, 2005; Gordon et al., 2009). This has a very different energetic consequence compared to that of the isometric force actuator, simply because the actuator is supporting body weight from an ideal orientation underneath the body. Essentially, this solution represents the dynamic equivalent of a wheel, which allows for continuous support even as it rolls in a straight path along the ground. Another example of such a system is the gliding of an ice skater. The legs simply bear the weight of the body but do no work to displace the body.

Perhaps a system utilizing a vertically oriented actuator might take advantage of the rigid strut-like leg in the inverted pendulum and use the actuator to provide impulses at the stepping transition. Although such a walking mechanism is theoretically possible, there is little the actuator could do without requiring a tension force in the leg to keep it grounded, or else launch itself into the air during actuation.

In the following section, we discuss the potential of walking robots that require multiple actuators to accomplish efficient walking gaits.

## 3.2. Multiple-Actuator Designs

### 3.2.1. Inverted Pendulum with Telescopic Leg Actuators

The fully passive inverted pendulum model has been used to characterize the fundamentals of human walking for many decades (Cavagna and Margaria, 1966; Alexander, 1980). Although it remains a successful model for describing aspects of natural gait, it is limited by its capacity to predict motor responses during atypical walking gaits. Here, the word atypical specifies any such gait where the inverted pendulum is not naturally selected (e.g., Groucho walking, running, skipping, etc.). This is somewhat peculiar given that all forms of typical *and* atypical gaits still utilize the same morphological leg. Thus, an alternative way to think about the inverted pendulum is as a motor control strategy for effective bipedal walking. Specifically, it is the minimal energetic cost associated with the distinctive arced trajectory of the inverted pendulum that allows for efficient bipedal walking. Although focus is generally on the minimal work required for the inverted pendulum during single stance, a bipedal system *does* require an instantaneous impulse to redirect the CoM from downward to upward at the step-to-step transition (assuming a steady, periodic gait), and this impulse *does* impart a quantifiable cost on the system. Of course, in reality, the biological biped does not utilize ideal impulses (instantaneous with infinite magnitude), but rather, it imparts impulse-like forces (high magnitude, relatively short burst duration) to manage CoM redirection. These impulsive forces largely align with the orientation of the legs in the form of a push-off and a heel strike force, which both contribute to the characteristic double-humped profile of the vertical ground reaction force, as discussed in Part I (**Figure 7B**).

To test whether these dynamics are optimal without explicitly constraining them (such as with the inverted pendulum model), two telescopic legs with linear actuators are utilized to provide optimized force profiles that manage the CoM trajectory with minimal mechanical work. A similar model was utilized by Srinivasan and Ruina (2006). Even though the model used the same mechanism (telescopic leg actuators) for all conditions, it spontaneously discovered an optimal walking gait at slow speeds and an optimal running gait at high speeds. It also discovered a hybrid pendular-running gait at intermediate speeds. Although humans do not naturally employ pendular-running locomotion, evidence that various avian species use a similar pattern have since been described (Usherwood, 2010).

Here we employ a similar model, also with two massless telescopic leg actuators and a point mass body (**Figure 7A**). The equations of motion are detailed.

$$m\_{\mathcal{L}}\ddot{\mathbf{x}}\_{\mathcal{L}} = \sum\_{\{l,r\}} F\_l \left( \frac{\mathbf{x}\_{\mathcal{L}} - \mathbf{x}\_{\mathcal{fl}}}{L\_l} \right) \tag{10}$$

$$m\_{\mathcal{C}}\ddot{\mathbf{y}}\_{\mathcal{C}} = \sum\_{\{l,r\}} F\_l \left(\frac{\mathbf{y}\_{\mathcal{C}}}{L\_l}\right) - m\_{\mathcal{C}}\mathbf{g} \tag{11}$$

where ¨*xc* is the horizontal acceleration of the CoM, *Fi* is the leg actuator force for both left (*l*) and right (*r*) legs, *xf* is the position of the foot contact (where the force vector originates from; the foot contact is a constant parameter since a non-slip contact is assumed) for both legs and *L* is the effective leg length of each limb, as formulated below.

$$L = \sqrt{\left(\varkappa\_{\mathcal{L}} - \varkappa\_{f}\right)^{2} + \varkappa\_{\mathcal{L}}^{2}} \tag{12}$$

In order to ensure the model does not take advantage of unreasonable leg length values (e.g., *L ≫ ds*), a path constraint was applied to the optimization. The constraint mandates that a leg actuator cannot produce force if the CoM is further away from the foot contact than the maximum leg length indicates.

$$F\_{\text{Leg}}\left(L\_{\text{max}} - L\right) > 0\tag{13}$$

A control optimization protocol was applied (as described in the single actuator methods) that included a work-based cost and a force-rate-squared cost for each leg actuator (Eq. [8]). The forcerate-squared term serves to penalize highly impulsive forces in favor of more realistic, smooth leg forces. The mechanical power of the leg actuators (*W*˙ *leg*) utilized in the cost function is shown as a function of leg force (*Fleg*) and leg length velocity ( *L*˙).

$$
\dot{W}\_{\text{Leg}} = F\_{\text{Leg}} \dot{L} \tag{14}
$$

$$\dot{L} = \frac{\left(\varkappa\_{\mathcal{L}} - \varkappa\_{f}\right)\dot{\varkappa}\_{\mathcal{L}} + \jmath c \dot{\jmath}\_{\mathcal{L}}}{L} \tag{15}$$

GRF of this model are shown in comparison to empirical data (**Figure 7B**). Many key features of human walking are reflected in the vertical GRF of the model. For example, the model oscillates between periods of single stance (a single leg provides force) and double stance (both legs provide simultaneous force). The characteristic double-humped profile is also notable in the optimal solution of the model. The hump towards the end of stance occurs due to active extension of the trailing leg and replicates the preemptive push-off found in human walking. Recall, the preemptive push-off does positive work to reorient the CoM velocity vector more perpendicularly to the force vector of the coming collisional impulse at heel strike (**Figures 4B and 7C,D**). This impulse manifests in the signal as the hump at the beginning of the next stance leg and occurs due to extension forces of the forward leg resisting compression. Similar to human walking, this sequencing helps to maintain momentum with minimal loss at the step-to-step transition. Horizontal GRF are also similar—both showing a deceleration phase towards the beginning of stance and an acceleration phase towards the end of stance. Finally, the point of zero horizontal acceleration occurs approximately at mid-stance (CoM is above the foot contact position).

Overall, the optimal solution of this model takes advantage of the passive dynamics of the inverted pendulum during the majority of single stance by holding a rigid leg (constant radius trajectory means the leg does not extend, and this has no work-based cost since leg velocity is zero). However, the model deviates from this pattern at the step-to-step transition and relies on impulsive forces by both legs simultaneously in order to manage the redirection of the CoM from down to up. The majority of the model's work-based cost is accumulated at this transition, however, it is managed as efficiently as possible, short of using ideal impulses (recall these solutions are penalized by a force-rate-squared cost for more realistic force profiles).

## 3.2.2. Forced Coupled Oscillator Model (No Actuator Cost)

The inverted pendulum with telescopic leg actuators is arguably the most realistic model for human walking, as compared to other walking mechanisms described in the contribution thus far. This is because previous models considered rigid strut-like legs (as well as, in one case, collapsible legs) and relied on a fixed-orientation actuator to provide force directly to the CoM. However, humans use legs themselves as actuators (non-fixed orientation) to apply force to the body. Still, it may be useful to consider a composite of the two strategies, where a total of three actuators are available to the model: two telescopic legs plus an additional vertical force applied directly to the CoM. Essentially, this allows the model to deconstruct the GRF into distinct signals that are distributed among the different actuators, thereby implying optimal function based on the orientation and magnitude of the resulting force vectors.

Specifically, a coupled oscillator mechanism is used to consider a more specific form of actuator force applied to the CoM. The coupled oscillator mechanism consists of a linear actuator that drives a point mass (*mL*) in vertical oscillations off the body (**Figure 8A**). The influence of these forces is manifested through the reaction force of the actuator on the body CoM (*mc*). In this model, the added point mass of the coupled oscillator mechanism can be thought of in two ways: (1) as an additional load that the walking mechanism carries or (2) as a portion of the existing CoM now split into two pieces (in either case, *mL* < *mc*). Although this distinction does affect force magnitudes, we account for this by reporting forces in units of body weight, where 1 *BW* = *g*(*mc* + *mL*). This is analogous to a horse's head bobbing up and down during locomotion. The mass of the head is a portion of the total body weight and the neck muscles are the actuator to help drive (and control) this load, although in this case much of this oscillation is likely passively managed by the complex nuchal ligament (Gellman and Bertram, 2002). Regardless, the

in grey whilst the total force is shown in black. (C) Kinematics of CoM trajectory are shown for mid-stance [asterisk notes mid-stance as a common point in the gait cycle between panes (A) and (C)], double stance and mid-stance again. The dashed box indicates the step-to-step transition region shown in (D) where the CoM velocity vector is reoriented from the beginning of push-off (*Vpo*) through to the middle of double stance (*Vds*) and to the end of heel strike (*Vhs*). Force vectors of both legs (*Fl* and *Fr*) and the CoM are shown for multiple snapshots over the transition (thin lines are sequential vectors over the transition). The point of maximal mechanical power is shown at the middle of double stance where the legs do positive and negative work simultaneously, even though the summed vector appears perpendicular to the CoM velocity vector (misleadingly implies zero work).

oscillation of the head is thought to have an impact on the wholebody locomotion of the animal, as the head typically makes up about 10% body mass.

Additional constraints are modeled such that the load is driven with a continuous periodic motion—the consequence of which is an average actuator force equivalent to the load's weight. With these constraints, the actuator is prohibited from merely performing Groucho patterns (constant vertical force to the CoM) like the single actuator design described previously. This is because the constant reaction force required to bear CoM weight would result in an equal and opposite force accelerating the coupled load in the downward direction for the duration of the step, making a periodic pattern infeasible. Instead, the actuator force must provide equal amounts of positive and negative work to maintain steady state kinematics of the load. Since an average upward force is required for the actuator to maintain full support of the load's weight, a constant loading effect is felt (in the downward direction) at the CoM of the walker, in addition to the dynamic oscillation force.

distributed over all three actuators. It should be noted that the vertical range of the load's trajectory in pane (A) is scaled down for illustrative purposes.

The equations of motion of the previous model are expanded to include the forces imparted by the coupled oscillator mechanism.

$$m\_l \ddot{\mathbf{x}}\_{\mathcal{C}} = \sum\_{\{l,r\}} F\_l \left( \frac{\mathbf{x}\_{\mathcal{C}} - \mathbf{x}\_{\hat{fl}}}{L\_l} \right) \tag{16}$$

$$m\_{\mathcal{C}}\ddot{\mathcal{Y}}\_{\mathcal{C}} = \sum\_{\{l,r\}} F\_l \left(\frac{\mathcal{Y}\_{\mathcal{C}}}{L\_l}\right) - F\_m - m\_{\mathcal{C}}\mathbf{g} \tag{17}$$

where *mt* is the total system mass (*mc* = 0.8*mt* and *mL* = 0.2*mt*) and *Fm* is the optimized force of the coupled oscillator actuator. Equations describing motion for the added point mass, *mL*, are shown below.

$$m\_L \ddot{\mathbf{x}}\_L = m\_L \ddot{\mathbf{x}}\_{\mathcal{C}} \tag{18}$$

$$m\_L \ddot{\mathbf{y}}\_L = F\_m - m\_L \mathbf{g} \tag{19}$$

Furthermore, leg length and mechanical power of the leg actuators (as well as the maximum leg length constraint) are implemented per equations [12-15].

First we consider the optimal solution for the model described with no actuator cost (i.e., work done by the coupled oscillator actuator imparts no cost influence on the optimal solution, however work done by the leg actuators is considered) (**Figure 8A,C**). In this case, the GRF shows a prominent single hump, as opposed to the more typical double-humped profile observed in the model without the coupled oscillator. Essentially, the legs provide isometric, weight-bearing forces (body plus average loading of coupled oscillator) during the stance phase of the gait while the third actuator takes over forces that facilitate mechanical work done to redirect the CoM near the step transition. The summation of the leg actuator and the coupled oscillator force profiles replicates the summed forces of the familiar double-humped pattern, which is responsible for bearing body weight and oscillating the body (inertial force) (**Figure 8C**). In many ways the solution is unsurprising given that the double-humped profile is already known to be an optimal pattern. The only difference is that the optimization spontaneously seizes on a strategy that delegates the energetically expensive work-based portion of the force profile to the actuator (since there is no cost penalty to do so) and the legs maintain the inverted pendulum portion of stance since these forces are largely isometric (i.e., constant leg length with zero work done).

## 3.2.3. Including Actuator Cost

The coupled oscillator model described above requires essentially no work of the telescopic leg actuators. As such, it is a passive gait, from the perspective of the biped since the leg actuators are used mostly as rigid struts. However, it is useful to consider whether there is any utility in the coupled oscillator strategy beyond the supplementation of free mechanical work available via the coupled oscillator actuator. Therefore, the same model is used to consider an optimal solution that seeks to minimize actuator work in the coupled oscillator as well as work done by the legs. Additionally, a force rate penalty is utilized for all three actuators to avoid unrealistic impulsive forces. The equation for actuator work is listed below, and the resulting optimal solution is shown in **Figure 8B,D**.

$$
\dot{W}\_m = F\_m \dot{d} \tag{20}
$$

$$
\dot{d} = \dot{\mathbf{y}}\_L - \dot{\mathbf{y}}\_\mathbf{c} \tag{21}
$$

The solution looks quite different from that which neglected the coupled oscillator actuator cost. Instead of the actuator providing dramatic sweeping impulses to the load/CoM system, it acts like a rigid strut. The force oscillations observable in **Figure 8D** facilitate a kinematic trajectory that changes in tandem with the body point mass. As a result, the displacement between the two point masses (*d*) is constant, and the relative velocity (˙ *d*) is zero. Thus, the actuator is not used to perform mechanical work (**Figure 8B**). Indeed, the cost of this solution is the same as the model with no coupled oscillator (**Table 1**). Ultimately, this result suggests that the coupled oscillator actuator cannot reduce the cost of the overall system, even though it has already been shown capable of reducing leg work. In order to understand why this mechanism cannot reduce the cost overall, the apparent cost of the actuator was manipulated. Specifically, a weighting coefficient, *Cm* was introduced in order to discount the cost of the coupled oscillator actuator's mechanical power in the objective function during optimization.

TABLE 1 | Cost summary for models. The non-dimensional work is shown for all relevant actuators (legs, actuator at the center of mass and total). Work is indicated with not applicable ("na") if the particular model does not include such an actuator.


*\*Cost of actuator work is not considered for this optimal solution.*

*†Cost of actuator work is not considered for this optimal solution but actuator constraints on stroke, force capacity and voltage supply are implemented.*

$$\mathbf{C}\_{m}\dot{W}\_{m} = \mathbf{C}\_{m}\mathbf{F}\_{m}\dot{\mathbf{d}}\tag{22}$$

$$\mathbf{0} < \mathbf{C}\_{m} < 1$$

By implementing a weighting coefficient, the energetic benefit of the actuator's force oscillations is less obscured by its diminished cost, allowing suboptimal solutions to be evaluated. **Figure 9** shows the full work (i.e., no discount) done by the actuator, as well as the leg work and total work of the legs plus the actuator over a range of weighting coefficients. Force profiles for optimization solutions are also shown (same format as in **Figure 8C,D**) for the following weighting coefficients: *Cm* = 0.05, 0.35, 0.65 *and* 0.95. As expected, the force profiles are very similar to the case of no actuator work considered when *Cm* = 0.05. However, the force oscillations become less pronounced at higher *Cm* values, until they begin to converge on a rigid strut solution when *Cm* = 0.95 (**Figure 9**).

At very high discounts (*Cm* = 0.05) the full actuator work increases drastically, although leg work is greatly reduced. At low discounts (*Cm* = 0.95) and even moderate discounts, total work plateaus to the cost of the model with no coupled oscillator, while actuator work diminishes and the legs take up more and more of the cost. Essentially, the energetic advantage that the coupled oscillator actuator provides to the legs is overshadowed by its full cost, and as a result, the optimal solution uses the actuator as a rigid strut (no work), unless its cost is artificially discounted.

It is perhaps surprising that the addition of a coupled oscillator actuator cannot improve upon the energetics of a bipedal mechanism without it. Indeed, the step-to-step transition is costly in part because the orientation of the legs during double stance means that both positive and negative work must be done simultaneously on the body in order to redirect the CoM trajectory (Donelan et al., 2002a, Donelan et al., 2002b). The nonvertical orientation of the legs (in contrast to the vertical actuator) means that a larger force magnitude—and consequently, more work—is required to alter the body trajectory from downward to upward.

Although it is unclear exactly why work of the actuator is more expensive than the work it saves the legs, there are a few identifiable factors that contribute to its cost. First, in order to offload the legs during their high mechanical power at double stance, the load must be accelerated downward to incite a positive reaction on the body, and this incurs a cost. Next, this action must be paid back with positive acceleration in order to maintain a positive/negative net work balance (this is required to have a steady state, repeatable pattern). Ideally, the positive acceleration (negative reaction force) can be supported by the legs with isometric force during single stance (i.e., no extra energetic cost to the legs), however actuator work is still required to brake the load from its acceleration and then lift it up against gravity. The consequence of these factors and their interactions is such that any use of the actuator (beyond isometric force) costs more than it saves.

Ultimately these results indicate that the economy of a walking machine would not benefit from the implementation of a coupled oscillator mechanism as described. Still, the concept may retain its utility in a system where reducing leg work (rather than work overall) is desirable.

coefficients (0 < *Cm* < 1). This reduces the apparent cost of the actuator and thus, alters the optimization solution. Actuator work contributes most of the work for a low coefficient and almost no work for a high coefficient. The opposite is true of the legs. Ground reaction force profiles are shown for four different weighting coefficients (solid grey lines are single leg forces, dashed grey lines are actuator reaction forces and solid black lines are a summation of both). The total work of the model approaches the work done by the model with no coupled oscillator actuator at higher coefficients.

## 3.2.4. Applying Realistic Actuator Constraints

One way to translate the coupled oscillator model into a realworld context is to imagine a human walking with such a mechanism mounted to a body harness. Although this design concept would not benefit the energetics of the whole system (person plus machine), it could still prove a useful strategy for reducing leg work and mechanical power required by the person to walk.

In this example, two linear shaft motors (model: S320T, Nippon Pulse America Inc., Radford, Virginia) are used. The two motors are controlled to act in unison and with a parallel configuration (one mounted anterior to the torso and the other mounted posterior to the torso). The summed effect of the two motors embodies the theoretical actuator allowing known loads with vertical oscillations to apply impulses to the CoM (front and back actuators are used to minimize pitch moments since the harness can only be mounted at the surface of the torso, a small moment arm distance from the true CoM). Similar to the model, reaction forces of the permanent magnets (mounted to the frames) are felt by the user's body through the attaching harness. It is hypothesized that an individual will choose motor patterns based on the principle of energy minimization, in which, the optimal work-based solutions discovered by the optimal control problem reflect the coupled oscillator interaction chosen. Although current literature suggests that humans sometimes adapt gait patterns to accommodate elastic load oscillations to reduce metabolic exertion (Rome et al., 2005, 2006; Castillo et al., 2014; Ackerman and Seipel, 2014), more evidence is needed to show that humans can employ energy minimization strategies consistent with the interactions proposed by the forced coupled oscillator mechanism described. Still, realistic system constraints and considerations can be implemented for the applied problem.

In order to consider the dynamics of the actuators in this applied system, the variable *Fm* is updated.

$$F\_m = K\_F i\_a - c\_d \dot{d} \tag{23}$$

where *ia* is the armature current, *KF* is the motor force constant that relates current and force and *cd* is the damping coefficient that characterizes viscous damping of the motor.

Three additional constraints are implemented to simulate a more realistic system: (1) motor/load kinematic oscillation range is limited by stroke; (2) maximum force capacity is limited by the motors; (3) maximum voltage is limited by a direct current power supply (model: PS16L80, Advanced Motion Controls, Camarillo, California). These constraints are described mathematically.

$$-\frac{S}{2} \le d \le \frac{\xi}{2} \tag{24}$$

$$-F\_{m,max} \le F\_m \le F\_{m,max} \tag{25}$$

$$-V\_{\rm PS,rms} \le V \le V\_{\rm PS,rms} \tag{26}$$

where *d* is the displacement of the load relative to the body point mass (*d* = *yL − yc*), *S* is the motor stroke, *Fm*,*max* is the maximum acceleration force, *VPS*,*rms* is the root-mean-square voltage available from the power supply and*V* is the total voltage draw, determined from Kirchhoff 's Voltage Law.

$$V = V\_{R\bar{t}} + V\_{em\bar{f}} + V\_{ind}$$

where *VRi* is the voltage at the armature resistance, *Vemf* is the voltage due to back electromotive force (emf) and *Vind* is the voltage due to inductance. By assuming that force is proportional to current and noting Ohm's Law, we derive:

$$V = \frac{R\_d}{K\_F} F\_m + K\_{emf} \dot{d} + \frac{L\_{ind}}{K\_F} \dot{F}\_m \tag{27}$$

where *Ra* is the armature resistance, *Kemf* is the motor back emf constant, and *Lind* is inductance. Note that a motor controller is chosen specifically for this system (model: DMC4123, Galil Motion Control, Inc., Rocklin, California) with sinusoidal amplifiers (D3520), however actuation performance is not further limited since constraints of the other equipment are more restrictive.

When the actuator dynamics and constraints are implemented, the optimization converges on a solution that utilizes a positive pulse of motor reaction force applied to the body (negative force on the load) near the middle of double stance (where maximal leg power is produced; **Figure 7D**, **Figure 10B,C**). Essentially, this allows for redirection of the CoM while the load is effectively weightless (i.e., *Fm ≈* 0), from the perspective of the legs. However, this offloading must be paid back in order to maintain a steady state pattern and so a negative reaction soon follows. The sequencing is beneficial overall since the positive pulse helps to offload the legs during a time of high mechanical power output (near the middle of double stance) and the negative pulse hinders the legs during a time of diminished mechanical power (closer toward single stance). It should also be noted that much of the negative pulse is provided by damping force (and some armature current) since load velocity peaks shortly after the positive pulse (~90° phase delay; **Figure 10C**).

During single stance, the total reaction force of the motor is near the weight of the load, due mostly to the armature current (although some damping force is present). The effect of this force during single stance does not contribute much to the cost of the solution since mechanical power is largely zero due to the constant leg length (inverted pendulum strategy). However, leg force decreases slightly over stance in an asymmetrical pattern as it provides isometric weight-bearing force that is offloaded slightly by

FIGURE 10 | Inverted pendulum model with telescopic leg actuators and a coupled oscillator at the body center of mass. (A) Point mass trajectories of body (*mc*) and load (*mL*) are shown for the optimal solution where the actuator cost is not considered. However realistic actuator dynamics (damping) and constraints (e.g. stroke, motor force capacity, peak voltage available from power supply) are implemented. (B) Vertical ground reaction forces are shown for individual legs, the total actuator reaction force on the center of mass (damping plus force due to armature current) and the summation of forces. Note that the grey dashed line is the same as in (C) where both terms of actuator reaction force are shown (damping and force due to armature current). Note, the actuator reaction force and its components are scaled by the bracket and asterisk indicated at the bottom right of pane (B). (D) Armature voltage normalized to maximum voltage available from the power supply is shown since this is the only restricting actuator constraint affecting the solution.

an increasing damping force (**Figure 10B,C**). This damping occurs due to the body CoM slowing its vertical motion relative to the load as it rises to mid-stance and then begins to fall away from the load. This pattern of reaction force continues into the beginning of push-off and helps to unload the legs slightly during this time. Eventually the positive pulse of reaction force occurs again at the next step and the cycle repeats.

Given that the actuator system provides beneficial offloading to the legs at a time when total motor voltage is not nearly saturated (~67% *VPS*,*rms*; **Figure 10D**), it is fair to question why higher force magnitudes are not used. However, the positive pulse must be paid back with negative reaction force (positive force on the load) and the maximum voltage becomes saturated at forces just beyond the weight of the load (**Figure 10D**), leaving little room for additional oscillation. In fact, the maximum force allowed by the system can be calculated as follows (assuming ˙ *d* = 0, *F*˙*<sup>m</sup>* = 0):

$$V\_m \left( V = V\_{PS,rms} \right) = \frac{K\_F}{R\_a} V\_{PS,rms} \tag{28}$$

With the parameters of the system selected, maximum force production is limited to approximately 115% the weight of the load. This limitation comes from the voltage available from the power supply rather than the force capacity of the motors themselves. In fact, the motor force itself only ever approaches about 33% of the motor force capacity, and as such, this constraint does not limit the solution. Likewise, the maximum stroke range used in the solution is around 22% of that available, and so this constraint also does not limit the solution.

Given that the actuator system is heavily restricted in its ability to pay positive reaction forces back with negative forces beyond the weight of the load, it must rely heavily on the damping force that dominates immediately following the positive pulse. As well, the motor can provide some additional negative force beyond the voltage limitation at this time since the back emf voltage reduces the overall voltage draw.

The strategy just outlined reduces the leg work accumulated over a step, even with the limitations of the actuator constraints. However, the overall system expends more work in total, since the actuator strategy is more expensive than the savings it provides to the legs (**Table 1**). Still, if the design goal of such a device is to offload the leg work done by a human wearing an exoskeleton, then the solution presents this potential.

## 4. Other Considerations

## 4.1. Leg Swing Dynamics

The reader may have noticed that the complication of leg swing dynamics has not played a formative role in the development of walking models discussed here. Although this is an important aspect of locomotion that ultimately cannot be ignored, we have chosen to focus on the underlying mechanisms that have a dominant influence on the energetics of whole-body trajectory management (Donelan et al., 2002a; Kuo et al., 2005). There is some evidence that swinging the leg consumes approximately 10–33% of metabolic expenditure during bipedal walking (Doke et al., 2005; Gottschall and Kram, 2005; Umberger, 2010), however the dynamics of a pendular leg (or more specifically, a double pendulum) can likely be facilitated with mostly passive dynamics.

For example, a slightly more complex and more thoroughly actuated model replicating human gait (Hasaneini et al., 2013) spontaneously employs a bang-coast-bang strategy to power leg swing in walking. Specifically, a quick burst impulse is used to accelerate the leg forward (first bang), then the leg swings with mostly passive dynamics (coast) and another quick burst impulse is used to decelerate the leg before the next touchdown (second bang). It has previously been recognized that similar activation patterns govern natural leg swing in humans (Mochon and McMahon, 1980; Doke et al., 2005; Doke and Kuo, 2007). Furthermore, the bang-coastbang strategy has generally been demonstrated an optimal mode of movement control when initial and final conditions require a similar state (e.g., initial velocity equals final velocity) (Srinivasan and Ruina, 2006; Srinivasan, 2011).

The mechanical cost of a bang-coast-bang leg swing is proportional to the leg's rotational velocity squared, given that the impulse must do work to impart kinetic energy (*W* = <sup>1</sup> <sup>2</sup> *<sup>I</sup>ω*<sup>2</sup> ) for a desired travel of the leg over the duration of swing (Srinivasan, 2011). Ultimately, the rotational velocity of the leg is related to the stride length that the foot must sweep through and the time duration of the swing. Assuming that double stance is relatively short, it then follows that the time duration of swing is approximately equal to step frequency. Thus, step length and step frequency should play an important role in determining the energetic cost of leg swing. Walking is associated with relatively low speeds (i.e., low step frequency and step length), and so it is predicted that the leg swing cost should also be low, as compared to other gaits such as running. In addition, step frequency and step length were constrained to the same values in all models (see Eq. [7]), and as such, there is likely a general increase of the cost surface for solutions presented here. However, the unaccounted cost of leg swing should not change the optimal solutions presented, since a global shift in the cost surface does not change its shape nor the location of the minimum.

It should be noted that this speculation assumes a decoupling between the leg dynamics and the rest of the body. However, it is easy to imagine that oscillations from a coupled oscillator mechanism, for example, may have an influence on the passive nature of the double pendulum leg, and thus, a more complicated energetic interaction. More detailed and thorough models should be developed to answer such questions about the energetics of leg swing and determining interactions.

## 4.2. Mechanical Work, Metabolic Energy and Electrical Power Consumption

All of the models presented here utilize a mechanical work-based cost for optimization. Although it is ultimately the metabolic energy that most likely influences motor control choices regarding movement patterns in humans, a work-based cost was chosen instead. For one, work is easily quantifiable as a mechanical variable, whereas metabolic energy requires the consideration of a more complicated physiological interaction. For example, the metabolic energy associated with isometric contraction (no work) is costlier for force generation than it is for force maintenance (Russ et al., 2002).

A simple approximation of the metabolic energy associated with work done by the muscles is determined by considering the differential efficiency of muscle contraction (25% for concentric contraction and −120% for eccentric contraction). However, given that only steady state gaits were considered by the optimizations, equal amounts of positive and negative work must be done over a step. Thus, the differential conversion from work to metabolic energy should not change any of the optimal results, other than the overall value associated with cost.

Also, since the models are meant to represent theoretical walking mechanisms that can be thought of as either robots or simple abstractions of human bipeds, it is unclear that metabolic cost is even the most appropriate cost to consider. Given that different actuators consume energy in different ways, it seems appropriate to consider mechanical work, since it is a physical requirement that all actuators must consume at least this energy (biological or artificial). An electromagnetic shaft motor was considered for implementation in the coupled oscillator mechanism, and as such, electrical power could have been used for the optimization. However, this cost scales somewhat differently from simple mechanical work, and so this changes the cost scaling comparisons of the leg actuators relative to the oscillator motor. Consequently, mechanical work was used as a more generally comparable energetic cost.

## 5. Cost Results Summary

In this contribution, we have outlined multiple reductionist walking mechanisms. Although each model is limited by the inherent physics of its individual makeup, they all test the employment of strategies reflecting one or more principles important to efficient bipedal locomotion. Although the single actuator Groucho design allows for zero work to be done over a step, this mechanism represents a trivial solution, which is already epitomized by wheeled mechanisms, and these systems have their own considerations less relevant to truly legged machines (e.g., typically requires some form of infrastructure, such as a road, since the effective radius is invariant). The horizontal actuator inverted pendulum model utilizes a bang-coast-bang approach in order to ensure continuous periodic motion of the CoM, however this model imparts a large cost on the actuator, since it must provide impulses to slow the CoM to a full stop and reaccelerate up to speed with every step. The sequencing of positive and negative work is restricted to operate suboptimally (effective heel strike before push-off) simply because a resting motion is necessary at the stepto-step transition. The energetic cost of this model is unnecessarily excessive relative to more economic designs discussed thereafter (*cost ∼*= 22.05*x*10*−*2; **Table 1**).

The inverted pendulum with telescopic legs represents a model that can replicate dynamics more similar to human walking. The total cost of the leg actuators is approximately 3.5 times less than the fixed-horizontal actuator model (*cost ∼*= 6.25*x*10*−*2; **Table 1**), even though it has twice the number of actuators. This result is largely due to the extra degree of freedom given to the CoM so it can deviate from a constant radius profile. This is important because it allows for a continuous periodic gait pattern that maintains momentum (minimizes leg work) at the step-to-step transition rather than bringing the system to rest with every step. Still, the orientation of the legs at this transition (non-vertical) also exists as a limitation to what is possible for energy minimization, since positive and negative work of each leg must be done simultaneously, and this is somewhat wasteful.

The coupled oscillator mechanism is used to take advantage of inverted pendulum motion during stance and vertical actuation at the step-to-step transition. When the cost of the coupled oscillator actuator is not considered, it completely takes over the expensive portion of the gait required for redirecting the CoM motion from downward to upward, and only uses the legs to bear isometric loads with mostly zero leg deflection during single stance (almost no work in this portion). The cost of the legs is essentially null in this model however the work done by the coupled oscillator actuator is prohibitive (*cost ∼*= 301.25*x*10*−*2; **Table 1**).

When the cost of the coupled oscillator actuator is considered, the optimization converges on a strategy that uses the actuator for isometric force production only. This is the dynamic equivalent of returning the load mass back to the CoM, and exists essentially as a null result. The optimal pattern exists as it does because the cost of using the actuator to perform work is costlier than not using it at all. The resulting cost is equivalent to the model with no coupled oscillator actuator (*cost ∼*= 6.25*x*10*−*2; **Table 1**).

Finally, the coupled oscillator model is optimized with no actuator cost, but with more realistic system dynamics and constraints deemed potentially restricting from data sheets of commercially available equipment. The resulting optimal pattern utilizes impulsive forces to reduce the weight of the load at costly double stance. It also takes advantage of damping forces to help oppose the relative acceleration of the load over the duration of single stance. This results in a ground reaction force, which is somewhat asymmetrical. The overall cost of the model is approximately 18.9% higher than with no coupled oscillator (*cost ∼*= 7.43*x*10*−*2; **Table 1**), however leg work is still reduced by about 16.6% (*leg work ∼*= 5.21*x*10*−*<sup>2</sup> ; **Table 1**).

## 6. Models and Their Solutions in Context

We began this contribution by recognizing an alternate definition for the fundamental task of locomotion as the optimal dynamic interaction between the system mass and the external environment as mediated by mechanisms available to the organism (**Figure 1**). Most exoskeleton designs tend to focus on principles directed at specific mechanisms of gait. For example, a variety of active ankle exoskeletons have been developed in recent years with the strategy of providing mechanical power directly at the ankle joint during push-off and have achieved successful reductions in metabolic consumption ranging from 6–24% the cost of unassisted walking (Sawicki and Ferris, 2008; Malcolm et al., 2013, 2014; Zhang et al., 2017). Although this approach clearly has potential for success when the mechanism of focus is well understood in the context of its role in whole-body energetics, a different approach is to consider strategies that influence the interaction between the organism and its environment more directly.

Indeed, we began this discussion by entertaining the notion that the leg actuators in Collins et al.'s variation on the passive dynamic walker (Collins et al., 2005) could mostly be replaced with a single actuator at the CoM. The dialogue that followed eventually culminated in the coupled oscillator exoskeleton as a more elaborate manifestation of this approach to control an optimal interaction at the body more directly. Even though the resulting optimal strategy turned out to be similar (apply impulsive forces to the body near push-off), we have shown that this type of actuation does not necessarily need to be applied at the ankle joint, at least in theory. This is an important insight given that carrying loads (e.g., actuator, transmission, battery, etc.) at the foot can result in a cost increase 4.4 times greater than carrying the same load at the waist and 1.7 times greater at the shank or thigh (Browning et al., 2007). Furthermore, the coupled oscillator strategy does not seek to minimize loading (as an ankle exoskeleton might), but rather *requires* some loading to operate. As such, the weight of the actuator, transmission, battery, etc. actually helps to generate the reaction forces that benefit leg work. In fact, increased loading could potentially minimize the necessary stroke required, assuming that voltage constraints are improved over the power supply currently suggested in the model. Of course, empirical studies are still needed to verify the theoretical potential of a coupled oscillator exoskeleton in practice.

Overall, we view the control optimization models discussed here as a direct exploration of how the interaction (system mass and external environment) can be optimized and to what extent. Although focus is directed at the optimal interaction and not at the mechanism, it is impossible to facilitate the interaction in the absence of a mechanism. As such, we rely on reductionist abstractions of real mechanisms. For example, biological legs with sophisticated musculature and joint spaces are collapsed into simple telescopic actuators that can actively extend. Electrical windings and ferrous shafts mounted to body harnesses are replaced with an extensive actuator driving a point mass load. Although some may view these simplifications as inaccurate depictions that do a disservice to complex systems in real life, the reductionist nature of such mechanisms allows for clearer interpretation of what makes an interaction optimal in the first place (i.e., less moving parts).

## References


This is not to say that the details of a mechanism are not important. To the contrary, appropriate tuning of mechanisms (e.g., spring stiffness), for example, can greatly affect the performance of an exoskeleton (Sawicki and Khan, 2016). However, the design process of such devices is well-served by a prior understanding of its effect on energetic exertion at the whole-body level (assuming this is the goal), before focusing on such details as tuning. This is arguably validated by the fact that ankle exoskeletons have likely benefitted from the prior understanding of the importance of pushoff on the energetics of human walking.

To some degree, the practice of reductionist actuation modelling may be interpreted as an arbitrary thought experiment. However, we maintain that each variation of the bipedal walker is a new opportunity to gather insight on the fundamental barriers to efficient actuation in locomotion. The results of such practice if interpreted carefully—can lead to important advances in the perspective that roboticists and biologists hold on the science of animal and machine locomotion.

## Author Contributions

JB conceptualized the manuscript and performed literature review. RS developed mathematical derivations of the models and performed all optimization analyses in MATLAB. Both RS and JB contributed to the composition and editing of the manuscript.

## Funding

This work was supported by Natural Sciences and Engineering Research Council (NSERC) of Canada Discovery Grant (312117- 2012) and University of Calgary Eyes High Doctoral Recruitment Scholarship (EHDRS).

## Acknowledgments

The authors thank Delyle Polet for his advice and conversations regarding the optimization models presented in this publication.

*Concepts and applications (J.E.A. Bertram)*. Hoboken, USA: Wiley-Blackwell, 1–25.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2018 Schroeder and Bertram. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# **Modeling and Control of Adjustable Articulated Parallel Compliant Actuation Arrangements in Articulated Robots**

## *Wesley Roozing\**

*Department of Advanced Robotics, (Fondazione) Istituto Italiano di Tecnologia, Genova, Italy*

Considerable advances in robotic actuation technology have been made in recent years. Particularly the use of compliance has increased, both as series elastic elements as well as in parallel to the main actuation drives. This work focuses on the model formulation and control of compliant actuation structures including multiple branches and multiarticulation, and significantly contributes by proposing an elegant modular formulation that describes the energy exchange between the compliant elements and articulated multibody robot dynamics using the concept of power flows, and a single matrix that describes the entire actuation topology. Using this formulation, a novel gradient descent based control law is derived for torque control of compliant actuation structures with adjustable pretension, with proven convexity for arbitrary actuation topologies. Extensions toward handling unidirectionality of elastic elements and joint motion compensation are also presented. A simulation study is performed on a 3-DoF leg model, where series-elastic main drives are augmented by parallel elastic tendons with adjustable pretension. Two actuation topologies are considered, one of which includes a biarticulated tendon. The data demonstrate the effectiveness of the proposed modeling and control methods. Furthermore, it is shown the biarticulated topology provides significant benefits over the monoarticulated arrangement.

**Keywords: compliant joints, force/torque control, series-parallel elastic actuation, energy efficient actuation, articulated robots**

## **1. INTRODUCTION**

Recent years have seen a paradigm shift in the field of robotic actuation from stiff, mainly position controlled concepts to compliant actuators in force control. This increased focus on use of compliance has taken place by addition of elastic elements both in series with servo drives, and in parallel to the main actuation of robotic systems. Many of the proposed concepts take inspiration from biological systems, in both their topology as well as the capacity for energy storage and release during motion. In robotic systems, they provide significant further benefits such as improved force control performance and physical robustness against impacts.

Out of the concepts proposed in literature, compliance in series with the actuation drive, known as series elastic actuation (SEA) and pioneered by Pratt in the 1990s (Pratt and Williamson, 1995), has been the most widely adopted. SEAs have evolved to become the core component of nearly all articulated robots. Addition of compliant elements in parallel to the main actuation drives, known

#### *Edited by:*

*Monica A. Daley, Royal Veterinary College, United Kingdom*

#### *Reviewed by:*

*Daniel F. B. Haeufle, Hertie-Institut für klinische Hirnforschung (HIH), Germany Dongming Gan, Khalifa University, United Arab Emirates*

## *\*Correspondence:*

*Wesley Roozing wesley.roozing@iit.it*

#### *Specialty section:*

*This article was submitted to Bionics and Biomimetics, a section of the journal Frontiers in Robotics and AI*

*Received: 17 November 2017 Accepted: 15 January 2018 Published: 12 February 2018*

#### *Citation:*

*Roozing W (2018) Modeling and Control of Adjustable Articulated Parallel Compliant Actuation Arrangements in Articulated Robots. Front. Robot. AI 5:4. doi: 10.3389/frobt.2018.00004*

**51**

as parallel elastic actuation (PEA), has seen less adoption than SEA. However, their benefits have been repeatedly demonstrated, particularly in terms of energy efficiency: in actuator test bench setups (Mettin et al., 2010; Haeufle et al., 2012; Mathijssen et al., 2015, 2016; Plooij et al., 2016), hopping robots (Liu et al., 2015), bipedal walkers (Yang et al., 2008; Mazumdar et al., 2016), and humanoids (Shirata et al., 2007). Another field of application is that of prostheses, where parallel compliance has been utilized in prosthetic ankles (Au et al., 2009; Realmuto et al., 2015; Jimenez-Fabian et al., 2017) and knees (Rouse et al., 2013; Pfeifer et al., 2015), to reduce the motor torque required to produce the desired deflection-torque profiles.

A common challenge with parallel compliance is that during some stages of the motion the torque generated by the parallel element does not correspond well to the desired torque on the joint. The result of this is that the main actuation drive has to work against the parallel compliance in order to obtain the desired joint torque or motion. To address this, many works employ unidirectional elements (Au et al., 2009; Mettin et al., 2010; Realmuto et al., 2015; Mazumdar et al., 2016; Jimenez-Fabian et al., 2017), clutches/switches (Haeufle et al., 2012; Rouse et al., 2013; Liu et al., 2015; Plooij et al., 2016), secondary motors to change the pretension (Mathijssen et al., 2015, 2016; Roozing et al., 2015, 2016), or a combination of these concepts to engage and disengage the parallel elements at desired moments.

Many biological systems have been found to contain biarticulated muscle structures, where a single muscle spans multiple joints. The human body incorporates many biarticular muscles; for example, the rectus femoris and hamstrings, which span the hip and knee joints as an antagonistic pair, the biceps that spans the shoulder and elbow, and the gastrocnemius muscle, which spans the knee and ankle joints. In the field of biomechanics, biarticulated muscles have been identified to transfer mechanical power between joints (Schenau, 1989; van Soest et al., 1993; Prilutsky and Zatsiorsky, 1994), used, for example, to greatly increase jumping height.

Considering the benefits demonstrated in biological systems, several authors have sought to employ multiarticulated actuation in articulated robots. In such contexts, motor drives and elastic elements that drive the joints of a robotic system are sometimes referred to as (actuation) branches. In Klein and Lewis (2009), the transfer of mechanical power between joints was experimentally demonstrated in a leg that models all nine major muscle groups in the human lower limb in the saggital plane. In Iida et al. (2008) and Niiyama et al. (2007), biarticulation was used in walking and jumping, respectively. Salvucci et al. (2014) showed how biarticulation can improve the end-effector force ellipsoid. The recently introduced compliant bipedal walker (Loeffl et al., 2016) also included a biarticulated tendon spanning ankle and knee. Babič et al. (2009)showed the benefits of a biarticulated compliant tendon spanning the ankle and knee joints in terms of jumping height through optimized motions of—and experiments with—a jumping robot.

In Tsagarakis et al. (2014) and Roozing et al. (2015, 2016), a 1- DoF leg prototype was designed that combines a high power SEA main drive with a parallel compliant high efficiency energy storage branch with adjustable pretension using a secondary motor. Using a novel distributed controller that actively utilizes both branches, the authors experimentally verified the potential of both mechanism and controller, demonstrating a 65% reduction in electrical power consumption when compared to conventional SEA, while performing cyclic squatting motions. The concept, its design optimization and control methods were generalized to multi-DoF systems and biarticulated actuation configurations in Roozing et al. (2016). Simulation studies performed on a 2-DoF leg demonstrated significant improvements in electrical energy efficiency and reduction in peak torque and electrical power requirements, compared to SEA only, while performing elliptical trajectories with the hip in a squatting motion. A biarticulated actuation arrangement was shown to further improve energy efficiency, compared to an arrangement utilizing solely monoarticulated parallel compliance.

This article builds upon these existing concepts and focuses on the model formulation and control of compliant actuation arrangements including multiple branches and multiarticulation, and contributes by:


This article is structured as follows. Section 2 builds up the proposed model formulation, starting at single-joint, single-branch systems and expanding into multi-DoF, multiactuator systems with multiarticulation. Section 3 briefly discusses the design optimization method originally presented in Roozing et al. (2016), followed by the proposed control strategies and an illustrative example in Section 4. A simulation study to validate the proposed methods is presented in Section 5, followed by concluding remarks and suggestions for future work in Section 6.

## **2. COMPLIANT ACTUATION**

In general, the torque *τ* generated on a single joint with configuration *q* by a single compliant tendon can be written as

$$
\tau(q) = -k \, n \, (p + n \, q), \tag{1}
$$

where *k* denotes the linear tendon stiffness, *n* denotes the transmission ratio, and *p* denotes the pretension position, or the position where element is at rest length. The sign of *n* indicates the direction of *q* that increases the elongation of the tendon. The elongation ∆ of the element is thus given by

$$
\Delta = \mathfrak{p} + \mathfrak{n}\,q.\tag{2}
$$

In implementations of elastic elements with high energy storage, unidirectional elements are often used, such as those constructed of natural rubber, usable in elongation and not in compression. For those, the torque is thus dependent on the sign of ∆:

$$\tau(q) = \begin{cases} -k \, n \, \Delta & \Delta > 0 \\ 0 & \text{Otherwise.} \end{cases} \tag{3}$$

We will explicitly take this property into account in the synthesis of our control strategies in Section 4.

## **2.1. Adjustable Parallel Compliance**

While parallel compliance can provide many benefits, the parallel branches may not be continuously required, nor may their static properties be suitable for every task or configuration required of the robot. In these cases, adjustability is a desirable property of the parallel branches, that may be exploited to further increase the effectiveness of the system. In general, for the compliant arrangements considered here, three parameters may be considered for adjustment: pretension, transmission ratio(s), and engageability. Generally, stiffness of mechanical elastic elements cannot be adjusted directly; instead, adjustable transmission ratio is commonly utilized. **Figure 1** gives a graphical overview of the three types of adjustability. Adjustable pretension can in some sense be considered series-elastic actuation; however, in contexts where the stiffness value is relatively small and the compliance augments some main drive, this is commonly referred to as parallel compliance with adjustable pretension.

Each method has its respective benefits and drawbacks:


Due to the binary nature of clutch mechanisms, we shall focus on the other two, namely adjustable pretension and adjustable transmission ratio. We consider the impact of these methods of adjusting compliance properties on generated torque, by returning to (1). For adjustable pretension, we take the derivative w.r.t. *p*:

$$\frac{\delta\tau}{\delta p} = -k\,\eta,\tag{4}$$

which does not depend on *p*, showing the adjustment is linear, and is independent of *q*, i.e., changing *p* results in a constant offset of *τ* . For adjustable transmission ratio, we take the derivative w.r.t. the transmission ratio *n*:

$$\frac{\delta\tau}{\delta n} = -k\,p - 2\,k\,n\,q,\tag{5}$$

which is a function of *n*, hence the adjustment is not linear. It can be observed the first term results in a change in offset of *τ* for *p ̸*= 0, and the second term shows that the change of slope of *τ* (*q*) scales with 2 *n*. As noted before, for *n* = 0 *→ τ* = 0, allowing to effectively disengage the compliant element.

## **2.2. Multiarticulation**

In this section, we formulate multiarticulated compliant branches, that span any number of joints. Assuming an articulated robot with *N* joints, and a configuration vector given by **q** = [*q*1, *q*2, *. . .* , *qN*] *<sup>T</sup> ∈ Q* where the joint space *Q ⊂ ℜ<sup>N</sup>* , the deflection ∆ *∈ ℜ* of a single multiarticulated branch is given by

$$
\Delta = p + n\_1 q\_1 + n\_2 q\_2 + \dots + n\_N q\_N,\tag{6}
$$

where *n*<sup>1</sup> *. . . n<sup>N</sup> ∈ ℜ* denote the transmission ratios for each of the *N* joints, shown also in **Figure 2**. Again, the sign of each *n<sup>i</sup>* indicates the direction of the corresponding joint *q<sup>i</sup>* that increases the elongation of the tendon.

The torque *τ<sup>i</sup> ∈ ℜ* applied to the *i*th joint can then be written as

$$\tau\_i = -k \, n\_i \left( p + n\_1 \, q\_1 + n\_2 \, q\_2 + \dots + n\_N \, q\_N \right), \quad i = 1 \dots N,\tag{7}$$

where *k* denotes the stiffness of the branch. Contracting the transmission ratios into vector form, we can write the torque *<sup>τ</sup> ∈ ℜ<sup>N</sup>* applied to all *N* joints as

$$\boldsymbol{\tau} = -\mathbf{t}^T \boldsymbol{k} \left( \boldsymbol{p} + \mathbf{t} \, \mathbf{q} \right) \in \mathbb{R}^N,\tag{8}$$

where the row vector **<sup>t</sup>** = [*n*1*, <sup>n</sup>*2*, . . . , <sup>n</sup>N*] *∈ ℜ<sup>N</sup>* both maps the joint configurations to elastic element elongation, and maps the produced linear tendon force back to joint torques. The deflection is written using **t** as ∆ = *p* + **t q**, and the linear tendon force *f ∈ ℜ* is equal to *f* = *k* (*p* + **t q**).

In terms of adjustability of multiarticulated configurations, adjusting *p* affects the torque on all joints linearly:

$$\nabla\_{\mathfrak{p}} \boldsymbol{\tau} = -\mathbf{t}^T \boldsymbol{k} \in \mathfrak{R}^N,\tag{9}$$

whereas adjusting the transmission ratios **t** affects joints nonlinearly and is also dependent on **q**:

$$\nabla\_{\mathbf{t}} \boldsymbol{\tau} = -k(\mathbf{q}\,\mathbf{t})^T - kI(\boldsymbol{p} + \mathbf{t}\,\mathbf{q}) \in \mathbb{R}^{N \times N},\tag{10}$$

where *I* denotes the N *×* N identity matrix. It can be observed the first term arises from the change in elongation of the element due to the changed transmission ratio, and the second diagonal term arises from the change in conversion ratio from linear tendon force to torque on the joints.

## **2.3. Multiple Branches**

In this section, we expand the previous section to a unified formulation for multiple, possibly multiarticulated branches. Supposing we have *M* parallel elastic branches, we gather all their respective **<sup>t</sup>** vectors in an *actuation topology* matrix *<sup>T</sup> ∈ ℜ<sup>M</sup>×<sup>N</sup>* , that fully describes the actuation topology:

$$T = \begin{bmatrix} \mathbf{t}\_1 \\ \dots \\ \mathbf{t}\_M \end{bmatrix},\tag{11}$$

which gives rise to the vector of deflections: **<sup>∆</sup>** <sup>=</sup> **<sup>p</sup>** <sup>+</sup> *<sup>T</sup>* **<sup>q</sup>** *∈ ℜ<sup>M</sup>*, and correspondingly the total torque *<sup>τ</sup> ∈ ℜ<sup>N</sup>* on the robot exerted by the branches:

$$\boldsymbol{\tau} = -\boldsymbol{T}^T \boldsymbol{K} (\mathbf{p} + T\mathbf{q}),\tag{12}$$

where *<sup>K</sup> ∈ ℜ<sup>M</sup>×<sup>M</sup>* is the diagonal matrix of stiffness values. Note that throughout this article superscript [*·*] *T* denotes transpose, whereas *T* denotes the matrix. The vector of linear tendon forces **<sup>f</sup>** *∈ ℜ<sup>M</sup>* follows as **<sup>f</sup>** <sup>=</sup> *<sup>K</sup>* (**<sup>p</sup>** <sup>+</sup> *<sup>T</sup>* **<sup>q</sup>**). Similar to the single branch case, adjusting the pretensions **p** for *M* actuators yields:

$$\nabla\_{\mathbb{P}}\tau = -T^{T}K \in \mathfrak{R}^{N \times M}.\tag{13}$$

For adjustable transmission ratios, calculating *∇<sup>T</sup> τ* yields a 3D tensor, of which the components for the *m*th actuator are given by

$$\nabla\_{\mathbf{t}\_m} \tau = -k\_m \left( \mathbf{q} \,\mathbf{t}\_m \right)^T - k\_m I \left( p\_m + \mathbf{t}\_m \,\mathbf{q} \right) \in \mathfrak{R}^{N \times M}. \tag{14}$$

Considering each **t***<sup>m</sup>* is of dimension *N*, this means that up to *N M* variables are involved. Of course, usually *T* can be considered quite sparse since all tendons are not driving all joints.

In both cases, the gradient with respect to the joint configurations is:

$$\nabla\_{\mathbf{q}} \boldsymbol{\tau} = -\boldsymbol{T}^{\mathrm{T}} \boldsymbol{K} \boldsymbol{T} \in \; \mathfrak{R}^{\mathrm{N} \times \mathrm{N}}.\tag{15}$$

Stopping for a moment to consider the different dynamics of adjustable pretension and adjustable transmission of multiarticulated compliance, we find the latter arguably provides more freedom in shaping the provided torque than the former, due to changing the slope and the larger number of degrees of freedom (in multiarticulation). As aforementioned, this also adds the potential benefit of disengaging elements entirely from desired joints. However, significant drawbacks exist due to the nonlinear behavior on a potentially much larger configuration space, combined with increased complexity in realizing such structures. Therefore, at this point, we choose to focus on adjustable pretension in our modeling and control formulation.

We now proceed with a modular model formulation using energy exchange through the concept of power ports. Taking the time derivative of the deflections **∆**, we find the rate of change of the deflection of the elastic elements is given by

$$
\dot{\Delta} = \dot{\mathbf{p}} + T\dot{\mathbf{q}}.\tag{16}
$$

Given that the power flow into an elastic element is given by the force multiplied by rate of displacement (i.e., *P* = *f* ∆˙ ), we find from port-Hamiltonian theory that (**f***,* **<sup>∆</sup>**˙ ) *∈ ℜ<sup>N</sup>* and (*τ,* **q**˙) *∈ ℜ N* describe an *N*-dimensional power port that exchanges energy between the rigid body robot and compliant actuation branches driving it. This power flow is the sum of each of the power flows in/out of the individual elastic elements; indeed, power may flow between the elastic elements as well.

This concept is depicted graphically in **Figure 3**, using Bond graph notation. The first diagram shows the notation using **t** vectors, and the bottom diagram shows how the *T* matrix completely describes the power flow between actuators and robot. This formulation has several advantages for rapidly evaluating different actuation topologies; by simply modifying *T* the transmission ratios and actuation configuration of tendons can be quickly modified. It also enables modularity of the modeling and simulation procedures by separating actuator dynamics from the articulated multibody dynamics of the robot.

## **3. OPTIMIZATION OF DESIGN PARAMETERS**

In this section, we briefly discuss the optimization of design parameters presented in Roozing et al. (2016). Based on gravitational load and inertial properties, the compliance design parameters can be chosen to achieve desired compensation torque over the joint workspace, resulting in higher energy efficiency and reduction of peak torque/power requirements on the main joint actuators. The optimization procedure considers the transmission ratios contained in *T*, elastic element stiffness values contained in *K*, and pretension positions in **p** as optimization variables. We first define the error vector **e***<sup>d</sup>* for a leg configuration **q** as

$$\mathbf{e}\_d(\mathbf{q}, \varphi) = \boldsymbol{\tau}(\mathbf{q}, \varphi) + \boldsymbol{\zeta}(\mathbf{q}) \in \mathbb{R}^N,\tag{17}$$

where *τ*(**q**, *φ*) denotes the net tendon torques (12) acting on the leg DoFs, and *φ* contains the considered design parameters *T*, *K* and **p**. The function *ζ*(**q**) denotes the vector function of desired torques; here, we consider gravitational joint torque compensation, *ζ*(**q**) = **g**(**q**). For highly dynamic systems for which the desired dynamic behavior is known at design time, *ζ* can be chosen to include inertial, Coriolis, and damping components for

efficient execution of those motions. This is done by designing the dynamic behavior and then obtaining the required actuation torques during each phase of the task through inverse dynamics, thus obtaining *ζ* (**q**). It is also possible to optimize for multiple tasks simultaneously by obtaining *ζ* (**q**) as a weighted linear combination of multiple tasks. Here, however, we consider the desired motions not to be known at design time.

The total error *Ed*(*φ*) *∈ ℜ* is defined as the integrated *l* 2 -norm over a subset of the workspace:

$$E\_d(\varphi) = \int\_{\mathbf{q} \in \mathfrak{Q}\_d} ||\mathbf{e}\_d(\mathbf{q}, \varphi)||\_2 \, \mathrm{d}\mathbf{q},\tag{18}$$

where *Q<sup>d</sup> ⊂ Q* is a subset of the joint workspace on which to optimize and depends on the specific robot. For robots for which the desired trajectories are known *a priori*, *Q<sup>d</sup>* can be set to this trajectory in joint space. Using the *l* 2 -norm approximates minimizing the electrical power consumption directly, as the electrical power of a BLDC motor can be approximated (neglecting back-EMF and electrical dynamics) by its squared torque. An optimal solution *φopt* minimizes *Ed*:

$$\begin{aligned} \mathfrak{L}\mathfrak{\varphi}\_{\mathfrak{q}\mathfrak{p}t} &= \min\_{\mathfrak{q}} \ E\_d(\mathfrak{\varphi}) \\ \text{s.t.} &\underline{\mathfrak{\varphi}} \le \mathfrak{\varphi} \le \overline{\mathfrak{\varphi}}, \end{aligned} \tag{19}$$

where *φ* and *φ* denote the lower and upper bounds of *φ*, respectively. Note that this design optimization procedure includes **p** into the optimization as a parameter. As such, it attempts to optimize the design such that it provides the desired torques over the joint space as accurately as possible without pretension adjustment. The control strategies presented in the next section exploit the fact that pretension is adjustable, which can be used to further increase efficiency of such systems.

## **4. CONTROL STRATEGIES**

Various control strategies can be employed to effectively utilize adjustability of (parallel) compliance. In Roozing et al. (2015) and Roozing et al. (2016), inversion of the peractuator pretension–torque relations was utilized to obtain the pretension position references that lead to the desired torques. To handle coupling resulting from multiarticulation, the equations were solved in a cascaded manner. However, this method generalizes poorly for arbitrarily complex structures and requires a degree of designer intuition.

In the following sections, we propose two alternative methods to solve the torque control problem through adjustable pretension, employing the multi-DoF, multiactuator formulation of Section 2.3. The first relies on the (pseudo)inverse of the topology matrix *T*, which is a generalization of the previous method. We show this method suffers from limitations in certain situations, with regards to coupling and unidirectionality of elastic elements. The second relies on gradient descent, which allows to simultaneously take coupling and unidirectionality of the elastic elements as well as achievable pretension adjustment speeds into account.

## **4.1. (Pseudo)inverse**

Returning to the multi-DoF, multiactuator torque equation (12), we observe that it can be solved for **p**:

$$\mathbf{p}^\* = -\left(T^T\mathbf{K}\right)^{-1}\boldsymbol{\tau}^\* - T\mathbf{q},\tag{20}$$

where *τ*\* denotes the desired torque, and **p**\* denotes the resulting desired pretension positions, respectively. If *T* is not full rank, the pseudoinverse may be used in (20). This method is suitable for position controlled pretension as in Roozing et al. (2016), and is a multi-DoF generalization of the method presented in that work. However, using the (pseudo)inverse, it is not possible to take unidirectionality of elastic elements into account. Suitable preprocessing of the desired torque vector can resolve this issue in certain cases, however, this is not a general solution, hence this method is feasible only if the resulting **∆** *≥* 0 or if bidirectional elastic elements are used.

## **4.2. Gradient Descent**

To obtain a gradient descent based torque control law, we start with the torque control error **e**, defined as **e** = *τ <sup>∗</sup> <sup>−</sup> <sup>τ</sup> ∈ ℜ<sup>N</sup>* . Taking the gradients with respect to **p** and **q**, we obtain the rate of change of **e** as

$$
\dot{\mathbf{e}} = \left(\nabla\_\mathbf{P} \mathbf{e}\right) \dot{\mathbf{p}} + \left(\nabla\_\mathbf{q} \mathbf{e}\right) \dot{\mathbf{q}},\tag{21}
$$

where for now we have assumed the use of bidirectional elastic elements, or equivalently, **∆** *≥* **0**, i.e., no branches are in slack. Section 4.2.1 introduces an extension for when this assumption does not hold. Since we can assume that the desired joint torques do not depend on **p**, we have *∇***<sup>p</sup>** *τ <sup>∗</sup>* = **0**, and the above equation can be rewritten using the definition of **e** as

$$\dot{\mathbf{e}} = -\left(\nabla\_{\mathbf{p}}\boldsymbol{\tau}\right)\dot{\mathbf{p}} + \left(\nabla\_{\mathbf{q}}\boldsymbol{\tau}^\* - \nabla\_{\mathbf{q}}\boldsymbol{\tau}\right)\dot{\mathbf{q}},\tag{22}$$

where *∇***<sup>q</sup>** *τ ∗* depends on high-level controller and robot dynamics, and from Section 2.3 we recall:

$$
\nabla\_{\mathbf{p}} \boldsymbol{\tau} = -\boldsymbol{T}^T \boldsymbol{K},
$$

$$
\nabla\_{\mathbf{q}} \boldsymbol{\tau} = -\boldsymbol{T}^T \boldsymbol{K} \boldsymbol{T}.\tag{23}
$$

At this point, we introduce the squared *l*2-norm of **e** as our error measure. Using the results above, the chain rule, and *∇∥***e***∥* 2 <sup>2</sup> = 2 **e**, we compute the gradient with respect to **p**:

$$\begin{split} \|\nabla\_{\mathbf{p}} \|\mathbf{e}\|\_{2}^{2} &= \left(\nabla\_{\mathbf{p}} \mathbf{e}\right) \left\|\mathbf{e}\right\|\_{2}^{2} \\ &= 2 \left(T^{T}K\right)^{T} \left(\boldsymbol{\tau}^{\*} - \boldsymbol{\tau}\right) . \end{split} \tag{24}$$

Setting rate of change of **p** as **p**˙ = *−γ<sup>e</sup> ∇***p***∥***e***∥* 2 2 , where 0 *< γ <sup>e</sup> ≤* 1 is a suitable scaling constant, ensures asymptotic convergence of **e** given **q**˙ = **0**; Section 4.2.2 discusses the extension to **q**˙ *̸*= **0**. Furthermore, note that (24) does not depend on *∇***<sup>q</sup>** *τ ∗* , i.e., the controller is independent of the specific robot dynamics or its high-level controllers.

By taking the second-order gradient of the squared *l*2-norm of **e**, we show that it is globally convex, and thus **e** converges to the global minimum:

$$\begin{split} \nabla\_{\mathbf{p}}^{2} \|\mathbf{e}\|\_{2}^{2} &= \nabla\_{\mathbf{p}} \left[ 2 \left( T^{T} K \right)^{T} \left( \boldsymbol{\tau}^{\*} - \boldsymbol{\tau} \right) \right] \\ &= 2 \left( T^{T} K \right)^{T} \left( T^{T} K \right), \end{split} \tag{25}$$

which is positive definite as the quadratic form is always positive definite. This proves global asymptotic convergence of the error.

#### 4.2.1. Constraint

The previous section assumed that either bidirectional elastic elements were used, or equivalently, unidirectional elements for which the elongation **∆** *≥* **0**. This section adds a dynamic potential function of which we take the gradient, so that the control algorithm will never attempt to descend in directions that run the tendons into slack, and, conversely, avoids that tendons are run into slack due to joint motion.

To enforce unidirectionality constraints while maintaining continuity and global convexity, we add a quadratic constraint potential term *c*(**p**), given by:

$$c(\mathbf{p}) = -\gamma\_{\rm const} \parallel \Delta^{-}(\mathbf{p}) \parallel\_{2}^{2},\tag{26}$$

where **∆***<sup>−</sup>*(**p**) = min(**∆**(**p**), 0) is the element-wise minimum, i.e., the constraint is only active for branches that are currently in slack. *γconst ∈ ℜ* is a large scaling constant. By adding the constraint potential gradient, **p**˙ is given by

$$\dot{\mathbf{p}} = -\gamma\_\varepsilon \nabla\_\mathbf{p} ||\mathbf{e}||\_2^2 + \nabla\_\mathbf{p} \ c(\mathbf{p}),\tag{27}$$

where *∇***p***∥***e***∥* 2 2 is given by (24) and *∇***<sup>p</sup>** *c*(**p**) = *−*2 *γconst* **∆***−*(**p**). Similar to (25), the second-order gradient of *c*(**p**) results in a quadratic form which is globally convex. This constraint replaces the slack control component of the control strategy described in Roozing et al. (2016). Achievable values of **p** due to mechanical constraints can be similarly imposed in a convex manner.

## 4.2.2. Compensating for <sup>q</sup>˙ *̸*<sup>=</sup> <sup>0</sup>

To ensure the convergence of the error under non-zero joint motion, we extend the above gradient descent based control law with an additional term taking this motion into account. Given **q**˙, we solve **e**˙ = **0** for **p**˙ in (22):

$$\begin{aligned} 0 &= -\left(\nabla\_{\mathbf{p}}\tau\right)\dot{\mathbf{p}} + \left(\nabla\_{\mathbf{q}}\tau^\* - \nabla\_{\mathbf{q}}\tau\right)\dot{\mathbf{q}}\\ \left(\nabla\_{\mathbf{p}}\tau\right)\dot{\mathbf{p}} &= \left(\nabla\_{\mathbf{q}}\tau^\* - \nabla\_{\mathbf{q}}\tau\right)\dot{\mathbf{q}}\\ \dot{\mathbf{p}} &= \left(\nabla\_{\mathbf{p}}\tau\right)^{-1}\left(\nabla\_{\mathbf{q}}\tau^\* - \nabla\_{\mathbf{q}}\tau\right)\dot{\mathbf{q}}\\ &= -\left(T^T K\right)^{-1}\left(T^T K T + \nabla\_{\mathbf{q}}\tau^\*\right)\dot{\mathbf{q}}, \end{aligned} \tag{28}$$

which we will refer to as **p**˙ *dq*. This yields the rate of change of **p** needed to compensate for the change in **q**, and thus keep the error constant. The first term is equal to *−T* **q**˙, and simply ensures that **p** + *T* **q**, i.e., the elongation **∆**, remains constant. The second term is equal to *−*(*T <sup>T</sup> K*) *−*1 (*∇***<sup>q</sup>** *τ ∗* ) **q**˙ and compensates the change in desired torque due to *∇***<sup>q</sup>** *τ ∗ ̸*= **0**. Of course, this last term requires knowledge of how the desired torques will change as the joint configurations change and is generally not trivial to implement. Combining (28) with (27):

$$\dot{\mathbf{p}} = -\gamma\_\varepsilon \nabla\_\mathbf{p} ||\mathbf{e}||\_2^2 + \nabla\_\mathbf{p} c(\mathbf{p}) + \gamma\_{dq} \dot{\mathbf{p}}\_{dq},\tag{29}$$

we obtain the rate of change of **p** that results in global asymptotic convergence of **e**. The scaling constant 0 *≤ γdq ≤* 1 avoids excessive adjustment of the pretension to compensate the joint motion, which for high gear ratios may reduce energy efficiency, and is dependent on the mechanical implementation of the actuators.

#### 4.2.3. Computing the Adjustment Velocities

The rate of change of **p** given by (29) may not be achievable in practice due to speed limitations following from the mechanical implementation. Hence, **p**˙ is scaled as follows to obtain the reference adjustment velocity **p**˙ *∗* :

$$
\dot{\mathbf{p}}^\* = \alpha \,\dot{\mathbf{p}},\tag{30}
$$

where

$$\alpha = \begin{cases} \frac{p\_{\text{max}}}{\max(|\dot{\mathbf{p}}|)} & \max(|\dot{\mathbf{p}}|) > p\_{\text{vmax}} \\ 1 & \text{Otherwise} \end{cases},\tag{31}$$

and *pvmax* denotes the maximum achievable adjustment velocity. This ensures none of the branches are commanded beyond their speed limit, which would result in not descending the gradient of the error norm in the correct direction.

## **4.3. Rankedness of** *T*

The case of *T* not being full rank has one important consequence; the solution is redundant. An intuitive interpretation of this is the example of two antagonistic branches driving a single joint, in which increasing the tension of both in a certain proportion (given by their relative transmission ratio and stiffness values) does not result in a change of net torque. This is an example of a single rank deficiency of *T*, resulting in a line in the **p** configuration space providing identical joint torques. For more complex systems, *T* may be multiple rank deficient.

Since optimal energy efficiency is obtained by minimizing the tension throughout the system that loads the pretension mechanisms, a unique solution may be obtained in the null space of the obtained solution. In the following, we suggest two extensions toward this end.

#### 4.3.1. Pseudoinverse

When using the pseudoinverse based pretension control of Section 4.1, the following extension may be used, minimizing the squared *l*2-norm of the deflections **∆** in the null space of the solution of (20):

$$\min\_{\mathbf{x}} \|\Delta\|\_{2}^{2} = \|\mathbf{p}\_{psdo} + Z\mathbf{x} + T\mathbf{q}\|\_{2}^{2}$$

$$\text{s.t.}\\\Delta \ge \mathbf{0} \text{ and } \underline{\mathbf{p}} \le \mathbf{p}\_{psdo} + Z\mathbf{x} \le \overline{\mathbf{p}},\tag{32}$$

where *Z* = ker (*−T <sup>T</sup> K*), **p***psdo* denotes the pseudoinverse solution for **p**\* given by (20), and **p**, **p** denote the lower and upper bounds on **p**, respectively. Given a solution **x***opt* of (32), the new value for the desired pretension positions **p**\* is given by **p***psdo* + *Z* **x***opt*.

#### 4.3.2. Gradient Descent

For the gradient descent based solution of Section 4.2, one may add a gradient term *ctens* (**p**):

$$\varepsilon\_{\text{tens}}(\mathbf{p}) = -\gamma\_{\text{tens}} \|\Delta(\mathbf{p})\|\_2^2,\tag{33}$$

for which the gradient w.r.t. **p** is given by *∇***<sup>p</sup>** *ctens*(**p**) = *−*2 *γtens* **∆**(**p**). This gradient is then added to (29). For simplicity and illustration of the core ideas of this work however, we shall focus on systems with full rank *T* for the remaining sections.

## **4.4. An Illustrative Example**

To illustrate the core ideas behind the gradient descent based control law, we start with a simple example of a biarticulated robot with two joints and two adjustable compliant tendons in a static configuration (**q**˙ = 0). The actuation topology is described by

$$T = \begin{bmatrix} \mathbf{t}\_1 \\ \mathbf{t}\_2 \end{bmatrix} = \begin{bmatrix} -\mathbf{0}.1 & -\mathbf{0}.2 \\ \mathbf{0} & \mathbf{0}.3 \end{bmatrix},\tag{34}$$

i.e., the first tendon is biarticulated, and the second is monoarticulated. The first joint is driven only by the first tendon, and the second joint is driven by both tendons in an antagonistic manner. We assume the tendons to be unidirectional. The stiffness matrix *K* is given by *K* = diag(1000,1000), and the joint configuration **q** = [0,0]*<sup>T</sup>* . The reference torques are set to *<sup>τ</sup>*\* <sup>=</sup> [10, *<sup>−</sup>*30]*<sup>T</sup>* Nm in this example. Furthermore, we set the constraint parameter *γconst* = 10<sup>8</sup> and gradient descent parameter *<sup>γ</sup><sup>e</sup>* <sup>=</sup> <sup>5</sup> *<sup>×</sup>* <sup>10</sup>*<sup>−</sup>*<sup>6</sup> . Lastly, we assume a maximum adjustment velocity of *pvmax* = 0.05 m/s. The landscape of the squared *l*2-norm is shown in **Figure 4**, together with six example evolutions with varying initial conditions for **p**. They can be seen to all converge to the global minimum, indicated by the vertical dashed line.

The time evolutions of *∥***e***∥* 2 2 , *τ*, and **p** are shown in **Figure 5**. As the desired torques can be achieved with **∆** *≥* **0** and *T* is full rank, the error norm converges to zero for all evolutions. One can observe that while **p** takes relatively long to converge (bottom figures), this is beneficial: the error norm is very small after 5 s (top-left figure), and further adjustment of the pretension yields only small reduction of the error. Out of these six example evolutions, numbers 1–4 have initial conditions where at least one of the two branches is in slack. It can be seen that the constraint

**FIGURE 4** | Gradient descent: squared *l*2-norm of e. The superimposed red lines show example evolutions (see also time evolutions in **Figure 5**) of p and the corresponding squared *l*2-norm of the error. They can be seen to converge to the global minimum, indicated by the vertical dashed line.

described in Section 4.2.1 is effective, driving the branches out of slack at the maximum velocity. From the time evolutions of *p*<sup>1</sup> and *p*<sup>2</sup> (bottom figures), one may be tempted to think there is undesired overshoot in the pretension positions (e.g., evolution 1). However, this "overshoot" is desired, as due to the biarticular coupling between the joints, this reduces the torque error norm while the other pretension position converges.

## **5. SIMULATION STUDY**

In this section, we present a simulation study on the model of a planar 3-DoF leg prototype which was recently developed Roozing et al. (2018). The prototype follows the same design concepts as in Roozing et al. (2015) and Roozing et al. (2016), with all joints driven by torque-controlled series-elastic actuators, augmented with parallel compliant branches with adjustable pretension. The model includes viscous friction components at the joints, actuator dynamics with friction in the motor drives and drive trains, and elastic element internal damping. Furthermore, low-level torque control is implemented for the SEAs, velocity control is implemented for the parallel pretension motors, and voltage and current limits are imposed. For more details on their dynamics modeling, we refer the reader to Roozing et al. (2016). The design features three actuated degrees of freedom: ankle, knee and hip, and is semi-anthropomorphic, with similar mass and mass distribution to the human limb. The trunk link is loaded with an additional 20 kg, simulating the weight of a full humanoid in two-legged stance.

A diagram of the model is shown in **Figure 6**. In this case, two actuation topologies are considered; one that includes two monoarticulated parallel elastic branches on knee and ankle, and one where one of the two branches is biarticulated, spanning the

ankle and knee joints. The design parameters were optimized following the procedure outlined in Section 3, and the actuation topology matrices are given by

$$T\_{mono} = \begin{bmatrix} -0.07 & 0 & 0\\ 0 & 0.06 & 0\\ 0 & 0 & 0 \end{bmatrix},$$

$$T\_{bi} = \begin{bmatrix} -0.07 & -0.0352 & 0\\ 0 & 0.06 & 0\\ 0 & 0 & 0 \end{bmatrix},\tag{35}$$

and the stiffness matrices are given by *Kmono* = diag(5900,8600,0) and *Kbi* = diag(5900,8600,0), respectively. As evidenced by the zero columns in (35), the hip joint is not augmented with a parallel branch.

In this study, we first perform a number of point-to-point motions in Cartesian space with the hip of the robot, keeping the

torso upright. Each pose is maintained for 10 s to clearly illustrate the transient behavior of the proposed control strategy given the system's parameters. **Figure 7** shows the joint configuration references and tracking for the biarticulated configuration; the monoarticulated configuration is not shown for brevity, however, tracking is almost identical. The figure is augmented to show the leg poses at various time instances, showing the wide range of executed motions. Phases A–D and F denote the aforementioned static poses. The second part of the reference motion involves a cyclic Cartesian trajectory of the hip in an elliptical squatting motion, to demonstrate its behavior under highly dynamic motion. This part is denoted as phase E in **Figure 7**.

The robot is controlled with simple joint-level impedance controllers as high-level control strategy, providing the torque references for the gradient descent based controller of Section 4.2. As in Roozing et al. (2015) and Roozing et al. (2016), the SEAs are torque controlled to ensure the desired net torques are always achieved at the joints. We set the gradient descent parameter *<sup>γ</sup><sup>e</sup>* <sup>=</sup> <sup>1</sup> *<sup>×</sup>* <sup>10</sup>*<sup>−</sup>*<sup>6</sup> , the constraint parameter *γconst* = 10<sup>2</sup> , and *γdq* = 0.1. The maximum pretension adjustment velocity of this system is approximately 3 cm/s, imposed by the transmission ratio, chosen electric motors and supply voltage of 48 V.

The results are shown in **Figures 7** and **8**. The torque plots for the ankle (**Figures 8A,B**) and knee (**Figures 8C,D**) confirm that indeed the net torques *τ*<sup>1</sup> and *τ*<sup>2</sup> are nearly identical when comparing the mono- and biarticulated cases, showing that the SEAs can effectively ensure the desired net torque is achieved at the joints, and that the motions are comparable.

Considering on the torque provided by the parallel elastic tendons (red lines in **Figures 8A–D**) for both joints in both cases, they can be observed to converge to the net desired torque, causing the required SEA torque to converge to zero, unless the desired joint torque is not feasible given the tendon actuation topology. For example, negative ankle torques cannot be provided by the ankle tendon, causing the tendon torque to converge to zero and the SEA providing the full negative torque (e.g., phases C and D, where the center of pressure is behind the ankle joint and the ankle needs to provide negative torque). Furthermore, from the elastic element elongation shown in **Figures 8G,H**, it can be observed that the constraint (Section 4.2.1) effectively constrains the unidirectional tendons to zero elongation. These results show the gradient descent based control approach is effective at achieving torque control of the system using (multi-)articulated compliant arrangements.

During the cyclic motion part of the reference, the tendon torques are unable to converge to the reference torque entirely, as the pretension adjustment speed limits do not allow for it (and the load motion compensation parameter *γdq* = 0.1); however, their smaller adjustments combined with the optimized design do lead to a substantial reduction of the error, causing the SEAs to need to deliver only a fraction of the net joint torque. This in turn allows to design for small, light, efficient motors. In the monoarticulated knee case, the SEA is providing less than 10 Nm peak torque out of approx. 70 Nm required net peak torque. In the biarticulated case, the SEAs are providing less than 5 Nm on both the knee and ankle joints. In the monoarticulated ankle case, a smaller reduction in

torque requirements is observed; the dependence of ankle load on the configuration of both joints results in the monoarticulated tendon not providing a torque that matches well with the required torque, despite substantial pretension adjustment of the ankle tendon (**Figure 8E**).

Comparing the two actuation topologies, we observe that the biarticulated configuration is both able to provide the desired net joint torques more accurately, as well as needing significantly smaller pretension adjustments to achieve them. This conclusion is further strengthened by comparing the squared *l*2-norm of the error for both cases, shown in **Figures 8I,J**. We can therefore conclude that the biarticulated configuration is more suitable for the system under consideration.

## **6. CONCLUSION AND FUTURE WORK**

This work has developed a novel model formulation of compliant actuation structures for articulated robots, including multiple branches and multiarticulation. The modular formulation employs a single matrix to describe the entire actuation topology, and formulates the energy exchange between the compliant elements and articulated multibody robot dynamics using N-D power flows.

Using this formulation, we derived a novel gradient descent based control law for compliant actuation structures with adjustable pretension, with proven convexity for arbitrary actuation topologies. Unidirectional elastic elements were considered through the inclusion of a convex constraint into the formulation.

A simulation study on a 3-DoF leg model using two different actuation topologies demonstrated that the gradient descent

## **REFERENCES**


based control method is effective for torque control of the parallel tendons, leading to asymptotic convergence of the error. Additionally, the results illustrate that the chosen actuation topology and optimization of its design parameters are also fundamental for optimal performance.

We believe this control strategy is promising, and future work will include the application of this strategy to the 3-DoF hardware prototype, which is currently under development and will allow for rapid interchange of several actuation topologies, including those considered in this work. In terms of future work, the proposed formulation lends itself very well to the inclusion of energy expenditure; the magnitude of pretension adjustment can be considered in the context of energy consumed by the motors to do so. Furthermore, whereas in the presented simulation study serieselastic main drives were augmented with parallel elastic tendons, we believe effective systems can be designed that employ only such elastic tendons, in multiarticulated configurations, similar to the human anatomy. Lastly, extensions toward predictive control in an energy efficiency context are promising.

## **AUTHOR CONTRIBUTIONS**

This work was fully performed by WR.

## **FUNDING**

This work was supported by European Commission projects WALK-MAN (611832), CENTAURO (644839), and CogIMon (644727).

mutilated gears. *IEEE/ASME Trans. Mechatron.* 20, 594–602. doi:10.1109/ TMECH.2014.2307122


*Conference on Intelligent Robots and Systems, 2007. IROS 2007* (San Diego: IEEE), 3635–3640.


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2018 Roozing. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Evaluation and Analysis of Push-Pull Cable Actuation System Used for Powered Orthoses

### Svetlana Grosu\*, Carlos Rodriguez–Guerrero, Victor Grosu, Bram Vanderborght and Dirk Lefeber

MECH Department, Vrije Universiteit Brussel (VUB) and Flanders Make, Brussels, Belgium

Cable-based actuation systems are preferred in rehabilitation robotics due to their adequate force transmission and the possibility of safely locating the motors away from the patient. In such applications, the cable dynamics represents the prescribing component for the system operating loads and control. A good understanding of the actuation, based on cable-conduit transmission, is therefore becoming mandatory. There are several types of cable-conduit configurations used for the actuation. Currently, there is lack of information in literature with regard to the push-pull cable type. Therefore, the main focus of this contribution is to evaluate push-pull cable-based actuation used within wearable robotic devices. This study includes working principle description of push-pull cable actuation with its characteristic advantages and drawbacks. The use of push-pull cables in bidirectional force transfer with remote actuation is investigated being integrated in a test-stand setup of a novel gait rehabilitation device. The experimental results and close analysis of the push-pull cable-based actuation system outline its performance, the overall dynamic behavior and the transmission efficiency of push-pull cables used for powered orthoses.

#### Edited by:

Barbara Mazzolai, Fondazione Istituto Italiano di Technologia, Italy

#### Reviewed by:

Dongming Gan, Khalifa University, United Arab Emirates Arturo Forner-Cordero, Universidade de São Paulo, Brazil

\*Correspondence:

Svetlana Grosu sgrosu@vub.be

#### Specialty section:

This article was submitted to Bionics and Biomimetics, a section of the journal Frontiers in Robotics and AI

Received: 18 December 2017 Accepted: 21 August 2018 Published: 11 September 2018

#### Citation:

Grosu S, Rodriguez–Guerrero C, Grosu V, Vanderborght B and Lefeber D (2018) Evaluation and Analysis of Push-Pull Cable Actuation System Used for Powered Orthoses . Front. Robot. AI 5:105. doi: 10.3389/frobt.2018.00105

Keywords: push-pull cable, cable-conduit, exoskeleton, rehabilitation robotics, cable-based actuation

## 1. INTRODUCTION

In the development of novel rehabilitation robotic devices engineers are faced with the challenge of combining suitable design concepts, high performance actuator technologies and dedicated control strategies in view of improved physical human-robot interaction (HRI). According to a number of investigations on different actuation approaches for exoskeletons, the low power/weight and force/angle ratios are still major drawbacks (Herr, 2009). Classical designs including high power actuators tend to be relatively expensive. Typically these are bulky, heavy, and have a high mechanical output impedance due to necessary power transmission. In addition, actuators directly integrated on the joints would add unnecessary weight to the orthoses. In order to compensate for their own weight, the size of the motors must increase, escalating the required power from joint to joint. This will conduct to a significant increase in total system mass and inertia. The solution suggested in several contributions (Morrell and Salisbury, 1998; Sugar, 2002; Zinn et al., 2004; Veneman, 2006; Slavnic et al., 2014; Guerrero et al., 2015 ´ ) proposes relocating all actuators to the static base of the system and decoupling the dynamics of the actuator and the load, by using a compliant element, e.g., a spring, between both. This way, mass and inertia of the movable part can be significantly reduced, thus, allowing an ergonomic kinematic design.

Cable driven actuation, such as: open-ended cables (Tsai, 1994), endless cables, Bowden cables, and push-pull cables are a promising alternative when a combination of lightweight, high strength, compact designs, safety, compliance, and dynamic motions are required. Generally, the torque capacity of cablebased actuators is a function of the strength of the cable. As well, the efficiency of cable drives can reach up to 96%, on condition that they are properly implemented (Townsend and Salisbury, 1988). Initially, open-ended cables were used in actuation of robotic devices, but they were limited to providing only tension force and no compression force, so that an extra device was needed to hold the cables in tension, complicating the design and the control of the transmission system. To overcome this problem, a new generation of cables was developed: endless tendon drives (Tsai, 1994), Bowden cables (Veneman et al., 2005; Sulzer et al., 2009), and push-pull cables (PPC) (Winter and Bouzit, 2007; Grosu et al., 2014; Slavnic et al., 2014; Xu et al., 2014; ´ Guerrero et al., 2015).

In recent years, cable-conduit actuation gained significant attention in rehabilitation robotics (Springer and Ferrier, 2002; Wege and Hommel, 2005; Veneman, 2006; Dovat et al., 2008; Sulzer et al., 2009; Slavnic et al., 2013; Wu et al., 2015), mainly due to advancements in high-strength cable materials which support the transmission of high forces and offer the possibility of locating actuators away from the patient. However, these advantages are overshadowed by the nonlinear dynamic behavior caused by friction between the cable and the conduit. Backlash effect is more evident due to compliance and friction within the conduit when the actuation direction is changed. This issue can result in a loss of precision and has to be compensated for in control algorithms. As an example of this approach, an adaptive backlash inverse controller was developed by Agarwal et al. (2010) that dynamically estimates the model parameters and compensates for changes in friction influenced by the conduit curvature. These drawbacks were reported also in various studies where cable actuation is used (Townsend and Salisbury, 1988; Kaneko et al., 1991; Panchaphongsaphak et al., 2006; Agrawal and Peine, 2008). The backlash effect produced by using cabledriven mechanisms on surgical robot was evaluated in Peine et al. (2012). Some researchers have investigated means of reducing the coefficient of friction in cable and housing systems (Sammons, 1983; Carlson et al., 1990). LeBlanc (1990) showed efficiency depends upon the angle of wrap, the types of the cable and housing used. The effect of friction coefficient and other variables on frictional losses in upper-limb prostheses was researched in Carlson et al. (1995). The following parameters of interest were investigated: type of the cable and cable housing; the angle through which the cable bends; bending radius and the amount of tension in the cable. Various techniques are adopted in practice to reduce the friction effects, e.g., by using PTFE-coated steel cables and keeping wide angles for cable-wrapping (Letier et al., 2006). But, these hardwarebased solutions can reduce the friction levels only to a certain degree. The other way to deal with cable-conduit nonlinear dynamics is to implement effective controllers (Agarwal et al., 2010; Vitiello et al., 2013; Slavnic et al., 2014; Guerrero et al., ´ 2015) where the control parameters must be adjusted to the certain configuration of the cable (Panchaphongsaphak et al., 2006).

From the available cable-driven solutions further in this work we propose to focus on push-pull cable actuation. This paper provides a detailed description of general PPC technical specifications. The main goal is to investigate the transmission efficiency, mechanical design and implementation of PPC actuation system into exoskeletons. The authors describe issues related to working principles, geometric, kinematics, and dynamics particularities of the PPC actuation system.

Another goal was to estimate the motor torque τ<sup>m</sup> in the experimental setup in conditions that only force sensors are available.

Following section 1 where the state-of-the-art of cablebased transmission systems was presented, section 2 will describe the materials and methods. Here, the working principle, transmission efficiency, and friction characteristics of the PPC are discussed. The description of experiments using an orthosis test setup which is a simplified version of a novel gait rehabilitation device CORBYS, can be found at the end of the section. The experimental results of the PPC actuation system are presented in section 3. In sections 4 and 5 the authors present main observations, results, and conclusions of the complete work.

## 2. MATERIALS AND METHODS

## 2.1. Physical Characteristics of Push-Pull Cable Transmissions

Push-pull control assemblies are designed to provide smooth, positive, and precise transmission of the mechanical motion for medium to heavy-duty push-pull applications. The general structure of a PPC consists of an inner member, made from a wire rope and armored with a polished flat band wrap covered with an inner tube. The external layer is represented by an extruded plastic mantle of great strength and durability, see **Figure 1**. The inner member can easily slide in low friction lifetime lubricant. Also, the end borders of the cable are featured with stainless materials and seals, ensuring protection against foreign matter and corrosion.

The main parameters influencing the cable performance are the normal forces on the cable determined by cable tension or preload, the friction coefficients resulting from material combinations and velocity of the inner member. Furthermore, cable and conduit stiffnesses play an important role in the definition of stick-slip behavior and consequently, the mechanical bandwidth of the transmission.

As mentioned before, friction between the inner member and the external conduit usually has an impact on the assembly efficiency. This is also the case for durability and control where the friction factor depends upon the total degrees of bending in the cable. The friction can be expressed by the equation, described in Schiele et al. (2006):

$$\frac{F\_{in}}{F\_{out}} = e^{-\mu\theta} \tag{1}$$

where Fin/Fout is the ratio of input to output forces, µ is the kinetic coefficient of friction and θ is the total wrap angle.

The total wrap angle (θ) of the cable system is represented in **Figure 2** and defined in the equation:

$$
\theta = \theta\_1 + \theta\_2 = \sum\_{i=1}^n \theta\_i \tag{2}
$$

This is why cable manufacturers advise keeping the push-pull cables as straight as possible in a setup.

There is only limited literature on experimental evaluation of friction between the cable and outer conduit for push-pull cables. The experimental results on PPC static friction evaluation were presented in Slavnic et al. (2014) ´ , with the focus on the dependence of static friction on the bending angle while using different cable loads. For these experiments the PPC of 2 m length has been considered, while the bending angle of the cable was set to 180, 360, 540, and 720◦ . The actuation pulling force was exerted on one end of the cable and recorded at the moment when the other end of the cable started the movement. In **Figure 3** can be observed that the effectiveness of the PPC for bending angles between 180◦ and 720◦ is in the range of ≈ 85–40%. While the maximal efficiency is achievable if the cable is mentioned straight and it goes up to ≈95%. Furthermore, according to PPC cables manufacturers the efficiency factor may vary due to length, strike, movement direction, bend radius and temperature. In this sense cable features such as structural modifications, cable size, end connectors types and cable lengths should be adapted to

the design requirements. The selection of the proper push-pull cable, generally, is a function of the desired input force. However, in a real cable-conduit based setup, changing the bending angle modifies the cable preload and therefore also has a fundamental effect on the cable efficiency. Consequently, the cable preload increases the amount of friction as the normal forces between the cable and external conduit get bigger.

## 2.2. Test-Stand Mechanical Design

In the initial testing phase and the evaluation process of PPCbased actuation system, a test-stand setup of the CORBYS gait rehabilitation device (Slavnic et al., 2014 ´ ) was built by project partners from SCHUNK. This test-stand was meant to prove the actuation system design concept through various experiments, as well as supporting development of the sensor processing and control algorithms to be used in the final CORBYS system prototype. The test-stand device, see **Figure 4** consists of an

orthosis leg and PPC-based actuation system. The frame is supporting three PRL+ motors that actuate the orthosis joints in the sagittal plane. The orthosis leg includes three revolute joints: at the ankle, knee, and hip. The mass of the orthosis is 9.2 kg. Three push-pull cable elements are connected between the actuators and joints of the orthosis leg. The selected PPCs have different geometry properties (diameters, lengths etc.) since different moments are required for the three joints actuation.

There are two U9C force cells implemented on extremities of each PPC: one force cell is located close to the actuator, the second one is on the joint side.

In **Table 1** are presented the general technical specifications of the PPCs used for CORBYS test-bed actuation system. [t]

## 2.3. Working Principle of PPC Actuation System

In contrast to Bowden cables, that can transfer force only in pulling direction, PPCs are bi-directional, able to transfer force in pulling and pushing directions. The PPC cables are therefore larger in diameter, stiffer, and able to transmit larger forces.

**Figure 5** shows one of the actuated orthosis joints with the relative kinematic and dynamic variables. In order to

TABLE 1 | Push-pull cables technical specifications, integrated in the test-stand device.


transmit rotational motion from motors to the orthosis joints via PPC cables, rotational motion of the motor has to be transformed into linear displacement. Then, on the joint side the linear displacement of the PPC has to be transformed to a rotational motion. For this reason, mechanically simple slidercrank mechanisms are used. The motor generates a moment τ<sup>m</sup> that is converted to the cable force f<sup>s</sup> by the slider-crank mechanism. This force is subsequently transmitted to the orthosis joint via PPC cables. The overall efficiency of the PPC actuation system is determined by the efficiencies of the sub-systems components: the efficiency of the PRL+ motor, the efficiency of the slider-crank mechanisms and the efficiency of the PPC.

**Figure 6** shows the slider-crank mechanism displaced on the motor side of the PPC actuation system. The length of the crank is denoted by r, while l<sup>s</sup> is the length of the connecting rod. The angular displacement of the crank is represented by α and d is the normal displacement between the crank pivot point and the slider line.

The crank pivot point is marked with O where the coordinate x-y system is located. The angle that the connecting rod makes with the slider line is denoted as β . With point C we indicate the revolute joint between the crank and the connecting rod. S marks the revolute joint between the connecting rod and the slider. Position of point S according to x-y coordinate system is expressed as following:

$$\mathbf{S} = \begin{bmatrix} \boldsymbol{\chi\_s} \\ \boldsymbol{\chi\_s} \end{bmatrix} = \begin{bmatrix} -r\sin\alpha & -l\_s\cos\beta \\ r\cos\alpha & +l\_s\sin\beta \end{bmatrix} = \begin{bmatrix} \boldsymbol{\chi\_s} \\ \boldsymbol{d} \end{bmatrix} \tag{3}$$

The angle <sup>β</sup> and angular velocity <sup>β</sup>˙ can be calculated using the following expressions:

$$\beta = \sin^{-1}\left(\frac{r\cos\alpha - d}{l\_s}\right) \tag{4}$$

$$\beta = -\frac{r\sin\alpha}{l\_s\cos\beta}\,\dot{\alpha}\tag{5}$$

The lengths of the connecting l<sup>s</sup> and crank rods r are different for the slider-crank mechanisms displaced on the orthosis joints and on the motor side, as can be seen in **Table 2**.

The operating range of the slider-crank mechanism is a function of the lengths of the crank, connecting the rod and the distance of the slider line from the crank pivot, therefore:

$$
\alpha\_{\max} = \pm \cos^{-1} \frac{d - l\_s}{r} \tag{6}
$$



In order to express the motor torque related to the force transmitted via PPC and later to the joint moment, static force analysis of the slider-crank mechanism is described below. The mass and inertia parameters of the connecting rod are neglected, while the inertia of the crank plate is taken into consideration. **Figure 7** shows the force diagram of the slider-crank mechanism. The motor torque M<sup>m</sup> is applied by the motor around the vector of the crank joint that is normal to the image plane and passes through the crank pivot point O . The applied momentum M<sup>m</sup> is converted by the lever r to the force F<sup>m</sup> at the other side of the crank link (point C).

The force F<sup>m</sup> is equal to:

$$F\_m = \frac{\mathbf{r}\_m}{r} \tag{7}$$

The force F<sup>s</sup> that acts on the push-pull cable end is represented by:

$$F\_s = F\_{ma} \cos \beta \tag{8}$$

where Fma – the axial force transmitted by the connected rod is expressed as:

$$F\_{ma} = \frac{F\_m}{\cos(\beta + \alpha)}\tag{9}$$

## 2.4. Description of Experiments 2.4.1. Goals

Several experiments were performed in order to evaluate the transmission efficiency of the PPCs by comparing forces measured on the motor and respectively, the orthosis joint sides of the cable (Fin vs. Fout). Based on these force measurements, Equations (4, 7–9) were used to calculate the torque required for the orthosis joints actuation.

The other point of interest during the experiments, was to observe the effect of velocity changes on operational and output forces.

### 2.4.2. Experimental Setup

For the experiments was used the test-stand device described in section 2.2. It was necessary to define several conditions related to the experimental setup, namely:


### 2.4.3. Experimental Protocol

For the experiments we considered actuating consecutively the hip and knee joints of the orthosis. The ankle joint was unactuated during all the experiments. However, the ankle PPC remained connected between the ankle joint and the motor. Actuation of the hip and the knee joints is realized by implying to the motor a sinewave of a constant amplitude. The amplitude is selected in such a way that the actuation in both directions is noticeable, therefore to illustrate the effect of the push and pull movements of the cable. The curvatures of PPCs are not restrained or guided in any specific way.

The force measurements are provided by the two force sensors (U9C), located on the joint (indicated in graphs by JF) and the motor sides (indicated in graphs by MF) of each PPC. This data was used later to determine the torque parameter τ<sup>m</sup> with the expressions (4, 7–9).

The hip and knee joints were tested in separate trials. Therefore, two sets of experiments can be distinguished, for the hip and respectively, for the knee. Each set consisting of two tests with different motor velocities following a sinewave amplitude.

For the hip joint actuation experiments motor amplitude was set to ±25◦ . To vary the velocity, sinewave frequency was set to 0.15Hz and then to 0.05Hz.

For the knee joint experiments the motor amplitude was set to 30◦ , starting from the initial position when the orthosis leg is in vertical position. In contrast to the hip actuation experiments, hier PPC was always in tension due to the fact that the orhosis knee joint has limited range of motion and does not allow extension. Therefore, the dynamics of PPC in contraction was not characteristic for the knee actuation experiments. However, the same frequencies as the ones used in the hip experiments were used, 0.15Hz and then 0.05Hz.

## 2.4.4. Data Processing

Force and position measurements were collected from all the experiments and exported to MATLAB. Each experiment was performed in several trials. Out of continuous data stream an arbitrary period of 30 s was selected for further data processing. The same period was considered for all four experiments.

## 3. RESULTS

## 3.1. Hip Joint Actuation, First Set of Experiments

In **Figures 8**, **9** are illustrated the joint force (JF) and motor force (MF), measured by sensors located on both ends of the hip pushpull cable, function of the motor position angle. Two phases can therefore be distinguished in the figures, pulling (represented by a negative force on a graph with gray background) and pushing (represented by a positive force on a graph with white background). The transaction from pulling to pushing movement can be identified according to force readings. When the orthosis achieves the equilibrium position the force sensors register zero value. Pushing phase starts when the orthosis passes the equilibrium position and therefore sensors register positive readings. Considering that orthosis equilibrium position is not when the orthosis is perfectly vertical as the initial condition for the experiments, more pulling force is required to reach the desired amplitude (sinewave is offset from the equilibrium position).

**Figure 10** shows joint (denoted by JAbsP on the graph) and motor angular positions(MPD) during the experiments with

hip joint actuation. It can be noticed that the joint angle is following the output sinewave trajectory of the motor. These graphs demonstrate an appropriate position control without deviations from the imposed motion (no oscillations as have been seen in the force readings, from **Figures 8**, **9**).

It can be also observed that the angle of the joint exceeds that of the motor by 2–5◦ . This appears due to the various geometrical dimensions of the slider-crank elements used on the joint and respectively, the motor sides, as specified in **Table 2**.

## 3.2. Knee Joint Actuation, Second Set of Experiments

In the second set of experiments, see **Figures 11**, **12** the knee joint motor has been actuated, while the hip and the ankle motors remained blocked.

**Figure 13** shows the joint (JAbsP on the graph) and motor angular positions (denoted by MPD on the graph) during the experiments with the knee joint actuation. We can see the output sinewave trajectory of the motor of 30◦ and the output angle of the joint. The joint closely follows the trajectory imposed by the motor.

In **Figures 14**, **15** we can see the calculated motor torque function of input force Fin, obtained from the hip and knee actuation sets of experiments. According to the graphs we can observe almost linear relation between motor torques and forces.

In **Table 3** are displayed push-pull cables efficiencies calculated based on peak forces (Fin and Fout ) measured in the experiments. According to this data we can conclude that the difference between the efficiencies values is not considerable for the two selected speeds. Also, it can be observed that in pushing

TABLE 3 | Hip and knee push-pull cables efficiencies during pushing and pulling phases, measured at the maximum force magnitudes.


phase the efficiency is slightly higher as less force is required to perform the movement. Therefore, the efficiency depends on the operating loads, which is in line with the specifications from literature, presented in the introduction part of this article. It can be also observed that in contrast to the hip PPC, the knee PPC is operating with less efficiency. This can be explained by the different geometry and of the knee cable, which is longer and thinner with an increased bending curvature.

## 4. DISCUSSION

In this paper were addressed the critical points of cabledriven transmissions, including their dynamic behavior, main advantages, and drawbacks with the focus on the push-pull cable configuration. PPC-based transmission has been implemented in actuation of a new gait rehabilitation robot CORBYS, but in order to test the performance of actuation system its simplified version (a test-stand) has been used for the experiments. One of the main goals of experiments was to estimate the transmission efficiency of the PPC and to observe the overall dynamic behavior of the system. The obtained results suggest to think about following control optimization strategies for compensation of friction, nonlinearities, and backlash issues.

All types of cable-conduit transmissions present drawbacks, such as nonlinear friction highly dependent on curvature and geometry of the cable. The dynamic behavior and functional characteristics of Bowden and PPC transmissions are very similar, but there are also few differences. In comparison to PPC, Bowden cable-based actuation systems can operate only in pulling direction. Taking into account that PPCs are larger in diameter and more stiff, they are able to transmit larger forces compared to the Bowden cables. Moreover, PPCs have the ability to transfer forces in two directions, pushing and pulling.

Considering hip joint experiments, if we compare force readings provided by the motor side force cell (MF on a graph, in RED) and joint force cell (JF on a graph, denoted in BLUE), it can be observed that the higher force is needed at the motor side. This outcome appears due to the energy losses caused by nonlinear friction along PPC actuation mechanism.

When the motor achieves the maximum imposed angle position (−25◦ or 25◦ ) it changes the actuation direction. Starting from that moment, according to the graphs, it can be observed that the gravity helps the movement and JF and MF are overlapping. According to the experimental results illustrated in **Figures 8**, **9**, in limits of the selected frequencies, the required force magnitudes are similar. However, for higher velocities, due to the dynamics of PPC, the force oscillations are more evident. Considering these oscillations we can state that the PPC performs better in tension.

In contrast to the hip actuation experiments, the push-pull cable in knee joint experiments was always in tension because the orthosis leg was not passing through the equilibrium position. Therefore, observing the PPC dynamics while in contraction (which is pushing phase) is missing for the knee actuation experiments. The difference in force readings between the motor sensor (MF) and joint sensor (JF) can be visualized on the graphs. This difference is produced by losses of energy due to the dynamics of the actuation system and push-pull cable nonlinear friction. This force difference is more evident in contrast to the hip PPC, due to the physical characteristics of the PPC used for the kne actuation joint. Namely, it is longer, thinner (see **Table 1**) and was changing cable curvature due to the orthosis motion. Similar to the hip joint actuation set of experiments, selected velocities do not have any considerable effect on force output magnitudes.

Taking into account that the exact model of the PRL+ actuator is difficult to obtain due to its nonlinear characteristics, the motor torque τ<sup>m</sup> was determined based on sensors force measurements (MF) and using formulas 4, 7–9 presented in theoretical part of section 2.

The establishing of the motor torque τ<sup>m</sup> was one of the goals followed in this contribution. This torque will be used as a reference for further investigations on torque control strategies as an alternative to the position control used so far. Torque control is especially important for the applications where humanrobot interaction is involved, such as in exoskeleton applications. This solution permits to avoid integration of the expensive torque sensors in the system and reduces its mechanical design complexity.

## 5. CONCLUSION

Cable-based actuation systems show many advantages over classical actuators when implemented in various robotics related applications. The employment of this type of actuation in wearable devices permits the dislocation of the actuators from the orthosis so that inertia of the motors is not imposed on human

## REFERENCES


body, therefore, improving safety and functional aspects. Still, issues related to control challenges, specific mechanical design requirements and assumptions have to be considered. These aspects were addressed in this paper together with the evaluation of cable-based force transmission system for powered orthoses.

Several experiments were performed to evaluate the transmission efficiency of the PPCs by comparing forces measured on the motor and respectively, the orthosis joint sides of the cable. Based on these force measurements, the torque required for the orthosis joints actuation was computed.

Additionally, we noticed that the efficiency of the PPC force transmission is highly dependent on the configuration of the actuation components of the system, such as mechanical design, geometry of the PPC, bending angles, and cable preloads.

The other objective during the experiments, was to observe the effect of velocity changes on operational and output forces. According to obtained results the force amplitude does not change considerably but the force oscillations are more evident at higher velocities.

According to the obtained results, can be concluded that all specific features and compromises typical for cable-conduitbased transmissions are also characteristic to PPC actuation. Still, using PPC actuation in certain applications could be preferred due to the number of individual advantages. For example, capability to transfer larger forces in two directions and less complex mechanical construction of the actuation system. The use of PPC actuation systems is advised in applications where the light-weight design and transmission of large forces are required, definitely a solution to consider in wearable devices.

## AUTHOR CONTRIBUTIONS

SG, CR-G, and VG were working together on experimental setup and post-processing of results. BV and DL were providing guidance in this research study, as well they contributed in a writing process of this article.

## FUNDING

This research work has been funded by the Research Foundation - Flanders (FWO) under grant no. S000118N SBO Exo4Work project.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Grosu, Rodriguez–Guerrero, Grosu, Vanderborght and Lefeber. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Efficiency and Power Limits of Electrical and Tendon-Sheath Transmissions for Surgical Robotics

*Christopher R. Wagner\* and Evangelos Emmanouil*

*Medical Technology, Cambridge Consultants, Cambridge, United Kingdom*

A popular design choice in current surgical robotics is to use mechanical cables to transmit mechanical energy from actuators located outside of the body, through a minimally invasive port, to instruments on the inside of the body. These cables enable high performance surgical manipulations including high bandwidth control, precision position control, and high force ability. However, cable drives become less efficient for longer distances, for paths that involve continuous curves, and for transmissions involving multiple degrees of freedom. In this paper, we consider the design tradeoffs for two methods of transmitting power through an access port with limited cross sectional area and curved paths - tendon/sheath mechanical transmissions and electrical wire transmissions. We develop a series of analytic models examining fundamental limits of efficiency, force and power as constrained by access geometry, material properties, and safety limits of heat and electrical hazards for these two transmission types. These models are used to investigate the potential of achieving the required mechanical power requirements needed for surgery with smaller access ports and more difficult access pathways. We show that an electrical transmission is a viable way of delivering more than sufficient power needed for surgery, highlighting the opportunity for next-generation actuators to enable more minimally invasive surgical devices.

Keywords: surgical robotics, tendon sheath transmissions, cable drives, efficiency, minimally invasive surgery, shape memory alloy (SMA)

## Introduction

Current generation laparoscopic surgical robots, such as the daVinci Xi (Intuitive Surgical, Sunnyvale, CA, USA) are high performance general purpose machines for supporting surgeons in executing surgical tasks. They allow for high bandwidth control, precision position control, high force ability, and high endurance (Taylor et al., 2016). To achieve these benefits, surgical robots use rigid shafts in combination with mechanical cables around pulleys to transmit mechanical energy from outside of the body to the distal joints inside the body. This is an effective and efficient transmission for straightline access from the entry port to the surgical site when using stiff materials and low friction bearings.

Access is limited, however, to a straight line from entry point to relevant anatomy. We would like to extend the minimally invasive benefits of surgical robotics - increasing dexterity with reduced trauma and reduced healing time - to more surgical procedures that involve longer and more tortuous access pathways (**Figure 1**). Candidates include natural orifice transluminal endoscopic surgery (NOTES) and augmenting manipulation capabilities for flexible endoscopic procedures (McGee et al., 2006; Burgner-Kahrs et al., 2015). Additional procedures, such as vascular access, or additional benefits,

#### *Edited by:*

*Navvab Kashiri, Fondazione Istituto Italiano di Technologia, Italy*

#### *Reviewed by:*

*Muneaki Miyasaka, Nanyang Technological University, Singapore Gianluca Palli, Università degli Studi di Bologna, Italy*

#### *\*Correspondence:*

*Christopher R. Wagner chris.wagner@cambridgeconsultants. com*

#### *Specialty section:*

*This article was submitted to Robotic Control Systems, a section of the journal Frontiers in Robotics and AI*

> *Received: 22 December 2017 Accepted: 12 April 2018 Published: 18 June 2018*

#### *Citation:*

*Wagner CR and Emmanouil E (2018) Efficiency and Power Limits of Electrical and Tendon-Sheath Transmissions for Surgical Robotics. Front. Robot. AI 5:50. doi: 10.3389/frobt.2018.00050*

**74**

such as sutureless entry wounds, can also be considered if the access port diameter were further reduced from its current minimum of 5 mm (Ferguson and O'Kane, 2004; Tacchino et al., 2009).

There are several approaches to deliver the mechanical energy needed to carry out surgical manipulations from the outside of the body to the inside. One approach is to use a tendon/sheath drive - continue to use mechanical cables, but embedded in a stiff sheath that can provide the reaction force against which to actuate the tendons. This approach has the obvious limitation that as the access diameter is reduced, and the access path length increases and becomes curvier, the performance of the transmission will decrease due to friction, stiffness and inertia of the tendon. Further, there becomes a tradeoff with number of achievable degrees of freedom as each requires additional tendons which require additional cross sectional area.

An alternate transmission that would not suffer from these access path limitations is an electrical transmission - a wire. The efficiency of a wire to transmit power is high; however, creating a minimally invasive surgical robot based on an electrical transmission has the obvious drawback that the actuators now need to be located on the inside of the body. Examples of this approach exist (Takayama et al., 1997; Mineta et al., 2001; Yeung and Gourlay, 2012; Lee et al., 2014), though the actuator size is now the dominant factor. Still, if the actuators of sufficient performance could be placed significantly closer to the surgical site inside the body, this would remove another limitation of current surgical robots - the large size of the systems. Actuators on the inside of the body would likely not suffer from gravitational effects nearly as much; the knock-on effect of a small increase in distal actuator size resulting in a larger set of proximal actuators for a serial arm configuration would be avoided. And, there are plausible actuator technologies that have a significantly higher work density that traditional electrical motors, including shape memory alloy and piezoelectrics (Huber et al., 1997), that would achieve sufficiently low internal actuator volumes.

In this paper, we present a series of analytic models that investigate the design tradeoffs involved in considering these two transmission approaches for surgical robotics. We first establish a model describing the limits of heat uptake in the body, which is the fundamental limit for both transmission types. We then develop a model of tendon/sheath power transmission, establishing limits of efficiency, force, and power based on access geometry and material properties. Similarly, we develop a model for electrical transmission of power into the body, taking into account access geometry and electrical safety limits to identify efficiency and power limits. Using these models, we then evaluate their relative ability to efficiently deliver power and required performance under the access constraints posed by minimally invasive surgery. The results highlight the overall opportunity for high performance minimally invasive robotics with more stringent access geometries than current systems.

## Heat Dissipation as Fundamental Limit to Power Delivery

A fundamental limiting factor when delivering power into the human body is the corresponding power lost to heat along the length of the transmission. If the temperature rises too high, cell death and permanent tissue damage can result (Rossmanna and Haemmerich, 2014). Medical device regulations provide guidance as to safe temperature limits that can be applied to the body (International Electrotechnical Commission, 2014), depending on the length of applied time. However, these temperature limits need to be translated into power limits, as parameterised by tissue thermal properties, to be useful as design guidance.

In this section, we present an analytic model of local heat propagation to relate known safe temperature limits to corresponding power limits. These power limits are used in the transmission models presented in later sections. The following model derives thermal power limits for a cylindrical geometry (such as a mechanical or electrical wire) giving off heat into surrounding tissue in steady state, based on a solution to the well known bioheat equation (Incropera et al., 2011). While other bioheat models exist, the use of the bioheat equation is widespread and a range of measurements exist for the model constants for different tissues (Kerdok et al., 2006; Hasgall et al., 2015). Also, use of the bioheat equation encapsulates the key properties of heat removal in bulk tissue (diffusion and heat loss through capillary perfusion) and does not require us to make an estimate of an imprecise convection term.

This model is useful as an initial estimate for heat limits, and its analytic nature is useful on which to base further calculation. For more complicated geometries and tissue interactions, a mesh style solver may be required to derive more precise limits.

As derived more fully in the Appendix, an estimate for the upper bound of heat power that the body can safely dissipate through a cylinder of radius *r*, per unit length, is given by

$$H\_{\text{max}}\left(r\right) = 2\pi kBr \frac{K\_1\left(Br\right)}{K\_0\left(Br\right)}\left(T\_1 - T\_a\right) \tag{1}$$


where

$$B = \sqrt{\frac{\omega \rho\_b c\_b}{k}}\tag{2}$$

and *ω*, *ρb*,*cb*, *k* are tissue thermal properties as described in **Table 1**, *T*1 is the surface temperature of the cylinder, and *K*0 ( ...) and *K*1 ( ...) are zeroth and first order modified Bessel functions of the second kind, respectively.

## Efficiency Limits of Mechanical Tendon/sheath Transmissions

Current laparoscopic surgical robots use mechanical cable drives for tool actuation inside the body as well actuating external body motion. These systems rely on tense cables running between pulleys, which are well modelled in the literature (Miyasaka et al., 2015, 2016) to enable higher performance through closed loop control (Haghighipanah et al., 2015; Rosen et al., 2017). Fundamental efficiency limits of this style of cable/ pulley transmission have also been established (Townsend and Salisbury, 1988), as well as size and parameter tradeoffs as applied to surgical tools (Friedman, 2011).

However, the principles of operation of cable/pulley transmissions do not directly extend to curved paths. Instead, use of a mechanical cable drive through a continuous curved path requires use of a sheath around the cable to provide opposing tangential and axial forces. Models for friction and control of these systems exist in the literature, with varying levels of detail relating to complex effects such as tendon viscoelasticity and hysteresis (Palli and Melchiorri, 2006; Palli et al., 2009, 2012; Agrawal et al., 2010; Do et al., 2015; Choi et al., 2017). We extend here a simple tendon/sheath model (Palli and Melchiorri, 2006; Palli et al., 2009), in combination with conservative loss

assumptions, to derive power and efficiency limits. Based on these limits, we can compare the performance of cable/sheath transmissions to other transmissions, especially in cases where a cable/sheath approach may intuitively seem less efficient, such as for long, curvy paths. Key to these models are the incorporation of the design constraints that will relate to MIS surgery, such as access diameter and path length, as well as performance outputs, such as efficiency, force, and degrees of freedom.

## Efficiency Model

In this section, we derive an expression for the efficiency limits of tendon/sheath transmissions, based on validated models that exist in the literature. We start with a simple tendon/sheath model that relates input and output tensions as parameterised by a radial path geometry (**Figure 2**, **Table 2**), and a friction coefficient relating tendon tension with friction forces (Palli and Melchiorri, 2006; Palli et al., 2009). A single parameter (curvature) is used to parameterise the radial path geometry, where more complex paths can be modelled with a simple radial path with equivalent accumulated angle (Do et al., 2015). This model also accounts for tendon stretch but does not assume any stretch or losses due to the sheath. Because we are only concerned with power and efficiency limits and not more complex effects such as tendon hysteresis, we make a conservative assumption where we only transmit energy when pulling and all energy associated with hysteresis is assumed lost. We re-derive the solution to the model to be explicit about the contribution of pretension so that it can be correctly incorporated into an expression for efficiency.

As derived more fully in the Appendix, we separate an explicit pretension term (*T*0) from the input and output tendon tensions:

$$\begin{aligned} T\_{\bar{l}n} &= T\_{\bar{l}m,\bar{w}} + T\_0\\ T\_{out} &= T\_{out,\bar{w}} + T\_0 \end{aligned} \tag{3}$$

The output tension capable of doing work *Tout*,*w* is given by

$$T\_{out,w} = T\_{in,w} e^{-\frac{\mu L}{R}} - T\_0 \left(1 - e^{-\frac{\mu L}{R}}\right) \tag{4}$$

and the stretch of the tendon *δw* only due to input tension (and not pretension) is

#### Table 2 | Mechanical Nomenclature.


$$\delta\_{\mathcal{W}} = \frac{T\_{\dot{l}n,\mathcal{W}}R}{EA\mu} \left(1 - e^{-\frac{\mu L}{R}}\right) \tag{5}$$

We can now write the relationship for efficiency, relating work in to work out. We do not need to include the prestretch, and use *Tin*,*w* and *Tout*,*w* as the input and output tensions that account for transmitted work. Assuming an input motion of distance *D* at tension *Tin*,*w*, then

$$W\_{\rm in} = DT\_{\rm in,w} \tag{6}$$

and

$$W\_{\rm out} = \left(D - \delta\_{\rm w}\right) T\_{\rm out,w} \tag{7}$$

so efficiency *η* becomes

$$\begin{split} \eta &= \frac{W\_{out}}{W\_{in}} \\ &= \frac{\left(D - \delta\_{\text{w}}\right) T\_{out,\text{w}}}{DT\_{in,\text{w}}} \end{split} \tag{8}$$

Expanding terms and simplifying gives a final equation for efficiency:

$$\begin{split} \eta &= \quad \varepsilon^{-\frac{\mu L}{R}} \left( 1 - \frac{T\_0}{T\_{in,w}} \left( e^{\frac{\mu L}{R}} - 1 \right) \\ &- \frac{R}{EA\mu D} \left( T\_0 \left( 2 - e^{\frac{\mu L}{R}} - e^{-\frac{\mu L}{R}} \right) + T\_{in,w} \left( 1 - e^{-\frac{\mu L}{R}} \right) \right) \right) \end{split} \tag{9}$$

Examining (9), we note that as *D* approaches infinity, the efficiency limit can be simplified to:

$$\eta\_{lim} = e^{-\frac{\mu L}{R}} \left( 1 - \frac{T\_0}{T\_{in,w}} \left( e^{\frac{\mu L}{R}} - 1 \right) \right) \tag{10}$$

which corresponds to the overall work delivered being large compared to tendon losses. This applies to a tendon/sheath drive where power is transmitted continuously in one direction, such as in a closed loop. Note that this expression is not simply the tension ratio of *Tout*,*w* to *Tin*,*w*, but contains an explicit term that decreases efficiency with the increase of pretension.

## Degrees of Freedom and Bend Radius

This analysis is meant to support an understanding of the achievable performance for a given cross sectional access area, path geometry, and material property limits. As observed above, the efficiency of a tendon drive relates to the cross sectional tendon area, where larger tendons of the same material result in stiffer and thus more efficient tendon drives. To relate this result to the total cross sectional access area, we also need to account for sheath stiffness as well as degrees of freedom.

We make the simplifying assumption that sufficient sheath material must exist so that the sheath stiffness must at least match the tendon stiffness (when loaded axially). If this was not the case, the sheath stiffness would dominate and performance would be limited.

Using this assumption, and the limit that *ndof* degrees of freedom can be controlled by as few as *n* + 1 tendons (Tsai, 1999), this gives an upper bound to the number of degrees of freedom achievable for a given access area. Solving the following set of equations:

$$\begin{array}{ll}E\_{\text{sheath}}A\_{\text{sheath}} &= n\_{\text{trendon}}E\_{\text{trendon}}A\_{\text{trendon}},\\A\_{\text{total}} &= A\_{\text{sheath}} + n\_{\text{trendon}}A\_{\text{trendon}},\\n\_{\text{def}} &= n\_{\text{tendon}} - 1\end{array} \tag{11}$$

and using a cylindrical expression for area gives an upper bound estimate for the number of achievable degrees of freedom for a tendon sheath system, for a given geometry and material properties.

$$n\_{dof} = \frac{r^2 E\_{sheet}}{r\_{tendon}^2 \left(E\_{tendon} + E\_{sheet}\right)} - 1\tag{12}$$

This expression is a conservative bound, as it does not account for additional area needed for sliding tolerances, working channels, or close packing adjustments. Also, an equal stiffness assumption was used to relate the cross sectional area devoted to the sheath material versus tendon material; more complex failure modes like buckling are not accounted for. Further, if this limit is used, it assumes a monolithic sheath whose bending is limited by the material properties, not by additional geometry features (such as notches to increase flexibility). Finally, a common design simplifying control of the system uses 2 tendons for each degree of freedom, which further exacerbates the conservative bound.

If we restrict ourselves to material strain limits, then an estimate of the corresponding minimal bend radius relative to tool radius can be calculated using estimates of strain at yield:

$$R\_{min} = \frac{r}{\epsilon\_{yield}}\tag{13}$$

## Power and Force Limits

Power delivery through a tendon sheath system is limited by the maximum force achievable per tendon, and the maximum velocity achievable per tendon without causing heat damage due to frictional losses.

Force limits for wire rope can be modelled with an exponential fit relating radius to breaking strength; see (Friedman, 2011) for experimental fits to several material types. The working limit of a wire rope is then related to the breaking limit through a safety factor. An expression for this working limit tension *TWL* is given by

$$T\_{WL} = \frac{1}{s} \alpha r\_{tendon}^{\beta} \tag{14}$$

where *α* and *β* are the breaking limit fit parameters, *rtendon* is the radius of the tendon, and *s* is the safety factor.

The maximum velocity for a tendon (at this maximum tendon force, delivering maximum power) can be derived from the cable and heat models given above. We observe that the power loss between input and output should never exceed the heat limit of the tissue through the sheath. For large motions, this can be expressed as

$$T\_{\dot{m}}\nu - T\_{out}\nu \le H\_{\text{max}}\left(r\right)L \tag{15}$$

where v is the velocity of the tendon, and L is the path length. Note that we use *Tin* and *Tout* which incorporates pretension, instead of *Tin*,*w* and *Tout*,*w*, as the heat loss depends on the total tension, not just the delivered power.

Because *Tout* exponentially decreases along the length of the path, the point that will have the most loss due to friction will be at the beginning, where absolute tension and loss per unit length is greatest. Thus, we can derive the maximum allowable tendon velocity by taking the derivative with respect to path length of the above expression, and solving for *v* at L = 0.

$$\begin{array}{rcl}d\left(T\_{in}\nu - T\_{out}\nu\right) &=& \frac{d\left(H\_{\max}\left(r\right)L\right)}{dL} \\ \frac{dL}{\phantom{I} - \frac{dT\_{out}}{dL}\nu}\nu &= H\_{\max}\left(r\right) \\ T\_{in}e^{-\frac{\mu L}{R}}\frac{\mu}{R}\nu &= H\_{\max}\left(r\right) \end{array} \tag{16}$$

Evaluating at L = 0, using *TWL* as the maximum input tension, and solving for v gives an expression for the maximum allowable velocity *vmax*:

$$\nu\_{\text{max}} = \frac{H\_{\text{max}}\left(r\right)R}{\mu T\_{WL}} \tag{17}$$

For this expression note that *r* is the outer radius of the sheath (or sheath bundle), not the radius of the tendon material.

## Efficiency Limits of Electrical Cables

In this section, we develop a similar analytic model to investigate the ability of an electrical wire to transmit power within the body, subject to an access geometry constraint. The model also takes into account basic electrical safety constraints which would apply when transmitting power into the body; namely, electrical breakdown causing current flow through tissue.

## Coaxial Wire Model

For this analysis, we use a coaxial wire structure (**Figure 3**) to account for the voltage and return carrying lines, as this provides the opportunity for increased safety for power delivery in the body if the outer conductor is at the same voltage as the body. A full safety analysis, however, including additional mitigations such as galvanic isolation and over-current detection is outside the scope of this work.

In this coaxial model, the outer radius is *r*, the radius of the inner conductor is *ri*, and the cross sectional area of the two conductors are equal to account for the return path of the current (**Table 3**). Thus, the width of insulation *w* can be calculated from equating the conductor areas

$$
\pi r^2 - \pi \left(r\_{\bar{l}} + \omega\right)^2 = \pi r\_{\bar{l}}^2 \tag{18}
$$

and solving for *w*. This gives

$$\mathbf{w} = \sqrt{r^2 - r\_i^2} - r\_i \tag{19}$$

#### Table 3 | Electrical Nomenclature.


Also note that for an actual wire used in the body, an additional biocompatibility layer would need to exist outside of the outer conductor. This layer has been omitted in this model because it can be made thin (e.g., less than 10 um) in practice.

## Current Limits

Given the above limit of heat into the body along a cylinder, we can derive the maximum current through the wire by Joule heating using the resistivity of the wire. Given that the absolute resistance of a wire is related to the length of the wire, we start with all equations taking into account the length explicitly.

So, from above:

$$H\_{\max}\left(r,l\right) = lH\_{\max}\left(r\right) \tag{20}$$

Resistance of a wire *R* is proportional to length and the resistivity of the material (*ρwire*), and inversely proportional to area. In the coaxial wire model, the cross sectional area of the current conducting portions of the wire is given by *A* = 2*πr* 2 *<sup>i</sup>* . The resistance equation then becomes

$$R\left(r\_i, l\right) = l\frac{\rho\_{wire}}{A} \tag{21}$$

$$l = l \frac{\rho\_{wire}}{2\pi r\_i^2} \tag{22}$$

Assuming the same amount of forward and return current in the separate conductors in the coax wire, the resistivity and heat can be related using an equation for Joule heating:

$$P\_{heat}\left(r, r\_i, l\right) = 2I\left(r, r\_i, l\right)^2 \mathcal{R}\left(r\_i, l\right) \tag{23}$$

Solving for I, and maintaining our dependence on coax geometry parameters *r*, *ri* and *l*:

$$I\_{\max}\left(r, r\_i, l\right) = \sqrt{\frac{H\_{\max}\left(r, l\right)}{2R\left(r\_i, l\right)}}\tag{24}$$

$$=\sqrt{\frac{lH\_{\text{max}}\left(r\right)}{l\frac{\mathcal{Q}\_{\text{wire}}}{\pi r\_i^2}}}\tag{25}$$

$$=r\_i \sqrt{\frac{\pi}{\rho\_{wire}} H\_{max} \left(r\right)}\tag{26}$$

Thus, we observe that the *Imax* does not depend on the length of the wire.

## Voltage Limits

The above analysis determines the maximum heat in the body as limited by the heat dissipation ability of tissue. However, another effect corresponds to limit the voltage used inside the body, which thus limits the maximum power delivered. The risk is that with high voltages, the voltage exceeds the breakdown voltage (dielectric strength) of the insulator, leading to an electrical hazard. Thus, voltages are typically limited inside the body. In the case of the coaxial cable, one form of protection is setting the outer conductor at the same potential as the body. However, we still need to protect against an internal breakdown of the insulator to prevent the outer conductor achieving a high voltage.

The breakdown voltage is related to the material properties of the insulator and the geometry with the following relationship:

$$V\_{breakdown} = d\boldsymbol{w} \tag{27}$$

where *Vbreakdown* is the breakdown voltage, *d* is the dielectric constant of the material, and *w* is the width of the insulation. The maximum operating voltage is usually related to the breakdown voltage with a safety factor (at least a factor of 5 smaller than the breakdown voltage):

$$V\_{operating} = \frac{V\_{breakdown}}{s} \tag{28}$$

where *s* is the desired safety factor.

Expressed in terms of our coaxial model geometry parameters, *Voperating* is

$$V\_{operating} = \frac{d}{s} \left(\sqrt{r^2 - r\_i^2} - r\_i\right) \tag{29}$$

## Power and Efficiency Limits

With the above relationships for voltage and current as related to geometry, we can now derive an estimate for efficiency and power limits. First, we identify the optimal inner conductor radius and corresponding insulator thickness for a given outer radius. As insulator thickness increases, the allowable drive voltage and thus power increases; however, available cross sectional area decreases, causing resistance to increase, which decreases power delivery.

Power can be expressed as the product of *Imax* (which is limited by thermal limits into the surrounding tissue) and *Voperating* (which is limited by breakdown voltage of the insulator and a safety factor).

$$P\_{\max}\left(r, r\_{\bar{i}}\right) = I\_{\max}\left(r, r\_{\bar{i}}\right) V\_{\text{operating}}\left(r, r\_{\bar{i}}\right) \tag{30}$$

We observe that there is an optimal inner conductor radius that exists for each external radius. We can explicitly solve for this *ri*,*max* by taking the derivative of the power expression with respect to *ri*, and setting equal to zero.

$$P\_{\max} = \sqrt{\pi} r\_i \sqrt{\frac{1}{\rho\_{\text{wire}}} H\_{\max} \left( r \right)} \frac{d}{s} \left( \sqrt{r^2 - r\_i^2} - r\_i \right)$$

$$\frac{dP\_{\max}}{dr\_i} = \frac{\sqrt{\pi} d \sqrt{\frac{1}{\rho\_{\text{wire}}} H\_{\max} \left( r \right)}}{s \sqrt{r^2 - r\_i^2}} \left( r^2 - 2r\_i^2 - 2r\_i \sqrt{r^2 - r\_i^2} \right) \tag{31}$$

We see that *dPmax*/*dri* will be zero when

$$\left(r^2 - 2r\_i^2 - 2r\_i\sqrt{r^2 - r\_i^2}\right) = 0\tag{32}$$

Solving for *ri*, and choosing the expression that will result in real values yields

$$r\_{i,max} = r \sqrt{\frac{1}{2} \left( 1 - \frac{\sqrt{2}}{2} \right)}\tag{33}$$
 
$$\approx 0.38r$$

Substituting this result back into our power expression, collecting known constants into a single term Λ and simplifying, we can derive the final expression for *Pmax*:

$$P\_{\max} = \frac{\Lambda dr^2}{s} \sqrt{\frac{1}{\rho\_{\text{wire}}} H\_{\max}(r)}\tag{34}$$

where

$$
\Lambda \approx \mathbf{0.37} \tag{35}
$$

Note that *Pmax* is the *input* power maximum, which is derived from voltage and heat safety limits which will apply at the beginning of the wire. The power transmitted needs to take into account the losses in the wire, which relates to the length. Since we have derived the limits based on power lost to heat, the output power maximum simply becomes

$$P\_{\max,out} = \frac{\Lambda dr^2}{s} \sqrt{\frac{1}{\rho\_{\text{wire}}} H\_{\max} \left(r\right)} - lH\_{\max} \left(r\right) \tag{36}$$

Finally, we can write an equation for the efficiency of transmission *η*, which relates the input power and the power lost to Joule heating in the wire. Note that this relationship does depend on length, as the total power lost to heat increase per unit length of wire.

$$\eta\_{\text{wire}}\left(r\right) = 1 - \frac{lH\_{\text{max}}\left(r\right)}{P\_{\text{max}}\left(r\right)}\tag{37}$$

## Results

In this section, we evaluate and compare the previously derived models of heat limits, mechanical transmissions, and electrical transmissions using representative values of tissue constants and design parameters that apply to minimally invasive surgery. We first establish the mechanical performance target (including force and power) that a transmission for MIS surgery is attempting to achieve, then explore the ability of mechanical and electrical transmissions to meet that performance target. For both transmissions, we examine the corresponding efficiency for different access constraint geometries when achieving the identified performance target. Finally, we also consider additional performance or safety related metrics for each of the transmission types; namely degrees of freedom for the mechanical transmission, and voltage levels for the electrical transmission.

## Mechanical Performance Targets for Surgical Manipulations

This analysis is meant to provide suitable models and parameters to aid in the design of transmissions for minimally invasive surgical robots. Therefore, it is useful to establish a mechanical performance target that, if the transmission met this target, there would be a reasonable assumption that a surgery could be carried out.

Mechanical power requirements of surgical manipulation tasks are not stated directly in the literature, though we can use independent reported task measurements to estimate an upper bound. The BlueDRAGON system has been used to measure surgeon motions and interaction forces during minimally invasive tasks (Markvicka, 2014), and reports mean and SD handle velocities (about the trocar) of 0.047 rad/s ± 0.056 rad/s while grasping during a bowel handling task (Brown et al., 2004). The forces measured, however, were at the tool handle, so are not representative of the tool/tissue interaction forces. Wagner et al. reports a histogram of forces for a minimally invasive gall bladder blunt dissection task, where all forces with a duration longer than 100 ms were below 10 N (Wagner et al., 2007).

If we assume a distance of 0.15 m from port to tool tip in (Brown et al., 2004) (half the length of a standard MIS tool shaft), and calculate the velocity that accounts for 95% of all grasp motions, this gives

$$\begin{array}{l}\nu = \left(0.047 + 2 \times 0.056\right) \times 0.15\\= 0.024 m/s\end{array} \tag{38}$$

Combining this upper bound velocity with the upper bound force limit gives a conservative upper bound mechanical power target for continuous manipulation of

$$\begin{array}{l}P\_{\text{target}} = F\nu\\ = 10 \times 0.024\\ = 0.24W \end{array} \tag{39}$$

We will use this coarse power target estimate as a baseline to compare the achievable power limits of the mechanical and electrical transmissions. We emphasize that this estimate is a continuous power upper bound; peak power demands may exceed these values.

## Heat Limits in Tissue

The fundamental limit to power delivery into the body relates to efficiency of the corresponding transmission, and the ability of

Wagner and Emmanouil Transmission Efficiency Limits for Surgical Robotics

Table 4 | Tissue thermal properties from (Hasgall et al., 2015).


the body to dissipate the excess heat. Using representative tissue thermal properties (**Table 4**), and the maximum allowed body temperature of 43 degrees C in steady state (EN 60601-1, Clause 11.1), we can derive heat limits for different radii of cylindrical tool transmissions (**Figure 4A**). This model assumes sufficient tissue surrounding the cylinder to dissipate the heat (**Figure 4B**) - if this amount of tissue is not available, more stringent limits should be used. We observe that the amount of heat that tissue can dissipate around a cylinder depends significantly on the type of tissue, and varies somewhat linearly with radius.

## Mechanical Tendon/sheath Transmission for MIS Surgical Robotics Efficiency

Because the main efficiency expression (9) relates many effects, we explore the relative magnitude of these effects by choosing a plausible operating point (**Table 5**) for MIS surgical robotics, then vary individual parameters around that operating point. We consider a transmission that might be used for a colonoscopy procedure - a relatively long access path through the bowel of 0.5 m, with a continuous path radius of 0.1 m. We assume use of stainless steel wire rope as the tendon material, with a low coefficient of friction against the sheath (using the stainless steel/ Teflon coefficient of friction of 0.04). We assume a tendon pull of distance 0.01 m, and a similar tendon radius (0.22 mm) as cables used on current surgical robotic tools (Friedman, 2011), as well as similar pretension (9% of working limit, approx 3.2 N).

Table 5 | Default parameters for tendon/sheath efficiency evaluation.


The model reveals several interesting trends relating design parameters to efficiency, beyond those intuitively expected. First, increasing pretension decreases efficiency (**Figure 5A**), which is a tradeoff with other design effects such as tolerancing and backlash. Low force and low distance motions suffer in efficiency (and increasing burden on control), as losses due to cable stretch and friction dominate work delivered (**Figure 5B**). Path length and radius both serve to reduce efficiency, but as a ratio (**Figure 5C**). Finally, stiffness of the tendon from material property and cross sectional area serves to increase efficiency (**Figure 5D**).

### Degrees of freedom

An estimate of maximum degrees of freedom, using the design parameters listed in **Table 5** is shown in **Figure 6**. This is an estimate of the achievable degrees of freedom using the *same* tendon radius as the operating point described above, so a similar per-degree of freedom performance. Similarly, if we make the assumption that the sheath material is continuous, then the achievable minimum bend radius can be calculated using estimates of strain at yield (**Figure 6B**).

Examining these results, the achievable degrees of freedom increase exponentially for a small increase in tool radius. However, the predicted minimum bend radius for the listed materials also increases exponentially. This model result highlights a common design principle: the tradeoff between the performance benefits achieved by stiff

FIGURE 5 | Effects of various tendon/sheath design parameters on efficiency, for default parameters given in Table 5. X's represent parameters used in other graphs. (A) Effects of pretension and pull distance. (B) Effects of input tension and friction coefficient. (C) Effects of path length and radius. (D) Effects of tendon material and size.

system.

materials and the corresponding achievable minimum bend radius. Strategies to mitigate this usually involve trading off some amount of performance, or increasing the overall tool radius (to add material cross sectional area to achieve axial stiffness) but incorporating small radius articulating regions to achieve lower bend radius. Note that the large achievable strain and relatively high stiffness make nitinol an ideal candidate for monolithic sheaths.

#### Power and force limits

The required velocity to achieve the heat-limited power throughput increases exponentially as the tendon radius decreases (**Figure 7B**). For the operating geometry and materials listed above, the velocities needed to reach heat damage to tissue are significantly higher than our target tool tip velocity of 24 mm/s. However, the working limit of tendons, at the sizes that are currently used in tools, is close to the 10 N force target (**Figure 7A**). This indicates that mechanical transmissions, for this size scale and application, are largely force limited, not power limited.

## Electrical Transmission for MIS Surgical Robotics

Using the electrical transmission models derived previously, we examine the performance of an electrical transmission under the same access constraints and heat limits as used for the mechanical tendon/sheath investigation. We also investigate current and voltage limits, as these can relate to other design considerations such as actuator compatibility and safety.

#### Current and voltage limits

Using similar tissue properties as above, we can find current limits for different radii of the coaxial wire as limited by the heat

properties of tissue (**Figure 8A**). Current scales somewhat linearly with outer radius for the size scales relevant to MIS surgery.

Voltage limits can be determined using cross sectional area, insulator dielectric properties, and an assumed safety factor. For this analysis, we use a safety factor of 5 to match that used in the mechanical analysis. Achievable voltages for PVC (dielectric constant 15 *kV*/*mm*) and PTFE (dielectric constant 60 *kV*/*mm*), biocompatible insulator materials with a range of formulations for flexibility and toughness, are shown in **Figure 8B**, with an air gap (dielectric constant 3.0 *kV*/*mm*) insulator for reference. Note that significant voltages can be achieved even with a small insulation width.

## Power and efficiency limits

With the same path length and tissue properties as used in the above mechanical transmission analysis, we can estimate the maximum power output for a coaxial cable with similar access constraints. Assuming a copper cable (with resistivity of 1.68 *<sup>×</sup>* <sup>10</sup>*−*<sup>8</sup> <sup>Ω</sup>*m*) with PVC insulation, the maximum power achievable for a 0.5 m wire length is shown in **Figure 9A**. Note that the maximum power capacity for this coaxial wire far exceeds the mechanical power requirements estimated earlier.

Similar to above, the efficiency in different tissues across outer radius is shown in **Figure 9B**. We observe that the efficiency of a wire transmission can exceed 99%, even for wires with *l* = 1 m and *r* < 0.2 mm, and subject to safety constraints within the body.

## Discussion

The work presented identifies the efficiency limits of transmissions used for small access diameter surgical robotics. We present efficiency and power limit models for tendon/sheath mechanical transmissions and electrical wire transmissions derived from first principles and from existing models in the literature. The models incorporate access constraints as well as safety limits required when implementing surgical systems in a medical device design context. We then used the models to investigate achievable power and efficiency limits using representative values for minimally invasive robotic surgery.

The results of the analysis validated the initial hypothesis - for longer path lengths, higher path curvature, and higher pretension, the mechanical transmission decreased in efficiency. A key observation is that the decrease was significant in the size scales and parameters relevant for MIS surgery. Mechanical transmissions achieved 85% efficiency for shorter, straighter paths, to below 25% for longer, curved paths with higher pretension.

For similar path geometries and access constraints, electrical coaxial wire (coax to provide a current return path and an isolation barrier) achieved high efficiencies - over 99% efficient power transfer, even with small diameter access. Again, the key observation is that this efficiency result holds for the sizes relevant to MIS surgical robotics. An important qualification, however, is that this efficiency does not take into account the efficiency of the actuator, which is likely to be lower than the corresponding efficiency of the mechanism required to convert tendon motion into tool motion. We discuss this further in the next section.

We also investigated the ability of the two transmissions to deliver other performance criteria required for surgical manipulations beyond efficiency; specifically, force, velocity and power. We observed that, for current materials used in tendon/sheath construction, the tendon working limit is near the force limit required for surgical manipulations. The velocities of the cables at those working limits, however, are much smaller than those imposed by the heat limit, implying the ability of a tendon/sheath system to deliver significantly more absolute power if tendon velocities are increased. Thus, tendon/sheath transmissions will struggle to deliver the same performance for longer access paths or smaller cross-sectional area if tendons are used in the same manner (at forces and velocities similar to those needed for surgery).

The electrical transmission, from an absolute power delivery standpoint, has the ability to deliver significantly more power than required for surgical manipulations. For example, a 1 mm radius coaxial wire as described can deliver over 10 kW of power in a safe manner, which is many orders of magnitude above the target 0.25 W mechanical power needed for surgical manipulations. Again, the limiting factor in this case would be the actuator, not the transmission, and 10 kW of continuous power could not be dissipated by the body. However, this highlights the potential opportunity for smaller MIS robots, given sufficient actuation technology.

## Candidate Actuators for Locating Inside the Body

The results presented show that the efficiency of a wire to transmit power is high even under the access constraints posed by MIS surgery; however, creating a full minimally invasive surgical robot based on an electrical transmission requires locating the actuators on the inside of the body. While a complete analysis of optimal actuator technology for surgical robotics is out of scope for this paper, it is worthwhile to mention candidate actuation technologies with sufficient work-density characteristics that could enable practical implementations of existing surgical tools.

Overviews of actuator technology show several smart material actuation technology that are significantly more work dense than traditional moving coil actuators (Huber et al., 1997). These include piezoelectric actuators and shape memory alloy. Piezoelectric technology is both significantly more work dense than traditional motors (up to 10<sup>9</sup> *W*/*m*3 as compared to <sup>2</sup> *<sup>×</sup>* 106 *<sup>W</sup>*/*m*3) and more efficient (above 99% as compared to 50–80%). However, the drawback is that the strain achieved per stroke is small, so some additional transmission would be required to convert the output into forces and displacements useful for the task.

Similarly, shape memory alloy has a higher work density than moving coil transducers (up to 108*W*/*m*<sup>3</sup> ), but has some drawbacks. Primarily, shape memory alloys rely on a thermal effect to generate actuation, and so are poor in terms of efficiency (1–2%). Ongoing research, however, is examining approaches for increasing the efficiency and thermal properties of shape memory alloy (Thrasher et al., 1994; Pathak, 2010; Nespoli et al., 2010; Salerno et al., 2014; Khan et al., 2016), making it a better candidate for an internal actuator.

MIS robotic manipulators incorporating both miniature moving coil actuators and SMA actuators have been developed. Mineta et al. and Takayama et al. (Takayama et al., 1997; Mineta et al., 2001) developed catheter-like manipulators incorporating SMA actuators. In their work they showed it is possible to practically incorporate SMA actuators in MIS positioners that could for example guide a monopolar electrosurgery tool for dissection and cauterisation operations. Hideki and Salerno (Okamura et al., 2009; Salerno et al., 2014) explored the possibility of employing SMA actuators in robotic grippers with encouraging results. Lee et al. and Yeung et al. (Yeung and Gourlay, 2012; Lee et al., 2014) designed small robotic tools with moving coil actuators integrated in the tool and robot body and could apply up to 10 N of grasping force. This strategy could be better suited for tools such as grippers and needle drivers, and even staplers with an appropriate reduction ratio.

## Model Limitations

Our investigation focused on the power and efficiency limits of power transmission into the body. These are not the only source of limiting factors to consider, however, when designing a surgical robot. One of the main omissions of the previous analysis is establishing a mechanical ground against which to apply force. There are a number of solutions that exists whose application depends on the specific surgical manipulation. For example, multiple robots entering the body from different ports can increase stiffness of the base (Mahoney et al., 2017). Additionally, forces internal to the robot do not require an external mechanical ground, so multiple armed systems can be effective. Finally, the robot can use alternative anchoring strategies, such as cuffs or balloons, to establish mechanical ground distally.

The efficiency limits developed for tendon/sheath transmissions considered a number of material property and geometry effects. Two similar effects not incorporated include the stiffness of the sheath material and the effects of the surrounding tissue. If these stiffnesses were low compared to the tendon stiffness, these would further reduce the observed efficiency of the system. Further, we didn't consider effects of any secondary transmissions, such as a pulley at the distal end to enable jaw rotation.

Similarly, we also did not consider all electrical transmission effects. We assumed a DC current in our analysis, but AC current may be more practical, depending on the actuator technology. If AC power delivery was warranted, then skin depth effects (and known mitigations such as Litz wire) should also be considered. Further, we did not account for data transmission effects - if the same conductor was used to transmit control commands as well as power, conductor effects might limit data transmission bandwidth which could potentially limit the available degrees of freedom.

## Future Directions

The results for the mechanical tendon/sheath transmission showed that current systems work close to the force limits of the materials, but far away from the heat limits of the body. This implies that more power can be transmitted into the body for the same access geometry, if the transmission operated at higher velocities. This comes with its own set of challenges - if a tendon is operating with a direct link to an output degree of freedom, changes in desired output direction would require a change in direction of a high velocity tendon. This would require low backlash and low tendon inertia; properties that are difficult to achieve with today's tendon materials. Further, that high tendon velocity would need to be converted into a lower velocity output motion, requiring an additional transmission (in the gearing sense) at the output.

This leads to the related consideration of busing - using the same power delivery line for multiple degrees of freedom. Or, put another way, allowing all of the power delivery potential of the transmission cross sectional area to pass through a single degree of freedom. This is straightforward to imagine in the case of electrical transmission to multiple switched actuators, but more difficult to envisage in the mechanical context. One mechanical busing scheme to consider is a hydraulic transmission, with a series of controlled valves to gate power to the corresponding degree of freedom. Identifying the control and valve technology remains a challenge, but hydraulics have the advantage of graceful failure modes on puncture - assuming biocompatible hydraulic fluids and a sufficiently stiff delivery tube, high pressures will quickly dissipate as the working fluid is incompressible. Mechanical busing could potentially change how power is delivered mechanically, which is not force limited, and could allow transmission around an operating point of peak efficiency.

The electrical efficiency models presented in this work motivate the opportunity for continued investigation into sufficiently efficient and work-dense electrical actuators for MIS surgical robotics. Correct use of these actuators can enable equal or better mechanical performance as mechanical cable drives, with little of the external mechanical infrastructure and size required. This points to a vision for the future of surgical robotics, where all of the MIS benefits can be delivered with a small robotic system.

## References


Finally, we observe that for both mechanical and electrical transmissions, efficiency estimates indicate that current systems are far from the allowable continuous heat limits that can be safely accounted for by the body. This highlights the opportunity to further decrease access size, and increase the range of procedures that the benefits of surgical robotics can be applied to.

## Author Contributions

CW derived models, carried out the analysis, and was the primary author of the text. EE carried out additional analysis, verification of results, and contributed to the text. Both were involved in developing the underlying ideas and direction of the work, revising the manuscript, and approving the submitted version.

## Acknowledgments

We would like to thank a number of colleagues for helpful review and discussion of this paper, including Al Mashal, Dan Cowan, Alan Sanders, Donal Taylor, Rodrigo Zapiain, Baudouin Geraud, Tom Parker, Steve Gardner, Rob Rudolph, Matt Neighbour, Georgina Koffler, Sergio Malorni, Marco Rizzardo, and Simon Karger.


**Conflict of Interest Statement:** Both authors were employed by the company Cambridge Consultants, Ltd during the writing of the paper.

*Copyright © 2018 Wagner and Emmanouil. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Appendix

## Heat propagation through a cylinder

Here we present the derivation of an analytic model of the local heat propagation from a cylinder (such as a wire) into surrounding tissue in steady state using the bioheat equation (Incropera et al., 2011). The derivation of the presented here initially follows the one presented by Yue et al (Yue et al., 2004), a solution of the bioheat equation in steady state in cylindrical coordinates. We solve for different boundary conditions, however, allowing the analysis of the heat flux through the heat-generating wire. Several main results are presented here: (1) the analytic solution to the bioheat equation given boundary conditions of two known temperatures at fixed radii, (2) a simplification of this result with the outer radius set at infinity, and (3) the corresponding heat flux through the wire given a temperature constraint to nearby tissue. Please see **Table 1** for nomenclature used.

More formally, given a cylindrical pipe of tissue with outer radius *r*2 and inner radius *r*1, and with known temperatures at the radii *T*2 and *T*1 respectively, determine the corresponding heat flow into the cylindrical pipe through the inner radius, at steady state.

We start with the bioheat equation in cylindrical coordinates for one dimension, in steady state, given by:

$$\frac{1}{r}\frac{d}{dr}\left(r\frac{dT}{dr}\right) + \frac{\omega\rho\_b\varepsilon\_b}{k}\left(T\_a - T\right) + \frac{\dot{q}\_m}{k} = 0\tag{40}$$

From this equation, temperature *T* at a particular radius *r* is related through the main effects of diffusion, perfusion from capillary action, and metabolic heat (*q*˙*m*). Other parameters include *<sup>ω</sup>* the perfusion rate [*m*3/(*<sup>s</sup> · <sup>m</sup>*3)] of blood through capillaries in a particular tissue, *ρb* the density (*kg*/*m*<sup>3</sup> ) of blood, *cb* the specific heat [*J*/ ( *kg · K* ) ] of blood, *k* the tissue thermal conductivity [*W*/ ( *m · K* ) ], and *Ta* arterial blood temperature.

To have a safe limit that applies even if the tissue is deep within the body, we are deriving a solution such that all of the heat put into the system through *r*1 is removed through perfusion effects. This has the advantage of avoiding a convective boundary condition, which are difficult to parameterise for internal tissue interactions. Also, we assume steady state to analyze the worst case limit of continual heat input into the body. We set the metabolic term *qm* to 0 for this analysis, as its effect will be negligible.

To solve (40), first use a series of substitutions to rewrite the equation into a differential equation form with a known solution. If we let

$$\begin{array}{c} B = \sqrt{\frac{\omega \rho\_b c\_b}{k}}\\ A = B^2 T\_d\\ \Phi \left( r \right) = A - B^2 T \left( r \right) \end{array} \tag{41}$$

and substitute into the bioheat equation (40), we are left with a differential equation of the form

$$\frac{d^2\Phi}{dr^2} + \frac{1}{r}\frac{d\Phi}{dr} - B\Phi = 0\tag{42}$$

By inspection, this is a zero-order modified Bessel differential equation with known solution given by

$$\Phi\left(r\right) = C\_1 I\_0 \left(Br\right) + C\_2 K\_0 \left(Br\right) \tag{43}$$

where *I*0 is a zeroth order modified Bessel function of the first kind and *K*0 is a zeroth order modified Bessel function of the second kind. To solve for the constants *C*1 and *C*2, we use the boundary conditions of known temperature at specified radii

$$T\left(r\_1\right) = T\_1\tag{44}$$

$$T\left(r\_2\right) = T\_2\tag{45}$$

along with the above substitutions (41). Solving the two resulting equations gives

$$C\_1 = \frac{B^2 \left(-T\_1 K\_0 \left(Br\_2\right) + T\_2 K\_0 \left(Br\_1\right) - T\_a K\_0 \left(Br\_1\right) + T\_a K\_0 \left(Br\_2\right)\right)}{I\_0 \left(Br\_1\right) K\_0 \left(Br\_2\right) - I\_0 \left(Br\_2\right) K\_0 \left(Br\_1\right)}\tag{46}$$

$$C\_2 = \frac{B^2 \left(\left(T\_1 - T\_a\right) I\_0 \left(Br\_2\right) - \left(T\_2 - T\_a\right) I\_0 \left(Br\_1\right)\right)}{I\_0 \left(Br\_1\right) K\_0 \left(Br\_2\right) - I\_0 \left(Br\_2\right) K\_0 \left(Br\_1\right)}\tag{47}$$

Substituting this result into (43) gives the analytic solution for the bioheat equation in steady state, for fixed temperature boundary conditions:

$$\begin{aligned} T\left(r\right) &= \frac{T\_a\left(I\_0\left(Br\_1\right)K\_0\left(Br\_2\right) - I\_0\left(Br\_2\right)K\_0\left(Br\_1\right)\right)}{I\_0\left(Br\_1\right)K\_0\left(Br\_2\right) - I\_0\left(Br\_2\right)K\_0\left(Br\_1\right)} \\\\ &- \frac{\left(T\_1 - T\_a\right)I\_0\left(Br\_2\right) - \left(T\_2 - T\_a\right)I\_0\left(Br\_1\right)}{I\_0\left(Br\_1\right)K\_0\left(Br\_2\right) - I\_0\left(Br\_2\right)K\_0\left(Br\_1\right)}K\_0\left(Br\right) \\\\ &+ \frac{T\_1K\_0\left(Br\_2\right) - T\_2K\_0\left(Br\_1\right) + T\_aK\_0\left(Br\_1\right) - T\_aK\_0\left(Br\_2\right)}{I\_0\left(Br\_1\right)K\_0\left(Br\_1\right) - I\_0\left(Br\_1\right)K\_0\left(Br\_1\right)}I\_0\left(Br\right) \end{aligned} (48)$$

However, using this solution directly can lead to unrealistic results when trying to derive heat flux limits at *r*1, as nothing in the solution limits the heat flux at *r*2. To overcome this limitation, solve (48) in the limit as *r*2 goes to infinity. At this limit, we can safely assume there is no external heat sink effect biasing the flux calculation at *r*1, and all heat removal contributions are from perfusion and diffusion. Given that we will use the maximum safe steady state temperature in the body at *r*1, the heat power limit estimate will hold given that the temperature falls off within reasonable distances.

*I*0 ( *Br*1 ) *K*0 ( *Br*2 ) *−I*<sup>0</sup> ( *Br*2 ) *K*0 ( *Br*1

Knowing that *limx→∞I*<sup>0</sup> ( *x* ) = *∞* and *limx→∞K*<sup>0</sup> ( *x* ) = 0 lets us derive the following expression for *limr*2*→∞Tfull* ( *r* ) by inspection, which we term *Tinf* ( *r* ) :

$$T\_{inf}\left(r\right) = T\_a + \frac{T\_1 - T\_a}{K\_0 \left(Br\_1\right)} K\_0 \left(Br\right) \tag{49}$$

*Tinf* ( *r* ) is the analytic solution of the bioheat equation given no outer radial constraint on temperature. Note that for this limit, *T*2 is not a parameter - it naturally falls out as body temperature (*Ta*).

Using *Tinf* ( *r* ) , we can now use Fourier's law of heat conduction to relate the rate of temperature change (*dT*/*dr*) to the power flux (*Q*˙ ) due to heat through *r*1. To be clear on units, we are using *Q*˙ to denote heat power (in units of Watts) and *q*˙ to denote heat flux in units of Watts per square meter, where *q*˙ = *Q*˙ /*a* where *a* is area.

The one dimensional version of Fourier's law, in cylindrical coordinates, is:

$$
\dot{q} = -k \frac{dT}{dr} \tag{50}
$$

where *k* is the thermal conductivity. Substituting in for heat flux gives:

$$\frac{\dot{Q}}{a} = k \frac{dT}{dr} \tag{51}$$

Because we are solving for heat flux through a cylinder, use *a* = 2*πrl*, where *l* is the length of the cylinder.

$$\frac{\dot{Q}}{2\pi r l} = k \frac{dT}{dr} \tag{52}$$

Then, rearrange to find *Q*˙ /*l*, which is the heat power per unit length of the cylinder, which we will define as H(r):

$$H\left(r\right) = \frac{\dot{Q}}{l} = 2\pi rk \frac{dT}{dr} \tag{53}$$

This is the heat power that is transmitted through a cylinder, per unit length, for a given temperature differential and thermal conductivity *k*. Carrying out the derivative *dTinf*( *r* ) *dr* and substituting into (53) gives

$$H\_{\text{max}} = 2\pi k Br\_1 \frac{K\_1 \left(Br\_1\right)}{K\_0 \left(Br\_1\right)} \left(T\_1 - T\_a\right) \tag{54}$$

where

$$B = \sqrt{\frac{\omega \rho\_b c\_b}{k}}\tag{55}$$

which is an analytic result to estimate the maximum heat power (in W/m) that specific tissues in body can can safely dissipate for a given cylindrical geometry.

## Tendon/sheath transmission force balance

Here we derive a simple tendon/sheath model that relates input and output tensions as parameterised by a radial path geometry, and a friction coefficient relating tendon tension with friction forces. This is the same model as presented by Palli et al. (Palli and Melchiorri, 2006; Palli et al., 2009), but we re-derive the solution to the model to be explicit about the contribution of pretension (not in the original derivation) so that it can be correctly incorporated into an expression for efficiency.

As stated in (Palli and Melchiorri, 2006), we model the force balance of a small section of tendon (**Figure 2**):

$$
\Delta T = -F\_f = -\mu \text{Nsign}\left(\dot{\epsilon}\right) \tag{56}
$$

where ∆*T* is the change in tension, *Ff* is the force due to friction, *µ* is the friction coefficient, and *N* is the normal force. *sign* ( *ϵ*˙ ) ensures that friction force is acting opposite the direction of motion of the tendon. We will neglect this direction term for the remaining analysis, as we are focusing on deriving an expression for efficiency limits which can be derived sufficiently from a single-direction analysis.

The normal force *N* is given by:

$$N = T\Delta\gamma = T\frac{\Delta x}{R} \tag{57}$$

where ∆*γ* is the subtended angle, ∆*x* is the length, and *R* is the radius of curvature of the tendon element. Then, for an infinitesimal section of tendon:

$$dT = -\mu T \frac{d\mathbf{x}}{R} \tag{58}$$

Solving for *T* ( *x* ) gives

$$T\left(\mathbf{x}\right) = T\_{\text{int}}e^{-\frac{\mu\mathbf{x}}{\mathcal{R}}}\tag{59}$$

where *Tin* is the input tension at *x* = 0.

Similarly, to solve for the cable stretch *δ* as a function of distance, we start with the stretch along the tendon section:

$$
\Delta \delta = T \begin{pmatrix} x \\ \end{pmatrix} \begin{matrix} \Delta x \\ \end{matrix} \tag{60}
$$

where *E* is the elasticity of the tendon material, and *A* is the tendon cross sectional area. For the infinitesimal tendon section, the differential equation becomes:

$$\frac{d\delta}{dx} = \frac{1}{EA} T\left(x\right) \tag{61}$$

whose solution is:

$$\delta\left(\mathbf{x}\right) = \frac{T\_{\text{ff}}R}{EA\mu} \left(1 - e^{-\frac{\mu x}{R}}\right) \tag{62}$$

Thus, our model to relate tensions and tendon stretch, for a tendon of length *L*, path radius *R*, and friction coefficient *µ* is:

$$\begin{array}{c} T\_{out} = T\_{in} e^{-\frac{\mu L}{R}}\\ \delta = \frac{T\_{in} R}{EA\mu} \left(1 - e^{-\frac{\mu L}{R}}\right) \end{array} \tag{63}$$

Again, this model accounts for tension and stretch for single direction motions, but does not account for hysteresis effects when tendon motion reverses direction.

Now, we introduce an explicit pretension term *T*0 as a component of both input and output tension, where *Tw* is the remaining tension that is doing work:

$$\begin{cases} T\_{in} = T\_{in,\omega} + T\_0\\ T\_{out} = T\_{out,\omega} + T\_0 \end{cases} \tag{64}$$

Solving for *Tout*,*w* is then

$$\begin{aligned} T\_{out,w} &= T\_{out} - T\_0 \\ &= T\_{in}e^{-\frac{\mu L}{R}} - T\_0 \\ &= \left( T\_{in,w} + T\_0 \right) e^{-\frac{\mu L}{R}} - T\_0 \\ &= T\_{in,w}e^{-\frac{\mu L}{R}} - T\_0 \left( 1 - e^{-\frac{\mu L}{R}} \right) \end{aligned} \tag{65}$$

We also carry out the same separation for the tendon stretch *δ* into a prestretch term *δ*0 and the stretch due to the input work tension *δw*:

$$\begin{array}{lcl}\delta\_{\mathsf{W}} &= \delta - \delta\_{0} \\ &= \frac{\left(T\_{in,\mathsf{w}} + T\_{0}\right)R}{EA\mu} \left(1 - e^{-\frac{\mu L}{R}}\right) - \frac{T\_{0}R}{EA\mu} \left(1 - e^{-\frac{\mu L}{R}}\right) \\ &= \frac{T\_{in,\mathsf{w}}R}{EA\mu} \left(1 - e^{-\frac{\mu L}{R}}\right) \end{array} \tag{66}$$

# The Making of a 3D-Printed, Cable-Driven, Single-Model, Lightweight Humanoid Robotic Hand

#### *Li Tian1 , Nadia Magnenat Thalmann1 \*, Daniel Thalmann2 and Jianmin Zheng1*

*1Nanyang Technological University (NTU), Singapore, Singapore, 2École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland*

Dexterity robotic hands can (Cummings, 1996) greatly enhance the functionality of humanoid robots, but the making of such hands with not only human-like appearance but also the capability of performing the natural movement of social robots is a challenging problem. The first challenge is to create the hand's articulated structure and the second challenge is to actuate it to move like a human hand. A robotic hand for humanoid robot should look and behave human like. At the same time, it also needs to be light and cheap for widely used purposes. We start with studying the biomechanical features of a human hand and propose a simplified mechanical model of robotic hands, which can achieve the important local motions of the hand. Then, we use 3D modeling techniques to create a single interlocked hand model that integrates pin and ball joints to our hand model. Compared to other robotic hands, our design saves the time required for assembling and adjusting, which makes our robotic hand ready-to-use right after the 3D printing is completed. Finally, the actuation of the hand is realized by cables and motors. Based on this approach, we have designed a cost-effective, 3D printable, compact, and lightweight robotic hand. Our robotic hand weighs 150 g, has 15 joints, which are similar to a real human hand, and 6 Degree of Freedom (DOFs). It is actuated by only six small size actuators. The wrist connecting part is also integrated into the hand model and could be customized for different robots such as Nadine robot (Magnenat Thalmann et al., 2017). The compact servo bed can be hidden inside the Nadine robot's sleeve and the whole robotic hand platform will not cause extra load to her arm as the total weight (150 g robotic hand and 162 g artificial skin) is almost the same as her previous unarticulated robotic hand which is 348 g. The paper also shows our test results with and without silicon artificial hand skin, and on Nadine robot.

Keywords: robotic hand, modeling, 3D printing, cable-driven system, grasp planning

## INTRODUCTION

The idea of automata was created very early in the human history more than 2,000 years ago. In China, Lu Ban made an artificial bird, which was able to fly with its wings (Needham, 1974). The Greek engineer, Ctesibius, applied knowledge of pneumatics and hydraulics to produce the first organ and water clocks with moving figures (Rosheim, 1994). After that, various mechanical designs were produced towards a trend of complexity and precision. "The Writer automaton" was built in the 1770s using 6,000 moving parts by Pierre Jaquet-Droz (Percy and Timbs, 1840). It can write any custom text up to 40 letters long, and text is coded on a wheel where characters are selected one by one. Actually, the writer basically fit the definition of a "Robot" from the Robot Institute of America (1979): "A

#### *Edited by:*

*Jörn Malzahn, Fondazione Istituto Italiano di Technologia, Italy*

#### *Reviewed by:*

*Kensuke Harada, Osaka University, Japan Vincent Wall, Technische Universität Berlin, Germany*

*\*Correspondence:*

*Nadia Magnenat Thalmann nadiathalmann@ntu.edu.sg*

#### *Specialty section:*

*This article was submitted to Humanoid Robotics, a section of the journal Frontiers in Robotics and AI*

*Received: 18 May 2017 Accepted: 21 November 2017 Published: 04 December 2017*

#### *Citation:*

*Tian L, Magnenat Thalmann N, Thalmann D and Zheng J (2017) The Making of a 3D-Printed, Cable-Driven, Single-Model, Lightweight Humanoid Robotic Hand. Front. Robot. AI 4:65. doi: 10.3389/frobt.2017.00065*

**91**

reprogrammable, multifunctional manipulator designed to move materials, parts, tools, or specialized devices through various programmed motions for the performance of a variety of tasks." However, its hands are not articulated. A human hand is the most articulated parts of the human body and grasping is one of the most common and important gestures that humans use when interacting with surrounding objects. The robotic hand is also a kind of automaton or robot. However, not much work about robotic hand can be found in the history until the first modern industrial robots "Unimates" in 1960s (Siciliano and Khatib, 2016). Since the creation of the first humanoid robot Eric (Riskin, 2016), researchers have been aiming to develop dexterity robotic hands for humanoid robots to make them grasp-like humans. Many different styles of robotic hands have been fabricated over the last 30 years. As electricity had been widely used as an easy-get and high-efficiency power resource, most of them are actuated by electrical motors or pneumatic motors (Siciliano and Khatib, 2016). It is undeniable that robotic hands have come close to replicating human hands. However, no such robotic hand works exactly like a human hand in terms of appearance and physical characteristics, which include movement and force. From the mechanical perspective, the main difficulty comes from two aspects, mechanism and actuation. First, the human hand has 15 movable joints, of which, some joints have more than one degree of freedom (DOF). These characteristics make it difficult to model and fabricate a human-like robotic hand. We also need to take into account the motion range of different joints as they have different limitations. Second, muscles and tendons drive our body motion in high efficiency (Woledge, 1998). There is currently no good solution to simulate the force with precise motions of fingers.

The state-of-the-art robotic hands usually have complex mechanical structures and control methods (Melchiorri et al., 2013) (Xu and Todorov, 2016). The development of 3D print technology, in some ways, has reduced the cost and complexity of making dexterity robotic hand. There are several robotic hands that can be 3D printed (Slade et al., 2015; ten Kate et al., 2017). However, according to their test report, the functionalities such as movable joint, DOF, and motion range, are not as good as these state-of-the-art robotic hands. Our robotic hand aims to mimic the grasping behavior of the human hand while staying simple in making and control of the robotic. With the help of Fused Deposition Modeling, 3D printed rigid endoskeleton (Tavakoli et al., 2017) and functional articulations non-assembly joint (Cali et al., 2012) are easy to apply to the robotic hand. It is also possible to integrate every joint into a single articulated hand 3D model. Human fingers have bones and tendons but do not contain muscles (Agur and Dalley, 2009). The muscles that move the finger joints are in the palm and the forearm. The muscles actuate the fingers through long elastic tendons, which are linked to the finger bones. This paper analyses the anatomy of the human hand and proposes a mechanical model of robotic hands with considerations of DOF and constraints. Further assisted by advanced geometric modeling techniques, we have created a new robotic hand 3D model. Compare to other 3D printed robotic hands; our robotic hand has a neat and efficient actuation system, which reproduced all human hand's joints and their motion range. The fabrication of our robotic hand is simple with low cost. We also controlled the weight of our robotic hand and made it compatible with different robots. Compared to the human hand, the robotic hand has adopted similar mechanical characteristics and motion range of each joint. The cable-driven method is created to mimic this human motion system. The servo motors, cables, and 3D printed parts function like muscles, tendons, and bones, respectively. We use six servo motors to actuate the fingers in a cable-driven system. The interlocking design of fingers and thumb provide a simple and practical way to simulate human grasps. In particular, our robotic hand uses a modular design, which makes it easy to mount on the Nadine robot or other humanoid robots. The experiments show that the created robotic hand can grasp different objects using plenty of gestures, which are based on hand taxonomy.

With advancement in the development of artificial silicon skin, the latest humanoid robots look more and more humanlike (Hirukawa, 2005). However, it also brings the "uncanny valley" (Mori, 1970) problem. The behaviors of the humanoid robot, like natural grasp, can help solve this issue. The artificial skin increases the force required to actuate fingers. We tested the different combination of the artificial skins and the actuators. We also specifically implemented it on Nadine robot, and our results are shown in this paper Section "Grasp experiments with artificial silicon skin."

The rest of the paper is organized as follows. Section "Related Works" gives an overview of existing robotic hands. Section "Designs and Fabrication" details the design and fabrication. Section "Limitation of the Hardware" describes the hardware limitations. Section 5 presents our experiments of evaluating the performance of the design in terms of grasping objects, followed by the conclusion and future work in Section "Conclusion and Future Work."

## RELATED WORKS

A robotic hand can be evaluated by many criteria including DOFs, motion ranges of each joint, accuracy, speed, grasping trajectory, grasping force, weight, and appearance and so on. It is a challenge to make a robotic hand with excellent features in all areas. However, based on the pre-established purpose of a robotic hand, researchers focus on important features for certain usage. We will briefly review past works related to robotic hands in three categories based on their roles. They are Prosthetic Robotic Hand, Research Purpose Robotic Hand, and Humanoid Robot's Robotic Hand.

## Prosthetic Robotic Hand

One statistic report has concluded that low-income countries have 30 million or more people with amputation and most of them cannot afford prosthetic care (Malbran, 2011). A well-functioning prosthetic robotic hand will help them greatly improve the quality of life. For a commercial prosthetic hand designed to help people grip objects used in daily life, the ability to grasp is the first and most important function that designers need to consider. Here, the "grasp" refers to a static posture with an object held securely with one hand (Feix et al., 2009a). There are many different kinds of objects used in our daily lives. Cutkosky (1989) provided a comprehensive and detailed organization of human grasps, which include 15 different postures. Unfortunately, due to the limitation of cost, weight, and high grasping success rate, most commercial prosthetic hands are greatly simplified in design. Commercial prosthetic hands can simulate only a limited number of gestures.

The typical commercial prosthetic hands, such as Bebionic hand, i-Limb hand, are made of laser-cut metal, motors, and screws. It usually comes in the shape of a human hand but is heavier than a human hand. The most common design consists of 11 joints, 6 DOFs (the thumb has one more DOF as compared to the other fingers), and 5 or 6 actuators. The robotic hand's hardware, motion control system, and power supply are integrated together. The grasp is robust and built to last. Bebionic hand (Medynski and Rattray, 2011) (**Figure 1A**) and i-Limb hand (Belter et al., 2013) (**Figure 1B**) are two representatives of commercial prosthetic hand. In recent years, several prosthetic hands also use 3D print technology to build the hand. The designer of the Tact hand (**Figure 1C**) claimed that although their robotic hand cost only US\$250, its performance meets or exceeds those of current commercial prosthetic hands (Slade et al., 2015). There is another 3D printed prosthetic hand called Dextrus hand (Phillips et al., 2015) (**Figure 1D**), which uses the 16-joint design that comprises more joints in each finger, resulting in a more human-like grasp.

## State of the Art Robotic Hand

Many researchers believe that the human hand is the perfectly engineered product of nature (Kalganova et al., 2015). They envision to build a robotic hand, which could function as close as possible to a human hand. Different from the prosthetic robotic hand, this robotic hand does not emphasize much about the weight, cost, and how it is linked to a human arm. The only aspect it focuses on is the simulation of the motion of a human hand to a robotic hand. The Stanford/JPL hand (1983) is one of the first dexterous hands to be invented (Salisbury and Roth, 1983). Most research work related to robotic hands has been conducted before 2010 by H. D. Bos.1

The Shadow Hand (Rothling et al., 2007) (**Figure 2A**) has 40 actuators and 20 DOFs. The UB hand IV (Melchiorri et al., 2013) (**Figure 2B**) is another example of a dexterous hand, which is closest to the human hand regarding functionality. Zhe Xu and his partners have made a low-cost modular, 20-DOF anthropomorphic robotic hand in 2013 (Xu et al., 2013) (**Figure 2C**), and a highly biomimetic robotic hand in 2016 (Xu and Todorov, 2016) (**Figure 2D**).

Without the limitation of the number of actuators used, researchers can simulate more than 20 DOFs in a single robotic hand. Pneumatic control and electric motor control are two common solutions for the actuator of a robotic hand. Although the accuracy of the actuator improved drastically over the past decades, the size and weight of the actuator have not reduced

<sup>1</sup>Evolution of Robot Hands Author: H.D. Bos Supervisor: ir. M. Wassink: https:// www.ram.ewi.utwente.nl/aigaion/attachments/single/363.

Figure 2 | (From left to right) (A) Shadow hand (Rothling et al., 2007), (B) UB hand IV (Melchiorri et al., 2013), (C) Xu Zhe's anthropomorphic robotic hand (Xu et al., 2013), and (D) Xu Zhe's highly biomimetic robotic (Xu and Todorov, 2016).

significantly. Too many actuators will yield a big and heavy robotic hand control system. Therefore, this hinders the use of this form of robotic hand for most daily applications. "Underactuation" is a widely used concept in robotics (Birglen et al., 2007). It means having fewer actuators than the DOFs. Several robotic hands can have only one actuator. Researchers have used hardware lock way (Kontoudis et al., 2015), adaptive synergy (Catalano et al., 2014) to adjust the control of the robotic hand's fingers. In contrast, they have more complex mechanical structure to alter the control of fingers. We will explain our design in chapter three.

## Humanoid Robot's Robotic Hand

Different humanoid robots are made based on a variety of application purposes. The Atlas robot (de Waard et al., 2013) (**Figure 3A**) from Boston Dynamics has been developed for outdoor search and rescue. Several diverse, powerful non-humanoid robotic hands can be linked to its arm one at a time for use in various scenarios. The ASIMO (Sakagami et al., 2002) (**Figure 3B**) by Honda Motor Co., Ltd. has a couple of dexterous humanoid hands, which can open the cover of a cup. Design details of these two state-of-the-art robots are still kept confidential. The i-Cub (Metta et al., 2008) (**Figure 3C**), which was developed by the RobotCub Consortium, has the learning ability to grasp objects. In our opinion, a possible drawback is that they look like a robot more than a human. The Inmoov (Langevin, 2014) robot's most parts are 3D printable (**Figure 3D**), and its hand is one of the references to our robotic hand.

## Conclusion on Robotic Hands and Design Goals

There is no robotic hand currently available that is suitable for the Nadine robot. We cannot adopt non-humanoid robotic hands for the Nadine robot as it needs a couple of humanoid robotic hands to match its physical appearance. Prosthetic robotic hands have a human-like model and robust performance. However, they are too big and heavy for the Nadine robot. Nonetheless, they showed us that fewer DOFs such as five or six could also handle many grasping jobs. The robotic hands for research purpose have superior grasping abilities, but their actuation control systems are too large and complex to be integrated into the Nadine robot.

So we need to design and make a new robotic hand for the Nadine robot, and our design goals are as follows. (1) The total weight of the robotic hand (with actuators) should be less than 300 g. From the previous test of the Nadine robot, an overweighted robotic hand will significantly reduce her arm's motion range. (2) The robotic hand should look like a real hand and have similar joints and motion range to mimic the human grasping gestures. (3) This robotic hand should be simple to make and easy to use.

## DESIGNS AND FABRICATION

In this section, we study the biomechanical features of a human hand such as the bones and joints in order to create robotic hand's model. We apply the constraints of the human hand to simplify its motion model without significantly reducing its functionalities. Then, we design our new 3D hand model based on our understanding of hand motions.

## Human Hand Features

A human hand has a total of 27 bones (Agur and Dalley, 2009). There are five bones in the palm, eight bones in the wrist, and 14 bones in five fingers (**Figures 4A,B**). The thumb consists of three joints named distal-interphalangeal (DIP) joint, interphalangeal joint, and trapeziometacarpal (TM) joint, whereas the other four fingers comprise three joints named DIP joint, proximalinterphalangeal (PIP) joint, and metacarpophalangeal (MCP) joint. Every normal human hand has 15 movable joints, which support the motions of the fingers.

In **Figure 4C**, we show a human hand with a total of 27 DOFs (Magnenat Thalmann et al., 2017). The thumb contributes five DOFs, and 16 DOFs are by four fingers. The remaining six DOFs from the wrist are called "global motion," as they control the motion of the whole hand. All 21 DOFs from the fingers are called "local motion," which controls the motion of the fingers.

## Constraints of Human Hand

Lin et al. (2000) give a formal representation of the constraints of human hands. There are three types of constraints:


Figure 3 | Humanoid robots (first row from left to right): (A) Atlas robot, (B) ASIMO robot, (C) i-Cub robot, and (D) InMoov Robot.

Type III. People have a habit of making standard gestures. We usually bend all the fingers at the same time to make a fist, instead of one by one.

## Simplification of the Hand Model

The five DOFs from the MCP joints, which represent fingers abduction and adduction motion, have a narrow motion range from −15 to 15° (Lin et al., 2000). The experiment from previous research on robotic hand shows that they are rarely involved in the grasping action (Xu and Todorov, 2016), so we have removed them from our robotic hand's model. The four DOFs in the wrist come from the original mechanical design of the Nadine robot. The robotic arm will assist the robotic hand to make the global motion within the 3D working space. **Figure 4D** shows the mechanical model of it with a total of 20 DOFs.

## 3D Modeling and 3D Printing Technology

With the latest advancement in computer aided design (CAD) technology, people can create a robotic hand's 3D model easily and quickly. The next issue to address would be how to fabricate a low-cost robotic hand within a short period. In traditional techniques of building a robot arm or a robotic hand, the parts are cut into specific shapes and linked together with glue or screws. However, with the rise of 3D printing technology (Berman, 2012), more and more robotic hands can be made using the 3D printer. 3D printing technology provides an easy way to make the conversion between 3D models to real objects using CAD software and a 3D printer. For example, we have used 3DS Max™ to make the 3D model of our robotic hand in polygons mesh. The 3D printer we utilized, uPrint™ SE Plus,2 can print out a hand-sized object within 20 h regardless of the complexity of the model. In the future, we plan to use the latest 3D printing technology to improve the quality of our robotic hand. The "Mark X Composite Printer"3 can use a new print material called carbon fiber to print. This material has some superior characteristics in comparison to aluminum. The Mark X print bed clicks into place with 10 μm accuracy.

For articulated robotic hands, the structure of the joints is one of the most critical designs that will affect the functionality. The finger joint of a good robotic hand should be able to act like a knuckle of a human hand. It not only links the two neighboring finger's phalanges but also supports the motion of the fingers and grasping the power of the robotic hand. For commercial prosthetic hands, "linkage spanning" and "tendon linking" are two widely used joint coupling methods (Belter et al., 2013). "Linkage spanning" provides the best stability when moving the joint, while "tendon linking" gives more flexibility depending on the material used and the mechanical design. Each of the Inmoov robot's finger, which is made from six separate parts, needs three pins and adhesive to link the parts together. In comparison, our robotic hand's finger has adopted a new 3D model design inspired by non-assembly, articulated models (Cali et al., 2012). The new design of the joints integrates pin linkages to the fingers with an interlocking method. **Figure 5A** depicts the exploded view of its finger's 3D model. The final product of the finger is shown below in blue (**Figure 5B**). This finger is ready to use right after the printing is completed with no additional assembly work required. The 3D model of InMoov robot's finger is shown for comparison in **Figure 5C**.

This method can also be applied to the design of the thumb section. The thumb plays a more imperative role as compared to the other fingers in the grasping action. From our robotic hand's DOF model, the thumb has two DOFs in the TM joint. In many other robotic hands' 3D model, TM joint is split into two joints to represent two DOFs, for example, Inmoov hand (**Figure 5F**), Tact hand, and Dextrus hand. Nonetheless, this design will make the robotic hand, unlike a human hand. "Ball joints" are spherical bearings that allow a limited range of smooth movements in all directions. For robotic hands, "ball joints" can be used for simulating a joint with more than one DOF, such as a wrist's joint or a thumb's joint. EthoHand (Konnaris et al., 2016) has a ball-jointed thumb, which is controlled by three motors and six tendons (**Figure 5G**).

<sup>2</sup>uPrint SE Plus: http://www.stratasys.com/3d-printers/idea-series/uprint-se-plus. 33D printer mark X: http://www.chemtron.com.sg/3d-printers/mark-x-compositeprinter.

As part of the 3D model, our robotic hand integrates the "ball joint" mechanism to the robotic hand's palm (**Figures 5D,E**). This design gives the TM joints of the robotic hand the ability to move in any direction. With the help of two cables, the thumb's four motions, abduction/adduction, and flexion/extension can be simulated from this joint. This design improves the pinch function of the robotic hand.

These 3D printed based robotic hands greatly reduce the number of parts, time, and cost to make a copy. According to Inmoov's website, more than 100 people have fabricated an Inmoov hand. Our robotic hand has an even more easy-to-make hand 3D model. Our robotic hand can be 3D printed in seven separate parts, including five fingers and two pieces of half palm. The seven parts can be linked together to function as a 15-joint robotic hand (**Figure 6A**). The posterior palm has a smooth shape, which is an improved design of the flexy hand (Burn et al., 2016), an opensourced 3D modeled hand. After combining models of the seven parts together, it forms an entire piece of the robotic hand's 3D model (**Figure 6B**). This combined 3D model can be 3D printed out as a whole (**Figure 6C**), thus saving assembly time and making mass production more straightforward.

## Finger Actuation

The simplest way to actuate the fingers is to use 16 actuators to control the 16 DOFs. Nevertheless, it will result in a heavy and complex control system. By applying the constraints of human hand Type II, we have used the cable-driven method to control it, which is similar to the Dextrus hand and the Inmoov hand. One cable actuates each finger, but the thumb has an additional cable for adduction motion (**Figure 6D**). Previous tests on those low DOFs robotic hands already show that six or seven DOFs are good enough for most grasping gestures (Slade et al., 2015; Konnaris et al., 2016).

Our robotic hand has six actuators to control 15 joints and six DOFs. Each finger of Nadine's hand has three movable joints. A thread (0.5 mm diameter Nylon of 11 kg tensile strength) goes through the inner part of the whole finger to control the movement. When pulling the control thread, the finger will move starting from the DIP joint, which has the least resistance force of all finger joints. We set the motion range of the joints similar to a real hand's by adjusting the length of the joint's connector part. **Table 1** shows our robotic hand (Robotic hand from IMI) joint moving angles as compared to the other robotic hands and human hand.

## Motion Control System

For a cable-driven robotic hand, each finger is usually controlled by two cables and one motor. The motor uses a round or a two-arm horn to pull the two cables. The turning of the motor actuates the flexion/extension motion of the corresponding finger. Our robotic hand supports both two-cable and one-cable design. The one-cable

#### Table 1 | Hand joint moving angle.


design uses one cable to control each finger, which only actuates the flexion motion of the finger. Our robotic hand is 20% smaller than an adult human hand, thus allows the hand to be inserted into a normal-sized silicon artificial skin. The elasticity of the artificial skin will return the fingers to an extended position once the cables are relaxed. The one-cable design only requires one-arm horn of the motor which is smaller than a two-arm horn. Moreover, this design allows us to position the servo motors' bed in steps, at the same time preventing any collision between servo motors. This results in a compact control system. The two-cable design takes up more space but enables the hand to function without artificial skin. It uses a Raspberry Pi II™ as the controller, sending pulse-width modulation signals to control the motion of each motor.

As the Nadine robot uses the pneumatic motor for its actuator and the torque force is not very strong, the weight of the new hand should be as light as possible to avoid exerting a heavy load onto the joint of the wrist. Our robotic hand weighs 150 g, which is much lighter than any of the existing hands. The three main reasons are (1) the 3D printed parts are not heavy due to the hollow design of the palm and fingers. It also saves printing materials and print time; (2) the servo motor used for the actuator of it (HITEC HS-5070MH) is only 12.7 g each; (3) it can be powered externally, doing away the need for a battery to be provisioned inside the hand. **Table 2** shows the Nadine hand's mass compared to other robotic hands.

## Wrist Design and Servo Bed

We have two wrist designs for two robots. The robotic hand model originally has a four-hole wrist connector, which is Table 2 | Mass of the robotic hand.


designed to link to the wrist of Nadine robot (**Figure 7B**). A small servo bed for HITEC HS-5070MH has been fixed on the Nadine robot's forearm (**Figure 7C**). The connector is interchangeable with another design to fit other robots. For **Figure 7A**, we have referenced the wrist part of Inmoov hand and made it as a part of our hand model. We also increased the size of the fingers to suit the new thin artificial hand skin. Then, we can test our robotic hand with Inmoov hand's forearm and actuators.

## Silicon Artificial Skin

We purchased customize artificial hand skin from RenShan silicon rubber production company.4 The force needed to drive the finger is estimated based on the hardness and thickness of the silicon used to make the artificial skin. Based on our previous test experience, artificial skin, which is too thin (less than 2 mm) will be easily torn during the grasping tests. We tested the 5, 8,

<sup>4</sup>http://cn.made-in-china.com/showroom/renshanzhiping.

and 12° hardness silicon skin in 1.5, 3, and 5 mm conditions. We found HS-5070MH motor can only fully drive 5° hardness, 1.5 mm artificial skin, so, we have chosen this artificial skin for our later experiment.

## LIMITATION OF THE HARDWARE

The grip strength of a male adult is generally up to 50 kg (Mathiowetz et al., 1985). It is much higher than the force that a small-sized servo motor can produce. We searched online for a small size and high torque actuator. The HITEC HS-5070MH has a dimension of 23.6 mm × 11.6 mm × 29 mm and provides a torque of 3.8 kg.cm when powered by 7.4 V. Although the grasp is contributed by five servo motors, significant torque is consumed by the cable-driven system. Thus, the grip strength is much lesser than a human hand. The heaviest object we tested is a small tin weight 200 g (**Figure 8H**). Our 3D print material (PLA) is also unable to withstand such high force and can easily break when its thickness is less than 1.5 mm. We tried to use a 1 mm dimension pin to link the joints of the finger, and we found it is easily to crack when the finger is driven by a 3.8 kg.cm torque servo motor. As the torque of servo motors continues to increase and the strength of 3D print materials improve constantly, these two related issues should be solved in early future. We neither calculated the efficiency of our system nor the tip force of each finger. They are not the main goal of our robotic hand. The evaluation of our robotic hand will be performed by direct grasping experiment.

Autonomous grasping requires precise motion control. For global motion control, it should move the robotic hand not only to a suitable grasping position but also in a correct orientation. For local motion, it should move each finger to form the planned grasping gestures in the correct sequence. However, due to the mechanical design and the tolerance of the actuators, which manage the Nadine robot's arm and hand, we are unable to precisely control the grasp path of fingers. However, as compliant fingers design, this hand is still able to create all the important grasp gestures in the experiments. We are still working on the precise control with the cable-driven method.

The surface of 3D printed robotic hand is solid and smooth; it is hard to create enough friction when holding a heavy object. To solve this issue, we have added Blu Tack to the contact points so as to increase the coefficient of friction and improve the grasping force. Blu Tack is a reusable putty-like pressure-sensitive adhesive produced by Bostik, commonly used to attach lightweight objects (such as posters or sheets of paper) to walls or other dry surfaces. The Blu Tack is removed in the later experiments with the artificial skin.

## EXPERIMENTS

To evaluate the overall performance of our robotic hand, we have conducted grasping experiments using different objects from everyday life. After that, we test the robotic hand with and without the artificial skin. Last but not least, the grasping experiments were tested on the Nadine robot using the new robotic hand.

## Grasping Experiment Based on Cutkosky's Taxonomy

In Cutkosky's human hand taxonomy (Cutkosky, 1989), there are 16 grasping types, which have different gestures. He divided the grasp into two major categories, power and precision. The "power grasp" emphasizes on security and stability whereas the "precision grasp" emphasizes on dexterity and sensitivity. These two categories have several subcategories based on the geometry of the target. For all grasping tests, we placed the target objects at

a fixed position in front of our robotic hand, and then we sent a command to control each finger to approach the target object. A few gestures were generated only after multiple trials.

For "power grasp," it is divided into prehensile and non-prehensile based on whether clamping is required. Non-prehensile grasping is usually used for objects bigger than the hand. Due to the mechanical limitation of our robotic hand, its thumb is unable to move to the palm's plane. Therefore, we have used a palm and four fingers as the platform to support the target (**Figure 8A**). When the target must be clamped, prehensile grasping is chosen in which the fingers and palm confine the object. The basic geometric considerations of the objects are critical. If the object is thin, we use Lateral Pinch as shown in **Figure 8B**. In **Figures 8C,D**, fingers surround the object in radial symmetry when the object is compact like a CD or a ball. For a long object, fingers surround the object in wrap symmetry, as depicted in **Figures 8E–I**.

"Precision grasp" has two subcategories based on the target's geometry. For a compact object, fingers support the object in the shape of a disk and a sphere, as shown in **Figures 9A–C**. For a long object, the thumb opposes one or several fingers (**Figures 9D–G**).

Grasping analysis is dependent on several parameters such as finger force, gesture, and friction between the hand and the target object, making it difficult and complex. In these experiments, we focus on some important features of the robotic hand. In conclusion, for power grasp, our robotic hand can hold an object as heavy as 300 g and lift it up with a stable grip. Heavier objects will prevent the grasping gesture as the driven force is insufficient. That means the grasp fails and the object could slip

out. For precision grasp, it can hold different objects with suitable gestures based on their shapes. We are testing our robotic hand with more grasp gestures (Feix et al., 2009b; Deimel and Brock, 2016).

## Grasp Experiments with Artificial Silicon Skin

We test our robotic hand with an artificial silicon skin. The skin will limit the driven force and hence might reduce the motion range of the fingers. However, compared to the first experiment, it will make the robotic hand fit to Nadine robot and achieve a human-like grasp. We tested four different shape objects with two common grasp gestures. In the experiments, we pinched a card and a spoon. We also held a stapler and a slim cylinder. Similar to the first experiment, we manually placed the objects in front of our robotic hand and controlled the fingers to grasp (**Figure 10**). We are still working with our vendor for a softer and more human-like artificial skin.

We have tested our robotic hand on the Nadine robot. The test target is a small-sized toy. We used the plotted motions from the virtual Nadine in Section "Grasp Experiments with artificial silicon skin" and applied to the Nadine robot controller. The results show that the Nadine robot can grasp the object as per the plotted motions. **Figure 11** provides four screenshots of the Nadine robot grasp test.

## CONCLUSION AND FUTURE WORK

We have described the modeling and fabrication of a new 3D printable robotic hand for the humanoid robot. We studied how a human hand functions at first, and then tried to replicate the important features on our robotic hand. The robotic hand has a simple and practical design, which successfully simulated most human hand gestures in our grasp experiment. The robotic hand weighs 150 g. The test shows that it can handle most of the important grasping configurations. This robotic hand can greatly improve the overall performance of humanoid robots.

In future, we plan to improve it in several ways:


## AUTHOR CONTRIBUTIONS

NT has initiated the research on Nadine's robot hand as she is the Principal Investigator of Nadine social robot research. LT has designed and implemented Nadine's robot hardware hand and done state of the art. All authors have discussed the research every week and have contributed to the writing and editing of the paper.

## FUNDING

This research is supported by the BeingTogether Centre, a collaboration between Nanyang Technological University (NTU) Singapore and University of North Carolina (UNC) at Chapel Hill. The BeingTogether Centre is supported by the National

## REFERENCES


Research Foundation, Prime Minister's Office, Singapore under its International Research Centres in Singapore Funding Initiative.

## SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at http://www.frontiersin.org/article/10.3389/frobt.2017.00065/ full#supplementary-material.


*Conference on Robotics and Automation (ICRA)*, 3485–3492. doi:10.1109/ ICRA.2016.7487528

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2017 Tian, Magnenat Thalmann, Thalmann and Zheng. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# A Physical Model Suggests That Hip-Localized Balance Sense in Birds Improves State Estimation in Perching: Implications for Bipedal Robots

Darío Urbina-Meléndez 1,2†, Kian Jalaleddini 3†, Monica A. Daley <sup>4</sup> and Francisco J. Valero-Cuevas 1,3 \*

<sup>1</sup> Department of Biomedical Engineering, University of Southern California, Los Angeles, CA, United States, <sup>2</sup> School of Engineering, National Autonomous University of Mexico, Mexico City, Mexico, <sup>3</sup> Division of Biokinesiology and Physical Therapy, University of Southern California, Los Angeles, CA, United States, <sup>4</sup> Comparative Biomedical Sciences, Royal Veterinary College, London, United Kingdom

#### Edited by:

Stéphane Viollet, Aix-Marseille Université, France

#### Reviewed by:

Anick Abourachid, National Museum of Natural History, France Pierre Moretto, Université de Toulouse, France

#### \*Correspondence:

Francisco J. Valero-Cuevas valero@usc.edu

†These authors have contributed equally to this work.

#### Specialty section:

This article was submitted to Bionics and Biomimetics, a section of the journal Frontiers in Robotics and AI

Received: 23 December 2017 Accepted: 19 March 2018 Published: 04 April 2018

#### Citation:

Urbina-Meléndez D, Jalaleddini K, Daley MA and Valero-Cuevas FJ (2018) A Physical Model Suggests That Hip-Localized Balance Sense in Birds Improves State Estimation in Perching: Implications for Bipedal Robots. Front. Robot. AI 5:38. doi: 10.3389/frobt.2018.00038 In addition to a vestibular system, birds uniquely have a balance-sensing organ within the pelvis, called the lumbosacral organ (LSO). The LSO is well developed in terrestrial birds, possibly to facilitate balance control in perching and terrestrial locomotion. No previous studies have quantified the functional benefits of the LSO for balance. We suggest two main benefits of hip-localized balance sense: reduced sensorimotor delay and improved estimation of foot-ground acceleration. We used system identification to test the hypothesis that hip-localized balance sense improves estimates of foot acceleration compared to a head-localized sense, due to closer proximity to the feet. We built a physical model of a standing guinea fowl perched on a platform, and used 3D accelerometers at the hip and head to replicate balance sense by the LSO and vestibular systems. The horizontal platform was attached to the end effector of a 6 DOF robotic arm, allowing us to apply perturbations to the platform analogous to motions of a compliant branch. We also compared state estimation between models with low and high neck stiffness. Cross-correlations revealed that foot-to-hip sensing delays were shorter than foot-to-head, as expected. We used multi-variable output error state-space (MOESP) system identification to estimate foot-ground acceleration as a function of hip- and head-localized sensing, individually and combined. Hip-localized sensors alone provided the best state estimates, which were not improved when fused with head-localized sensors. However, estimates from head-localized sensors improved with higher neck stiffness. Our findings support the hypothesis that hip-localized balance sense improves the speed and accuracy of foot state estimation compared to head-localized sense. The findings also suggest a role of neck muscles for active sensing for balance control: increased neck stiffness through muscle co-contraction can improve the utility of vestibular signals. Our engineering approach provides, to our knowledge, the first quantitative evidence for functional benefits of the LSO balance sense in birds. The findings support notions of control modularity in birds, with preferential vestibular sense for head stability and gaze, and LSO for body balance control,respectively. The findings also suggest advantages for distributed and active sensing for agile locomotion in compliant bipedal robots.

Keywords: balance, lumbosacral organ, vestibular system, birds, perch, compliant robot, co-localized sensing, distributed sensing

## 1. INTRODUCTION

All terrestrial vertebrates have linear and angular acceleration sense localized to the vestibular system of the inner ear. It is well-known that birds use a variety of reflexes mediated by internal signals to stabilize their head during walking and flying (Maurice et al., 2006). Uniquely among living animals, birds appear to have two specialized balance-sensing organs: the vestibular system of the inner ear and an additional balance sensor located between the hips called the lumbosacral organ (LSO) (Necker, 2006) which has been proposed to be especially useful for terrestrial locomotion (Necker, 2005, 2006). Birds have long flexible necks, with head motions tightly coupled to gaze control (Necker, 2007; McArthur and Dickman, 2011; Pete et al., 2015). Consequently, the vestibular system is not closely nor tightly coupled to the torso. In contrast, the LSO is located in the sacrum between the hips, near the CoM. Having a balance organ at the torso is likely to be beneficial to legged locomotion and balance because the hip joint plays an important role on controlling the position of the CoM of the whole body with respect to the foot (Abourachid et al., 2011). Here we consider and contrast the functional implications of hip-localized (LSO) and head-localized (vestibular) balance-sense.

Generally speaking, keeping balance is a task that many legged-animals perform to prevent falling or rotating about the foot point after perturbations (Vukobratovic et al., 2012). Specifically, a balance-sensing organ produces afferent signals to detect current body posture and motion to determine the movements required to achieve or maintain a desired posture and motion. In birds, direct neurophysiological evidence has clearly established that they must possess balance sense that is independent of the vestibular system (Abourachid et al., 2011). They retain the ability to reflexively compensate for body rotations even after labyrinthectomy and spinal cord transection to eliminate descending inputs influenced by the vision and vestibular senses (Abourachid et al., 2011). This neurophysiological evidence, along with particular anatomical features of avian lumbosacral region (below), suggests a balance sensing function of the LSO that complements proprioceptive information from the vestibular system, as well as mechanoreceptors in the skin, joints and muscles.

Anatomically, the LSO is located within an enlargement of the lumbosacral region of the spinal column, between the 27 and 38th segments (Streeter, 1904). The LSO presents a suite of features unique to the spinal column of birds, including bilateral protrusions of neural tissue identified as mechanosensors (accessory lobe (AL) neurons), located adjacent to ligaments supporting the spinal cord (Schroeder and Murray, 1987; Necker, 2005, 2006; Yamanaka et al., 2008). The spinal cord is dorsally bifurcated in this region and supports a "glycogen body" (GB) centered on top. The entire region is enclosed by bony canals with a distinct concentric ring structure (Necker, 2006). The arrangement of the canals, AL, ligaments, and GB is reminiscent of the vestibular system (Necker, 2006) and invites functional analogy to an accelerometer. Each AL contains mechanoreceptors (Schroeder and Murray, 1987; Necker, 2006; Yamanaka et al., 2008), with commissural axons projecting to last-order premotor interneurons in the spinal pattern generating network (Eide and Glover, 1996; Necker, 2006). The AL neurons within the LSO exhibit spontaneous firing and phase-coupled firing in response to vibrational stimulation between 75 and 100 Hz, and ablation of these neurons disrupts standing balance (Necker, 2006). Thus, multiple lines of anatomical and neurophysiological evidence suggest balance-sensing function of the LSO.

Despite evidence of LSO hip-localized balance-sense in birds, no previous studies have provided quantitative evidence for the functional benefits of LSO as an adaptation for posture balance sensing of posture-relevant information. We hypothesize that hip-localized balance sense provides two main functional advantages compared to head-only balance-sense: (1) reduced sensorimotor delay and (2) more accurate state estimation of foot-ground acceleration due to closer proximity to the feet. Here we use a physical model of a perching guinea fowl subject to foot-ground perturbations to test the hypothesis that hiplocalized balance sense enables more rapid sensing and accurate state estimation compared to only a head-localized balance sense.

Most birds "perch" (balance with the feet attached to the substrate) when they alight on elevated objects such as branches; therefore we focus on perching as a conveniently simple and ecologically relevant balancing behavior. We built a simple physical model of a standing guinea fowl perched on a horizontal platform (i.e., feet attached to the platform). The horizontal platform was attached to the end effector of a 6 DOF robotic arm, allowing us to apply perturbations analogous to motions of a compliant branch. The physical model provides a first approximation of the muscle-tendon viscoelastic properties that provide leg compliance. We approximated LSO and vestibular balance sensors using 3-D accelerometers located at hips and at the head, respectively. We used system identification to estimate foot-ground acceleration as a function of hip- and head-localized sensing, individually and combined.

## 2. MATERIALS AND METHODS

## 2.1. Physical Model of a Guinea Fowl

A skeletal model of a guinea fowl was built by interconnected and hinged aluminum bars (**Figures 1**, **2**). Friction was reduced by using bearings at the hip, knee, ankle, and foot. The general body size, limb segment lengths and configuration were based on guinea fowl anatomy from the literature (Gatesy and Biewener, 1991; Daley et al., 2007; Gordon et al., 2015), with a hip height of 20 cm.

This physical model focused on approximating the guinea fowl's (i) LSO (hip) and vestibular (head) balance sensing systems location, (ii) body center of mass location and limb configuration in a standing posture and (iii) visco-elastic mechanical properties of the muscle-tendon-driven limbs. This model was meant as a first approximation of the key physical features, to allow a quantitative comparison of the information available at hipand head-localized balance sensors. It was not meant to be an exhaustive exploration of the effects of posture, material properties, and muscle-tendon actions. Such considerations could be the subject of future work.

The toes of the model were firmly attached to a platform. Thus, the guinea fowl model "perched" while maintaining an upright standing posture. This posture was maintained by the passive tensions in rubber bands that cross the hip, knee, ankle and metatarsal joints without further assistance or active support (**Figure 2**). We pre-tensioned rubber bands across joints to represent the tendon-driven functional anatomy of a guinea fowl. These rubber bands also have viscoelastic properties that approximate the passive mechanical properties of "muscles" held at a constant activation level when holding the standing posture. The origins and insertions of the rubber bands were adjusted to have large enough moment arms at each joint to overcome gravity and maintain posture even when perturbed by the moving platform. Our focus was not to explore effects of varying muscle activation patterns for standing postures, but instead to find a set of tensions in the rubber bands sufficient to maintain standing posture and propagate the perturbations from the platform through the skeletal anatomy.

We used two interchangeable necks, each with different stiffness to test the effects of muscle coactivation on balance sensing at the head. Each neck was 25 cm long and curved as shown in **Figures 1**, **2**. The first neck was made of 12.7 mm diameter Ultra-Flex Corrugated Steel Sleeving (McMaster-Carr, part 54885K21). The second was 19.05 mm diameter Abrasion-Resistant Polyurethane Rubber Rod (McMaster-Carr, part 8695K155).

## 2.2. Instrumentation

The end-effector of a 6 degrees of freedom (DOF) AdeptSix 300 robotic arm (Omron Adept Technologies, Inc, San Ramón, CA) hold the platform where the model perched (**Figure 3**). We used 3-D accelerometers at the following locations on the model: (i) head to represent the vestibular system; (ii) hip to represent the LSO sensor, and (iii) between the feet to record the reference perturbations or "foot acceleration" (**Figure 1**). All

FIGURE 1 | Physical model of the skeleton of the guinea fowl made of articulated aluminum plates and an elastic tube for a neck. The location of the sensors can be seen on the floor between the model's feet, on its pelvis between the hips, and on its head. The joints of the model are, starting from the pelvis: the hip, knee, ankle and metatarsal joints. The transparent sphere around the accelerometer between the feet indicates the scale of random displacements 20 mm in radius.

accelerometers were MEMS inertial sensors Model LIS344ALH (ST Microelectronics, Geneva, Switzerland).

## 2.3. Trials

Each trial replicated a scenario that a guinea fowl might face while perching on a tree branch which is subject to perturbations from weather and other animals. Our goal was not to replicate natural perturbation exactly, but to provide a general test of our hypothesis that the LSO has benefits over the vestibular system for rapid and accurate state estimation for balance.

Each trial consisted of a series of 3,000 random, uncorrelated displacements generated by the robotic arm. Each displacement was a center-out-and-back movement in a random direction to the surface of spheres with 2, 5, 10, and 20 mm in radius. Trials were block-randomized across sphere sizes. We recorded a total of eight trials (4 sphere sizes × 2 necks stiffnesses) (**Table 1**).

## 2.4. Data Acquisition

We used a high-performance National Instruments (NI) PXI-8108 computer, upgraded with 4 GB DDR2 RAM and a 500 GB SSD. An NI PXI-6254 ADC card recorded the accelerations signals. The data acquisition hardware was housed in the NI PXI-1042 chassis. We acquired data at the sampling rate of 1 kHz.

## 2.5. Data Analysis

### 2.5.1. Estimation of Neck Stiffness

To estimate the effective neck stiffness, we performed a bootstrap analysis of 1,000 trials by randomly selecting 30 s segments

FIGURE 2 | Photographs of the physical model of the skeleton of the guinea fowl. On the left the complete model is shown, on the middle and right sections details of the elastic linkages that are required for the robot to maintain a standing posture can be seen.

from each trial. We then found the resonant frequency (the frequency with maximal power) of accelerations at the head. The effective muscle stiffness was estimated from:

$$K\_i = m\_i f\_i^2 \tag{1}$$

where i is the neck number, m<sup>i</sup> the mass and f<sup>i</sup> the resonant frequency.

## 2.5.2. Estimation of Sensory Delay at the Hip and Head

We calculated cross-correlation of foot acceleration against that recorded from hip or head to estimate the propagation delays of TABLE 1 | Each trial consisted of 3,000 random center-out-and-back displacements (center-surface of a sphere).


the applied mechanical perturbations. The delay was taken as the lag where the cross-correlation was maximal.

## 2.5.3. Estimation of the Time History of Foot Acceleration

We used a data-driven modeling approach to estimate the time history of the foot acceleration given the time history of signals recorded at the sensory sites (hip and head). To this end, we trained state-space models (in the least-squares sense) to predict foot acceleration from the hip or head accelerations. We used MOESP state-space identification (Verhaegen and Dewilde, 1992; Verhaegen and Verdult, 2007) implemented in the Statespace Model Identification (SMI) MATLAB toolbox (Haverkamp and Verhaegen, 1997). The state-space model is represented as follows:

$$\begin{cases} \mathfrak{x}(k+1) &= A\mathfrak{x}(k) + Bacc\_{\mathsf{sensor}}(k) \\ acc\_{\mathsf{foot}}(k) &= \mathsf{Cx}(k) + Dacc\_{\mathsf{sensor}}(k) \end{cases} \tag{2}$$

where accsensor(k) is the input signal (acceleration signal recorded from the hip or neck) and accfoot(k) is the measured foot acceleration. x(k) is the state variable, and A, B, C, D are the unknown state-space matrices. We set the model order to three after inspecting the singular values of the extended observability matrix as described in the previous work (Haverkamp, 2000). The model order of three resulted in 21 parameters that was significantly less than the number of 4,000 available training data points for each training run. Since the number of free parameters was much <10% of the training data, the model is not over-parameterized and cannot learn noise and the stochastic behavior.

We assessed the performance of the model in predicting the foot acceleration (accˆ foot). By running the identified models in the prediction mode, we compared the predictions to the actual measured signals, accfoot. We quantified the difference using the identification Variance Accounted For (VAF) expressed as:

$$\% \text{VAF} = 100 \left( 1 - \frac{\nu ar(\hat{a} \text{\hat{c}} c\_{\text{foot}}(k) - a c c\_{\text{foot}}(k))}{\nu ar(a c c\_{\text{foot}}(k))} \right) \tag{3}$$

where 100% indicates a perfect prediction of all the variability in the measured signals, and 0% means no meaningful prediction.

### 2.5.4. Boot-Strap Analysis and Statistics

To estimate the robustness of the analyses (cross-correlation, system identification, etc.), we performed a 100 trials bootstrap study (random sampling with replacement) (Efron and Tibshirani, 1994). For each trial, we randomly chose 40 s windows from the measured data, performed the cross-correlation and system identification analyses, and then calculated summary statistics across the 100 measures. We performed student's t-test for statistically significant differences between conditions. Values for central tendency and variance are reported as medians (interquartile range) unless stated otherwise.

## 3. RESULTS

We first present the differences in neck stiffness, then the effect of sensor location and neck stiffness on (i) sensing delay, and (ii) estimation of foot acceleration.

The necks made from two different materials have different bending stiffnesses whose estimates are shown in **Figure 4**. Since we measured the dynamical response of the entire physical model (see section 4), each direction and magnitude of perturbation induced a different dynamical response that resulted in a different acceleration measured at the head. This led to different resonant frequencies to be multiplied by the mass of the neck (Equation 1). Note that we would obtain different estimates of neck stiffnesses if the square of the resonant frequency at the head were multiplied by the mass of the whole model. Doing this would have given us an approximation of the stiffness of the whole body, which besides the neck, has a fixed stiffness. Also, if the complete body mass were considered, mass differences between trials with different necks would have been smaller, resulting in a constant bias that would not change the statistical differences between the estimates of neck stiffness. The median neck stiffnesses were 0.67 ( 0.26–1.05) N/m and 1.25 (0.56–1.55) N/m for the low and high stiffness necks, respectively. Student t-test shows the average neck stiffness are significantly different (p < 0.05).

As expected, an accelerometer at the hip generally detected foot acceleration with shorter delays than the accelerometer at the head. Foot-to-hip median delays were 0.02 (0–0.03) s and 0.03 (0.005–0.065) s, respectively for the low and high stiffness necks. Foot-to-head median delays were longer, measured at 0.095 (0.06–0.135) s for the low stiffness neck, and 0.055 (0.02–0.07) s for the high stiffness one (**Figure 5**). The variability was quite

large as the shown information collapses data across different acceleration axes and different sphere experiments (perturbation magnitudes).

Foot-to-hip delays were significantly shorter than foot-tohead delays (p < 0.05) for the low stiffness neck, but not significantly different for the high stiffness neck (**Figure 5**). A stiffer neck reduced the delays for information sensed at the head. This resulted in hip and neck delays that were very similar with no statistical difference.

Estimates of acceleration at the feet are more accurate when using signals from the hip-mounted accelerometers than from the head-mounted accelerometers. **Figure 6** shows an example where acceleration at the feet is estimated from the hip- and headmounted accelerometer, overlaid with the ground-truth signal measured at the feet.

Hip-localized estimates of the foot acceleration accounted for 30.81–48.96% of variance (% VAF as defined in Equation 3) against 15.59–22.19% of head-localized estimates (**Figure 7**). This figure summarizes the estimation results by pooling together data from both neck stiffnesses. Prediction of foot acceleration as a function of neck type is shown in **Figure 8**. Particularly, **Figure 7** shows data separated as a function of perturbation magnitude. It demonstrates that independently of the perturbation magnitude, the estimate of foot acceleration from the hip was always more accurate than that from the head sensor. Moreover, sensory fusion (combining info from both sensors) did not significantly improve the foot acceleration estimation. Therefore, sensory fusion did not provide additional benefits beyond hip-only sensing.

We have found that when only head-localized accelerometers were available, the high stiffness neck improved estimates of foot acceleration compared to the low stiffness neck (**Figure 8**). With the low stiffness neck, the median VAF was 15.11 (11.38–21.74)%, while it was 17.95 (10.18–29.19)% for the high stiffness one. These data groups were statistically different (p < 0.05).

and 0.03 s respectively for the low stiffness and high stiffness necks.

## 4. DISCUSSION

To validate the anatomical and neurophysiological evidence of LSO balance sensing function in birds, we present a quantitative investigation of the functional benefits of hip-localized balance sense. Here we investigated the perturbation sensing dynamics of a physical model of a guineafowl perched in a standing posture. We explored two proposed functional advantages of hip-localized compared to head-localized balance sense: minimization of sensorimotor delay and improved estimation of foot-ground acceleration, due to closer proximity of the sensor to the feet. To our knowledge, this is the first study to quantitatively analyze the practical benefits of hip-localized sensing of accelerating for balance control. We find that a hip-localized acceleration sensor—analogous to the LSO—provides shorter delays and improved state estimation of feet acceleration during substrate perturbations.

In particular, our experimental paradigm applied displacements at the feet, where we also measured the "ground truth" acceleration of the moving substrate on which the bird is perched. We then compared the ability to sense and reconstruct that ground truth acceleration on the basis of accelerations measured at the hip and head. We find that the location of these simulated balance sense organs has important consequences to how a bird (a model of a guinea fowl, in this case) could use acceleration information from hip-localized balance sense for bipedal perching, standing and locomotion. A second level of analysis focused on the material properties of the neck of the physical model. One was (less stiff) corrugated tubing, and the other (more stiff) solid rubber tubing. These material differences were designed to explore the effect of muscle co-contraction at the neck as a means of active sensing, or at least modulation of the utility of head-localized balance sense.

Before discussing the results in detail, it is important to clarify some features of our experimental approach to balance sense. A salient feature of our experimental results is the variability in our results, as in **Figure 4**. Shouldn't the bending stiffness of each neck be thought as a single number? Similarly, shouldn't the foot-to-hip delays be constant and the same independently of neck stiffness (**Figure 5**)? Recall that the stiffness of the system is inferred from the resonant frequency of the acceleration measured at the head. The acceleration at the head is a function of the the dynamical response of the entire guinea fowl model to input perturbations. In fact, we are measuring the frequency response and delays of the coupled oscillations of the legs held in a standing position by rubber bands, plus the pelvis and neck. Given that this physical structure is only symmetric in the sagittal plane, its dynamical response will depend on the direction of the 3D perturbations—which naturally results in variability in our results. Nevertheless, the corrugated tubing condition ("low stiffness neck") leads to perturbation responses at the head that,

in general and on average, reflect a lower stiffness for this lumpedparameter analysis. Similarly, foot-to-hip delays were, in general and on average, shorter than the feet-to-head delays. In a sense, instead of "neck stiffness," the results in **Figure 4** may be better called the "apparent stiffness lumped at the head." But given that the purpose of this analysis is to test for the effect of the material properties of the neck on time delays and estimation accuracy, we chose not to belabor this point and simply call it "neck stiffness." After all, (i) the neck is the only body part that was swapped, and (ii) changes in material properties only at the neck better reflect the potential effects of muscle co-contraction at the neck in the guinea fowl.

There are limitations to our approach that, while worth mentioning, we believe do not challenge the validity of our results. Importantly, our physical model can only approximate the anatomy and muscle mechanics of the guinea fowl. Our multi-link articulated structure approximates only the general link-segment arrangement and length proportions of the animal skeleton, and the viscoelastic rubber bands only roughly approximate the properties of muscle-tendon linkages. Similarly, we did not consider the proprioceptive signals coming from the joints, skin and muscles that could also contribute to state estimates of foot acceleration. While these limitations prevent us from claiming that our results are direct parallels of how a guinea fowl would respond neuro-mechanically to perturbations, it is nevertheless a valid means to test for differences in sensory signals as a function of sensor location and neck stiffness. Moreover, we explicitly avoided making the assumption that the skeleton of the guinea fowl was simply a set of links rigidly fused at a given posture. Rather, we used rotating hinges at the joints, where the posture of the model was achieved by appropriately setting the lengths and tensions of the rubber bands to approximate muscle-tendon actions to maintain posture at rest. This mechanical structure as a first approximation—provides a biomechanically realistic instantaneous response to a perturbation at the feet, and avoids other multiple assumptions associated with a computational model (Martins et al., 2009). The results we present here are an analysis of the aggregate acceleration responses to a sequence of center-out 3-dimensional perturbations. As such, we consider the details of each response only implicitly. Future research could explore the moment-to-moment details of the responses within an individual perturbation.

The biological interpretation of these results hinges on the assumption that the functional benefits of hip-localized balance sense could translate into selective evolutionary pressure to promote the anatomical specialization of the LSO in evolutionary time. This assumption is supported by two fundamental controltheoretical notions: (i) that delays are detrimental because they make any causal closed-loop controller (biological or engineered) more unstable (Gu et al., 2003) and (ii) that having a more faithful estimate of a perturbation improves the corrective

FIGURE 7 | Comparison of estimation accuracy of foot acceleration from the hip, head and their fusion as a function of perturbation magnitudes. Hip-to-feet compared to head-to-foot acceleration estimation was more accurate (p < 0.05). Fusion of the hip and head information did not improve estimation of the foot acceleration beyond that obtained with hip information along.

type. Increasing neck stiffness improved estimation of foot acceleration from acceleration measured at the head. Low stiffness and high stiffness neck data were statistically different (with p < 0.05 and indicated with an asterisk). The median %VAF was 15.11 and 17.95 for the low stiffness neck and high stiffness neck respectively.

response, and thus improving performance, economy and stability.

The simplest interpretation of the time delays hinges on the notion that a causal feedback controller has knowledge of the past, but not of the present (strictly speaking) or future. Therefore, it cannot execute anticipatory control actions and is thus limited by its closed-loop bandwidth. In contrast, biological systems are well-known to produce anticipatory motor commands (Aruin and Latash, 1995; Westwick and Perreault, 2011), as well as short-latency reflexive responses (Sinkjær et al., 1999; Jalaleddini et al., 2017b,a). Anticipatory strategies are considered to be critical adaptations to mitigate the deleterious effects of large transmission and processing delays inherent to neural systems (Bean, 2007; Faisal et al., 2008). Nevertheless, any voluntary, anticipatory or reflexive action would benefit from shorter delays. This point is supported by the observation of many morphological and physiological adaptations in the nervous systems to reduce time delays such as increased axonal diameter, myelination and saltatory conduction.

The biological relevance of state estimation (Kalman, 1960; Simon, 2006) relates to the fact that physiological sensory signals contain task-relevant information, but not necessarily in the coordinates and units used by the controller. In particular, some version of the "state" of the system is encoded in sensory coordinates and units that are different from those used by the neural controller to select, plan and execute a response. This means that any raw sensory signal (e.g., acceleration at the LSO or vestibular system) must first be interpreted to extract useful information (e.g., the details of the perturbation at the feet). The MOESP state-space identification technique is but one example of a state estimator (Verhaegen and Dewilde, 1992; Verhaegen and Verdult, 2007). To test our hypothesis, it suffices to show that a hip-localized balance sensing organ is better at sensing, estimating, and reconstructing the perturbations at the feet than a head-localized one, **Figure 6**. On the same figure, we only show forward/backward accelerations (i.e., along the y axis), which are the most destabilizing during locomotion. It has been shown that lateral (i.e., side-to-side) movements are more stable than forward/backward movements because stance width naturally provides a stabilizing effect (Dean et al., 2007). Whether and how the concept of state estimation applies to the nervous system, however, is yet unresolved (Loeb, 2012).

Necker stated in the concluding paragraph of his 2006 paper that "The local organization of the neuronal network [of the LSO] favors rapid and hence effective control," with no further elaboration (Necker, 2006). We now present what is, to the best of our knowledge, the first concrete evidence that a hip-localized balance sense organ (like the LSO) is an effective source of faster and better sensing of posture-relevant information. Faster sensing is evidenced by the shorter time delays for hip-localized vs. a head-localized accelerometers. Moreover, our results also show that the time delays for head-localized balance sense organs can be shortened by cocontracting neck muscles (i.e., a stiffer neck). From the state estimation point of view, however, we find that hip-localized balance sense organs are superior, and do not benefit from sensory fusion with head-localized acceleration independently of neck stiffness. Therefore, we conclude that hiplocalized balance sense indeed promotes more rapid and effective control.

These results have important implications for how the evolution of hip-localized balance sense by the LSO might have contributed to the unique sensorimotor control features of birds. In particular, it has long been recognized that birds have relatively "modular" function and control of wings, legs and tail compared to other vertebrates (Gatesy and Dial, 1996). The functional dissociation between forelimb (wing) for aerial locomotion and hindlimb (leg) for terrestrial locomotion is paralleled by increased autonomy of their respective sensorimotor control networks (Biederman-Thorson and Thorson, 1973; Jacobson and Hollyday, 1982; Sholomenko and Steeves, 1987; Ho and O'Donovan, 1993; McArthur and Dickman, 2011). The presence of a local and distributed balance sensing organ that is directly integrated with hindlimb spinal networks has likely contributed to this modular control organization. The mechanosensing neurons of the LSO project directly to pre-motor neurons in the spinal cord (Eide, 1996; Necker, 2006). This suggests the balance sense information produced by the LSO is likely to contribute to rapid and effective control because it is processed locally. Such local processing is advantageous because involving the brain in the response could introduce counterproductive time delays.

While our results focus on perching, hip-localized balance sense is likely beneficial for other postural and locomotor tasks. We designed our perturbations to simulate sensory inputs analogous to bird perching on a branch subject to varied 3- D movements such as wind, movements of other animals, etc. During perching, a bird is exposed to 3-D substrate perturbations, for which short-latency reflex responses could suffice, if sufficiently rapid sensing is available. This is similar to the observed knee and ankle strategies in the control of human upright posture (Bingham et al., 2011), or slipgrip mechanisms for human finger manipulation (Cole and Abbs, 1988). Moreover, such rapid and informative sensing is also critical to low-level (distributed, spinal or sub-cortical) sensorimotor processing to control short-latency responses to perturbations (Lawrence et al., 2015a,b) that ultimately supports long-latency control of voluntary function in general. The LSO is directly integrated with the hindlimb spinal motor control networks (Eide, 1996; Necker, 2006), suggesting that hiplocalized balance sense is likely relevant to all hindlimb-mediated behaviors, including perching, standing balance, over-ground locomotion and arboreal locomotion. Birds effectively have two distinct balance sensorimotor processing centers: the "cerebral brain," responsible for executive function and navigation, and the "sacral brain," responsible for low-level, short latency control of terrestrial perching, standing and locomotion.

Adopting lessons from the millions of years of biological evolution poses intriguing and exciting possibilities for the engineering evolution of robust and versatile bipedal robots. There are well-known forms of morphological control where the structure of the body co-evolves with the nervous system (or controller) to simplify and improve open- or closed-loop control (Lipson and Pollack, 2000; Valero-Cuevas et al., 2007; Pete

REFERENCES


et al., 2015). At the other extreme we have the classical robotics approach to fully centralized control that depends on algorithms that process sensory information and issue motor commands. The LSO provides support for an intermediate alternative, where one can have the benefits of morphological adaptations and central control—but supplemented by distributed neural control centers informed by distributed balance sense organs like the LSO.

## AUTHOR CONTRIBUTIONS

DU-M designed and constructed the physical model of the guinea fowl, wrote the Title, Abstract, Introduction, did renders of the physical model of the guinea fowl and put together different author ideas and perspectives. KJ guided the NI DAQ implementation and data analysis activities: system identification analysis, bootstrap analysis and statistics. Together, DU-M and KJ implemented the NI DAQ system, programmed the AdeptSix 300 robotic arm, did data analysis on MATLAB, wrote the Methods section created and edited the figures. MD and FV-C provided the initial idea of giving an engineering quantitative analysis for functional benefits of the LSO balance sense in birds. They wrote most of the Discussion section and validated: (i) the design and construction of the physical model of the guinea fowl, (ii) data analysis activities, and (iii) each of the paper sections and figures. All the authors contributed to editing the paper for style, clarity, succinctness and grammar.

## FUNDING

Research reported in this publication was supported by the National Institute of Arthritis and Musculoskeletal and Skin Diseases of the National Institutes of Health under Awards Number R01 AR-050520 and R01 AR-052345, and by the Department of Defense under award number MR150091 to FV-C. The content of this endeavor is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the Department of Defense. This work was also supported by Fonds Québécois de la Recherche sur la Nature et les Technologies to KJ.

## ACKNOWLEDGMENTS

Author MD would like to thank Alexander Spröwitz for discussions on the potential balance sensing function of the lumbosacral organ of birds.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Urbina-Meléndez, Jalaleddini, Daley and Valero-Cuevas. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Active Viscoelasticity of Sarcomeres

Khoi D. Nguyen† , Neelima Sharma† and Madhusudhan Venkadesan\*

*Department of Mechanical Engineering and Materials Science, Yale University, New Haven, CT, United States*

The perturbation response of muscle is important for the versatile, stable and agile control capabilities of animals. Muscle resists being stretched by developing forces in the passive tissues and in the active crossbridges. This review focuses on the active perturbation response of the sarcomere. The active response exhibits typical stress relaxation, and thus approximated by a Maxwell material that has a spring and dashpot arranged in series. The ratio of damping to stiffness in this approximation defines the relaxation timescale for dissipating stresses that are developed in the crossbridges due to external perturbations. Current understanding of sarcomeres suggests that stiffness varies nearly linearly with neural excitation, but not much is known about damping. But if both stiffness and damping have the same functional (linear or not) dependence on neural excitation, then the stress relaxation timescale cannot be varied depending on the demands of the task. This implies an unavoidable and biologically unrealistic trade-off between how freely the crossbridges can yield and dissipate stresses when stretched (injury avoidance in agile motions) vs. how long they can maintain perturbation-induced stresses and behave like a solid material (stiffness maintenance for stability). We hypothesize that muscle circumvents this trade-off by varying damping in a nonlinear manner with neural excitation, unlike stiffness that varies linearly. Testing this hypothesis requires new experimental and mathematical characterization of muscle mechanics, and also identifies new design goals for robotic actuators.

Keywords: muscle viscoelasticity, sarcomere mechanics, active perturbation response, variable impedance, stress relaxation timescale, dynamic modulus, sinusoidal response

## 1. INTRODUCTION

A muscle develops mechanical forces when neurally or electrically excited, and also when externally perturbed (Rack and Westbury, 1974; Kirsch et al., 1994; Lindstedt and Hoppeler, 2016). The perturbation response 1F<sup>p</sup> of passive tissues and the active excitation-dependent perturbation response 1F<sup>a</sup> add to the baseline force F<sup>a</sup> generated due to neural excitations and to F<sup>p</sup> due to passive tissues (**Figure 1A**). Passive refers to mechanical responses in the absence of neural stimulation while active responses require neural stimulation, and consume metabolic energy. Notably, the active resistance to stretch is one of the first mechanical responses of muscle when stimulated, even before it begins to develop tension (McMahon, 1984).

Active and passive perturbation responses play an essential role in animal motor control because they are faster than any neural response, including the fastest of reflexes (Bizzi et al., 1982; Brown and Loeb, 2000; Dickinson et al., 2000; Hogan and Buerger, 2005; Holmes et al., 2006; Nishikawa et al., 2007; Biewener, 2016; Roberts, 2016). These responses have also been called preflexes or mechanical feedback (Brown and Loeb, 2000; Nishikawa et al., 2007). In robotics as well, the perturbation response of actuators are employed advantageously when appropriately tuned to the task and the environment's mechanical response (Hogan, 1984; Pratt and Williamson, 1995; Hogan and Buerger, 2005; Buerger and Hogan, 2007; Vanderborght et al., 2013). However, current actuator

#### Edited by:

*Monica A. Daley, Royal Veterinary College, United Kingdom*

#### Reviewed by:

*Eric D. Tytell, Tufts University, United States Allison Arnold, Harvard University, United States Gregory S. Sawicki, University of North Carolina at Chapel Hill, United States*

#### \*Correspondence:

*Madhusudhan Venkadesan mv@classicalmechanic.net*

*†These authors have contributed equally to this work.*

#### Specialty section:

*This article was submitted to Bionics and Biomimetics, a section of the journal Frontiers in Robotics and AI*

Received: *23 December 2017* Accepted: *25 May 2018* Published: *14 June 2018*

#### Citation:

*Nguyen KD, Sharma N and Venkadesan M (2018) Active Viscoelasticity of Sarcomeres. Front. Robot. AI 5:69. doi: 10.3389/frobt.2018.00069*

**113**

FIGURE 1 | (A) A muscle provides, at every length, a passive tension *Fp* and an active tension *Fa* that varies with neural input. Furthermore, an external perturbation to its length elicits additional passive and active contributions, 1*Fp* and 1*Fa*, respectively. The tendons attached to the muscle work in series and act to transmit the total tension *F<sup>T</sup>* the muscle outputs, i.e., *F<sup>T</sup>* = *F<sup>p</sup>* + 1*F<sup>p</sup>* + *F<sup>a</sup>* + 1*Fa*. (B) The active perturbation response arises from the contractile machinery of sarcomeres, and can be measured by applying length perturbations 1*x* to an isolated skinned muscle fiber. The perturbation response 1*Fa* is approximated by a Maxwell-type material that is comprised of a spring and dashpot in series. The spring has stiffness *k*(*u*) and dashpot has damping *b*(*u*), both of which vary with the neural input *u*. The ratio of the damping to stiffness determine a stress relaxation timescale τrelax. For a step length perturbation, the active perturbation response 1*Fa*(*t*) decays exponentially over the relaxation time.

technologies do not yet match the ability of muscle to vary its active perturbation response by several fold, such as stiffness that may vary by over 50× in muscle (Hunter and Lafontaine, 1992; Madden, 2007; Anderson et al., 2012; Vanderborght et al., 2013; Hines et al., 2017). Therefore the mechanical capabilities of skeletal muscle continues to be sought-after by roboticists not only in terms of their force, work and power generation capabilities (Madden, 2007), but also in terms of their active perturbation response (Madden et al., 2004).

In this review, we examine the active perturbation response using the formalism of frequency-dependent dynamic modulus, i.e., the ratio of the force response 1F<sup>a</sup> to externally imposed small sinusoidal length perturbations 1x (**Figure 1B**). Emphasizing qualitative over detailed quantitative comparisons with data, we use approaches from materials science (de Gennes, 1979; Barnes et al., 1989) to simplify the possibly complicated active response in terms of spring-like and dashpot-like elements (**Figure 1B**). In particular, because the active perturbation response of muscle does not have a single, simple rest length and decays with time, its simplest representation is a Maxwell material with a spring and dashpot in series (Maxwell, 1867; Barnes et al., 1989) whose stiffness k and damping b depend on the neural input u.

We use examples from motor control in section 2 to illustrate the role of active perturbation responses, and present an overview in section 3 of how a muscle's active perturbation response is characterized. We elaborate the Maxwell model in section 4, examine its neural modulation in section 5, and summarize the main conclusions of this review in section 6.

## 2. ACTIVE PERTURBATION RESPONSE IN MOTOR CONTROL

Active perturbation response of muscle depends strongly on the animal's intent and biomechanical context. At one extreme, muscle may behave like a stiff solid that undergoes no appreciable strain (e.g., co-contracted muscles to stiffen a joint). At another extreme it may offer little resistance and yield freely like a fluid (e.g., biceps in throwing). In general, muscles exhibit every behavior in between the extremes depending on the level of neural excitation (Cui et al., 2008; Sponberg and Full, 2008; Farahat and Herr, 2010; Hu et al., 2011; Sawicki et al., 2015). Consider three representative examples with different active perturbation responses of muscles and their analogs in robots.

First, when elastic energy storage and recovery are important, a muscle typically behaves as a stiff strut (Alexander and Bennet-Clark, 1977; Zajac, 1989; Biewener and Roberts, 2000; Roberts, 2016) so that most of the externally imposed strain is elastically stored in and recovered from the tendon (but see George et al., 2013). Use of the tendon's series elasticity to store and recover elastic energy in running is a well-known example (Cavagna et al., 1964). Robotic actuators employ a similar approach using series elastic elements for energy storage and recovery, in addition to protecting the actuator from shock loads (Raibert, 1986; Pratt and Williamson, 1995).

Second, when stabilizing a joint through muscle stiffening or regulating limb stiffness for controlling interactions with the surroundings, the stiffness of muscle is more continuously and appropriately varied to the dynamics and mechanics of the task being performed (Lacquaniti and Maioli, 1987; Lacquaniti et al., 1993; McIntyre et al., 1996; Burdet et al., 2001; Hogan and Buerger, 2005; Cui et al., 2008; Hu et al., 2011). In dynamical contexts, muscle's active perturbation response is generalized to an active impedance (Hogan, 1984; Hogan and Buerger, 2005) that may be approximately understood in terms of spring-like and dashpot-like responses to the external strain, strain-rate and the neural excitation. Varying limb impedance by modulating the impedance of the driving actuators has also been central to controlling interactions in robots (Hogan and Buerger, 2005; Vanderborght et al., 2013).

Third, some tasks involve quick transitions of muscles from a fluid-like to a solid-like response. For example, in throwing, the elbow's joint angular velocity exceeds 5000◦ /s before the elbow rapidly brakes and stiffens at the end to avoid injuries (Roach et al., 2013). Antagonistic muscles to this movement, such as the biceps brachii have to yield with little resistance like a weak dashpot would, in order to not prematurely decelerate the arm and to avoid injuries when a rapid stretch is imposed upon them (Lindstedt et al., 2001; LaStayo et al., 2003). At the end of the motion, the biceps brachii provide active braking to safely decelerate the arm without themselves suffering damage, thus transitioning into impedance control and ending with high stiffness. Such transitions in the material properties of actuators have not been studied or used in robots.

These examples illustrate that muscle's active perturbation response bridges the gap between two extremes. One extreme is the ability to maintain internal stresses that arise from an external perturbation for prolonged periods of time, lasting several minutes (Rancourt and Hogan, 2001; Loram et al., 2007), so that the muscle may function as a solid spring-like material for stability, elastic energy storage in tendons and so on. The other extreme is the ability to rapidly dissipate perturbation-induced internal stresses in mere tens of milliseconds (Roach et al., 2013) so that the muscle may function as a fluid dashpot-like material that enables agile and rapid motions without suffering damage.

## 3. CHARACTERIZATION OF MUSCLE'S ACTIVE PERTURBATION RESPONSE

Current understanding of muscle's mechanical behavior may be encapsulated by simple mathematical models such as Hilltype models of muscles (Zajac, 1989), Huxley-type models of sarcomeres (Walcott, 2014), and more intricate models of noncrossbridge elements such as the winding filament model of titin (Nishikawa et al., 2012; LeMoyne et al., 2014). Such simplified models are essential to elucidate underlying biological principles (Herzog, 2000), and to facilitate intensive computations such as applications of optimal control to study motor behaviors (Todorov, 2004). These mechanical models used to understand and characterize muscle have clear analogs in the centuries-old development of constitutive models of viscoelasticity in materials science (Barnes et al., 1989). An important lesson from materials science (Barnes et al., 1989) and from recent developments in soft (Wyss et al., 2007; Goldenfeld, 2018) and active or biological matter (Mizuno et al., 2007; Marchetti et al., 2013) has been that simple models, although often quantitatively inaccurate, guide experimental design and form the basis for fundamental mechanistic understanding.

Among viscoelastic constitutive models of materials, the Voigt model with a spring and dashpot in parallel, and the Maxwell model with a spring and dashpot in series are the two simplest approximations (Barnes et al., 1989). The Voigt model has been applied to characterize the viscoelastic properties of passive tissues, including that of muscle (Fung, 2013). But it is qualitatively wrong for the active perturbation response of muscle because it implies a single fixed length and the persistence of elastic stresses forever. We therefore consider the Maxwell model (**Figure 1B**) or a generalization called the standard linear model, which is a combination of a Maxwell body in parallel with a second weak spring representing parallel passive elasticity. These models are not literal representations of microscopic springs and dashpots, but their stiffness and damping are the respective linearized parameterizations of the reversible (elastic) and irreversible (viscous) components of the dynamic response to perturbations. In this sense, they are applicable to passive biological materials (Fung, 2013), as well as to the perturbation response of active biological materials (Deng et al., 2006; Mizuno et al., 2007).

We briefly summarize how muscle generates forces, with special focus on how it resists perturbations, and point the reader to more thorough examinations of the century-old topic of force generation in muscle (Herzog, 2000). Neural excitation in the form of a train of action potentials increases the concentration of freely available intracellular Ca2<sup>+</sup> ions, and in turn increases the number of crossbridges formed between interdigitating filaments of actin and myosin (**Figure 1B**). The crossbridges form transient load-bearing links between actomyosin filaments. By executing a power stroke, each crossbridge contributes approximately 2pN to the active tension Fa. It also acts as a molecular spring with a stiffness of approximately 1pN/nm and contributes to the active perturbation response 1F<sup>a</sup> (Finer et al., 1994). The collective behavior of crossbridges is such that they store elastic stresses when the sarcomere is externally perturbed, but slowly dissipate the stresses as crossbridges detach and re-attach elsewhere (Huxley, 1974). The presence of a relaxation timescale for stress dissipation suggests that the active perturbation response resembles a Maxwell viscoelastic material (Palmer et al., 2007).

## 3.1. Dynamic Modulus and Other Measures of Perturbation Response

The dynamic modulus is the ratio of the active perturbation response 1F<sup>a</sup> to externally applied small stretches 1x, and depends on the excitation level (or intracellular [Ca2+]) and the rate (frequency) at which the stretch is applied. The behavior resembles a pure spring if the stresses 1F<sup>a</sup> persist without any decay, and resembles a pure dashpot if the stresses decay (exponentially for a linear dashpot). Most materials exhibit both behaviors, depending on the timescales of the applied stretch and of observation. For example, a Maxwell body responds to a sudden stretch with a sharp pure spring-like transient, followed by a slower pure dashpot-like dissipation of stress (**Figure 1B**). These responses to small perturbations have also been characterized and identified as the short-range elastic component (SREC) (Rack and Westbury, 1974; Campbell, 2010), the complex modulus, or the sinusoidal response (Kawai and Brandt, 1980; Palmer et al., 2007).

The dynamic modulus arising from crossbridge dynamics is not the same as the slopes of the well-known force-length and force-velocity curves of the sarcomere (Zajac, 1989), which are sometimes misinterpreted as stiffness and damping. Although the slope of these respective curves have physical units of stiffness and damping, they do not represent the dynamic perturbation response of the sarcomere. For example, the force-length curve for a sarcomere near its natural length of ≈ 2.2µm has zero slope, which leads to a misinterpretation that an excited sarcomere has no stiffness at its natural length, a provable fallacy (Rack and Westbury, 1974). Seen from the perspective of dynamic systems, the sarcomere may be characterized as a material with a frequency-dependent perturbation response, and the forcelength relationship quantifies the zero-frequency stiffness alone (Kawai and Brandt, 1980). It is tempting to set aside these complicated (and complex) perturbation responses of muscle as biological artifacts. However, as we argue here, the frequencydependent perturbation response of muscle, and its modulation through neural excitation, are central to muscle's utility as a biological actuator.

The perturbation response of muscle has also been extensively used as a window into its microscopic functioning (McMahon, 1984). Among the many insights gleaned on sarcomere function, there remain major open questions such as the molecular origins of force enhancement (Rassier, 2012) and thixotropy (Campbell, 2010). Force enhancement (or depression) is the persistence of additional stresses for several seconds when an active sarcomere is externally stretched (contracted Edman et al., 1982). Thixotropy is a term borrowed from passive materials to referto the history dependence of a cyclically stretched sarcomere so long as the time elapsed between consecutive stretches is sufficiently small (Campbell and Moss, 2002). These and similar studies have revealed that besides the actomyosin contractile machinery, muscle's force generation and perturbation response are additionally affected by other factors such as the surrounding passive tissues (Roberts, 2016), pressure and viscosity of the intraand inter-cellular fluid (Baron et al., 2017; Sleboda and Roberts, 2017), and non-crossbridge but calcium-sensitive components like titin (Herzog, 2014; Nishikawa, 2016). Frequency-dependent dynamic modulus is related to the perturbation response protocols used in force enhancement and thixotropy studies, but generalizes to an examination of multiple timescales by adopting established protocols from rheology.

The frequency-dependent dynamic modulus of a sarcomere, myofibril, muscle fiber or a whole muscle is mostly attributable to the crossbridges only when the imposed stretches are sufficiently small so as to not forcibly detach bound crossbridges (McMahon, 1984). A natural concern therefore is whether the dynamic modulus is relevant for animal motor behaviors. Forcibly detaching crossbridges leads to microscopic damage that helps build muscle if the extent of damage is sufficiently small (LaStayo et al., 2003), but excessive damage is injurious. Thus when muscle is highly externally stretched, it is important for the sarcomere itself to remain stiff and enable the softer series elastic elements such as the tendon, the aponeuroses, and other passive tissues to accommodate a majority of the strain. In muscles with short tendons or when forcibly stretched by amounts that cannot be accommodated by tendons, the crossbridges must unbind and dissipate stresses fast enough so that a majority are not forcibly detached. Thus non-injurious perturbation response involves small elastic strains at individual crossbridges although the whole muscle or joint may experience large motions, and large crossbridge strains imply dashpot-like stress dissipation. We therefore argue that the small strain dynamic modulus of the muscle is relevant to non-pathological function, and is indeed known to play a role in a large variety of tasks (Bizzi et al., 1982; Lacquaniti and Maioli, 1987; Lin and Rymer, 1998, 2000, 2001; Loram et al., 2007; Cui et al., 2008; Rancourt and Hogan, 2009; Hu et al., 2011).

## 4. ACTIVE PERTURBATION RESPONSE OF A MAXWELL MATERIAL

A Maxwell-type response is modeled by a series arrangement of a spring with stiffness k(u) and dashpot with damping b(u), and captures the relaxation behavior of the active perturbation response 1F<sup>a</sup> (**Figure 1B**). The dependence on neural input u represents the fact that the number of crossbridges and therefore the active response of a sarcomere varies with neural excitation (Herzog, 2000). Upon imposing a step perturbation to the length, the spring initially assumes all the externally applied strain and 1F<sup>a</sup> = ℓk(u) where ℓ is the amplitude of the imposed step. The dashpot dissipates the stored elastic stress over a timescale τrelax = b(u)/k(u) and 1F<sup>a</sup> decays exponentially in time. The response resembles an elastic solid against perturbations completed over a duration shorter than the relaxation timescale τrelax and a viscous fluid against longer perturbations.

The perturbation response of a Maxwell material is characterized by its frequency-dependent dynamic modulus K. The imposed length perturbation 1x(t) is distributed between the displacement 1x<sup>s</sup> of the spring and 1x<sup>d</sup> of the dashpot so that 1x = 1x<sup>s</sup> + 1xd. Using the constitutive laws for a spring and dashpot, the active perturbation response is given by

$$\frac{d}{dt}\Delta\mathbf{x} = \frac{1}{k(\mu)}\frac{d}{dt}\Delta F\_a + \frac{1}{b(\mu)}\Delta F\_a. \tag{1}$$

The normalized dynamic modulus K/k(u) is found using the Fourier transform of Equation (1), and depends on the frequency ω of the applied sinusoidal length perturbation and the stress relaxation timescale τrelax according to

$$\frac{K}{k(\mu)} = \left| \frac{\hat{f}}{\hat{\boldsymbol{x}}} \right| = \frac{\boldsymbol{\omega} \tau\_{\text{relax}}}{\sqrt{1 + \{\boldsymbol{\omega} \tau\_{\text{relax}}\}^2}},\tag{2}$$

$$
\pi\_{\text{relax}}(\mu) = \frac{b(\mu)}{k(\mu)},\tag{3}
$$

where <sup>x</sup>ˆ(ω) and <sup>ˆ</sup> f(ω) are the Fourier transforms of 1x(t) and 1Fa(t), respectively. For fast perturbations with ωτrelax ≫ 1, the dynamic modulus K ≈ k(u), and the active perturbation response resembles a pure spring. For slower perturbations with ωτrelax ≪1, the dynamic modulus K ≈ τrelaxk(u) ω and decreases linearly with frequency. The prefactor τrelaxk(u) is equal to damping b(u), and the perturbation response resembles a pure dashpot.

The active perturbation response of skinned muscle fibers (Kawai and Brandt, 1980; Palmer et al., 2007) indeed resembles a Maxwell material's active response 1F<sup>a</sup> plus a weak passive parallel elastic spring 1F<sup>p</sup> (**Figure 2A**). The dynamic modulus measurement of skinned cardiac muscle fibers using sinusoidal perturbations of different frequencies is shown in **Figure 2A**, where activated fibers were held at the plateau region of the force-length curve and perturbed sinusoidally. Data from skeletal muscle (Kawai and Brandt, 1980; Miller et al., 2010; Palmer et al., 2013) show a similar response but those experiments did not perturb at high enough frequencies, because of which the dashpot-like response is evident but not the spring-like response.

Cardiac muscles have many physiological differences from skeletal muscles, but both are governed by similar biomechanical principles by virtue of relying on sarcomeres for active force production. Whether the Maxwell body is attributed to crossbridge dynamics, titin, or other active components of the sarcomere, the sinusoidal analysis experiments indicate that a single timescale, namely the stress relaxation timescale τrelax, separates the fluid and solid behaviors of the active perturbation response.

A non-dimensional version of the Maxwell model has no parameters and is obtained by normalizing the dynamic modulus K by the excitation-dependent high-frequency stiffness k(u), and normalizing the frequency ω by the reciprocal of the stress relaxation timescale τrelax (**Figure 2B**). Therefore if the relaxation timescale τrelax were made excitation-dependent so that the corner of the frequency response shifts right at lower excitation levels, it would also reduce the high frequency stiffness (red vs. green curves in **Figure 2B** inset). Thus, at the same frequency of perturbation, the active response may transition from a stiff solid to a weakly visous fluid if the excitation is sufficiently reduced.

## 5. VARYING THE RELAXATION TIMESCALE

The stress relaxation timescale τrelax of the active perturbation response is the ratio of the sarcomere's damping to its stiffness, and may vary with neural excitation depending on how stiffness and damping vary (**Figure 2B**). While it is known that the stiffness increases linearly with neural input (Rack and Westbury, 1974; Kirsch et al., 1994), it remains unknown how damping may vary. To examine the neural modulation of τrelax, we consider a linear form for the stiffness k(u), but allow a general power-law for how damping b(u) varies with neural input according to

$$k(u) = k\_0 u,\tag{4}$$

$$b(u) = b\_0 u^d.\tag{5}$$

Thus the stress relaxation timescale is given by the ratio of damping to stiffness as

$$
\pi\_{\text{relax}} = \pi\_0 u^{d-1}.\tag{6}
$$

Neural excitation is parameterized by the normalized variable 0 ≤ u ≤ 1, and τ<sup>0</sup> = b0/k<sup>0</sup> is the relaxation timescale at maximal excitation.

Recall that evidence from motor control (section 2) indicates that the stress relaxation timescale increases with increasing neural drive. Highly activated muscle resembles an elastic solid whose relaxation timescale is much greater than the duration of the experiment or behavior (green shading in **Figure 2C**), and minimally activated muscle resembles a weakly viscous fluid whose relaxation timescale is much smaller than the fastest perturbation in the experiment or behavior (brown shading in **Figure 2C**).

Only for d > 1 is the qualitatively correct behavior observed for τrelax as a function of u (blue solid line, left panel of **Figure 2C**), and if both stiffness and damping scale equally with excitation (d = 1) the timescale becomes an invariant quantity (blue solid line, middle panel **Figure 2C**). For d < 1 the modulation of τrelax with increasing u makes the active perturbation response weak spring-like for weak neural input and strong dashpot-like for strong neural input (blue solid line, right panel **Figure 2C**). This case would correspond to large crossbridge strains and frequent forced detachments of crossbridges at both extremes of muscle function, solid or fluid.

In all three cases, the mean active tension F<sup>a</sup> would increase with the excitation u (red dashed line, **Figure 2C**), and therefore the dependence of the relaxation timescale τrelax on the neural input u is equivalent to being dependent on the mean active tension Fa. The functionally desirable behavior is therefore one of a force-dependent slowing of stress relaxation in the active perturbation response.

Hill-type muscle-tendon models that incorporate a series elastic tendon, and treat the force-length and force-velocity curves as the stiffness and damping of the active perturbation response, exhibit a similar frequency response to the Maxwell model (Figures 16, 18, and 19 in Zajac, 1989). The highpass filtering characteristics of those models arise because the dynamic modulus of the active element is effectively zero and it behaves as a pure damper in response to perturbations. Thus the active response plus the passive elastic spring of the tendon is simply a Maxwell model. Furthermore, when the Hill-type models operate on the ascending limb of the forcelength curve, the zero-frequency stiffness of the active response is non-zero and the stress relaxation timescale (corner in the frequency response) becomes excitation-dependent. However, the functional dependence of the stress relaxation timescale

muscle, and ω is the frequency of a sinusoidal perturbation. Against perturbations with periods shorter than τrelax, the response resembles an elastic solid, and it resembles a viscous fluid for slower perturbations. inset: If the relaxation time for the active perturbation response of a sarcomere depends on its neural input *u*, we hypothesize that the sarcomere may switch between solid and fluid behaves by varying *u*. For example, the active perturbation response may be fluid-like at low neural input (green) and solid-like at high neural input (red). (C) For the hypothesized dependence τrelax ∝ *u d*−1 (Equation 6), τrelax is increasing with increasing *u* if *d* > 1, invariant with *u* if *d* = 1, and decreasing with increasing *u* if *d* < 1. Each case results in different functional consequences for a sarcomere and ultimately for muscles when coupled with the fact that a sarcomere's active tension *Fa* increases with *u*. A sarcomere can vary between a solid behavior with high active tension (mechanically stable against external perturbations, green shading) and a fluid behavior with low active tension (freely yields with minimal resistance, brown shading) only if *d* > 1.

on the excitation in the Hill-type models corresponds to the unrealistic d < 1 scenario, where increasing excitation makes the muscle-tendon more fluid-like. This increased fluid-like response may however be relevant in dynamic tasks such as locomotion, where submaximally activated muscle operates primarily on the ascending limb of the force-length curve (Holt and Azizi, 2016).

The active perturbation response of muscle is also modulated through neural feedback circuits, the stretch reflex being the fastest of them (McMahon, 1984). Even those fast reflexes take over 50ms in humans and therefore it can impact the perturbation response only at frequencies 20Hz or below. At these lower frequencies, reflexes have the capability of altering the frequency response, including the effective damping (Lin and Rymer, 1998, 2001). The analysis presented in this paper does not incorporate the effect of neural reflexes, and neither do the single fiber experiments using sinusoidal analysis (Kawai and Brandt, 1980; Palmer et al., 2007, 2013; Miller et al., 2010).

## 6. CONCLUSION

We hypothesize that an excitation-dependent increase in the damping associated with the active perturbation response should outstrip the stiffness increase. This is necessary to explain observed muscle behavior and accomplish tasks that require the muscle to mimic phase transitions between a fluid-like and solid-like active response. The microscopic origins of how the stress relaxation timescale may vary is an important avenue for future research. Whether it shares mechanisms with force-enhancement, thixotropy or even the well-known load-dependent changes in the detachment rate of actin-bound myosin remains to be discovered. Our proposal for studying the active perturbation response of sarcomeres has analogies to the applications of rheology in the fields of active and biological matter (Deng et al., 2006; Mizuno et al., 2007; Wyss et al., 2007; Marchetti et al., 2013).

Based on the known uses of muscle in control, and current theories of sarcomere function, we have argued for treating the active perturbation response of the sarcomere as a Maxwelltype material with the elastic and viscous elements in series. We note that our conclusion differs from some authors who express a preference for a Voigt element, although they conclude that both models could fit their experimental data (Ford et al., 1977). A significant reason for the differing viewpoints is because we explicitly separate the active perturbation response from all other force production components, namely the passive perturbation response, and the active and passive mean force generation, while (Ford et al., 1977) have lumped them together.

A renewed examination of the active perturbation response of muscle and its control through neural excitation may provide new design targets for engineered actuators for use in agile animal-like robots. However, significant technological challenges remain in developing actuators that can modulate stiffness and damping. The prevalent use of high-bandwidth feedback control to mimic viscoelasticity creates fragile devices that are sensitive to sensor malfunction, time-delays and noise (Hogan and Buerger, 2005). On the other hand, novel soft actuators (Majidi, 2014; Rus and Tolley, 2015; Hines et al., 2017) such as dielectric (Anderson et al., 2012) or nematic (Hines et al., 2017) elastomers, twisting cable muscles (Haines et al., 2016) or pneumatic actuators (Wehner et al., 2014; Peele et al., 2015) resemble a Voigt material rather than a Maxwell material. Therefore, they suffer the severe limitations imposed by a fixed

## REFERENCES


neutral length and a strong parallel elastic component. As an alternative, variable stiffness and damping capabilities may be achieved through mechanical design of a transmission element (Vanderborght et al., 2013). These designs also have a key limitation, namely that force, stiffness and damping are typically varied through independent control inputs and thereby lead to high control complexity. We do not imply that matching the success of muscles as actuators needs mimicry of its microscopic structure. Rather, developing novel actuators with a Maxwelltype perturbation response that can undergo large changes in the stress relaxation timescale may prove fruitful in mimicking the beneficial principles underlying muscle's versatile use by animals.

## AUTHOR CONTRIBUTIONS

KN and NS contributed equally. MV conceived the paper. KN synthesized the literature on sarcomere mechanics. NS synthesized the motor control literature. KN, NS, and MV wrote and edited the paper together.

## ACKNOWLEDGMENTS

Bertrand Tanner and Samuel Walcott for comments and discussions. Funding support from the Raymond and Beverly Sackler Institute for Biological, Physical and Engineering Sciences at Yale.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Nguyen, Sharma and Venkadesan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# A General Approach to Achieving Stability and Safe Behavior in Distributed Robotic Architectures

Stefan S. Groothuis <sup>1</sup> \*, Gerrit A. Folkertsma<sup>1</sup> and Stefano Stramigioli 1,2

*<sup>1</sup> Robotics and Mechatronics Group, Technical Medical Centre, University of Twente, Enschede, Netherlands, <sup>2</sup> Bio-mechatronics and Energy-Efficient Robotics, ITMO University, St. Petersburg, Russia*

This paper proposes a unified energy-based modeling and energy-aware control paradigm for robotic systems. The paradigm is inspired by the layered and distributed control system of organisms, and uses the fundamental notion of energy in a system and the energy exchange between systems during interaction. A universal framework that models actuated and interacting robotic systems is proposed, which is used as the basis for energy-based and energy-limited control. The proposed controllers act on certain energy budgets to accomplish a desired task, and decrease performance if a budget has been depleted. These budgets ensure that a maximum amount of energy can be used, to ensure passivity and stability of the system. Experiments show the validity of the approach.

#### Edited by:

*Navvab Kashiri, Fondazione Istituto Italiano di Technologia, Italy*

#### Reviewed by:

*Dongming Gan, Khalifa University, United Arab Emirates Takeshi Hatanaka, Osaka University, Japan*

\*Correspondence: *Stefan S. Groothuis s.s.groothuis@utwente.nl*

#### Specialty section:

*This article was submitted to Bionics and Biomimetics, a section of the journal Frontiers in Robotics and AI*

Received: *22 December 2017* Accepted: *27 August 2018* Published: *08 October 2018*

#### Citation:

*Groothuis SS, Folkertsma GA and Stramigioli S (2018) A General Approach to Achieving Stability and Safe Behavior in Distributed Robotic Architectures. Front. Robot. AI 5:108. doi: 10.3389/frobt.2018.00108* Keywords: robotics, passivity-based control, energy budgeting, interaction, safety

## 1. INTRODUCTION

For any controlled robotic system which interacts with an unknown environment, stable interaction, and safety are requirements which cannot be compromised and have to be ensured under all situations. This is even more so for physical human-robot interaction (pHRI), meaning that a robotic system interacts mechanically with a human (**Figure 1**). An example of a field of application where pHRI is fundamental is in assistive robotics, for instance as developed in the European SoftPro project (Synergy-based Open-source Foundations and Technologies for Prosthetics and RehabilitatiOn). In this project soft synergy-based robotics technologies are developed to design new prostheses, exoskeletons, and assistive devices for upper limb rehabilitation (SoftPro, 2017).

Safety during pHRI is often achieved by limiting performance aspects of the robotic system, e.g., by limiting the maximum velocity or generated force of the system. This may mean that the inherent performance of the robot is decreased since these measures are often implemented in physical ways by, for instance, low power motors and mechanical slip clutches, as opposed to controlled limitations that are bandwidth dependent and, therefore, not strictly safe in all situations (Groothuis et al., 2013). Furthermore, controlled safety measures are often implemented using digital computers, so they act on the physical system in a discrete (sampled) way.

Safety and stability have been addressed by passivity-based control schemes. Passivity means that the stored energy in a system is always less than or equal to the initial amount of stored energy plus the amount that has been added to it (Willems, 1972). In other words: passive systems cannot generate energy themselves. In Schindlbeck and Haddadin (2015), a task-based energy-tank method to obtain passivity was introduced, which cancels out non-passive terms in the system by

properly choosing the tank dynamics. At the same time, the tank allows the system to use a certain maximum amount of energy to execute a particular task. This means that during pHRI, a robot can never inject more energy into a human operator or user than what is determined by the controller. This prevents the unbounded growth of system states, and thereby increases safety. Although these continuous-time control laws were successfully implemented on a robotic arm and were proved to be passive, it is important to consider the effects of time discretization, e.g., computation delays, measurement digitization (quantization), and (variable) communication delays or even communication loss between a high-level controller, lower-level actuation controllers, or any other distributed architecture. If, for instance, communication loss results in a failure of torque command updates to any of the low-level controllers, passivity cannot be guaranteed any longer. Time delays are a common problem in telemanipulation and haptic interfaces, where passivity has already been used to stabilize systems subject to time delays. In Lee and Huang (2010), the Passive Set-Position Modulation (PSPM) method was proposed. This method passivates a system by implementing a spring coupling with damping injection between a system's position and its commanded setpoint. The setpoint can be modulated up to what is allowed to keep passivity. This method does require a model of the system. The Passivity Observer/Passivity Controller (PO/PC) as presented in Ryu et al. (2004, 2005) implements a passivity observer, monitoring the energy flowing into and out of a system, while using a passivity controller to dissipate any excess energy, i.e., energy that was not first injected intentionally, that is generated by a system. Experiments were shown, for which a precise measurement of the interaction forces was necessary. It was noted that achieving system passivity may be difficult or impossible due to actuator saturations.

This work proposes a unified energy-based modeling and control paradigm for distributed controlled robotic systems in which passivity for guaranteed stability during pHRI is used, while energy limits are imposed for safety with respect to humans. Passivity is enforced at the actuation layer, i.e., the place in a system where control messages are translated to physical energy flowing into the system. Because it is enforced at the actuation layer, no model of the complete system is necessary. Also, the approach is modular, making the extension of the system straight-forward. The paradigm is based on the fundamental notion of energy in a system, and the energy exchange between systems during interaction. A universal framework that models actuated and interacting robotic systems is proposed, which is used as the basis for energy-based and energy-limited control. This control may be a continuous-time physical (or physically equivalent) controller, or a digitally implemented discrete-time controller. Fundamentally, the controllers are distributed, as opposed to centralized or decentralized, in the sense that decision-making is done not only in the supervisory controller but also in the lower-level actuation controllers. This is analogous to organisms with a central and peripheral nervous system in case of reflex movements for instance. Furthermore, the controllers act on certain energy budgets to accomplish a desired task and take appropriate measures if a budget has been depleted. These budgets ensure that a maximum amount of energy can be used, to ensure passivity of the system. The allocation of the budgets that the controllers may use is proposed. In Schindlbeck and Haddadin (2015), this estimated allocation was done based on the virtual controller energy. Here, also the error energy is included, i.e., the difference between the desired stored energy and the actual stored energy, to more accurately estimate the necessary budget, and which is furthermore divided into individual actuator budgets. A strategy to follow in case a budget has been depleted is presented. With this approach the actuators, that are responsible for energy injection, become "energy-aware," i.e., they become aware of the amount of energy that is exchanged with a system (Stramigioli, 2015; Folkertsma et al., 2018). The high-level coordinating control loop and the adherence to passivity are separated since passivity is enforced locally in the distributed actuation controllers that are as close as possible to the mechanical system.

The paper is structured as follows: section 2 introduces concepts from port-based modeling, and presents a generic model for a robotic system. In section 3 passivity, and energyaware systems and actuation are presented. Section 4 presents the energy distributions in the systems, and treats the approach to estimate budget requirements. These requirements are translated to individual actuation budgets that are allocated. Interaction experiments with a setup were performed that are presented in section 5, and the proposed approach presented in this paper and the experimental results are discussed in section 6. The paper concludes with section 7.

## 2. NATURE-INSPIRED PORT-BASED MODELING AND CONTROL FRAMEWORK

## 2.1. Energetic Modeling Through Interconnections

Energy is a fundamental property of all physical systems. A robot is a physical system which follows the laws of nature and can exchange physical energy with the environment (a wall, an object, or a human) through a mechanical interaction. Oliver Heaviside's energy current principle states that if energy goes from one place to another, or from one subsystem to another, it

has to travel the space in between, and cannot simply disappear and reappear (Yavetz, 1995). Together with the first principle of thermodynamics, i.e., the energy of an isolated system is constant, it implies that if a system is broken into parts the system can be decomposed energetically in subsystems that exchange energy. An energy increase in one subsystem needs to be accompanied by an energy decrease of the same amount in one or more other subsystem(s). This transport of energy from one subsystem to the other can be modeled by the concept of a power port through which energy can leave one system and enter another, as shown in **Figure 2** for systems 6<sup>1</sup> and 62. The corresponding instantaneous energy change is a power flow and can be expressed as a tensor contraction of a variant and a covariant tensor which are called flow f and effort e (Duindam et al., 2009). In **Figure 2**, a positive power flow is directed from 6<sup>1</sup> to 6<sup>2</sup> and is indicated with a half-arrow called a bond. The flows and efforts in the mechanical domain correspond to velocities and forces, respectively<sup>1</sup> . In multibody dynamics, the flows will be twists and are elements of the Lie algebra T ∈ se(3), and efforts will be wrenches and are the dual elements belonging to the dual Lie algebra W ∈ se<sup>∗</sup> (3). A power port may then be indicated with (T, W).

The port-Hamiltonian formalism for the modeling of systems makes use of this principle (van der Schaft, 2006). The dynamics of any physical system can be modeled in a consistent energetic way by describing it as the interconnection of subsystems that can store energy (generalized energy storage of potential and kinetic energy), reversibly and power continuously transform efforts and flows (transformers, gyrators, and junction structures representing Kirchhoff's laws), and irreversibly transform energy to heat (resistors, dampers). The power continuous connections are composed of elements that together form a Dirac structure, which is a mathematical structure in which no energy is generated or dissipated, but only transformed and distributed (van der Schaft and Cervera, 2002). The Dirac structure determines how the ports are interconnected. Furthermore, this methodology allows to describe open systems, by defining "unconnected" ports which can be used to interconnect the system with another system. An important fundamental feature of this formalism is that the interconnection of systems in this form will again result in a system of the same form, giving rise to a "system algebra."

## 2.2. The Nervous System for Robot Control

Like humans, many organisms regulate their movements using a nervous system. The human motor control system is comprised of several components. The central nervous system (CNS) consists of the brain and spinal cord, and mainly the brain acts as high-level supervisor responsible for cognition and planning. The peripheral nervous system (PNS) consists of the nerves to connect all parts of the body to the CNS, and is responsible for local lower-level control (for instance reflex behavior together with circuits in the spinal cord) and activation of muscles. The musculatory system to move the body is supported by the skeletal system, and the latter defines the kinematic structure and its constraints, i.e., the possible movements of the body. This is a layered or hierarchical approach that can very beneficially be applied to the control of robotic systems. It is the inspiration for the universally applicable model for possibly interacting robotic systems, which is proposed and presented in **Figure 3**.

Starting from the righthand side in **Figure 3**, the robotic "skeleton" is the mechanism with a certain kinematic structure, i.e., the load. It is the power continuous interconnection of an energy storage element C, associated to the kinetic and potential energy storage of the mechanism. Inherent friction or damping is represented by R, which irreversibly transforms energy to heat, increasing the entropy. If additional physical damping is desired to specifically lower the kinetic or potential energy in the system, that energy does not have to be dissipated but can be transformed appropriately and stored in other storage elements. This makes the additional desired damping regenerative, using a transformer indicated with RS. This buffered energy can be reintroduced as useful kinetic or potential energy. The system can interact with the environment by an energy exchange, which may cause the energy storage of the mechanical system to change.

The mechanism can be actuated to do useful work and to behave in a desired way. Many actuators, as depicted to the left of the mechanism, for instance electric motors, may power the mechanical system in the same way that many muscles actuate the skeleton. The power amplifiers take their energy needed for actuation from the energy source S, which is an "infinite source" within the context of the system or situation.

The power amplifier is controlled by a local actuation controller, like the distributed PNS that locally controls the activation of muscles, as shown to the left of the actuation. This is a relatively fast controller that can act on either high-level commands or on local situations. A reflex arc in the human body, retracting one's hand from a hot surface for instance, is a local control circuit during which an appropriate action is taken without the brain being involved (Purves et al., 2012).

High-level commands can come from the supervisory control system that does the cognition and planning, like the CNS, as depicted on the far lefthand side in **Figure 3**. It can change a system's behavior by shaping the energy in the system. The energy in storage element C can increase if energy needs to be removed from the system, while it may decrease if more energy needs to be provided to the system. Any damping behavior is virtual and does not necessitate the dissipation of physical energy from the system. Instead, it can be regenerative using RS, rerouting energy back into a storage element.

This novel philosophy of modeling and controlling a robotic system is similar to the human motor control system. It is beneficial for implementing desired system

<sup>1</sup>The reverse correspondence may hold in some situations, for instance in case of generalized bond graph models. A treatment of various modeling decisions, however, is beyond the scope of this work.

properties like stability through passivity because of the explicit energetic modeling through interconnections and the layered or hierarchical control system approach.

## 3. ENERGY-AWARE SYSTEMS

Before desired energetic properties can be implemented in the system, it needs to have a way to estimate the energy that was injected or extracted. This section explains the concept of passivity, and presents a method to achieve energy-awareness in robotic systems, which was published previously (Folkertsma et al., 2018) and is summarized here for completeness.

## 3.1. Necessary Passivity

In physical systems, the property of passivity is an energybased measure of stability. It is a special case of dissipativity, as introduced in Willems (1972), that arises naturally in physical dynamical systems. General dissipativity is defined by considering a system

$$
\dot{x} = f(\mathbf{x}, \boldsymbol{\omega}), \quad z = g(\mathbf{x}, \boldsymbol{\omega}), \tag{1}
$$

where x is the system state, w the input, and z the output, which take their values in their respective manifolds X, W, and Z. The supply function is a mapping

$$s: \mathcal{W} \times Z \mapsto \mathbb{R}.$$

The system (1) is said to be dissipative if there exists a storage function H : X 7→ R such that

$$H(\mathbf{x}(t\_1)) \le H(\mathbf{x}(t\_0)) + \int\_{t\_0}^{t\_1} s(\mathbf{w}(t), \mathbf{z}(t)) \, dt \tag{2}$$

for t<sup>1</sup> ≥ t0. The system is conservative, i.e., non-dissipative, if the equality holds in (2). In physical systems, a natural choice for the storage function H is the total energy in the system; the supply function is the supplied power of the input port of the system, e.g., the contraction of mechanical force and velocity, i.e., P = F T x˙ in coordinates. Indeed, systems that contain only passive physical elements, i.e., masses, springs, and dampers, can never contain more energy than initially present. Consequently, the energy is bounded and thus a system that is overall passive is always stable (van der Schaft, 1999; Ortega et al., 2008).

If a robot interacts with an environment, the total dynamical system that should be considered is the coupled system of the robot with the environment. This can be represented as two systems coupled by an energy connection that transfers energy from one system to the other. An environment is unknown or very difficult to model adequately, and therefore it is not trivial to ensure that the coupled system is stable if feedback control is used that only considers the model of the system. The environment is not merely a disturbance, but it is part of the system.

When the system with which the controlled robot is interacting is passive, a necessary condition for the stability of the interconnected system is that the controlled robot, as seen from the port (T, W), is passive, or in other words: the energy which can be extracted via (T, W) is bounded. This is proved in Stramigioli (2015), by constructing a passive environment which would keep on extracting energy from the controlled system in case the controlled system would not be passive. In such a case, the state of the passive environment would diverge, resulting in instability. If the environment with which the controlled robot is interacting is active, the robot should not only be passive, but should be designed in a way that its damping injection is sufficient to dissipate enough energy generated by the active environment.

## 3.2. Discrete Energy-Aware Control and Actuation

When considering that the robot should be passive, a sufficient and effective way to achieve the passivity requirement is to use control by interconnection (Stramigioli, 2001). This considers, for example, the control of a system through the connection of parts which may be physically interpretable. Control of a system is then achieved by physically adapting the system by, for instance, attaching springs and dampers, or by any other energy bounded virtual dynamics. This method will not compromise passivity, and, therefore, stability. The classical way of control is to apply forces with actuators which are steered by digital controllers. This method can very likely compromise passivity of the system, because the actuators can possibly inject an unbounded amount of energy into the system. Normally, when using the second method, any state or signal of the robot is measured and an appropriate force or torque F is calculated and applied to the system, without considering what the injected power P = F T x˙ would be. This "energy-ignorant" way of control can result in an active system, thereby endangering stability.

An unbounded injection of energy can also occur in case time delays are present in the system. Passivity will be lost due to time delays since an unknown amount of energy may have been injected into or extracted out of the system in between sampling moments. Therefore, the physical energy should be monitored, giving rise to passive sampling (Stramigioli et al., 2005). Two conditions are necessary to estimate the energy, which are:


The energy transfer 1H(k) through a generic power port during a time step T is given by the integral of the power:

$$\begin{aligned} \Delta H(k) &= \int\_{kT}^{(k+1)T} e\_d^T(k) f(s) ds \\ &= e\_d^T(k) \int\_{kT}^{(k+1)T} f(s) ds \\ &= e\_d^T(k) (\varkappa((k+1)T) - \varkappa(kT)). \end{aligned}$$

A slight adaptation can be applied to obtain a computable form:

$$
\Delta H(k-1) = e\_d^\mathrm{T}(k-1)(\mathfrak{x}(kT) - \mathfrak{x}((k-1)T)).\tag{3}
$$

The effort is held constant using a ZOH, while the flow is a continuous time physical variable that is sampled. If this energy sampling concept is applied in an actuator, that actuator becomes aware of the energy it injects or extracts. This type of actuator is the Embedded Energy-Aware Actuator, or E2A 2 (Folkertsma et al., 2018). Using this type of actuator ensures passive behavior from a control perspective at the interface of the signal and energy domains, i.e., at the actuator. The actuator takes its energy from the source S in **Figure 3** to perform work, while that energy is monitored as virtual energy according to (3).

## 3.3. Beyond Passivity: Safe Interaction

Passivity ensures that never more energy can be extracted from a system than what has been added to it previously. This, however, does not entail safety with respect to humans during pHRI. Extracting a certain limited amount of energy present in a system is a passive interaction, but that limited energy may be delivered in such a way that human injury can occur. The amount and rate of energy exchanged should be within safe levels, which is the maximum energy that can be transferred during interaction without causing injury. This amount is based on a criterion like the Maximum Power Index (Newman et al., 2000; Alami et al., 2006), which is an approximation of the Wayne State Concussion Tolerance (Greenwald et al., 2008). Injury to the head is quantified as maximum power that can be transferred, which over time is transferred energy. Considering a robot that may move and interact, this limits its kinetic energy within safe levels as set a priori. Previous work has presented a way to let the system adhere to energy limits by decreasing elasticity or damping to limit the potential or kinetic energy, respectively (Tadele et al., 2014). This has been shown to be applicable to higher dimensional systems that are controlled by 6D spatial springs as well (Raiola et al., 2018).

## 4. DISTRIBUTED ENERGY AND BUDGETING

## 4.1. Physical and Virtual Energy Storage

**Figure 3** shows the model of an energy-based framework that can be used to describe the modeling and control of generic robotic systems. Energy is stored in several places, and the distinction can be made between virtual and physical energy. Physical energy is associated to the mechanism and actuation, and is stored as kinetic and potential energy in the robot. Virtual energy storage is associated to the supervisory controller and the lower-level actuation controllers, and, if implemented digitally, only exists as numbers in software. For the robot to perform desired tasks, and to do mechanical work, it needs a certain amount of energy. This energy may be present as stored energy in the mechanism, or should be injected through the environment or the actuation. To ensure a passive system, the amount of energy that the actuators inject is monitored by motor controllers through the use of the E2A 2 actuators. A virtual energy budget, representing the physical energy that may be injected by an actuator, is defined, and if that budget is depleted no further physical energy is allowed to be injected in the system by that actuator. The supervisor has an energy budget to distribute among the individual actuators based on the high-level control implementation. When the controllers are implemented digitally, budgeting is done in a discrete way, allocating energy each time step.

Due to the distributed nature of the system, the supervisor will likely be implemented on a different system than the actuation controllers. That means that the sampling frequency at which they operate may be different. It is assumed in the remainder that the supervisor has a sampling time of T<sup>k</sup> , with sampling moments k, while the actuation controllers have a sampling time of Tn<sup>j</sup> , with sampling moments n<sup>j</sup> for actuation controller j. Furthermore, it holds in general that T<sup>k</sup> > Tn<sup>j</sup> ∀ j, i.e., the supervisor is slower than the actuation controllers.

## 4.2. Energy Requirements

A fundamental question that arises is: "How can the various budgets be determined such that stability is guaranteed and system performance is not limited by conservative budgeting?."

One way to answer this question is to use a teaching approach. A robot can be externally manipulated and thereby "shown" a certain motion, which it uses to observe the evolvement of system states. These states are directly related to the kinetic and potential energy (changes) in the system, that will have to be injected by actuators, and are, hence, the actuation controller budget requirements. However, it might be cumbersome or even impossible to teach a robot a certain motion, and this seems only useful for repetitive tasks and motions. Therefore, a model-based approach is proposed here. A model of the robot is very likely developed for designing a controller, which can be directly used for estimating energy requirements during motions.

Consider a generic dynamic model of a robot in joint coordinates q, i.e.,:

$$M(q)\ddot{q} + C(q,\dot{q})\dot{q} + B\dot{q} + \mathcal{g}(q) = \mathfrak{r} + \mathfrak{r}\_{\text{ext}},\tag{4}$$

where M(q) is the inertia matrix of the robot, C(q, q˙) is the matrix associated with Coriolis and centrifugal forces, B is the joint damping matrix, g(q) are the configuration dependent potential forces, τ are the controlled joint torques, and τext are other external forces acting on the joints.

A certain motion task that is to be executed by the robot requires an amount of energy to be converted into kinetic energy. An accelerated motion will always correspond to a change of kinetic energy Ekin, and if a system moves along a gravitational field, for instance increasing and decreasing its height, the potential energy Epot will change as well. Some energy is dissipated as heat and, thereby, irreversibly removed from the system. These losses are due to friction, for instance. These energies are bounded in case of servoing tasks like position setpoint regulation from an initial condition. Therefore, the energy requirement to accomplish such tasks can be found in a straight-forward way. However, in case of periodic motions that may continue indefinitely, the energy requirement cannot be given as one energy budget that should suffice for accomplishing the task, since the required energy of the dissipative system will increase to infinity as the periodic motion execution time tends to infinity. An energy requirement for a motion during a certain time window can, however, be given. This means that for a finite time window, the energy requirement is also finite. Therefore, energy budget allocations for generic tasks and movements can only usefully be done for finite time windows. Since controllers are mainly implemented in a discrete way in computers running at a certain sampling interval, it makes sense to consider the energy requirements during each time step and providing an energy budget suitable for that time step.

In Schindlbeck and Haddadin (2015), the accurate tracking of a desired contact force was considered for which the required energy budget was estimated based only on the virtual energy in the impedance controller. A quadratic potential energy function in work space was defined, with which the energy budget was initialized. Here, the energy allocation for a time step is based on the virtual energy present in the controller, and on the kinetic and potential energy errors, i.e., the difference between the energies associated to desired system states and the actual system states. Furthermore, this energy is divided into the individual actuator controller budgets, and will be allocated as such.

### 4.2.1. Impedance Controller

Suppose a robot's end-effector is controlled to track a desired trajectory (rd,r˙d) in work space. A generic impedance controller can be given by:

$$F\_{imp} := -K\_r \, \tilde{r} - B\_r \dot{r},\tag{5}$$

in which Fimp is the virtually applied work space force to the endeffector, K<sup>r</sup> is the work space elasticity matrix, r˜ = r − r<sup>d</sup> is the deviation between the actual and desired end-effector work space configurations, and B<sup>r</sup> is the work space damping matrix. The joint coordinates can be transformed to work space coordinates using a Jacobian mapping, as given by:

$$
\dot{r} = J\_r(q)\dot{q},\tag{6}
$$

in which Jr(q) is the configuration dependent Jacobian matrix mapping joint velocities q˙ to end-effector velocities r˙. The dual relation maps end-effector forces to joint forces or torques by:

$$
\pi = J\_r^\mathrm{T}(q)F,\tag{7}
$$

in which F is the end-effector force in work space. With (7) the impedance controller of (5) can be expressed as joint space

$$\pi\_{imp} = J\_r^{\mathrm{T}}(q) F\_{imp} = -J\_r^{\mathrm{T}}(q) (K\_r \, \tilde{r} + B\_r \dot{r}),\tag{8}$$

in which τimp is the controlled end-effector force expressed as joint torques.

#### 4.2.2. Virtual Controller Energy

The mapping in (6) relates velocities, but, equivalently, infinitesimal displacements, and displacement errors, can be related by:

$$
\delta r = J\_r(q)\delta q,\\
$$

$$
\delta \tilde{r} = J\_r(q)\delta \tilde{q}.\tag{9}
$$

The change in stored controller energy due to a small displacement away from the desired (equilibrium) configuration can be given by:

$$E\_{contr} := \frac{1}{2} \delta \tilde{r}^{\mathrm{T}} K\_{r} \delta \tilde{r},$$

which, together with (9), can be expressed as a function of the joint configuration by:

$$E\_{contr} = \frac{1}{2} \delta \tilde{q}^{\mathrm{T}} J\_r^{\mathrm{T}}(q) K\_r J\_r(q) \delta \tilde{q},$$

which can be written as:

$$E\_{contr} = \frac{1}{2} \delta \tilde{q}^{\mathrm{T}} K\_r^{q} \delta \tilde{q},$$

where K q r is the end-effector controller elasticity expressed in joint space, i.e., the pull back of K<sup>r</sup> , given by:

$$K\_r^q := J\_r^\Gamma(q) K\_r J\_r(q).$$

#### 4.2.3. Energy Stored in Mechanism

Besides this controller energy, the energy in the mechanism may deviate from the energy that would be present in case the system states are as desired. That means that the total energy deviation can be given by:

$$
\tilde{E}\_{tot} := \tilde{E}\_{kin} + \tilde{E}\_{pot} + E\_{contr}, \tag{10}
$$

which is the energy that the actuation can still add to the system, and in which the kinetic and potential energy errors are defined as:

$$
\tilde{E}\_{kin} := E\_{kin\_d} - E\_{kin}, \tag{11}
$$

$$
\tilde{E}\_{\text{pot}} := E\_{\text{pot}\_d} - E\_{\text{pot}}.\tag{12}
$$

The kinetic and potential energies of the robot can be given by:

$$E\_{kin} := \frac{1}{2} \dot{q}^{\mathrm{T}} M(q) \dot{q},$$

$$E\_{pot} := \sum\_{i=1}^{I} m\_i \lg h\_i,$$

while the corresponding desired energies are defined by:

$$E\_{kin\_d} := \frac{1}{2} \dot{q}\_d^\mathrm{T} \mathcal{M}(q\_d) \dot{q}\_d,\tag{13}$$

$$E\_{\mathbb{P}^{ot\_d}} := \sum\_{i=1}^{I} m\_i \,\, g \, h\_{d\_i}. \tag{14}$$

Hence, the (desired) potential energy is defined by the (desired) center of mass height h<sup>i</sup> and mass m<sup>i</sup> of each individual body. It may be assumed that the inertia matrix in the desired configuration M(qd) is approximated by the inertia matrix in the actual configuration M(q), which is valid in nominal situations in which the time step is relatively small compared to the system dynamics.

#### 4.2.4. Individual Actuator Budgeting

Equation (10) is the total energy deviation of the robot as expressed in joint coordinates. For a distributed system, individual actuator budgets should be derived, so that each actuator explicitly adheres to the passivity requirements. Therefore, the energies derived before are separated to isolate individual actuator contributions:

$$
\vec{E}\_{contr} := \frac{1}{2} \text{diag}(\delta \tilde{q}) K\_r^{\tilde{q}} \delta \tilde{q},
$$

in which diag(. . .) is an operator transforming a vector into a diagonal matrix, and <sup>E</sup>Econtr is an array of individual actuator energy contributions. Note that the sum over all elements of array <sup>E</sup>Econtr is the scalar <sup>E</sup>contr. The same is done with the kinetic energy:

$$
\vec{E}\_{kin} := \frac{1}{2} \text{diag}(\dot{q}) M(q) \dot{q},
$$

and the potential energy:

$$
\vec{E}\_{pot} := \text{vec}\left(m\_i \lg h\_i\right),
$$

where vec(. . .) is an operator that creates an array from scalar elements i, where i identifies a mechanism body. The same is done with (13) and (14) for the desired energies.

The energy deviations of the actuation controllers can now be given in an array as the sum of the various energy contributions:

$$\begin{aligned} \vec{E}\_{tot} &:= \begin{bmatrix} \tilde{E}\_1, \ \tilde{E}\_2, \ \dots, \ \tilde{E}\_N \end{bmatrix}^\mathrm{T} \\ &= \vec{\tilde{E}}\_{kin} + \vec{\tilde{E}}\_{pot} + \vec{E}\_{contr}, \end{aligned}$$

in which <sup>E</sup>˜ j is the energy deviation associated to actuation controllerj, and <sup>E</sup> E˜ kin and <sup>E</sup> E˜ pot are the array forms of (11) and (12).

## 4.3. Energy Budget Allocation

Each supervisor time step k the actuation controller budgets are replenished up to an energy level that is calculated using the error energy. The new energy setpoint at time k = k<sup>s</sup> for controller j is indicated with Es<sup>j</sup> (k), which is sampling moment nk<sup>s</sup> for the actuation controllers.

Defining Eb<sup>j</sup> as the energy budget of controller j, E − bj = Ebj (nk<sup>s</sup> <sup>−</sup> 1), and <sup>ǫ</sup> as some small amount of energy which is explained in section 4.4, the allocation at time k can be defined as:

$$E\_{s\_j}(k) := \begin{cases} \tilde{E}\_j(k-1) \ge 0: & \tilde{E}\_j(k-1) \\\\ \tilde{E}\_j(k-1) < 0: & \begin{cases} E\_{b\_j}^- > 0: & E\_\epsilon \\\\ E\_{b\_j}^- \le 0: & \begin{cases} E\_{b\_j}^- > \tilde{E}\_j(k): & E\_{b\_j}^- \\\\ E\_{b\_j}^- \le \tilde{E}\_j(k): & \tilde{E}\_j(k) \end{cases} \end{cases} \end{cases} \tag{15}$$

Hence, a controller budget Eb<sup>j</sup> can maximally increase up to the error energy <sup>E</sup>˜ j . If more energy is still in the actuation controller budgets than what is necessary for overcoming the deviation in the states, no additional energy is needed and energy may even be removed. Excess energy will flow back to the supervisor to be distributed in future time steps. When the appropriate energy setpoint has been determined, the corresponding budget level is set to this setpoint, i.e., Eb<sup>j</sup> (nk<sup>s</sup> ) = Es<sup>j</sup> (k).

This means that the energy budget setpoint is an absolute energy level that resets a local controller budget to the setpoint, as opposed to a relative energy level causing an increase or decrease of the current local controller budget. The former method prevents drift-like issues, e.g., unnecessary virtual energy dissipation if energy budget messages are lost in the communication network.

It is assumed that the actuation controllers operate at a higher frequency than the supervisor. This means that the energy budget that is allocated is the permissible energy to be used during multiple time steps n until a new budget is allocated at supervisor time step k+1. The energy that has been used during an actuation controller time step is:

$$E\_{used\_j}(n-1) := \tau(n-1) \cdot (q(n) - q(n-1)),$$

which is the computable form as given in (3). The energy used since the energy budget was updated by a new setpoint at time nk<sup>s</sup> < n is:

$$
\bar{E}\_{used\_{\bar{j}}}(n) = \bar{E}\_{used\_{\bar{j}}}(n-1) + E\_{used\_{\bar{j}}}(n-1),
$$

and the energy left in the budget of an actuation controller is then:

$$E\_{b\_j}(n) = E\_{s\_j}(k) - \ddot{E}\_{used\_j}(n). \tag{16}$$

With this implementation of the E2A 2 , a control message communicated over a system bus is no longer just a setpoint ς<sup>j</sup> for torque, velocity, or position, but also includes the energy Es<sup>j</sup> as the new energy budget that may be used: σ<sup>j</sup> := ςj , Es<sup>j</sup> . Es<sup>j</sup> replaces the local energy budget Eb<sup>j</sup> , while ς<sup>j</sup> is executed until the next setpoint σ<sup>j</sup> is received, or until the energy budget Eb<sup>j</sup> has been depleted.

## 4.4. Energy Budget Usage

The impedance controller of (5) is realized with joint torque control, which means that the setpoints σ will be of the form σ : = (τ , Es). A physical way of representing a force controller on joint level that controls τ and that only has a limited energy budget that can be expended to perform a task, is shown in **Figure 4**. This system is based on an energy storage element with state s and stored energy H(s), that is coupled to a transformer with transformation ratio u as set by a computational unit (CU). The transformer determines how the state of the storage element is transformed to a controlled force τ applied to the mass m. In case of a unit storage element the stored energy as a function of the state is defined by:

$$H(s) := \frac{1}{2}s^2.$$

The effort e generated by the storage element is a force given by:

$$e := \frac{\mathrm{d}H(s)}{\mathrm{d}s} = s.$$

The force τ on the mass m is given by:

τ = u · e,

So to apply a desired force τ<sup>d</sup> on the mass, the computational unit will calculate a transformation ratio by:

$$u := \frac{\mathfrak{t}\_d}{s},$$

such that

$$
\mathfrak{r} = u \cdot e = \frac{\mathfrak{r}\_d}{s} \cdot e = \frac{\mathfrak{r}\_d}{s} \cdot s = \mathfrak{r}\_d. \tag{17}
$$

If the energy budget has depleted, no more energy may be injected in the system by the actuator, so the power flow P out of the actuator may not be positive, since that would further inject energy in the system. However, it may be negative as that will extract energy from the connected system. Therefore, the actuation controller should be parameterized as a function of the budget's energy content (Folkertsma et al., 2018; Raiola et al., 2018). More precisely, the calculation of the transformation ratio is parameterized as follows:

$$u = \begin{cases} \frac{\mathbf{r}\_d}{s} & \text{if } ((H(s) > \epsilon) \lor (P < 0)),\\ \frac{\mathbf{r}\_d}{\mathbf{y}^T}s & \text{if } 0 < H(s) \le \epsilon,\\ 0 & \text{otherwise}. \end{cases} \tag{18}$$

Here, γ = √ 2 ǫ, and ǫ is some small amount of energy. If enough energy is in the storage element, i.e., H(s) > ǫ, u can be such that τ = τ<sup>d</sup> according to (17). This also holds if the power is directed such that energy is extracted from the system instead of injected (P < 0). If the energy has decreased below ǫ, the force that will be applied is proportional to the energy content. This decreases the rate at which the energy budget depletes (Raiola et al., 2018). In discrete time, in which the physical storage H(s) will be a discrete calculated budget E<sup>b</sup> , there is no way to assure that the energy budget will never become negative, because of the discrete sampling of a continuous time system and the corresponding time delays. Therefore, the condition that H(s) < 0 is included, which means that u = 0.

The minimum to which the calculated energy can become negative is <sup>E</sup>bmin : <sup>=</sup> <sup>R</sup> (n+1)·T<sup>n</sup> n·T<sup>n</sup> P(t) dt. Since this is always a finite and generally small amount, it does not compromise stability if it is properly taken into account by subtracting the amount of negative energy from a new budget allocation setpoint.

## 4.5. Energy Budget Depletion

Energy in the actuation controllers is allocated from the supervisor budget, and, therefore, that budget will decrease by the same amount. It will further decrease due to dissipation in the system, which should be replenished to prevent the supervisor budget from completely depleting. If that happens, the system is unable to perform any action if no energy is added in the supervisor or to the mechanism through interaction. It may also be that not enough energy is allocated to an actuator during a time window, due to model inaccuracies for instance, which results in the system not being able to accomplish a required motion to satisfy a certain task. Depending on the application an actuation controller transmission ratio of 0, meaning an actuator force of 0, may not be desirable. Supposing only one actuator, the energy contents of the system are then not changed by actuation, which means that kinetic energy can only decrease by dissipation, i.e., a motion is not braked but only damped. Possible solutions are provided here.

#### 4.5.1. Braking the System

Whenever, a local energy budget is depleted the system may be braked by removing energy. This can be achieved by engaging a local P-controller that controls a force in the opposite direction of the movement. In that way deceleration is achieved while regenerating kinetic energy as energy in the local motor controller budget, since P = τ · ˙q < 0.

### 4.5.2. Exchanging Energy

Local actuation controllers may be given the ability to communicate with each other to exchange (parts of) energy budgets. If one controller doesn't need to use its complete budget while another needs additional energy, energy can be shifted to the controller with the depleted budget. This does not alter the total energy contents of the system and, therefore, does not compromise passivity.

### 4.5.3. Self-Replenishing Local Budgets

In a strict application of the passivity requirement through energy awareness, having insufficient energy can be considered as the inherent safety of the implementation. When a looser approach is followed, while notably still requiring unconditional stability, additional energy may be supplied to the particular budget that has been depleted. This can be done by the local controller by replenishing the energy budget with an amount of energy that is strictly less than the previously allocated energy budget. If this is recursively done, the amount of additional energy will always be finite. It is important that the self-replenished energy is communicated to the supervisor, such that it can keep track of the total energy in the system and decrease its own budget with the same amount to keep a consistent, and passive, total energy level.

The actuation controller's energy budget is given by (16). The additional energy that may be generated at time step n, Ebadd (n), is proposed to be:

$$E\_{b\_{add}}(n) := \frac{E\_s(k)}{a^l} \tag{19}$$

in which a > 1 is a factor that determines what portion of the previously allocated energy is allocated again, and l ∈ [1, L] is a counter keeping track of the number of times energy was added. L is given by the amount of time steps n in each (slower) time step k, which under communication loss can mean that L → ∞. This would mean that the total energy added is:

$$E\_{b\_{add}} := \lim\_{L \to \infty} \sum\_{l=1}^{L} \frac{E\_s(k)}{a^l} = \frac{E\_s(k)}{a-1}.$$

For <sup>a</sup> <sup>&</sup>gt; 1, the additional energy that may be used is <sup>E</sup>badd <sup>&</sup>lt; <sup>∞</sup>, while for <sup>a</sup> <sup>=</sup> 2 it becomes <sup>E</sup>badd <sup>=</sup> <sup>E</sup>s(k).

## 4.6. Complete System

A physical representation of the complete system is shown in **Figure 5**, which resembles the model as shown in **Figure 3**. It consists of actuation controllers as the one in **Figure 4** that attach to the actuators in the joints of a robot. A supervisor connects to the actuation controllers, and allocates energies in the individual controllers and sets force setpoints in the computational units. These local controllers are energy-aware and adhere to passivity requirements, which means that the mechanism's stability is guaranteed. Time delays or even communication loss between the supervisor and the actuation controllers do not compromise that guarantee.

## 5. CASE STUDY

Experiments were performed to validate the proposed modeling and energy-aware control. This was done with a five-bar linkage

system capable of end-effector movements in the horizontal plane. It uses two motors M<sup>1</sup> and M<sup>2</sup> to drive four bars in the horizontal plane. Of the four joints in the setup two are dependent due to the kinematic constraints of the linkage. Therefore, there are two actuated degrees of freedom to control two configuration variables of the end effector in the plane: x and y position, or either one of the positions x or y and the orientation. This system may be used as a slave system that is capable of versatile movements and opposing forces. This system is coupled to a similar, but smaller-scale, five-bar linkage system, both shown in **Figure 6**.

## 5.1. Implementation

The slave system consists of two Faulhaber 3890-048CR DCmotors that drive two links of 15.3 cm in length via a capstan transmission with a speed reduction factor of 7.3. Due to this transmission and low transmission ratio, the actuators are backdrivable. Two other links of 23.8 cm in length are connected to both driven links, and are coupled at the end effector to form a parallel mechanism. The rotation of the motors is measured with optical motor shaft encoders. The two motors are driven by ESCON 50/5 motor controllers in current controlling mode. The master system is similar to the slave system, using two Maxon RE25 DC-motors, each driving a link of 6.3 cm without a gear transmission, to which links of 7.5 cm are connected and coupled at the end effector to form a parallel mechanism. Again, motor rotation is measured using optical motor shaft encoders. The two motors can be driven by ESCON 24/2 motor controllers.

A kinematics and dynamics model of the two systems in the setup were developed. The forward kinematics model is based on straight-forward planar geometry which treats the two "legs"

FIGURE 6 | Two coupled five-bar linkage systems are used as the experimentation setup. The smaller system on the left is termed the master while the larger system on the right is the slave.

of the systems as 2-DOF planar serial mechanisms, which are coupled by the constraint that the end effectors of both "legs" should be in the same point in space. The end effector position of the parallel mechanism R can then also be expressed as the angles of both motors (q1, q2) after the transmissions, i.e., R(q1, q2): = - Rx(q1, q2), Ry(q1, q2) T . The analytical Jacobian J is then given by the partial derivatives of R(q1, q2) with respect to q<sup>1</sup> and q2. A physical system model of both the master and the slave setups has been developed in the bond graph modeling language with the modeling software package 20-sim (Controllab Products B.V., 2016). Energy budget calculations were done using the Euler-Lagrange model of (4).

The controller as presented in section 4 was included in the setup model. Using the 20-sim 4C package (Controllab Products B.V., 2016), realtime C-code was generated of the controller submodel which was compiled for a Gumstix embedded Linux computer. This Gumstix is incorporated on a RAMstix board that provides inputs and outputs for, for instance, encoders and motor controller PWM signals (Robotics and Mechatronics, 2018). This way, simulated behavior with a model and designed controller is directly implemented on a hardware setup for experimentation. The controller that was implemented is a combination of the supervisor controller and the actuation controllers as presented in **Figure 3**. In a distributed system the actuation controllers are in general separate controllers accepting setpoint commands to steer the power amplifiers. Here, they are implemented on the same embedded computer as the supervisor for ease of implementation, but can be run at a different sampling frequency than the one set for the supervisor. This can therefore emulate a fast motor controller communicating with a slower supervisor.

## 5.2. Experimental Method

A Cartesian planar impedance controller is implemented on the slave system, as given in (5). The master system can dictate the virtual equilibrium point in space of the slave r<sup>d</sup>

by a transformation Z between the master and slave position workspaces S<sup>m</sup> and S<sup>s</sup> :

$$Z := \mathbb{S}\_m \mapsto \mathbb{S}\_s \tag{20}$$

in which subscript m and s indicate the master and slave system, respectively. If r<sup>m</sup> ∈ S<sup>m</sup> and r<sup>s</sup> ∈ S<sup>s</sup> , the affine workspaces transformation is defined as:

$$r\_d = \alpha \left(r\_m - r\_{m\_0}\right) + r\_{s\_0},\tag{21}$$

in which α is a scalar parameter isotropically scaling the workspace, and rm<sup>0</sup> and rs<sup>0</sup> are the initial positions of the master and slave end effectors, respectively.

This master and slave setup lends itself for the application of repetitive motion rehabilitation given to a patient (the slave operator) by a therapist (the master operator). During experimentation, the slave system is used to perform a back-andforth motion with an arm. The master operator may support the person by slightly preceding the person's intended movement and thereby causing the impedance controller to pull the person's arm along. The master operator may also obstruct the person by opposing the intended motion causing the person to push against the impedance controller or even completely deviating from the path, to train motion accuracy. Specifically, a back-andforth motion was performed with the slave in the master's initial end effector position, and after a while the master position is moved to obstruct the slave operator. Furthermore, the virtual spring's equilibrium point of the impedance controller, as set by the master system, is tracked in free space, and the slave system is manipulated (to charge the virtual impedance controller energy) and then released to assess whether instability occurs. These experiments are performed with the energy-based control paradigm as presented, as well as with a traditional controller that is unaware of injected energy, both implemented in discrete time.

Unfortunately, it was observed that the slave system has a relatively high stiction and friction that are position dependent, which is most likely due to bearing misalignement and highly tensioned tendons. Therefore, free space tracking of a virtual spring equilibrium point is relatively inaccurate and jerky. No immediate changes could be made to the setup to solve this issue.

## 5.3. Experiments

The experiments were performed with an impedance controller, which in general is a virtual spring and damper. In this case, however, due to the high friction setup the impedance controller was implemented as only an isotropic spring with elasticity of 50 N/m. The workspace scaling parameter was set at α = 2.

### 5.3.1. Understanding the Energy-Based Controller

To show the workings of the energy-based control paradigm, an identifying experiment was done at 100 Hz with an initial supervisor energy budget of 1 J, which may be used to inject energy in the system via the motor controllers. The end effector position of the master is shown in **Figure 7**, the end effector position of the slave is shown in **Figure 8**, and the energy levels associated to the slave are shown in **Figure 9**. During the experiment one slave motor was externally manipulated between 7 and 20 s, followed by dictating an equilibrium point trajectory by the master while the slave is kept fixed between 22 and 35 s, followed by a free moving slave tracking the equilibrium point as set by the master between 35 and 50 s. In the initial configuration, the motor controller budgets are empty, since the error energy is zero. That means that the motor controllers do not allow any positive power to flow from the motors to the mechanical system. However, it can be seen in **Figure 9** that upon external manipulation of only one slave motor, the corresponding budget (Controller 1) increases due to the power outflow of the motor being negative, i.e., the motor absorbs power from the system. This means that the total energy in the system, which is the sum of the supervisor and motor controller budgets, is increased. When keeping the slave fixed while manipulating the master (in such a way that again only one slave motor would have to be actuated), energy from the supervisor does flow to the motor controller budget. Note that only that particular motor budget receives energy to perform a task, as properly calculated according to (15).

This allocated energy is what has been calculated to be necessary in order for the slave to compensate the error energy in the system. When the master end effector returns to its initial position, the energy content in Controller 1 decreases and flows back to the supervisor, since that energy is not necessary anymore for decreasing the error energy. Note that the total energy in the system is constant. When releasing the slave and performing the same motion with the master as before, energy is allocated from the supervisor to the budget, and the total energy decreases due to dissipation. "Controllers" indicates the sum of the motor controller energy budgets, "Mechanism" indicates the kinetic energy of the slave, and "Virtual" indicates the virtual impedance controller energy.

### 5.3.2. Traditional vs. Energy-Based

The supervisor and the motor controllers are both set at a sampling frequency of 100 Hz, and the back-and-forth motion, tracking the virtual equilibrium point with a free moving slave, and manipulating and suddenly releasing the slave were sequentially performed with both a traditional and the energybased controller.

The performance of the traditional controller is shown in **Figures 10**–**12**. It can be seen that the discrete implementation of the traditional impedance controller at this sampling frequency is stable during back-and-forth motion and free space tracking of the equilibrium point. However, manipulating the slave and suddenly releasing it induces unstable oscillations in the system, which means that passivity has been lost.

The performance of the energy-based controller is seen in **Figures 13**–**15**, which is similar to the traditional controller. However, now all sequential experiments, including charging and suddenly releasing the slave, remain stable as opposed to the traditional controller.

The energy-based controller is aware of the energy that is injected into the system and can, therefore, ensure passivity and stability of the system. The traditional controller may violate passivity and allow so much energy injection in the system that it becomes unstable. The effectiveness of this paradigm has been clearly demonstrated in the previous experiments.

FIGURE 13 | Master end effector position; energy-based 100 Hz supervisor.

## 6. DISCUSSION

The experiments shown here verify that the energy-based controller is able to keep the passivity of a system, as opposed to a traditional straight-forward implementation. This shows that the proposed approach is a viable method to control robotic architectures to ensure stability while obtaining adequate performance. The energy-based controller will not create instability in the system due to the awareness of the amount of energy that has been injected in the system. Also, performance is not considerably decreased due to the appropriate estimation of the actuation controller budgets in this situation. The traditional controller is unaware of the injected energy and will become unstable and, therefore, unusable in certain situations. A complete and persistent communication loss between the supervisor and the actuation controllers will also not cause instability, since the actuators are aware of the injected energy. It is noted that this approach is independent of the chosen control algorithm in the high-level supervisor. Here, an impedance controller is implemented and is mainly used for interaction, while it was briefly shown that is also works for a position regulation task without interaction. If an energy error can be defined between the desired system states and the actual system states, and the two conditions as mentioned in section 3 are fulfilled, energy budget allocations can be estimated and passivity can be successfully implemented using this approach.

The experiments performed here were done with a controller that is based on a model of the system. Energy budget estimations may deteriorate when the controller approaches the model validity boundaries, which can cause the energy estimation to be too generous or conservative. This mainly affects the performance of the system, since the energy budgeting is still limited, so unbounded growth of system states cannot occur. Moreover, the proposed method assumes perfect measurements of the actuation torque and velocity or position, while they may be different from their setpoint or may be quantized to implement in a digital controller. The effects of these issues have been partially investigated in Stramigioli et al. (2005), and should be further researched in the future.

An important aspect to realize, which is also not treated in this work is that energy budgets will be communicated over a network architecture as packets. This means that the total energy is not only the sum of the energy budgets in the supervisor and individual controllers, but also the "traveling" energy in the network. To accurately keep passivity of the system, this aspect should be taken into account.

## 7. CONCLUSION

This paper proposed a unified energy-based modeling and energy-aware control paradigm for robotic systems. It presented an energy-based modeling framework that is applicable to any robotic system, in which the energy transfer between subsystems is made explicit. It considers the separation of a high-level, and possibly relatively slow, supervisory control system, and the lower-level, and likely relatively fast, actuation controllers. By implementing energy-awareness on the actuation controllers, stability can be guaranteed through passivity, even under large time delays or communication losses. The actuation controllers can expend an energy budget to fulfill a certain task, and are incapable of injecting more energy if the budget has depleted. An approach to allocate these budgets was proposed, while a strategy was presented to follow when a budget has depleted. Experiments validated that the proposed method is capable of stably controlling an

## REFERENCES

Alami, R., Albu-Schaeffer, A., Bicchi, A., Bischoff, R., Chatila, R., De Luca, A. (2006). "Safe and dependable physical human-robot interaction in anthropic domains: state of the art and challenges," in 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems (Beijing), 1–16.

Controllab Products B.V. (2016). 20-sim.


interacting mechanical five bar linkage system, as opposed to a traditional controller which destabilizes the system quickly.

## AUTHOR CONTRIBUTIONS

SG was involved in the development of the conceptual framework that is presented in the manuscript and the implementation on a physical system, and has taken the lead in writing the manuscript and shaping it into its current form. SS was involved in the preliminary work related to this research topic, and contributed to the conceptualization of the framework presented in the manuscript. GF was involved in earlier work which has formed the substantial basis for the presented work in this manuscript, and was involved in furthering the research topic. Moreover, GF has provided help with proof reading. Also, SS has provided help with proof reading and shaping the manuscript into its final form.

## FUNDING

This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No. 688857 (SoftPro). The content of this publication is the sole responsibility of the authors. The European Commission or its services cannot be held responsible for any use that may be made of the information it contains.

for collaborative robots. IEEE Robot. Automat. Lett. 3, 1237–1244, doi: 10.1109/LRA.2018.2795639


SoftPro (2017). SoftPro. Available online at : http://softpro.eu


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Groothuis, Folkertsma and Stramigioli. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Highly-Integrated Hydraulic Smart Actuators and Smart Manifolds for High-Bandwidth Force Control

*Victor Barasuol 1\*, Octavio A. Villarreal-Magaña 1, Dhinesh Sangiah 2, Marco Frigerio 1,3, Mike Baker 2, Robert Morgan 2, Gustavo A. Medrano-Cerda 1, Darwin Gordon Caldwell 1 and Claudio Semini 1*

*1 Department of Advanced Robotics, Istituto Italiano di Technologia, Genoa, Italy, 2 Moog (United Kingdom), Tewkesbury, United Kingdom, 3 Faculty of Engineering Sciences, KU Leuven, Leuven, Belgium*

#### *Edited by:*

*Monica A Daley, Royal Veterinary College, United Kingdom*

#### *Reviewed by:*

*Carlo Ferraresi, Dipartimento di Ingegneria Meccanica e Aerospaziale (DIMEAS), Politecnico di Torino, Italy Kevin Haninger, Fraunhofer Institute for Production Systems and Design Technology, Germany*

> *\*Correspondence: Victor Barasuol victor.barasuol@iit.it*

#### *Specialty section:*

*This article was submitted to Humanoid Robotics, a section of the journal Frontiers in Robotics and AI*

*Received: 24 December 2017 Accepted: 12 April 2018 Published: 14 June 2018*

#### *Citation:*

*Barasuol V, Villarreal-Magaña OA, Sangiah D, Frigerio M, Baker M, Morgan R, Medrano-Cerda GA, Caldwell DG and Semini C (2018) Highly-Integrated Hydraulic Smart Actuators and Smart Manifolds for High-Bandwidth Force Control. Front. Robot. AI 5:51. doi: 10.3389/frobt.2018.00051*

Hydraulic actuation is the most widely used alternative to electric motors for legged robots and manipulators. It is often selected for its high power density, robustness and highbandwidth control performance that allows the implementation of force/impedance control. Force control is crucial for robots that are in contact with the environment, since it enables the implementation of active impedance and whole body control that can lead to a better performance in known and unknown environments. This paper presents the hydraulic *Integrated Smart Actuator (ISA)* developed by Moog in collaboration with IIT, as well as smart manifolds for rotary hydraulic actuators. The ISA consists of an additive-manufactured body containing a hydraulic cylinder, servo valve, pressure/position/load/temperature sensing, overload protection and electronics for control and communication. The ISA v2 and ISA v5 have been specifically designed to fit into the legs of IIT's hydraulic quadruped robots HyQ and HyQ-REAL, respectively. The key features of these components tackle 3 of today's main challenges of hydraulic actuation for legged robots through: (1) built-in controllers running inside integrated electronics for high-performance control, (2) low-leakage servo valves for reduced energy losses, and (3) compactness thanks to metal additive manufacturing. The main contributions of this paper are the derivation of the representative dynamic models of these highly integrated hydraulic servo actuators, a control architecture that allows for highbandwidth force control and their experimental validation with application-specific trajectories and tests. We believe that this is the first work that presents additive-manufactured, highly integrated hydraulic smart actuators for robotics.

Keywords: hydraulic actuation, legged robot, servo valve, integrated smart actuator, dynamic modeling, control architectures, additive manufacturing, low leakage

## 1. Introduction

Hydraulic actuation is the most widely used alternative to electric motors for legged robots and manipulators. It is not only the high power-to-weight ratio and the high control bandwidth (Mavroidis et al., 1999; Siciliano and Khatib, 2007) that make hydraulic actuators interesting. Other major advantages are related to the inherent properties of hydraulic oil that acts as a lubricant as well as cooling liquid. Despite these advantages, hydraulic actuation suffers from a number of short-comings. The recently published *Survey on Control of Hydraulic Robotic Manipulators with Projection to Future Trends* (Mattila et al., 2017) mentions two of them: First, the difficult


TABLE 1 | Comparison of the specifications of some servo valves used in hydraulic legged robots.

*All values are taken from the data sheets mentioned in the footnotes 1 to 6. An asterisk (\*) indicates that a value has been estimated from a plot.*

controller design due to the nonlinearity of the hydraulic system dynamics and second, the low energy efficiency of traditional closed-loop hydraulic systems. Additionally, compact hydraulic actuation components are rare on today's market and their compact integration into articulated robots is challenging (Semini et al., 2011). Hydraulic servo actuators have been used for several decades in legged robots. Marc Raibert's early hopping robots (e.g., the 3D one-legged hopping machine) were driven by hydraulic actuators that combined a low friction cylinder, position sensor, velocity sensor and pressure control servo valve (Raibert, 1986). Raibert continued using similar custom actuators for the robots developed by his company Boston Dynamics, Inc. (BDI). The legs of *BigDog*, for example, are powered by a custom hydraulic actuator with a Moog Series 30 servo valve<sup>1</sup> , cylinder, load cell and potentiometer (Buehler et al., 2005). The servo valves allowed the actuators to be controlled in force as well as in position. Subsequently developed BDI robots like LS3, Cheetah, Wildcat, ATLAS and Spot use hydraulics<sup>2</sup> , but no detailed information about the servo actuators is available. IIT's HyQ robot uses a custom hydraulic servo actuator that consists of a Moog E024 servo valve<sup>3</sup> , a Hoerbiger cylinder, a custom hydraulic manifold, 2 pipes, a load cell, an absolute and relative joint encoder, and electronics for sensors and valve amplifiers (Semini et al., 2011). Boaventura et al. presented highperformance force control on these actuators (Boaventura et al., 2015). Another force-controlled hydraulic actuation unit was developed by Hyon et al. for a light-weight hydraulic leg (Hyon et al., 2013) that was later used as the basis for the actuation of the joints of a hydraulic humanoid robot called *TaeMu* (Hyon et al., 2017). The actuator units use PSC AS110 servo valves<sup>4</sup> . The *BabyElephant* robot (Gao et al., 2014) is powered by custommade hydraulic actuators called *Hy-Mo*. These actuators consist of a hydraulic cylinder, an electric motor to move the spool of the valve and pressure sensors to measure the two chamber pressures (Wang et al., 2014). Another example is the hydraulic force control unit of the hydraulic quadruped robot BITDOG (Lu et al., 2015). The servo actuator consists of a hydraulic servo valve, hydraulic actuating cylinder, displacement sensors, pressure sensors and shock absorber. The actuator's active compliance controller was presented in Lu et al. (Lu et al., 2015). The ROBOCLIMBER is a 4,000 kg quadruped machine with force-controlled prismatic legs. Montes et al. present various force control strategies for the hydraulic cylinders (Montes and Armada, 2016).

Besides academic prototypes, there are also a few hydraulic servo actuators available on the market, such as the Moog *A085 Series Servo Actuators*<sup>5</sup> that combine high performance cylinders, linear feedback devices and servo valves in one assembly. These actuators were recently installed on a Menzi Muck walking excavator to implement active chassis balancing with force controlled cylinders (Hutter et al., 2017). The force was estimated with hydraulic pressure sensors inside the two cylinder chambers. Other commercial examples are the linear and rotary actuators developed by KNR systems (Kim et al., 2014) that feature KNR series KSV070 servo valves<sup>6</sup> . A combination of KNR actuators and Star Hydraulics series 200 servo valves<sup>7</sup> are used for the hydraulic quadruped robot Jinpoong developed by the Korea Institute of Industrial Technology (KITECH) (Cho et al., 2016).

A related class of integrated hydraulic actuators are electrohydrostatic actuators (EHA) that combine an electric motor, hydraulic pump, small tank, and rotary/linear actuator into one unit. Two examples from academia are the following: Alfayad et al. have recently presented the *IEHA - Integrated Electro Hydraulic Actuator* for a hydraulic humanoid called *Hydroid* (Alfayad et al., 2011). Kaminaga et al. have been developing EHAs for robotic hands and recently for the humanoid robot called *Hydra* (Kaminaga et al., 2014). The remainder of this paper will exclusively focus on actuators driven by servo valves.

Most high-performance, hydraulic legged robots (see list above for examples) rely on miniature servo valves to control the actuators in their legs. The fast response and high control bandwidth of servo valves, when compared to other types of valves, allow the implementation of force/torque control, which is an important building block to achieve robust locomotion (Semini et al., 2015). The high bandwidth of servo valves, however, has its price: The pilot stage has a continuous internal leakage (the so-called *tare leakage*) that routes part of the flow directly from the pressure port back to the return port. This *wasted* flow leads to a generally low energy efficiency of servo hydraulics. Note that the tare leakage does not include the spool null leakage.

**Table 1** compares the most important specifications of the abovementioned servo valves. The last column show the specifications of the ISA v5 that will be introduced next.

<sup>1</sup>http://www.moog.com/content/dam/moog/literature/ICD/Moog-Valves-30series-datasheet-en.pdf

<sup>2</sup>The most recently presented *SpotMini* is BDI's smallest and lightest (30 kg) quadruped and it uses electric actuation.

<sup>3</sup>http://www.moog.com/literature/ICD/e024seriesmicrovalves-ds.pdf 4http://www.psc-net.co.jp/category/1359255.html

<sup>5</sup>http://www.moog.com/products/actuators-servoactuators/industrial/hydraulic/ a085-series-hydraulic-servo-actuators.html

<sup>6</sup>http://rnd.knrsys.com/english/view.html?id\_no=48

<sup>7</sup>http://www.star-hydraulics.co.uk/assets/data-docs/product%20pdfs/ST-200- 2017-2-En.pdf

This paper presents two versions of the new Integrated Smart Actuators (ISA) developed by Moog in collaboration with IIT, as well as a smart manifold on a rotary actuator. The ISA consists of a hydraulic cylinder, servo valve, various sensors, overload protection and electronics for control and communication. Its body is additively manufactured (AM) in a titanium alloy, allowing a very compact design with integrated flow paths and wire channels. The two presented versions ISA v2 and ISA v5 (**Figure 1A,B**) have been specifically designed to fit into the legs of IIT's hydraulic quadruped robots HyQ (Semini et al., 2011) and the newest version HyQ-REAL (under construction). The last column of **Table 1** shows the key specifications of the new ultra-low leakage valve inside the ISA v5 and smart manifolds (see Section 2).

The main contributions of this paper are the derivation of the representative dynamic models of these highly integrated hydraulic servo actuators, a control architecture that allows for high-bandwidth force control and their experimental validation with applicationspecific trajectories and tests. The key features of these components tackle the disadvantages of hydraulic actuation for legged robots through: (1) built-in controllers running inside integrated electronics, (2) low-leakage servo valves, and (3) compactness thanks to metal additive manufacturing. We believe that this is the first work that presents additive-manufactured, highly integrated hydraulic smart actuators for robotics. The ISA has been mentioned in a paper on additive manufacturing by Moog's Guerrier et al. presented at the 2016 conference on Recent Advances in Aerospace Actuation Systems and Components (Guerrier et al., 2016). A short overview of the ISA has been presented during the IROS 2016 workshop on *The Mechatronics behind Force/Torque Controlled Robot Actuation: Secrets and Challenges* (Semini et al., 2016).

This paper is structured as follows: Section 2 presents the main features and specifications of the ISA and smart manifolds. Section 3 describes the control architecture running on the actuator's embedded ARM processor, and Section 4 explains the mathematical model of the actuator dynamics. Section 5 presents the results of simulation and experiments, followed by the conclusions in Section 6.

## 2. ISA and Smart Manifold Features

The linear actuator ISA v2 was developed with the goal of integrating standard hydraulic components by means of additive manufacturing to create an optimized actuator unit. The ISA v2 has a high response valve and the most important sensors for position/force control and self-protection (temperature and mechanical impacts). **Figure 2A** shows a CAD rendering of the ISA v2 with a section to illustrate the main features of the actuator and integrated components.

The use of ISAs in a legged robot has several advantages on system level: First, it reduces the overall complexity of the machine, since the various actuator components are combined into one device. Sensor wires are routed inside the AM body and several components are merged into the same electronic board (e.g., microcontroller, valve amplifier, Inertial Measurement Unit (IMU), temperature sensor). Fewer and shorter wires result in higher reliability and less signal noise. Second, it reduces the total robot weight, and increases its ruggedness. For an illustrative comparison, **Figure 3** shows the linear actuator units of a HyQ leg with all the components that belong to one unit. Note that the shown A/D converter and communication electronic boards - shown on the left of the bottom-right picture - are connecting

to 3 actuators in total. The black box to its right hosts the electronics for 6 valve amplifiers. The total estimated weight of the components of 1 actuator unit (excluding the electronics) is 1.3 kg. The corresponding ISA v2 on the other hand weighs only 0.92 kg and additionally includes pressure relief valves for overload protection, as well as all electronics to close local control loops.

The need for more efficient actuators in autonomous robots triggered the redesign of the linear actuator ISA v2 to achieve the version ISA v5, with a compromise between high performance and energy wasting.

The combination of additive manufacturing and standard parts allows for customisation of actuators and retain all of the functionalities that of the ISA. This idea has been used to create the smart manifolds to provide most of the ISA technology to custom made rotary actuators as shown in **Figure 2B**.

In this section we present the most important features and specifications, regarding the mechanical and electronic components, of the integrated smart actuators and smart manifolds.

## 2.1. Integrated Servo Valve for High Performance (ISA V2)

The integrated servo valve is a derivative of the high performance version of the Moog E024 series sub-miniature servo-valves (MOOG Inc, 2015) (valve used in the HyQ robot). The high bandwidth of around 320 Hz allows high performance force and position control, as previously demonstrated in (Boaventura et al., 2015). The frequency response for the high response valve (HRV) used in the ISA v2 is shown in **Figure 4**.

## 2.2. Integrated Servo Valve for High Performance with Improved Efficiency (ISA V5)

ISA v5 incorporates an ultra-low leakage valve (ULLV) technology to considerably reduce throttling losses and improve the efficiency of the overall unit. The pilot stage of the valve is modified to improve tare leakage and still have high bandwidth

of greater than 100 Hz as shown in **Figure 4**. The second stage peak leakage is reduced with valve overlap and dual gain flow slots. **Figure 5** compares the flow gain of the valve configuration to that of the high response valve used in ISA v2. Here the v5 has low-flow gain near null and high-gain stroke greater than 25%. The low gain flow slots near null considerably reduces losses and reduces nonlinearity in the flow curve as opposed to purely overlapped valves. **Figure 6** compares the leakage flow of ISA v5 (ULLV) to that of ISA v2 (HRV). When the actuator is stationary the leakage of ISA v5 is approximately 36% of ISA v2. This results in a power saving of approximately 112W per actuator.

## 2.3. Unequal Area Flow Slots (ISA V5)

ISA v5 has unequal area flow slots which match the area ratio of the actuator. This results in constant actuator velocity during retraction and extension and removes the need for gain scheduling based on the piston motion direction.

## 2.4. Integrated Sensors for Position, Pressure and Force Measurement

A 1Mbps absolute position sensor is used to measure the position of the piston rod. The two cylinder chamber pressures of the actuator are measured using strain gauge based miniature pressure sensors. Actuator force is measured using a strain gauge based load cell integrated in the actuator tail stock. The electronic board additionally includes an IMU and ports for optional sensors.

## 2.5. Integrated Electronics to Control Actuator Position/Force

The design of the Remote Electronics Unit (REU) follows Moog's commercial aircraft flight control standards. It closes several control loops (see Section 3) on an ARM processor and offers various communication options such as, e.g., EtherCAT, CAN and Serial bus.

FIGURE 5 | Flow gain curves for the HRV (blue line) and ULLV (black line). The HRV presents a constant flow increasing of about 2.3 l/min/mA until near 2 mA, when the valve gain start decreasing. The ULLV, instead, presents a low flow gain of 0.4 l/min/mA and a high flow gain of *±*2.5 l/min/mA inside and outside the current range of *±*1.2 mA, respectively. Note: these curves were obtained from experimental measurements considering the valve redundant coils (2 coils) in series. The ISA and smart manifolds integrate these valves with the coils connected in parallel to increase the operation safety. In this case, the scale of the current is doubled and the current command ranges from *±*10 mA.

## 2.6. Valve Spool Feedback

The smart manifold has a mechanical feedback valve (MFB valves), i.e., the spool position is indirectly controlled from the valve current command. The ISA v2 and v5, instead, have electric feedback valves (EFB valve) and the spool position can be fed back in a specific control loop. Electric feedback technology allows for a valve response less sensitive to the null bias (which is dependent on the return and supply pressures and on the oil temperature).

FIGURE 6 | Valve tare and null-region leakages for the high response valve (blue curve) and for the ultra-low leakage valve (black curve). Experimental measurements show about 0.06 l/min and 0.28 l/min of tare leakage for the ULLV and HRV valves, respectively. Leakage peaks at spool null region are about 0.5 l/min for the HRV and 0.17 l/min for the ULLV. Considering an operating pressure supply of 20.7 MPa, the ULLV may save about to 41 W due to tare leakage and up to 112 W at the spool null region.



## 2.7. Integrated Overload Protection

Pressure relief valves limit the pressure inside the two cylinder chambers. These valves vent to return if the chamber pressures reach 22 MPa thus resulting in an effective and repeatable overload protection.

**Tables 2, 3** show the most relevant mechanical and electrical features of the ISAs and smart manifolds, respectively.

## 3. Control Architecture

This section describes the standard control loops available on the Remote Electronics Unit (REU) that is integrated in the smart actuators (i.e., ISA v2, ISA v5 and smart manifolds). The REU, also called thumb-REU for its small board size, has standard nested control loops that can be configured according to the actuator hardware. For example, for the more innovative lines (the ISA v2 and v5 with electric feedback valves), it is available a control block with a set of functionalities to control the spool position. For the case of smart manifolds, which have mechanic feedback valves and no spool position measurements, such control block can be reconfigured to control another state of the actuator. By available control block we mean a block with a set of control functionalities that can be used by the control designer. Such functionalities are, e.g.,: PID controllers, feedforward terms, feedback terms, filters, selection of control loop frequency, saturation functions, anti wind-up for integral actions, gain scheduling, offsets, and logic blocks for nonlinear compensations.

The standard firmware of the thumb-REU provides four control blocks where each one can be configured to be associated to one of the following actuator variables: piston/rotor position, pressure difference between the actuator chambers (i.e., ∆*P*), actuator force/

TABLE 3 | Electrical specifications of the boards for the ISA (both v2 and v5) and the smart manifold.


FIGURE 7 | Control diagram illustrating the arrangement of control blocks for each actuator. (A) Control architecture for the smart manifold mounted on the custom rotary actuator with three nested loops: rotor position control, torque control (at load level using a torque sensor), and the valve current control. (B) Control architecture for the ISA v5 with four nested loops: piston position, force control (at load level using a load cell), spool position control, and valve current control. The control loop frequency selected for each block is indicated on the top-left corner of each respective translucent color box. A velocity compensation term (*KvX*˙ ) and a valve current compensation block DGC (due to the valve dual-gain feature) is used in both control architectures. Details on the PID actions and additional terms are described in Section 5.

torque, spool position and valve current. Each control block, though, can be activated or deactivated at will, to achieve different control strategies. For example, to perform pure actuator position control, force/torque control, or even activate only the valve current loop to use the thumb-REU as a simple valve amplifier.

**Figure 7** shows a simplified view of each control architecture (arrengement of the available control blocks) that was used to test the ISA v5 (shown in **Figure 1B**) and the smart manifold with the custom made rotary actuator (shown in **Figure 2B**). The frequency of the control loops range from ~1 kHz for the piston/rotor position to ~10 kHz for the valve current. Both control diagrams show a force loop with an outer position loop that leads to an impedance control, which is an essential strategy for legged systems and will be used in the HyQ-REAL robot.

Both control architectures shown in **Figure 7** have a force/torque control loop with feedback of the force/torque measured at the load level (instead of performing hydraulic force/torque control), what minimizes tracking issues due to the internal hydraulic actuator friction. To compensate for the pressure dropping due to the piston/ rotor motion, a velocity compensation term is used to inject extra flow into the chambers. The extra current command from the velocity compensation is modulated according to the dual-gain feature of the ULLV to avoid a flow over compensation, at the valve region of high gain, that can make the overall control loop to be unstable.

For the case where the velocity compensation is introduced at the level of the spool control loop (see **Figure 7B**), *Kv* is tuned to take into consideration the relationship between the spool opening and the valve flow. Details on the PID actions, of each block, and the additional terms considered for the test of the actuators are described in Sec. 5.

## 3.1. Current Loop Tuning

The current control loop of the smart actuators is the innermost loop and it is the loop that has negligible coupling with the load dynamics. Moreover, such decoupling allows for a control tuning only dependent on the internal dynamics of the valve, i.e., the valve coil dynamics.

Given the importance of the current loop dynamics for the outer loops, we design a current controller that aims for a small phase lag between the desired and actual valve current. We propose a control structure in a 2-DOF (degrees of freedom) configuration where: (1) the closed-loop has the forward path composed of a proportional action, in series with a lead-lag compensation of second-order; and (2) a gain *Kpf* applied on the demand to reduce steady-state errors. The gain *Kpf* applied on the demand compensates for the steady-state error due to the absence of integral action (avoided to reduce the phase lag in the response). The lead-lag compensation is implemented to supress high-frequency resonance modes of the valve coil that are excited as the proportional action is increased. **Figure 8A** shows a step input response test performed in openloop to obtain the static gain, here denoted as *Ksg* between the desired current and the actual current. **Figure 8B**, instead, shows the apearance of a high-frequency resonance mode, around 1.8 kHz,

for a step input response test when the current loop is closed with a certain proportinal action.

The transfer function of the current controller, named as PID + LLC in **Figure 7**, is described in the Laplace domain as:

$$C\left(s\right) = K\_P \frac{\left.\omega\_{lag}^2 \left(s^2 + 2\zeta\_{lead}\omega\_{lead}s + \omega\_{lead}^2\right)}{\left.\omega\_{lead}^2 \left(s^2 + 2\zeta\_{lag}\omega\_{lag}s + \omega\_{lag}^2\right)}\right)}\tag{1}$$

where *Kp* is the proportional gain of the current loop, *wlead* and *wlag* are the natural frequencies for the lead and lag compensations, respectively, and *ζlead* and *ζlag* are the damping ratios for the lead and lag compensations, respectively.

The gain *Kpf* can be computed from *Kp* and *Ksg* as:

$$K\_{\mathcal{P}\mathcal{f}} = \frac{1 + K\_{\mathcal{P}} K\_{\mathcal{S}\mathcal{g}}}{K\_{\mathcal{P}} K\_{\mathcal{S}\mathcal{g}}} \tag{2}$$

The values used for the current loops of both the ISA and the Smart Manifold are described in (**Table 4**). With such tuning one obtains a current control loop with response time of less than 2 ms.

## 4. Actuator Modeling

In this section the dynamics of the hydraulic actuators are described8 . The main goal is to provide a representative mathematical model, TABLE 4 | Controller gains implemented in the system.


*Gains marked with an asterisk (\*) may change depending on the task.*

from the valve to the load dynamics, in order to: (1) complement the simulation of the rigid body dynamics of a legged system making it more realistic; (2) help designing new control strategies to improve force/torque tracking performance and robustness; and (3) to serve as a tool to understand the impact of the mechanical design of parts on the controller performance (e.g., friction and backlash). In Section 5 the model is used to simulate the experiments where the actuator is tested under representative conditions of legged systems.

The power conversion in hydraulic actuation relies on the transmission of fluid by means of a pump to a hydraulic actuator. The role of the actuator is to transform back the hydraulic energy into mechanical energy, which is then transmitted to a mechanical device (e.g., a robot leg).

In the following paragraphs a description of the pressure, the flow, the valve spool and the load dynamics of hydraulic actuators is given. The rate of change of the pressure with respect to time in a given chamber is represented by the so-called *Continuity Equation*, which can be used to obtain the pressure dynamics for a linear hydraulic actuator as

$$\dot{P}\_A = \frac{\beta\_{\text{eff}}}{V\_{0A} + A\_A \chi\_p} \left( Q\_A - A\_A \dot{\mathbf{x}}\_p - \mathbf{C}\_{li} \left( P\_A - P\_B \right) \right), \tag{3}$$

$$\dot{P}\_B = \frac{\beta\_{\text{eff}}}{V\_{0B} + A\_B \left(L\_{\text{P}} - \chi\_{\text{P}}\right)} \left(Q\_B + A\_B \dot{\mathbf{x}}\_{\text{P}} - C\_{li} \left(P\_B - P\_A\right)\right), \tag{4}$$

where *PA* and *PB* are the pressures inside the chambers, *V*0*A* and *V*0*B* are the dead volumes coming from the valve inside the chambers, *AA* and *AB* the piston/vane areas where pressure is exerted in each of the chambers, *QA* and *QB* are the flows inside the chambers, *xp* is the piston position, *Lp* represents the cylinder stroke length

<sup>8</sup>An extended version of this section will be published in the Springer book *Humanoid Robotics: A Reference* in the chapter *Actuator Modeling and Simulation* by Jorn Malzahn, Victor Barasuol, and Klaus Janschek.

and *Cli* is a coefficient related to leakage. The *Bulk Modulus* is a measure for the compressibility of the fluid (commonly denoted by *β*). Due to flexibility of the compartments that contain the fluid and undissolved air pollution in the fluid, changes in the volume may not be captured by *β*. In this case, the Bulk Modulus *β* is replaced by the *Effective Bulk Modulus βeff* .

From Equations (3) and (4), the pressures in each of the chambers can be obtained and the hydraulic force delivered by the piston is given by

$$f\_h = A\_A P\_A - A\_B P\_B.\tag{5}$$

The pressure dynamics equations for a rotary actuator can be obtained in a similar fashion with respect to the linear case by replacing both piston areas *AA* and *AB* with the volumetric displacement of the motor *Dm*, the piston position *xp* with the angular position of the rotor *θa* and the cylinder stroke length *Lp* with the range of the motor *Lm*. The hydraulic torque delivered by the rotor can also be obtained from the pressure dynamics as

$$
\tau\_H = D\_m \left( P\_A - P\_B \right). \tag{6}
$$

It can be noticed that the pressure dynamics not only depend on the position of the piston or the rotor, but they also depend on the flows going into each of the chambers of the actuators. These flows (*QA* and *QB*) can be obtained from the dynamics of the valve. However, the valve is one of the elements that renders the modeling and control of hydraulic systems more complex, due to its nonlinear behavior. **Figure 9** shows a schematic drawing of a valve which controls the flows *QA* and *QB*. The spool position determines the amount of flow that goes through the orifices of the valve.

In the case of our experimental setups, a novel dual-gain Ultra Low Leakage Valve (ULLV) developed by Moog and IIT is being used. The spool position is controlled by means of an input current. The relation between input current and spool position is usually modeled as a second order system [see (Boaventura et al., 2015)] of the form

$$X\_V \left( s \right) = \frac{\omega\_n^2 K\_{spool}}{s^2 + 2\zeta \omega\_n s + \omega\_n^2} I \left( s \right), \tag{7}$$

where *Xv* ( *s* ) represents the spool position in the Laplace domain, *I* ( *s* ) is the valve current in the Laplace domain, *Kspool* relates the current input with the spool position in steady state, *ωn* is the natural frequency of the spool and *ζ* is the damping coefficient of the spool. However, for the ULLV a frequency domain analysis (depicted in **Figure 4**) showed higher order, which have been fit to a third-order transfer function given by

$$X\_{\nu}\left(s\right) = \frac{\omega\_n^2 K\_{loop}}{\left(s^2 + 2\zeta\omega\_n s + \omega\_n^2\right)s + \omega\_n^2 K\_{loop}} X\_{\nu}^{\tau}\left(s\right) \tag{8}$$

where *x<sup>r</sup> <sup>v</sup>* is the spool position reference, *Kloop* = 540 rad/s, *ωn* = 6132 rad/s and *ζ* = 1.5. The relationship between the spool position reference *X<sup>r</sup> v* ( *s* ) and the spool current command *Ir* ( *s* ) , in Laplace domain, are approximated by the following equations:

$$X\_{\nu}^{r}\left(s\right) = K\_{spool}I\left(s\right) \tag{9}$$

$$I\left(s\right) = \frac{p\_{\mathcal{L}}}{s + p\_{\mathcal{L}}} I\_{\mathcal{I}}\left(s\right),\tag{10}$$

where *Kspool* = 0.0356 mm/mA and *pc* = 2,000 rad/s is the pole of the first order systems that approximates the current loop response of the current controller described in Sec. 3.

With the definition of the spool dynamics, one can obtain the flow going through an orifice using the following expression:

$$Q = k\_{\mathcal{V}} \left( \mathbf{x}\_{\mathcal{V}} \right) \propto\_{\mathcal{V}} \sqrt{\Delta P}, \tag{11}$$

where *kv* ( *xv* ) is the so-called *valve gain*, and it is a factor that depends on the discharge coefficient, orifice area gradient and the density of the fluid (Merritt, 1967). Normally, *kv* is computed based on experiments with nominal input current and nominal flow, making use of a flow-meter. In the case of the ULLV, plots relating the flow going through the valve with respect to the input current were provided. These plots are depicted in **Figure 5**. The plots indeed show clearly the zones were the gain value changes, in the case of the ULLV. The low and high values of *kv* were estimated from these plots.

The flows *QA* and *QB* obtained from the differences between *Q*1, *Q*2, *Q*3 and *Q*4 in **Figure 9** are computed according to the following equations (considering a critically centered valve)

$$Q\_A = Q\_1 - Q\_2,\tag{12}$$

$$Q\_1 = k\_{\nu 1} \left( \chi\_{\mathcal{V}} \right) \text{sg} \left( \chi\_{\mathcal{V}} \right) \text{sign} \left( P\_S - P\_A \right) \sqrt{|P\_S - P\_A|}, \tag{13}$$

$$Q\_2 = k\_{\nu 2} \left( \varkappa\_{\nu} \right) \text{sg} \left( -\varkappa\_{\nu} \right) \text{sign} \left( P\_A - P\_T \right) \sqrt{|P\_A - P\_T|}, \tag{14}$$

$$Q\_B = Q\_4 - Q\_3,\tag{15}$$

$$Q\_3 = k\_{\nu 3} \left( \varkappa\_{\nu} \right) \text{sg} \left( \varkappa\_{\nu} \right) \text{sign} \left( P\_B - P\_T \right) \sqrt{|P\_B - P\_T|}, \quad \text{(16)}$$

$$Q\_4 = k\_{\nu4} \left(\mathbf{x}\_{\nu}\right) \text{sg}\left(-\mathbf{x}\_{\nu}\right) \text{sign}\left(P\_S - P\_B\right) \sqrt{|P\_S - P\_B|}, \quad \text{(17)}$$

where the function *sg* ( *x* ) is defined as:

$$\text{sg}(\mathbf{x}) = \begin{cases} \mathbf{x} & \text{for } \mathbf{x} \ge \mathbf{0} \\ \mathbf{0} & \text{for } \mathbf{x} < \mathbf{0}, \end{cases} \tag{18}$$

and *PT* and *PS* are the return and supply pressures, respectively. The variable valve gains *kvi* ( *xv* ) , with *i* = 1..4, are equally modeled according to the ULLV flow gain curve shown in **Figure 5** and Eq. 9 as:

$$\mathbf{k}\_{\nu l}(\mathbf{x}\_{\nu}) = \begin{cases} 6.667 \times 10^{-6} \, m^3/s & \text{for} \quad |\mathbf{x}\_{\nu}| < \mathbf{x}\_{d\mathbf{g}} \\ 4.167 \times 10^{-5} \, m^3/s & \text{for} \quad |\mathbf{x}\_{\nu}| \ge \mathbf{x}\_{d\mathbf{g}}, \end{cases} \tag{19}$$

where *xdg* is equal to 42.7 × 10−3 mm.

It is possible to consider fluid compressibility, elasticity of the fluid container and fluid resistance in the model, but these effects are highly nonlinear and are out of the scope of this paper.

Hydraulic force (in the case of linear actuators) or hydraulic torque (in the case of rotary actuators) is applied onto a mechanical device. In the case of this study, an experimental setup was built in order to test a custom-made rotary actuator integrated with a smart manifold. Additionally, the ISA v5 linear servo actuator was mounted on the Knee Flexion/Extension (KFE) joint of the hydraulically actuated leg of HyQ-REAL. These experiments are explained in Section 5.

## 5. Simulation and Experimental Results

To evaluate the performance of the ISA v5 and the smart manifold, two experimental setups were devised. The first setup consists of the smart manifold mounted on a custom made rotary actuator, which drives a metal wheel. This setup is shown in **Figure 10A**. For the second experimental setup, the ISA v5 was mounted on a leg of the HyQ-REAL to drive the KFE joint. This setup is depicted in **Figure 10B**. This section includes the simulation and experimental results on the rotary actuator setup (**Figure 10A**) and the experimental results on the KFE joint driven by the ISA v5 (**Figure 10B**).

The experiments were performed using the control strategy depicted in **Figure 7** and **Table 4** shows the specific values of the controller gains for each of the loops (unless it is indicated differently). It is worth mentioning that in the rotary actuator there is no spool loop. This is due to the fact that the connection of the first and second stage of the valve is done mechanically. On the other hand, the stages of the valve in the leg experimental setup is done through electric feedback, which gives rise to the spool control loop.

## 5.1. Integrated Smart Manifold and Rotary Actuator

The core of the test rig is made of three main components: the smart manifold, a custom-made rotary actuator and a set of sensors useful for control and analysis (position, torque, pressure, among others). All of these elements are shown in **Figure 2B**. This configuration of integrated electronics along with a rotary hydraulic actuator is suitable for actuating a legged system, for example, a Hip

FIGURE 10 | Pictures of the experimental setups of the smart manifold and the ISA v5: (A) Smart manifold mounted on custom made rotary actuator driving a wheel; (B) ISA v5 driving the knee joint of a HyQ-REAL leg.

Abduction/Adduction (HAA) joint of a quadruped robot, such as the ones on HyQ-REAL.

For the experiments, a metal wheel was mechanically connected to the rotary actuator, as it can be seen in **Figure 10A**. The weight, shape and dimensions of this wheel were designed in order to approximate the rotary inertia of the leg of HyQ-REAL, which is known from CAD data. The mathematical model of the test setup (along with the hydraulics model of Section 4) was obtained for simulation and control purposes. A schematic drawing of the linear equivalent of the model is shown in **Figure 11** and its mathematical description is given by

$$I\_a \ddot{\theta}\_a = \tau\_H - \tau\_f - K\_s \left(\theta\_a - \theta\_l\right) - D\_s \left(\dot{\theta}\_a - \dot{\theta}\_l\right), \tag{20}$$

$$I\_l \ddot{\theta}\_l = \mathcal{K}\_\mathcal{s} \left(\theta\_a - \theta\_l\right) + D\_\mathcal{s} \left(\dot{\theta}\_a - \dot{\theta}\_l\right) \tag{21}$$

where *Ia* and *Il* represent the rotational inertia of the actuator rotor and the load, respectively, *θa* and *θ<sup>l</sup>* correspond to the angular position of the actuator and the load, respectively, *τH* is the hydraulic torque coming from the actuator, *τf* is the friction torque acting on the mechanical rotor, *Ks* stands for the torque sensor stiffness and *Ds* is the damping present in the torque sensor.

In this model, the inertia of the torque sensor is being neglected since it is considered to be very small (approximately <sup>2</sup> *<sup>×</sup>* 103 times smaller with respect to the load inertia). We consider as well that the sensor and the rotor are rigidly connected to each other (sensor position is equal to actuator position). We use a Lund-Grenoble Friction Model for the friction torque acting on the actuator rotor (Aberger and Otter, 2002), with a stiction level of *Ts* = 2 and a Coulumb friction level of *Tc* = 1.9, identified from experiments. Nonlinear effects such as backlash between the load and the sensor (actuator) position are considered using a similar approach as in (Merzouki et al., 2003). We also model the hard-stops of the system as spring-damper systems with no pulling effects. The proposed model is considered accurate enough for simulation and controller design. The specific values of the parameters of the model, such as stiffness and inertias, were obtained from CAD models and experiments. The experiments were designed to identify damping coefficients of the involved dynamic elements.

In addition, the inherent characteristics of the sensors (e.g., resolution and sampling frequency) and the frequencies implemented in the various loops are also included in the simulation. This information is taken from the datasheets provided by the manufacturers of the sensors and electronic boards. In further studies, variations on the system due to quantization errors or sampling frequencies can be analyzed using the proposed

TABLE 5 | Rotary actuator parameters.


simulation. The motor parameters used in simulation are the ones of the custom made actuator provided by the manufacturer and are shown in **Table 5**. To obtain *V*0*A* and *V*0*B* the following expressions using the pipe volume *Vp* are used:

$$V\_{0A} = V\_P + D\_m \theta\_{a,}\tag{22}$$

$$V\_{\rm OB} = V\_{\rm P} + D\_m \left( L\_m - \theta\_a \right) \,. \tag{23}$$

For the first experiment, a reference signal for a rotary joint of a leg was obtained from simulation. The simulation corresponds to the robot HyQ-REAL performing a trotting gait with a forward velocity of 1 m/s. **Figure 12** shows the position and torque tracking performance achieved, both during simulations and experiments. It is worth noting the resemblance between the generated signals both in simulation and experiments. We consider that the achieved performance is sufficient to show the capabilities of the designed actuators in robot locomotion applications. Performance can be further improved by including the nonlinearities related to the hydraulic force and the valve dynamics in the controller design in a similar fashion as in (Boaventura et al., 2015).

blocked condition.

For the second experiment, the rotor actuator position was blocked, and a step torque reference was given in order to evaluate the torque tracking performance of the system. **Figure 13** shows an example of the torque tracking during simulation and the experiments. The actuator eliminates the steady state error with a rise time of approximately 8 ms. These results match between simulation and experimental data. This step response suggests that the achievable bandwidth goes from 50 Hz to 100 Hz during blocked condition.

A difference between simulation and experiments can be noted in **Figure 13**, where low-amplitude oscillations are present before reaching steady state during experiments. One cause of this oscillatory behavior can be attributed to the contact model between the setup and the mechanical end-stop when the actuator is blocked. This assumption is supported by the fact that in the case of the rotary actuator, a stiff metal to metal contact was used to keep the motor in a constant position. This contact shows low-amplitude oscillations. In the case of the linear actuator experiments (shown in **Figure 14**) the contact was kept between metal and rubber. The compliance displayed by the rubber and its possible nonlinear stiffness, potentially increase the amplitude of the oscillation. This

effect was also tested in simulation and the result coincide with this assumption. Added to the contact dynamics, a hunting effect originated due to the overlapping of the valve around the null position may also accentuate this oscillatory behavior.

To evaluate the performance of the rotary actuator against impacts, we provided a periodic reference trajectory that results in continuous impacts against the mechanical end-stop. An example of these tests in simulation and experiments is shown in **Figure 15**. It can be seen that the magnitude of the applied torque in order to cope with this impulse-like disturbances remains within the actuator limits (i.e., 170 Nm). These results show that the hydraulic servo actuators are able to deal with this kind of perturbation remaining stable and avoiding saturation. It is also important to point out that the reference signal in the case of the experiment shows some spikes after the impact. These sudden increments in the reference are mainly originated from the velocity measurements and the rotor velocity error at the beginning of the contact.

The final experiment for the rotary actuator is designed to emulate the presence of external disturbances into the system. The perturbation is introduced by attaching a weight to the edge of the load wheel of the experimental setup. A sequence of snapshots of this experiment is shown in **Figure 16**. The weight and rope slack used to apply the external disturbance could lead to a torque disturbance impulse with an amplitude larger than 250 Nm if the wheel was perfectly blocked or its joint had infinite joint stiffness. **Figure 17** shows such large impacts considering the case where the system has load position control without impedance torque control. For the test shown in **Figure 17** we consider a pure position control where the output of the

position PID controller block *Fr* becames the direct command signal *Uv* to the hydraulic valve (see **Figure 7A**). As it can be seen, when torque control is not implemented, the time that the system takes to stabilize is much larger. On the other hand, when torque control is applied (as depicted in **Figure 18**), steady-state is reached much faster, with a lower level of oscillations.

Additionally, we tested our control scheme for different values of controller gains. This set of gains is considered to contain a range of impedances suitable for quadruped robot locomotion. Based on our experience with the HyQ-series robots, depending on the application, these values of impedances could be implemented in systems ranging from 80 kg to 120 kg. It can be seen that even with a compliant set of gains, the actuator is able to cope with a disturbance. These experiments show the relevance of implementing torque and impedance control when dealing with impacts.

Studying the comparison between the experimental data and the simulation, the proposed setup proved to be useful for controller design, since the same strategy with the same control parameters (controller gains and frequencies of the control loops) were chosen with similar results in performance. Added to the fact that the designed smart actuators presented in this paper

fulfill the requirements of legged systems, it has been shown that the integration of the electronics and hydraulic actuation aids greatly in the design of new control strategies for hydraulic systems.

## 5.2. ISA V5 Driving Knee Joint of HyQ-REAL

For the linear actuator experiments, the ISA v5 was mounted on the KFE joint of the experimental setup for the leg of the hydraulically actuated robot HyQ-REAL. Position control and force tracking were also tested. Due to time constraints, a detailed simulation study of the implementation of the linear smart actuator was not carried out along with the experiments, mainly because additional experiments are required in order to properly identify the parameters of the leg. Nevertheless, the tests presented in this section contributed not only in the evaluation of the performance of the ISA v5, but also gave important insight about key parameters, such as the bandwidth of the system. It remains as future work to develop a simulation environment for behavior analysis and controller design, such as the one developed for the smart manifold integrated with the custom made rotary actuator.

Similarly to the position tracking experiment performed with the rotary actuator, a sine wave signal was set as reference for the enconder position of the KFE. **Figure 19** shows the tracking performance of the system. Implementing the control strategy described in **Figure 7**, it can be seen that the tracking is similar to the one achieved with the rotary actuator.

The second experiment for the linear actuator is similar to the force control against the end-stop for the rotary actuator. In the case of the leg, the foot was put in contact with a wooden pallet (as it can be seen in the lower part of the picture displayed in **Figure 10B**) and a step reference for force was given, for low and high force values. The results of these experiments can be seen in **Figure 14**.

For the last experiment, the leg was dropped from a height of 10 cm in order to evaluate the performance of the ISA when subjected to critical impacts, generally present while executing locomotion tasks. The total weight of the leg is approximately of 10.2 kg and the moving part from the slider that is attached to the leg (see **Figure 10B**) weighs approximately 5 kg. For this experiment the impedance values were *Kp* = 100000 N/m and *Kd* = 4000 Ns/m. **Figure 20** shows the force tracking during this experiment. The impact of the leg takes place around 0.1, at which the reference signal sent by the position control loop starts being tracked by the force loop. It can be noticed, that the actuator is able to stabilize the leg, considerably below its maximum output force (7500 N). The size of the peak appearing right after the time of the impact is due to the time the system takes to perceive and respond against the external disturbance (the ground reaction force).

The results of the experiments show a similar behavior as the ones displayed by the rotary actuator for similar tests, after tuning the controllers in order to account for the dynamics of the leg. This represents a positive result towards the implementation of a systematic method to design, test and implement controllers for legged systems in the future. Regarding this last point, one of essential task for further research, is to take advantage of the possibility to provide a detailed model of the integrated actuators shown in this paper, and develop control strategies that could improve the performance based on these models such as the feedback linearization (Boaventura et al., 2015).

## 6. Discussion and Conclusions

This paper presented a detailed description of the Integrated Servo Actuators ISA v2, ISA v5 and the smart manifold integrated with a custom made rotary actuator, developed by Moog and IIT. The main features and specifications of the components that build up the system were conceived in order to satisfy the requirements of legged systems and to overcome the current shortcomings of hydraulically actuated robots. The devices here described successfully integrate the electronics for controller implementation along with the components that comprise a high-performance hydraulic actuation system.

A detailed dynamic model of the actuator is given and was verified to be representative after comparing the simulation and the experimental results. The main goals of providing such dynamic model and a detailed parametrization of the actuator features are: to help on the elaboration of new control strategies; to make the simulation of multi-legged systems more realistic by integrating the actuator dynamics; to help designing mechanical parts by foreseeing the impact of their geometric tolerances (e.g., that lead to backlash) and properties (e.g., that lead to friction or structure stiffness).

Differences in the results between experimental and simulated tests are expected due to the difficulty to model the fast and nonlinear dynamics present in the systems. Sources of mismatching dynamics are likely to be due to, e.g.: inaccuracies in the environment stiffness model; the asssunption of a test bench stiffness infinitly rigid; the nonlinearities of the valve around the spool null position that can not be precisely modeled; and the innacuracies in the friction modeling. Most of these modeling mismatching and assumptions tend to cause a difference between simulation and experimental results when the actuator performs velocities close to zero (condition where friction modeling is critical) or goes into a steady state where the valve spool tends to be positioned around null. At this critical conditions, high controller gains might lead to limit cycles as a steady state.

A control strategy implemented on the integrated electronic boards of the servo actuators is explained in detail. This control strategy was tested on two experimental setups and the experimental results for the smart manifold integrated with the rotary actuator and the ISA v5 mounted on the knee joint of HyQ-REAL leg were presented. The force/torque control performance, shown in Sec. 5, suggests a control bandwidth between 50 Hz and 100 Hz for low amplitude signals (about 10% of the maximum output force/

torque). Considering the control bandwidth found in state-of-theart papers for related applications (Paine et al., 2014; Boaventura et al., 2015; Hutter et al., 2016), that ranges from 30 Hz to 60 Hz for small signals around 10%, the control performance obtained with the smart actuarors is relevant, promissing and are part of the current state-of-the-art.

In its current state, the main limitations noticed in the smart actuators are at firmware level. Future work includes implementing new functionalities to test different control strategies (e.g., nonlinear control approaches) as well as tools to run system identification algorithms and to evaluate the control performance of the various control loops (e.g., frequency response analysis).

Future work also comprises new steps towards the system modeling. A detailed characterization for simulation and controller design of the experimental setup of the ISA v5 along with the HyQ-REAL leg will be obtained. Moreover, different control strategies that might improve the performance of the overall system (such as model based strategies like feedback linearization) will be tested both in simulation and experiments. Finally, the novel servo actuators here described, will be implemented on HyQ-REAL, the newest version of the hydraulically actuated quadrupeds developed in IIT.

## Author Contributions

VB, DS, MF, MB, RM contributed to the development of the actuators and setups. VB, OV-M, DS designed and performed

## References


the experiments. VB, OV-M and GM-C did the modeling part and performed the simulations. CS, MB and DC coordinated the teams. VB, OV-M, DS and CS prepared the manuscript and figures.

## Funding

This work was supported by Moog Inc and Istituto Italiano di Tecnologia (IIT), with additional funding from the European Union's Seventh Framework Programme for research, technological development and demonstration under grant agreement n° 601116 as part of the ECHORD++ (The European Coordination Hub for Open Robotics Development) project under the experiment called HyQ-REAL.

## Acknowledgments

We would like to thank Gonzalo Rey (previously at Moog) for pushing the development of the ISA forward; James Smith (Moog), Kulkarni Laxman (Moog), Iain Chisholm (Moog), Ian Whiting (Moog), Ian Barthlett (Moog), Justin Fellows (Moog), and Vanishree Rao (Moog) for the development of the ISA; and Alex Posatskiy (previously at IIT) and Yannick Berdou (previously at IIT) for the design of the robot leg and experimental setups.


Raibert, M. (1986). *Legged Robots That Balance*. MA, United States: The MIT Press.


*Engineers Part I Journal of Systems and Control Engineering* 225 (6), 831–849. doi: 10.1177/0959651811402275


**Conflict of Interest Statement:** Authors DS, MB and RM are employed by company Moog (UK). All other authors declare no competing interests and have no involvement or role in the company.

*Copyright © 2018 Barasuol, Villarreal-Magana, Sangiah, Frigerio, Baker, Morgan, Medrano-Cerda, Caldwell and Semini. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Configuration-Dependent Optimal Impedance Control of an Upper Extremity Stroke Rehabilitation Manipulandum

### Borna Ghannadi\*, Reza Sharif Razavian and John McPhee

*Systems Design Engineering, University of Waterloo, Waterloo, ON, Canada*

Robots are becoming a popular means of rehabilitation since they can decrease the laborious work of a therapist, and associated costs, and provide *well-controlled* repeatable tasks. Many researchers have postulated that human motor control can be mathematically represented using optimal control theories, whereby some cost function is effectively maximized or minimized. However, such abilities are compromised in stroke patients. In this study, to promote rehabilitation of the stroke patient, a rehabilitation robot has been developed using optimal control theory. Despite numerous studies of control strategies for rehabilitation, there is a limited number of rehabilitation robots using optimal control theory. The main idea of this work is to show that impedance control gains cannot be kept constant for optimal performance of the robot using a feedback linearization approach. Hence, a general method for the real-time and optimal impedance control of an end-effector-based rehabilitation robot is proposed. The controller is developed for a 2 degree-of-freedom upper extremity stroke rehabilitation robot, and compared to a feedback linearization approach that uses the standard optimal impedance derived from covariance propagation equations. The new method will assign optimal impedance gains at each configuration of the robot while performing a rehabilitation task. The proposed controller is a linear quadratic regulator mapped from the operational space to the joint space. Parameters of the two controllers have been tuned using a unified biomechatronic model of the human and robot. The performances of the controllers were compared while operating the robot under four conditions of human movements (impaired, healthy, delayed, and time-advanced) along a reference trajectory, both in simulations and experiments. Despite the idealized and approximate nature of the human-robot model, the proposed controller worked well in experiments. Simulation and experimental results with the two controllers showed that, compared to the standard optimal controller, the rehabilitation system with the proposed optimal controller is assisting more in the active-assist therapy while resisting in active-constrained case. Furthermore, in passive therapy, the proposed optimal controller maintains the position error and interaction forces in safer regions. This is the result of updating the impedance in the operational space using a linear time-variant impedance model.

Keywords: optimal impedance control, linear quadratic regulator, operational space, rehabilitation manipulandum, human-robot interaction, stroke rehabilitation

#### Edited by:

*Navvab Kashiri, Fondazione Istituto Italiano di Technologia, Italy*

#### Reviewed by:

*Zhijun Zhang, South China University of Technology, China Elias Kosmatopoulos, Democritus University of Thrace, Greece*

> \*Correspondence: *Borna Ghannadi bghannad@uwaterloo.ca*

#### Specialty section:

*This article was submitted to Robotic Control Systems, a section of the journal Frontiers in Robotics and AI*

Received: *31 October 2017* Accepted: *09 October 2018* Published: *01 November 2018*

#### Citation:

*Ghannadi B, Sharif Razavian R and McPhee J (2018) Configuration-Dependent Optimal Impedance Control of an Upper Extremity Stroke Rehabilitation Manipulandum. Front. Robot. AI 5:124. doi: 10.3389/frobt.2018.00124*

## 1. INTRODUCTION

## 1.1. Motivation

Upper extremity motor impairments are common among poststroke patients. If the rehabilitation therapy is stimulating and intense, it will be effective in treatment of disabilities (Richards and Malouin, 2015). Thus, upper extremity rehabilitation robots including robotic manipulanda<sup>1</sup> ("InMotion Arm" and "ReoGo") and robotic exoskeletons ("ArmeoPower," and "ArmeoSpring") have been commercially introduced to the clinical setting (Maciejasz et al., 2014; Proietti et al., 2016). Although, in some studies, advantages of these robots over traditional therapy methods are minor (Brewer et al., 2007; Wisneski and Johnson, 2007; Lo et al., 2010; Mazzoleni et al., 2013; Maciejasz et al., 2014), their use cannot be ignored since they can provide well-controlled repeatable tasks, progress evaluation measurements and entertaining user-interfaces (Reinkensmeyer, 2009; Kowalczewski and Prochazka, 2011).

When stroke management is supported by effective care, rehabilitation costs can be substantially reduced (Krueger et al., 2012). Effective stroke care includes rapid assessment and rehabilitation with efficient outcomes in physical and functional recovery (Hebert et al., 2016). Efficient physical recovery is a qualitative measure, and a healthy subject is assumed to have an efficient physical activity level. Hence, if a stroke rehabilitation approach can improve the physical activity of a stroke patient to the level of a healthy subject, the rehabilitation can be considered effective, i.e., it cannot do any better.

Studies have shown that a human interacts with the environment while minimizing an error and effort or, in general, a cost function (Todorov and Jordan, 2002; Franklin et al., 2008). In other words, the human's central nervous system (CNS) optimally controls human interaction. In rehabilitation therapy, there is an interaction between the stroke patient and a therapist or robot (or in general, an environment). To promote effective therapy, if the stroke patient's CNS cannot maintain the optimality goal, this internal optimal control problem should be solved externally with the aid of assistive devices (Jarrassé et al., 2012). Thus, we assume that the use of optimal control methods in rehabilitation robotics is well-suited to assisting an impaired CNS. This assumption is consistent with previous studies, such as Hunt et al. (1999) who used optimal control theory in a feedback balance control mechanism to maintain standing of paraplegic subjects, Emken et al. (2005) who considered rehabilitation robot training as an optimization problem and designed an optimal controller for assist-as-needed (active-assisted) therapy, Ibarra et al. (2014) and Ibarra et al. (2015) who developed an optimal controller for ankle rehabilitation, Mombaur (2016) who uses optimal control theory to predict natural (healthy human) movement and improve the device performance in rehabilitation technologies, Wang et al. (2017) who used optimal control to maintain patient's safety and comfort during elbow rehabilitation, and Corra et al. (2017) who implemented optimal control to adjust the gains of a controller for arm rehabilitation.

## 1.2. Control Strategies in Rehabilitation Robotics

Control strategies for rehabilitation robots can be divided into two general subgroups: (1) High-level control scenarios for stimulating neural plasticity, and (2) Low-level control scenarios to implement high-level scenarios (Maciejasz et al., 2014). Highlevel control scenarios include: assistive, corrective (coaching) and resistive (challenge-based) control modes. Among these modes, assistive control is the core element in post-stroke rehabilitation therapy. In assistive mode, three types of low-level control scenarios are implemented on these robots: (1) Passive control, (2) Triggered passive control, (3) Partially assistive control. Passive trajectory tracking and impedance-based control methods, which are types of passive and partially assistive control scenarios, respectively, are widely used in these robots (Maciejasz et al., 2014; Proietti et al., 2016).

In robotic rehabilitation, because of physical interaction of the patient with a mechanical device, safety is a fundamental element in the design of a low-level control scenario. Thus, impedance-based control scenarios are more applicable for robotic rehabilitation (Marchal-Crespo and Reinkensmeyer, 2009; Maciejasz et al., 2014; Proietti et al., 2016), since conventional position/force control scenarios (passive trajectory tracking) do not consider dynamic interaction of the humanrobot system (Hogan, 1985). Furthermore, assist-as-needed therapy, which encourages voluntary participation of the patient, is implementable through the impedance-based control scenarios.

In impedance-based control, the amount of assistance/resistance (i.e., compliance) can be adjusted by controlling the impedance gains. However, in the presence of a variable admittance environment (i.e., different patients) or different trajectories (i.e., robot configurations), the interaction force and configuration will exacerbate inefficiency of the controller with non-optimal gains. For example, a resistivecapacitive impedance control with therapist-adjustable constant stiffness and damping ratios is implemented in the upper extremity rehabilitation manipulandum from the Toronto Rehabilitation Institute (TRI) and Quanser Consulting Inc., but these gains cannot be adjusted optimally using trial and error by the therapist (Huq et al., 2012).

Besides other methods of partially assistive control (e.g., attractive force-field control, model-based assistance, learningbased assistance, counter-balance-based assistance, and performance-based adaptive control), adaptive and optimal forms of impedance control have been developed to deal with variable admittance environments. Hussain et al. (2013) used an adaptive impedance control for patient-cooperative therapy of a lower-limb exoskeleton, and they verified the controller performance using an experimental setup. In more recent studies, optimal impedance controls for an exoskeleton gait trainer and elbow rehabilitation robot were developed (Dos Santos and Siqueira, 2016; Wang et al., 2017). The proposed methods were implemented in a computer simulation, and the real-time performance of the controllers was not discussed. In an exoskeleton, the impedance control

<sup>1</sup>End-effector-based rehabilitation robots

is defined in the joint space, while in a manipulandum, the impedance model is in the operational space. Thus, the controllers developed for exoskeletons are not suitable for a manipulandum. Furthermore, exoskeleton controllers are developed for some sort of predefined rhythmic motions and they are not implementable for random reaching movements. Beside recent studies on exoskeletons, Maldonado et al. (2015) used stiffness-based tuning for an adaptive impedance control of an upper extremity manipulandum; the method was verified using computer simulations only and its real-time capabilities were not mentioned.

In some studies, to improve impedance control performance, the compliance has been controlled by an outer-loop force control (Erol and Sarkar, 2007; Siciliano et al., 2009; Ghannadi et al., 2014a). Depending on the controller structure and use of series elastic actuators, the compliance term can be controlled by an inner-loop force control in the presence of an outerloop impedance control. For example, Perez-Ibarra et al. (2017) used an H-infinity force control to implement this approach. This hybrid impedance-force controller can be implemented by different methods such as weighted sum (Moughamir et al., 2005) or robust Markovian approach (Jutinico et al., 2017). However, this method only controls the compliance (i.e., interaction force) term, and the impedance gains are not optimal.

In a recent study, to select optimal target impedance for a lower limb exoskeleton, a method for estimating human admittance using particle swarm optimization was proposed (Taherifar et al., 2017). Overall, a general solution for an optimal impedance problem can be obtained with optimization techniques (i.e., an optimal control approach). Such techniques can adapt to variable admittance environments and different robot configurations. However, real-time control of the system limits the utilizable non-linear optimization methods. Ding et al. (2010) used a musculoskeletal human model (without including muscle dynamics) together with surface elecromyography (sEMG) signals to implement model-based assistance control on a rehabilitation exoskeleton. In Ghannadi et al. (2017a), we used a nonlinear model-predictive approach to control human-robot interaction in an upper extremity manipulandum. The method was verified using computer simulation, but experimental tests were not performed because of inefficient computation for realtime implementation.

Learning-based methods can also be used to evaluate the optimal impedance gains (Ge et al., 2014; Modares et al., 2016). Ge et al. (2014) implemented an adaptive linear quadratic regulator (LQR) to estimate the impedance gains, and Modares et al. (2016) used reinforcement learning to solve an LQR problem and achieve optimal impedance gains. However, the validity of the proposed methods was verified using simulation studies, and real-time implementation was not discussed. Other than LQR, H-infinity control approaches can be used to achieve optimal performance. In Kim et al. (2015), an H-infinity impedance control is implemented for an upper extremity exoskeleton. Compared to an LQR controller, the H-infinity controller is more robust because it can handle uncertainties in the impedance model. Design of an H-infinity controller depends on the selection of a weighting function, whereas in an LQR control, the optimal state feedback gain matrix is favorable. Thus, initial design of an H-infinity approach may take more effort than an LQR controller. Furthermore, H-infinity may have large numerical variations that require increased numerical precision, thereby increasing the computation cost for real-time implementation (Glover and Packard, 2017).

Since multi-link manipulanda are controlled in the joint space to achieve the desired impedance at the end-effector in the operational space, the optimal impedance gains should be assigned to the different robot configurations. For different configurations, the manipulability ellipsoid in robotics is introduced to determine the easiest manipulation direction (Yamashita, 2014). Thus, a method is required to optimally change the impedance gains based on the robot's manipulability ellipsoid. Hogan (2017) proposed an optimal impedance control for a one-dimensional system, the standard optimal impedance control (SOIC), which minimizes an cost function with position and force penalty. This problem was solved using covariance propagation equations. To the knowledge of the authors, there is no other optimal impedance control approach that has resolved different robot configuration problem independently.

## 1.3. Research Objective

The control input for a conventional impedance control using nonlinear feedback linearization is in terms of the interaction force, which is defined based on the impedance model with timeinvariant gains (see **Appendix C**). There is a trade-off between tracking accuracy and interaction force in the operational space, and increasing one of them may decrease the other. An impedance model (i.e., gains) can regulate this trade-off efficiently if they are adjusted optimally for different robot configurations. Tuning this trade-off is important since this can help the patient to safely (i.e., with an optimally safezoned interaction force) follow a desired trajectory with optimal accuracy. As discussed in section 1.2, different studies have tried to provide the best trade-off in robotic rehabilitation. However, there is a lack of research in the design of realtime optimal impedance control for different configurations of rehabilitation manipulanda. Furthermore, previous lowlevel controls of rehabilitation manipulanda have not included human-robot interactions for the adjustment of the robot controller.

To find optimal impedance gains, we restate the problem definition using optimal control theory: the best trade-off between tracking error and interaction force in the operational space can be revisited as finding an optimal control law that minimizes the tracking and effort error in the operational space. If this control law can satisfy the impedance model with time-variant gains, optimal target impedance will be achieved because these gains are the resulting optimal solution. The objective of this work is to design a general real-time optimal impedance control for rehabilitation manipulanda. This controller is designed to reduce therapist intervention (with fewer gain adjustments) and improve the quality of therapy in terms of safety (less interaction force based on robot manipulability) and rehabilitation (optimal tracking).

In our previous study, we presented an optimal impedance control (OIC) for an upper extremity stroke rehabilitation robot (Ghannadi and McPhee, 2015); adjustment and performanceevaluation of the controller were done by simulating the robot interacting with a musculoskeletal upper extremity model (Ghannadi et al., 2014b). The current paper is an extension to our previous study. Here, a general method that optimally adjusts impedance gains for variable robot configurations is developed and tuned by simulating the human-robot system. The proof that justifies the existence of a linear time variant (LTV) impedance model is provided. The controller is implemented on a Quanser Consulting Inc./TRI robot. Then, the performance of the controller in terms of interaction force and tracking accuracy is evaluated and compared to the SOIC (Hogan, 2017) through simulations and experiments. In experiments, a complete dynamic model of the robot including joint and end-effector frictions, and joint stiffness are considered.

This paper is organized as follows. First, in the section 2, the modeling procedure and controller design are presented. Second, in the section 3, simulation and experiment descriptions and assessment criteria are provided. Next, in the section 4, OIC simulation and experimental results are compared to SOIC. Finally, in the section 5, contributions and future work are presented.

## 2. MODELING AND CONTROL

In this section, first, the human-robot system model (which is used in a model-in-loop simulation) is described. Next, the proposed controller design is discussed.

## 2.1. Model Development

The upper extremity stroke rehabilitation robot is a 2 degreeof-freedom (DOF) parallelogram arm that moves the hand in the horizontal plane to perform reaching movements for therapy (**Figure 1B**). This robot is driven by two DC motors that share the same axis of rotation, and are connected to the 2 DOF arm through disc and timing belt mechanisms. Since the proposed controller is particularly suited for backdrivable robots, the simulation model of the rehabilitation robot is assumed to have negligible frictional forces so that the robot can be backdriven. Hence, it is modeled as a frictionless planar parallelogram linkage in the MapleSimTM software package.

The musculoskeletal arm is considered a planar 2 DOF linkage with 19 muscles lumped in 6 muscle groups (Ghannadi et al., 2014b) (**Figure 1A**). In this model, upper extremity tendons were treated as rigid elements<sup>2</sup> , and the passive elements of the arm muscles were assumed to have less contribution than the active elements in muscle forces. Hence, the contractile element of the Hill-type muscle model is used to model muscle dynamics, and forward static optimization (Ghannadi et al., 2014b) is implemented to solve the muscle force sharing problem while tracking the desired curvilinear path (Zadravec and Matjaci ˇ c, 2013 ´ ) (**Figure 1A**) with minimum jerk and a bellshaped tangential speed (Flash and Hogan, 1985) under robot operation. This musculoskeletal arm is also developed in the MapleSimTM software package.

These two models are integrated in MapleSimTM and connected to each other by a free rotational revolute joint with a force sensor. There are eight inputs to the human-robot system consisting of two robot motor torque inputs (TR1,2 ) and six muscle activations (a1..6). In this system, the number of outputs is six, where two are from motor encoders (q1,2), two are from the force sensor (FextZ,<sup>X</sup> ), and two are the musculoskeletal model joint angles (θ1,2).

In contrast to admittance control, impedance control can be used for backdrivable systems. Thus, for implementing the proposed optimal controller, we assume that the friction is negligible so that the robot can be backdriven. In simulations, the robot model has no friction and the musculoskeletal model has only approximate parameters for the muscles and inertial properties; thus, we do not expect a close quantitative match between simulation and experimental results. Nevertheless, the model will be effective for the design and tuning of a feedback controller if a good qualitative match between simulation and experimental results is achieved.

## 2.2. Optimal Control Method

In an optimal control structure, it is desired to carry out a desired task while minimizing a cost function. The dynamic equation of the robot excluding frictional forces is as follows:

$$\mathbf{T}\_R - \mathbf{J}\_R^T \mathbf{F}\_{int} = \mathbf{M}\_R(\mathbf{q})\ddot{\mathbf{q}} + \mathbf{C}\_R(\mathbf{q}, \dot{\mathbf{q}})\dot{\mathbf{q}} = \mathbf{F}\_R(\mathbf{q}, \dot{\mathbf{q}}, \ddot{\mathbf{q}}) \tag{1}$$

where **T**<sup>R</sup> is the vector of robot motor torques, and **J**<sup>R</sup> is the robot geometric Jacobian. **F**int is the robot to human interaction force in the global coordinates, and it is equal to the measured force by the force sensor (i.e., **F**int = **F**ext). **M**<sup>R</sup> is the robot inertia (mass) matrix, and **C**<sup>R</sup> is the robot Coriolis-centrifugal matrix. The state-space representation for the robot dynamics can be expressed as:

$$\dot{\mathbf{x}}\_{\mathbf{q}} = \begin{Bmatrix} \dot{q}\_1 \\ \dot{q}\_2 \\ \mathbf{M}\_R^{-1}(\mathbf{u} - \boldsymbol{\Gamma}\_R(\mathbf{q}, \dot{\mathbf{q}}, \mathbf{0})) \end{Bmatrix} = \mathfrak{F}(\mathbf{x}\_{\mathbf{q}}, \mathbf{u}), \tag{2}$$

where:

and

$$\mathbf{u} = \mathbf{T}\_R - \mathbf{J}\_R^T \mathbf{F}\_{ext, \bullet} \tag{3}$$

$$\mathbf{x}\_{\mathbf{q}} = \begin{Bmatrix} \mathbf{q} \\ \dot{\mathbf{q}} \end{Bmatrix} = \begin{Bmatrix} q\_1 \\ q\_2 \\ \dot{q}\_1 \\ \dot{q}\_2 \end{Bmatrix}. \tag{4}$$

The objective is to develop a real-time controller that optimizes impedance gains at different configurations. Since the statespace representation (2) is nonlinear, application of nonlinear

<sup>2</sup>The compliance of tendon is proportional to its slack length. Thus, a tendon is compliant if its normalized slack length is large (≥ 10), and it is very stiff when it is equal to 1 (Zajac, 1989). For most muscles in the upper extremity this value is around 1; hence, the stiff tendon assumption seems to be valid.

optimal control approaches will be limited by the computation time. On the other hand, if (2) was linear, a linear optimal controller (such as LQR or H-infinity) could solve this problem in real-time. Since the robot performs preplanned point to point reaching tasks in the horizontal plane (Lu et al., 2011), we can perform Jacobian linearization on the robot dynamics along the preplanned rehabilitation trajectory to apply a systematic linear control technique, which can allow for real-time control. In recent robotic rehabilitation simulation studies (Ge et al., 2014; Modares et al., 2016), LQR was used to implement optimal impedance control. Hence, we also use an LQR approach to solve the optimal impedance control problem. The LTV state-space equation of the robot's error dynamics will be:

$$\dot{\mathbf{x}}\_{\mathbf{q}d} - \dot{\mathbf{x}}\_{\mathbf{q}} = \left. \frac{\partial \widetilde{\mathbf{g}}}{\partial \mathbf{x}\_{\mathbf{q}}} \right|\_{\mathbf{x}\_{\mathbf{q}} = \mathbf{x}\_{\mathbf{q}d}, \mathbf{u} = \mathbf{u}\_{d}} (\mathbf{x}\_{\mathbf{q}d} - \mathbf{x}\_{\mathbf{q}}) + \left. \frac{\partial \widetilde{\mathbf{g}}}{\partial \mathbf{u}} \right|\_{\mathbf{x}\_{\mathbf{q}} = \mathbf{x}\_{\mathbf{q}d}, \mathbf{u} = \mathbf{u}\_{d}} (\mathbf{u}\_{d} - \mathbf{u}) $$
 
$$= \dot{\widetilde{\mathbf{x}}}\_{\mathbf{q}} = \mathbf{A}\_{\mathbf{q}} \widetilde{\mathbf{x}}\_{\mathbf{q}} + \mathbb{B}\_{\mathbf{q}} \widetilde{\mathbf{u}}, \tag{5}$$

where subscript d indicates the desired value of a variable, and accent ∼ denotes the error of the desired variable with respect to its actual value. and are the state and input matrices, respectively. The desired control input is defined by the following equation:

$$\mathbf{u}\_d = \Gamma\_R(\mathbf{q}\_d, \dot{\mathbf{q}}\_d, \ddot{\mathbf{q}}\_d). \tag{6}$$

At each operational point, which is defined every 1000/ν ms of the rehabilitation trajectory, the model is linearized and the interaction force is applied to the robot. ν is the sampling-time frequency which is measured in Hz. It is worth noting that, if very few operational points are defined, the system may be biased into optimizing for static situations. At each operational point, the controllability (**C**) and observability (**O**) matrices are defined as:

$$\mathcal{C} = \begin{bmatrix} \mathbf{B\_{q}} \ \mathbf{B\_{q}} \mathbf{B\_{q}} \ \mathbf{A\_{q}^{2}} \mathbf{B\_{q}} \ \mathbf{A\_{q}^{3}} \mathbf{B\_{q}} \end{bmatrix}\_{4 \times 8}, \quad \mathcal{O} = \begin{bmatrix} \mathbf{I} \\ \mathbf{A\_{q}} \\ \mathbf{A\_{q}^{2}} \\ \mathbf{A\_{q}^{3}} \end{bmatrix}\_{16 \times 4}, \tag{7}$$

where is an identity matrix.

At each operational point, there is an LTV impedance model which is relating the end-effector operational space error (e<sup>ρ</sup> <sup>6</sup>) to interaction force error:

$$-\widetilde{\mathbf{F}}\_{ext}^{\Sigma} = \mathbf{M}\_{imp}\widetilde{\boldsymbol{\tilde{\rho}}}^{\Sigma} + \mathbf{B}\_{imp}\widetilde{\boldsymbol{\tilde{\rho}}}^{\Sigma} + \mathbf{K}\_{imp}\widetilde{\boldsymbol{\rho}}^{\Sigma},\tag{8}$$

here, it is assumed that the desired interaction force is equal to zero, that is <sup>e</sup>**<sup>F</sup>** 6 ext = −**<sup>F</sup>** 6 ext. <sup>e</sup><sup>ρ</sup> is the end-effector position error in the Cartesian coordinates, and subscript imp stands for the impedance model. **M**, **B**, and **K** are mass, damping, and stiffness coefficients (impedance gains), respectively. These gains are time-dependent. These gains are time-dependent. Superscript 6 denotes that the corresponding vector is defined in the end-effector's n-t coordinates (i.e., the operational space; see **Figure 1B**). If **R**6 is the rotation matrix transforming the n-t coordinates to the Cartesian coordinates, <sup>e</sup><sup>ρ</sup> <sup>6</sup> can be obtained from the following equations:

$$\begin{cases} \widetilde{\rho}^{\Sigma} = \mathbf{R}\_{\Sigma}^{\mathrm{T}} \widetilde{\rho}, \\\ \mathbf{R}\_{\Sigma} = \begin{bmatrix} \cos(\theta\_{\Sigma}) \ -\sin(\theta\_{\Sigma}) \\ \sin(\theta\_{\Sigma}) \ \cos(\theta\_{\Sigma}) \end{bmatrix}, \end{cases} \tag{9}$$

where θ<sup>6</sup> is defined in **Figure 1B**.

The LTV state-space Equation (5) is in terms of errors, so we can use the infinite time<sup>3</sup> LQR to optimally control the robot along the desired trajectory. For the LQR approach, the quadratic cost function is:

$$\mathfrak{J}\_{\mathbf{q}} = \frac{1}{2} \int\_{0}^{\infty} \left( \widetilde{\mathbf{x}}\_{\mathbf{q}}^{T} \mathbf{Q}\_{\mathbf{q}} \widetilde{\mathbf{x}}\_{\mathbf{q}} + \widetilde{\mathbf{u}}^{T} \mathbf{R}\_{\mathbf{q}} \widetilde{\mathbf{u}} \right) dt. \tag{10}$$

The above cost function is for minimizing the joint space error together with the consumed energy. An impedance control approach controls the robot performance in the operational space as in (8) (Siciliano et al., 2009; Hogan, 2017). Thus, for an optimal impedance control it will be desired to minimize the operational space error together with the operationally applied force (effort) error while satisfying (8). In other words, the following cost function is more appropriate than (10):

$$\mathfrak{J}\_{\Sigma} = \frac{1}{2} \int\_{0}^{\infty} \left( \widetilde{\mathbf{x}}\_{\Sigma}^{T} \mathbf{Q}\_{\Sigma} \widetilde{\mathbf{x}}\_{\Sigma} + \widetilde{\mathbf{F}}\_{\Sigma}^{T} \mathbf{R}\_{\Sigma} \widetilde{\mathbf{F}}\_{\Sigma} \right) dt,\tag{11}$$

where <sup>e</sup><sup>6</sup> is the operational space transformation of the applied force error in the Cartesian space (e):

$$\begin{cases} \mathbf{F} = \mathbf{J}\_R^{-T} \mathbf{u} = \mathbf{J}\_R^{-T} \mathbf{T}\_R - \mathbf{F}\_{ext}, \\ \widetilde{\mathbf{F}} \cong \mathbf{J}\_{Rd}^{-T} \widetilde{\mathbf{u}}. \end{cases} \tag{12}$$

To solve the LQR problem with the updated cost function in (11), we use the mapping from the operational space into the joint space and then solve the LQR problem with the ordinary cost function in (10).

#### 2.2.1. Mapping the Operational Into Joint Space

We define the joint, Cartesian and operational state errors as follows:

$$\widetilde{\mathbf{x}}\_{\mathbf{q}} = \begin{Bmatrix} \mathbf{q}\_d - \mathbf{q} \\ \dot{\mathbf{q}}\_d - \dot{\mathbf{q}} \end{Bmatrix} = \begin{Bmatrix} \widetilde{\mathbf{q}} \\ \widetilde{\mathbf{q}} \end{Bmatrix},\tag{13}$$

$$\widetilde{\mathbf{x}}\_{\rho} = \begin{Bmatrix} \boldsymbol{\rho}\_{d} - \boldsymbol{\rho} \\ \dot{\boldsymbol{\rho}}\_{d} - \dot{\boldsymbol{\rho}} \end{Bmatrix} = \begin{Bmatrix} \widetilde{\boldsymbol{\rho}} \\ \widetilde{\boldsymbol{\rho}} \end{Bmatrix},\tag{14}$$

$$\widetilde{\mathbf{x}}\boldsymbol{\Sigma} = \begin{Bmatrix} \boldsymbol{\rho}\_d^{\boldsymbol{\Sigma}} - \boldsymbol{\rho}^{\boldsymbol{\Sigma}} \\ \dot{\boldsymbol{\rho}}\_d^{\boldsymbol{\Sigma}} - \dot{\boldsymbol{\rho}}^{\boldsymbol{\Sigma}} \end{Bmatrix} = \begin{Bmatrix} \widetilde{\boldsymbol{\rho}}^{\boldsymbol{\Sigma}} \\ \widetilde{\boldsymbol{\rho}}^{\boldsymbol{\Sigma}} \end{Bmatrix}.\tag{15}$$

Based on the inverse kinematics of the robot, the geometric Jacobian definition (Siciliano et al., 2009) and first-order Taylor series expansion, the relation between the Cartesian and joint space errors can be defined as:

$$\begin{cases} \widetilde{\boldsymbol{\rho}} \cong \mathbf{J}\_{Rd} \widetilde{\mathbf{q}},\\ \widetilde{\boldsymbol{\rho}} \cong \mathbf{j}\_{Rd} \widetilde{\mathbf{q}} + \mathbf{J}\_{Rd} \widetilde{\mathbf{q}}, \end{cases} \tag{16}$$

Thus, the operational state error in terms of the joint state error can be defined by the following equation:

$$\widetilde{\mathbf{x}}\_{\rho} \cong \begin{bmatrix} \mathbf{J}\_{Rd} & \mathbf{0} \\ \mathbf{J}\_{Rd} & \mathbf{J}\_{Rd} \end{bmatrix} \widetilde{\mathbf{x}}\_{\mathbf{q}}.\tag{17}$$

Consider **Figure 1B**, the operational coordinate (6:n-t) is the rotated and translated Cartesian coordinate (G:ZX) by angle θ<sup>6</sup> and desired position vector ρ<sup>d</sup> , respectively; thus, the relation between the operational and Cartesian space errors can be defined as:

$$\begin{cases} \widetilde{\rho} = \mathbf{R}\_{\Sigma} \widetilde{\rho}^{\Sigma}, \\ \widetilde{\rho} = \dot{\mathbf{R}}\_{\Sigma} \widetilde{\rho}^{\Sigma} + \mathbf{R}\_{\Sigma} \widetilde{\rho}^{\Sigma}. \end{cases} \tag{18}$$

By defining ̟<sup>6</sup> as the skew symmetric matrix of the angular velocity (θ˙6), the operational state error can be defined in terms of the Cartesian state error as:

$$
\widetilde{\mathbf{x}}\_{\boldsymbol{\theta}} = \begin{bmatrix}
\mathbf{R}\_{\boldsymbol{\Sigma}} & \mathbf{0} \\
\mathbf{w}\_{\boldsymbol{\Sigma}} \mathbf{R}\_{\boldsymbol{\Sigma}} & \mathbf{R}\_{\boldsymbol{\Sigma}}
\end{bmatrix} \widetilde{\mathbf{x}}\_{\boldsymbol{\Sigma}}.\tag{19}
$$

Finally, the operational and joint state errors can be related as:

$$
\widetilde{\mathbf{x}}\_{\Sigma} \cong \begin{bmatrix} \mathbf{R}\_{\Sigma} & \mathbf{0} \\ \mathbf{w}\_{\Sigma} \mathbf{R}\_{\Sigma} & \mathbf{R}\_{\Sigma} \end{bmatrix}^{-1} \begin{bmatrix} \mathbf{J}\_{Rd} & \mathbf{0} \\ \mathbf{\dot{J}}\_{Rd} & \mathbf{J}\_{Rd} \end{bmatrix} \widetilde{\mathbf{x}}\_{\mathbf{q}} = \begin{bmatrix} \mathbf{J}\_{Rd}^{\Sigma} & \mathbf{0} \\ \mathbf{\dot{J}}\_{Rd}^{\Sigma} & \mathbf{J}\_{Rd}^{\Sigma} \end{bmatrix} \widetilde{\mathbf{x}}\_{\mathbf{q}} = \mathbf{T}\_{\mathbf{q}}^{\Sigma} \widetilde{\mathbf{x}}\_{\mathbf{q}}. \tag{20}
$$

<sup>3</sup>This research is focused on "Errand Completion Tasks" as opposed to "Time Management Tasks" (Sohlberg and Mateer, 2001), i.e., we assume that the timing in performing the rehabilitation task is not critical.

### 2.2.2. Building Updated LQR Matrices

We define <sup>6</sup> and <sup>6</sup> in the operational space cost function (11) as positive definite diagonal matrices. Using the mapping Equation (20) we can correlate the first terms of the two quadratic cost functions (10,11); thus, **<sup>q</sup>** can be defined as:

$$\mathbb{Q}\_{\mathbf{q}} = \left(\mathfrak{T}\_{\mathbf{q}}^{\Sigma}\right)^{T} \mathbb{Q}\_{\Sigma} \mathfrak{T}\_{\mathbf{q}}^{\Sigma}. \tag{21}$$

Note that **<sup>q</sup>** is positive definite, since <sup>6</sup> is positive definite. Since:

$$
\widetilde{\mathbf{F}}\_{\Sigma} = \mathbf{R}\_{\Sigma} \widetilde{\mathbf{F}}, \tag{22}
$$

considering (12) we can also rearrange the energy term in the joint space cost function (10) as:

$$\begin{split} \widetilde{\mathbf{u}}^{T} \mathbf{R}\_{\mathbf{q}} \widetilde{\mathbf{u}} &= \left( \mathbf{J}\_{Rd}^{T} \widetilde{\mathbf{F}} \right)^{T} \mathbf{R}\_{\mathbf{q}} \left( \mathbf{J}\_{Rd}^{T} \widetilde{\mathbf{F}} \right) = \left( \mathbf{J}\_{Rd}^{T} \mathbf{R}\_{\Sigma} \widetilde{\mathbf{F}}\_{\Sigma} \right)^{T} \mathbf{R}\_{\mathbf{q}} \left( \mathbf{J}\_{Rd}^{T} \mathbf{R}\_{\Sigma} \widetilde{\mathbf{F}}\_{\Sigma} \right) \\ &= \widetilde{\mathbf{F}}\_{\Sigma}^{T} \mathbf{R}\_{\Sigma}^{T} \mathbf{J}\_{Rd} \mathbf{R}\_{\mathbf{q}} \mathbf{J}\_{Rd}^{T} \mathbf{R}\_{\Sigma} \widetilde{\mathbf{F}}\_{\Sigma} \geqslant 0. \end{split} \tag{23}$$

Now since <sup>6</sup> is positive definite, if:

$$\mathbb{R}\_{\mathbf{q}} = \left(\mathbf{R}\_{\Sigma}^{T}\mathbf{J}\_{Rd}\right)^{-1} \mathbb{R}\_{\Sigma} \left(\mathbf{J}\_{Rd}^{T}\mathbf{R}\_{\Sigma}\right)^{-1} = \left(\mathbf{J}\_{Rd}^{\Sigma}\right)^{-1} \mathbb{R}\_{\Sigma} \left(\mathbf{J}\_{Rd}^{\Sigma}\right)^{-T}.\tag{24}$$

**<sup>q</sup>** is also positive definite unless the robot is at a singularity point. Based on (24), minimizing the energy term in the joint space cost function (10) will indirectly minimize the energy term in the operational space cost function (11).

#### 2.2.3. Optimal Impedance Control

With the updated LQR matrices, the optimal impedance controller scheme takes the structure shown in **Figure 2**. Using (3), the driving torque will be:

$$\mathbf{T}\_R = \mathbf{u} + \mathbf{J}\_R^T \mathbf{F}\_{ext},\tag{25}$$

where the control input **u** is defined such that it should optimally control the error dynamics (by <sup>−</sup>e**u**) while applying the nominal control input ( ◦ **u**). Thus, it will have the following form:

$$\begin{cases} \mathbf{u} \stackrel{\Delta}{=} \stackrel{\circ}{\mathbf{u}} - \widetilde{\mathbf{u}}, \\ \stackrel{\circ}{\mathbf{u}} = \mathbf{u}\_d - \mathbf{J}\_{Rd}^T \mathbf{F}\_{ext}, \\ \widetilde{\mathbf{u}} = -\mathbb{K} \widetilde{\mathbf{x}}\_{\mathbf{q}}. \end{cases} \tag{26}$$

Note that the nominal control input is equal to the desired system dynamics (desired control input) minus the torque caused by the interaction force at any desired location. This subtraction (**u**<sup>d</sup> − **J** T Rd**F**ext) at a zero tracking error will lead to a zero desired interaction force in (8). Finally, the Equations (25), (26) are used to satisfy the impedance model (8) (as shown in **Appendix A**), in order to overcome the robot dynamics and interaction force.

## 3. CONTROLLER ASSESSMENT

Here, the simulation procedure for controller tuning, assessment and comparison is presented. Then, the experimental procedure for validation of the simulation results is discussed. Finally, assessment criteria for controller evaluation are provided.

## 3.1. Simulations

In robotic rehabilitation, it is usually desired to follow a predefined path. During a path-following task, at least three therapy cases can occur (Ding et al., 2007; Amirabdollahian, 2011):


Here, to evaluate the performance of the controller during a rehabilitation procedure, four modes of movement are considered: impaired, healthy, delayed and time-advanced hand

movement along the specified path. Each of these modes will lead to one of the above therapy cases.

In the impaired hand movement mode, the upper extremity of the patient is totally dysfunctional (zero muscle activation), so the passive case will occur. In the healthy mode, the patient has normal timing and coordination, so one of the active cases can happen depending on the healthy subject performance. The delayed hand movement mode is used to model a stroke patient who needs assistance during therapy, and it leads to the activeassisted case. The time-advanced mode models a patient with rapid hand movements; thus, the active-constrained case will be enabled. The performance of the proposed controller (OIC) is compared to the SOIC (Siciliano et al., 2009; Hogan, 2017) (see **Appendix C**), which is also designed for the robot to perform in four modes of the movement.

For LQR weights, matrix <sup>6</sup> is defined to have less position error along the normal to the path, and matrix <sup>6</sup> is considered to have less force error along the path (see **Appendix B** for choosing LQR weights). To run simulations, we should consider a trajectory for manipulation. Based on Ghannadi et al. (2014b) the trajectory is approximated by a smooth curvilinear path with a large radius of curvature (**Figure 1B**). Then, a cubic spline interpolation approach is used to generate the path with a bellshaped tangential speed profile and minimum jerk (see **Figure 3**).

Generated MapleSimTMmodels are exported as optimized MATLAB <sup>R</sup> S-functions into the Simulink/MATLAB <sup>R</sup> environment. Sampling-time frequency of the simulations is set to ν = 1000 Hz, and a fixed-step Euler solver is selected to solve the ordinary differential equations.

## 3.2. Experiments

To evaluate the performance of the controllers during a rehabilitation procedure, a healthy male subject performed four modes of movement similar to the simulations (see **Figure 1A**). To this end, the following protocols are considered:


In the last three protocols, the point on the curvilinear path at each simulation time step is defined by a small circular region. The subject should try to keep the end-effector position inside that region while tracing the curvilinear path. In other words, the small circular regions define the accuracy required for tracing the curvilinear path. To reduce the effect of random/noisy movements, each mode for each controller was performed in 10

trials. Tests of the two controllers were alternated randomly to reduce the effect of learning.

In our controller design, the optimality criterion is to minimize the tracking and applied force errors in the operational space. As discussed in the Introduction, the CNS optimally controls a human's environmental interaction. Hence, in a rehabilitation task, aligned with the CNS decision making process, the subject tries to minimize the interaction force and tracking error as much as possible. For the active-assisted (or active-constrained) therapy case, if the amount of assisting (or resisting) force can be increased with minimal change in the position tracking error induced by the subject's latency (or rapid movement), at first, the subject will decrease (or increase) muscle activations. However, later in the next stages of the therapy, unimproved tracking accuracy with higher interaction force will entice the subject to reduce the tracking error in order to decrease the applied interaction force, and this is done by increasing (or decreasing) muscle activations. In other words, in active-assisted therapy, an impaired subject may feel more assistive force if they are not able to minimize the tracking error. The increase in assistive (or resistive) force is regulated by the impedance model. In contrast to SOIC, this amount is achieved optimally in OIC based on the robot's configuration. That is why, later in the section 4, the optimal increase in interaction force in OIC is more significant than in SOIC.

The optimal increase in interaction force in OIC is in contrast to a conventional proportional-integral-derivative (PID) controller, which tries to minimize the tracking error while increasing the interaction force. With the PID controller, the subject will not have any motivation to minimize the tracking error, since the robot has already reduced it. Furthermore, if the subject suffers from stiff joints or muscle fatigue, the PID controller will increase his muscle activities, since he will resist against the robot movement and this may lead to injuries. It is worth noting that, in OIC, if the assistive (or resistive) force increases more than a certain amount so-called optimal interaction force, which is regulated by controller gains (similar to the PID controller), the tracking error will decrease and the subject will try to provide resisting force by increasing muscle activities.

Robot motors are rated at 115 mN-m of continuous torque. The continuous force at the hand (or end-effector) is limited to 13 N per plane of motion. Driven joint angles are measured by two optical encoders (with 4,000 count/revolution, which results in a sensitivity of 0.8 mm/count in detecting changes in the Cartesian space) connected to the motors. The disc and the belt mechanism increase the output torque by a ratio of 307/16. The robot end-effector has a 6-axis force-torque (FT) sensor, which measures the human-robot interaction forces and torques in body frame. The FT sensor has been calibrated to tolerate maximum 250 N on the horizontal plane and 1000 N normal to the plane. Sensor resolution is 1/24 N in the horizontal plane and 1/48 N normal to the plane.

The robot has frictional joints with stiffness and the manipulator moves on a frictional surface. These frictions are modeled using three continuous velocity-based frictional models (Brown and McPhee, 2016). Hence, robot dynamic Equation (1) is updated (refer to Ghannadi et al., 2017b for the detailed dynamic parameter identification of the robot):

$$\mathbf{T}\_R - \mathbf{J}\_R^T \mathbf{F}\_{int} = \mathbf{M}\_R(\mathbf{q})\ddot{\mathbf{q}} + \mathbf{C}\_R(\mathbf{q}, \dot{\mathbf{q}})\dot{\mathbf{q}} + \mathbf{K}\_R(\mathbf{q} - \mathbf{q}\_0) + \mathbf{f}\_T + \mathbf{J}\_R^T \mathbf{f}\_F = \boldsymbol{\Gamma}\_R(\mathbf{q}, \dot{\mathbf{q}}, \ddot{\mathbf{q}}).\tag{27}$$

where **f**<sup>F</sup> is the friction force under the end-effector in the global coordinates, **f**<sup>T</sup> is the friction torque vector at the joints, and **K**<sup>R</sup> is a 2 × 2 symmetric joint stiffness matrix.

The robot's computer software interface includes Simulink/MATLAB<sup>r</sup> which uses Quanser's real-time control software driver (QUARC). To control the robot, the driver software uses Quanser's data acquisition (DAQ) card (Q8). The driver and application software communicate through TCP/IP and shared memory protocol. To read the FT sensor data, a National Instruments DAQ card (PCI-6229) is used. This card is compatible and operable by the QUARC software. Sampling-time frequency of the experiments is set to ν = 500 Hz, and a fixed-step Euler solver is selected to solve the ordinary differential equations.

## 3.3. Assessment Criteria

Muscle activities during reaching tasks in upper extremity rehabilitation have been used as a measure for performance evaluation (Wagner et al., 2007). As discussed, if the assistive (or resistive) force is greater than a threshold (i.e., optimal interaction force) then the subject will try to provide resisting force by increasing muscle activities, which may cause injuries. Maintaining this increase in the assistive (or resistive) force less than the threshold will decrease (or increase) muscle forces (i.e., activations); this is in good correlation with the goals of robotic rehabilitation therapy (Amirabdollahian, 2011). Therefore, to capture the effect of increased assistance (or resistance) to the subject, we assess the simulated performance of the OIC and SOIC controllers using the activation results from the musculoskeletal model interacting with the robot. Furthermore, the dynamic response of the system is used to evaluate the controllers in terms of the amount of interaction force and tracking error.

## 4. RESULTS AND DISCUSSIONS

## 4.1. Muscular Activities

Since muscle activities less than 0.003 are mostly caused by suboptimal results and round-off calculation errors, muscles with activations less than this amount are not reported. Instead, active muscles with activations more than 0.003 are studied. These muscles are: Muscle 1, mono-articular shoulder flexor; Muscle 4, mono-articular shoulder extensor; and Muscle 5, bi-articular shoulder-elbow flexor.

As shown in **Figure 4**, for the delayed hand movement in both controllers, the robot assistance has decreased the amount of muscle activation compared to the other modes, for most of the path. In the delayed hand movement mode, when the robot detects a subject's latency, it increases the assistive force compared to the healthy hand movement. This increase in assistive force decreases the subject's muscular activities. Despite the increase in assistive force, the tracking accuracy has not

improved. Thus, compared to the healthy subject, the subject with lower muscular activities will feel higher interaction force but unimproved tracking accuracy.

For the time-advanced mode in both robot controllers, the amount of maximum muscle activations is higher than the other modes. Similar to the delayed hand movement, despite the increase in resistive force, the tracking accuracy has not changed significantly. Therefore, compared to the healthy subject, the subject with higher muscular activities will feel more interaction force but with unimproved tracking accuracy. In the healthy hand movement mode, both controllers result in the same amount of activation.

Simulation results for both the delayed and time-advanced hand movements (in OIC and SOIC) indicate that activeassisted and active-constrained therapies have been invoked, respectively. In the active-assisted case, muscular activities of the subject are too low for the task to be finished independently; hence, the robot assists him. In the active-constrained case, the robot resists the subject with high muscular activities. From the delayed hand movement results (weak tracking performance with high interaction force), we can speculate that the subject will try to improve the tracking accuracy in the next stages of therapy by applying more muscle force, thereby decreasing the interaction force. Similarly, in the next stages of the therapy with time-advanced hand movement (more resisting force with increased muscular activities), the subject may try to reduce the tracking error and resisting force by decreasing muscular activities. In other words, continuing the robot therapy will achieve levels of muscle activation close to those for a healthy subject.

Although, both controllers are successful in implementing active-assisted and active-constrained therapies, in the delayed hand movement mode, the decrease in muscle activation for the OIC is more than the SOIC (see the root mean square (RMS) values for the delayed mode in **Figure 5**). Thus, in OIC, if the delayed hand movement subject wants to improve tracking accuracy and decrease the assistive force, he can have a wider range of muscle activations compared to SOIC. With a higher assistive force compared to the SOIC, and a wider range for the muscle activation changes, the OIC can be used for a wider range of applications and patients in active-assisted therapy. In other words, the OIC is more effective in active-assisted therapy.

## 4.2. Dynamic Response

Normalized interaction forces and position errors in the operational space are compared for four modes (see **Figures 6**, **7** for simulation and experimental results). Interaction force results are normalized to the maximum applied force in the

FIGURE 5 | RMS of active muscle activations in three modes using OIC (solid fill) and SOIC (crosshatch fill). Circled numbers are corresponding to the active muscle numbers.

horizontal plane to show similar trends to the approximate and highly idealized simulation model. Both in experiments and simulations, tangential interaction force plots show that the amount of assistance (in the delayed condition) or resistance (in the time-advanced condition) for the OIC is slightly more than the SOIC. Both in simulations and experiments, normal

interaction force amount in the impaired hand movement mode for the OIC is not more than the SOIC, while for the other modes, the OIC results in higher values than the SOIC. This is because the position error in the normal direction is reduced by the OIC. However, the normal position error for the impaired hand movement mode in the SOIC is significantly more than the OIC. This shows that the optimal performance of the SOIC, especially in experiments, has failed to deal with impaired patients. The tangential position error is similar for both controllers.

shaded area denotes twice the standard deviation at each instance of experiment.

In simulations with the SOIC (**Figure 6**), after 3 s of the simulation, normal position error for the time-advanced hand movement is strictly increasing, and this will result in instability issues. However, this does not happen in experiments, since robot instability limited the selection of higher gains for the SOIC. Thus, in experiments, the robot in the SOIC is set to be more compliant. In simulations for the SOIC, between the position error and the interaction force, there is a linear relationship which is due to the linear time invariant (LTI) impedance model of the controller. However, for the OIC this relationship is nonlinear, and this is because of the LTV impedance model of the controller. One cannot see this nonlinear relationship because the robot's frictional forces have changed the system behavior and made it linear.

Regarding the controller structure, for the OIC, the statespace model is controllable and observable because at each operational point (and any intermittent interval), **C** and **O** are rank 4; furthermore, the dominant pole position of the LQR controller (which is the closest eigenvalue of - **<sup>q</sup>** − **q** to the imaginary axis) at each operational point has a negative real value, which makes the system critically damped. On the other hand, the optimum values of SOIC are such as to result in an under-damped system with a damping ratio of <sup>√</sup> 2/2.

## 5. CONCLUSION

In this study, we designed and verified a modified LQR controller (i.e., OIC) for optimal impedance control, which indirectly considers the operational space and interaction forces. This modified LQR controller was compared to the SOIC (which is based on the feedback linearization approach). Despite some similarities to the SOIC, the OIC has proven to be more efficient in passive, active-assisted, active-constrained therapy since it updates the impedance gains optimally during a reaching task (at different robot configurations). Physiologically, this efficient behavior causes less muscle activations in active-assisted therapy. Dynamically, the controller is more robust to disturbances caused by unknown dynamics, and the tracking error and interaction force are in a safer region.

Since the QUARC software does not support online LQR gain adjustments using an LQR s-function during experiments, an offline gain selection is done based on the desired configuration of the robot. In online gain selection, the gains are updated based on the current configuration of the robot. Hence, in offline gain selection, the implemented controller can be classified as an optimal passive trajectory tracking controller. In recent experiments, we managed to perform online gain adjustment with MATLAB's built-in LQR controller, but the results were similar to the offline gain selection results presented in this paper. In the offline gain selection experiments, we generated different modes similar to the simulations. In other words, we maintained the current configuration of the robot close to the desired configuration. That is why, similar results from the offline and online gain selection experiments are obtained. The controller's computational cost is the same as that of the SOIC, even if the LQR gains are adjusted online. In OIC, therapists will be able to modify the controller with a single parameter c in (36), which represents the effort/state balance weight; the inclusion of a single calibration parameter contributes to the superiority of the OIC over SOIC.

Here, an integrated human-robot dynamic system is used to fine-tune the controller gains. This method is advantageous for efficient tuning of the robot controllers in experiments. A good qualitative agreement between experiments and simulations verifies the effectiveness of this method.

Our proposed controller and tuning method can be used in any rehabilitation manipulandum system. Possible improvements for this method are as follows. First, for a linear robot model, the OIC assumes an apparent mass for the robot equal to its mass matrix, while the SOIC permits

## REFERENCES


offline changes to the robot's apparent mass. For considering the patient interaction dynamics, the robot's apparent mass should vary online as a function of the input frequencies of the system. However, neither the OIC nor SOIC offer such updates. Moreover, in regards to experiments, the unknown dynamics of the robot presents a challenging issue, independent of the controller. As a part of our future work, we will present a method to implement an OIC on the robot which also allows for online changes to the robot's apparent mass. Second, in the impedance model (8), the desired interaction force is assumed to be zero, while for implementing any high-level controller that deals with variable admittance environments (different patients or the same patients at different stages of their therapy) this desired interaction force should be updated by an outer-loop control law. In our future work, we will also develop the outer-loop controller to enhance the proposed OIC.

## ETHICS STATEMENT

This study was carried out on a single subject in accordance with the recommendations of the Tri-Hospital Research Ethics Board (THREB) and the University of Waterloo Office of Research Ethics (ORE) with written informed consent from the subject. The subject gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the Tri-Hospital Research Ethics Board (THREB) and the University of Waterloo Ethics Board.

## AUTHOR CONTRIBUTIONS

BG development of the proposed method. RS technical help for conducting experimental trials. JM supervisor of the project.

## ACKNOWLEDGMENTS

This work was funded by the Natural Sciences and Engineering Research Council of Canada (NSERC) and the Canada Research Chairs (CRC) program. The authors wish to thank Quanser Consulting Inc. for providing the upper extremity rehabilitation robot, and TRI for collaborating.


in 9th International Conference on Rehabilitation Robotics, 2005. ICORR 2005 (Chicago, IL: IEEE), 307–312.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Ghannadi, Sharif Razavian and McPhee. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

By applying the control law (25) and (26) to the robot dynamics (27) and substituting **u**<sup>d</sup> from (6), we get:

$$\mathbf{K}\widetilde{\mathbf{x}}\_{\mathbf{q}} + \Gamma\_R(\mathbf{q}\_d, \dot{\mathbf{q}}\_d, \ddot{\mathbf{q}}\_d) - \mathbf{J}\_{Rd}^T \mathbf{F}\_{ext} = \Gamma\_R(\mathbf{q}, \dot{\mathbf{q}}, \ddot{\mathbf{q}}).\tag{A1}$$

Using Taylor series expansion and (13), (A1) can be rearranged as:

$$\begin{split} \mathbf{J}\_{Rd}^{T} \mathbf{F}\_{ext} &= \mathbf{K} \widetilde{\mathbf{x}}\_{\mathbf{q}} + \mathbf{M}\_{Rd} \widetilde{\dot{\mathbf{q}}} + \left. \frac{\partial \Gamma\_{R}}{\partial \dot{\mathbf{q}}} \right|\_{\mathbf{q} = \mathbf{q}\_{d}, \dot{\mathbf{q}} = \dot{\mathbf{q}}\_{d}} \widetilde{\dot{\mathbf{q}}} + \left. \frac{\partial \Gamma\_{R}}{\partial \mathbf{q}} \right|\_{\mathbf{q} = \mathbf{q}\_{d}, \dot{\mathbf{q}} = \dot{\mathbf{q}}\_{d}} \widetilde{\mathbf{q}} \\ &= \mathbf{M}\_{Rd} \left[ \mathbf{0} \: \mathbf{I} \right] \widetilde{\dot{\mathbf{x}}}\_{\mathbf{q}} + \left[ \begin{array}{c} \mathbf{K}\_{P} \ \: \mathbf{K}\_{D} \end{array} \right] \widetilde{\mathbf{x}}\_{\mathbf{q}}, \end{split} \tag{A2}$$

where:

$$\begin{cases} \mathbb{K} = \left[ \mathbb{K}\_1 \; \mathbb{K}\_2 \right], \\\mathbb{K}\_P = \left. \frac{\partial \Gamma\_R}{\partial \mathbf{q}} \right|\_{\mathbf{q} = \mathbf{q}\_d, \dot{\mathbf{q}} = \dot{\mathbf{q}}\_d} + \mathbb{K}\_1, \\\mathbb{K}\_D = \left. \frac{\partial \Gamma\_R}{\partial \dot{\mathbf{q}}} \right|\_{\mathbf{q} = \mathbf{q}\_d, \dot{\mathbf{q}} = \dot{\mathbf{q}}\_d} + \mathbb{K}\_2. \end{cases} \tag{A3}$$

The following equations can be derived from (20):

$$\begin{aligned} \widetilde{\mathbf{x}}\_{\mathbf{q}} & \cong \begin{bmatrix} \mathbf{J}\_{Rd}^{\Sigma} & \mathbf{0} \\ \mathbf{J}\_{Rd}^{\Sigma} & \mathbf{J}\_{Rd}^{\Sigma} \end{bmatrix}^{-1} \\ \widetilde{\mathbf{x}}\_{\Sigma} & \coloneqq \begin{bmatrix} \left(\mathbf{J}\_{Rd}^{\Sigma}\right)^{-1} & \mathbf{0} \\ -\left(\mathbf{J}\_{Rd}^{\Sigma}\right)^{-1} \mathbf{J}\_{Rd}^{\Sigma} \left(\mathbf{J}\_{Rd}^{\Sigma}\right)^{-1} & \left(\mathbf{J}\_{Rd}^{\Sigma}\right)^{-1} \end{bmatrix} \widetilde{\mathbf{x}}\_{\Sigma} = \mathfrak{X}\_{\Sigma}^{\mathbf{q}} \widetilde{\mathbf{x}}\_{\Sigma}, \text{(A4)) \end{aligned}$$

$$
\widetilde{\dot{\mathbf{x}}}\_{\mathbf{q}} \cong \dot{\mathfrak{T}}\_{\Sigma}^{\mathbf{q}} \widetilde{\mathbf{x}}\_{\Sigma} + \mathfrak{T}\_{\Sigma}^{\mathbf{q}} \widetilde{\dot{\mathbf{x}}}\_{\Sigma}.\tag{A5}
$$

Thus, (A2) can be written as:

$$\begin{split} \mathbf{J}\_{Rd}^{T}\mathbf{F}\_{ext} &= \mathbf{M}\_{Rd}\Big[\mathbf{0}\ \mathbf{I}\right] \mathfrak{T}\_{\Sigma}^{\mathbf{q}}\widetilde{\mathbf{x}}\_{\Sigma} \\ &+ \mathbf{M}\_{Rd}\Big[\mathbf{0}\ \mathbf{I}\right] \dot{\mathfrak{T}}\_{\Sigma}^{\mathbf{q}}\widetilde{\mathbf{x}}\_{\Sigma} + \left[\mathbf{K}\_{P}\ \mathbf{K}\_{D}\right] \mathfrak{T}\_{\Sigma}^{\mathbf{q}}\widetilde{\mathbf{x}}\_{\Sigma}. \text{ (A6)} \end{split} $$

(A6) is corresponding to the LTV impedance model (8), if:

$$\begin{cases} \mathbf{M}\_{imp} = (\mathbf{J}\_{Rd}^{\Sigma})^{-T} \mathbf{M}\_{Rd} (\mathbf{J}\_{Rd}^{\Sigma})^{-1}, \\ \mathbf{B}\_{imp} = (\mathbf{J}\_{Rd}^{\Sigma})^{-T} \mathbf{K}\_{D} (\mathbf{J}\_{Rd}^{\Sigma})^{-1} - 2 \mathbf{M}\_{imp} \mathbf{j}\_{Rd}^{\Sigma} (\mathbf{J}\_{Rd}^{\Sigma})^{-1}, \\ \mathbf{K}\_{imp} = (\mathbf{J}\_{Rd}^{\Sigma})^{-T} \mathbf{K}\_{P} (\mathbf{J}\_{Rd}^{\Sigma})^{-1} - \left(\mathbf{M}\_{imp} \mathbf{j}\_{Rd}^{\Sigma} + \mathbf{B}\_{imp} \mathbf{j}\_{Rd}^{\Sigma}\right) (\mathbf{J}\_{Rd}^{\Sigma})^{-1}. \end{cases} \tag{A7}$$

## APPENDIX B: CHOOSING LQR GAINS

Matrices <sup>6</sup> and <sup>6</sup> have diagonal weights:

$$\begin{cases} \mathbb{Q}\_{\Sigma} = \text{Diag}\left( \mathbb{Q}\_{\Sigma1}, \mathbb{Q}\_{\Sigma2}, \mathbb{Q}\_{\Sigma3}, \mathbb{Q}\_{\Sigma4} \right), \\\mathbb{R}\_{\Sigma} = \text{Diag}\left( \mathbb{R}\_{\Sigma1}, \mathbb{R}\_{\Sigma2} \right), \end{cases} \tag{A8}$$

where these weights are chosen such that the cost function results in the allowable error associated with the state or effort, in other words:

$$\begin{cases} \mathbb{Q}\_{\Sigma,i} = \mathcal{y}\_{tol,i}^{-2} & (i = 1..4), \\\mathbb{R}\_{\Sigma,j} = cF\_{tol,j}^{-2} & (j = 1..2), \end{cases} \tag{A9}$$

TABLE A1 | Experimental values of the OIC coefficients.


TABLE A2 | Experimental values of the SOIC coefficients.


in which ytol,<sup>k</sup> and Ftol,<sup>k</sup> are the allowable amount of the k th element of the state (e**x**6) and effort (6) vector errors, respectively. These weights should also be adjusted such that the (A7) results in positive definite impedance gains. Coefficient c > 0 will be controlled by the therapist to adjust the effort/state balance. The coefficients of the experiments for the OIC are given in **Table 1**.

## APPENDIX C: STANDARD OPTIMAL IMPEDANCE CONTROL

For the robot dynamics (27), using nonlinear feedback linearization (inverse dynamics approach), we define the control law as (Siciliano et al., 2009):

$$\mathbf{T}\_R = \mathbf{J}\_R^T \mathbf{F}\_{ext} + \mathbf{F}\_R(\mathbf{q}, \dot{\mathbf{q}}, \mathbf{y}),\tag{A10}$$

where **y** is the outer loop control law and is defined such that it changes manipulator behavior to a linear impedance under interaction force error. In other words, it is desired to have the linear impedance model in the operational space as in (8) with time invariant coefficients. This impedance model can be achieved if the outer loop control law is defined as:

$$\mathbf{y} = \mathbf{J}\_{\mathcal{R}}^{-1} \mathbf{R}\_{\Sigma} \mathbf{M}\_{imp}^{-1} \left( \mathbf{M}\_{imp} (\dot{\mathbf{b}} - \mathbf{J}\_{\mathcal{R}}^{\Sigma} \dot{\mathbf{q}}) + \mathbf{B}\_{imp} \widetilde{\boldsymbol{\rho}}^{\Sigma} + \mathbf{K}\_{imp} \widetilde{\boldsymbol{\rho}}^{\Sigma} + \widetilde{\mathbf{F}}\_{\text{ext}}^{\Sigma} \right), \tag{A.11}$$

where:

$$\begin{cases} \dot{\mathbf{b}} = \mathbf{R}\_{\Sigma}^{T} \widetilde{\boldsymbol{\rho}}\_{d} - \dot{\mathbf{w}}\_{\Sigma} \widetilde{\boldsymbol{\rho}}^{\Sigma} + \mathbf{w}\_{\Sigma} \mathbf{w}\_{\Sigma} \widetilde{\boldsymbol{\rho}}^{\Sigma} + \mathbf{w}\_{\Sigma} \mathbf{R}\_{\Sigma}^{T} (\mathbf{J}\_{R} \dot{\mathbf{q}} - 2 \dot{\boldsymbol{\rho}}\_{d}), \\ \dot{\mathbf{J}}\_{R}^{\Sigma} = \mathbf{R}\_{\Sigma}^{T} \dot{\mathbf{j}}\_{R} - \mathbf{w}\_{\Sigma} \mathbf{R}\_{\Sigma}^{T} \mathbf{J}\_{R}, \\ \dot{\mathbf{R}}\_{\Sigma} = \mathbf{w}\_{\Sigma} \mathbf{R}\_{\Sigma}, \\ \mathbf{w}\_{\Sigma} = \begin{bmatrix} 0 & -\dot{\boldsymbol{\rho}}\_{\Sigma} \\ \dot{\boldsymbol{\rho}}\_{\Sigma} & 0 \end{bmatrix}. \end{cases} \tag{A12}$$

For this controller, with a predefined diagonal mass coefficient matrix, the standard optimum stiffness and damping are as follows (Hogan, 2017):

$$\begin{cases} \mathbf{M}\_{imp} = \text{Diag}\left(M\_{imp1}, M\_{imp2}\right), \\ \mathbf{B}\_{imp} = \text{Diag}\left(\frac{F\_{\text{bol1}}}{\mathcal{Y}\_{\text{bol1}}}, \frac{F\_{\text{bol2}}}{\mathcal{Y}\_{\text{bol2}}}\right), \\ \mathbf{K}\_{imp} = \text{Diag}\left(\sqrt{\frac{2F\_{\text{bol1}}M\_{imp1}}{\mathcal{Y}\_{\text{bol1}}}}, \sqrt{\frac{2F\_{\text{bol2}}M\_{imp2}}{\mathcal{Y}\_{\text{bol2}}}}\right). \end{cases} \tag{A13}$$

The coefficients of the experiments for the SOIC are given in **Table 2**.

Frontiers in Robotics and AI | www.frontiersin.org

# Individual Leg and Joint Work during Sloped Walking for People with a Transtibial Amputation Using Passive and Powered Prostheses

*Jana R. Jeffers1 \* and Alena M. Grabowski 1,2*

*1Applied Biomechanics Laboratory, Department of Integrative Physiology, University of Colorado Boulder, Boulder, CO, United States, 2Applied Biomechanics Laboratory, Eastern Colorado Healthcare System, Department of Veterans Affairs, Denver, CO, United States*

People with a transtibial amputation using passive-elastic prostheses exhibit reduced prosthetic ankle power and push-off work compared to non-amputees and compensate by increasing their affected leg (AL) hip joint work and unaffected leg (UL) ankle, knee, and hip joint and leg work during level-ground walking. Use of a powered ankle–foot prosthesis normalizes step-to-step transition work during level-ground walking over a range of speeds for people with a transtibial amputation, but the effects on joint work during level-ground, uphill, and downhill walking have not been assessed. We investigated how use of passive-elastic and powered ankle–foot prostheses affect leg joint biomechanics during level-ground and sloped walking. 10 people with a unilateral transtibial amputation walked at 1.25 m/s on a dual-belt force-measuring treadmill at 0°, ±3°, ±6°, and ±9° using their own passive-elastic and a powered prosthesis (BiOM T2, BionX Medical Technologies, Inc., Bedford, MA, USA) while we measured kinematic and kinetic data. We calculated AL and UL prosthetic, ankle, knee, hip, and individual leg positive, negative, and net work. Use of a powered compared to passive-elastic ankle– foot prosthesis resulted in greater AL prosthetic and individual leg net work on uphill and downhill slopes. Over a stride, AL prosthetic positive work was 23–30% greater (*p* < 0.05) during walking on uphill slopes of +6°, and +9°, prosthetic net work was up to 10 times greater (more positive) (*p* ≤ 0.005) on all uphill and downhill slopes and individual leg net work was 146 and 82% more positive (*p* < 0.05) at uphill slopes of +6° and +9°, respectively, with use of the powered compared to passive-elastic prosthesis. Greater prosthetic positive and net work through use of a powered ankle–foot prosthesis during uphill and downhill walking improves mechanical work symmetry between the legs, which could decrease metabolic cost and improve functional mobility in people with a transtibial amputation.

Keywords: joint work, individual leg work, amputee, uphill walking, downhill walking

## INTRODUCTION

Typically, people with a transtibial amputation are prescribed a passive-elastic energy storage and return (ESAR) prosthesis that is made of carbon fiber and functions like a spring with no ability to generate power *anew* or to articulate. When people with a unilateral transtibial amputation use such passive-elastic prostheses, they have 10–30% higher metabolic demands to walk at the

#### *Edited by:*

*Monica A. Daley, Royal Veterinary College, United Kingdom*

#### *Reviewed by:*

*Alexandra Voloshina, Stanford University, United States Kiisa Nishikawa, Northern Arizona University, United States*

*\*Correspondence: Jana R. Jeffers jana.jeffers@colorado.edu*

#### *Specialty section:*

*This article was submitted to Bionics and Biomimetics, a section of the journal Frontiers in Robotics and AI*

*Received: 29 September 2017 Accepted: 11 December 2017 Published: 22 December 2017*

#### *Citation:*

*Jeffers JR and Grabowski AM (2017) Individual Leg and Joint Work during Sloped Walking for People with a Transtibial Amputation Using Passive and Powered Prostheses. Front. Robot. AI 4:72. doi: 10.3389/frobt.2017.00072*

same speeds as non-amputees (Torburn et al., 1995; Waters and Mulroy, 1999; Hsu et al., 2006) and compensate for the lack of prosthetic push-off work with increased unaffected leg (UL) and decreased affected leg (AL) step-to-step transition work (Herr and Grabowski, 2012; Adamczyk and Kuo, 2015; Russell Esposito et al., 2016). People with a transtibial amputation using an ESAR prosthesis also exhibit slower preferred walking velocities (Herr and Grabowski, 2012; Russell Esposito et al., 2014), increased sagittal plane angular momentum (Pickle et al., 2016), and increased knee joint adduction moments in their UL (Grabowski and D'Andrea, 2013) compared to non-amputees. In addition, when walking on level ground using passive-elastic prostheses, people with a transtibial amputation exhibit an increase in knee flexion in their AL compared to their UL at heel-strike and activate their AL biceps femoris more than their UL biceps femoris, suggesting that greater work is absorbed at the knee (Isakov et al., 2000). The increased AL knee flexion has been attributed to the shape of the prosthetic socket that is designed to increase patellar tendon loading for patellar tendon bearing sockets (Isakov et al., 2000). However, when people with a transtibial amputation walked on level ground with a conventional solid-ankle cushioned heel prosthesis, there was almost no positive or negative sagittal plane knee power during the first half of the stance phase (Winter and Sienko, 1988). The advent of ESAR prosthetic feet has resulted in no changes to knee sagittal plane range of motion compared to use of older, conventional solid-ankle cushioned heel prostheses (Postema et al., 1997) though it is not yet known how more advanced prostheses affect knee sagittal plane moments and powers. Normative knee moments and powers could improve symmetry between legs and mechanical energy transfer across the AL knee joint of people with transtibial amputations.

Previous modeling and experimental studies have found that people with a unilateral transtibial amputation walking 0.6–1.6 m/s over level ground while using a passive-elastic prosthesis compensate for reduced ankle push-off work with increased UL and AL hip positive work (Zmitrewicz et al., 2006; Silverman et al., 2008; Adamczyk and Kuo, 2015). But, in an often-cited study regarding compensatory strategies adopted by people with a transtibial amputation during walking on level ground, subjects used prostheses (solid-ankle cushioned heel) that were not designed to restore push-off energy to the wearer (Winter and Sienko, 1988). These studies also included some subjects who had a transtibial amputation due to vascular disease (Winter and Sienko, 1988; Silverman et al., 2008). People who undergo a transtibial amputation due to vascular disease, as opposed to a traumatic or congenital amputation, typically require even higher metabolic energy to walk at the same speeds as non-amputees on level ground and have a slower preferred walking speed (Torburn et al., 1995). Higher metabolic cost and a slower preferred walking speed could also be attributed to the redistribution of positive push-off work from the ankle to the hip, similar to the redistribution of joint work in elderly populations (Franz and Kram, 2013). Though many studies have shown a 10–30% higher metabolic cost for people with either a traumatic or dysvascular transtibial amputation using a passiveelastic prosthesis during level-ground walking compared to nonamputees walking at the same speed (Torburn et al., 1995; Waters and Mulroy, 1999; Hsu et al., 2006), a recent study of young subjects (average age 29 years) found that people with a traumatic transtibial amputation using a passive-elastic ESAR prosthesis during levelground walking do not have an increased metabolic cost compared to non-amputees over a range of speeds (0.74–1.68 m/s) (Russell Esposito et al., 2014). Furthermore, the biomechanical effects of using a passive-elastic ESAR prosthesis on level-ground walking step-to-step transition work are inconclusive. Herr and Grabowski (2012) found that use of a passive-elastic ESAR prosthesis resulted in significantly higher leading leg negative and significantly lower trailing leg positive step-to-step transition work during levelground walking over a range of speeds (0.75–1.75 m/s) (Herr and Grabowski, 2012). However, Russell Esposito et al. (2016) found that use of a passive-elastic ESAR prosthesis did not significantly affect leading leg step-to-step transition work during level-ground walking compared to non-amputees. Thus, the metabolic and biomechanical effects of walking on level ground while using a passive-elastic ESAR prosthesis are unclear.

A commercially available powered ankle–foot prosthesis (BiOM) has been developed that contains a one degree of freedom ankle articulation (plantar- and dorsi-flexion) and generates battery-powered mechanical push-off work in late stance through series-elastic actuation. A state space controller, which is based on level-ground biological ankle work loops (moment vs. angle curve) during steady speed walking, is used to govern the response of the BiOM (Au et al., 2007) based on prosthetic ankle position (angle) from the encoder. To tune the response of the BiOM to the wearer, tuning parameters within the device are adjusted until the wearer's net prosthetic ankle work is within 2 SDs of average non-amputee ankle work values (BionX Medical Technologies, Inc., 2016). Use of this powered ankle–foot prosthesis has normalized the metabolic costs and biomechanics (preferred walking speed, stepto-step transition work) during level-ground walking at speeds of 0.75–1.50 m/s for people with a transtibial amputation compared to non-amputees (Herr and Grabowski, 2012; Russell Esposito et al., 2016). However, to our knowledge, only three studies have investigated how use of a powered prosthesis affects the metabolic cost and biomechanics of uphill walking compared to use of a passive-elastic prosthesis. Use of the powered prosthesis normalized metabolic cost while walking on level ground and normalized trailing leg step-to-step transition work on both level ground and a 5° uphill ramp (Russell Esposito et al., 2016). Use of the powered compared to passive-elastic prosthesis reduced hamstring muscle activation on uphill slopes of +3°, +6°, and +9° (Pickle et al., 2017) and reduced the range of sagittal plane whole-body angular momentum on slopes of −10°, −5°, 0°, and +5° (Pickle et al., 2016). It remains unclear how use of a powered ankle–foot prosthesis affects leg joint work contributions during uphill and downhill walking over a range of slopes compared to use of a passive-elastic prosthesis. Use of a powered ankle–foot prosthesis that can restore leg joint biomechanics and work during uphill and downhill walking could normalize metabolic cost and improve the overall function of people with transtibial amputations.

Biomimetic mechatronic devices, such as prostheses, orthoses, and exoskeletons, have been designed to match nonamputee leg joint biomechanics in order to restore function in individuals with a physical impairment and augment function in unimpaired individuals (Zoss et al., 2005; Au et al., 2009; Cherelle et al., 2012; Collins et al., 2015). When non-amputees walk uphill at 10°, the hip and ankle work combine to provide 86 and 95% of leg positive and net leg work, respectively, and to walk downhill at −10° the muscles acting at the knee perform 58 and 81% of the negative and net leg work (DeVita et al., 2007). Furthermore, as non-amputees walk uphill at 21.3°, peak positive hip moment more than doubles, peak positive ankle moment increases 19%, and peak positive knee moment almost doubles, compared to walking on level ground (Lay et al., 2006). Conversely, when non-amputees walk downhill at −21.3°, peak positive hip moment remains constant, peak positive ankle moment decreases 44% and the magnitude of peak negative knee moment increases over fourfold compared to walking on level ground (Lay et al., 2006). It is not yet clear how people with a transtibial amputation using powered and passive-elastic prostheses adapt to walking on uphill and downhill slopes. It is likely that the design of future mechatronic devices such as powered ankle–foot prostheses should mimic healthy non-amputee gait, but the effects of current device designs for walking on various slopes for people with a leg amputation are unknown. This information would be useful for determining if prostheses can restore function to people with an amputation and/or the modifications needed to improve prosthetic design. Thus, we sought to determine the effects of using a passive and powered ankle–foot prosthesis on leg joint biomechanics of people with a transtibial amputation walking on level-ground and at a range of uphill and downhill slopes.

People with transtibial amputations using passive-elastic prostheses have increased UL and decreased AL step-to-step transition work compared to non-amputees, but have equivalent step-to-step transition work when using a powered ankle–foot prosthesis during level-ground and inclined walking. Thus, we hypothesized that UL total individual leg positive and net work would decrease, and AL total individual leg positive and net work would increase with use of the powered compared to passiveelastic prosthesis when walking at each uphill and downhill slope. Use of a passive-elastic prosthesis results in lower AL prosthetic ankle work, no change in AL knee range of motion, and increased UL and AL hip positive work compared to use of a powered ankle–foot prosthesis on level ground and on a 5° incline. Because a powered ankle–foot prosthesis provides push-off power and net positive work, we hypothesized that UL ankle and hip joint positive and net work would decrease, AL prosthetic ankle positive and net work would increase and hip joint positive and net work would decrease, and knee joint work would remain unchanged when walking at each slope while using a powered compared to passive-elastic prosthesis.

## MATERIALS AND METHODS

## Subject Recruitment

Ten healthy adults with a traumatic unilateral transtibial amputation (6 M, 4 F, mean ± SD: age 42 ± 11 years, height 1.7 ± 0.08 m, and mass without a prosthesis 77.3 ± 14.8 kg) (**Table 1**) provided written informed consent according to the Declaration of Helsinki and US Department of Veterans Affairs institutional review board. Subjects self-reported that they were at a Medicare functional classification level of K3 or higher, and free of neurological, cardiovascular, and musculoskeletal disease other than that associated with a unilateral transtibial amputation.

## Experimental Protocol

#### Tuning of the Powered Prosthesis

First, a certified prosthetist from BionX Medical Technologies aligned the powered prosthesis (BiOM T2, BionX Medical Technologies, Inc., Bedford, MA, USA) to each subject. We then placed reflective markers on subjects' lower limbs according to a modified Helen Hayes marker set. We placed markers over joint centers and clusters of four markers over each segment. We also placed reflective markers on the AL at the approximate locations of the prosthetic foot 1st and 5th metatarsal heads, posterior calcaneus, and medial and lateral malleoli, matching the locations on the UL. We placed "malleoli" markers for the powered prosthesis on the encoder, which coincides with the center of rotation in the sagittal plane (**Figure 1**). Subjects then walked using the BiOM at

TABLE 1 | Subject anthropometrics and their own passive-elastic prosthetic foot model.


FIGURE 1 | "Malleolus" marker placement on the encoder/center of rotation for the BiOM powered prosthesis.

1.25 m/s on a dual-belt force-measuring treadmill (Bertec Corp., Columbus, OH, USA) for a series of 45-s trials at slopes of 0°, ±3°, ±6°, and ±9° while we simultaneously measured kinematics at 100 Hz (Vicon, Oxford, UK) and ground reaction forces (GRFs) at 1,000 Hz. We filtered GRFs using a fourth order recursive Butterworth filter with a 30 Hz cutoff and filtered kinematic data using a sixth-order recursive Butterworth filter with a 7 Hz cutoff in a custom Matlab (Mathworks, Natick, MA, USA) script. Perpendicular GRF data from each leg were used to determine ground contact with a 20 N threshold.

To objectively tune the BiOM, we calculated prosthetic ankle angles, moments, powers, and net mechanical work normalized to body mass, including prosthetic mass using Visual 3D software (C-Motion, Germantown, MD, USA) after each 45-s trial and compared these data with averages from 20 non-amputees walking at the same speed and slopes (Jeffers et al., 2015). Similar to Ventura et al. (2011), we did not adjust the rigid segment model foot or shank in Visual 3D and used inertial properties inherent in the Visual 3D anatomical model due to the similar weight of the BiOM and an anatomical foot and shank (21.6 N). We then iteratively and systematically tuned the BiOM using a tablet with software provided by the manufacturer (BionX Medical Technologies, Bedford, MA, USA) until the lab-measured kinematic and kinetic data for AL prosthetic ankle range of motion, peak moment, peak power, and net work normalized to body mass matched the UL and non-amputee averages within 2 SDs at each slope (**Figure 2**). We manipulated tuning parameters including "Stiffness," "Power Fast," "Power Slow," "Power Timing Fast," "Power Timing Slow," "Power Sensitivity," "Stiffness Duration," "Stance Dampening," "Cadence Range," and "Hard Stop Sensitivity." We tuned the BiOM at each slope so that each subject replicated biological ankle biomechanics on that slope. We used the tuning parameters determined during up to two tuning sessions for each subject and each slope throughout the remainder of the experimental protocol.

### Kinetic and Kinematic Data Collection

Subjects walked on the treadmill while using the powered prosthesis for approximately 10 h over five experimental sessions on separate days at the same speed and slopes prior to the session where we measured kinetic and kinematic data for analyses. All experimental sessions were separated by at least 22 h and no more than 2 weeks. The first two sessions were each 2–3 h long and the third through fifth sessions were each 1.5 h long. Then, during the sixth session (approximately 2.5 h long), we simultaneously measured kinematics at 100 Hz and GRFs at 1,000 Hz while subjects walked at 1.25 m/s on a dual-belt force-measuring treadmill (Bertec Corp., Columbus, OH, USA) at slopes of 0°, ±3°, ±6°, and ±9° using the powered prosthesis tuned for each slope and their own passive-elastic prosthesis. Each trial was approximately 1-min long and we randomized the trial order. We used the same marker set as described above and placed "malleoli" markers for the passive-elastic prosthesis on the medial and lateral edges of the carbon fiber prosthesis at the most dorsal point of the keel. We

FIGURE 2 | Prosthetic ankle (A) range of motion, (B) peak moment, (C) peak power, and (D) net work from the iterative BiOM tuning process for subjects walking at 1.25 m/s over a range of slopes. We tuned the BiOM to match average unaffected leg (UL) data (black) within 2 SDs (gray shaded area) for ankle (A) range of motion, (B) peak moment, and (C) peak power. We tuned the BiOM to match average data from non-amputees (black) within 2 SDs for ankle (D) net work. BiOM prosthetic ankle data (ensemble average) collected at the end of the tuning day (white) were numerically higher (closer to UL and non-amputee data) for most biomechanical variables at all slopes compared to the final day of the protocol (gray). We used data collected on Day 6 for analyses. In some cases, the symbols overlap.

filtered GRFs using a fourth-order recursive Butterworth filter with a 30 Hz cutoff and filtered kinematic data using a sixth-order recursive Butterworth filter with a 7 Hz cutoff. Perpendicular GRF data from each leg were used to determine ground contact with a 20 N threshold. We calculated sagittal plane joint powers with Visual3D software (C-Motion, Germantown, MD, USA). Using a custom Matlab (MathWorks, Natick, MA, USA) script, we integrated joint power with respect to stride time to determine joint work over a stride (heel-strike to heel-strike of the same foot). We summed ankle, knee, and hip joint work over a stride to calculate leg work. We averaged at least five consecutive strides for each subject for each condition and calculated an ensemble average of all 10 subjects.

## Statistical Analyses

Prior to choosing a statistical approach to determine the effects of speed, slope, and their interaction on individual leg and joint mechanics, we tested for linearity and normality of the data with RStudio statistical software (RStudio, Boston, MA, USA). We determined linearity by visually inspecting residuals and Q-Q plots in RStudio (Kim, 2015). Similarly, we determined normality by visually inspecting histograms in RStudio. The data were not linearly related but were normally distributed. Because our hypotheses are based on changes in leg or joint work at each slope and on the effects of using each prosthesis on leg and joint work, we used one-way repeated measures ANOVAs with prosthetic foot type (powered BiOM or passive-elastic ESAR) as the independent variable and leg or joint positive, negative or net work as the dependent variable with a significance level of 0.05 at each slope. We removed data outliers (total 260 of 3360 individual data points) from statistical analyses if they fell outside the first or third interquartile range (R Studio, Boston, MA, USA).

## RESULTS

During the tuning sessions, we were able to tune the powered prosthesis (BiOM) such that prosthetic ankle net work matched average non-amputee ankle net work within 2 SDs at all slopes (**Figure 2**). We were also able to match prosthetic ankle range of motion, peak moment, and peak power to the UL ankle averages within 2 SDs on all slopes (**Figure 2**). Similarly, on the final day of the protocol when we collected kinematic and kinetic data, the tuning established during the tuning sessions and used throughout the acclimation trials resulted in prosthetic ankle biomechanics that matched non-amputee average ankle net work within 2 SDs (**Figure 2**). Prosthetic ankle biomechanics also matched UL average ankle joint range of motion within 1 SD for all slopes, peak ankle moment within 2 SDs for all slopes, and peak ankle power within 2 SDs on level and all uphill slopes (**Figure 2**). While we matched the mechanics of the powered prosthesis to either the UL ankle or non-amputee ankle averages within 2 SDs on both the tuning and data collection days, when averaged across all slopes there was a numeric 20% decrease in ankle range of motion, 9% decrease in peak ankle moment, and 28% decrease in peak ankle power on the final day of the protocol compared to the tuning days. There was an average numeric 37% decrease in net ankle work done by the powered prosthesis on all slopes except at −3°, where the prosthetic ankle net work increased by almost fourfold from 0.007 to 0.033 J/kg on the final day of the protocol compared to the tuning days.

We found no effect of prosthetic foot type on UL positive, negative, or net work for all slopes (*p* > 0.05, **Figure 3**). We found no effect of prosthetic foot type on AL positive work (*p* > 0.05) or on AL negative work for all slopes (*p* < 0.001, **Figure 3**). There was an effect of prosthetic foot type on AL net work at uphill slopes of +6° and +9° (*p*< 0.05, **Figure 3**). AL net work was 146%,

and 82% more positive on +6°, and +9° slopes, respectively, with use of the BiOM compared to ESAR prosthesis (**Figure 3**).

We found no effect of prosthetic foot type on UL ankle or hip joint positive, negative, or net work for all slopes (*p* > 0.05, **Figure 4**). However, we did find an effect of prosthetic foot type on AL positive ankle work for uphill slopes of +6° and +9° (*p* < 0.05, **Figure 4**) and on AL net ankle work for all slopes (*p* ≤ 0.001, **Figure 4**). AL positive ankle work increased 89 and 55% at +6° and +9°, respectively, with use of the BiOM compared to ESAR prosthesis (**Figure 4**). In addition, at +3° there was a trend for AL positive ankle work to be 44% greater with use of the BiOM compared to ESAR prosthesis (*p* = 0.0575, **Figure 4**). AL net ankle work was greater (i.e., more positive) for all slopes with use of the BiOM compared to ESAR prosthesis. Specifically, at downhill slopes of −9°, −6°, and −3° AL net ankle work was 94, 109, and 155% more positive, respectively, with use of the BiOM compared to ESAR prosthesis (**Figure 4**). At 0°, +3°, +6°, and +9°, AL net ankle work increased 3.5-, 3.4-, 6.7-, and 9.4-fold, respectively, with use of the BiOM compared to ESAR prosthesis (**Figure 4**). In other words, the BiOM provided almost ten times as much net ankle work as an ESAR prosthesis at the steepest uphill slope of +9°. We also found a significant effect of prosthetic foot type on AL negative ankle work at −9°, −3°, +6°, and +9° (*p* < 0.05, **Figure 4**). Specifically, AL negative ankle work was 35–45% less negative with use of the BiOM compared to ESAR prosthesis (**Figure 4**). We did not find an effect of prosthetic foot type on AL hip joint positive, negative, or net work (*p* > 0.05, **Figure 4**). Similarly, we found no effect of prosthetic foot type on UL or AL knee joint positive, negative, or net work (*p* > 0.05, **Figure 4**).

## DISCUSSION

In contrast to our hypothesis, which was based on results from level-ground and inclined walking, there were no changes in UL individual leg net work when subjects used the BiOM compared to their own ESAR prosthesis when walking on any slope. These results are in contrast with previous studies (Herr and Grabowski, 2012; Russell Esposito et al., 2016) that found leading (unaffected) leg step-to-step transition work was normalized with use of the BiOM powered ankle–foot prosthesis compared to an ESAR prosthesis on both level ground and a 5° incline. However, and in partial support of our hypothesis, AL individual leg net work was more positive (i.e., increased) at uphill slopes of +6° and +9° with use of the BiOM compared to ESAR prosthesis; but we found no effect of prosthetic foot type on individual leg total positive work. This increase in AL individual leg net work with use of the BiOM powered prosthesis compared to an ESAR prosthesis

prosthesis [unaffected leg (UL) is black, affected leg (AL) is gray], and passive-elastic energy storage and return (ESAR) prosthesis (UL is white and AL is hashed) during walking at 1.25 m/s across a range of slopes. \* indicates a significant difference in AL work between use of the BiOM powered and passive-elastic ESAR prosthesis.

is due to an increase in AL prosthetic ankle positive work and AL prosthetic ankle net work, which is in partial support of our hypothesis. Our hypotheses were based on level-ground walking studies; however, the BiOM state space controller is based on a biological ankle work loop for level-ground walking and provides net positive prosthetic ankle work on all slopes. Thus, it is possible that the BiOM is not optimized for walking up and down slopes. It is also possible that at moderate uphill and all downhill slopes, the substantial increase in AL prosthetic ankle positive and net work with use of the BiOM was absorbed at the knee or within the socket–limb interface and, thus, did not translate into significantly higher AL individual leg net work. Or, it is possible that the increased prosthetic push-off work provided by the BiOM was not enough for the user to overcome the compensation strategy typically adopted by individuals with a transtibial amputation using passive-elastic prostheses (Silverman et al., 2008; Adamczyk and Kuo, 2015) or that our subjects did not utilize the compensation strategy at all – as evidenced by no change in hip work, though AL prosthetic ankle positive and net work increased with use of the BiOM. Specifically, and in contrast to our hypotheses, there were no differences in hip or knee joint positive or net work for either leg. By contrast, previous studies found an increase in both UL work and AL hip work during level-ground walking with a passive prosthesis compared to non-amputees (Winter and Sienko, 1988; Silverman et al., 2008; Adamczyk and Kuo, 2015). The type of prosthesis may affect compensation strategies, however, the type of ESAR prosthesis used by subjects in Silverman et al. and Adamczyk and Kuo is not clear (Silverman et al., 2008; Adamczyk and Kuo, 2015), and Winter and Sienko (1988) had subjects use solid-ankle cushioned heel prostheses. Furthermore, these studies included some subjects who underwent a leg amputation due to vascular disease (Winter and Sienko, 1988; Silverman et al., 2008). These people typically have a slower preferred walking speed than those who have a congenital or traumatic amputation (Torburn et al., 1995), which could have exacerbated any compensation strategy when walking at the same speed—faster than preferred speed—as people with a traumatic or congenital leg amputation using a passive-elastic prosthesis.

Another possible explanation for the increase in AL and prosthetic ankle net work, but no change in UL or AL hip work with use of the powered compared to a passive-elastic prosthesis, could be that the provided prosthetic ankle push-off work is similar to the work provided by the uni-articular soleus, rather than the bi-articular gastrocnemius. An ankle–foot prosthesis does not span the knee and thus is uni-articular. Neptune et al. used musculoskeletal modeling and predicted that horizontal trunk propulsion/acceleration is primarily provided by the soleus and rectus femoris during late stance in non-amputees walking on level ground (Neptune et al., 2008). However, in experimental studies of non-amputees that measured muscle activation of the plantar-flexors during level-ground walking, ankle push-off work was primarily due to medial gastrocnemius activation, while the soleus played a small role in providing push-off work (Gottschall and Kram, 2003; Franz and Kram, 2013). Therefore, it is possible that a powered ankle–foot prosthesis that can only replace the function of the uni-articular soleus is incapable of fully replicating biological ankle function during walking. Furthermore, musculoskeletal modeling studies have found that the use of powered or passive-elastic prostheses increases whole-body sagittal plane angular momentum compared to non-amputees and that neither device is capable of providing power to the trunk similar to the biological gastrocnemius (Pickle et al., 2016, 2017). Future studies should investigate the mechanical energy loss and transfer from the prosthetic ankle to the residual limb, and investigate the role of the socket–limb interface in this energy transfer. Future prosthetic designs may need to incorporate a connection that crosses the knee joint to improve the energy transfer to and from the prosthesis in order to potentially normalize biomechanics (Endo et al., 2009).

We integrated sagittal plane joint power with respect to time to determine joint work. Joint power, and thus the net work done at the ankle, knee, and hip, was more positive when walking uphill and more negative when walking downhill when subjects used either prosthesis (**Figure 4**). Similar to non-amputees (Lay et al., 2006; DeVita et al., 2007), hip and ankle power became more positive with steeper uphill slopes (Figures S1 and S2 in Supplementary Material) and knee power became more negative with steeper downhill slopes (Figure S3 in Supplementary Material) when subjects with a transtibial amputation used either prosthesis. In support of our hypothesis, UL and AL sagittal plane knee joint work remained unchanged with use of the BiOM compared to an ESAR prosthesis. This result is in line with Postema et al. (1997) and Winter and Sienko (1988), who found no difference in sagittal plane AL knee range of motion or power during level-ground walking at a self-selected walking velocity with the use of an ESAR compared to conventional solid-ankle cushioned heel prosthesis. Furthermore, and similar to Winter and Sienko (1988), subjects exhibited little to no knee power in the first half of stance (Figure S3 in Supplementary Material), unlike non-amputees.

We iteratively tuned the BiOM prosthesis to match the average biological ankle sagittal plane range of motion, peak moment, peak power, and net work from 20 non-amputees at each slope (Jeffers et al., 2015). Though subjects used the same tuning parameters, prosthetic components, and alignment established in the tuning sessions for all experimental sessions, they likely modified the way that they walked while using the BiOM during the final experimental session compared to the tuning sessions (**Figure 2**). It is possible that after acclimation to walking while using the powered prosthesis, the tuning parameters should be further adjusted as the user modifies his or her gait. Previous studies that have analyzed the use of the BiOM prosthesis on level ground or up a 5° incline were completed in fewer experimental sessions (1–3 sessions in previous studies vs. 6 sessions for the present study), used over-ground measurements and relied on data collected from the BiOM's on-board microprocessor (Herr and Grabowski, 2012; Russell Esposito et al., 2016), rather than from independent treadmill-based motion capture and GRF data. Furthermore, these previous studies used force plates mounted in a walkway to collect GRF data from only a few steps (Herr and Grabowski, 2012; Russell Esposito et al., 2016) and, thus, differences could exist between our data, which measured GRFs from multiple consecutive steps at a set speed, and these studies. The differences in tuning strategies and protocol length may also potentially explain the differences in our results compared to others. Based on our ankle joint mechanics data (**Figures 2** and **4**), subjects were able to match the BiOM prosthetic ankle net work to within 2 SDs of average biological ankle values during tuning and on the final day of our protocol, though prosthetic ankle net work numerically decreased by an average of 37% on all slopes except −3°. Thus, it is possible that tuning on the first day of a longer protocol and using the same tuning strategy throughout could affect prosthetic ankle mechanical power output, net work or range of motion. Future studies are needed to better understand the interaction of the user and the prosthesis during acclimation and the effects of different tuning strategies. Future studies may also be needed to measure the effects of matching the response of the BiOM to within 1 SD of the average of non-amputees.

Despite the differences in prosthetic ankle biomechanics between the tuning days and the final day of the protocol, we found a significant increase in AL prosthetic net work for all slopes and a significant increase in AL prosthetic positive work at uphill slopes of +6° and +9° with use of the BiOM compared to ESAR prosthesis. Based on previous studies of individuals with impaired or no ankle function (Winter and Sienko, 1988; Powers et al., 1994; Silverman et al., 2008; Collins and Kuo, 2010; Adamczyk and Kuo, 2015), a reduction in ankle push-off work is related to slower preferred walking speed, increased kinematic and kinetic asymmetry, increased metabolic energy expenditure, and increased AL hip positive work production. Thus, by increasing AL prosthetic ankle positive and net work, use of a powered ankle–foot prosthesis may increase preferred walking speed, improve kinematic and kinetic symmetry between legs, decrease metabolic demand, and decrease reliance on the muscular work performed by the AL hip joint and UL on uphill and downhill slopes for people with a transtibial amputation. The changes in leg joint mechanics when using a powered compared with an ESAR prosthesis during walking over a range of slopes may, therefore, result in improved functional mobility and, thus, quality of life, specifically when navigating uphill slopes (Burger and Marincek, 1997; Ehde et al., 2001; Ephraim et al., 2005).

Use of a powered prosthesis decreases frontal plane knee moments, which have been associated with knee osteoarthritis, in the UL during walking on level ground compared to use of an ESAR prosthesis (Morgenroth et al., 2011; Grabowski and D'Andrea, 2013). Furthermore, use of a powered prosthesis decreased sagittal plane angular momentum on a range of slopes compared to an ESAR prosthesis (Pickle et al., 2016). We calculated sagittal plane knee joint power but did not include contributions to or changes in frontal or transverse plane powers. It is possible that the reduction in frontal plane knee moments is still observed when using the powered prosthesis to walk uphill and downhill. In future studies, we intend to investigate frontal plane joint moments when people with a unilateral transtibial amputation walk uphill and downhill using passive-elastic and powered prostheses.

We normalized all values to each subjects' mass with the respective prosthesis and there was an average 1.6 kg increase in body mass when wearing the BiOM compared to wearing their own ESAR prosthesis. While Mattes et al. (2000) found that adding mass to an ESAR prosthesis—until the total mass and inertia matched the intact limb, similar to the powered prosthesis (Herr and Grabowski, 2012)—resulted in greater gross metabolic power of approximately 21 W per 1 kg added, it is unclear how the combination of prosthetic ankle power and added distal mass relative to the residual limb change joint work contributions or individual leg work. Finally, while we attempted to match AL sagittal plane prosthetic ankle biomechanics to UL sagittal plane ankle biomechanics by adjusting tuning parameters in the BiOM powered prosthesis, prosthetic ankle range of motion, peak moment, peak power, and net work were 20–40% numerically lower on the final day of data collection than during the tuning sessions. Future studies are planned to investigate the effects of systematically varying powered ankle–foot prosthetic tuning parameters on the biomechanics of level-ground and sloped walking to determine the effects of tuning and appropriate acclimation times. Appropriate acclimation could influence the way current prostheses are tuned to an individual patient in the clinical setting and, thus, the efficacy of using a powered ankle–foot prosthesis during daily activities.

## CONCLUSION

Previous studies of people with a transtibial amputation using passive-elastic and powered prostheses have primarily focused on level-ground walking. We found that with use of the BiOM powered compared to a passive-elastic ESAR prosthesis, AL sagittal plane prosthetic ankle positive work increased for uphill slopes of +6° and +9° and prosthetic ankle net work increased for all slopes (−9° to +9°). Similarly, with use of the BiOM powered compared to a passive-elastic prosthesis, AL net work increased for uphill slopes of +6° and +9°. There were no differences in AL knee or hip positive or net work nor did unaffected joint or leg work change when using the BiOM powered compared to a passive-elastic prosthesis. Greater prosthetic ankle positive and net work through use of a powered prosthesis could improve kinematic and kinetic symmetry between the legs of people with a transtibial amputation during walking on slopes and, thus, improve preferred walking speed, metabolic cost, functional mobility, and quality of life.

## ETHICS STATEMENT

This study was carried out in accordance with the recommendations of Ethical Principles and Guidelines for the Protection of Human Subjects of Research, Colorado Multiple Institutional Review Board with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the Colorado Multiple Institutional Review Board.

## AUTHOR CONTRIBUTIONS

AG conceived of the experiment, and AG and JJ designed the experimental protocol. JJ recruited subjects, collected and analyzed data, and wrote custom code. AG analyzed data, reviewed, and edited the manuscript. All authors contributed to the writing of the manuscript.

## FUNDING

This work was supported by a Career Development Award #A7972-W from the United States (U.S.) Department of Veterans Affairs Rehabilitation Research and Development Service. The contents do not represent the views of the U.S. Department of Veterans Affairs or the United States Government. NCT01784003.

## SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at http://www.frontiersin.org/articles/10.3389/frobt.2017.00072/ full#supplementary-material.

FIGURE S1 | Affected leg (AL) and unaffected leg (UL) hip power (ensemble average) during the stance phase for subjects using a BiOM powered prosthesis (black) and their own passive-elastic energy storage and return (ESAR)

## REFERENCES


prosthesis (gray) during walking at 1.25 m/s on uphill slopes (A–C), level ground (D), and downhill slopes (E–G). Solid lines represent subjects' UL data and dashed lines represent subjects' AL data. This figure is for visualization of joint power over a stride, which we integrated to calculate joint work.

FIGURE S2 | Affected leg (AL) prosthetic and unaffected leg (UL) ankle power (ensemble average) during the stance phase for subjects using a BiOM powered prosthesis (black) and their own passive-elastic energy storage and return (ESAR) prosthesis (gray) during walking at 1.25 m/s on uphill slopes (A–C), level ground (D), and downhill slopes (E–G). Solid lines represent subjects' UL data and dashed lines represent subjects' AL data. This figure is for visualization of joint power over a stride, which we integrated to calculate joint work.

FIGURE S3 | Affected leg (AL) and unaffected leg (UL) knee power during the stance phase (ensemble average) for subjects using a BiOM powered prosthesis (black) and their own passive-elastic energy storage and return (ESAR) prosthesis (gray) during walking at 1.25 m/s on uphill slopes (A–C), level ground (D), and downhill slopes (E–G). Solid lines represent subjects' UL data and dashed lines represent subjects' AL data. This figure is for visualization of joint power over a stride, which we integrated to calculate joint work.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2017 Jeffers and Grabowski. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Case Study: A Bio-Inspired Control Algorithm for a Robotic Foot-Ankle Prosthesis Provides Adaptive Control of Level Walking and Stair Ascent

Uzma Tahir <sup>1</sup> , Anthony L. Hessel <sup>1</sup> , Eric R. Lockwood<sup>2</sup> , John T. Tester <sup>2</sup> , Zhixiu Han<sup>3</sup> , Daniel J. Rivera<sup>2</sup> , Kaitlyn L. Covey <sup>2</sup> , Thomas G. Huck <sup>1</sup> , Nicole A. Rice<sup>1</sup> and Kiisa C. Nishikawa<sup>1</sup> \*

<sup>1</sup> Center for Bioengineering Innovation and Department of Biological Sciences, Northern Arizona University, Flagstaff, AZ, United States, <sup>2</sup> Department of Mechanical Engineering, Northern Arizona University, Flagstaff, AZ, United States, <sup>3</sup> BionX Medical Technologies, Inc., Bedford, MA, United States

#### Edited by:

Jörn Malzahn, Fondazione Istituto Italiano di Technologia, Italy

#### Reviewed by:

Arturo Forner-Cordero, Universidade de São Paulo, Brazil Jana R. Jeffers, University of Colorado Boulder, United States

> \*Correspondence: Kiisa C. Nishikawa kiisa.nishikawa@nau.edu

#### Specialty section:

This article was submitted to Bionics and Biomimetics, a section of the journal Frontiers in Robotics and AI

Received: 17 November 2017 Accepted: 19 March 2018 Published: 11 April 2018

#### Citation:

Tahir U, Hessel AL, Lockwood ER, Tester JT, Han Z, Rivera DJ, Covey KL, Huck TG, Rice NA and Nishikawa KC (2018) Case Study: A Bio-Inspired Control Algorithm for a Robotic Foot-Ankle Prosthesis Provides Adaptive Control of Level Walking and Stair Ascent. Front. Robot. AI 5:36. doi: 10.3389/frobt.2018.00036 Powered ankle-foot prostheses assist users through plantarflexion during stance and dorsiflexion during swing. Provision of motor power permits faster preferred walking speeds than passive devices, but use of active motor power raises the issue of control. While several commercially available algorithms provide torque control for many intended activities and variations of terrain, control approaches typically exhibit no inherent adaptation. In contrast, muscles adapt instantaneously to changes in load without sensory feedback due to the intrinsic property that their stiffness changes with length and velocity. We previously developed a "winding filament" hypothesis (WFH) for muscle contraction that accounts for intrinsic muscle properties by incorporating the giant titin protein. The goals of this study were to develop a WFH-based control algorithm for a powered prosthesis and to test its robustness during level walking and stair ascent in a case study of two subjects with 4–5 years of experience using a powered prosthesis. In the WFH algorithm, ankle moments produced by virtual muscles are calculated based on muscle length and activation. Net ankle moment determines the current applied to the motor. Using this algorithm implemented in a BiOM T2 prosthesis, we tested subjects during level walking and stair ascent. During level walking at variable speeds, the WFH algorithm produced plantarflexion angles (range = −8 to −19◦ ) and ankle moments (range = 1 to 1.5 Nm/kg) similar to those produced by the BiOM T2 stock controller and to people with no amputation. During stair ascent, the WFH algorithm produced plantarflexion angles (range −15 to −19◦ ) that were similar to persons with no amputation and were ∼5 times larger on average at 80 steps/min than those produced by the stock controller. This case study provides proof-of-concept that, by emulating muscle properties, the WFH algorithm provides robust, adaptive control of level walking at variable speed and stair ascent with minimal sensing and no change in parameters.

Keywords: biomechanics, level walking, muscle model, powered prosthesis, preflex, stair ascent, trans-tibial amputation

## INTRODUCTION

The development of prostheses is expanding rapidly, resulting in a new generation of robotic devices that behave like the limbs they are designed to replace (Aaron et al., 2006; LeMoyne, 2016). Despite the demonstrable success of the new technologies, significant challenges remain. Compared to intact limbs, state-ofthe-art powered prostheses are limited in terms of their speed and adaptability. Foot-ankle prostheses are typically used for either walking (Herr and Grabowski, 2012) or running (McGowan et al., 2012), but not both. Adaptation to changing conditions or variation in terrain remains a significant issue (Farrell and Herr, 2011; Sinitski et al., 2012; Tkach and Hargrove, 2013; Kannape and Herr, 2014). Advances in prosthesis development have been driven largely by technology (e.g., light-weight materials, long-life batteries, programmable electronics, and wireless communication), rather than by advances in understanding of the biological principles underlying human movement.

Powered, ankle-foot prostheses have shown great promise in normalizing gait for people with a unilateral trans-tibial amputation (Aldridge et al., 2012; Sinitski et al., 2012; Agrawal et al., 2013; Gates et al., 2013; Grabowski and D'Andrea, 2013; D'Andrea et al., 2014; Esposito et al., 2014). By assisting users through powered plantarflexion during stance and dorsiflexion during swing, the BiOM T2 prosthesis normalizes metabolic costs, preferred walking speed, and ankle biomechanics (Herr and Grabowski, 2012). However, not all users benefit equally (Gardinier et al., 2017) and many challenges remain, especially for ambulation over varying terrain (Aldridge et al., 2012; Pickle et al., 2016; Russell Esposito et al., 2016).

While provision of motor power permits faster preferred walking speeds than can be produced using only passive devices (Herr and Grabowski, 2012), the use of active motor power raises the issue of control; specifically, when and how much torque assistance to provide under varying terrain conditions (Farrell and Herr, 2011; Tkach and Hargrove, 2013; Kannape and Herr, 2014). State-based control schemes typically depend on pattern recognition algorithms to select among a set of control strategies that may differ among the phases of a particular gait (e.g., stance vs. swing phases of level walking; Au et al., 2007) or among gaits associated with different terrains (e.g., level walking vs. stair ascent; Wilken et al., 2011). It is commonly presumed that, because control approaches typically exhibit no inherent adaptation to varying terrain conditions, some combination of mechanical sensing, manual actuation (e.g., Alimusaj et al., 2009), or other volitional signals (e.g., EMG; Kannape and Herr, 2014) are required to detect the need for a transition from one control strategy to another, and to deliver the appropriate torque for the new conditions (Tkach and Hargrove, 2013).

In contrast, muscles adjust their stiffness instantaneously in response to changes in load ("preflexes") without requiring input from the nervous system (Nichols and Houk, 1976; Dickinson et al., 2000; Monroy et al., 2007; Nishikawa et al., 2007, 2013). When an applied load stretches a muscle, its stiffness increases to resist overstretch. Likewise during unloading, muscles become more compliant. Muscles behave as non-linear, self-stabilizing springs (e.g., Rack and Westbury, 1974; Richardson et al., 2005), and play an important role in control of movement (Hogan, 1985), particularly in response to unexpected perturbations (Daley et al., 2009; Daley and Biewener, 2011). Although difficult or impossible to test experimentally, it seems likely that muscles contribute generally to motor control (Seiberl et al., 2013, 2015; Hessel et al., 2017).

We recently developed a novel "winding filament" hypothesis for muscle contraction (Nishikawa et al., 2012; Nishikawa, 2016), which provides a biologically plausible mechanism to account for the intrinsic adaptive properties of muscle (Monroy et al., 2007; Nishikawa et al., 2007, 2013). In the winding filament hypothesis, the engagement of the titin spring upon muscle activation provides for a mechanism by which nearly invariant muscle force output can be produced when muscles are activated at varying initial positions (Nishikawa et al., 2013). The winding of titin on the thin filaments upon activation provides for changes in muscle stiffness, not only as a function of muscle recruitment but also in response to applied length changes.

The goals of the present study were to develop a control algorithm based on the winding filament hypothesis (WFH), to implement the algorithm using the powered BiOM T2 foot-ankle prosthesis as a platform, and to test its robustness by comparing performance during level walking at variable speed and stair ascent. In computer science, robustness refers to the property that algorithms perform well not only under the conditions for which they were designed, but also under different conditions that stress the original design assumptions. For this study, we tested the robustness of WFH and BiOM T2 stock controllers by comparing their performance during level walking at variable speed, the task for which their design was optimized, vs. stair ascent, a novel condition with different biomechanical requirements for which the controllers were not explicitly optimized.

## METHODS

We first describe the WFH muscle model and the BiOM T2 prosthesis, and next describe our methods for implementing the control algorithm using the BiOM prosthesis as a platform and for subject-specific tuning of each algorithm. We then describe the methods used to compare the performance of the WFH and BiOM stock controllers during level walking and stair ascent, and lastly we describe methods for statistical analysis of data.

## Muscle Model Based on the Winding Filament Hypothesis

Previous attempts to use bio-inspired neuromuscular control approaches have been limited by the use of Hill models which fail to predict the history-dependence of muscle force (Lee et al., 2013; McGowan et al., 2013). The WFH proposes that muscle cross bridges not only translate but also rotate the thin filaments, which wind titin upon them, storing elastic energy during isometric force development (Nishikawa et al., 2012). In this way, the WFH accounts for history-dependent muscle properties including force enhancement with stretch and force depression with shortening (Nishikawa et al., 2013; Nishikawa, 2016). A muscle model (**Figure 1**) based on the WFH includes

a contractile element (CE) that represents myosin cross bridges, a damper (Cce) in parallel with the CE that represents the muscle force-velocity relationship, a pulley that represents actin filaments, and two springs representing titin (kts) and series elastic elements (kss).

for further explanation and equations.

The contractile element is a linear force generator, similar to the CE of Hill muscle models (Zajac, 1989). The CE is characterized by an active force-length relationship, fl(Xm), where X<sup>m</sup> is muscle length, based on overlap between actin and myosin filaments, and a maximum isometric force (P0) related to muscle cross-sectional area. In the model, the active forcelength relationship was measured in mouse soleus muscles and approximated using a second order polynomial (Petak, 2014). Activation of the CE results in counter-clockwise rotation of the pulley. This rotation causes the titin spring to wind on the pulley, increasing strain in the free portion of the spring (Xts). Displacement of the titin spring produces a force (Fts) that limits further counter-clockwise rotation of the pulley. When the CE deactivates, the titin force (Fts) rotates the pulley clockwise back to its initial angular position. The parallel damper (Cce) has different coefficients for lengthening (Cce\_l) and shortening (Cce\_s), a linear approximation to the lengthening and shortening sides of the force-velocity relationship (Zajac, 1989).

The undamped, linear titin spring (kts) is connected in series and in parallel to the CE via a cable wrapped around the pulley in a no-slip configuration (**Figure 1**). The force of the titin spring (Fts) is modulated by activating or deactivating the contractile element, or by applied forces that displace the pulley toward or away from the CE, which decreases or increases the strain in the titin spring, respectively (**Figure 1**). An undamped, linear spring (kss) in series with the pulley system (**Figure 1**) represents the tendon, aponeurosis, and extracellular matrix of the muscle. The series spring (kss) attaches at the axle of the pulley and is deflected by translation of the pulley but not by its rotation.

The model derivation follows Hooke's law, in which force is the product of the spring deflection (Xi) and spring constant (ki). Similarly, the force of the damper is the product of the velocity of the contractile element (x˙CE) and the directional damping constant (Cce\_l or Cce\_s). The balance of forces is calculated by superposition of the two degrees of freedom (translation and rotation) about the pulley (Petak, 2014; Lockwood, 2016). Euler's method is used to calculate the velocity of the damper and the changes in spring lengths at each time step, updated in real time at 500 Hz.

The force of the contractile element (Fce) is given by Equation (1):

$$Fe = \operatorname{act}(t) \* f(X\_m) \* P\_0 \tag{1}$$

where activation level, act(t), is an input parameter ranging from 0 to 1, whose value is specified for each stage of the gait cycle and is tuned to user preference (see section Subject-Specific Tuning). The force-length relationship, fl(Xm) equals 1 at the plateau of muscle optimal length (L0) and decreases toward 0 at shorter or longer muscle lengths (Zajac, 1989). The peak isometric force (P0) is also tuned to user preference prior to experimental walking trials.

The rotational force balance around the pulley is given by Equations (2, 3), and the translational force balance is given by Equations (4, 5):

#### Rotational force balance

$$\text{'Fe} + \text{Cce}\dot{\text{X}}ce = \text{'tsXts} \tag{2}$$

$$
\dot{\mathbf{X}}ce = \begin{array}{c} \frac{kts\mathbf{X}ts - \text{Fce}}{\text{Cce}} \end{array} \tag{3}
$$

Translational force balance

$$\text{Fce } + \text{ } \alpha \text{e } \dot{\text{X}} \text{ce } + \text{ } k \text{tsX} \text{ts} = \begin{bmatrix} \text{ks}(\text{X}m \, - \, \text{X}p) \end{bmatrix} \tag{4}$$

$$\dot{\mathbf{X}}ce \;= \frac{ks(\mathbf{X}m - \mathbf{X}p) - \mathbf{F}ce - \mathbf{k}ts\mathbf{X}ts}{\mathbf{C}ce} \tag{5}$$

where Fce is the force of the contractile element, Cce is the directional damping constant of the contractile element, x˙ce is the velocity of the contractile element, kts is the titin spring constant, Xts is the titin spring displacement; kss is the series elastic spring constant, X<sup>m</sup> is the muscle length, and X<sup>p</sup> is the pulley displacement. Given input parameters X<sup>m</sup> from the BiOM's position sensor and act(t) from subject-specific tuning (see section Subject-Specific Tuning), the velocity of the contractile element (x˙ce) is determined by substitution from the sum of rotational (Equations 2, 3) and translational (Equations 4, 5) forces acting on the pulley. The rotational and translational forces are independent and combine using superposition to yield the net force acting on the pulley (Equation 6)

$$\text{Superposition \"Xce = } \frac{\text{ktsXts } Fce}{\text{Cce}} + \frac{\begin{bmatrix} \text{kss} \begin{pmatrix} Xm \ - \ Xp \end{pmatrix} - Fce \text{ ktsXts} \end{bmatrix}}{\text{Cce}} \tag{6}$$

The derivation assumes equilibrium about the pulley at all times (t), and disregards both pulley mass and non-conservative forces (e.g., friction). The muscle model was validated using isokinetic lengthening and shortening data from mouse soleus muscles (Petak, 2014) to demonstrate that it accurately accounts for intrinsic muscle properties, including force enhancement with stretch and force depression with shortening.

## BiOM T2 Prosthesis Platform

Currently, three types of lower limb prosthetic devices are available commercially for persons with a trans-tibial amputation: passive, quasi-passive, and powered prostheses (LeMoyne, 2016). The BiOM T2 provides powered plantarflexion and dorsiflexion during variable-speed walking (Herr and Grabowski, 2012). It performs negative and positive work by employing a series-elastic actuator comprising a transverse-flux motor and ball-screw transmission in series with a carboncomposite leaf spring (Au et al., 2007; Eilenberg et al., 2010; Markowitz et al., 2011). The motor's rotary motion is converted into linear motion through the ball-screw transmission. The in-series leaf spring improves motor efficiency by storing and returning some of the energy delivered by the motor, storing energy for prosthetic ankle angles < 90◦ and becoming detached at angles > 90◦ . A carbon-composite foot at the base of the prosthesis provides additional compliance in the heel and forefoot. The mass of the prosthesis is 2 kg, designed to emulate the mass of a biological foot and partial shank of an 80 kg person (Dempster, 1955). The overall configuration is autonomous; all of the electronic components and the lithium-polymer battery that provides energy to the motor are housed within the prosthesis.

A wireless communication system (Bluetooth) allows for ankle stiffness and power delivery to be adjusted in real time while a person with an amputation walks using the prosthesis. The magnitude and timing of power delivery is calculated within the prosthesis and then adjusted for each wearer to match the performance of a biological ankle during the initial prosthesis fitting. The sensors include motor shaft and ankle joint output encoders, and a 6 degree-of-freedom inertial measurement unit (IMU) comprising three accelerometers and three rate gyroscopes. The BiOM stock controller employs a state-based approach to command ankle torques using a set of algorithms that are implemented in specific stages of the walking gait cycle (early swing, late swing, early stance, midstance, late stance). Previous studies demonstrate that the BiOM prosthesis significantly outperforms passive-elastic prostheses, and permits metabolic energy costs, preferred walking velocities and biomechanical patterns over level terrain that are similar to those of people without amputation (Herr and Grabowski, 2012).

## Implementation of the WFH Control Algorithm Using the BiOM Prosthesis Platform

The WFH control algorithm was developed in MATLAB and translated to C code to replace the portion of the commercially available BiOM stock controller that determines the motor torque applied to the prosthetic ankle joint. The WFH algorithm (**Figure 2**) includes a simplified musculo-skeletal model of the

(represented by WFH muscle models, see Figure 1) with lengths (LmA and LmP), foot moment arms (FMAA and FMAP), shank attachment lengths (SALA and SALP), and muscle attachment angles (α, β) determined from published values or calculated using geometry (see Table 1). θ = ankle joint angle. See text for further explanation.

shank (tibia/fibula) and foot, using a simple hinge to represent the ankle joint. A pair of virtual muscles provides dorsiflexion and plantarflexion torque, similar to Au et al. (2005). The tibialis anterior and its synergists were approximated as an anterior muscle group (**Figure 2**) with muscle length (LmA), shank attachment length (SALA) and foot moment arm (FMAA). The soleus, gastrocnemius, and plantaris were approximated as a single posterior muscle group (**Figure 2**) with muscle length (LmP), shank attachment length (SALP) and foot moment arm (FMAP). The algorithm thus ignores the biarticular function of the gastrocnemius muscles (Cleather et al., 2015) which attach to the distal femur (Visser et al., 1990). Each virtual muscle group is represented by a WFH muscle model (see **Figure 1**) scaled using anthropomorphic estimates of length and maximum voluntary force (see **Table 1**).

Model parameters were determined using a combination of published values, a local optimization process, simulation-based tuning, and user preference (**Table 1**). Initial values of peak


[1] Petak (2014).

[2] Hoppeler and Flück (2002).

[3] Fukunaga et al. (1992).

[4] Maganaris and Paul (1999).

[5] Arnold et al. (2010).

[6] Maganaris and Paul (1999).

[7] Baxter et al. (2012). [8] Krishnaswamy et al. (2011).

AM, anterior muscle; PM, posterior muscle. Activation parameters Act(A2-4) and Act(P6) are given for Subject 1 and Subject 2, respectively. All other parameters were the same for both subjects. Modified from Petak (2014).

isometric force (P0, **Table 1**) were based on published values of the cross-sectional area of human shank muscles (Fukunaga et al., 1992; Arnold et al., 2010), scaled by 9.5 N/cm<sup>2</sup> for peak voluntary contraction (Hoppeler and Flück, 2002). Initial parameter values for muscle and tendon lengths, foot moment arms, and attachment angles of the anterior and posterior muscles were determined using published values (**Table 1**). Maximum isometric muscle force, P0, as well as spring and damping constants were optimized using a local optimization function (Matlab FMINCON), and adjusted for best fit to the observed ankle torque data collected during walking trials using the BiOM stock controller.

The ankle joint angle was calculated in real-time using input from the BiOM's ankle angle encoder and shank geometry. The law of cosines allows calculation of virtual muscle lengths from the BiOM's ankle angle encoder. Given virtual muscle length and activation act(t) for each muscle, the WFH algorithm calculates the ankle moment of each muscle, and computes net ankle joint moment which, after compensating for mechanical resistance, is sent as a command to the BiOM motor.

The walking gait cycle of humans consists of stance (∼60%) and swing (∼40%) phases (Vaughan et al., 1990). The BiOM state detection algorithm further distinguishes early swing (state 2), late swing (state 3), early stance (state 4), late stance (state 5), and powered plantarflexion (state 6) on the basis of torque and timing. During human walking, muscle activation patterns differ depending on the stage of walking (Krishnaswamy et al., 2011). The tibialis anterior muscle, which provides ankle dorsiflexion, is activated just prior to toe-off, and remains activated throughout swing and into early stance. The posterior muscles, soleus, and gastrocnemius, are active during the stance phase of walking and silent while the foot is in the air (Lichtwark and Wilson, 2006; Krishnaswamy et al., 2011).

The WFH control algorithm uses the BiOM state machine only to provide phase-dependent activation (0–100% of maximal isometric muscle force, P0) of the anterior and posterior muscles that approximates biological muscle activation patterns. The anterior muscle group is active (∼60–90% P0) during early swing, late swing (∼30–50% P0), and early stance (∼30–50% P0) and the posterior muscle group is only active (∼40–70% P0) during powered plantar flexion. The activation levels were adjusted to user preference during tuning sessions preceding experimental trials (see section Subject-Specific Tuning).

A series of simulations was conducted to determine the sensitivity of the model to the parameter values (see **Table 1**). For a representative level walking trial (Subject 1, 1.65 m/s), the parameter values were varied systematically over a wide range (typically 0.5–250% of the optimal value). The predicted ankle moment for each parameter set was compared to the experimentally observed ankle moment during the representative trial using error analysis and the results reported as R<sup>2</sup> . The sensitivity analysis showed that the WFH algorithm predicts similar ankle torques over a wide range of parameter values. A change of ±25% in parameter values resulted in a <2% change in R 2 for all variables except PoP, Cce\_sP, and Act(P6), for which R<sup>2</sup> decreased by 11, 4, and 9% respectively.

## Subject-Specific Tuning

Two healthy male adults (age 34 and 35 years; height 181 and 184 cm; weight 82 and 109 kg), with traumatic, unilateral trans-tibial amputation 10, and 11 years prior to the study and no neuromuscular disorders or injuries, gave free informed consent to participate in this study. Both subjects had prosthetic ambulation skills for variable cadence, traversing most environmental barriers, and for vocational, therapeutic, or exercise activity that demands prosthetic use beyond simple locomotion (i.e., K3/K4 ambulation).

At the time of the study, both subjects had owned and used a BiOM T2 prosthesis in daily ambulatory activities for 4 and 5 years. Both subjects used their own socket and footplate attached to a BiOM T2 prosthesis specifically modified for this study with a softer hard-stop spring that allowed up to 2◦ of dorsiflexion and with software enabled to run both stock and WFH controllers.

The BiOM stock and WFH controller parameters were tuned for each subject in three phases during at least two tuning sessions. Subject-specific tuning of the BiOM stock controller used standard operating procedures recommended by the manufacturer and included: (1) accounting for subject mass; (2) adjusting ankle stiffness at heel strike by increasing or decreasing early stance stiffness; and (3) adjusting the power provided at slow and fast walking speeds during powered plantarflexion based on user preference. For the WFH controller, P<sup>0</sup> and activation levels (0–100%) of the anterior and posterior muscle groups (**Table 1**) during each phase of walking were determined based on subject preference. Once parameters for the stock and WFH controllers were determined for each subject during level walking, the same parameter values were used in all trials (level walking and stair ascent) for that subject.

## Testing Subjects With a Trans-Tibial Amputation

## Metabolic Cost of Transport

We measured the metabolic cost of walking with the WFH vs. stock controllers at walking speeds of 0.75, 1.2, and 1.65 m/s. Gross rates of oxygen consumption and carbon dioxide production were measured using a ParvoMedics TrueOne 2400 (Sandy, UT) metabolic cart, while subjects walked on a Woodway Desmo (Waukesha, WI) treadmill. The order of velocities tested was randomized and subjects rested for at least 5 min between trials. Steady-state metabolic power (W) from 4–6 min of each trial was estimated using a standard equation (Brockway, 1987). The metabolic power was divided by each participant's weight and speed to calculate the metabolic cost of transport (J Nm−<sup>1</sup> ).

## Inverse Dynamics vs. BiOM Torque Sensor

Walking kinematics and kinetics were quantified at three speeds (0.75, 1.25, and 1.65 m/s) for each subject using an AccuGait Optimized force-plate (Advance Medical Technology, Inc.), eight ViconTM cameras, and Nexus 2.3 motion analysis software. An infrared timing system (Brower Timing Systems, Draper, Utah, USA) was used to determine average walking speed. Trials falling ± 0.05 m/s outside of the prescribed speed were discarded. The force-plate was embedded in a 2.9 m walkway. The cameras were operated at 100 Hz, the same rate as the force plate. IR-reflecting markers were placed on the subjects at standard locations (Winter, 1990; Davis et al., 1991) to track limb position. The Nexus 2.3 lower leg plug-in gait dynamics function was used to estimate ankle torque from inverse dynamics. Anthropometric variables were measured for each subject, including body mass (kg), height (mm), left and right leg length (mm), left and right knee width (mm), and left and right ankle width (mm).

For each algorithm, ankle moments estimated using the BiOM torque sensor and inverse kinematics (derived from video and force plate data) were compared at three speeds (0.75, 1.25, and 1.65 m/s). The BiOM's state machine was used to identify strides on the prosthesis side, with each stride starting at one heel-strike event (0% gait cycle) and ending at the next heel-strike event on the prosthetic side. For level walking, 7–12 strides were analyzed for each subject and each controller at each speed (n = 55 strides for Subject 1; n = 64 strides for Subject 2).

## Level Walking at Variable Speed

The BiOM's sensors, validated using inverse dynamics (see section Inverse Dynamics vs. BiOM Torque Sensor below) were used to estimate peak ankle moment (Nm/kg), plantarflexion angle during stance (degrees), and peak ankle power (W/kg) in the sagittal plane during level walking at three speeds (0.75, 1.25, and 1.65 m/s) and during stair ascent. Plantarflexion was defined as a negative ankle angle and dorsiflexion as a positive angle, where 0◦ represents the neutral position in which the foot is ∼perpendicular to the shank. Due to noise in the time stamps for the encoded data from the BiOM sensors and variation in the duration of each stride, the raw data were interpolated to obtain 100 data points for each stride based on total stride duration. The interpolated data were used to calculate ankle angular velocity and ankle power.

## Stair Ascent

Subjects were asked to ascend stairs in a step-over-step manner at self-selected speed and at 80 steps per minute. During stair ascent, subjects were asked to land on each step with the ball of the foot, as they naturally do with the intact limb (Kannape and Herr, 2014). To normalize the self-selected speed, subjects were given the prompt "walk upstairs at a comfortable, safe pace that would allow you to maintain a conversation with someone walking with you." Each subject took 2–6 level ground steps with the prosthesis and then transitioned to the first step with their intact limb. After ascending four stairs on the prosthesis side, the subjects continued walking on level ground for another two strides on the prosthesis side and then ascended four more stairs. A total of 11–62 steps was analyzed for each subject, speed and control algorithm (n = 179 steps for Subject 1; n = 96 steps for Subject 2).

To compare the robustness of stock and WFH algorithms, we defined robustness operationally as maintenance or improvement of prosthesis performance on stairs vs. level walking relative to average values for control subjects with no amputation performing the same tasks as published in Aldridge et al. (2012). For example, the average plantarflexion angle for Subject 1 using the WFH control algorithm was −11.6◦ during level walking at 0.75 m/s and it increased to −15.9◦ during stair ascent at 80 steps/min, so the algorithm is robust because the average plantarflexion angle of control subjects with no amputation ascending stairs at 80 msteps/min was −14.7◦ .

## Statistics

Statistical comparisons were performed using JMP Pro 13 (SAS Institute, Inc.). BiOM stock and WFH controllers were compared separately for each subjects. Peak ground reaction force was compared using two-way ANOVA (α = 0.05) with controller (BiOM stock vs. WFH), walking speed (0.75, 1.25, 1.65 m/s), and controller x speed as the main effects. One-way ANOVA (α = 0.05) was used to compare ankle moment estimates from the BiOM's torque sensor to the estimates based on inverse kinematics within each combination of controller and walking speed. For level walking, peak ankle moment (Nm/kg), plantarflexion angle (degrees), and ankle power (W/kg) were compared using two-way ANOVA (α = 0.05), with controller, walking speed and controller x speed as the main effects. For stair ascent, peak ankle moment (Nm/kg), plantarflexion angle (degrees), and ankle power (W/kg) were compared separately at each speed (self-selected vs. 80 steps/min) using one-way ANOVA (α = 0.05) with controller as the main effect. To compare robustness of stock and WFH algorithms, we used twoway ANOVA with controller (stock vs. WFH), condition (level walking vs. stair ascent) and controller × condition as main effects. The analysis was performed separately for slow (0.75 m/s level walking vs. 80 steps/min stair ascent) and medium speeds (1.25 m/s level walking vs. self-selected speed for stair ascent).

For parametric analyses, assumptions of normality were evaluated using Shapiro-Wilk tests within each combination of controller and speed. Equality of variances was evaluated using Levene tests for normally distributed data and Brown-Forsythe tests for non-normal data. Each data set was tested for normality and all comparisons between controllers for each speed and subject were tested for equality of variances before and after best Box-Cox transformations. For each subject, a total of 9 dependent variables (including vertical ground reaction force; peak ankle moment from BiOM torque encoder vs. inverse dynamics; and sagittal plane peak ankle moment, ankle plantarflexion angle, and ankle power during level walking and stair ascent) were measured for each controller (BiOM stock and WFH) and walking speed (two for stairs and three for level walking), for a total of 48 data sets per subject. Due to persistent violations of normality and homoscedasticity even after transformation (see results), between-controller comparisons were also tested using non-parametric Steel-Dwass tests.

## RESULTS

For Subject 1, 15 of 48 data sets failed the normality test and 10 of 32 comparisons failed the equal variances test after transformation. For Subject 2, 12 of 48 data sets failed the normality test and 11 of 32 comparisons failed the equal variance tests after transformation. ANOVA results are presented in **Tables 2**–**4**, and results from the more conservative Steel-Dwass tests are indicated by asterisks in **Figures 3**–**6**. Where ANOVA and non-parametric tests differed, we report the results of the more conservative non-parametric tests.

## Ground Reaction Force

The BiOM stock and WFH controllers produced similar vertical ground reaction forces (GRF) for both subjects during level walking at all speeds (**Figure 3A**). For both subjects, peak GRF increased with walking speed (two-way ANOVA, both P < 0.0001, **Table 2**). The subjects differed in effects of the controllers on peak GRF. For Subject 1, peak GRF was 6.7%



Results of two-way ANOVA after Box—Cox transformation for stock and WFH controllers during level walking at three speeds (0.75, 1.25, and 1.65 m/s). See Figure 3 for means ± SEM. <sup>a</sup>Not normally distributed after Box–Cox transformation;bVariances not equal after Box–Cox transformation.


TABLE 3 | Peak sagittal plane ankle moment, plantarflexion angle, and ankle power for two subjects during level walking at variable speed.

Results of two-way ANOVA after Box–Cox transformation for stock and WFH controllers during level walking at three speeds (0.75, 1.25, and 1.65 m/s). See Figure 5 for means ± SEM. <sup>a</sup>Not normally distributed; <sup>b</sup>Variances not equal.

TABLE 4 | Peak sagittal plane ankle moment (Nm/kg), plantarflexion angle (degrees), and ankle power (W/kg) for two subjects while ascending stairs.


Results of one-way ANOVA after Box–Cox transformation for stock vs. WFH controllers during stair ascent at self-selected speed and 80 steps/min. See Figure 6 for means ± SEM. <sup>a</sup>Not normally distributed; <sup>b</sup>Variances not equal.

higher on average for the stock controller at the two faster walking speeds (**Figure 3B**, Steel-Dwass tests, P < 0.0198). For Subject 2, peak GRF was 8.7% higher for the WFH controller at the lowest speed (**Figure 3B**; Steel-Dwass test, P = 0.0065).

## Metabolic Cost of Transport

The metabolic cost of transport for the two control algorithms differed between subjects. Subject 1 had the lowest cost of transport at all speeds using the stock controller (**Figure 4**). The WFH-controller performed nearly as well as the stock controller at 0.75 m/s but walking was less efficient with the WFH controller than with the stock controller at the faster speeds. Subject 2 had the lowest cost of transport when using the WFH-controller at the slow and intermediate speeds, but the stock controller had the lowest cost of transport at the fastest speed (**Figure 4**).

## Inverse Dynamics vs. BiOM Torque Sensor

The accuracy of the BiOM torque sensor was assessed by comparing the peak ankle moment estimated by the sensor to the peak ankle moment estimated from inverse dynamics within each combination of control algorithm (stock vs. WFH) and speed (0.75, 1.25, and 1.65 m/s) for each subject (12 comparisons total). In only one of 12 ANOVA comparisons was there a difference in peak ankle moment between estimates from the BiOM's sensors and inverse dynamics after Box-Cox transformation (Subject 1, 0.75 m/s; p = 0.0041). In this case, the BiOM torque sensor underestimated the peak torque by ∼8% relative to the estimate from inverse dynamics. Because these results suggest that the BiOM's sensor provide a reliable measure of ankle moment, we used data from the BiOM sensors for subsequent comparisons of algorithm performance.

## Level Walking at Variable Speed

During level walking, maximum forces of the virtual muscletendons unit (MTU) were ∼800 N for the anterior muscle model and ∼1200 N for the posterior muscle model, which are within the range of peak shank muscle forces in observed in human studies (Arnold et al., 2010). Ankle moment profiles were similar for the BiOM stock and WFH control algorithms at all speeds for both subjects (**Figure 5A**). Peak ankle moment increased significantly with walking speed for both controllers (**Figure 5B**, two-way ANOVA, p < 0.0001, **Table 3**). Subject 1 achieved a 3.6% higher peak ankle moment during walking using the BiOM stock controller at the intermediate speed (**Figure 5B**, Steel-Dwass test, P = 0.0036), while Subject 2 achieved a 13.2% higher peak ankle moment using the WFH algorithm at all speeds (**Figure 5B**; Steel-Dwass tests, P < 0.017).

For Subject 1, peak plantarflexion angle achieved during stance (**Figures 5C,D**) was 38% larger on average for the stock algorithm than for the WFH algorithm at all three speeds (**Figure 5D**, **Table 3**; Steel-Dwass tests, P < 0.0004). For Subject 2 (**Figure 5D**), the plantarflexion angle was 20% smaller for the WFH algorithm than the stock algorithm at the slowest speed (Steel-Dwass test, P = 0.0001), similar for both controllers at the intermediate speed, and 10% larger for the stock controller at the fastest speed (Steel-Dwass test, P = 0.0004). Ankle power (**Figures 5E,F**) increased with speed for both algorithms (twoway ANOVA, p < 0.0001; **Table 3**) and was significantly larger for the stock algorithm (average 44% for Subject 1 and 29% for Subject 2) for both subjects at all speeds (**Figure 5F**, Steel-Dwass tests, P ≤ 0.0092). Both subjects reported a preference for the stock controller during level walking, which was not surprising based on their extensive previous experience using the BiOM prosthesis with the stock controller.

## Stair Ascent

Using the same parameters as for level walking, ankle moment profiles were again similar for the stock and WFH controllers

transport (J/Nm) for two subjects at three walking speeds (0.75, 1.2, and 1.65 m/s). Broken lines = Subject 1; solid lines = Subject 2.

when ascending stairs (**Figure 6A**). There was no consistent pattern of variation in peak ankle moment (**Figure 6B**, **Table 4**) between stock and WFH controllers. For Subject 1, the stock controller had an 12.4% higher peak moment at the self-selected speed (**Figure 6B**, Steel-Dwass test, P < 0.0001), and the WFH controller had a 3.7% higher peak moment at 80 steps/min (**Figure 6B**, Steel-Dwass tests, P ≤ 0.0007). For Subject 2, there was no difference between controllers at the self-selected speed, and the stock controller had a 146% higher peak ankle moment at 80 steps/min (**Figure 6B**, Steel-Dwass tests, P = 0.0001).

For both subjects at self-selected speed and 80 steps/min, the plantarflexion angle (**Figures 6C,D**) was significantly greater for the WFH controller than for the stock controller (**Figure 6D**, Steel-Dwass tests, P < 0.0001). The WFH controller increased ankle angle by 35 and 20.2% at self-selected walking speed and by 383 and 1193% at 80 steps/min for Subject 1 and Subject 2, respectively. For both subjects and both speeds, ankle power (**Figures 6E,F**) was greater for the WFH controller than for the stock controller (**Figure 6F**, Steel-Dwass tests, P < 0.0093), increasing by 0.7 and 39% at self-selected speed and by 255 and 435% at 80 steps/min for Subject 1 and Subject 2, respectively.

The mechanics of level walking and stair ascent are markedly different. In contrast to level walking, stair ascent involves two cycles controlled dorsiflexion and powered plantarflexion; the first cycle pulls the center of mass up from the previous stair, and the second cycle pushes the center of mass up to the next stair (Wilken et al., 2011; Aldridge et al., 2012). Both the BiOM stock and WFH algorithms produced relatively smooth transitions from level steps to stair steps and back to level steps (**Figure 7**. Pull-up moments in early stance were similar for both algorithms (**Figures 7A,C**), but push-off moments later in stance were higher relative to pull-up moments for the stock (**Figure 7C**) compared to WFH controller (**Figure 7A**). Plantarflexion angles were also smaller and more variable for the stock (**Figure 7D**) compared to the WFH (**Figure 7B**) controller, especially during the transitions from level to stairs and vice versa. The WFH controller produced reliable moments and plantarflexion angles during level to stairs transitions and vice versa with minimal sensing (i.e., ankle angle input only) and no change in parameters.

Both subjects expressed a strong preference for the WFH algorithm when ascending stairs. One subject reported that the

FIGURE 5 | Ankle kinematics and kinetics during level walking at variable speed using WFH (gray) and BiOM stock (black) controllers. (A) Average ankle moment (Nm/kg) ± SEM vs. % gait cycle. (B) Peak ankle moment (Nm/kg) ± SEM for two subjects at three walking speeds. (C) Average plantarflexion angle (degrees) ± SEM vs. % gait cycle. (D) Peak plantarflexion angle (degrees) ± SEM for two subjects at three walking speeds. (E) Average ankle power (W/kg) ± SEM vs. % gait cycle. (F) Peak ankle power (W/kg) ± SEM for two subjects at three walking speeds. Data in (A,C,E) are from for Subject 1 walking at 1.25 m/s. Asterisks (\*) represent statistical differences (P < 0.05) using Steel-Dwass non-parametric tests.

WFH controller appeared to compensate for the BiOM's weight, so he did not feel that he had to carry the prosthesis up the stairs using his own muscles. Both subjects reported that the WFH algorithm allowed them to ascend stairs in a more natural step-over-step manner than the commercially available stock controller which they had more experience with using.

stair ascent and back to level walking at self-selected speed using the WFH (gray, A,B) and stock (black, C,D) controllers. Eight consecutive strides are shown in each figure. The subject first takes two strides on level ground, then ascends four stairs on the prosthetic side, and finally takes two level strides at the top of the stairs. For the WFH controller, there is no change in muscle activation or other parameters during transitions from level walking to stair ascent and back to level walking.

## Robustness of the WFH Algorithm

To illustrate the robustness of the WFH algorithm, we evaluated the behavior of the anterior and posterior muscle models during level walking at 1.25 m/s and stair ascent at self-selected speed (**Figure 8**). The reciprocal lengths of the anterior and posterior muscles are determined strictly by the ankle angle input (**Figure 8A**) as a function of their moment arms and tendon stiffness. During level walking (solid line), the virtual anterior muscle is stretched during late stance (**Figure 8A**) and shortens rapidly during early swing, as observed in previous studies during normal walking (Lichtwark and Wilson, 2006). The posterior muscle shows a reciprocal pattern, stretching during swing and shortening during stance to power plantarflexion. During stair ascent, the peak plantarflexion angle was larger and occurred later in the gait cycle than during level walking, as also observed in previous studies (Lichtwark and Wilson, 2006; Spanjaard et al., 2007).

In the WFH algorithm, the net ankle moments produced by the muscle models are the sum of: (1) the contractile element forces (Fce) determined by the activation parameters; (2) spring forces determined by a combination of activation and muscle length (XmA, XmP); and (3) damping forces determined by the velocity of the contractile element (x˙CE). During level walking, the anterior muscle (**Figure 8B**) produces a dorsiflexion moment during swing, and the posterior muscle (**Figure 8C**) produces a plantarflexion moment during stance. The net ankle moment (**Figure 8D**), the sum of anterior and posterior muscle moments, is the command sent to the motor.

The difference in net moment produced by the WFH algorithm during level walking and stair ascent illustrates the algorithm's robustness. The only difference between level walking and stair ascent was the ankle angle input (**Figure 8A**). There was no change in virtual muscle activation or any other parameters. Yet, the plantarflexion moment produced by the posterior muscle (**Figure 8C**) rose earlier during stance and was larger during late stance during stair ascent than during level walking. The net torque (**Figure 8D**) also rose earlier during stance producing a double peak (**Figure 8D**, dashed line) in net ankle moment typical of normal stair ascent (Sinitski et al., 2012) in contrast to the single peak produced during level walking (**Figure 8D**, solid line). By emulating muscle intrinsic response to length changes (provided as input via the BiOM ankle angle sensor), the algorithm provides robust control of ankle moment during stair ascent without requiring a change in parameters.

Statistical analysis demonstrated the difference in robustness between WFH and stock controllers in the two subjects who participated in this case study. For this analysis, peak ankle moment, maximum plantarflexion angle, and peak ankle power were compared between the controllers at two speeds (slow and medium) for each subject, for a total of 12 two-way ANOVA

tests. A controller was considered to be robust if the change in average value of the dependent variable (e.g., peak ankle moment) either stayed the same during stair ascent or became closer to the average value of that variable observed in control subjects with no amputation performing the same task, published in Aldridge et al. (2012). Of 12 comparisons, both controllers showed robust behavior from level walking to stairs for peak ankle power at medium speed (both subjects) and for peak ankle torque in Subject 2 at medium speed. Neither controller showed robustness for peak ankle torque of Subject 1 at both speeds. For all other comparisons (7/12), the WFH algorithm showed robust performance during stair ascent whereas the stock controller did not. Parameter values of the stock controller moved significantly away from values for control subjects for maximum plantarflexion angle of both subjects at both speeds, for peak ankle torque of Subject 2 at the slower speed, and for peak ankle power of both subjects at the slower speed (two-way ANOVA, post-hoc t-tests, all P < 0.05).

## DISCUSSION

Although this case study was limited to only two subjects, it represents one of the few studies that includes subjects with extensive experience (4–5 years) using a powered prosthesis for daily activities. Despite the similar history of the two subjects in terms of the time since amputation and experience using the BiOM T2, there were differences between subjects in performance between the stock and WFH controllers during level walking. Subject 1 had a lower cost of transport, larger peak ankle moment, and larger plantarflexion angle during stance when using the stock controller than when using the WFH controller. The reverse was true for Subject 2, for whom the cost of transport was lower, peak ankle moment was larger, and plantarflexion angle during stance was larger at slow and intermediate speeds when using the WFH controller than when using the stock controller. For level walking, the BiOM stock controller produced higher ankle power for both subjects at all speeds. Given their extensive experience using the stock controller, we expected a larger difference in performance between controllers than was observed.

During stair ascent, the WFH controller increased both plantarflexion angle and ankle power for both subjects at the self-selected speed and at 80 steps/min. In a previous study of stair ascent using the BiOM T2 prosthesis, Aldridge et al. (2012) demonstrated that the BiOM T2 prosthesis improved ankle plantarflexion angle by ∼10◦ relative to an elastic storage and return prosthesis at both 80 steps/min (−4.5◦ vs. +5.8◦ ) and self-selected speed (−4.9◦ vs. +5.8◦ ). However, even with the BiOM T2 prosthesis, the plantarflexion angle was significantly lower compared to controls with no amputation (−14.7◦ at 80 steps/min and −15.2◦ at self-selected speed). In the present study, both subjects achieved ankle plantarflexion angles averaging from −15.1 to −16.7◦ at self-selected speed and −16.2 to −19.5◦ at 80 steps/min when using the WFH controller, equivalent to those of control subjects in the previous study (Aldridge et al., 2012). In contrast in the present study, the stock controller produced significantly smaller ankle plantarflexion angles averaging −11.2 to −13.9◦ at the self-selected speed and −0.9 to −3.4 at 80 steps/min.

The experienced BiOM users in the present study had larger plantarflexion angles during stair ascent at the self-selected speed (−11.2 to −13.9◦ ) than subjects in Aldridge's et al. (2012) study (−4.9◦ vs. +5.8◦ ), who had acclimated to the BiOM for only ∼43days on average. This difference could be due to the greater experience of the subjects in the present study, to the modified hard-stop spring which allows greater dorsiflexion than the commercially available BiOM T2 prosthesis, or both.

Although more ankle power is required to accelerate the center of mass during stair ascent than during level walking (Wilken et al., 2011), it appears that maintaining ankle kinematics is also important for stair ascent. Ankle plantarflexion plays an important role in transitioning off the trailing limb onto the leading limb, and a decrease in ankle plantarflexion requires increased hip extension to raise the center of mass (Aldridge et al., 2012). Although Aldridge et al. (2012) reported no difference in ankle plantarflexor moment or ankle power between the BiOM T2 prosthetic limb and controls with no amputation, the intact limb generated more ankle power than control subjects, and use of the asymmetric "hip strategy," typically used by people with a unilateral trans-tibial amputation while ascending stairs (Powers et al., 1997; Alimusaj et al., 2009), was not reduced. An important question for future work is whether, by increasing both plantarflexion angle and ankle power during stair ascent compared to the stock controller, the WFH controller can reduce or eliminate use of the "hip strategy," which increases both gait asymmetry and muscular effort. Increased plantarflexion and ankle power when climbing stairs, as well as smooth transitions between different terrains, may provide a significant improvement in quality of life and cardiovascular fitness for persons with an amputation (Sansam et al., 2009; Sagawa et al., 2011).

The ability of the WFH controller to produce walking at variable speed and stair ascent with minimal real-time sensing (only ankle angle) and no change in virtual muscle activation or other parameters is a significant achievement due to the fundamental differences in gait when ambulating these terrains (Wilken et al., 2011). While the phases of level walking from heel-strike to heel-strike include controlled plantarflexion, controlled dorsiflexion, powered plantarflexion, and swing (Au et al., 2008), the phase transitions during stair ascent include an additional pair of controlled dorsiflexion and powered plantarflexion phases before the swing phase. The first pair of controlled dorsiflexion and powered plantarflexion phases pull the center of mass up from the previous stair, whereas the second pair pushes the center of mass up to the next stair (Wilken et al., 2011; Aldridge et al., 2012). Subjects using the WFH algorithm transitioned smoothly from level walking to stair ascent and vice versa using the same set of equations and parameter values. Ankle kinematics and kinetics were much more variable for both subjects during transitions from level to stairs and back when using the stock controller. The adaptation of ankle torque assistance provided by the WFH controller during gait transitions depends only upon the different ankle angle input, which in turn represents the effects of external forces applied to the virtual muscles at the ankle joint (see **Figure 8**).

Although many previous studies presume that different operational modes are required for ambulation in different terrains (Tkach and Hargrove, 2013), the present study provides proof of concept that a controller based on muscle intrinsic properties can provide adaptive torque control using the same set of equations and parameters across different gaits and terrains. In principle, this is the same control strategy that animals and humans use during unexpected perturbations when muscles instantaneously adjust their force and stiffness in response to applied length changes long before reflex feedback can modify muscle activation (Daley et al., 2009; Daley and Biewener, 2011). By demonstrating the sufficiency of muscle intrinsic properties for control of level walking and stair ascent, the results suggest the likelihood that these properties may contribute to control of voluntary human movements.

## LIMITATIONS

One major limitation of this study is the small sample size of two subjects. Although many previous studies have compared the BiOM T2 prosthesis to passive and quasi-passive prostheses and to control subjects with no amputation, during both level walking (Herr and Grabowski, 2012) and stair ascent (Aldridge et al., 2012; Pickle et al., 2014), a much larger sample is needed to test the repeatability of the results from this small case study. Future studies are also needed to assess whether the WFH algorithm can improve the kinematics and kinetics not only of the affected limb but also the unaffected limb during level walking and stair ascent, as well as regulation of whole body angular momentum.

There are also some limitations associated with the design of the WFH control algorithm that could be addressed in future studies. The first is use of square-waves to simulate activation of the virtual muscles during the different phases of the gait cycle. The sensitivity analysis showed that square-wave activation of the virtual muscles at specific phases during the gait cycle results in fluctuation of the damping forces when muscle activation changes abruptly. These fluctuations in simulated muscle force reduce dorsiflexion during swing at the stance-swing transition, and also increase plantarflexion moment in early stance, which may decrease efficiency and performance of the WFH controller during level walking. Future work should include development of activation strategies that reduce discontinuities in muscle activation across the gait cycle. In the long term, a control strategy for virtual muscle activation that eliminates the requirement for state-based control and provides volitional control, such as EMG from the residual limb, would also likely improve performance and energy efficiency.

## CONCLUSION

The results of this case study of two experienced BiOM users provide proof-of-concept that a WFH control algorithm based on muscle intrinsic properties can produce ankle kinematics and kinetics during level walking at variable speed and stair ascent that are similar to those produced by the BiOM's stock control algorithm and by people with no amputation. The robust WFH controller transitions from level walking to stairs and vice versa with no change in muscle activation or other parameters, and without requiring information about the user's intended activity. Future work should address optimization of algorithm performance and assessment of its impact on the kinematics and kinetics of the ankle, knee, and hip on affected and unaffected sides, as well as whole-body biomechanics.

## ETHICS STATEMENT

This study was carried out in accordance with the recommendations of Ethical Principles and Guidelines for the Protection of Human Subjects in Research and Northern Arizona University's Institutional Review Board for the Protection of Human Subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the Institutional Review Board for the Protection of Human Subjects at Northern Arizona University.

Animal studies were carried out in accordance with the recommendations of the United States Department of Agriculture (USDA) Animal Welfare Act and Regulations (AWA), the Guide for the Care and Use of Laboratory

## REFERENCES


Animals, Public Health Services Policy on Humane Care and Use of Laboratory Animals, Occupational Safety and Health Administration (OSHA), and Environmental Protection Agency (EPA) regulations, Northern Arizona University's Institutional Animal Care and Use Committee policies and procedures. The research was approved by Northern Arizona University's Institutional Animal Care and Use Committee.

## AUTHOR CONTRIBUTIONS

The experiments were designed by KN, ZH, and JT. UT, AH, and ZH: recruited subjects. UT, EL, AH, KC, and DR: improved the experimental design, performed the experiments, and collected data. UT, EL, KC, NR, and DR: analyzed data using custom code. UT, NR, and TH: prepared the figures. UT, JT, and KN: prepared the first draft of the manuscript. All authors reviewed and edited the manuscript.

## FUNDING

This work was supported in part by awards from the W. M. Keck Foundation, the National Science Foundation (IOS-0732949, IOS-1025806, IOS-1456868; IIP-1237878, and IIP-1521231), the Technology Research Initiative Fund of Northern Arizona University, and the Achievement Rewards for College Scientists Foundation (AH).

## ACKNOWLEDGMENTS

The authors wish to thank Dr. Kathleen Ganley for use of the force-plate and motion capture system and Dr. Pamela Bosch for use of the treadmill and metabolic cart at the NAU—Phoenix Biomedical Campus. Kirsten Lopez, Nathanial Johnson, Shelby Martin, and Isaac Romero assisted in data collection and analysis. Drs. Zachary Lerner and Stan Lindstedt provided constructive comments on earlier versions of this manuscript.


during walking at different speeds? Clin. Orthop. Relat. Res. 472, 3044–3054. doi: 10.1007/s11999-014-3647-


LeMoyne, R. (2016). Advances for Prosthetic Technology. Tokyo: Springer.


performance during fast SSCs of human m. adductor pollicis. Physiol. Rep. 3:e12401. doi: 10.14814/phy2.12401


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Tahir, Hessel, Lockwood, Tester, Han, Rivera, Covey, Huck, Rice and Nishikawa. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.