Abstract
Humans can adapt to abruptly changing situations by coordinating redundant components, even in bipedality. Conventional adaptability has been reproduced by various computational approaches, such as optimal control, neural oscillator, and reinforcement learning; however, the adaptability in bipedal locomotion necessary for biological and social activities, such as unpredicted direction change in chase-and-escape, is unknown due to the dynamically unstable multi-link closed-loop system. Here we propose a switching adaptation model for performing bipedal locomotion by improving autonomous distributed control, where autonomous actuators interact without central control and switch the roles for propulsion, balancing, and leg swing. Our switching mobility model achieved direction change at any time using only three actuators, although it showed higher motor costs than comparable models without direction change. Our method of evaluating such adaptation at any time should be utilized as a prerequisite for understanding universal motor control. The proposed algorithm may simply explain and predict the adaptation mechanism in human bipedality to coordinate the actuator functions within and between limbs.
Introduction
We can adaptively operate our bipedal body by cooperating with others in an emergency (Hutchins, ; Fujii et al., ) and sometimes competing with others (Yamamoto et al., ; Fujii et al., ). Current technology can succeed in reproducing such real-time adaptation in video game tasks (Mnih et al., ) and overcoming unpredicted deficits (Yoshihara et al., ; Cully et al., ), although such adaptation is limited to a certain part of the agent's body. However, with regard to bipedal locomotion, which is more dynamically unstable than that of more than four-legged species (Golubitsky et al., ), researchers have not paid attention to the adaptive movements when motor commands change suddenly in response to a change in the situation, such as chase-and-escape behavior (Kamimura and Ohira, ; Fujii et al., ), which have been acquired in over the course of evolution as biological (Carvalho et al., ) and social (Helbing et al., ) features essential for life activities. For example, it is considered that a sudden intentional direction change opposite to the original direction, such as in interpersonal sports (Fujii et al., ), is quite difficult to achieve and thus has been ignored in the fields of robotic engineering (Koolen et al., ; Kuindersma et al., ) and computational neuroscience (Taga et al., ; Song and Geyer, ), with the focus primarily being placed on bipedal adaptation to external disturbances. Here, we refer to this as switching adaptation in bipedal locomotion because both motor commands (i.e., situation or task) and motor system requirements will switch in this case.
Although most previous studies on human motor control were based on the optimal control theory (Todorov and Jordan, ; Scott, ), which is considered to be physiologically related to the cerebellum (Shadmehr and Krakauer, ), this theory cannot necessarily apply to universal motor control. The theory focuses on optimizing the system based on the centralized invariant cost functions, such as the deviation of target trajectory (Uno et al., ) or motor cost as muscle activity (Anderson and Pandy, ), such as in arm movement. However, an unstable multi-link closed-loop system with large inertia and a narrow base of support in abruptly changing situations, such as switching adaptation in bipedal locomotion, is difficult to control optimally. This is because it cannot determine the optimal target trajectory due to the large control component with physiological constraints (e.g., joints and muscles), the observation component with cognitive constraints (e.g., ground and opponent) and the context (e.g., the predicted optimal strategy could be defeated by the opponent's counter-attack; Fujii et al., ). Thus, switching adaptation in bipedal locomotion, which is difficult to control even in current robotics (Koolen et al., ; Kuindersma et al., ), is an excellent example to shed more light on the mystery of universal motor control.
Human bipedality, which is considered to be the result of adaptations to environmental variabilities (Carvalho et al., ), is one of the controversial problems to control. While the efficiency of bipedal locomotion in the optimal control theory (Srinivasan and Ruina, ) was explained by the dynamics only in the ground phase, neural oscillator control (Taga et al., ) that is physiologically located in the spinal central pattern generator (Grillner, ; Dimitrijevic et al., ) can reproduce the whole of aperiodic adaptive bipedal locomotion in a self-organized manner rather than explicitly calculating the target trajectory or joint torques. However, the oscillator system is considered to be limited in cyclic movement with adaptation only to external disturbances (Thelen et al., ; Taga et al., ). For example, active adaptation to a changing situation will result in excessive deviation from the aperiodic locomotion generated by the oscillator (e.g., in the opposite direction) because the motor command itself changes drastically. In recent years, using a physiological reflex model, the diversity of walking including a direction change of 50° was reproduced (Song and Geyer, ), but in situations such as escape or pursuit, robustly faster direction change at any time (Fujii et al., ) is needed. Furthermore, it is unknown which factors make such adaptive bipedal locomotion difficult because previous locomotion models (Taga et al., ; Song and Geyer, ) including multiple neural oscillators, peripheral reflexes and multi-link body dynamics were implemented in a complicated manner, whereas as far as the passive walk, the previous model simply accomplished it (McGeer, ). Therefore, as a prerequisite for such adaptability, it is important to examine a minimal control model that achieves direction change at any time in the opposite direction with a small number of components and a simple algorithm, and to establish a methodology for evaluating it.
Distributed autonomous control, in which autonomous components implicitly function as a whole by interacting with each other without central control, such as in multi-agent (Couzin et al., ) or multi-link (Watanabe et al., ) biological systems, is applicable to real-time adaptation to the rapid impairment of components (Yoshihara et al., ). This control system is biologically plausible than explicit simulation because the system can perform self-modeling (Bongard et al., ) to adapt to the situation beyond its framework. The differences and advantages of the distributed autonomous control compared with the neural oscillator control are that the local components autonomously set the local target and have flexibility in the rule-based interaction among components. Among the autonomous system, self-repairing robots (Bongard et al., ; Cully et al., ) are remarkable, but the switching adaptation task in this study requires more improvisational adaptation (e.g., within 1 s). The mobility control (Yoshihara et al., ) based on the design of autonomous systems, in which an autonomous mobile component moves prior to an immobile component, can execute arm reaching movement when confronting a real-time deficit of the component with improvisational adaptation. We thus assumed that mobility control can be a key factor in the switching adaptability with a minimal algorithm due to the real-time adaptability without the explicit control of the components. However, in bipedal locomotion, in addition to the control of the center of mass in locomotion (equal to endpoint control in arm movement), balance and leg swing control are necessary and often conflict, so not only the operation of equivalent rules for each component but also the switching of rules according to the situation should be important.
In this paper, we adopted switching autonomous system, which extended (i.e., incomplete) distributed autonomous control scheme, because the current task can be accomplished by solving multiple conflicting functions. For example, it would be more advantageous for multiple actuators to switch roles to maintain balance by the leading leg and to move the center of mass by the trailing leg (Yamashita et al., ). In neurophysiology, this mechanism may be related to postural control in the reticulospinal tracts found in cats (Mori et al., ), but its mechanisms of interaction and switching the function of actuators (i.e., muscles) have remained unknown. We therefore implemented an adaptive bipedal model into role-switching for propulsion, balance, and leg swing control with switching mobility control. The objective of this study is to propose a new control algorithm and evaluation methodology of a switching adaptive model for performing bipedal locomotion as a prerequisite for universal motor control.
Materials and methods
Model overview
In this study, we constructed a three-mass model as a toy model (i.e., a minimally redundant model) of a sidestep locomotor system (Figure 1A). The three masses were linked with three actuators, springs, and dampers that represent the legs and inter-leg (i = 1–3: inter-leg, right leg and left leg, respectively). For simplicity of the spatially symmetrical configuration of three actuators, inter-leg actuator 1 was modeled as a hip abductor and adductor muscles to swing the legs. Passive parameters are partly based on the human-like model in a previous study (Taga et al., ), as shown in Table S1. In this model, the segments were stretchable, but if a leg exceeded a certain length (1.1 times its initial length), we increased the elastic coefficients (Table S1). We also increased the leg elasticity in the foot contact phase compared with that in the flight phase (Table S1).
Figure 1
The model can perform 2D lateral locomotion by sending appropriate commands to the actuators (Figure 1A) according to the following equation of motion:
where x is a position vector of the three mass points, Fai is an active force vector generated from the three actuators, g is a gravity acceleration vector, and Fpassive is a passive force vector including viscoelasticity of the leg, its extension limit, and auxiliary action in the trunk (Table S1). The last auxiliary viscoelasticity prevents falling if the horizontal distance between the trunk and either leg is within 0.15 m. This value of 0.15 m is heuristically determined based on the trade-off of falling and propulsion in observation. We improved distributed autonomous control (Figures 1B–D), which is based on the rule that the velocity commands are determined from the instantaneous “mobility” of each actuator in real time. This rule will be given in an autonomous decentralized form, which is explained in the paragraphs below.
Switching mobility control
For switching mobility control, here we consider the velocity command for actuator i. Command using positional information is not appropriate in this study because the calculation of the precise target trajectory is not needed. Velocity sensing and command may be reasonable such as due to the utilization of visual optical flow in a self-driven agent. It is assumed that the sensory (i.e., proprioceptive) information of the system including actuator lengths and angles and these derivative values was used. In this section, we consider the following two steps to construct the model: (i) First, the mobility index was defined as the difference between each local actuator's desired velocity () and the actual velocity vi. (ii) Based on the mobility index, global desired velocity vd was allocated preferentially to mobile actuators and control input in each actuator was determined.
In the first step, actuator i divides vd into two components: a local vector and a residual vector . The former is the component of vd that actuator i could generate through its own stretching and shortening and the latter is the component that actuator i is incapable of generating in the current leg posture (Figure 1B):
exi is a unit vector to produce the force in actuator i. exi in the inter-leg is defined as the unit vector from the trailing leg to the leading leg. Switching coefficient ai is basically 1 for propulsion but switches for balancing and leg swing based on the related segment sensory information (the schematics are shown in Figure S1). A notable difference from a previous robot arm study (Yoshihara et al., ) is that the local desired velocity is modified by coefficient ai in the situations because of a temporal constraint to apply the force to the ground. When i is 2 or 3 (i.e., right or left leg) in the flight phase (Figure S1 right), ai was set to 1/2 because the leg cannot apply force to the ground and the contribution of the leg to trunk movement halved (Figure 1C: , Figure 1D: ). Additionally, when the trunk approaches the anterior leg within 0.2 m, ai was set to −1 to prevent falling, regardless of being in the flight or supported phase (Figure 1C: ). When the posterior leg in the flight phase extended over its natural length, ai was also set to −1 to attract the leg to the trunk as the swing for the next step (Figure 1C: ). These two corrections reflect the balance and leg swing, those conflict with propulsion, respectively.
When i is 1 (i.e., inter-leg: Figure S1 left), ai depended on the phase of both legs. In the double support phase, ai was set to 0 because of a lack of contribution to trunk velocity. In the double flight phase, ai was set to 1/2 in the same manner for both legs. Additionally, when the posterior leg length was over 0.6 times the natural length in the double flight phase or the anterior leg support phase, ai was set to −1 to attract the posterior leg to the trunk. This value of 0.6 was heuristically determined based on the following observation: if it is too large, the model sometimes cannot perform the leg swing, and if it is too small, it cannot move in the desired direction. Because of the dependence on kinematic sensory information of other segments, this system is not purely autonomous. However, this switching system contributed to achieving the task by resolving the trade-off between propulsion and balance.
For the adaptation under various environmental conditions, the mobility measure ki must evaluate the instantaneous mobility of each joint appropriately, which requires calculation of kinematic and dynamic mobilities (Yoshihara et al., ). The kinematic mobility is the ability of actuator i to move the trunk, and represented by the absolute value of the local desired velocity, i.e., geometric state of the actuator. Dynamic mobility is the same ability as determined by the dynamic properties of the actuator, which is represented by the difference between the actual state and the geometric state of the actuator. The mobility measure ki is defined as the corrected ratio of dynamic mobility to kinematic mobility:
where vi is the velocity produced at the trunk generated by actuator i, and ε1 and ε2 are small values (ε1: 10−10, ε2: 10−4) to avoid dividing by zero. The denominator and numerator are related to the kinematic and dynamic mobility of actuator i, respectively. The mobility ki is supposed to take a value of 0 in an immobile actuator, and 1 in a mobile actuator.
Next, by using ki, , and (Equations 2–4), we intended to design a real-time controller that would make the most mobile actuator work dominantly, and make the other actuators work cooperatively in order to satisfy vd. Actuator i basically tries to move according to its own local vector , and require the other actuators to create its residual vector . The required velocity from actuator j to actuator i, , is defined as a projection of to exi:
The mobility ki of each actuator then determines how the actuators interact with each other. We express this as follows:
is the velocity command for actuator i. The first term functions as an inhibitory interaction from actuator j, which prevents actuator i from moving according to its own local velocity . In contrast, the second term functions as an excitatory interaction make actuator i work cooperatively and generate the residual velocity of actuator j. In this study, we considered the minimal model which fully connected among three components (i.e., the nearest neighbors equal to the full connections). In more biological model, note that the number of the connection will increase and we should examine the connection configuration, such as based on the nearest neighbors. The velocity command, , is transformed to torque as follows:
where Gi is the proportional gain of actuator i. We heuristically set it to 3,000 kg/s to perform the task.
Simulation and statistics
Initial horizontal and vertical positions of the trunk were set as 0 and 0.92 m, respectively. Initially, three masses were kept motionless in an equilateral triangular posture and double support stance. The time step in the simulation was set to 10−5 s. To examine the parameter sensitivity, we ideally should use the human parameter for the verification. Although we used the skeletal parameters based on the human parameters (Taga et al., ), the passive joint viscoelasticity should be approximated when we expressed it as a scalar value (Taga et al., ), because muscle-tendon complex has many components with various viscoelastic properties. Based on the assumption that we can learn the (sub) optimal parameters in various motor tasks, we approximated the passive joint viscoelasticity with the invariant scalar value. Instead, we examined the following three parameters from as broad a range as possible: the elasticity of legs, the elasticity of ground, and the proportional gain of actuators. We simulated the models to multiply each parameter by 10−1, 10−1/2, 1, 101/2, and 10. The details and the results were given by Figure S3. In short, in the case of multiplying 10−1/2 or 101/2 by the each original parameter, the model had sensitivity enough to accomplish the direction change at any time.
To quantify the switching adaptability performance, the reaching time was calculated as the time interval from the direction change command to the movement at 2 m displacement after the direction change. To compare the switching mobility model with the conventional models, we reproduced the two model simulations in forward walking models (Taga et al., ; Song and Geyer, ). In the neural oscillator model13, we used the cited parameters, motion equations and set the constant input as 6. The time step in the simulation was set to 10−6 s. In the reflex control model15, we used the freely available MATLAB code, and set the type of model to normal walk and the time step of extraction to 10−3 s. Other parameters were set as the defaults, including 1.3 m/s as the initial horizontal moving velocity.
For bivariate correlations, we used Pearson's correlation coefficient. For comparing the reaction time between during two different phases, we used the unpaired t-test. Both statistics are described with the corresponding degrees of freedom (denoted by a subscript). For all the statistical calculations, p < 0.05 was considered significant. All simulations and statistical analyses were performed using MATLAB 2016a Statistics and Machine Learning Toolbox (The MathWorks, Inc., MA, USA).
Results
Bipedal locomotion with switching mobility control
We first set the target speed to 2 m/s and simulated straightforward locomotion without direction change (Video S1). Figure 2A shows the time series of the target and actual velocities of the trunk mass. Similar to actual human bipedal locomotion (Bruijn et al., ), the target speed was not always achieved because the acceleration of the body can occur only at the moment when the foot is grounded. The trunk velocity is mostly obtained at the time of contact of the trailing leg (Figures 2A,B) because the leading (right) leg was not designed to overtake the trailing leg in the sidestep (Yamashita et al., ). In particular, in this model, alternate grounding of the leading and the trailing feet did not always occur (Figure 2B, Video S1). Thus, the locomotion of our model was neither strictly walking nor running. As is the case for skipping with the repeated same foot contact (Minetti, ) and galloping with both the double support and flight phase (Yamashita et al., ), in the model, the same foot sometimes repeatedly contacted and showed a double support and flight phase. Our model temporally changed the four gaits characteristics, and thus cannot be categorized. The longer term (60 s) characteristics are shown in Figure S2. The mobility index of the three actuators (Figure 2C) alternately increased and decreased to play their roles as determined by switching coefficients (Figure 2D). Mobility seemed to increase in the flight phase in both leg actuators (2 and 3) and at the phase with either leg grounded in inter-leg actuator 1 (Figure 2C). The switching coefficients seemed to switch appropriately to propulsion (Figure 2D red, e.g., grounded in actuators 2 and 3), balance (blue, e.g., grounded in actuators 2 and 1; in flight in actuator 3), and swing (orange or light blue, in flight in actuator 2 or 3) separately in each actuator (Figure 2D: the algorithm is given in Section Materials and Methods).
Figure 2
Direction change at various timings
During the sidestep in Figure 2, we switched the target speed to −2 m/s at various timings and moved the trunk to 2 m in the opposite direction from that moment. The vertical axis in Figure 2E shows the reaching time toward 2 m after the direction change command with a 0.03-s interval (horizontal axis). First, the switching mobility model achieved direction change at any time using only the three actuators. The reaching time and its variation are the model performance (mean and standard deviation of reaching time: 2.734 ± 0.567 s) and a new evaluation method of the switching adaptation in bipedal locomotion. The results showed the reaching time increased during the trailing leg stance compared with the other timings (2.940 ± 0.644 s vs. 2.604 ± 0.474 s, t99 = 3.01, p = 0.0033). Figure 3 shows examples of the faster direction change (reaching time: 1.478 s) after 1.861 s from the start of the simulation (Figure 3B, Video S2) and in the delayed direction change (reaching time: 4.280 s) after 2.161 s (Figure 3B, Video S3). The faster trial involved a change in direction to switch the mobility index and the switching coefficient and to include fewer steps in a shorter cycle compared with the slower trial. As a coarse grained explanation at the direction change timing, the trailing leg stance increased the reaching time because the trailing leg propelled the body (before the direction change) and then will make the body difficult to change the inverse direction.
Figure 3
As fundamental kinematic characteristics to investigate the fine-grained fluctuation of the reaction time, we examined the relationship of step numbers and foot height with the direction change performance (Figure 4). The reaching time was significantly increased with a greater number of steps for both leading and trailing feet (Figure 4A, leading: r99 = 0.542, p = 4.7 × 10−9, trailing: r99 = 0.509, p = 5.4 × 10−8). It was also significantly increased with maximum foot height (Figure 4B) for the leading foot (r99 = 0.283, p = 4.1 × 10−3), but not that for the trailing foot (r99 = 0.03, p = 0.976). These results suggest that the faster direction change was derived from the movement with less motor cost, estimated by fewer steps and a smaller leading foot height. However, an underlying cause of difference between the trials in the faster and the slower reaching time was difficult to explain directly because the behaviors were interrelated and generated from the closed-loop structure. This may be generated from a subtle dynamic state difference and the subsequent accumulation of integration error in the non-integrable system.
Figure 4
Comparison with conventional models
To reveal the difference in motor output in the different architectures, we reproduced two previous forward walk models with a neural oscillator (Taga et al., ) and reflex control (Song and Geyer, ; details are given in Section Materials and Methods). Note that because there is no sidestep bipedal model, except for ours, detailed comparisons between the proposed and previous models are impossible (furthermore, these models have parameter sensitivity, so we cannot match the locomotion velocities). We thus focused on the fundamental locomotion characteristics which differed greatly beyond the mere specific parameters of the models. We first compared the amplitude and variance of the inter-step interval for both legs with a large number of steps (90 stable steps from the 11th to the 100th step) to quantify the stability in locomotion (Figures 5A–C). The switching mobility model had much larger variance in step interval, despite the lower horizontal moving velocity (leading leg: 0.298 ± 0.247 s, trailing leg: 0.257 ± 0.190 s at a mean velocity of 0.575 m/s) compared with the forward walking models (neural oscillator model: 1.119 ± 0.001 s at a mean velocity of 1.544 m/s, reflex model: 1.229 ± 0.004 s at 1.200 m/s; in both models, the intervals for the left and right legs were the same). The step interval variance in the switching mobility model was also much larger than the measured human sidestep walking data (Yamashita et al., ; ~1.0 ± 0.1 s at 1.3 m/s). In addition, from a visual analysis, in the switching mobility model, sidestepping was performed in a less efficient way with higher foot raising (Figure 5D) or a repeated grounded phase in the same leg (Figure 2C, Video S1) at the expense of switching adaptability. We calculated the peak foot height frequency (Figures 5D–F) and showed that the leading leg was sometimes raised higher than in the comparable models, in which the foot height was strictly controlled, especially in the computational reflex model (Song and Geyer, ).
Figure 5
Discussion
In this study, we constructed a minimal distributed autonomous model achieving bipedal change in direction at any time with only three actuators, but without accurate features representing the whole human body structure, such as a central pattern generator and a lower limb joint, as reproduced in the previous comparable models (Taga et al., ; Song and Geyer, ), that is, without explicit optimization and cyclic movement in neural oscillators. Simultaneously, we also developed a new method for evaluating the ability to perform robustly faster direction change during bipedal locomotion (Figure 2E). In previous studies of adaptive bipedal locomotion in robotic engineering or neurophysiology, such as when encountering an obstacle in the environment or an external disturbance (Taga et al., ; Song and Geyer, ; Koolen et al., ; Kuindersma et al., ), it was not considered whether the bipedal model can achieve direction change at any time, which is not explicitly implemented (i.e., unpredictable for the model). Thus, to discuss this control problem, it is necessary to reconstruct the frameworks from the viewpoints of engineering control and neurophysiology, as explained below. The expansion of the frameworks should be necessary for understanding universal motor control.
From the viewpoint of engineering control, the switching mobility model showed switching adaptability at the expense of efficiency because it is difficult for the bipedal locomotion model to satisfy the criteria of both efficiency and adaptability. Previous research using an inverted pendulum locomotion model (Srinivasan and Ruina, ) explained the energy efficiency in various locomotion patterns based on the optimal control theory. With regard to efficiency, the switching mobility model showed highly costly movement in which a period of repeated grounding of the same leg occurred (Figure 3), in contrast to that in the conventional models (Taga et al., ; Song and Geyer, ), even without a change in direction. Furthermore, at the time of changing direction, it sometimes takes a long time to adapt to the command to change direction (Figure 3B) because of a greater number of steps (Figure 4A) and a higher leading leg raise (Figure 4B). This leg raise was controlled in the neural oscillator model (Taga et al., ) and strictly computed in the reflex model (Song and Geyer, ) at a lower level than in the switching mobility model without direction change (Figure 5). In adaptive bipedal locomotion, it is difficult to calculate the optimal trajectory, so we confirmed the higher motor cost in the switching mobility model, estimated from the much larger variance in step interval and much higher leg raise, than in the conventional models. This is also because there is no mathematical guarantee of efficient movement in mobility control (Yoshihara et al., ), which is different from the explicit optimal control or neural oscillator control in cyclic movement. Instead, switching mobility control in the switching mobility model would have an advantage regarding adaptability, even if the movement switches from a cyclic to a non-cyclic pattern and vice versa and has difficulty in control, such as large instability as in bipedal chase-and-escape in interactive sports (Fujii et al., ,). We were not able to perform a direct comparison with these control algorithms because there are no models to show the switching adaptability (and performing a sidestep); however, a future model can be compared by the proposed evaluation method in which the model performs direction change at various timings.
Neurophysiologically, our switching mobility control algorithm suggests the presence of reflex-like switching functions of propulsion, balancing, and leg swing within and between limbs to achieve the task. The algorithm does not directly reflect the neural mechanism, but we can consider similarity with human neurophysiology by a process of elimination. The proposed model does not explicitly control the actuator movements like cerebellum (Shadmehr and Krakauer, ) and does not directly generate quasiperiodic movements such as using neural oscillators (Taga et al., ) physiologically located in spinal central pattern generators (Grillner, ; Dimitrijevic et al., ). Instead, the previous work suggests that diverse movements can be reproduced by incorporating the multiple reflexes in the spinal feedback circuitry without the central pattern generator (Song and Geyer, ). Among these architectures, we can find the similarity between the reflex control and the autonomous distributed control (Yoshihara et al., ; Watanabe et al., ), which incorporates interaction among components and environments (i.e., feedback of ground reaction force) and can implicitly adapt to a rapid change of command. In other words, the point at which each actuator autonomously sets and executes a target movement according to the situation in the switching mobility model matches that in the physiological reflex mechanism in which the inputs and outputs are locally automated. Our switching distributed autonomous control algorithm provided the relationship among the actuators (i.e., muscles) to switch functions of propulsion, balancing, and leg swing within and between limb levels. In a manner similar to long-latency reflexes, which would possess an internal model of limb dynamics (Kurtzer et al., ), the actuator in the switching mobility model rapidly switched functions according to the situation. There is no physiological evidence from the neural circuitry of a response of such inter-limb reflexes to a drastic change of command in the long neural pathway, but we believe that it would be needed for the switching adaptation in bipedal locomotion. The neural mechanism involved would be complicated because of possible involvement with both voluntary and reflex control overlapping in their neural substrates (Kurtzer et al., ); however, it may be simply explained by the simple interaction rule of our switching mobility control. This constructive approach can contribute to understanding intelligent motor control including biologically (Carvalho et al., ) and socially (Helbing et al., ) essential activities, such as escape from enemies, pursuit of prey, and search for food.
However, there are some problems with the above neurophysiological claims. One is that it claims to be based only on the similarity in the architectures without neurophysiological evidences. This is considered as a general problem in finding evidences of long-latency reflexes, which overlapped with voluntary movements in their neural substrates (Kurtzer et al., ). Second is validation with real-world human data satisfying both sidestepping (Yamashita et al., ) and unpredicted change direction (Fujii et al., ). The latter study showed that the unpredicted competitive situation (i.e., requiring the faster movement) delayed the first step initiation more than 100 ms, suggesting the human can decrease the delay more than the current switching adaptability model. The real human locomotion mechanism includes central pattern generators, peripheral reflexes, and spring-damper system. The spring-damper property, which contributes high speed locomotion according to a quadruped robot study (Kimura et al., ), should be further investigated.
Third is the sensitivity of the simulation to the choice of some of the model parameters. Our supplementary results (Figure S3) showed that the switching adaptability model had strong sensitivity to the parameters. We suppose the parameterization may be related with inherent adaptation to the individual musculoskeletal system and might be relatively independent of the motor control adaptation. As a further alternative approach, for example, evolutionary algorithm (Song and Geyer, ) and reinforcement learning (Lillicrap et al., ) efficiently worked in previous bipedal locomotion studies. These are complementary relationship in terms of inter- and intra-generation progress, respectively. The reflex model (Song and Geyer, ) optimized the control parameters with the covariance matrix adaptation evolution strategy (Hansen, ). Reinforcement learning enables the acquisition of efficient and adaptive locomotion by trial and error on models, like humans actually do, probably in the basal ganglia (Doya, ). However, if learning a case of sudden change through evolutionary algorithm and reinforcement learning (also in learning of humans as an experimental condition; Fujii et al., ), there is a possibility that it will not mean “unpredictable sudden change” when performing an adaptive bipedal locomotion (Shinya et al., ). In other words, it can be considered that the explicit control rule should be difficult to estimate in principle in these approaches (i.e., human bipedal locomotion can also predictively adapt to the situation; Shinya et al., ). The evolutionary algorithm and reinforcement learning will also have the advantage to acquire the efficient movement in our less efficient model. Satisfying the requirements of efficiency (e.g., obtained explicitly by using optimal control and implicitly by evolutionary algorithm or reinforcement learning) and switching adaptability proposed by our study will further contribute to solving the general motor control problem.
Statements
Author contributions
KF conceived the original idea of the model. KF and YYo designed the model. KF, YYo, HT, and YYa analyzed data and wrote the paper.
Funding
This work was supported by a Grant-in-Aid for JSPS fellows 26-407 and Exploratory Research 16K12995. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Acknowledgments
The authors would like to thank M. Yano of Tohoku University, S. Hagio of Tokyo University and D. Yamashita of Japan Institute of Sports Sciences for useful discussions and S. Song of Carnegie Mellon University for his walking model.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Supplementary material
The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fnhum.2017.00298/full#supplementary-material
References
1
AndersonF. C.PandyM. G. (2003). Individual muscle contributions to support in normal walking. Gait Posture17, 159–169. 10.1016/S0966-6362(02)00073-5
2
BongardJ.ZykovV.LipsonH. (2006). Resilient machines through continuous self-modeling. Science314, 1118–1121. 10.1126/science.1133687
3
BruijnS. M.MeijerO. G.BeekP. J.van DieënJ. H. (2013). Assessing the stability of human locomotion: a review of current measures. J. R. Soc. Interface10:20120999. 10.1098/rsif.2012.0999
4
CarvalhoS.BiroD.CunhaE.HockingsK.McGrewW. C.RichmondB. G.et al. (2012). Chimpanzee carrying behaviour and the origins of human bipedality. Curr. Biol.22, R180–R181. 10.1016/j.cub.2012.01.052
5
CouzinI. D.KrauseJ.JamesR.RuxtonG. D.FranksN. R. (2002). Collective memory and spatial sorting in animal groups. J. Theor. Biol.218, 1–11. 10.1006/jtbi.2002.3065
6
CullyA.CluneJ.TaraporeD.MouretJ.-B. (2015). Robots that can adapt like animals. Nature521, 503–507. 10.1038/nature14422
7
DimitrijevicM. R.GerasimenkoY.PinterM. M. (1998). Evidence for a spinal central pattern generator in humans. Ann. N. Y. Acad. Sci.860, 360–376.
8
DoyaK. (2000). Complementary roles of basal ganglia and cerebellum in learning and motor control. Curr. Opin. Neurobiol.10, 732–739. 10.1016/S0959-4388(00)00153-7
9
FujiiK.DaichiY.TetsuyaK.IsakaT.KouzakiM. (2015a). Preparatory body state before reacting to an opponent: short-term joint torque fluctuation in real-time competitive sports. PLoS ONE10:e0128571. 10.1371/journal.pone.0128571
10
FujiiK.IsakaT.KouzakiM.YamamotoY. (2015b). Mutual and asynchronous anticipation and action in sports as globally competitive and locally coordinative dynamics. Sci. Rep.5:16140. 10.1038/srep16140
11
FujiiK.YokoyamaK.KoyamaT.RikukawaA.YamadaH.YamamotoY. (2016). Resilient help to switch and overlap hierarchical subsystems in a small human group. Sci. Rep.6:23911. 10.1038/srep23911
12
FujiiK.YoshiokaS.IsakaT.KouzakiM. (2013). Unweighted state as a sidestep preparation improve the initiation and reaching performance for basketball players. J. Electromyogr. Kinesiol.23, 1467–1473. 10.1016/j.jelekin.2013.08.001
13
FujiiK.YoshiokaS.IsakaT.KouzakiM. (2015c). The preparatory state of ground reaction forces in defending against a dribbler in a basketball 1-on-1 dribble subphase. Sports Biomech.14, 28–44. 10.1080/14763141.2015.1026931
14
GolubitskyM.StewartI.BuonoP. L.CollinsJ. J. (1999). Symmetry in locomotor central pattern generators and animal gaits. Nature401, 693–695. 10.1038/44416
15
GrillnerS. (1985). Neurobiological bases of rhythmic motor acts in vertebrates. Science228, 143–148. 10.1126/science.3975635
16
HansenN. (2006). The CMA evolution strategy: a comparing review, in Towards a New Evolutionary Computation, eds LozanoJ. A.LarrañagaP.InzaI.BengoetxeaE. (Berlin; Heidelberg: Springer), 75–102.
17
HelbingD.FarkasI.VicsekT. (2000). Simulating dynamical features of escape panic. Nature407, 487–490. 10.1038/35035023
18
HutchinsE. (1995). Cognition in the Wild. Cambridge, MA: MIT Press.
19
KamimuraA.OhiraT. (2010). Group chase and escape. New J. Phys.12:053013. 10.1088/1367-2630/12/5/053013
20
KimuraH.FukuokaY.CohenA. H. (2007). Biologically inspired adaptive walking of a quadruped robot. Philos. Trans. R. Soc. Lond. A Math. Phys. Eng. Sci.365, 153–170. 10.1098/rsta.2006.1919
21
KoolenT.BertrandS.ThomasG.De BoerT.WuT. F.SmithJ.et al. (2016). Design of a momentum-based control framework and application to the humanoid robot atlas. Int. J. Hum. Rob.13:1650007. 10.1142/S0219843616500079
22
KuindersmaS.DeitsR.FallonM.ValenzuelaA.DaiH. K.PermenterF.et al. (2016). Optimization-based locomotion planning, estimation, and control design for the atlas humanoid robot. Auton. Robots40, 429–455. 10.1007/s10514-015-9479-3
23
KurtzerI. L.PruszynskiJ. A.ScottS. H. (2008). Long-latency reflexes of the human arm reflect an internal model of limb dynamics. Curr. Biol.18, 449–453. 10.1016/j.cub.2008.02.053
24
LillicrapT. P.HuntJ. J.PritzelA.HeessN.ErezT.TassaY.et al. (2015). Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971.
25
McGeerT. (1990). Passive dynamic walking. Int. J. Rob. Res.9, 62–82. 10.1177/027836499000900206
26
MinettiA. E. (1998). The biomechanics of skipping gaits: a third locomotion paradigm?Proc. Biol. Sci.265, 1227–1235. 10.1098/rspb.1998.0424
27
MnihV.KavukcuogluK.SilverD.RusuA. A.VenessJ.BellemareM. G.et al. (2015). Human-level control through deep reinforcement learning. Nature518, 529–533. 10.1038/nature14236
28
MoriS.MatsuiT.KuzeB.AsanomeM.NakajimaK.MatsuyamaK. (1998). Cerebellar-induced locomotion: reticulospinal control of spinal rhythm generating mechanism in cats. Ann. N. Y. Acad. Sci. 860, 94–105. 10.1111/j.1749-6632.1998.tb09041.x
29
ScottS. H. (2004). Optimal feedback control and the neural basis of volitional motor control. Nat. Rev. Neurosci.5, 534–546. 10.1038/nrn1427
30
ShadmehrR.KrakauerJ. W. (2008). A computational neuroanatomy for motor control. Exp. Brain Res.185, 359–381. 10.1007/s00221-008-1280-5
31
ShinyaM.FujiiS.OdaS. (2009). Corrective postural responses evoked by completely unexpected loss of ground support during human walking. Gait Posture29, 483–487. 10.1016/j.gaitpost.2008.11.009
32
SongS.GeyerH. (2015). A neural circuitry that emphasizes spinal feedback generates diverse behaviours of human locomotion. J. Physiol. Lond.593, 3493–3511. 10.1113/JP270228
33
SrinivasanM.RuinaA. (2006). Computer optimization of a minimal biped model discovers walking and running. Nature439, 72–75. 10.1038/nature04113
34
TagaG.YamaguchiY.ShimizuH. (1991). Self-organized control of bipedal locomotion by neural oscillators in unpredictable environment. Biol. Cybern.65, 147–159. 10.1007/BF00198086
35
ThelenE.UlrichB. D.NilesD. (1987). Bilateral coordination in human infants - stepping on a split-belt treadmill. J. Exp. Psychol. Hum. Percept. Perform.13, 405–410. 10.1037/0096-1523.13.3.405
36
TodorovE.JordanM. I. (2002). Optimal feedback control as a theory of motor coordination. Nat. Neurosci.5, 1226–1235. 10.1038/nn963
37
UnoY.KawatoM.SuzukiR. (1989). Formation and control of optimal trajectory in human multijoint arm movement - minimum torque-change model. Biol. Cybern.61, 89–101. 10.1007/BF00204593
38
WatanabeW.KanoT.SuzukiS.IshiguroA. (2012). A decentralized control scheme for orchestrating versatile arm movements in ophiuroid omnidirectional locomotion. J. R. Soc. Interface9, 102–109. 10.1098/rsif.2011.0317
39
YamamotoY.YokoyamaK.OkumuraM.KijimaA.KadotaK.GoharaK. (2013). Joint action syntax in Japanese martial arts. PLoS ONE8:e72436. 10.1371/journal.pone.0072436
40
YamashitaD.ShinyaM.FujiiK.OdaS.KouzakiM. (2013). Walk-, run- and gallop-like gait patterns in human sideways locomotion. J. Electromyogr. Kinesiol.23, 1480–1484. 10.1016/j.jelekin.2013.08.005
41
YoshiharaY.TomitaN.MakinoY.YanoM. (2007). Autonomous control of reaching movement by “mobility measure”. Int. J. Rob. Mech.19, 448–458. 10.20965/jrm.2007.p0448
Summary
Keywords
sensory-motor system, multi-link system, closed-loop system, autonomous distributed control, flexible bipedal locomotion
Citation
Fujii K, Yoshihara Y, Tanabe H and Yamamoto Y (2017) Switching Adaptability in Human-Inspired Sidesteps: A Minimal Model. Front. Hum. Neurosci. 11:298. doi: 10.3389/fnhum.2017.00298
Received
15 March 2017
Accepted
22 May 2017
Published
07 June 2017
Volume
11 - 2017
Edited by
Mikhail Lebedev, Duke University, United States
Reviewed by
Rahul Goel, University of Houston, United States; Dai Owaki, Tohoku University, Japan; Kazuki Nakada, Hiroshima City University, Japan
Updates
Copyright
© 2017 Fujii, Yoshihara, Tanabe and Yamamoto.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Keisuke Fujii keisuke198619@gmail.com
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.