Adaptive Interlimb Coordination Mechanism for Hexapod Locomotion Based on Active Load Sensing

1. Introduction

Insects exhibit versatile interlimb coordination patterns to move around adaptively. For example, some insects possess various gait patterns (e.g., wave gait, tetrapod gait, and tripod gait) that change in response to changes in locomotion speed and loads applied to the body (Wilson, 1966; Dean, 1991; Zollikofer, 1994; Wosnitza et al., 2013). Furthermore, they can generate feasible locomotor patterns in response to leg amputation (Hughes, 1957; Delcomyn, 1991; Grabowska et al., 2012). In addition to steady walking, they flexibly change limb motion between walking and searching steps when they walk on uneven terrain with some gaps in the foothold (Pearson and Franklin, 1984; Theunissen and Dürr, 2013; Theunissen et al., 2014, 2015). While insects exhibit long stride steps during steady walking (i.e., walking) on uneven terrain, they exhibit short searching steps where the limb repeats a retracting and protracting motion in a short stride distance to obtain secure footholds. Understanding these flexible interlimb coordination mechanisms underlying insect adaptive walking sheds new light on developing adaptive legged robots that can move around stably through rough environments (e.g., disaster sites).

Biological and modeling studies have investigated decentralized control mechanisms underlying adaptive insect locomotion through comparative studies focusing on distinct insect species, e.g., stick insects and cockroaches (Ayali et al., 2015a). Stick insects (e.g., Phasmida) can climb unpredictable environments, and their relatively slow locomotor patterns allow researchers to address underlying sensory-motor mechanisms. Biological studies have elucidated that thoracic neural circuits generate rhythmic locomotor patterns neither sensory input from the leg nor the descending command from the brain (Mantziaris et al., 2020). Furthermore, sensory input in the limbs contributes to modifying intra- and interlimb coordination for adaptive stick insect locomotion. In contrast to stick insects, cockroaches (e.g., Blattaria) exhibit fast and stable locomotion and are ideal insects to address the interaction between neural control and body dynamics. While their conservative tripod gait patterns are generated by CPG, their flexible body can negotiate uneven terrains (Full et al., 1998; Watson et al., 2002; Weihmann et al., 2017). Furthermore, recent studies elucidate the sensory feedback mechanism underlying cockroach‘s locomotion in which signals from mechanoreceptors modulate muscle contractions to establish interlimb coordination (Ayali et al., 2015b; Weihmann et al., 2017). Although the mathematical models for insect‘s interlimb coordination have been developed differently depending on the focusing insect animals (e.g., stick insects and cockroaches), the common distributed control mechanisms in distinct insect animals have induced to unify them into common limb coordination model (Koditschek et al., 2004; Büschges et al., 2008; Daun-Gruhn, 2011; Toth et al., 2013). However, these unified models are too complex to analyze and apply to legged robots in simple manners.

In contrast to complex models describing the insect‘s sensory-motor system with large numbers of differential equations, redacted models significantly help us test hypotheses and interpret the substantial interlimb coordination mechanism underlying insect locomotion (Kimura et al., 1993; Dürr et al., 2004; Kukillaya et al., 2009; Owaki et al., 2017). The simple models reduced dimensions by using simple elementary processes, e.g., phase oscillators and reflexes, to generate interlimb coordination. For example, Cruse et al. proposed a series of reflex rules based on the behaviors of stick insects (Dürr et al., 2004). They predicted the pathway of sensory-motor modulation for interlimb coordination in the insect animal's thoracic nervous system. For another example, Owaki et al. proposed a simple CPG model where one phase oscillator controls each limb's stride motion and demonstrated that phase modulations depending on loads of limbs contribute to generating various gaits locomotion speed and leg amputation (Owaki et al., 2017). Regarding adaption to uneven environments, however, previous models still require recruiting a large number of neural components for modulating interlimb coordination depending on situations (Durr, 2001; Bläsing, 2006; Schilling et al., 2013a,b; Ngamkajornwiwat et al., 2020). This is because the limb without a stable foothold should adaptively change its foot trajectory and frequency comparing other limbs to search steady footholds. Therefore, the development of a simple interlimb coordination mechanism involving searching behavior will contribute to deeply understanding the essential mechanism underlying flexible insect locomotion.

To this end, this study develops a simple interlimb coordination model to extract substantial mechanisms underlying adaptive hexapod locomotion, including searching behavior on uneven terrain. We hypothesize that a simple local sensory feedback mechanism, “active load sensing,” plays an essential role in generating flexible hexapod interlimb coordination patterns in flat and uneven environments. In this scheme, each limb actively moves and detects the reaction force from the ground to judge whether it plays a pivotal role in maintaining the steady support polygon. As a result of the simulation experiments, a hexapod robot that could generate flexible gait patterns in response to locomotor speed and leg amputation was developed. Furthermore, the robot flexibly changed its limb behaviors between the walking step in steady walking and the searching step depending on the lack of the foothold. During particular limb searching, other limbs flexibly modulate their interlimb coordination through the same mechanism in walking in flat terrain. These results suggest that a simple decentralized control mechanism exploiting physical interaction between body and environment (e.g., the proposed active load sensing) allows insects to generate flexible interlimb coordination for flat terrain and unpredictable environments.

The remainder of this paper is organized as follows: Section 2 exploits the proposed simple interlimb coordination mechanism; Section 3 presents the results of the 3D simulation; Sections 4 and 5 present the discussion and conclusion.

2. Model

According to insect behaviors, the insects adaptively generate long limb strides for walking and short limb strides for searching. Besides, the periods of one limb stride locally and drastically change during a pass through uneven terrains. Therefore, modeling based on phase oscillators is required to discontinuously modulate the phase (e.g., phase reset) and also modulate limb trajectories, resulting in a complex interlimb coordination mechanism. To develop a simple model, this study employs two feedforward limb control modes and four fundamental transition rules that induce a hexapod robot to generate walking and searching behaviors. In the following modeling section, we first explain a robot model in the simulation environments. Then, we illustrated two basic limb control modes “swing mode” and “stance mode” and fundamental transition rules.

Regarding the mechanical structure, a robot consists of six identical limb units and a rigid trunk unit, as shown in Figure 1. Each limb has three degrees of freedom: joint α connects the trunk unit and limb unit and generates the protraction and retraction motion by changing the angle of the joint ${\theta }_{i,j}^{\alpha }$ (i = R, L and j = 1, 2, 3). The other two joints β and γ generate flexing/extending and elevating motions by changing the angles of the joints ${\theta }_{i,j}^{\beta }$ and ${\theta }_{i,j}^{\gamma }$.

FIGURE 1

Figure 1. The mechanical structure of the hexapod robot. Each limb has three degrees of freedom. Joint α generates retracting and protracting motions. Joints β and γ generate elevating motions. Each limb has a controller to generate limb motion in a decentralized manner.

Regarding the basic components of a limb controller, each limb has a controller with two control modes, that is, stance modes and swing mode to generate limb stride motions, as shown in Figure 2A. The controller state is described with the symbol M_i,j. When M_i,j = Swing mode, the limb controller is in the swing mode, and the limb generates protracting motion. Furthermore, the proposed model has two stance modes: early stance mode and late stance mode. In both stance modes, the limb generates retracting motion for kicking the ground.

FIGURE 2

Figure 2. Overview of an interlimb coordination mechanism for hexapod locomotion. (A) Two control modes(swing mode and stance mode) and transition conditions. (B) Schematics of threshold values in limb trajectory. (C) Neural connectivity between the limbs in the proposed model.

In the proposed robot mode, we simplify the coordination between the joints (i.e., intralimb coordination) to realize a specific foot trajectory. In all control modes, the joint α is controlled to achieve the target joint angular velocity ${\overline{\stackrel{.}{\theta }}}_{i,j}^{\alpha }$. In the swing mode (M_i,j = Swing mode), ${\overline{\stackrel{.}{\theta }}}_{i,j}^{\alpha }$ is set to a positive constant value ω_sw to generate the protracting motion, whereas in the stance modes (M_i,j = Early stance mode, Late stance mode), ${\overline{\stackrel{.}{\theta }}}_{i,j}^{\alpha }$ is a negative constant value ω_st to generate the retracting motion. The joints β and γ are controlled to achieve a joint target angle ${\bar{\theta }}_{i,j}^{\beta }$ and ${\bar{\theta }}_{i,j}^{\gamma }$, respectively, so that the foot moves along a specific trajectory, as shown in Figure 2B. Details of the foot trajectory design are described in Appendix A. Note that there is no additional control mode to stand stably (not walk) in the proposed mode. To realize a transition between walking and standing, the target angular velocity of alpha joint ${\bar{\theta }}_{i,j}^{\alpha }$ will change between negative for walking and zero for standing. However, in the present study, we focus on flexible changes walking and searching and set the parameter ${\bar{\theta }}_{i,j}^{\alpha }$ constant value for limb stride motion.

To generate adaptive interlimb coordination patterns, each controller should switch the control modes depending on the situation. The present study proposes four simple transition rules between the swing and stance modes, as shown in the overview control scheme (Figure 2C). Note that most rules conduct in a decentralized manner by exploiting physical interactions between the whole body and the environment. The details of the four simple rules are explained in the following sections.

Rule (i): Stretch Reflex

In the first rule, the limb changes its control modes between the swing and stance mode at the anterior extreme position (AEP) and posterior extreme position (PEP) of the foot to generate periodic limb stride motion (Figure 3). If the angle of joint α in the swing mode reaches a positive threshold angle θ_AEP (${\theta }_{i,j}^{\alpha }\geqq {\theta }_{\text{AEP}}$), then the limb controller changes its mode from swing mode to early stance mode. In contrast, if the joint angle α in the stance mode reaches a negative threshold angle θ_PEP (${\theta }_{i,j}^{\alpha }\leqq {\theta }_{\text{PEP}}$), then the limb controller changes its mode from stance to swing. The above transitions are described as follows:

$\begin{array}{ll}\{\begin{array}{c}\text{if }{\theta }_{i,j}^{\alpha }\geqq {\theta }_{\text{AEP}}\text{ then }{M}_{i,j}=Swingmode\to Earlystancemode,\\ \text{if }{\theta }_{i,j}^{\alpha }\leqq {\theta }_{\text{PEP}}\text{ then }{M}_{i,j}=Latestancemode\to Swingmode.\end{array}& (1)\end{array}$

Note that after mode transition from the swing to stance at the AEP point, the limb first becomes the early stance mode (M_i,j = Early stance mode), not the late stance mode (M_i,j = Late stance mode).

FIGURE 3

Figure 3. Transition mechanism based on α joint angle, ${\theta }_{i,j}^{\alpha }$. (A) Top view of the robot and threshold joint angles, θ_AEP and θ_PEP for the anterior extreme position (AEP) and posterior extreme position (PEP) transitions, respectively. (B) Example of changes in the control mode via transition rule (i).

Rule (ii): Searching Reflex

The second rule realizes adaptive switching between stepping and searching behavior depending on the lack of footholds. Although insects usually exhibit long retracting and protracting motions to generate stride lengths, the insect repeats short retracting and protracting motions to search for the next foothold in response to the foothold gaps (Pearson and Franklin, 1984; Theunissen and Dürr, 2013; Theunissen et al., 2014).

To implement the flexible changes between the stepping and searching behaviors, the present study assumes a simple transition rule for the transition from the stance mode to the swing mode as follows:

$\begin{array}{l}\text{if }(N_{i,j}^h<N_{\text{SRH}}^h)\wedge ({\theta }_{i,j}^{\alpha }<{\theta }_{\text{SRH}}^{\alpha })\text{then }{M}_{i,j}\\ =Earlystancemode\to Swingmode,& (2)\end{array}$

where $N_{i,j}^h$ is a horizontal component of ground reaction force (GRF) applied at the i, j limb (driving force is positive), $N_{\text{SRH}}^h$ is a threshold value for detecting where the limb obtains the foothold, and ${\theta }_{\text{SRH}}^{\alpha }$ is a constant value describing a range of joints α for the searching behavior. According to rule (i), the protracted limb changes the control mode from swing to stance and starts to retract. If the limb has no propulsive force after the retraction motion, then the limb changes to the swing mode immediately, resulting in the protracting motion (Figure 4).

FIGURE 4

Figure 4. Transition rule (ii) to secure foothold. (A) Top view of the body and a range of α joint angles, ${\theta }_{\text{SRH}}^{\alpha }$. (B) Example of changes in control mode via transition rule (ii). When ${\theta }_{AEP}>{\theta }_{i,j}^{\alpha }>{\theta }_{\text{SRH}}^{\alpha }$ after the transition in AEP, the robot protracts the limb and tends to kick the ground. (C1) If the limb cannot perceive the ground reaction force (GRF), the limb controller changes to the swing mode, resulting in a short step. (C2) If the limb successfully kicks the ground, the limb remains in the stance mode, resulting in a long step.

Rule (iii): Active Load Sensing at the Beginning of the Swing Phase

The third rule is attempted to secure a support polygon at the beginning of the swing phase in a decentralized manner. The support polygon is a convex horizontal region whose vertices correspond to the support limbs. For example, in Figure 5A, the support polygon comprises the contact points R1, R2, L2, and L3. When the center of mass (COM) lies in the support polygon, static stability is achieved during locomotion.

FIGURE 5

Figure 5. Active load sensing scheme for detecting free and responsible limbs in hexapod locomotion. (A) Free limb situation. After the R2 limb lifts, the center of mass (COM) is still in the supporting polygon. (B) Responsible limb situation. After the L2 limb lifts, the COM moves outside the support polygon. By exploiting the physical interaction between the body and the environment, each limb can simply modulate its control mode for the steady support polygon. (C) Top view of the body and a range of α joint angles, ${\theta }_{\text{ALS}}^{\alpha }$. (D) Example of changes in control mode via transition rule (iii).

To achieve static stability during locomotion through a decentralized control manner, this study classifies the stance limbs into two types: “free limb” and “responsible limb.” The free limb is a stance limb in which the robot maintains static stability when the concerned limb lifts off the ground. For example, consider the support polygon shown in Figure 5A, where the R1, R2, L2, and L3 limbs are in the stance phase. When the R2 limb lifts off, the new support polygons with R1, L2, and L3 still contain the COM of the insect, maintaining static stability. Therefore, the R2 limb can be classified as a free limb. In contrast to the free limb, the responsible limb is a stance limb in which the robot cannot keep the static polygon when the concerned limb lifts off the ground. For example, in Figure 5B, when the L2 limb lifts off, the COM of the insect is located outside the new support polygon with R1, R2, and L3, resulting in a lack of static stability. Consequently, the L2 limb in Figure 5B can be classified as a responsible limb. For stable and adaptive locomotion, the challenge is to instantly detect the free and responsible limbs and accordingly modulate the limb movements to maintain static stability.

The proposed study distinguishes between free and responsible limbs and modulates the limb control mode adaptively in a simple, decentralized manner. For detection of the limb state, the stance limb close to the PEP first attempts to lift off the ground. If the concerned limb perceives no GRF, then it can be interpreted as a free limb, and it changes the stance mode to swing mode. In contrast, if the concerned limb still perceives GRF, then it can be interpreted as a responsible limb and should be maintained in the stance mode. We describe these sequences of action and detection as “active load sensing.”

In the proposed model, we implement active load sensing around the PEP as follows. The lifting action is realized by other transition rules. Then, the sensory feedback mechanism based on active load sensing is described as follows:

$\begin{array}{l}\text{if }(N_{i,j}^v\geqq N_{\text{ALS}}^v)\wedge ({\theta }_{i,j}^{\alpha }>{\theta }_{\text{ALS}}^{\alpha })\\ \text{then }{M}_{i,j}\to Latestancemode.& (3)\end{array}$

where $N_{i,j}^v$ is a vertical component of GRF applied at the i, j limb, $N_{\text{ALS}}^v$ is a positive constant value for a threshold whether the limb is loaded or unloaded, and and ${\theta }_{\text{ALS}}^{\alpha }$ is a constant value to describe a blind-sector angle for active load sensing (Figure 5C). When ${\theta }_{\text{ALS}}^{\alpha }>{\theta }_{i,j}^{\alpha }>{\theta }_{\text{PEP}}$ during the swing mode, the limb maintains the protracting motion for lifting. After the lifting motion, if the protracting limb perceives GRF, it changes the control mode from swing to stance immediately like Figure 5D to achieve static stability.

Rule (iv): Sensory Feedback From Next Anterior Limb

In slow insect walking gaits (e.g., tetrapod gait and wave gait), the limbs of the ipsilateral side exhibit a metachronal wave from the tail to the head (i.e., wave gait; Wilson, 1966). Based on the insect walking trend, we assume the fourth transition rule in which each limb tends to switch its control mode from the stance mode to the swing mode when the anterior next limb reaches the PEP (Figure 6). More specifically, the enforcing early protraction refers to whether the angle of the next anterior joint α, ${\theta }_{i,j-1}^{\alpha }$, achieves a threshold angle ${\theta }_{\text{DRT}}^{\alpha }$ using the following equation:

$\begin{array}{l}\text{if }{\theta }_{i,j-1}^{\alpha }\leqq {\theta }_{\text{DRT}}^{\text{α}}\\ \text{then }{M}_{i,j}=Latestancemode\to Swingmode(\text{for }j\ne 1).& (4)\end{array}$

where ${\theta }_{\text{DRT}}^{\text{α}}$ is the threshold angle in the joint α that detects the limb closer to the PEP. Note that rule (iv) is the only transition rule that assumes the neural coupling between limbs in the proposed model.

FIGURE 6

Figure 6. Transition to swing phase depending on the position of next anterior limb. (A) Top view of the robot and threshold joint angle ${\theta }_{\text{DRT}}^{\alpha }$ for the effect from anterior to posterior limbs. (B) Changes in joint α of the anterior limb. When ${\theta }_{i,j-1}^{\alpha }<{\theta }_{\text{DRT}}^{\alpha }$, the limb sends a signal for the posterior limb to transition from the stance to swing mode. (C) Changes in joint α of the posterior limb. The posterior limbs move to the swing mode before reaching the PEP angle θ_PEP.

3. Results

To evaluate the proposed interlimb coordination mechanism, the present study conducts three kinds of simulation experiments: the emergence of typical hexapod locomotion, adaptation to leg amputation, and adaptation to gap environment. We use an open dynamics engine (ODE) to calculate the hexapod robot's three-dimensional physical dynamics in all experiments. The parameters in the simulation are heuristically determined as shown in Table 1 so that the robot can generate a typical tripod gait when the target angular velocities in the stance mode ω_st are the same as that in the swing mode ω_sw.

TABLE 1

Table 1. Parameters in simulation experiments.

3.1. Emergence of Typical Hexapod Gait Patterns

The first simulation experiment aims to evaluate how the proposed rules affect the locomotion patterns of the robot in response to various locomotion speeds. Regarding the experimental setup, the robot with an intact body (i.e., no leg amputation) walks on flat terrain. To address the flexibility of the locomotor patterns in response to locomotor speed, we conducted walking experiments with various swing-stance ratios. More specifically, we set constant values of ω_sw and ω_st, as shown in Table 1 for various locomotion frequencies. This setup is according that various insects likely maintain duration in the swing phase while they change the various durations in the stance phase (Wosnitza et al., 2013; Reinhardt and Blickhan, 2014; Weihmann et al., 2017; Dürr et al., 2018). As the phase oscillator based CPG models set the intrinsic frequency of periodic limb motion (Owaki et al., 2017), this study simply set limb swing speed of joint α to generate protract and retract motions in the swing and stance modes.

The results of the simulation experiments showed that the robot exhibited various gait patterns depending on the locomotor speed. When (ω_sw, ω_st) = (π/3, π/3), the robot exhibited synchronous coordination in two groups: L1 and R2 are L3 moves in phase, and R1 and L2 are R3 moves in phase as shown in Figure 7A. The interlimb coordination patterns correspond to the tripod gait. The locomotion speed is 10.7 [cm/s]. Additionally, when the target angular velocity in the stance mode ω_st decreases to π/6, the robot exhibits different coordination patterns: L1 and R3 synchronize, L2 and R1 synchronize, and L3 and R2 synchronize. These coordination patterns correspond to the typical tetrapod gait, where the two limbs are in the swing phase and the other four limbs support the body weight. The locomotion speed is 8.8 [cm/s]. Furthermore, the parameter ω_st decreases to π/15, and the robot exhibits a typical wave gait, as shown in Figure 7C where the ipsilateral anterior limbs move to the swing phase after the next posterior limb. These speed-dependent gait patterns of the robot correspond to the trends of insect locomotor patterns (Wilson, 1966). The locomotion speed is 2.4 [cm/s].

FIGURE 7

Figure 7. Various insect-like walking patterns depend on the speed ratio during the swing phase. Emerging gait patterns: (A) Tripod gait at (ω_sw, ω_st) = (π/3, π/3), (B) Tetrapod gait at (ω_sw, ω_st) = (π/3, π/6), and (C) Wave gait at (ω_sw, ω_st) = (π/3, π/15). The colored region represents the stance phase where the limb contacts the ground, while the white region represents the swing phase, where the limb has no ground contact. (D–F) show changes in the joint angle α from the beginning of the tripod, tetrapod, and wave gaits simulations, respectively.

For each locomotor condition, the proposed reflex rules modulate the interlimb coordination patterns as shown in Figures 7D,E. At the beginning of walk, the limb motions are frequently modulated by the reflex rules, for example, active load sensing (rule iii) in Figure 7D. As each interlimb coordination pattern converges, the reflex rules rarely modulate the limb's motion. This is because the locomotor patterns that emerge establish support polygons. With low stance speed (e.g., ω_st = π/15), the searching reflex (rule ii) and active load sensing (rule iii) rarely occurs, as shown in Figure 7F, because the long stance period contributes to maintaining the support polygons.

Although the robot exhibits speed dependent interlimb coordination patterns, several limbs show vague takeoff and touchdown, resulting in chattering in the gait diagram in the border between the swing phase and stance phase. This chattering is more conspicuous in a fast walking pattern like tripod gait (Figure 7A) than slow walking gait like metachronal wave gait (Figure 7B). This is because the low duty ratio in fast walking induces difficulty for limbs to translate next supporting polygon. In contrast, the large support polygon in the high duty ratio like Figure 7C facilitates the free limb to translate from the stance mode to the swing mode.

3.2. Adaptation to Leg Amputation

The second simulation experiment aims to evaluate the adaptability of the proposed model to leg amputation. In this simulation, we removed the middle limbs (L2 and R2), and the robot walked on leveled ground. We assume that the amputated limb does not induce the next posterior limb to change early from the stance to swing mode, and consequently, rule (iv) is invalidated. Additionally, the control parameters are the same as in the first simulation experiment, as shown in Table 1. Regarding angular velocities, we set (ω_sw, ω_st) = (π/3, π/15), referring to the low locomotion speed.

Figure 8 shows the results of the amputation. When L2 and R2 limbs are amputated, the robot generates feasible locomotor patterns that differ from the locomotor patterns by the intact robot. As shown in Figure 8A, the posterior limb on the ipsilateral side (e.g., L3 limb) moves before the anterior limb (e.g., L1 limb) despite no neural communication by the transition rule (iv). Figure 8B shows that the active load sensing (rule (iii)) modulates the responsible limb motions so as to generate feasible interlimb coordination at the beginning of walking. The emerging interlimb coordination well reproduces the actual amputated insect (Hughes, 1957; Graham, 1977; Dean, 1991; Grabowska et al., 2012). Furthermore, Figure 8C shows the trajectory of the walking robot with various combinations of leg amputation at specific periods. The robot can adapt to various combinations of leg amputations (e.g., middle limbs and hind limbs). However, when the L1 and R1 limbs are amputated, the robot falls forward, and it cannot generate feasible locomotor patterns. In the falling case, the COM moves the outside of the support polygon during the stance phase of the middle limbs. Although the proposed model still has room for improvement, it well reproduces parts of the insects' adaptive behavior (e.g., lateral sequence gait with L2R2 amputation) as well as the previous model (Owaki et al., 2017).

FIGURE 8

Figure 8. Results of adaptation to leg amputations. (A) Gait diagram of the walking robot with amputated L2 and R2 limbs. Regarding the target angular velocity, (ω_sw, ω_st) = (π/3, π/15). The robot exhibited a lateral sequence gait. (B) The history of each joint angle α from the beginning of walking with the leg amputation. (C) Robot trajectories with various combinations of leg amputation. The robot can move despite the leg amputations, whereas the robot stacks when the L1 and R1 limbs are amputated. Note that there is no direction control mechanism and the direction of robot movement changes depending on the physical interaction between the robot and the environment.

3.3. Adaptation to Gap Environment

The third experiment addresses the flexible transition between the stepping and searching behaviors in response to the lack of footholds. In this experiment, the robot walked on the ground with gaps and footholds of a specific width, as shown in Figure 9. To evaluate the effect of transition rule (ii), we compare the robot with and without transition rule (ii) and measure the success ratio over 30 trials for the two control conditions. In each trial, the initial joint α angles ${\theta }_{i,j}^{\alpha }$ are randomly set. When the transition rule (ii) is eliminated, the transition rule (i) at the AEP in Equation (1) is modulated as follows:

$\begin{array}{ll}\begin{array}{l}\text{if }{\theta }_{i,j}^{\alpha }\geqq {\theta }_{\text{AEP}}\text{ then }{M}_{i,j}=Swingmode\to Latestancemode.\end{array}& (5)\end{array}$

Because of this modulation, the controller has two states: M_i,j = 0 for the swing mode and M_i,j = 1 for the stance mode.

FIGURE 9

Figure 9. Experimental setup for locomotion on uneven terrain with gaps.

During gap crossing, the proposed model modulates the interlimb coordination and resulting in adaptive changes between walking and searching behaviors. At the beginning of the walk with the random, the active load sensing (reflex rule iii) and the effect from the posterior limb (reflex rule iv) modulates the interlimb coordination from the random initial condition to tetrapod gaits as shown in Figure 10. During the gap crossing, several limbs generate searching behaviors depending on the lack of foothold. Note that other limbs adaptively keep their control modes of stance mode by using to secure the support polygon by the feedback from the active load sensing. Then, after the gap crossing, all limbs modulate their interlimb coordination for stable locomotor patterns by the fundamental reflex rules. The robot with the searching reflex achieves 60% success ratio, while the robot without the searching reflex achieves a success ratio of under 30% (Figure 10B).

FIGURE 10

Figure 10. Results of walking experiments on uneven terrain. (A) Each history of α joint's angle during crossing gap sections. (B) Comparison of success ratios with and without the transition rule (ii). For each control condition, we conducted 30 trials with random initial angles for each limb's α joint. (C) Snapshots of successful gap crossing in the trial with transition rule (ii). (D) Snapshots of gap crossing failure during walking with transition rule (ii).

Figure 10C shows snapshots of the successful trial in the gap crossing by the robot with reflex rule (ii). The L1 limb retracts over the gap and does not obtain the foothold. Then, the L1 controller switches the control mode from the stance mode to the swing mode via the searching reflex. During the L1 limb's protraction, the robot body moves forward by other limbs' retractions, and consequently, L1 overcomes the gap and obtains a new foothold.

The robot with the searching reflex, however, fails to cross the gap due to stacking behaviors as shown in the snapshots of the failed trial (Figure 10D). In these snapshots, when the L1 and R1 limbs lift, the robot maintains the static support polygon with the L2, L3, R2, and R3 limbs. However, as the supporting limbs retract, the COM moves outside the support polygon, and consequently, the robot loses body balance during searching. Although the proposed model sometimes fails the crossing gap because of no sensory modulation during the stance phase, these results show that the proposed simple interlimb coordination mechanism play a pivotal role for the robot to change its limb behavior between walking step and searching step in response to the lack of footholds around AEP.

4. Discussion

The significance of the present study is to demonstrate that insect-like adaptive locomotor patterns (e.g., adaptation to locomotion speed, leg amputation, and gap crossing) can emerge via a simple chain of reflex mechanisms. Owing to the simplicity of the proposed model, the series of transition rules can be interpreted as a simple control strategy: each limb tries to create a static support polygon in a decentralized manner. This simple control strategy seems to be reasonable in insect locomotion because the insect's morphology (e.g., low COM due to the sprawled posture and a redundant number of limbs) has great advantages in securing support polygons. While complex neural network models help us to clarify the correspondence between the neural networks in the insect and the structure of neural modules in the modeling studies, the simple model allows us to understand the essences of the underlying control mechanism as well as introduce them to adaptive robot control.

While our model is abstracted, each reflex mechanism is similar to the biological findings. Rule (i) follows the reflex mechanism based on joint angles (Akay et al., 2004; Ekeberg et al., 2004). Rule (ii) and rule (iii) satisfy the physiological findings that sensory input signaling ground contact takes over the effects of command neurons for searching behaviors (Berg et al., 2015). Rule (iv) is similar to the effects from the posterior to the anterior limbs (Borgmann et al., 2009). Although our proposed model does not describe the details of the above sensory feedback mechanisms with interactions among sensor organs (e.g., mechanoreceptors), motor- and intern-neurons, the simple model integrates substantial sensory feedback mechanisms for adaptive interlimb coordination in response to locomotor frequency, leg amputation, and a gap of foothold.

Furthermore, the structure of the proposed model could shed new light on the control mechanism underlying insect adaptive searching behaviors. According to biological experiments, two control schemes, the “two motor patterns hypothesis” and “two control modes hypothesis,” have been proposed (Dürr et al., 2018). In the two motor patterns hypothesis, the control system has two distinct motor patterns for the long step of the walking limb and the short step of the searching limb. Walking and searching behaviors are realized by switching these motor patterns depending on the sensory information. In contrast, the two control modes hypothesis assumes a control mechanism for each swing phase and stance phase, and adaptive walking and searching steps emerge from reasonable switching between the control mechanisms. In this sense, our proposed model agrees with the two control modes hypothesis. Note that our simple model shows the significance of physical interaction in the two control modes hypothesis. Although each limb locally implements two control modes (namely, swing mode and stance mode) and simple reflex rules, the physical interaction with the environment globally affects among limbs and makes each limb free or responsible to support body weight. These interactions should be important for each limb to flexibly generate walking and searching steps as well as secure support polygon when an other limb is searching the foothold.

The failure case in the simulation experiments suggests that flexible coordination between the joints of one limb (e.g., intralimb coordination) is required to improve the adaptability of the proposed model. In the amputated experiments, the robot with amputated L1, R1 limbs cannot secure the support polygon by other limbs, and it tumbles. The simulated robot body model has COM at the middle of the trunk, the PEP limb position of the middle limbs induces the projected COM on the ground to go outside of the support polygon. These failure cases suggest that limbs should change the AEP and PEP position for a stable support polygon. According to insect behaviors, the actual insects modulate the AEP and PEP positions depending on the limb amputation and carrying loads (Delcomyn, 1991; Zollikofer, 1994).

Besides, in the gap crossing experiments, the COM also goes outside of the support polygon during the responsible limb's stance phase, whereas the anterior limbs searching footholds. This is because the target angular velocities of the α joints are set as constant values ω_sw and ω_st, and the supporting limb keeps retracting regardless of the projected COM going outside the support polygon, as shown in Figure 10C. In contrast, actual insect animals modulate their joint angular velocity depending on the situation (Watson et al., 2002). Furthermore, the searching behavior of each limb in simulation moves around the predesigned AEP in the sagittal plane whereas the insects (e.g., stick insect) spread the AEP of the forelimb forward and lateral (Theunissen and Dürr, 2013). These gaps in limb behaviors between the simulation and insect animals suggest that intra-limb coordination should be considered to generate flexible limb motion in both swing and stance modes.

Although we simplified the robot structure, a more insect-like limb structure may induce the robot to exploit physical interaction with the environment. The proposed model exhibits various insect-like gait patterns as shown in Figure 7; however, each limb shows a chattering step at the PEP position. This is because active load sensing is conducted by the limb lifting off the ground. Therefore, if the limb is responsible for the static supporting polygon, once the limb lifts off the ground, the responsible limb touches the ground again, resulting in chattering behaviors. Introducing a flexible foot segment like an insect's tarsus makes it possible to detect the limb's responsibility by sensing the strain of the flexible tip of the foot segment before the limb lifts off completely.

5. Conclusion

To elucidate the essential interlimb coordination mechanism underlying adaptive insect's walking and searching behaviors, we developed the simple model that consists of two control models (i.e., swing and stance modes) and four substantial reflex rules. Although the results of the simulation experiments suggest the requirement of additional control mechanisms for flexible intralimb coordination, the robot with the proposed simple interlimb coordination mechanism exhibits various speed-dependent gait patterns, adaptation to leg amputation, and flexible switching between the walking step and searching step during the gap crossing. These results show that simple decentralized control mechanism, e.g., active load sensing, and physical interaction with the environment generate the flexible changes between walking and searching limb behaviors with interlimb coordination for secure support polygons.

For further study, we will develop a physical robot considering the flexibility of the foot segment and evaluate the proposed model in a real-world environment. Furthermore, the intralimb coordination mechanism will be introduced in the proposed model so that each limb can adaptively change its stride speed and foot trajectory depending on the robot morphology and locomotor environments.

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.

Author Contributions

AF, TK, RK, and AI conceived the research. AF and WS conducted the experiments and analyzed the results. All authors developed the mathematical modeling and reviewed the manuscript.

Funding

This work was supported by JSPS KAKENHI [Grant-in-Aid for Scientific Research(S)] grant no. JP17H06150.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnbot.2022.645683/full#supplementary-material

Supplementary Movie 1. Tripod gait in Section 3.1.

Supplementary Movie 2. Tetrapod gait in Section 3.1.

Supplementary Movie 3. Wave gait in Section 3.1.

Supplementary Movie 4. The behavior of the robot with amputated L2 R2 limbs in Section 3.2.

Supplementary Movie 5. Gap crossing experiment in Section 3.3.

Supplementary Movie 6. The searching behavior of the robot with reflex rule (ii) in the successful trials in Section 3.3.

Supplementary Movie 7. The stacking behavior of the robot with reflex rule (ii) in the failed trial in Section 3.3.

References

Akay, T., Haehn, S., Schmitz, J., and Buschges, A. (2004). Signals from load sensors underlie interjoint coordination during stepping movements of the stick insect leg. J. Neurophysiol. 92, 42–51. doi: 10.1152/jn.01271.2003

PubMed Abstract | CrossRef Full Text | Google Scholar

Ayali, A., Borgmann, A., Büschges, A., Couzin-Fuchs, E., Daun-Gruhn, S., and Holmes, P. (2015a). The comparative investigation of the stick insect and cockroach models in the study of insect locomotion. Curr. Opin. Insect Sci. 12, 1–10. doi: 10.1016/j.cois.2015.07.004

CrossRef Full Text | Google Scholar

Ayali, A., Couzin-Fuchs, E., David, I., Gal, O., Holmes, P., and Knebel, D. (2015b). Sensory feedback in cockroach locomotion: current knowledge and open questions. J. Compar. Physiol. A 201, 841–850. doi: 10.1007/s00359-014-0968-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Berg, E. M., Hooper, S. L., Schmidt, J., and Büschges, A. (2015). A leg-local neural mechanism mediates the decision to search in stick insects. Curr. Biol. 25, 2012–2017. doi: 10.1016/j.cub.2015.06.017

PubMed Abstract | CrossRef Full Text | Google Scholar

Bläsing, B. (2006). Crossing large gaps: a simulation study of stick insect behavior. Adapt. Behav. 14, 265–285. doi: 10.1177/105971230601400307

CrossRef Full Text | Google Scholar

Borgmann, A., Hooper, S. L., and Büschges, A. (2009). Sensory feedback induced by front-leg stepping entrains the activity of central pattern generators in caudal segments of the stick insect walking system. J. Neurosci. 29, 2972–2983. doi: 10.1523/JNEUROSCI.3155-08.2009

PubMed Abstract | CrossRef Full Text | Google Scholar

Büschges, A., Akay, T., Gabriel, J. P., and Schmidt, J. (2008). Organizing network action for locomotion: insights from studying insect walking. Brain Res. Rev. 57, 162–171. doi: 10.1016/j.brainresrev.2007.06.028

PubMed Abstract | CrossRef Full Text | Google Scholar

Daun-Gruhn, S. (2011). A mathematical modeling study of inter-segmental coordination during stick insect walking. J. Comput. Neurosci. 30, 255–278. doi: 10.1007/s10827-010-0254-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Dean, J. (1991). Effect of load on leg movement and step coordination of the stick insect carausius morosus. J. Exp. Biol. 159, 449–471. doi: 10.1242/jeb.159.1.449

CrossRef Full Text | Google Scholar

Delcomyn, F. (1991). Perturbation of the motor system in freely walking cockroaches. ii. the timing of motor activity in leg muscles after amputation of a middle leg. J. Exp. Biolo. 156, 503–517. doi: 10.1242/jeb.156.1.503

PubMed Abstract | CrossRef Full Text | Google Scholar

Durr, V. (2001). Stereotypic leg searching movements in the stick insect: kinematic analysis, behavioural context and simulation. J. Exp. Biol. 204, 1589–1604. doi: 10.1242/jeb.204.9.1589

PubMed Abstract | CrossRef Full Text | Google Scholar

Dürr, V., Schmitz, J., and Cruse, H. (2004). Behaviour-based modelling of hexapod locomotion: linking biology and technical application. Arthropod Struct. Dev. 33, 237–250. doi: 10.1016/j.asd.2004.05.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Dürr, V., Theunissen, L. M., Dallmann, C. J., Hoinville, T., and Schmitz, J. (2018). Motor flexibility in insects: adaptive coordination of limbs in locomotion and near-range exploration. Behav. Ecol. Sociobiol. 72, 1–21. doi: 10.1007/s00265-017-2412-3

CrossRef Full Text | Google Scholar

Ekeberg, Ö., Blümel, M., and Büschges, A. (2004). Dynamic simulation of insect walking. Arthropod Struct. Dev. 33, 287–300. doi: 10.1016/j.asd.2004.05.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Full, R. J., Stokes, D. R., Ahn, A. N., and Josephson, R. K. (1998). Energy absorption during running by leg muscles in a cockroach. J. Exp. Biol. 201, 997–1012. doi: 10.1242/jeb.201.7.997

PubMed Abstract | CrossRef Full Text | Google Scholar

Grabowska, M., Godlewska, E., Schmidt, J., and Daun-Gruhn, S. (2012). Quadrupedal gaits in hexapod animals-inter-leg coordination in free-walking adult stick insects. J. Exp. Biol. 215, 4255–4266. doi: 10.1242/jeb.073643

PubMed Abstract | CrossRef Full Text | Google Scholar

Graham, D. (1977). The effect of amputation and leg restraint on the free walking coordination of the stick insectcarausius morosus. J. Compar. Physiol. 116, 91–116. doi: 10.1007/BF00605519

CrossRef Full Text | Google Scholar

Hughes, G. (1957). The co-ordination of insect movements: 11. the effect of limb amputation and the cutting of commissures in the cockroach (blatta oiuentalis). J. Exp. Biol. 34, 306–333. doi: 10.1242/jeb.34.3.306

CrossRef Full Text | Google Scholar

Kimura, S., Yano, M., and Shimizu, H. (1993). A self-organizing model of walking patterns of insects. Biol. Cybern. 69, 183–193. doi: 10.1007/BF00198958

CrossRef Full Text | Google Scholar

Koditschek, D. E., Full, R. J., and Buehler, M. (2004). Mechanical aspects of legged locomotion control. Arthropod Struct. Dev. 33, 251–272. doi: 10.1016/j.asd.2004.06.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Kukillaya, R., Proctor, J., and Holmes, P. (2009). Neuromechanical models for insect locomotion: Stability, maneuverability, and proprioceptive feedback. Chaos 19, 026107. doi: 10.1063/1.3141306

PubMed Abstract | CrossRef Full Text | Google Scholar

Mantziaris, C., Bockemühl, T., and Büschges, A. (2020). Central pattern generating networks in insect locomotion. Dev. Neurobiol. 80, 16–30. doi: 10.1002/dneu.22738

PubMed Abstract | CrossRef Full Text | Google Scholar

Ngamkajornwiwat, P., Homchanthanakul, J., Teerakittikul, P., and Manoonpong, P. (2020). Bio-inspired adaptive locomotion control system for online adaptation of a walking robot on complex terrains. IEEE Access 8, 91587–91602. doi: 10.1109/ACCESS.2020.2992794

PubMed Abstract | CrossRef Full Text | Google Scholar

Owaki, D., Goda, M., Miyazawa, S., and Ishiguro, A. (2017). A minimal model describing hexapedal interlimb coordination: the tegotae-based approach. Front. Neurorobot. 11:29. doi: 10.3389/fnbot.2017.00029

PubMed Abstract | CrossRef Full Text | Google Scholar

Pearson, K., and Franklin, R. (1984). Characteristics of leg movements and patterns of coordination in locusts walking on rough terrain. Int. J. Rob. Res. 3, 101–112. doi: 10.1177/027836498400300209

CrossRef Full Text | Google Scholar

Reinhardt, L., and Blickhan, R. (2014). Level locomotion in wood ants: evidence for grounded running. J. Exp. Biol. 217, 2358–2370. doi: 10.1242/jeb.098426

PubMed Abstract | CrossRef Full Text | Google Scholar

Schilling, M., Hoinville, T., Schmitz, J., and Cruse, H. (2013a). Walknet, a bio-inspired controller for hexapod walking. Biol. Cybern. 107, 397–419. doi: 10.1007/s00422-013-0563-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Schilling, M., Paskarbeit, J., Hoinville, T., Hüffmeier, A., Schneider, A., Schmitz, J., et al. (2013b). A hexapod walker using a heterarchical architecture for action selection. Front. Comput. Neurosci. 7:126. doi: 10.3389/fncom.2013.00126

PubMed Abstract | CrossRef Full Text | Google Scholar

Theunissen, L. M., Bekemeier, H. H., and Dürr, V. (2015). Comparative whole-body kinematics of closely related insect species with different body morphology. J. Exp. Biol. 218, 340–352. doi: 10.1242/jeb.114173

PubMed Abstract | CrossRef Full Text | Google Scholar

Theunissen, L. M., and Dürr, V. (2013). Insects use two distinct classes of steps during unrestrained locomotion. PLoS ONE 8, e85321. doi: 10.1371/journal.pone.0085321

PubMed Abstract | CrossRef Full Text | Google Scholar

Theunissen, L. M., Vikram, S., and Dürr, V. (2014). Spatial co-ordination of foot contacts in unrestrained climbing insects. J. Exp. Biol. 217, 3242–3253. doi: 10.1242/jeb.108167

PubMed Abstract | CrossRef Full Text | Google Scholar

Toth, T. I., Schmidt, J., Büschges, A., and Daun-Gruhn, S. (2013). A neuro-mechanical model of a single leg joint highlighting the basic physiological role of fast and slow muscle fibres of an insect muscle system. PLoS ONE 8, e78247. doi: 10.1371/journal.pone.0078247

PubMed Abstract | CrossRef Full Text | Google Scholar

Watson, J. T., Ritzmann, R. E., Zill, S. N., and Pollack, A. J. (2002). Control of obstacle climbing in the cockroach, blaberus discoidalis. i. kinematics. J. Compar. Physiol. A 188, 39–53. doi: 10.1007/s00359-002-0277-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Weihmann, T., Brun, P.-G., and Pycroft, E. (2017). Speed dependent phase shifts and gait changes in cockroaches running on substrates of different slipperiness. Front. Zool. 14, 1–15. doi: 10.1186/s12983-017-0232-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Wilson, D. M. (1966). Insect walking. Annu. Rev. Entomol. 11, 103–122. doi: 10.1146/annurev.en.11.010166.000535

PubMed Abstract | CrossRef Full Text | Google Scholar

Wosnitza, A., Bockemühl, T., Dübbert, M., Scholz, H., and Büschges, A. (2013). Inter-leg coordination in the control of walking speed in drosophila. J. Exp. Biol. 216, 480–491. doi: 10.1242/jeb.078139

PubMed Abstract | CrossRef Full Text | Google Scholar

Zollikofer, C. (1994). Stepping patterns in ants-influence of load. J. Exp. Biol. 192, 119–127. doi: 10.1242/jeb.192.1.119

PubMed Abstract | CrossRef Full Text | Google Scholar

Appendix A: Foot Trajectory

In the stance mode, the limb generates retracting motion to kick the ground with a target angular velocity in the α joint, ${\overline{\stackrel{.}{\theta }}}_{i,j}^{\alpha }$. More specifically, the target values for each joint are described to generate a specific state foot trajectory as follows:

$\begin{array}{ll}{\overline{\stackrel{.}{\theta }}}_{i,j}^{\alpha }={\omega }_{\text{st}},& (6)\end{array}$

$\begin{array}{ll}{\bar{\theta }}_{i,j}^{\beta }=\frac{\pi -{\bar{\theta }}_{i,j}^{\gamma }}{2}+{tan}^{-1}\left(\frac{Hcos{\theta }_{i,j}^{\alpha }}{W}\right)-\frac{\pi }{2},& (7)\end{array}$

$\begin{array}{ll}{\bar{\theta }}_{i,j}^{\gamma }={cos}^{-1}\left(\frac{L_{\text{UPR}}^2+L_{\text{BTM}}^2-L_{\text{st}}^2}{2L_{\text{UPR}}{L}_{\text{BTM}}}\right),& (8)\end{array}$

where L_UPR and L_UPR are the lengths of the upper and bottom links of the limb, respectively, and L_st is a parameter that reflects the target foot trajectory. The parameter L_st is calculated as follows:

$\begin{array}{ll}{L}_{\text{st}}=H^2+{(W/cos{\theta }_{i,j}^{\alpha })}^2.& (9)\end{array}$

where H and W are the target height of the body unit and width of the target foot trajectory, respectively.

In the swing mode, the limb generates a protracting motion along a round trajectory for ground clearance. As in the stance mode, the joint α is controlled to achieve the target angular velocity, where the joints β and β are controlled to achieve target angles. The target values are described as follows:

$\begin{array}{ll}{\overline{\stackrel{.}{\theta }}}_{i,j}^{\alpha }={\omega }_{\text{sw}},& (10)\end{array}$

$\begin{array}{ll}{\bar{\theta }}_{i,j}^{\beta }=\frac{\pi -{\bar{\theta }}_{i,j}^{\gamma }}{2}-\frac{\pi }{2},& (11)\end{array}$

$\begin{array}{ll}{\bar{\theta }}_{i,j}^{\gamma }={cos}^{-1}\left(\frac{L_{\text{UPR}}^2+L_{\text{BTM}}^2-L_{\text{sw}}^2}{2L_{\text{UPR}}{L}_{\text{BTM}}}\right),& (12)\end{array}$

where L_sw is a parameter that reflects the target foot trajectory. The parameter L_sw is calculated as follows:

$\begin{array}{ll}{L}_{\text{sw}}={(H-h({\theta }_{i,j}^{\alpha }))}^2+{(W/cos{\theta }_{i,j}^{\alpha })}^2,& (13)\end{array}$

$\begin{array}{ll}h({\theta }_{i,j}^{\alpha })=Hcos\left(\frac{\pi }{{\theta }_{\text{AEP}}-{\theta }_{\text{PEP}}}({\theta }_{i,j}^{\alpha }-\frac{{\theta }_{\text{AEP}}+{\theta }_{\text{PEP}}}{2})\right).& (14)\end{array}$

where $h({\theta }_{i,j}^{\alpha })$ is a function of ${\theta }_{i,j}^{\alpha }$ for the calculation of the target height of the foot. θ_AEP and θ_PEP are positive and negative constant values for the angular limitation of joint α at the AEP and PEP, respectively. By switching between the two control modes, the limb generates stride motion along the semicircular trajectory.