Decentralized Control Mechanism for Determination of Moving Direction in Brittle Stars With Penta-Radially Symmetric Body

1. Introduction

Most robots are designed to perform given tasks in predefined environments and lack the ability to autonomously determine and move toward their moving direction while adapting to various unpredictable situations. In contrast, animals can feasibly adapt to the unpredictable real-world and move toward their desired direction. Interestingly, this ability is not unique to higher organisms with sophisticated brains, but is inherent even in primitive living organisms (Meyer et al., 2017). This fact suggests that decisions are not made solely by a central controller, i.e., brain, and that decentralized control plays a significant role in animal locomotion. Many studies have been devoted to elucidate the decentralized control mechanism underlying animals' adaptive locomotion (Kimura et al., 2007; Schilling et al., 2013; Kano et al., 2017a,c), yet they have not succeeded in achieving both autonomous determination of moving direction and adaptation to unpredictable circumstances simultaneously. Clarifying it will help understand the mechanism of autonomous behaviors of animals as well as develop fully autonomous robots that can make decisions by themselves and behave adaptively and reasonably on this basis.

Brittle stars, a group of echinoderms that locomote on the sea floor, are a suitable model for addressing the above-mentioned issue. They have a penta-radially symmetric body in which five arms radiate from the central disc and are capable of moving omni-directionally (Figure 1) (Arshavskii et al., 1976b; Astley, 2012; Kano et al., 2012; Watanabe et al., 2012). They lack a central nervous system but instead possess a rather simple distributed nervous system consisting of radial nerves along the arms, which join a circumoral nerve ring at the disc (Figure 1) (Cobb and Stubbs, 1981). Despite such a simple nervous system, they can make decisions about their own movement (e.g., to escape from predators or to approach food), and once the decision is made, they locomote by coordinating their arm movements in real time (Arshavskii et al., 1976a,b; Astley, 2012; Matsuzaka et al., 2017; Clark et al., 2019). Moreover, they have outstanding resilience to bodily damage. Even after arbitrary loss of their arms, they promptly determine their moving direction and reorganize the coordination of the remaining arms to resume locomotion (Arshavskii et al., 1976a; Kano et al., 2017b).

FIGURE 1

Figure 1. Body and nerve structure of a brittle star (Ophiarachna incrassata).

It is likely that the moving direction is determined in the circumoral nerve ring by integrating sensory information detected at the arms. However, previous works on the locomotion of brittle stars (Arshavskii et al., 1976a,b; Astley, 2012; Kano et al., 2012, 2017b; Watanabe et al., 2012) did not elucidate the essential mechanism for the determination of moving direction. Recently, we have investigated the role of the circumoral nerve ring in the determination of moving direction through behavioral experiments (Clark et al., 2019). In particular, we observed the locomotion of brittle stars whose circumoral nerve ring was transected at various points. We found that brittle stars tended to move toward the direction opposite to the transected points, and that arms do not coordinate when the neural connection between the arms is lost by the transection. It is expected that these findings impart important insights into the determination of moving direction.

In this study, we propose a mathematical model that explains the behavioral findings on the nerve ring transection to deepen the understanding of the mechanism for the determination of moving direction in brittle star locomotion. In particular, based on our previous model for the adaptive inter-arm coordination of trimmed-arm brittle stars (Kano et al., 2017b), the extended model is proposed wherein the moving direction is modeled phenomenologically with an analogy of water tanks connected by tubes. We demonstrate via simulations that the proposed model well reproduces the results of the behavioral experiments.

2. Behavioral Findings

In this section, we present representative results of the behavioral experiments in which the circumoral nerve ring was transected in various ways (Clark et al., 2019). In particular, here we present the following four cases (Figure 2):

• Experiment 1. The nerve ring was cut in one place.

• Experiment 2. The nerve ring was cut in two places, with each arm retaining at least one nerve ring connection with an adjacent arm.

• Experiment 3. The nerve ring connection between each arm was cut (i.e., five separations). Moreover, an aversive stimulus was added to one of the arms.

• Experiment 4. The nerve ring was cut in two places but on both sides of the same arm. Moreover, an aversive stimulus was added to one of the neurally connected arms.

FIGURE 2

Figure 2. Results of the behavioral experiments: (A) Experiment 1, (B) Experiment 2, (C) Experiment 3, and (D) Experiment 4. Points where the nerve ring is cut and points where an aversive stimulus is added are indicated by scissors and blue circles, respectively. In the snapshots, the moving direction, points where the nerve ring is cut, and points where an aversive stimulus is added are indicated by black, orange, and blue arrows, respectively. Except for (B), data was taken from the work by Clark et al. (2019).

Note that in all experiments, we first observed the behavior of intact brittle stars for 10 min. Then, we cut the nerve rings and observed the behavior for a further 10 min. We used potassium chloride (KCL) solution as the aversive stimuli. Detailed procedures and the results of the experiments are provided in the work by Clark et al. (2019).

The results are shown in Figure 2. In Experiment 1, brittle stars tended to move in the direction opposite to the point where the nerve ring was cut (Figure 2A and Supplementary Movie 1). In Experiment 2, the arms connected by the nerve ring tended to coordinate; however, neurally disconnected arms did not tend to coordinate. Namely, it was often observed that the two connected arms coordinated to move in a certain direction, whereas the other three arms coordinated to move in the opposite direction (Figure 2B and Supplementary Movie 2). In Experiment 3, the arms did not coordinate and, thus, locomotion was not observed. When a stimulus was applied to one of the arms, only the stimulated arm responded whereas the other arms did not respond (Figure 2C and Supplementary Movie 3). In Experiment 4, the four neurally connected arms coordinated to move away from the stimulus, whereas the neurally isolated arm did not coordinate with the other arms and, thus, did not contribute to locomotion (Figure 2D and Supplementary Movie 4).

In summary, the following findings were obtained from the above experiments.

1. Brittle stars tend to move to a direction opposite to the point where the nerve ring was cut.

2. Neurally isolated arms cannot coordinate with other arms.

3. Sensory input (i.e., stimulus) is transmitted to neurally connected arms and enables them to coordinate.

Thus, it is likely that neural connection in the nerve ring plays an important role in the determination of moving direction.

3. Model

Based on the above findings, we propose a mathematical model. We have previously proposed a decentralized control model for the inter-arm coordination and succeeded in reproducing the locomotion patterns of brittle stars in which several arms were amputated (Kano et al., 2017b). This model was implemented in a brittle star-like robot, and it adapted to physical damages like real brittle stars; thus, it likely captures the essence of brittle star locomotion. However, the moving direction is determined by the central command. Thus, we here modify our previous model so that the moving direction is determined in a self-organized manner at the nerve ring. We review our previous model in section 3.1, and then we propose the modified model in section 3.2.

3.1. Review of Our Previous Work

The schematic of the body system in our previous model (Kano et al., 2017b) is shown in Figure 3A. The body consists of a pentagonal central disc and five arms radiating from its vertices. Each arm has only two degrees of freedom, i.e., yaw and pitch joints. Each arm can detect the ground reaction force parallel to the ground. The reaction forces acting on the right- and left-hand sides of the ith arm are denoted by F_{R, i} and F_{L, i}, respectively.

FIGURE 3

Figure 3. Schematics of the previous model (Kano et al., 2017b) and the proposed model. (A) Body system. (B) Definitions of d and r_i. The a_i values are also shown schematically. (C) Evaluation of reaction force from the environment when a_i−1 > a_i > a_i+1. (D) Outline of the decentralized control mechanism. Detailed explanations are provided in the main text. (E) Schematic of the proposed model. Five water tanks are connected by tubes. Water level denotes a_i. Dashed lines denote ā_i. Black arrows indicate water flow. Schematics in (A,C,D) were reproduced from Kano et al. (2017b).

It is assumed that the desired moving direction of the center of the body is predetermined and is denoted by the vector d. On this basis, parameters a_i (i = 1, 2, 3, 4, 5) are defined as

$\begin{array}{ll}{a}_i={\text{r}}_i·\text{d},& (1)\end{array}$

where r_i denotes the vector pointing the proximal end of the ith arm from the viewpoint of the center of the disc. Thus, a_i is large when the proximal end of the ith arm is oriented toward the direction of motion from the viewpoint of the center of the central disc (Figure 3B).

Using the a_i values, each arm can evaluate whether the detected reaction force assists with propulsion toward the desired moving direction or not. In particular, U_{R, i} and U_{L, i}, which are defined as

$\begin{array}{lll}\hfill U_{R,i}& =& \text{max}\left[a_i-a_{i-1},0\right]F_{R,i},\\ \hfill U_{L,i}& =& \text{max}\left[a_i-a_{i+1},0\right]F_{L,i},& (2)\end{array}$

represent to what extent the detected reaction force from the right/left assists with propulsion toward the desired moving direction, and the ith arm can make the evaluation on the basis of the values of U_{R, i} and U_{L, i}. For example, let us consider the case in which the ith arm is oriented toward the moving direction, i.e., a_i−1 > a_i > a_i+1 (Figure 3C). When the ith arm experiences a reaction force from the left, U_{L, i} is positive and, thus, the reaction force assists with propulsion. Meanwhile, when the ith arm receives a reaction force from the right, U_{R, i} is zero and, thus, the reaction force does not assist with propulsion.

Based on the above, the torque generated at each joint is determined according to the following rule (Figure 3D). First, each arm moves randomly to detect the reaction force against the ground (Figure 3Di). If the reaction force assists with propulsion toward the desired moving direction, the arm pushes against the ground, i.e., a power stroke begins (Figure 3Dii). Then, a recovery stroke begins when the joint angle reaches a certain threshold (Figures 3Diii,iv). On the other hand, if the reaction force impedes propulsion toward the desired moving direction, no action is generated (Figure 3Dv). Detailed mathematical formulas are provided in the Appendix and Kano et al. (2017b)

3.2. Proposed Model

In the above-mentioned model, a_i is determined by using the predetermined desired moving direction d (Equation 1). However, because real brittle stars likely determine their moving direction by integrating sensory information detected at each body part, it is natural to consider that a_i is determined in a self-organized manner. Hence, here we modify the model, focusing on how to control a_i.

Unfortunately, a neurophysiological basis for the determination of a_i is lacking at the present stage. Hence, we model a_i phenomenologically: we control a_i by considering an analogy with water tanks connected with tubes (Figure 3E). In this analogy, five water tanks, each of which corresponds to each arm, are located on a plane and they are connected with tubes. The water level of the ith tank represents the value of a_i. The water levels between adjacent tanks tend to decrease because water flows from a tank with higher level to that with lower level; thus, a_i evolves in a diffusive manner. In each tank, water is added or removed so that the water level approaches the target level. The target level increases and decreases when attractant and aversive stimuli are added to the corresponding arm, respectively.

Thus, the time evolution of a_i is described as follows:

$\begin{array}{ll}\tau {\dot{a}}_i={\overline{a}}_i-a_i+D(a_{i-1}+a_{i+1}-2a_i),& (3)\end{array}$

where τ is a time constant and D denotes the diffusion coefficient, which is related to the diameter of the tubes in Figure 3E. The target water level ā_i is given by

$\begin{array}{ll}{\overline{a}}_i=c_0+s_i,& (4)\end{array}$

where c₀ is a positive constant and s_i denotes the stimulus applied to the ith arm, which is positive and negative in the case of attractant and aversive stimulus, respectively.

It is assumed in our model that a nerve ring cut corresponds to a tube cut and that water outflows from the cross-section of the cut tube. Thus, in the case of a nerve ring cut at the place between the ith and (i+1)th arms, for example, the time evolution of a_i is calculated by replacing a_i+1 on the right-hand side of Equation (3) with zero.

We note that the definitions of U_R,i and U_L,i are slightly changed from Equation (2) as follows:

$\begin{array}{lll}\hfill U_{R,i}& =& \text{tanh}\{{\kappa }_u(\text{max}[a_i-a_{i-1},0]F_{R,i})\},\\ \hfill U_{L,i}& =& \text{tanh}\{{\kappa }_u(\text{max}[a_i-a_{i+1},0]F_{L,i})\}.& (5)\end{array}$

The hyperbolic tangent functions are introduced to make the parameter tuning feasible and to well mimic the behavioral findings, yet this change is not essential for the determination of moving direction.

4. Simulation

To validate the proposed model, we performed simulation experiments. Experimental conditions the same as Experiments 1–4 in section 2 were examined. The parameter values, which were determined by trial-and-error, are as follows: τ = 0.2 s, c₀ = 1.0, D = 1.0 × 10³, λ = 2.0 s⁻¹, β_yaw = 2.0, β_pitch = 30.0, γ_yaw = 5.0, γ_pitch = 10.0, κ_u = 1.0 × 10², γ_s = 80.0, θ_th = π/6 rad. The proportional gains of the yaw and joints are 1.5 and 0.7 kgs⁻², respectively, while the derivative gains of the yaw and pitch joints are 0.6 and 0.0 kgs⁻¹, respectively.

The results are shown in Figure 4. In Experiment 1, the simulated brittle star tended to move in the direction opposite to the point where the nerve ring was cut (Figure 4A and Supplementary Movie 5). In Experiment 2, the connected two arms coordinated to move in a certain direction, whereas the other three arms coordinated to move in the opposite direction (Figure 4B and Supplementary Movie 6). In Experiment 3, the arms did not coordinate and, thus, locomotion was not observed. Locomotion was not initiated even when a stimulus was applied to one of the arms (Figure 4C and Supplementary Movie 7). In Experiment 4, the four neurally connected arms coordinated to move away from the stimulus, whereas the neurally isolated arm did not coordinate with other arms and, thus, did not contribute to locomotion (Figure 4D and Supplementary Movie 8). Thus, the behaviors of real brittle stars shown in section 2 were generally well reproduced by the proposed model.

FIGURE 4

Figure 4. Simulation results: (A) Experiment 1, (B) Experiment 2, (C) Experiment 3, and (D) Experiment 4. Points where the nerve ring is cut and points where an aversive stimulus is added are indicated by scissors and blue circles, respectively. In the snapshots, points where the nerve ring is cut are indicated by pink lines. Points where aversive stimuli are added are indicated by pink circles. Mass points are colored orange and blue when they contact with and lift off the ground, respectively.

However, there still exists a difference between the simulated and real brittle stars. Each arm of the brittle star in Experiment 3 and the neurally isolated arm in Experiment 4 looked relaxed and did not actively push against the ground (Figures 2C,D). Meanwhile, in the simulation, these arms generate torques and actively pushed themselves against the ground (Figures 4C,D). This is because the terms max[a_i − a_i−1] and max[a_i − a_i+1] in Equation (5) are always positive for the neutrally isolated arm.

5. Conclusion and Future Work

In this study, we have focused on a brittle star which can autonomously determine the moving direction and move toward it adaptively under various circumstances by using its primitive distributed nervous system. Based on behavioral findings from the locomotion of brittle stars whose nerve rings were cut in various ways (Clark et al., 2019), we proposed a phenomenological mathematical model in which the direction of movement is determined in a self-organized manner, using an analogy with water tanks connected by tubes. We demonstrated via simulations that the behavioral findings can be well reproduced. Thus, our model likely captures the essence for the determination of moving direction in brittle stars.

We believe that this study imparts deep insights to biologists. While the previous models on animal locomotion (Kimura et al., 2007; Schilling et al., 2013; Kano et al., 2017a,c) focused on adaptation to various circumstances, the proposed model enables autonomous determination of the moving direction as well as adaptation. Thus, it will help understand the mechanism underlying autonomous behaviors of animals. Moreover, from an engineering perspective, the proposed control mechanism will be helpful in designing fully autonomous robots that can determine their moving direction as well as adapt to the circumstances encountered in real time, and it can potentially be used for disaster scenarios.

However, several problems still remain. First, a biological basis for the proposed model is still lacking. In particular, the physical meaning of a_i from a neurophysiological basis is unclear. Second, there still exists a discrepancy between the simulated and real brittle stars. Third, it is still unclear how the control mechanism of brittle star locomotion is related to that of other animals. Further neurophysiological studies and mathematical modeling on this basis are needed in the future. Extension of the proposed model to describe the coupling between the inter- and intra-arm coordination also remains as a future work.

Data Availability

The raw data supporting the conclusions of this manuscript will be made available by the authors, without undue reservation, to any qualified researcher.

Author Contributions

TK and AI: contributed the initial conception. TK, DK, HA, and EC: proposed the mathematical model. DK: performed simulations. TK: wrote the manuscript. DK, HA, EC, and AI: contributed to manuscript revision.

Funding

This work was supported in part by the Japan Science and Technology Agency, CREST (JPMJCR14D5) and by a Grant-in-Aid for Scientific Research (B) (No. 16KT0099) from the Ministry of Education, Culture, Sports, Science, and Technology (MEXT), Japan.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors would like to thank Dr. Akira Fukuhara, Kotaro Yasui, Shura Suzuki, and Taishi Mikami of Tohoku University for their helpful suggestions.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnbot.2019.00066/full#supplementary-material

Supplementary Movie 1. The behavior of a brittle star in Experiment 1.

Supplementary Movie 2. The behavior of a brittle star in Experiment 2.

Supplementary Movie 3. The behavior of a brittle star in Experiment 3.

Supplementary Movie 4. The behavior of a brittle star in Experiment 4.

Supplementary Movie 5. Simulation result for Experiment 1.

Supplementary Movie 6. Simulation result for Experiment 2.

Supplementary Movie 7. Simulation result for Experiment 3.

Supplementary Movie 8. Simulation result for Experiment 4.

References

Arshavskii, Y., Kashin, S., Litvinova, N., Orlovskii, G., and Fel'dman, A. (1976a). Coordination of arm movement during locomotion in ophiurans. Neurophysiology 8, 404–410. doi: 10.1007/BF01063603

PubMed Abstract | CrossRef Full Text | Google Scholar

Arshavskii, Y., Kashin, S., Litvinova, N., Orlovskii, G., and Fel'dman, A. (1976b). Types of locomotion in ophiurans. Neurophysiology 8, 398–404. doi: 10.1007/BF01063602

PubMed Abstract | CrossRef Full Text | Google Scholar

Astley, H. (2012). Getting around when you're round: quantitative analysis of the locomotion of the blunt-spined brittle star, ophiocoma echinata. J. Exp. Biol. 215, 1923–1929. doi: 10.1242/jeb.068460

PubMed Abstract | CrossRef Full Text | Google Scholar

Clark, E., Kanauchi, D., Kano, T., Aonuma, H., Briggs, D., and Ishiguro, A. (2019). The function of the ophiuroid nerve ring: how a decentralized nervous system controls coordinated locomotion. J. Exp. Biol. 222:jeb192104. doi: 10.1242/jeb.192104

PubMed Abstract | CrossRef Full Text | Google Scholar

Cobb, J., and Stubbs, T. (1981). The giant neurone system in ophiuroids I. The general morphology of the radial nerve cords and circumoral nerve ring. Cell Tissue Res. 219, 197–207. doi: 10.1007/BF00210028

PubMed Abstract | CrossRef Full Text | Google Scholar

Kano, T., Sakai, K., Yasui, K., Owaki, D., and Ishiguro, A. (2017a). Decentralized control mechanism underlying interlimb coordination of millipedes. Bioinspir. Biomimet. 12:036007. doi: 10.1088/1748-3190/aa64a5

PubMed Abstract | CrossRef Full Text | Google Scholar

Kano, T., Sato, E., Ono, T., Aonuma, H., Matsuzaka, Y., and Ishiguro, A. (2017b). A brittle star-like robot capable of immediately adapting to unexpected physical damage. R. Soc. Open Sci. 4:171200. doi: 10.1098/rsos.171200

PubMed Abstract | CrossRef Full Text | Google Scholar

Kano, T., Suzuki, S., Watanabe, W., and Ishiguro, A. (2012). Ophiuroid robot that self-organizes periodic and non-periodic arm movements. Bioinspir. Biomimet. 7:034001. doi: 10.1088/1748-3182/7/3/034001

PubMed Abstract | CrossRef Full Text | Google Scholar

Kano, T., Yoshizawa, R., and Ishiguro, A. (2017c). Tegotae-based decentralised control scheme for autonomous gait transition of snake-like robots. Bioinspir. Biomimet. 12:046009. doi: 10.1088/1748-3190/aa7725

PubMed Abstract | CrossRef Full Text | Google Scholar

Kimura, H., Fukuoka, Y., and Cohen, A. (2007). Dynamic walking of a quadruped robot on natural ground based on biological concepts. Int. J. Robot. Res. 26, 475–490. doi: 10.1177/0278364907078089

CrossRef Full Text | Google Scholar

Matsuzaka, Y., Sato, E., Kano, T., Aonuma, H., and Ishiguro, A. (2017). Non-centralized and functionally localized nervous system of ophiuroids: evidence from topical anesthetic experiments. Biol. Open 6, 425–438. doi: 10.1242/bio.019836

PubMed Abstract | CrossRef Full Text | Google Scholar

Meyer, B., Ansorge, C., and Nakagaki, T. (2017). The role of noise in self-organized decision making by the true slime mold physarum polycephalum. PLoS ONE 12:e0172933. doi: 10.1371/journal.pone.0172933

PubMed Abstract | CrossRef Full Text | Google Scholar

Schilling, M., Hoinville, T., Schmitz, J., and Cruse, H. (2013). Walknet, a bio-inspired controller for hexapod walking. Biol. Cybern. 107, 397–419. doi: 10.1007/s00422-013-0563-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Watanabe, W., Kano, T., Suzuki, S., and Ishiguro, A. (2012). A decentralized control scheme for orchestrating versatile arm movements in ophiuroid omnidirectional locomotion. J. R. Soc. Interf. 7, 102–109. doi: 10.1098/rsif.2011.0317

CrossRef Full Text | Google Scholar

Appendix

The joint torque is determined according to proportional-derivative control, and the time evolution of the target joint angles of the ith arm is defined as follows:

$\begin{array}{ll}{\dot{\Phi }}_i=\lambda ({\Xi }_i+{\text{P}}_i+{\text{R}}_i-{\Phi }_i),& (\mathrm{A1})\end{array}$

where λ is a positive constant and ${\Phi }_i={[{\varphi }_{\text{yaw},i},{\varphi }_{\text{pitch},i}]}^{\text{T}}$ with ϕ_{yaw, i} and ϕ_{yaw, i} being the target angles for the yaw and pitch joints, respectively. Note that the signs of the yaw and pitch joint angles are taken as positive when the arm bends leftward and upward with respect to the central disc, respectively. The vectors Ξ_i, P_i, and R_i contribute to the generation of the noise, power stroke, and recovery stroke, respectively. Hereafter, we describe mathematical formulas for these terms.

The noise vector Ξ_i is defined as

$\begin{array}{ll}{\Xi }_i=\left[\begin{array}{c}{\alpha }_{\text{yaw}}{\xi }_{\text{yaw},i}\\ {\alpha }_{\text{pitch}}{\xi }_{\text{pitch},i}\end{array}\right],& (\mathrm{A2})\end{array}$

where α_yaw and α_pitch are positive constants and ξ_{yaw, i} and ξ_{pitch, i} are uniform random numbers in the range [−1, 1], which vary with a fixed interval. It should be noted that the noise is essential because the robot cannot interact with the environment in the absence of the noise. Namely, if the noise is absent, the driving force of the robot does not exist, and the robot cannot detect the reaction force against the ground.

The power stroke vector, P_i is defined as

$\begin{array}{ll}{\text{P}}_i=\left[\begin{array}{c}-{\beta }_{\text{yaw}}(U_{R,i}-U_{L,i})\\ -{\beta }_{\text{pitch}}(U_{R,i}+U_{L,i})\end{array}\right],& (\mathrm{A3})\end{array}$

where β_yaw and β_pitch are positive constants, and U_R,i and U_L,i are defined in Equation (2) in the main text.

The recovery stroke vector R_i is defined as

$\begin{array}{ll}{\text{R}}_i=\left[\begin{array}{c}{\gamma }_{\text{yaw}}(S_{R,i}-S_{L,i})\\ {\gamma }_{\text{pitch}}(S_{R,i}+S_{L,i})\end{array}\right],& (\mathrm{A4})\end{array}$

where γ_yaw and γ_pitch are positive constants, and S_{R, i} and S_{L, i} are given by

$\begin{array}{l}{S}_{R,i}=\frac{1}{2}[1+\text{tanh}\{{\kappa }_s(-{\theta }_{\text{yaw},i}-{\theta }_{th})\}],\\ S_{L,i}=\frac{1}{2}[1+\text{tanh}\{{\kappa }_s({\theta }_{\text{yaw},i}-{\theta }_{th})\}],& (\mathrm{A5})\end{array}$

where κ_s is a positive constant, and θ_{yaw, i} is the yaw joint angle of the ith arm. When the joint angle reaches the threshold angle θ_th or −θ_th, S_{L, i} or S_{R, i} increases, respectively; thus, the target joint angle changes, causing the ith arm to move in the opposite direction by lifting off from the ground.

In summary, the noise vector Ξ_i is essential for detecting the reaction force against the ground (Figure 3Di). The power stroke vector P_i is 0 unless the ith arm detects a reaction force that assists with propulsion; however, P_i acts in such a way that the ith arm can push against the ground when it detects an assistive reaction force (Figure 3Dii). The recovery stroke vector R_i is nearly equal to 0 unless the yaw joint angle of the ith arm exceeds the threshold ±θ_th; however, R_i acts in such a way that the ith arm moves itself forward when the yaw joint angle reaches the threshold (Figures 3Diii,iv).