A Pathological Condition Affects Motor Modules in a Bipedal Locomotion Model

1. Introduction

Bipedal locomotion is a basic motor activity. Several studies suggest that animal locomotion is controlled by central pattern generators (CPGs) in the spinal cord (Grillner, 1975). CPGs provide a rhythmic motor activity across multiple muscles in a coordinated manner in both space and time (Guertin, 2009). Specifically, they generate coordinated flexor-extensor muscles' activity, and adapt gait patterns to environmental changes by using sensory feedback. The motor modules hypothesis (Dominici et al., 2011; Lacquaniti et al., 2012) proposes that the motor system groups muscles into a smaller number of functional modules based on CPGs. Physiological experiments suggest that human locomotion may involve the motor modules of synergistic muscle activations. In healthy adults, 4 or 5 motor modules are activated independently while walking (Ivanenko et al., 2004, 2005). Each module corresponds to a key phase of the gait cycle (Neptune et al., 2009). The 1st module acts to support the body in the early stance. The 2nd module acts to support both the body and propulsion in the later stance. The 3rd module contributes to decelerating the leg in the early and later part of the leg swing. The 4th module absorbs leg energy in the late part of the leg swing. Patients with neural disorders showed different combinations of motor modules, including the decrease of the number of motor modules (Ting et al., 2015). Previous studies demonstrated that locomotor rehabilitation improved walking ability in stroke patients while increasing the number of motor modules (Routson et al., 2013; Ting et al., 2015; Ferrante et al., 2016). Chronic stroke patients exhibit abnormal locomotion patterns, and 2 or more motor modules are merged in a single module, creating a timing overlap (Clark et al., 2010; Routson et al., 2013). These results suggest that motor modules are demonstrated as physiological markers of the status of patients with neural disorders (Cheung et al., 2012; Ting et al., 2015). On the contrary, some studies showed that locomotion recovery was not associated with changes in the number of motor modules (Hashiguchi et al., 2016; Tan et al., 2018). Stroke patients in the sub-acute phase showed a pattern of motor modules similar to those of healthy controls (Gizzi et al., 2011). Thus, how the nervous system in chronic stroke patients alters the motor modules remains unclear.

To address this question, we employed computer simulation of bipedal locomotion model (Taga et al., 1991; Taga, 1995; Ogihara and Yamazaki, 2001; Hase and Yamazaki, 2002; Jo and Massaquoi, 2007; Aoi et al., 2010, 2012; Allen et al., 2013). Specifically, we aimed to examine how the nervous system alters the organization of motor modules by building a two-dimensional bipedal locomotion model consisting of a musculoskeletal system and hierarchical CPGs. First, we built a normal locomotion model with 5 motor modules and various feedback inputs. By setting internal parameters by a genetic algorithm (GA) appropriately, the model succeeded to walk robustly. Then, we simulated a pathological condition. Motor evoked potentials (MEPs) are widely used to investigate the physiology of corticospinal condition (Hendricks et al., 2002). A previous study has shown a correlation between weaker lower-limb MEPs and lower gait ability (Piron et al., 2005). Therefore, we weakened output signals to the muscles on one leg to simulate the weak lower-limb MEPs. By this manipulation, the model failed to walk anymore. Then, we refit parameters by running additional GAs. Specifically, we modified one of the two sets of parameters by GAs: (a) reflex parameters to simulate patients who can compensate locomotion only by adapting reflexes, and (b) CPG parameters to simulate patients who need to adapt the controller itself for compensation. In both cases, the model succeeded to walk again, and found the only in the latter case, motor modules were merged.

2. Materials and Methods

2.1. Skeletal Model

Our two-dimensional skeletal model consists of 7 rigid links that represent the head, arms, torso (HAT), thighs, shanks, and feet (Figure 1A). The length, mass and moment of inertia for each segment are taken from Jo and Massaquoi (2007) as in Table 1.

FIGURE 1

Figure 1. Model overview. (A) Schematic representation of the proposed neuromuscloskeletal model. Left panel: Musculoskeletal model. (1) gluteus maximus (GM); (2) iliopsoas (IL); (3) biceps femoris long head (BFL); (4) rectus femoris (RF); (5) biceps femoris short head (BFS); (6) vastus (VA); (7) gastrocnemius (GC); (8) soleus (SO); (9) tibia anterior (TA). HAT, head, arms, and torso. Right panel: the neural system is a central pattern generator (CPG) model, which is organized hierarchically with a rhythm generator (RG) network and a pattern formation (PF) network. Two out of the four oscillators in RG (surrounded by the left dashed circle) induce phasic and rhythmic activity patterns for five motor modules in PF that activate muscles for the left leg (colored arrows from the right to left panels), whereas the other two oscillators in RG (surrounded by the right dashed circle) induce another five motor modules in PF for the right leg (arrows are not shown). (B) Hierarchical relationship between RG and PF. Two neurons in RG network generate five activity patterns with different phases in PF network. When the outputs of the RG network reach certain values (dashed circles), PF neurons are activated. One PF neuron represents one motor module.

TABLE 1

Table 1. Skeletal model parameters.

Each joint is modeled as a pin joint and has a linear viscous element. The coefficients of viscosity for the hip, knee, and ankle joints are 1.09, 3.17, and 0.943 Nms/rad, respectively (Aoi et al., 2010). The knee and ankle joint are locked to prevent hyperextension or hyperflexion.

The spring and damper coefficients are 2.0 × 10³Nm/rad and 5.0 × 10² Nms/rad for the knee joints, as well as 2.0 × 10³Nm/rad and 5.0 × 10² Nms/rad for the ankle joint. When the heels or toes make contact with the ground, they receive the ground reaction forces produced by springs and dampers. The spring and damper coefficients are 5.0 × 10³N/m and 1.0 × 10³ Ns/m horizontally and 2.5 × 10⁴N/m and 5.0 × 10² Ns/m vertically.

The equations of motion are derived by means of the Newton-Euler method (Taga et al., 1991; Taga, 1995). The general form of the equations is described as

$\begin{array}{ll}\ddot{x}=P(x)F+Q(x,\dot{x},GRF(x,\dot{x}),T_r(F_m)),& (1)\end{array}$

where x is a (21 × 1) vector of the inertial positions of 7 links and the inertial angles of 7 links, P is a (21 × 12) matrix, F is a (8 × 1) vector of constraint forces, Q is a (20 × 1) vector, GRF is a (8 × 1) vector of ground reaction forces on the feet; T_r is a (6 × 1) vector of torques, and F_m (18 × 1) is a vector of the muscle tensions to be explained in the next section.

Constraint forces on the joints are eliminated by using equations of kinematic constraints given as follows:

$\begin{array}{ll}C(x)\ddot{x}=D(x,\ddot{x})& (2)\end{array}$

where C is a (12 × 21) matrix and D is a (12 × 1) vector. By eliminating F from Equations (1) and (2), we obtain

$\begin{array}{l}\ddot{x}=P(x){\left[C\left(x\right)P\left(x\right)\right]}^{-1}[D\left(x,\ddot{x}\right)\\ -C\left(x\right)Q\left(x,\dot{x},GRF\left(x,\dot{x}\right),T_r\left(F_m\right)\right)]\\ +Q\left(x,\dot{x},GRF\left(x,\dot{x}\right),T_r\left(F_m\right)\right).& (3)\end{array}$

This equation can be numerically integrated, given the muscle tensions F_m.

2.2. Muscular Model

We used 18 principal muscles; 9 for each legs (Figure 1A): gluteus maximus (GM), iliopsoas (IL), biceps femoris long head (BFL), rectus femoris (RF), biceps femoris short head (BFS) vastus (VA), gastrocnemius (GC), soleus (SO), tibia anterior (TA). A muscle receives a signal from the corresponding α-motoneuron and generates muscle tension depending on the force-length and force-velocity relationship. We used the mathematical model described by Ogihara and Yamazaki (2001), as follows:

$\begin{array}{l}{F}_m={\bar{F}}_m^{\text{CE}}·k({\xi }_m)·h({\eta }_m)·{\alpha }_m+F_m^{\text{PD}}+F_m^{\text{PE}}\\ k({\xi }_m)=0.32+0.71exp\left[-1.112({\xi }_m-1)\right]sin\left[3.722({\xi }_m-0.656)\right]\\ h({\eta }_m)=1+tanh(3.0{\eta }_m)\\ F_m^{\text{PD}}=c_m^{\text{PD}}{\dot{L}}_m\\ F_m^{\text{PE}}=k_m^{\text{PE}}\left\{exp\left[15(L_m-{\bar{L}}_m)\right]-1\right\},& (4)\end{array}$

where F_m is the muscle tension generated at the mth muscle, ${\bar{F}}_m^{\mathrm{\text{CE}}}$ is the maximum muscle tension by contractile element (CE), k(ξ_m) is the force-length relationship, h(η_m) is the force-velocity relationship, α_m is the stimulation signal from the corresponding α-motoneuron (0 ≤ α_m ≤ 1), $F_m^{\mathrm{\text{PD}}}$ and $F_m^{\mathrm{\text{PE}}}$ are the forces generated by the damping and elastic elements. ξ_m and η_m are the normalized muscle length and contractile velocity divided by the muscle optimal length ${\bar{L}}_m$ and maximum muscle contractile velocity ${\bar{\dot{L}}}_m$, that is, where ${\xi }_m=L_m/{\bar{L}}_m$, ${\eta }_m={\dot{L}}_m/{\bar{\dot{L}}}_m$, L_m, and ${\dot{L}}_m$ are the muscle length and the contractile velocity, respectively. $c_m^{\mathrm{\text{PD}}}$ is the viscous coefficient, and $k_m^{\mathrm{\text{PE}}}$ is the coefficient of the elastic element.

2.3. Nervous System Model

We used Matsuoka neuron model (Matsuoka, 1987) as follows:

$\begin{array}{ll}\begin{array}{c}{\tau }_i\dot{u_i}=-u_i+{\displaystyle \sum _{j=1}^n}{w}_{ij}^{\text{CPG}}{y}_j-\beta v_i+u_{\theta }+{\text{Feed}}_i,\\ {\tau }_i^{\prime }{\dot{v}}_i=-v_i+y_i,\hfill \\ y_i=\text{max}(0,u_i),\hfill \end{array}& (5)\end{array}$

where u_i is the inner state of the ith neuron, v_i is a variable that represents the self-inhibition effect of the ith neuron. τ_i and ${\tau }_i^{\prime }$ are time constants, β is a coefficient, w_ij is a connecting weight from the jth neuron to the ith neuron. u_θ is an external input with a constant rate, and Feed_i is a feedback signal from the musculoskeletal system. Parameter values are show in Appendix.

We built a CPG model that is a two-layer hierarchical network composed of a rhythm generator (RG) network and a pattern formation (PF) network based on Li et al. (2013) (Figure 1A). The RG is composed of four Matsuoka neurons mutually inhibited to generate rhythm, whereas, a PF network contains five Matsuoka neurons with mutual and self-inhibition that corresponds to five motor modules. In response to rhythmic activation of the RG as an external input, the five modules are activated one by one sequentially with different phases (Figure 1B). PF neurons issue motor commands to α-motoneurons, which in turn activate muscles. α-motoneurons also receive feedback signals from various reflexes such as posture reflex and crossed extension reflex. α-motoneuron output α_m and reflex output Reflex_m are given as follows:

$\begin{array}{ll}{\alpha }_m=w_m^{\text{condition}}\frac{1}{1+\exp (-4({\displaystyle {\sum }_{i=1}^5{w}_{mi}^{\alpha }}{\text{PF}}_i+{\text{Reflex}}_m))},& (6)\end{array}$

$\begin{array}{ll}{\text{Reflex}}_m=\{\begin{array}{cc}{\displaystyle {\sum }_j(c_{mj}{\theta }_j+c_{mj}^{\prime }{\dot{\theta }}_j)+{\text{POS}}_m}& \text{GRF}>0,\hfill \\ 0\hfill & \text{otherwise,}\hfill \end{array}& (7)\end{array}$

where $w_m^{\mathrm{\text{condition}}}$, $w_{mi}^{\alpha }$, c_mj, and $c_{mj}^{\prime }$ are the weight coefficients, PF_i is the output of PF neuron. θ_j is the joint angle (j ∈ {hip, knee, and ankle} for each leg), POS_m is a posture control, and GRF is the vertical ground reaction force.

2.4. Parameter Search

Our model has 56 free parameters ($w_{mi}^{\alpha },c_{mj},c_{mj}^{\prime },{\mathrm{\text{POS}}}_m,{\mathrm{\text{Feed}}}_i$) need to be fixed for stable locomotion. We employed standard genetic algorithms (GAs) for searching these parameters (Noori et al., 2017). GAs are search algorithms based on the biological genetic mechanisms such as selection, crossover, and mutation (Hase and Yamazaki, 2002). The search mechanism is built on the interaction between individuals and the external environment. GAs comprise a set of individuals (the population) and a set of the genetic operators. The individuals have genes that are the potential solutions for the problem. GAs generate a sequence of populations by using genetic operators (selection, crossover, and mutation) among individuals. Individuals that achieved the highest evaluation can survive and generate children. The block diagram of a GA is presented in Figure 2. For the evaluation function, we used the following equation:

$\begin{array}{ll}{w}_{d}D+w_{s}S\to \text{max}& (8)\end{array}$

where w_d and w_s are the weight coefficients, D is the distance until the model falls down, and S is the number of steps taken while walking. We initialized 50 individuals randomly. We chose a one-point crossover with a probability of 0.7 and mutation with a probability of 0.1. We carried out impairment simulation for 5 times with 5 different seeds of the random number generator to examine the simulation results are unique. We confirmed that the model achieved stable bipedal locomotion for 5 s, and locomotion patterns with different seeds did not differ qualitatively. Essentially, humans acquire walking by trial and error. We mimicked this using GAs.

FIGURE 2

Figure 2. Block diagram of genetic algorithms. Initialization creates a population of random individuals. The evaluation phase tests individuals in the population for fitness. After that, a new generation is created through selection, crossover and mutation. This process is then repeated the required number of times was completed.

2.5. Normal and Pathological Locomotion Model

Normal locomotion was simulated by fitting the 56 free parameters by GAs while setting $w_m^{\mathrm{\text{condition}}}=1.0$ in Equation (3) for any m. After the model acquired normal locomotion, we simulated a pathological condition such as stroke. Specifically, we set $w_m^{\mathrm{\text{condition}}}=0.8$ for m = 1, ⋯, 6 and 0.6 for m = 7, ⋯, 9 to simulate weakening of MEP signals observed in stroke patients (Piron et al., 2005). Immediately after this manipulation, the model did not succeed to walk even for one step. Then, we examined two “simulated rehabilitation” scenarios for the model. The first scenario is that under the assumption of that the patients could recover the locomotion by compensating their reflexes, we refitted the 28 parameters for reflexes ($c_{mj},c_{mj}^{\prime }$). We call this scenario “reflex-compensation model.” The other scenario is that under the assumption of that the patients restore the locomotion by adapting the controller itself, we refitted the 48 parameters for feedback inputs to the CPG (Feed_i). We call this scenario “CPG-compensation model.” In both scenarios, the model reacquired the locomotion successfully, whereas the motor modules exhibited different dynamics.

2.6. Implementation

We implemented GAs using a message passing interface (MPI), which is a library for parallel computing. All programs were written in C language, and the fourth-order Runge-Kutta method was used to solve differential equations numerically. The step-size of time was set as 0.1 ms. In addition, HAT is controlled by both muscles and proportional-derivative control to maintain the model upright. The proportional-derivative control is determined by the angular velocity and the difference of the vertical angle. The angular velocity and angle coefficients are 1.0 × 10² and 1.0 × 10³, respectively.

3. Results

3.1. Acquisition of Normal Locomotion Pattern

After 3,000 generations of GAs, the model acquired stable bipedal locomotion, as shown in Figure 3A. The locomotion pattern resembles that of the human biped qualitatively. Figure 3B displays the muscle activations. The iliopsoas (IL) produced the activities in the middle of the gait cycle. The gluteus maximus (GM) and vastus (VA) achieved muscle activity at the beginning and end of the gait cycle. The tibia anterior (TA) obtained the activities at the beginning and middle to the end of the gait cycle. The soleus (SO) and gastrocnemius (GC) produced synchronous activity in the middle of the gait cycle. These muscle activations are consistent with the measured data (Ivanenko et al., 2005). Figure 3C shows the joint kinematics. The waveforms are similar to those previously reported in Ivanenko et al. (2005). The stance phase of the left leg and that of the right leg in gait cycle are 55.6% and 54.5%, respectively, showing that the kinematics of both legs are nearly symmetric. However, several differences between the simulation results and measured data were observed. For example, the RF produced the peak of the muscle activities in the middle of the gait cycle in the measured data. The period of the flexor angle on the knee joint was longer than the measured data.

FIGURE 3

Figure 3. The normal locomotion model. (A) Stick diagram of the generated normal locomotion. This is a snapshot of every 0.1 s. (B) Muscle tension (vertical axis) of each muscle during the gait cycle (horizontal axis). A gait cycle is the time period of movement during locomotion from the time when the heel of one foot hits the ground to when the heel of that same foot hits the ground again. Abbreviations as in the main text. (C) Joint angles in relation to the gait cycle for the left (left panel) and right (right panel) legs. HC and TO indicate the times for heel contact and toe off, respectively.

3.2. Walking Pattern of Pathological Condition

To build the pathological locomotion model, we weakened the output neural signal to the muscle on the left leg in the normal locomotion model. The pathological locomotion model fell down immediately when the model attempted to walk. After 200 generations of GA, the reflex-compensation model acquired bipedal locomotion by changing the reflex parameters (Figure 4A). After 1,000 generations of GA, the CPG-compensation model acquired bipedal locomotion by changing the CPG parameters (Figure 5A). In the reflex-compensation model, the step size was smaller than that in the normal locomotion (Figure 4A). In the muscle activations for the affected (left) leg, the activity in IL, SO, and GC observed in the normal locomotion disappeared (Figure 4B). In addition, the activities of the muscles shifted later in the IL, GM, RF, VA, BFS, and TA. The joint angles of the left leg (affected leg) is larger than that of the right leg (unaffected leg) (Figure 4C). The stance phase of the left leg and the right leg in the gait cycle are 60.9% and 50.9%, respectively, suggesting that the locomotor pattern is symmetric. On the other hand, in the CPG-compensation model, the step size varied across gait cycles compared with the normal locomotion (Figure 5A). In other words, the walking pattern of the pathological locomotion model was unstable. In the muscle activations for the affected (left) leg, the initial activity in VA, TA, and BFL observed in the normal locomotion disappeared. In contrast, the initial activity in the GM, RF, and BFS became larger than that in the normal locomotion model. Those muscle activities were generated by the reflex (Equation 7) to compensate for the disappearance of the initial activity in the muscles. In addition, the phases of the activities were advanced in the VA, BFL, TA, SO, and GC (Figure 5B). Related to the phase advances, the stance phase of the left leg (affected leg) became shorter than that of the right leg (unaffected leg) (49.9% and 75.2%, respectively), showing an asymmetric locomotion (Figure 5C).

FIGURE 4

Figure 4. The reflex-compensation model. (A) Stick diagram of the generated reflex-compensation locomotion. (B) Muscle tension of each muscle during the gait cycle in the reflex-compensation model (red-shaded area) and the normal locomotion model (gray line). Arrows indicate the disappearance of muscle activity. Triangles exhibit that the activities of the muscles shifted later. Abbreviations as in the main text. (C) Joint angles in relation to the gait cycle for the left (left panel) and right (right panel) legs in the reflex-compensation model (solid lines) and the normal locomotion model (dashed lines). Conventions as in Figure 3A.

FIGURE 5

Figure 5. The CPG-compensation model. (A) Stick diagram of the generated CPG-compensation locomotion. (B) Muscle tension of each muscle during the gait cycle in the CPG-compensation model (red-shaded area) and the normal locomotion model (gray line). Arrows indicate the disappearance of initial muscle activity. Asterisks show that the initial activity of muscles is larger than that in the normal locomotion model. Triangles exhibit that the activities of the muscles shifted earlier. (C) Joint angles in relation to the gait cycle for the left (left panel) and right (right panel) legs in the CPG-compensation model (solid lines) and the normal locomotion model (dashed lines). Arrows indicate the shifted stance phase compared to the normal locomotion model. Both legs are asymmetrical throughout the gait cycle. Conventions as in Figure 3A.

3.3. Comparison of Joint Angles Between Normal and Pathological Walking

Figure 6 illustrates angle-angle plots to confirm the error of gait trajectories obtained from the normal locomotion model and pathological locomotion models. These figures show that the trajectories for the reflex-compensation model were similar to these for the normal locomotion model (Figure 6B), whereas these for the CPG-compensation model was not (Figure 6C), especially ankle-hip and ankle-knee coordinates. Moreover, these error plots for the CPG-compensation model display that the angles of ankles were not periodic. The unnatural movements imply compensation for acquiring bipedal locomotion. Such a different ankle control strategy seems necessary for stroke patients to walk. In fact, various ankle-foot orthoses have been developed to adjust ankle movements for stroke patients (Ohata et al., 2011). Our simulation result on the CPG-compensation model supports this observation.

FIGURE 6

Figure 6. Angles of each joint for left and right legs (left and right panels, respectively) during locomotion in the (A) normal, (B) reflex-compensation, and (C) CPG-compensation models. The horizontal axis and vertical axis are joint angles (radian). Gray lines represents the angles for the normal locomotion model for comparison.

3.4. Comparison of Motor Modules Between Normal and Pathological Walking

Finally, we compared the activity patterns of the motor modules, 5 for left and 5 for right legs, in the normal and pathological conditions. Figure 7A illustrates them in the normal condition. All modules exhibit rhythmic activity with different phases; and thus, they become active one by one sequentially. In particular, the offset of the activity in the 5th module (the end of a locomotion cycle) is followed by the onset in the 1st module (the start of the next cycle) continuously. Figure 7B shows them in the reflex-compensation model. In this case, all the modules exhibited activities one by one sequentially as in the normal condition, although the phase of the 2nd module in both legs was delayed than that in the normal condition. Figure 7C shows the activity patterns in the CPG-compensation model. In this case, the 1st module for the affected (left) leg did not show marked activity. Moreover, in the 3rd, 4th, and 5th modules, the phases were advanced. We analyzed how long 2 motor modules were activated simultaneously, and plotted the durations between the 2nd and 3rd modules, those between the 3rd and 4th modules, and those between the 4th and 5th modules (Figure 8). The duration in the CPG-compensation model was the longest, whereas these were comparable in the Reflex-compensation model and the normal model (Figure 8B), suggesting that the activations of motor modules in the CPG-compensation model overlap temporally as if they are merged as a single module (Figure 8B). However, the overlap might be caused by a prolonged activity of each motor module. To address this, we analyzed the duration of 2nd, 3rd, and 4th modules for each condition by fitting the activities of the modules by a Gaussian function (Figure 8C). The Full-width at half maximum (FWHM) of the waveforms in the normal model, reflex-compensation model, and CPG-compensation model were 12.467, 13.730, and 13.528, respectively. Thus, the motor modules in the CPG-compensation model showed the same duration with those in the normal model, suggesting that the active durations are not extended. These results conclude that in the CPG-compensation model, motor modules are co-activated in time more than those in the normal and Reflex-compensation models, as if the modules act as a “merged” single module. The merging of motor modules is also observed in the unaffected leg, suggesting that the deficit in one leg affects the other leg.

FIGURE 7

Figure 7. Activity patterns of the motor modules for left and right legs (left and right panels, respectively) during locomotion in the (A) normal, (B) reflex-compensation, and (C) CPG-compensation model. The horizontal axis is time (s) and the vertical axis activity (arbitrary units). Gray lines represents the activity for the corresponding module in the normal model for comparison.

FIGURE 8

Figure 8. Analysis of activity patterns of the motor modules. (A) Schematic representation of activity patterns of the motor modules. One locomotion cycle is an activity period during the 1st to 5th motor modules. The locomotion cycle was set to 2π radian (dashed lines) to compare different locomotion cycles in each model. A duration of co-activation was defined when each module was active at the same time (the duration for modules 3 vs. 4 was shown). (B) The average of the co-activation for 3 locomotion cycles in each model. The horizontal axis is a comparison of motor modules and the vertical axis is the duration of co-activation. The colors of gray, blue, and red indicate normal model, reflex-compensation model, and CPG-compensation model, respectively. (C) The waveforms by a Gaussian function for the 3 locomotion cycles on the 2nd, 3rd, and 4th motor modules are displayed as normal model (gray), reflex-compensation model (blue), and CPG-compensation model (red).

4. Discussion

In this study, we carried out computer simulation of bipedal locomotion in normal and pathological conditions. In normal condition, our neuromuscloskeletal model, composed of a lower body with 7 links and 18 muscles, and a CPG controller with RG and PF organized hierarchically, walked successfully after fitting the internal parameters through a GA algorithm. In pathological conditions, input signals from the controller to one side of the leg were weakened, according to experimental results (Piron et al., 2005). This manipulation led to the immediate failure of stable walking. Then, we examined two scenarios to recover the stable walking. In one situation, parameters for proprioceptive feedback inputs were refitted, which simulates patients compensating locomotion by adapting reflexes against proprioceptive feedback signals. In this scenario, the model successfully acquired the stable walking again, and the activity patterns of motor modules were unaffected against the manipulation, although there were some marked differences in muscle activity patterns. In the other scenario, in which parameters for the CPG were refitted to simulate patients compensating the locomotion by adapting the controller, phases were advanced across multiple modules, and the modules tended to became active with identical phases, as if they were “merged” as a single module. These results suggest that if the compensation was made by adapting the reflex against feedback signals as light on sub-acute stroke patients would do, the motor modules were unaffected, whereas if the compensation was made by adapting the controller as severe on chronic patients would do, the modules were merged (Figure 9). These observations are consistent with experimental findings (Clark et al., 2010; Gizzi et al., 2011).

FIGURE 9

Figure 9. Hypothetical scheme of compensation in human locomotion. The left side is the reflex feedback loop. When the pathological patients acquired bipedal locomotion by modifying the reflex loop, their motor modules were not merged similar to sub-acute stroke patients (Gizzi et al., 2011). The right side is the CPG feedback loop. If these motor modules were merged, this suggests the patients that had changed their CPG feedback loop to acquire bipedal locomotion, as observed in chronic stroke patients (Clark et al., 2010).

4.1. Normal Locomotion Model

The model is built based on a musculoskeletal model (Taga et al., 1991; Taga, 1995; Ogihara and Yamazaki, 2001) and a CPG model (Jo and Massaquoi, 2007; Aoi et al., 2010, 2012). The resulting model contains 56 free parameters to be fitted to acquire stable walking. Previous studies have fitted these parameters by hand (Jo and Massaquoi, 2007), reinforcement learning (Matsubara et al., 2006; Li et al., 2013), and GAs (Ogihara and Yamazaki, 2001; Hase and Yamazaki, 2002). Notably, Aoi et al. (2019) compared gait patterns of healthy subjects with a computer simulation model with a CPG controller, where the 69 parameters were hand-tuned. We employed a GA algorithm for fitting the parameters that maximizes the walking distance and the number of steps. The GA succeeded to find the appropriate values for the 56 parameters. Eventually, our model acquired a symmetrical locomotion pattern (Figure 3A), which is consistent with normal human locomotion pattern (Ivanenko et al., 2005).

4.2. Pathological Locomotion Model

We then examined two possible scenarios for stroke patients to recover stable locomotion, reflex-compensation and CPG-compensation, where the former assumes that patients modify the reflexes while the latter they modify controller. We observed that in the former, the motor modules were not merged, whereas in the latter they were. Where does the different come from? Allen et al. (2013) built a pathological gait model based on the stroke patient data (Clark et al., 2010), and analyzed how different module patterns affect the locomotion. Specifically, they demonstrated that different merge combinations of motor modules results in difference locomotion impairments. However, they did not consider how such merging of motor modules would occur in stroke patients. On the other hand, we observed that CPG-compensation model showed the merge of motor modules to reacquire stable locomotion, suggesting that the merge occurs internally through the reacquisition process of locomotion. Meanwhile, we also observed that our reflex-compensation model did not merge the motor modules. This results seems consistent with the experimental findings by Gizzi et al. (2011), showing that sub-acute stroke patients did not show the merge of motor modules. These results suggest that depending on the duration after stroke, the patients would take different strategies to compensate their locomotion behaviors through the reacquisition process. The spinal cord network has the functions of adaptation by the feedback signals (Rossignol et al., 2006). In stroke patients, spinal cord is intact in general. For this reason, we propose that when the post-stroke period is short, the reflex feedback loop is affected, whereas when the duration is long, the CPG feedback loop is affected (Figure 9).

4.3. Implications for Adaptive Robot Locomotion

CPG-based bipedal robots change their locomotion patterns adaptively against environmental changes to some extent owing to the entrainment of the dynamical systems. For example, CPG-based robots can keep walking on a flat ground as well as a slope (Taga et al., 1991; Taga, 1995; Ishiguro et al., 2003; Matsubara et al., 2006; Aoi et al., 2010; Li et al., 2013, 2017) by sensing the acceleration change provided by proprioceptive feedback inputs to the controller. However, this ability requires an intact musculoskeletal system. When the system malfunctions, stable locomotion pattern would not be expected anymore. In this study, we demonstrated that even if the system malfunctions, as simulated stroke conditions, the system could recover the stable locomotion again by modifying internal parameters. However, depending on the choice of parameters to update, additional parameter search takes longer relearning time, and the strategy for restoring the locomotion patterns differs. In particular, motor modules are merged so that robots would exhibit only poor locomotion patterns in a certain condition. These results suggest that not all parameters should be updated to restore the locomotion ability.

4.4. Impact on Online Rehabilitation

Previous studies have shown that online rehabilitation has potentially useful attributes (Velliste et al., 2008). Therefore, various methods have been suggested for detecting motions of patients easily and accurately (Naseer et al., 2014; Holtzer et al., 2015; Khan et al., 2018). In particular, Khan et al. (2018) proposed a novel methodology that controls a prosthetic leg computing model using functional near-infrared spectroscopy (fNIRS) signals with little errors. This result can be effectively used for the rehabilitation of lower-limb amputees and patients with paralysis. This system currently supports only a few commands to generate motor torques to move forward or backward because of a crude resolution of fNIRS signals. In spite of this, more commands will be desirable. The fNIRS signals are measured during locomotion on a treadmill, and so the signals would reflect activities of CPGs within the brain. We could use our CPG model as a model of the generator of the fNIRS signals to enhance the signal resolution.

4.5. Limitations of This Study

Our musculoskeletal model is restricted in 2D, and the CPG model is abstracted mathematically. Thus, the model would not be suitable to study more detailed locomotion movements in a 3D space, as demonstrated by Kim et al. (2011) for example, or neuronal activity during locomotion in the brain. Rather, our model provides a crude closed-loop system model for human locomotion that can be somehow manipulated to simulate pathological situation. We will release the source code of our model under a opensource license to let researchers to elaborate the present model in future.

4.6. Suggestions for Rehabilitation Therapy

Hashiguchi et al. (2016) reported that the reduced number of motor modules in stroke patients was recovered to the original level by the rehabilitation for one month, and the tendency of the restoration was correlated with the improvement of muscle strengths and gait patterns by the rehabilitation. When the muscle strengths were weak, the patients would be impossible to compensate locomotion only by the reflexes, and so would need to change the motor strategy by changing the CPGs itself. This observation seems consistent with the present study, because compensation by the reflex did not affect motor modules, whereas that by the CPG did. Robot assisted locomotor training is a useful means for treatment of impaired locomotion (Jezernik et al., 2003; Kawamoto et al., 2013; Molteni et al., 2017; Watanabe et al., 2017) and the devices have been drastically improved to date (Beckerle et al., 2017; Gandolla et al., 2018; Mummolo et al., 2018). Tan et al. (2018) reported that robotic-assisted locomotor training for stroke patients improves walking speed, step cadence, stance duration percentage of gait cycle, but does not increase the number of motor modules. This implies that if more motor modules are recruited, the realized locomotion will be more sophisticated. Summarizing, stroke patients should be engaged with muscle exercises as well as gait training so that they could compensate the locomotion only by the reflex.

5. Conclusion

In this study, we investigated what conditions in stroke patients would result in the merge of motor modules. We built a musculoskeletal bipedal locomotion model with a neural controller. In a simulated stroke condition, the motor modules in the controller were merged functionally, suggesting that chronic stroke patients would modify internal parameters for the controller to recover locomotion. These results would thus provide insights on how the motor modules are merged in stroke patients for better rehabilitation. These findings also help to provide a means for robots to recover locomotion adaptively after malfunctioning of the controller.

Data Availability Statement

All datasets generated for this study are included in the manuscript/Supplementary Files.

Author Contributions

DI and TY designed the research, analyzed data, and wrote the paper. DI performed the research.

Funding

Part of this study was supported by NEDO Next Generation AI and Robot Core Technology and KAKENHI Grant Number 17K07049.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We would like to thank all the members in the Yamazaki lab for their continuous supports and helpful discussions on this study.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnbot.2019.00079/full#supplementary-material

References

Allen, J. L., Kautz, S. A., and Neptune, R. R. (2013). The influence of merged muscle excitation modules on post-stroke hemiparetic walking performance. Clin. Biomech. 28, 697–704. doi: 10.1016/j.clinbiomech.2013.06.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Aoi, S., Ogihara, N., Funato, T., Sugimoto, Y., and Tsuchiya, K. (2010). Evaluating functional roles of phase resetting in generation of adaptive human bipedal walking with a physiologically based model of the spinal pattern generator. Biol. Cybern. 102, 373–387. doi: 10.1007/s00422-010-0373-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Aoi, S., Ogihara, N., Funato, T., and Tsuchiya, K. (2012). Sensory regulation of stance-to-swing transition in generation of adaptive human walking: A simulation study. Robot. Auton. Syst. 60, 685–691. doi: 10.1016/j.robot.2011.12.005

CrossRef Full Text | Google Scholar

Aoi, S., Ohashi, T., Bamba, R., Fujiki, S., Tamura, D., Funato, T., et al. (2019). Neuromusculoskeletal model that walks and runs across a speed range with a few motor control parameter changes based on the muscle synergy hypothesis. Sci. Rep. 9:369. doi: 10.1038/s41598-018-37460-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Beckerle, P., Salvietti, G., Unal, R., Prattichizzo, D., Rossi, S., Castellini, C., et al. (2017). A human-robot interaction perspective on assistive and rehabilitation robotics. Front. Neurorobot. 11:24. doi: 10.3389/fnbot.2017.00024

PubMed Abstract | CrossRef Full Text | Google Scholar

Cheung, V. C., Turolla, A., Agostini, M., Silvoni, S., Bennis, C., Kasi, P., et al. (2012). Muscle synergy patterns as physiological markers of motor cortical damage. Proc. Natl. Acad. Sci. U.S.A. 109, 14652–14656. doi: 10.1073/pnas.1212056109

PubMed Abstract | CrossRef Full Text | Google Scholar

Clark, D. J., Ting, L. H., Zajac, F. E., Neptune, R. R., and Kautz, S. A. (2010). Merging of healthy motor modules predicts reduced locomotor performance and muscle coordination complexity post-stroke. J. Neurophysiol. 103, 844–857. doi: 10.1152/jn.00825.2009

PubMed Abstract | CrossRef Full Text | Google Scholar

Dominici, N., Ivanenko, Y. P., Cappellini, G., d'Avella, A., Mondì, V., Cicchese, M., et al. (2011). Locomotor primitives in newborn babies and their development. Science 334, 997–999. doi: 10.1126/science.1210617

PubMed Abstract | CrossRef Full Text | Google Scholar

Ferrante, S., Chia Bejarano, N., Ambrosini, E., Nardone, A., Turcato, A. M., Monticone, M., et al. (2016). A personalized multi-channel fes controller based on muscle synergies to support gait rehabilitation after stroke. Front. Neurosci. 10:425. doi: 10.3389/fnins.2016.00425

PubMed Abstract | CrossRef Full Text | Google Scholar

Gandolla, M., Guanziroli, E., D'Angelo, A., Cannaviello, G., Molteni, F., and Pedrocchi, A. (2018). Automatic setting procedure for exoskeleton-assisted overground gait: proof of concept on stroke population. Front. Neurorobot. 12:10. doi: 10.3389/fnbot.2018.00010

PubMed Abstract | CrossRef Full Text | Google Scholar

Gizzi, L., Nielsen, J. F., Felici, F., Ivanenko, Y. P., and Farina, D. (2011). Impulses of activation but not motor modules are preserved in the locomotion of subacute stroke patients. J. Neurophysiol. 106, 202–210. doi: 10.1152/jn.00727.2010

CrossRef Full Text | Google Scholar

Grillner, S. (1975). Locomotion in vertebrates: central mechanisms and reflex interaction. Physiol. Rev. 55, 247–304. doi: 10.1152/physrev.1975.55.2.247

PubMed Abstract | CrossRef Full Text | Google Scholar

Guertin, P. A. (2009). The mammalian central pattern generator for locomotion. Brain Res. Rev. 62, 45–56. doi: 10.1016/j.brainresrev.2009.08.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Hase, K., and Yamazaki, N. (2002). Computer simulation study of human locomotion with a three-dimensional entire-body neuro-musculo-skeletal model. JSME Int. J. Ser. C Mech. Syst. Mach. Elements Manufactur. 45, 1040–1050. doi: 10.1299/jsmec.45.1040

CrossRef Full Text | Google Scholar

Hashiguchi, Y., Ohata, K., Kitatani, R., Yamakami, N., Sakuma, K., Osako, S., et al. (2016). Merging and fractionation of muscle synergy indicate the recovery process in patients with hemiplegia: the first study of patients after subacute stroke. Neural Plast. 2016:5282957. doi: 10.1155/2016/5282957

PubMed Abstract | CrossRef Full Text | Google Scholar

Hendricks, H. T., van Limbeek, J., Geurts, A. C., and Zwarts, M. J. (2002). Motor recovery after stroke: a systematic review of the literature. Arch. Phys. Med. Rehabil. 83, 1629–1637. doi: 10.1053/apmr.2002.35473

PubMed Abstract | CrossRef Full Text | Google Scholar

Holtzer, R., Mahoney, J. R., Izzetoglu, M., Wang, C., England, S., and Verghese, J. (2015). Online fronto-cortical control of simple and attention-demanding locomotion in humans. NeuroImage 112, 152–159. doi: 10.1016/j.neuroimage.2015.03.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Ishiguro, A., Fujii, A., and Hotz, P. E. (2003). Neuromodulated control of bipedal locomotion using a polymorphic cpg circuit. Adapt. Behav. 11, 7–17. doi: 10.1177/10597123030111001

CrossRef Full Text | Google Scholar

Ivanenko, Y. P., Cappellini, G., Dominici, N., Poppele, R. E., and Lacquaniti, F. (2005). Coordination of locomotion with voluntary movements in humans. J. Neurosci. 25, 7238–7253. doi: 10.1523/JNEUROSCI.1327-05.2005

PubMed Abstract | CrossRef Full Text | Google Scholar

Ivanenko, Y. P., Poppele, R. E., and Lacquaniti, F. (2004). Five basic muscle activation patterns account for muscle activity during human locomotion. J. Physiol. 556, 267–282. doi: 10.1113/jphysiol.2003.057174

PubMed Abstract | CrossRef Full Text | Google Scholar

Jezernik, S., Colombo, G., Keller, T., Frueh, H., and Morari, M. (2003). Robotic orthosis lokomat: a rehabilitation and research tool. Neuromodulation 6, 108–115. doi: 10.1046/j.1525-1403.2003.03017.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Jo, S., and Massaquoi, S. G. (2007). A model of cerebrocerebello-spinomuscular interaction in the sagittal control of human walking. Biol. Cybern. 96, 279–307. doi: 10.1007/s00422-006-0126-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Kawamoto, H., Kamibayashi, K., Nakata, Y., Yamawaki, K., Ariyasu, R., Sankai, Y., et al. (2013). Pilot study of locomotion improvement using hybrid assistive limb in chronic stroke patients. BMC Neurol. 13:141. doi: 10.1186/1471-2377-13-141

PubMed Abstract | CrossRef Full Text | Google Scholar

Khan, R. A., Naseer, N., Qureshi, N. K., Noori, F. M., Nazeer, H., and Khan, M. U. (2018). fnirs-based neurorobotic interface for gait rehabilitation. J. NeuroEng. Rehabil. 15:7. doi: 10.1186/s12984-018-0346-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Kim, Y., Tagawa, Y., Obinata, G., and Hase, K. (2011). Robust control of cpg-based 3d neuromusculoskeletal walking model. Biol. Cybern. 105, 269–282. doi: 10.1007/s00422-011-0464-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Lacquaniti, F., Ivanenko, Y. P, and Zago, M. (2012). Patterned control of human locomotion. J. Physiol. 590, 2189–2199. doi: 10.1113/jphysiol.2011.215137

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, C., Lowe, R., and Ziemke, T. (2013). Humanoids learning to walk: a natural cpg-actor-critic architecture. Front. Neurorobot. 7:5. doi: 10.3389/fnbot.2013.00005

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, W., Szczecinski, N. S., and Quinn, R. D. (2017). A neural network with central pattern generators entrained by sensory feedback controls walking of a bipedal model. Bioinspir. Biomimet. 12:065002. doi: 10.1088/1748-3190/aa8290

PubMed Abstract | CrossRef Full Text | Google Scholar

Matsubara, T., Morimoto, J., Nakanishi, J., aki Sato, M., and Doya, K. (2006). Learning cpg-based biped bipedal locomotion with a policy gradient method. Robot. Auton. Syst. 54, 911–920. doi: 10.1016/j.robot.2006.05.012

CrossRef Full Text | Google Scholar

Matsuoka, K. (1987). Mechanisms of frequency and pattern control in the neural rhythm generators. Biol. Cybern. 56, 345–353. doi: 10.1007/BF00319514

PubMed Abstract | CrossRef Full Text | Google Scholar

Molteni, F., Gasperini, G., Gaffuri, M., Colombo, M., Giovanzana, C., Lorenzon, C., et al. (2017). Wearable robotic exoskeleton for overground gait training in sub-acute and chronic hemiparetic stroke patients: preliminary results. Eur. J. Phys. Rehabil. Med. 53, 676–684. doi: 10.23736/S1973-9087.17.04591-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Mummolo, C., Peng, W. Z., Agarwal, S., Griffin, R., Neuhaus, P. D., and Kim, J. H. (2018). Stability of mina v2 for robot-assisted balance and locomotion. Front. Neurorobot. 12:62. doi: 10.3389/fnbot.2018.00062

PubMed Abstract | CrossRef Full Text | Google Scholar

Naseer, N., Hong, M. J., and Hong, K.-S. (2014). Online binary decision decoding using functional near–infrared spectroscopy for the development of brain-computer interface. Exp. Brain Res. 232, 555–564. doi: 10.1007/s00221-013-3764-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Neptune, R. R., Clark, D. J., and Kautz, S. A. (2009). Modular control of human walking: a simulation study. J. Biomech. 42, 1282–1287. doi: 10.1016/j.jbiomech.2009.03.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Noori, F. M., Naseer, N., Qureshi, N. K., Nazeer, H., and Khan, R. A. (2017). Optimal feature selection from fnirs signals using genetic algorithms for bci. Neurosci. Lett. 647, 61–66. doi: 10.1016/j.neulet.2017.03.013

PubMed Abstract | CrossRef Full Text | Google Scholar

Ogihara, N., and Yamazaki, N. (2001). Generation of human bipedal locomotion by a bio-mimetic neuro-musculo-skeletal model. Biol. Cybern. 84, 1–11. doi: 10.1007/PL00007977

PubMed Abstract | CrossRef Full Text | Google Scholar

Ohata, K., Yasui, T., Tsuboyama, T., and Ichihashi, N. (2011). Effects of an ankle-foot orthosis with oil damper on muscle activity in adults after stroke. Gait Post. 33, 102–107. doi: 10.1016/j.gaitpost.2010.10.083

PubMed Abstract | CrossRef Full Text | Google Scholar

Piron, L., Piccione, F., Tonin, P., and Dam, M. (2005). Clinical correlation between motor evoked potentials and gait recovery in poststroke patients. Arch. Phys. Med. Rehabil. 86, 1874–1878. doi: 10.1016/j.apmr.2005.03.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Rossignol, S., Dubuc, R., and Gossard, J. P. (2006). Dynamic sensorimotor interactions in locomotion. Phys. Rev. 86, 89–154. doi: 10.1152/physrev.00028.2005

PubMed Abstract | CrossRef Full Text | Google Scholar

Routson, R. L., Clark, D. J., Bowden, M. G., Kautz, S. A., and Neptune, R. R. (2013). The influence of locomotor rehabilitation on module quality and post-stroke hemiparetic walking performance. Gait Post. 38, 511–517. doi: 10.1016/j.gaitpost.2013.01.020

PubMed Abstract | CrossRef Full Text | Google Scholar

Taga, G. (1995). A model of the neuro-musculo-skeletal system for human locomotion I. emergence of basic gait. Biol. Cybern. 73, 97–111. doi: 10.1007/BF00204048

PubMed Abstract | CrossRef Full Text | Google Scholar

Taga, G., Yamaguchi, Y., and Shimizu, H. (1991). Self-organized control of bipedal locomotion by neural oscillators in unpredictable environment. Biol. Cybern. 65, 147–159. doi: 10.1007/BF00198086

PubMed Abstract | CrossRef Full Text | Google Scholar

Tan, C. K., Kadone, H., Watanabe, H., Marushima, A., Yamazaki, M., Sankai, Y., et al. (2018). Lateral symmetry of synergies in lower limb muscles of acute post-stroke patients after robotic intervention. Front. Neurosci. 12:276. doi: 10.3389/fnins.2018.00276

PubMed Abstract | CrossRef Full Text | Google Scholar

Ting, L. H., Chiel, H. J., Trumbower, R. D., Allen, J. L., McKay, J. L., Hackney, M. E., et al. (2015). Neuromechanical principles underlying movement modularity and their implications for rehabilitation. Neuron 86, 38–54. doi: 10.1016/j.neuron.2015.02.042

PubMed Abstract | CrossRef Full Text | Google Scholar

Velliste, M., Perel, S., Spalding, M. C., Whitford, A. S., and Schwartz, A. B. (2008). Cortical control of a prosthetic arm for self-feeding. Nature 453, 1098–1101. doi: 10.1038/nature06996

PubMed Abstract | CrossRef Full Text | Google Scholar

Watanabe, H., Goto, R., Tanaka, N., Matsumura, A., and Yanagi, H. (2017). Effects of gait training using the hybrid assistive limb in recovery-phase stroke patients: a 2-month follow-up, randomized, controlled study. NeuroRehabilitation 40, 363–367. doi: 10.3233/NRE-161424

PubMed Abstract | CrossRef Full Text | Google Scholar