# An Improved Fuzzy Brain Emotional Learning Model Network Controller for Humanoid Robots

^{1}Cognitive Science Department, School of Information Science and Engineering, Xiamen University, Xiamen, China^{2}Institute of Mathematics, Physics and Computer Science, Aberystwyth University, Aberystwyth, United Kingdom^{3}Department of Electrical Engineering, Yuan Ze University, Tao-Yuan, Taiwan^{4}Department of Computer and Information Sciences, Northumbria University, Newcastle upon Tyne, United Kingdom

The brain emotional learning (BEL) system was inspired by the biological amygdala-orbitofrontal model to mimic the high speed of the emotional learning mechanism in the mammalian brain, which has been successfully applied in many real-world applications. Despite of its success, such system often suffers from slow convergence for online humanoid robotic control. This paper presents an improved fuzzy BEL model (iFBEL) neural network by integrating a fuzzy neural network (FNN) to a conventional BEL, in an effort to better support humanoid robots. In particular, the system inputs are passed into a sensory and emotional channels that jointly produce the final outputs of the network. The non-linear approximation ability of the iFBEL is achieved by taking the BEL network as the emotional channel. The proposed iFBEL works with a robust controller in generating the hand and gait motion of a humanoid robot. The updating rules of the iFBEL-based controller are composed of two parts, including a sensory channel followed by the updating rules of the conventional BEL model, and the updating rules of the FNN and the robust controller which are derived from the “Lyapunov” function. The experiments on a three-joint robot manipulator and a six-joint biped robot demonstrated the superiority of the proposed system in reference to a conventional proportional-integral-derivative controller and a fuzzy cerebellar model articulation controller, based on the more accurate and faster control performance of the proposed iFBEL.

## 1. Introduction

The control of uncertain nonlinear systems with multiple inputs and outputs often presents a great challenge, and the robotic motion control is such a typical case. Robots, especially humanoid robots, are widely used in domestic, medical and other industrial areas (Liu et al., 2015; Li et al., 2017; Wu et al., 2018; Zhou et al., 2018). A humanoid robot must accurately control its two manipulators and two legs, in order to generate hand reaching/grasping motions and biped-leg walking gaits. Such crucial motion abilities allow humanoid robots to work in complicated, dangerous, and even poisonous environments with reduced labor costs, health implication, and other associated complications.

The Sliding Mode Control (SMC) proves to be an effective control method for uncertain nonlinear systems, especially for humanoid motion control. Once the state of the system reaches a sliding surface, the state will remain on that surface regardless of system uncertainties and external disturbances (Lin and Hsu, 2015). Yet, control input chattering, usually led by a combination of uncertainties from multiple pathways, is often not expected in humanoid robot systems when SMC is applied. It has been found in several studies that the collaboration of an artificial neural network with a SMC controller can enhance non-linear approximation ability in reducing the chattering effect (Boldbaatar and Lin, 2015).

A neural network with good non-linear learning abilities is therefore of great appeal to the SMC model. Note that an association between a stimulus and its emotional consequence in the amygdala of the mammalian brain was discovered by LeDoux (1992). The inspiration from the emotional consequence then led to the development of the brain emotional learning network (BEL) controller, which has a good nonlinear approximation capability. Such a neural network is comprised of a sensory neural network in simulating the orbitofrontal cortex of the brain, and an emotional neural network representing the amygdala cortex (LeDoux, 1992; Lotfi and Akbarzadeht, 2013). The sensory neural network is responsible for the major output of the controller, while the emotional neural network has an indirect impact on the sensory neural network. Despite of the effectiveness in uncertain non-linear control, most BEL networks face the dilemma of slow learning convergence leading to difficulty in on-line control of the multiple joints of a humanoid robot.

Fuzzy neural networks (FNN) are another popular choice for uncertain nonlinear control systems with reasonable non-linear approximation ability, due to their rapid learning convergence and simple structure which is particularly favorable for on-line humanoid robotic control (Rubio, 2012, 2018; Aguilar-Iban et al., 2018; Rubio et al., 2018). A typical FNN integrates a fuzzy inference system and a neural network (Pan et al., 2016; Meda-Campana et al., 2018). The weights of the network are usually updated by taking only the output errors of the FNN as the learning assessment. To achieve better performance for uncertain nonlinear systems, the FNN must also consider the overall performance of uncertain nonlinear systems when adjusting the control parameters, as reported in Zhao and Lin (2017). Therefore, the combination of the rapid convergence of FNN and the nonlinear mapping capability of BEL seems to be a good idea for controlling humanoid robots.

We believe that the chattering effect of the SMC model is a very challenging issue. Although, many existing algorithms had been developed to deal with the chatting; the artificial neural network still plays an important role in the control of uncertain nonlinear system with multiple inputs and outputs. In addition, FNN is good at rapid convergence and BEL can ideally increase the network's nonlinear mapping capability. Therefore, we focused on a combined neural network to deal with the chattering problem. Based on these considerations, this paper proposes an improved brain emotional learning model network (iFBEL) for a humanoid robot controller, in an effort to achieve better human-like control performance with the support of more nonlinear approximation capabilities. The proposed iFBEL is comprised of two components, with one built from a conventional BEL and the other created by an FNN; and the resulted iFBEL thus enjoys the advantages of both sub-systems. The iFBEL works with a robust controller to replace the ideal sliding mode controller for better system performance. To ensure the convergence and robustness, the adaptive laws of the FNN and the robust controller are derived from the Lyapunov function. The iFBEL was validated and evaluated on a robot with a three-joint manipulator and a biped-leg system, although applications in other control fields can be readily identified. The experimental results demonstrate competitive performance of the proposed systems in dynamic humanoid robotic control.

The reminder of this paper is organized as follows: section 2 introduces a group of uncertain nonlinear systems controlled by a sliding mode controller. Section 3 reports the proposed improved fuzzy brain emotional learning model neural network. Section 4 describes the implementations of the network controller and the updating rules. Section 5 shows the experimental results and compares the performances with the conventional proportional-integral-derivative (PID) controller and the fuzzy cerebellar model articulation controller (FCMAC). Section 7 concludes the paper and points out future work.

## 2. Humanoid Robot Control by Sliding Mode Controller

In order to understand the proposed network-based control system and realize the importance of the proposed neural network, this section introduces a typical uncertain nonlinear system controlled by a sliding mode controller as the work's background.

A humanoid robot needs to control multi-joints. Without loss of generality, consider a class of *n*th-order uncertain nonlinear systems with *m*th-order input and output states expressed in the following form:

where * x*(

*t*) = [

*x*

^{(n−1)}(

*t*) …

*ẋ*(

*t*)

*x*(

*t*)] ∈ ℜ

^{m×n}is the system state vector, $u(t)={\left[{u}_{1}(t),{u}_{2}(t),\dots ,{u}_{m}(t)\right]}^{T}\in {\Re}^{m}$ is the control input vector,

*f*(

*(*

__x__*t*)) ∈ ℜ

^{m}is an unknown, but bounded, smooth nonlinear function,

*G*(

*(*

__x__*t*)) ∈ ℜ

^{m×m}is an unknown, but bounded, gain matrix, and $d(t)={\left[{d}_{1}(t),{d}_{2}(t),\dots ,{d}_{m}(t)\right]}^{T}\in {\Re}^{m}$ is an external bounded disturbance.

The nominal model of such a nonlinear system can be defined as

where *f*_{n}(* x*(

*t*)) is the nominal function of

*f*(

*(*

__x__*t*)), and ${G}_{n}=diag\left[{g}_{{n}_{1}}\dots {g}_{{n}_{m}}\right]\in {\Re}^{m\times m}$ is the nominal function of

*G*(

*(*

__x__*t*)), with

*g*

_{ni}being nominal gain constants, for

*i*= 1, 2, …,

*m*. Assume that

*g*

_{ni}> 0 for the existence of ${G}_{n}^{-1}$, Equation 1 can be represented as:

where *l*(* x*(

*t*),

*t*) = △

*f*(

*(*

__x__*t*)) + △

*G*(

*(*

__x__*t*))

*u*(

*t*) +

*d*(

*t*) is the lumped uncertainties and external disturbances. Let ${\underset{\_}{x}}_{d}(t)={\left[{x}_{d}^{(n-1)T}(t),\dots ,{\u1e8b}_{d}^{T}(t),{x}_{d}^{T}(t)\right]}^{T}\in {\Re}^{m\times n}$ be a desired trajectory in which the state of the system is tracked. The tracking error vector is defined as:

where *e*(*t*) = *x*_{d}(*t*) − *x*(*t*) is the tracking error.

An ideal sliding surface can be defined as

where $\stackrel{\u0304}{K}=\left[I,K\right]=\left[\begin{array}{cccc}I& {\lambda}_{1}I& \dots & {\lambda}_{n}I\end{array}\right]\in {\Re}^{m\times (m+1)n}$. All ${\lambda}_{j}={\left[{\lambda}_{1j}\dots {\lambda}_{nj}\right]}^{T}\in {\Re}^{n}$ are roots of the equation: ${q}^{n}+{\lambda}_{1}{q}^{n-1}+\cdots +{\lambda}_{n-1}q+{\lambda}_{n}=0$ in which *q* is the Laplace operator and is in the open left half-plane. The time derivative of Equation 4 leads to the following:

where __ ė__(

*t*) = [

*e*

^{(n)}(

*t*)

*e*

^{(n−1)}(

*t*) …

*ė*(

*t*)].

For the existence and reachability of this sliding surface, the control law of system is satisfied by the following inequation:

for σ_{i} > 0, *i* = 1, 2, …, *m*.

Taking 5 into 6, yields

If the dynamic and the lumped uncertainty of the system are known exactly, the ideal sliding mode controller is designed as:

where *sgn* is a sign function and *G*_{n} is a positive define matrix. However, it is difficult to obtain the dynamical functions of most nonlinear systems, and the lumped uncertainty is always unmeasurable. Therefore, the ideal sliding mode controller is unobtainable.

## 3. The Proposed iFBEL Network

The configuration of the proposed iFBEL is depicted in Figure 1, consisting of an BEL and the FNN in addition to the input and output spaces. The outputs of this network are *u*_{i} = *b*_{i} − *g*_{i} for *i* = 1, 2, …, *m*, in which, *b*_{i} are the outputs of the the BEL and *g*_{i} are the outputs of the FNN. The BEL network is comprised of the input space *I*, the association memory space *M*_{1}, the weight memory space *V*, and the sub-output space *B*. The FNN shares the same input space with the BEL, and it also includes the association memory space *M*_{2}, the receptive-field space *R*, the weight memory space *W*, and the sub-output space *G*. In particular, the FNN channel of iFBEL also contains a set of fuzzy reference rules (Lee, 1990) as represented as follows:

where *n*_{f} is the number of layers for each *m* input dimensions with each layer including *n*_{k} blocks and *n*_{l} = *n*_{f}*n*_{k} referring to the number of fuzzy rules, and ϕ_{ijk} represents the fuzzy set for *i*th input, *j*th layer and *k*th block; each fuzzy set's member function is implemented by the Gaussian function; ω_{jk} is the output weight in the consequent part; and *g*_{jk} is the rule's output. Note that: each fuzzy set's member function can be defined as rectangular, triangular or any continuously bounded function e.g., Gaussian or B-spline; in order to easily implement the iFBEL with the better non-linear approximation ability, the Gaussian function is adopted.

The aforementioned “spaces” are detailed as follows:

1. *Input Space I*: $p={\left[{p}_{1},{p}_{2},\dots ,{p}_{m}\right]}^{T}\in {\Re}^{m}$ is an input vector which is quantized into discrete regions (elements), where *m* is the number of input state variables. The number of elements, *n*_{e}, is termed as a resolution. *p* is delivered to the BEL and the FNN simultaneously as their inputs.

2. *Association Memory Spaces M*_{1} and *M*_{2}: Several elements are combined as a block; the number of blocks, *n*_{b} and *n*_{f} for the BEL and the FNN respectively, must be equal or greater than two. The association memory space of the BEL has *n*_{a}(= *m* × *n*_{b}) components, while that of the FNN has *n*_{c}(= *m* × *n*_{f}) components. Every component is represented as a Gaussian basis function; let φ denote a component for the BEL and *f* for the FNN:

where *i* = 1, 2, …, *m*, *j* = 1, 2, …, *n*_{b}, and *y*_{ij} and *z*_{ij} are the means and variances, respectively; and

where *i* = 1, 2, …, *m*, *j* = 1, 2, …, *n*_{f}, *k* = 1, 2, …, *n*_{k}, and *c*_{ijk} and *v*_{ijk} are the means and variances, respectively.

The block matrix of the BEL is defined as:

3. *Receptive-field Space R for FNN*: Every cell in this space is the product of the corresponding components of the association memory space *M*_{2}, which is defined as:

where *j* = 1, 2, …, *n*_{f}, and *k* = 1, 2, …, *n*_{k}. An example of the FNN with two input variables is shown in Figure 2, which has 4 layers (*n*_{f} = 4) for every input variable and 2 blocks (*n*_{k} = 2) for each layer. And *n*_{l} = *n*_{f}*n*_{k} is the number of receptive fields, such as *Aa, Bb*, …; ϕ_{jk} is associated with the *j*th layer and the *k*th block in the fuzzy rule as expressed in Equation 9. The block matrix of the FNN is defined as:

4. *Weight Memory Spaces V* and *W* : ν_{ijk} is the weight of the *i*th output, *j*th input, and *k*th block of the BEL; and ω_{ijk} is the weight of the *i*th output, *j*th layer, *k*th block of the FNN:

5. *Sub-output Space B* and *G*: The *i*th output (*b*_{i}) and the output vector (*b*) of the BEL, and the *i*th output (*g*_{i}) and the output vector (*g*) of the FNN are represented as follows:

6. *Output Space U*: The output of the proposed iFBEL is the combination of the outputs of the BEL and the FNN, in which the BEL works as a primary controller and the FNN as an emotion controller:

## 4. iFBEL-based Controller

The proposed intelligent controller, consisting of a sliding surface, an iFBEL network, and a robust controller, is shown in Figure 3. The iFBEL network and robust controller collaborate to imitate an ideal sliding mode controller. The updating rules of the BEL mechanism of the iFBEL network are followed by the brain emotional learning algorithm (Chung and Lin, 2015; Lin and Chung, 2015); and the adaptive laws of the FNN mechanism and robust controller are derived from the Lyapunov function. Besides, to ensure robust tracking performance.

The updating rules are detailed as follows. Subtracting 8 into 5, yields:

Assume that an optimal iFBEL ${u}_{BFC}^{*}$ exists in the ideal sliding model controller, *u*_{ISMC}, and that ϵ is a minimum error vector; thus, the weight matrixes of ${u}_{BFC}^{*}$ are represented as *V** and *W** for the BEL and the FNN, respectively. Then, the output of the optimal sliding model controller is:

where *u*_{BEL} and *u*_{FNN} are the outputs of the BEL and the FNN respectively, and Φ* and $\widehat{\Gamma}$ are the optimal matrix and estimated matrix of Φ and Γ respectively. The output of the proposed iFBEL controller is defined by:

where *u*_{RC} is the output of the robust controller, and $\widehat{V},\u0174,\widehat{\Phi}$ are the estimated matrices of *V**, *W**, Φ* respectively.

Taking 23 and 24 into 22, the following can be obtained:

where $\stackrel{~}{\Phi}={\Phi}^{*}-\widehat{\Phi}$, and $\u1e7c={V}^{*}-\widehat{V}$. A partially linear form of the receptive-field basis function vector $\stackrel{~}{\Phi}$ in the Taylor series is:

where Φ_{c} and Φ_{v} are defined by:

where $\stackrel{~}{c}={c}^{*}-\u0109,\u1e7d={v}^{*}-\widehat{v}$, β is a higher-order vector.

Rewriting 26 with $\stackrel{~}{\Phi}={\Phi}^{*}-\widehat{\Phi}$, yields:

Substituting 27 to 25, yields:

where $\omega ={W}^{*T}\beta +\stackrel{~}{W}({\Phi}_{c}\stackrel{~}{c}+{\Phi}_{v}\u1e7d)+\u03f5$ is a combined error of the FNN, and $\u1e7c={V}^{*}-\widehat{V}={\left[\stackrel{~}{{\nu}_{1}},\stackrel{~}{{\nu}_{2}},\dots ,\stackrel{~}{{\nu}_{m}}\right]}^{T}\in {\Re}^{m\times m{n}_{b}}$ is an approximation error weight matrix of the BEL. Consider a *H*_{∞} tracking performance (Chen et al., 2015) for the existence of ω and Ṽ as:

where η_{W}, η_{c}, η_{v} are diagonal positive constant learning-rate matrices, and λ_{i} is an attenuation constant. Set the initial conditions of the system as $s(0)=0,\stackrel{~}{W}(0)=0,\stackrel{~}{c}(0)=0,\u1e7d(0)=0$; then Equation 29 can be re-expressed as:

To approximate an ideal sliding mode controller, assume that the approximation error between the proposed iFBEL and an ideal controller are bounded; in other words, ω ∈ *L*_{2}[0, *T*_{1}] and Ṽ ∈ *L*_{2}[0, *T*_{2}] with ∀*T*_{1}, *T*_{2} ∈ [0, ∞]. Therefore ${\int}_{0}^{T}{\omega}_{i}^{2}(t)dt\le {N}_{1}$ and ${\int}_{0}^{T}{\stackrel{~}{\nu}}_{i}^{2}(t)dt\le {N}_{2}$, where *N*_{1} and *N*_{2} are two big positive constants. If λ = ∞, the minimum error cannot achieve approximation attenuation. If λ < ∞, the system is stable as shown by:

**THEOREM 1**. *For the nonlinear system with Multiple Inputs and Multiple Outputs as represented by Equation 1, the proposed iFBEL can be described by Equation 24, in which the updating rule of the BEL is designed as expressed in Equation 32, and the adaptive laws of the FNN and robust controller are designed as stated in Equations (34-36)*.

*where α is a learning-rate constant, and d consists of the input vector p and the output vector u _{BFC} with the learning constants γ and τ*.

*where* $R=diag\left[\begin{array}{cccc}{\lambda}_{1}& {\lambda}_{2}& \dots & {\lambda}_{m}\end{array}\right]\in {\Re}^{m\times m}$ *is a diagonal matrix of a robust controller to converge the proposed system with the update rules* $\stackrel{\xb7}{\u0174},\stackrel{\xb7}{\u0109}$ *and* $\stackrel{\xb7}{\widehat{v}}$, *and* λ_{i} > 0, *where i = 1, 2, …, m; thus, R is a positive definite matrix*.

PROOF. The Lyapunov function is given by:

Taking the derivative of the Lyapunov function and using 28, yields

Since $\stackrel{\xb7}{\widehat{V}}=0$ when *d*_{i} − *b* ≤ 0 and $\stackrel{\xb7}{\widehat{V}}=\alpha \xb7\Gamma \xb7\left[{d}_{i}-b\right]>0$ if *d*_{i} − *a* > 0, consider Ṽ ∈ *L*_{2}[0, *T*_{2}] leading to $-tr\left[{\u1e7c}^{T}{\alpha}^{-1}\stackrel{\xb7}{\widehat{V}}\right]\le 0$. Substituting 34–37 into 39, yields:

Integrating 40 from *t* = 0 to *t* = *T*, yields:

Since *V*(*T*) > 0 and *V*(0) > 0, Equations 30 and 31 lead to $\sum _{i=1}^{m}\underset{0}{\overset{T}{\int}}{s}_{i}^{2}(t)dt<\infty $.

## 5. Experimentation

To verify the effectiveness and efficacy of the proposed controller with the new iFBEL, it was applied to two typical humanoid robotic systems, including a three-joint robot manipulator and a six-joint biped robot. A comparative study is also included in this section to evaluate the performance of the proposed controller in reference to two important control approaches including a PID controller and an SMC with fuzzy cerebellar model articulation controller network (FCMAC) (Lin et al., 2009).

PID control is a classic control method, which is linearly combined by proportional control, integral control and differential control. The FCMAC network has the characteristics of rapid convergence, which enable the work to be suitable for the robotic control. The effectiveness of the FCMAC-based network controller has been demonstrated in many recent studies, such as Lin et al. (2016) and Zhao and Lin (2017). The experiments of both three-joint robot manipulator and six-joint biped robot are simulated in MATLAB R2016a. The configuration of the algorithm computer is set as follows: The CPU and the operating system of the development computer are Intel Core i5-4200U CPU@2.30GHz and Windows 10 professional. The source code of the algorithm can be found in this link^{1}.

The parameters for the robust controller and the iFBEL's Gaussian functions and weights are tuned by using Equations from 32 to 37. The learning rate parameters and iFBEL's network structure are set empirically.

### 5.1. Three-Joint Robot Manipulator

The first experiment was carried out using a relatively simple three-joint robot manipulator, to mainly practically evaluate the validity of the proposed system. The three-joint robot manipulator used in this experiment is illustrated in Figure 4; and the dynamic equation of such system is expressed as follows:

where *q* ∈ ℜ^{3} is the joint angle state vector, $\stackrel{\xb7}{q}\in {\Re}^{3}$ is the velocity vector, $\ddot{q}\in {\Re}^{3}$ is the acceleration vector, *M*(*q*) ∈ ℜ^{3×3} is the inertia matrix, $C(q,\stackrel{\xb7}{q})\in {\Re}^{3\times 3}$ is the Coriolis/Centripetal matrix, *g*(*q*) ∈ ℜ^{3} is the gravity vector, and *q* = [−0.2, 0.5, −0.3]^{T}, $\stackrel{\xb7}{q}=0$, $\ddot{q}=0$ are designated as the original state, *u* ∈ ℜ^{3} is the output torque. The detailed expression of $M(q),C(q,\stackrel{\xb7}{q}),g(q)$ and the nominal parameters of the manipulator are provided in Appendix 1.1.

The reference trajectories were given as ${q}_{d1}=\frac{1}{2}{[\frac{1}{2}(\mathrm{sin}(t+2.5)+0.7\mathrm{cos}(2t+1.5)],\mathrm{sin}(t)+\mathrm{sin}(2t),0.13-(\mathrm{sin}(t)+\mathrm{sin}(2t))]}^{T}$, $\stackrel{\xb7}{{q}_{d1}}=0$, $\ddot{{q}_{d1}}=0$. To evaluate the robustness of the proposed control system, the reference trajectories were modified as ${q}_{d2}=\frac{1}{2}{\left[\frac{1}{2}(sin(2t)+cos(t+1)),sin(2t)+cos(t+1),cos(2t)-sin(t)\right]}^{T}$, $\stackrel{\xb7}{{q}_{d2}}=0$, $\ddot{{q}_{d2}}=0$ at *t* = 15*s*, with the external disturbance of ${\tau}_{d}={\rho}_{1}\times {\left[0.2sin(2t),0.1cos(2t),0.1sin(t)\right]}^{T}$, where ρ_{1} = 1 is the amplification coefficient. In order to evaluate the proposed network's performances in various disturbance situations, two coefficients (ρ_{1} = 1.5 and ρ_{1} = 2) were also used in the experiments. The BEL and the FNN were characterized as follows:

• the number of elements for each state variable: *n*_{E} = 5 (elements);

• generalization: *n*_{C} = 4 (elements/block);

• the number of blocks for each state variable for both the BEL and FNN: *n*_{b} = *n*_{f} = 2 (blocks/layer) × 4 (layer) = 8 (blocks);

• the number of receptive fields: *n*_{E} = 2 (receptive fields/layer) × 4 (layer) = 8 (receptive fields).

The initial means of the Gaussian functions in the Association Memory Spaces were divided equally and set as [−1, 1] for the BEL, and [−2, 2] for the FNN. The initial variances were set as σ_{ij} = 0.1 for the BEL, and σ_{pq} = 0.1 for the FNN, where *i* = *p* = 1, 2, 3, and *j* = *q* = 1, 2…, 8. The weights of both the BEL and the FNN were initialized as zero and then automatically adjusted during the online training process. In addition, the learning rates were set as follows: η_{ω} = 20, η_{m} = 0.001, η_{v} = 0.001, α = 0.01, *b* = 0.1, *c* = 0.1.

The parameters of PID controller in the comparison experiments were set as: κ_{P} = 15, κ_{I} = 0.2, κ_{D} = 0.5, where κ_{P}, κ_{I} and κ_{D} are the coefficients of the proportional controller, integral controller and differential controller. FCMAC controller in the comparison experiments has the same parameters as FNN does.

The simulated position responses and the tracking errors at ρ_{1} = 1 are shown in Figure 5. To better distinguish these values for the three controllers, Figures 6, 7 show the amplified trajectory responses and the tracking errors at *t* = 0 and *t* = 15. In Figure 6, the PID controller required 1.4*s*, 1.3*s*, and 0.05*s* for Joints 1, 2, and 3 to converge, respectively, while the FCMAC required 1.2*s*, 1.3*s*, and 0.05*s* for these joints respectively; however, the proposed iFBEL controller just needed 1.1*s*, 1.3*s*, 0.03*s* for these joints, respectively. In addition, the iFBEL performed the best when *t* = 15*s*.

**Figure 5**. Trajectory responses (in the left) and tracking errors (in the right) of Joints 1, 2, and 3 at ρ_{1} = 1. The solid line indicates the performance of the iFBEL; the dotted line represents that of the PID controller; and the dot dash line implies that of the FCMAC controller. **(A)** Trajectory response and tracking error of Joint 1. **(B)** Trajectory response and tracking error of Joint 2. **(C)** Trajectory response and tracking error of Joint 3.

**Figure 6**. Amplified trajectory responses (in the left) and tracking errors (in the right) at *t* = 0 of Joints 1, 2, and 3. The solid line indicates the performance of the iFBEL; the dotted line represents that of the PID controller; and the dot dash line implies that of the FCMAC controller. **(A)** Amplified trajectory response and tracking error of Joint 1. **(B)** Amplified trajectory response and tracking error of Joint 2. **(C)** Amplified trajectory response and tracking error of Joint 3.

**Figure 7**. Amplified trajectory responses (in the left) and tracking errors (in the right) at *t* = 15 of Joints 1, 2, and 3. The solid line indicates the performance of the iFBEL; the dotted line represents that of the PID controller; and the dot dash line implies that of the FCMAC controller. **(A)** Amplified trajectory response and tracking error of Joint 1. **(B)** Amplified trajectory response and tracking error of Joint 2. **(C)** Amplified trajectory response and tracking error of Joint 3.

The accumulated RMSE values at ρ_{1} = 1 during the entire experiment are listed in Table 1, which also proved that the proposed iFBEL controller outperformed others. However, the difference among the three controllers is insignificant. The FCMAC and the PID controllers also generated good control performances in this experiment, because the three-joint manipulator system is not very complicated. The accumulated RMSE values under ρ_{1} = 1.5 and ρ_{1} = 2 are listed in Tables 2, 3, respectively. With the increase of disturbance, the errors of the three controllers also increased. However, the iFBEL also achieved the best performance under the two disturbance situations. This proves that the proposed iFBEL can well handle larger disturbances.

### 5.2. The Biped Robot

The configuration of the six-link biped robot used in this second experiment is illustrated in Figure 8. The experiment reported in the last sub-section was mainly used to validate the proposed system, but the experiment reported in this sub-section was primarily used to evaluate the efficiency and efficacy of the proposed control system. The dynamic equation of the robot is given as follow:

where *q* ∈ ℜ^{6}, $\stackrel{\xb7}{q}\in {\Re}^{6}$, $\ddot{q}\in {\Re}^{6}$ are the joint angle state vector, velocity vector and acceleration vector respectively, and *M*(*q*) ∈ ℜ^{6×6}, $C(q,\stackrel{\xb7}{q})\in {\Re}^{6\times 6}$, *g*(*q*) ∈ ℜ^{6} are the inertia matrix, the Coriolis/Centripetal matrix and the gravity vector respectively, *u* ∈ ℜ^{6} is the output torque. More details for $M(q),C(q,\stackrel{\xb7}{q}),g(q)$ and the nominal parameters of the biped robot can be found in Appendix 1.2.

This experiment also considered the phases of signal support of a gait cycle. The analysis planning and walking pattern generation are detailed in Appendix 1.3. The generated gait trajectory ${q}_{d}={\left[{\theta}_{1},{\theta}_{2},\dots ,{\theta}_{6}\right]}^{T}$,$\stackrel{\xb7}{{q}_{d}}=0$, $\ddot{{q}_{d}}=0$ were set as the reference trajectories of the biped robot. The initial angles of each joint were given as *q* = [0.37, 0.5, 0.75, −0.15, −0.56, 0.85]^{t},$\stackrel{\xb7}{q}=0$, $\ddot{q}=0$. τ_{d} = ρ_{2} × *exp*(−0.1*t*)_{6×1} was used in this experiment as the external disturbance, where ρ_{2} = 1 is the amplification coefficient.

The BEL and the FNN are characterized as the same with that used in the first experiment as reported in section 5.1, but with different initializations. In particular, the initial means of the Gaussian functions in the Association Memory Spaces in this experiment were divided equally and set as [−1.4, 1.4] for the BEL, and [−1.6, 1.6] for the FNN. The initial variances were set as σ_{ij} = 0.01 for the BEL and σ_{pq} = 0.5 for the FNN, where *i* = *p* = 1, 2…, 6, and *j* = *q* = 1, 2…, 8. The weights of both sub-systems were initialized from zero and then automatically adjusted during the online training stage. In this experiment, the learning rates were chosen as η_{ω} = 0.01, η_{m} = 0.001, η_{v} = 0.001, α = 0.01, *b* = 0.05, and *c* = 0.01.

The parameters of PID controller in the second experiment were set as: κ_{P} = 8, κ_{I} = 0.5, κ_{D} = 1.3. FCMAC controller in the second experiment also has the same parameters as FNN does.

The simulated position responses and the tracking errors at ρ_{2} = 1 led by the three controllers are illustrated in Figures 9, 10; with the performances of Joints 1, 2 and 3 illustrated in Figure 9 and those of Joints 4, 5, and 6 in Figure 10. The PID controller had a significant convergence delay, which therefore represented the worst performance within the three controllers. It is difficult from these figures to distinguish the performances led by the FCMAC and the iFBEL controllers, and thus the trajectories resulted from all the controllers in the range of [−1.4*s*, 1.4*s*] are magnified as displayed in Figures 11, 12 for better visualization and thus easier investigation.

**Figure 9**. Trajectory responses (in the left) and tracking errors (in the right) of Joints 1, 2, and 3 at ρ_{2} = 1. The solid line indicates the performance of the iFBEL; the dotted line represents that of the PID controller; and the dot dash line implies that of the FCMAC controller. **(A)** Trajectory response and tracking error of Joint 1. **(B)** Trajectory response and tracking error of Joint 2. **(C)** Trajectory response and tracking error of Joint 3.

**Figure 10**. Trajectory responses (in the left) and tracking errors (in the right) of Joints 4, 5, and 6 at ρ_{2} = 1. The solid line indicates the performance of the iFBEL; the dotted line represents that of the PID controller; and the dot dash line implies that of the FCMAC controller. **(A)** Trajectory response and tracking error of Joint 4. **(B)** Trajectory response and tracking error of Joint 5. **(C)** Trajectory response and tracking error of Joint 6.

**Figure 11**. Amplified trajectory responses (in the left) and tracking errors (in the right) of Joints 1, 2, and 3. The solid line indicates the performance of the iFBEL; the dotted line represents that of the PID controller; and the dot dash line implies that of the FCMAC controller. **(A)** Amplified trajectory response and tracking error of Joint 1. **(B)** Amplified trajectory response and tracking error of Joint 2. **(C)** Amplified trajectory response and tracking error of Joint 3.

**Figure 12**. Amplified Trajectory responses (in the left) and tracking errors (in the right) of Joints 4, 5, and 6. Solid line indicate iFBEL with the dotted one point PID controller and the dot dash one imply FCMAC controller. **(A)** Amplified trajectory response and tracking error of Joint 4. **(B)** Amplified trajectory response and tracking error of Joint 5. **(C)** Amplified trajectory response and tracking error of Joint 6.

From Figures 11, 12, it is clear that the PID controller could not converge rapidly in all the joints of the biped robot. The performances of the FCMAC and the iFBEL regarding all of the joints were very similar; both controllers rapidly converged the tracking errors. The tracking error amplitudes of the FCMAC controller in Joints 1, 2, 3, and 6 were larger than those of the iFBEL controller, which indicates the superiority of the proposed iFBEL controller.

The accumulated RMSE values are listed in Table 4. It is clear from this table that the convergence time of the iFBEL controller was shorter than those of the PID and the FCMAC for each joint. In this case, the RMSE values also proved that the proposed iFBEL controller achieved the best control performance within the three compared controllers used in this comparative study. The accumulated RMSE values at ρ_{2} = 1.5 and ρ_{2} = 2 are also given in Tables 5, 6, respectively. The iFBEL also achieved the best performance under the two disturbance situations.

## 6. Discussion

A humanoid robot usually consists of multiple joints and suffers many unexpected disturbances; therefore, the controller of humanoid robot must own the powerful non-linear approximation ability to handle these complex situations. Based on the results of the two simulations, the proposed iFBEL network successfully demonstrated a rapid convergence ability and a nonlinear mapping capability. In the two simulations, the iFBEL controller can always achieve the fastest reaction speed to reduce errors; in addition, the iFBEL controller still achieved the best performance in different disturbance patterns. Therefore, the proposed network is suitable for the control of humanoid robots.

Although the performance of iFBEL-based controller was better than those of the FCMAC and PID controllers, the iFBEL network's structure is more complicated than that of the FCMAC. To address this issue, we believe that a recurrent mechanism usually uses a simple network structure to achieve good dynamic performance. Therefore, in the future work, we will improve our method by embedding a recurrent network inside the iFBELC controller.

## 7. Conclusion

This paper proposed a novel humanoid robot controller, which integrates some components from a fuzzy neural network and a brain emotional learning model into a sliding mode controller for dynamic non-linear control. It has been theoretically proven that the proposed system is asymptotically stable, thus guaranteeing the convergence. Experimental results and comparative studies further verified this, and demonstrated precise position tracking, more favorable stability, and better performance in reference to the results generated from the recently-developed network controllers of PID and FCMAC.

This research can be further improved in several directions. The current iFBEL network does not include any recurrent mechanism, but such a mechanism can generally improve the dynamic performance of a network. Therefore, a future investigation will focus on the development of the recurrent feature to better support the iFBEL controller. In addition, the undesired chattering situation existing in the sliding surface has not been fully investigated; more efforts will focus on this issue. Furthermore, the proposed approach was only practically applied to the dynamic humanoid robot control in this work. It is worthwhile to apply the approach to a wider range of applications to fully discover its potential.

## Author Contributions

WF contributed to this work by developing the proposed method and preparing the experiments. FC contributed the implementation of the proposed method and writing the manuscript. C-ML conducted the statistical analysis of the experimental results. LY contributed the planning and analysis of the experiments and writing of the manuscript. CS contributed to the design of the proposed method. CZ contributed to the writing of the manuscript.

## Funding

This work was supported by the National Natural Science Foundation of China (Nos. 61673322, 61673326, and 91746103), the Fundamental Research Funds for the Central Universities (No. 20720160126), Natural Science Foundation of Fujian Province of China (Nos. 2017J01128 and 2017J01129), and the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No. 663830.

## Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

## Acknowledgments

We are very grateful to the reviewers for their constructive comments which have helped significantly in revising this work.

## Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/article/10.3389/fnbot.2019.00002/full#supplementary-material

## Abbreviations

BEL, Brain Emotional Learning; BFC, BEL and FNN Controller; FCMAC, Fuzzy Cerebellar Model Articulation Controller; FNN, Fuzzy Neural Network; iFBEL, improved Fuzzy Brain Emotional Learning; ISMC, Ideal Sliding Model Controller; PID, Proportional Integral Derivative; RC, Robust Controller; RMSE, Root Mean Square Error; SMC, Sliding Mode Control.

## Footnotes

## References

Aguilar-Ibanez, C., Suarez-Castanon, M. S., Mendoza-Mendoza, J., de Jesus Rubio, J., and Martnez-Garca, J. C. (2018). Output-feedback stabilization of the pvtol aircraft system based on an exact differentiator. *J. Intell. Robot. Sys.* 90, 443–454. doi: 10.1007/s10846-017-0660-0

Boldbaatar, E. A., and Lin, C. M. (2015). Self-learning fuzzy sliding-mode control for a water bath temperature control system. *Int. J. Fuzzy Syst.* 17, 31–38. doi: 10.1007/s40815-015-0015-6

Chen, C. H., Chung, C. C., Chao, F., Lin, C. M., and Rudas, I. J. (2015). Intelligent robust control for uncertain nonlinear multivariable systems using recurrent cerebellar model neural networks. *Acta Polytech. Hung.* 12, 7–33. doi: 10.12700/APH.12.5.2015.5.1

Chung, C. C., and Lin, C. M. (2015). Fuzzy brain emotional cerebellar model articulation control system design for multi-input multi-output nonlinear. *Acta Polytech. Hung.* 12, 39–58. doi: 10.12700/APH.12.4.2015.4.3

Huang, Q., Yokoi, K., Kajita, S., Kaneko, K., Arai, H., Koyachi, N., et al. (2002). Planning walking patterns for a biped robot. *IEEE Trans. Robot. Automat.* 17, 280–289. doi: 10.1109/70.938385

LeDoux, J. E. (1992). “Chapter the amygdala: neurobiological aspects of emotion, memory, and mental dysfunction,” in *Emotion and the Amygdala*, ed J. P. Aggleton (New York, NY: Wiley-Liss), 339–351.

Lee, C. C. (1990). Fuzzy logic in control systems: fuzzy logic controller. I. *IEEE Trans. Syst. Man Cyber.* 20, 404–418. doi: 10.1109/21.52551

Li, H., Liu, M., and Zhang, F. (2017). Geomagnetic navigation of autonomous underwater vehicle based on multi-objective evolutionary algorithm. *Front. Neurorobot.* 11:34. doi: 10.3389/fnbot.2017.00034

Lin, C. M., and Chung, C. C. (2015). Fuzzy brain emotional learning control system design for nonlinear systems. *Int. J. Fuzzy Syst.* 17, 117–128. doi: 10.1007/s40815-015-0020-9

Lin, C. M., Chung, C. M., and Hsu, C. F. (2009). “Adaptive cmac control system design for a class of nonlinear systems,” in *IEEE International Conference on Systems, Man and Cybernetics* (San Antonio, TX), 4508–4513.

Lin, C. M., and Hsu, C. F. (2015). Hybrid fuzzy sliding-mode control of an aeroelastic system. *J. Guid. Cont. Dyn.* 25, 829–832. doi: 10.2514/2.4955

Lin, F. J., Sun, I. F., Yang, K. J., and Chang, J. K. (2016). Recurrent fuzzy neural cerebellar model articulation network fault-tolerant control of six-phase permanent magnet synchronous motor position servo drive. *IEEE Trans. Fuzzy Syst.* 24, 153–167. doi: 10.1109/TFUZZ.2015.2446535

Liu, Z., Chen, C., Zhang, Y., and Chen, C. L. P. (2015). Adaptive neural control for dual-arm coordination of humanoid robot with unknown nonlinearities in output mechanism. *IEEE Trans. Cyber.* 45, 507–518. doi: 10.1109/TCYB.2014.2329931

Lotfi, E., and Akbarzadeht, M. R. (2013). “Emotional brain-inspired adaptive fuzzy decayed learning for online prediction problems,” in *IEEE International Conference on Fuzzy Systems* (Hyderabad), 1–7.

Meda-Campana, J. A., Araceli, G. M., Rubio, J. D. J., Tapia-Herrera, R., Hernandez-Cortes, T., Curtidor-Lopez, A. V., et al. (2018). Design of stabilizers and observers for a class of multivariable ts fuzzy models on the basis of new interpolation functions. *IEEE Trans. Fuzzy Syst.* 26, 2649–2662. doi: 10.1109/TFUZZ.2017.2786244

Pan, Y., Liu, Y., Xu, B., and Yu, H. (2016). Hybrid feedback feedforward: an efficient design of adaptive neural network control. *Neural Netw.* 76, 122–134. doi: 10.1016/j.neunet.2015.12

Rubio, J. D. J. (2018). Discrete time control based in neural networks for pendulums. *Appl. Soft Comput.* 68, 821–832. doi: 10.1016/j.asoc.2017.04.056

Rubio, J. D. J., Garcia, E., Aquino, G., Aguilar-Ibanez, C., Pacheco, J., and Zacarias, A. (2018). Learning of operator hand movements via least angle regression to be teached in a manipulator. *Evol. Syst.* doi: 10.1007/s12530-018-9224-1. [Epubh ahead of print].

Rubio, J. J. (2012). Modified optimal control with a backpropagation network for robotic arms. *IET Control Theory Appl.* 6, 2216–2225. doi: 10.1049/iet-cta.2011.0322

Shih, C. L., Li, Y. Z., Churng, S., and Lee, T. T. (1990). “Trajectory synthesis and physical admissibility for a biped robot during the single-support phase,” in *Proceedings of IEEE International Conference on Robotics and Automation, 1990*, Vol. 3 (Cincinnati, OH), 1646–1652.

Wu, Q., Lin, C.-M., Fang, W., Chao, F., Yang, L., Shang, C., and Zhou, C. (2018). Self-organizing brain emotional learning controller network for intelligent control system of mobile robots. *IEEE ACCESS* 6, 59096–59108. doi: 10.1109/ACCESS.2018.2874426

Zhao, J., and Lin, C. M. (2017). An interval-valued fuzzy cerebellar model neural network based on intuitionistic fuzzy sets. *Int. J. Fuzzy Syst.* 19, 1–14. doi: 10.1007/s40815-017-0321-2

Keywords: brain emotional learning network, humanoid robot control, Sliding mode control, neural network control, fuzzy neural network

Citation: Fang W, Chao F, Lin C-M, Yang L, Shang C and Zhou C (2019) An Improved Fuzzy Brain Emotional Learning Model Network Controller for Humanoid Robots. *Front. Neurorobot*. 13:2. doi: 10.3389/fnbot.2019.00002

Received: 18 June 2018; Accepted: 11 January 2019;

Published: 04 February 2019.

Edited by:

Feihu Zhang, Northwestern Polytechnical University, ChinaReviewed by:

Tianhua Chen, University of Huddersfield, United KingdomMuye Pang, Wuhan University of Technology, China

Jose De Jesus Rubio, Instituto Politcnico Nacional, Mexico

Copyright © 2019 Fang, Chao, Lin, Yang, Shang and Zhou. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Fei Chao, fchao@xmu.edu.cn