Online Tuning of PID Controller Using a Multilayer Fuzzy Neural Network Design for Quadcopter Attitude Tracking Control

This study presents an online tuning proportional-integral-derivative (PID) controller using a multilayer fuzzy neural network design for quadcopter attitude control. PID controllers are simple but effective control methods. However, finding the suitable gain of a model-based controller is relatively complicated and time-consuming because it depends on external disturbances and the dynamic modeling of plants. Therefore, the development of a method for online tuning of quadcopter PID parameters may save time and effort, and better control performance can be achieved. In our controller design, a multilayer structure was provided to improve the learning ability and flexibility of a fuzzy neural network. Adaptation laws to update network parameters online were derived using the gradient descent method. Also, a Lyapunov analysis was provided to guarantee system stability. Finally, simulations concerning quadcopter attitude control were performed using a Gazebo robotics simulator in addition to a robot operating system (ROS), and their results were demonstrated.


INTRODUCTION
In the 4th industrial revolution era, the application of multi-copters has significantly expanded, and it has thus attracted the interest of many researchers specialized in multi-copter control engineering. Multi-copters need to maintain accurate attitudes to ensure stable flight, so the development of accurate and stable controllers for multi-copters is essential. The most popular controller used for multi-copters is the cascaded proportional-proportional integral derivative controller (PPID) (Hernandez and Frias, 2016;Salas et al., 2019;Santos et al., 2019;Xuan-Mung and Hong, 2019a,b). Since PID is a linear controller, it is generally hard to use to achieve the highest control performance for non-linear control systems (Sarabakha et al., 2017). Furthermore, in the design of a PID controller, it is necessary to obtain the exact mathematical model of the control system and to optimize the gain value to achieve the desired performance. However, this work requires complexity calculations and an accurate modeling plant.
Gain tuning for PID controllers is time-consuming, as it depends on the knowledge of dynamic plants and the experience of expert operators (Kim et al., 2007). Mathematical models are different, as they depend on the size and appearance of multi-copters. Even if it is the same aircraft, the center of gravity can be different (Kurak and Hodzic, 2018). Therefore, the gain values are always flexible, and it may take longer to find suitable values. However, such tasks must be completed correctly, as an incorrect calculation can lead to terrible problems, such as falling or bumping into a people or objects. In past decades, many studies have been conducted to solve such problems (time-consuming, costly, etc.; Fatan et al., 2013;Kuantama et al., 2017;Noordin et al., 2017;Rouhani et al., 2017;Sumardi and Riyadi, 2017;Prayitno et al., 2018;Thanh and Hong, 2018;Chen et al., 2019;Rabah et al., 2019;Soriano et al., 2020). The Ziegler Nichols method is a well-known for tuning PID parameters (Azman et al., 2017). However, it produces oscillatory responses and yields overshoot problems (Zahir et al., 2018). Moreover, it cannot be applied in the online-tuning of parameters, and the control performance can be further improved.
In recognizing the above-mentioned limitation of traditional PID controllers, many researchers have provided methods for auto-tuning PID parameters (Amoura et al., 2016;De Keyser et al., 2016;Concha et al., 2017;Live et al., 2017;Mendes et al., 2017;Wang, 2017;Bernardes et al., 2019). In which, using the fuzzy neural network to tune PID parameter controllers has attracted the attention of many researchers. In 2018, Davanipour et al. proposed a self-tuning PID controller based on a fuzzy wavelet neural network (Davanipour et al., 2018). In 2019, Tripathy et al. introduced a fuzzy PID controller for load frequency control using spider monkey optimization (Tripathy et al., 2019). In 2017, Wang et al. provided an improved fuzzy PID controller design using predictive functional control structure . These studies could reduce the effort required for tuning the required parameters for PID controllers. However, most of their methods were TABLE 1 | The major differences and improvements among this study and related works.

Method
Using multilayer membership function neural network to online adjust the PID controller's parameters complicated, but the control performance could be further improved.
In recent years, many studies have investigated the fuzzy neural network (FNN), which is a combination of fuzzy systems and neural networks. Therefore, combined FNN systems have the advantages of both fuzzy systems and neural networks, such as human-like reasoning and learning capabilities. Using the neural network structure, FNN parameters can be trained online to achieve suitable values. According to the above discussions, this study presents a method for auto-tuning PID parameters using multilayer fuzzy neural network (PID-MFNN) to control the attitude the quadcopters. Compared with the previous studies Davanipour et al., 2018;Tripathy et al., 2019), our proposed PID-MFNN has some advantages, such as no need for offline training. The major differences and improvements among this study and related works are shown in Table 1. Also, the fuzzy weights and fuzzy membership functions can be updated online, and the multilayer structure is applied to improve the control performance. The main contributions of this study are summarized as follows: (1) the successful design of a multilayer structure was provided to improve the learning ability and flexibility of a FNN; (2) the adaptation laws for updating the parameters of a controller online were derived by using the gradient descent method; (3) the simulation results of controlling the attitude of a quadcopter were provided to illustrate the effectiveness of the control design method.
The rest of this paper is organized as follows. The quadcopter dynamics model and quadcopter controller are given in section Problem Formulation. The PID-MFNN structure and its learning parameter are described in section PID-MFNN Structure and Parameter Learning. The simulation results of controlling the quadcopter attitude are presented in section Simulation Results. Finally, the conclusions are given in section Conclusion.

Quadcopter Dynamics Model
The quadcopter schematic and its reference frames are shown in Figure 1. In which, the earth-frame and quadcopter body frame are denoted by E x E , y E , z E and B x, y, z . The dynamic equations of the quadcopter are given as: where θ , φ, and ψ stand for the three Euler angles: roll, pitch, and yaw, respectively; J x , J y and J z stand for the moment of inertia for the x, y, and z axes, respectively; l denotes the length of the quadcopter's arm; U 1 denotes the total thrust on the body in the z-axis; U 2 , U 3 , and U 4 denote the roll, pitch, and yaw torques, respectively, which are given as where F i = C t 2 i denotes the thrust of the i-th motor; i denotes the speed of the i-th motor; C t and C d stand for the thrust and drag coefficients, respectively.

Quadcopter Controller
The quadcopter controller scheme is given in Figure 2. There are two kinds of controllers that need to be considered in quadcopter control: position controller and attitude controller. The PPID attitude controller is given in Figure 3. The output of the outerloop controller is given byθ where G θ , G φ , and G ψ denote the proportional gain for the roll, pitch, and yaw outer-loop controller, respectively; e θ , e φ , and e ψ are the angle errors of the roll, pitch, and yaw, respectively. The output of the PID inner-loop controller is given by , are the PID gains for the roll, pitch, and yaw inner-loop controller; eθ , eφ, and eψ denote the angular velocity errors of the roll, pitch, and yaw, respectively.
To obtain suitable values for the PID gains in the outerloop and inner-loop controller, the PID-MFNN was designed in the following section to adjust the gains for the PPID controller online.
The auto-tuning algorithm structure of the PID-MFNN controller is shown in Figure 4 in which, the MFNN structure is applied to adjust the PPID controller gains online. As shown in Figure 5, the MFNN structure includes five spaces as follows:   1) The input space: the fuzzy input vector, I = i 1 , i 2 , ..., i n i , is prepared in this space. Each node corresponds to one input and is directly transferred to the next space.
2) The membership function space: the membership grades are calculated in this space using the fuzzy inputs and the Gaussian membership functions (GMFs). As shown in Figure 6, the multilayer fuzzy membership function is provided to improve the learning ability and flexibility of the FNN. The outputs of this space are computed as: where m ijk and v ijk are the mean and variance of the GMFs.
3) The fuzzy firing space: the fuzzy firing strength is calculated in this space using the membership grades as follows: where f kl is the fuzzy firing strength for the l-th rule in the k-th layer. Then, the fuzzy firing vector is presented as 4) The fuzzy weight space: in this space, the fuzzy weights w klo are provided to connect the fuzzy firing space and output space. The fuzzy weight vector for the o-th output is presented as 5) The output space: the final outputs of this space are the PID gains for the outer-loop and inner-loop controllers. Defuzzification is performed using the product operation as follows: Using the proposed MFNN to adjust the PID gains online, the desired performance of the control network can be achieved. The adaptation laws for updating the parameters of the proposed PID-MFNN controller online are explained in the following section.

Parameter Learning
First, considering the outer-loop controller, Equation (3) can be rewritten, where the gains G θ , G φ , and G ψ are obtained using the output of the proposed MFNN in Equation (10).
Frontiers in Neurorobotics | www.frontiersin.org By rewriting the Equation (6), A Lyapunov function is defined as E 1 (t) = 1 2 e˙ (t) 2 , using the gradient descent approach and chain rule, and the parameter update laws for PID-MFNN can be obtained as follows:  Then, considering the inner-loop controller, Equation (5) can be rewritten, where the gains K θ where α is replaced by p, i, d . By rewriting the Equation (4), A Lyapunov function can be defined as E 2 (t) = 1 2 (e (t)) 2 , using the gradient descent approach and chain rule, and the parameter update laws for PID-MFNN can be obtained as: whereŵ α kl ,m α ijk , andv α ijk are the connecting weight, mean, and variance of the inner-loop MFNN controller; T can be T φ , T θ , or T , which are the quadcopter transfer functions for the yaw, pitch and roll angles; η α w , η α m , and η α v are the positive learning rates.
Using the proposed adaptation laws in (16-18) and (23-25), the parameters of the proposed PID-MFNN can be obtained. Then, the suitable gains for the outer-loop and inner-loop PPID controller can be achieved.

The Convergence Analysis
The Lyapunov cost function is defined as Then, By applying the Taylor expansion, From (16), By rewriting (28) using (29) and (16), By rewriting (27) using (30), From (31), if η g w is chosen to satisfy 0 < η g w < 2 ζ 2 , then V(t) < 0. Consequently, the convergence of the adaptation law is guaranteed by the Lyapunov theorem. Similarly, the convergence of the adaptation laws for η α w , η α m , and η α v can be obtained.
The computational complexity of our approaches using big O notation is given as: + Big O notation for PID-CFNN: + Big O notation for PID-MFNN: where: T: is the simulation time. n j 1 : the number of membership functions in input 1. n j 2 : the number of membership functions in input 2. n k : the number of layers.
In our proposed network, we choose n j 1 = n j 2 = 3 and n k = 2.

SIMULATION RESULTS
This section presents the results of the quadcopter attitude control using the proposed PID-MFNN controller, which is performed using the Gazebo robotics simulator and ROS. The Gazebo simulation is a multi-robot simulator for outdoor environments. It supports the interface of the quadcopter's firmware, Pixhawk 4 (PX4). Therefore, it is easy to implement our algorithm on the quadcopter and to receive or transmit necessary data. The basic commands using MAVlink are also supported. Inside PX4, many terrains, sensors, and quadcopter types are provided using JavaScript. In this study, the 3D Robotics IRIS quadcopter was used without additional sensors, which were simulated on the basic terrain as a real world environment for verifying our designed control system. The wind disturbances of 1.0 m/s are considered as external disturbances. The quadcopter parameters used in this model were chosen as shown in Table 2.
The goal of the control system is to control the attitude of the quadcopter following the reference set-point trajectory: The Gazebo simulation interface is shown in Figure 7. The tracking performances of the quadcopter attitude under square wave and sine wave commands are given in Figures 8, 11, respectively. The black line presents the reference set-point trajectory. The red and blue lines present the quadcopter attitude output using the PID-MFNN controller and the PID conventional fuzzy controller (PID-CFNN), respectively. The control signals under the square wave and sine wave commands are shown in Figures 9, 12, respectively. The tracking errors under the square wave and sine wave commands are given in Figures 10, 13, respectively. The comparison results concerning the root mean square error (RMSE) between the proposed PID-MFNN controller and PID-CFNN controller are given in Tables 3, 4. From Figures 8-13, it can be seen that the proposed PID-MFNN controller has better control performance with a faster convergence speed and smaller tracking errors compared with the conventional fuzzy controller, which demonstrates the effectiveness of our proposed control method. The RMSE was used to evaluate the effectiveness of the control performance as follows: where n s is the number of samples. The parameter settings of the proposed PID-MFNN controller were chosen as n i = 2, n j = 3, and n k = 2; the initial GMF for the fuzzy rules are given as m    initial fuzzy weight is w klo = 0.2; the learning rates were chosen as η g w = η α w = η α m = η α v = 0.005; the sampling time was 0.01 s. The same parameters were chosen for PID-CFNN, but only the number of layers was n k = 1, and all GMFs were put in one layer.
The simulation results in Figures 8, 10 show that the proposed PID-MFNN controller has better performance than the PID-CFNN controller in terms of the rising time, settling time, and overshoot. The multilayer structure in our proposed controller, which contains a multilayer MF, provides it with a better scope regarding errors and disturbances compared with the single MF structure in PID-CFNN. The proposed PID-MFNN had a little longer computation time than the PID-CFNN due to the processing time of the multilayer structure. However, this did not affect the control performance, and the proposed PID-MFNN controller could achieve a smaller RMSE in both simulation cases. Apply some methods such as latent analysis in Wu et al. (2019) and Wu et al. (2020) to reduce the computational cost for the proposed multilayer fuzzy neural network will be our future work.
Remark 1. The conventional PID usually hard to achieve high control performance for controlling non-linear systems due to its parameters are usually tuned for local points based on a linearization method (Mohan and Sinha, 2008;Cetin and Iplikci, 2015). Therefore, it is not suitable for controlling highly complex and non-linear systems, such as quadcopters. However, in our proposed PID-MFNN method, we designed adaptation laws for updating the MFNN network parameters online, so the output of the MFNN network can auto-tune the PID gains. Therefore, our proposed PID-MFNN is suitable for quadcopters as well as other non-linear control systems.
Remark 2. By separating the member functions into a multilayer structure, the proposed method can better cover the changes in inputs. Moreover, with the proposed multilayer structure, the learning ability and flexibility of the network can be further improved.
Remark 3. The difficulties and limits in this study are described as follows. When applying our method in an experiment with real quadcopters, suitable initial parameters needed to be chosen. Otherwise, control could be lost. In our study, the simulation results were obtained using a Gazebo robot simulator, and it was easy to choose these parameters using the laws. Finally, the effectiveness of the proposed method was illustrated through the results of the conducted numerical simulations of quadcopter attitude control using a Gazebo robotics simulator and ROS. Our designed controller can also apply to a real quadcopter, and in our future work, we plan to apply some optimal algorithms to achieve suitable initial parameters.

DATA AVAILABILITY STATEMENT
The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.