Broad Learning Aided Model Predictive Control With Application to Continuous Stirred Tank Heater

This paper presents a data-driven predictive controller based on the broad learning algorithm without any prior knowledge of the system model. The predictive controller is realized by regressing the predictive model using online process data and the incremental broad learning algorithm. The proposed model predictive control (MPC) approach requires less online computational load compared to other neural network based MPC approaches. More importantly, the precision of the predictive model is enhanced with reduced computational load by operating an appropriate approximation of the predictive model. The approximation is proved to have no influence on the convergence of the predictive control algorithm. Compared with the partial form dynamic linearization aided model free control (PFDL-MFC), the control performance of the proposed predictive controller is illustrated through the continuous stirred tank heater (CSTH) benchmark.


INTRODUCTION
Tracking control plays an important role in the modern industrial process and has drawn much attention both in academia and industry over these years . As an efficient feedforward control strategy, model predictive control (MPC) has shown its efficiency in the tracking control of multivariable systems with various constraints. Compared with other control strategies where the control law is calculated offline, the core idea of MPC is to calculate an optimal control sequence by solving a finite optimal control problem online (Mayne and Rawlings, 2001). MPC is proved to be easy for implementation and maintenance in practical industry (Hidalgo and Brosilow, 1990;Yin and Xiao, 2017). Due to these advantages, MPC has been successfully used in the tracking control of the hybrid energy storage system, modular multi-level converters, wind generation system and many other applications in recent literatures (Qin and Badgwell, 2003;Garcia-Torres et al., 2019;Guo et al., 2019;Kou et al., 2019).
Despite the successful applications of MPC, most predictive controllers rely heavily on the precision of the predictive model. For model-based MPC, the predictive model is usually generated using the first principle, i.e., the controlled auto-regressive intergraded moving average (CARIMA) model (Zhang et al., 2018). However, it is arduous to set up a precise model for modern industrial process based on the first principle due to the complicated mechanism. Some alternative predictive models, such as the time-domain model, are also hard to obtain in practice (Patwardhan et al., 1998). Fortunately, there exists a huge number of underutilized process data with the development of the measurement and storage technology. Motivated by this, the data-driven predictive controller design has attracted the interest of researchers in recent years.
To propose the data-driven predictive control strategy, multivariable statistical analysis (MSA) techniques are used to identify the predictive model based on the available process data. Partial least square (PLS) and its modified forms have been paid more attention to due to the low computation complexity (Gao et al., 2018). To extract the dynamic characteristics of the process, a dynamic PLS based predictive control is proposed based on the assumed autoregressive exogenous (ARX) model (Kaspar and Ray, 1993). Subspace identification techniques are also used to design the predictive controller, where the state space model can be directly identified based on the process data (Yi, 2016;Garg and Mhaskar, 2017;Meidanshahi et al., 2017;Zhou et al., 2017). However, most investigations on subspace identification mainly focus on the identification of linear systems (Wu et al., 2013;Li et al., 2020). To cope with nonlinear systems, the neural network (NN) is often applied for the predictive controller design due to its strong learning ability in nonlinear model regression.
Over the past several years, fruitful achievements have been reported regarding the combination of NN and MPC. A dynamic neural network based MPC (DNN-MPC) is proposed and demonstrated to show superior control performance than the mathematical model based MPC for the industrial baker's yeast drying process (Yuzgec et al., 2008). A self-organizing recurrent neural network (RNN) aided nonlinear MPC is proposed (Han et al., 2016), where the control performance is evaluated through a wastewater treatment process. To improve the control performance for nonlinear process, the radial basis function neural network (RBF-NN) is applied to design the model predictive controller (Yu et al., 2011). Particularly, an adaptive splitting and merging RBF-NN based MPC has recently been successfully implemented in a zinc hydrometallurgy plant. However, these NN based MPC strategies often require much computation load, which limits its extension to the online implementation for realtime control performance.
In this paper, a random vector functional-link neural network (RVFLNN), i.e., the broad learning system (Chen and Liu, 2018), is incorporated into the predictive controller design. By using the incremental broad learning algorithm, the predictive model can be identified adaptively based on the online process data. In addition, a Jacobian matrix is used to approximate the predictive model to enhance the precision of the predictive model. The main contributions can be summarized as follows: 1) A data-driven predictive control strategy is proposed based on the broad learning algorithm with no prior knowledge of the system model; 2) The proposed data-driven predictive controller can be updated online based on the incremental broad learning algorithm while requiring little computation load; 3) The proposed broad learning aided MPC strategy is verified on the continuous stirred tank heater (CSTH) benchmark for temperature adjustment.
The rest of this paper is organized as follows. Section 2 introduces the basic knowledge of the model predictive control and the fundamental structure of the broad learning system. Section 3 discusses the incremental broad learning algorithm for new training data and proposes the incremental broad learning based MPC. Section 4 verifies the control performance of the proposed broad learning based MPC through the CSTH benchmark. Section 5 concludes this paper.

Basic Knowledge of Model Predictive Control
Given a nonlinear process described as follows: where y(t) ∈ R m is the output of the system, u(t) ∈ R l is the output of the controller, n u and n y represent the order of the system. The core idea of MPC is to optimize the following quadratic problem online or equivalently, where Y sp is the setpoint of the process,Ŷ is the prediction of the output and ΔU is the incremental value of the controller output to be calculated. λ 1 and λ 2 are two weights to make a trade-off between the tracking error and the cost of the control action.

Basic Knowledge of Broad Learning
As an improved neural network generated from the RVFLNN, broad learning has been proven to be an efficient algorithm for regression and classification, where it is important to extract the features from the training data. A random initialization of the features is always a popular way due to its easy computation. However, the feature extracted by random projection may suffer from unpredictability and needs guidance. To avoid this, a set of sparse features can be extracted from the training data using the sparse autoencoder (SAE) discussed in Gong et al. (2015). In this way, the alternating direction method of multipliers (ADMM) technique can be used to solve the lasso problem in the broad learning algorithm (Chen and Liu, 2018) to fine-tune the weight matrices. The details of the broad learning is given as below.
Assume that U ∈ R N×l and Y ∈ R N×m respectively denote the collected input and output data with N samples. The ith mapped feature Z i ∈ R N×α can then be calculated as where ϕ(·) denotes the projection function, W ei ∈ R l×α and β ei ∈ R N×α are two random matrices. Denote Z n ≡ [Z 1 , Z 2 , . . ., Z n ] as the concentration of the mapped feature. Z n can then be used to generate the jth enhancement node H j ∈ R N×c as where ζ(·) denotes the projection function, W hj ∈ R α×c and β hj ∈ R N×c are two random matrices.
Similarly, denote H m ≡ [H 1 , H 2 , . . ., H m ] as the collection of the enhancement nodes. In this way, the global model can be represented as where A m n ∈ R N×(nα+mc) denotes the input matrix and W m ∈ R (nα+mc)×p denotes the weight matrix.
It is straightforward that W m can be calculated as where (·) + represents the moore-penrose pseudoinverse. To reduce the computation complexity, the broad learning algorithm makes use of the ridge regression theory proposed in Hoerl and Kennard (2000). Assume that A m n is of full column rank, we have and consequently, The functional-linked broad learning model is shown in Figure 1. More details on the broad learning can be referred to Chen and Liu (2018).

Incremental Broad Learning Algorithm
Though broad learning has been proven to be efficient for classification and regression, it is not easy to regress the process model based on the online data due to the unpredictable noise, model uncertainty and dynamic characteristics. To implement the predictive control in the framework of the broad learning, it is essential to develop the equivalent incremental broad learning algorithm. By updating the parameters of the broad learning model, the precision of the regression model can be further enhanced. The incremental broad learning algorithm for new training data was briefly discussed in Chen and Liu (2018). The details of the incremental broad learning for new training data will be presented in this section.
According to Chen and Wan (1999), the stepwise updating algorithm of a functional-linked neural network for a new enhancement node can be equivalently represented as adding several columns to the input matrix. Denote the initial input matrix as A n , the final input matrix is then given as A n+1 [A n |a] when a new enhancement node a is added to the neural network.
Inspired by the moore-penrose pseudo inverse of partitioned matrix mentioned in Diaz and Gutierrez (2006), the pseudo inverse of A n+1 can be calculated as In this way, the new weights W n+1 can be calculated as Denote X a and Y a as a group of new training input data and the corresponding output data. The incremental algorithm broad learning for new training data can be demonstrated as Figure 2.
The original combination of feature nodes and enhancement nodes forms the following input matrix A m n : The input matrix u A m n trained by the new input data can be calculated as It is obvious that u A m n can be equivalently constructed as adding several rows generated by U a to the bottom of A m n . Denote the input matrix generated by the new training data U a as A u , u A m n can then be represented as where The pseudoinverse of u A m n can be calculated as where The weight matrix of the new broad learning model can be calculated as The incremental broad learning algorithm for new training data can be summarized in Algorithm 1.
Remark 1. In fact, the broad learning algorithm is herein used to the establish the predictive model for control purpose. Note that the predictive model does not need to be retrained when the new data arrives if the incremental broad learning algorithm is applied. The main computation is spent on the update of the new weights (i.e., u W m n ) and the pseudo inverse of input matrix (i.e., u A m n ) based on the previous value when the predictive model needs to be updated, which requires less computational load compared with the basic broad learning algorithm (i.e., requires retraining the network model with full data)

Incremental Broad Learning Based Predictive Control
For predictive control purpose, the cost function J(k) to be minimized at the k th instant can be given as follows, where Y sp (k + 1) and n y , n u , n p , and n c are the dimension of an output vector, the dimension of the input vector, the predictive horizon and the control horizon, respectively.
Remark 2. It is worth mentioning that n p should be set to be longer than the settling time in order to satisfy the convergence condition of the predictive control (Rossiter, 2003). n p has no influence on the convergence of predictive control if n p is greater than the settling time. However, n p is often determined based on the practical experience since the settling time is unknown. In this paper, the length of the control horizon n c and prediction horizon n p are both set to be 50. In the broad learning algorithm, the relationship between input training data U and output training data Y can be represented as Therefore, regardless of the activation function, the correlation between u(k) and y(k) can be presented as However, the activation function F(·) is necessary to extract the nonlinear characteristics. In the broad learning, the prediction of y(k) is calculated aŝ where f(net) 1 − e net 1 + e net .
Note that the nonlinearity brings challenges to the receding optimization of the predictive control since the cost function to be Frontiers in Control Engineering | www.frontiersin.org November 2021 | Volume 2 | Article 788492 minimized is nonlinear. Therefore, an appropriate approximation of the predictive model is necessary. According to Eq. 41, the prediction at the instant k can be rewritten asŷ (k) G(u(k)), Assume that u(k) is known, G(u(k)) can then be calculated easily via the general inverse. However, the incremental ΔU is to be calculated by solving the quadratic problem as Eq. 30. Take the prediction of the y(k + 1) at the instant k as an example,ŷ where Δu(k + 1) is unknown.
Since G(u) is continuous differentiable at any u, define the Jacobian matrix of G(u) as In this way, the approximation of G(u(k + 1)) can be given aŝ According to the definition of G(·) and f(net) as presented in Eqs 41, 42. It holds that f′(net) 1 − f(net) 2 ≤ 1 and G(u) is continuously differentiable for any u ∈ R nu . The Jacobian matrix J G (u) is Lipschitz continuous for u ∈ R nu . Therefore, there exists a Lipschitz constant c such that for Theorem 1. Let Eqs 41, 42, 47 holds, replacing G(u(k)) witĥ G(u(k)) has no influence on the convergence of the quadratic problem as shown in Eq. 30.
Proof. Define the deviation E G betweenĜ(u(k + 1)) and G(u(k + 1)) as According to the mean value theorem, we have Consider the equivalence of Riemann sum and integral, it yields E G G(u(k + 1)) − G(u(k)) − J G (u(k))Δu(k + 1) Consider that λ 1 and λ 2 are two weight constants and it holds that λ 2 ≫ λ 1 , it yieldŝ Therefore, the convergence ofĴ(k) is equivalent to the convergence of J(k), which thus completes the proof.
Remark 3. The Lipschitz constant c is related to the spectral radius of W. In practice, Δu is often constrained due to the mechanical structure of the actuators. The deviation E G can be acceptable with an appropriate W.
Denote W as The Jacobian matrix J G (u(k)) can be constructed as where + denotes the Hadamard product and In addition, the prediction of y(k + i) at the k th instant y(k + i| k) can be given as which leads to where Substituting Eqs 57, 59 into Eq. 30, the cost function can be rewritten as subject to the same constraint as Eq. 5, where E(k) Y sp (k) − Y(k).
The proposed predictive control is illustrated in Figure 3.

BENCHMARK STUDY ON CONTINUOUS STIRRED TANK HEATER
In this section, the proposed predictive control is applied to tune the temperature in the continuous stirred tank  Frontiers in Control Engineering | www.frontiersin.org November 2021 | Volume 2 | Article 788492 7 heater (CSTH) benchmark. The performance of the proposed control strategy is compared with the partial form dynamic linearization aided model free control (PFDL-MFC) strategy proposed in Hou and Jin (2013) and Hou and Jin (2011). The PFDL-MFC algorithm is chosen for comparison because both algorithms need little prior knowledge of the system model and calculate the optimal control input sequence based on Eq. 4.
The continuous stirred tank heater (CSTH) is a typical pilot plant for control performance evaluation and process monitoring. The experimental CSTH system is developed in the Automation Laboratory at Department of Chemical Engineering, IIT Bombay as shown in Figure 4 (Thornhill et al., 2008). It mainly consists of two separated water storage tanks. The water is heated separately and recycled between the two tanks. The CSTH benchmark can be regarded as a 5-input-3-output system. u 1 , u 2 , and u 3 represent the inputs controlled by separate valves. u 4 and u 5 represent the outputs of two heaters. The three outputs are the water temperature T 1 (Tank 1), T 2 (Tank 2) and the water level h 2 (Tank 2), respectively. Table 1 gives the parameters of the CSTH benchmark in steady state.
Consider that the heat input is added to the state variables in the form of polynomial functions (Thornhill et al.,2008), the controlled system is nonlinear even if the flow rate is steady. For simplicity, we only use u 4 and u 5 as the controller outputs to be calculated. The designed predictive controller aims to make T 1 track its setpoint. If not specified, the other variables are set to be steady state in Table 1. The practical constraints in the CSTH benchmark are set to be Smooth approximation is a popular method to enhance the tracking performance (Chi and Liang, 2015). By smooth approximation, the setpoint can be changed smoothly from one desired state to the other. In the simulation, the setpoint is given as y sp (k) y s1 , 0 < k ≤ 800 λ sp y sp (t − 1) + 1 − λ sp y s2 , 800 < k ≤ 1400 λ sp y sp (t − 1) + 1 − λ sp y s1 , 1400 < k ≤ 2000 where y s1 49, y s2 50 and the smoothing coefficient λ sp is set to be 0.98. Consider that the proposed predictive control requires initializing its predictive model based on the available process data. In this simulation, the process inputs are set to be random square waves with white noise in the first 200 s. The initial predictive model is calculated using the process data collected in the first 200 s. After the 200 th instant, the predictive model is updated based on the online process data.
Define the prediction error E p at the instant k as where n c and n p are both set to be 50.
Other parameters in the PF-MFPC algorithm are respectively set to be λ 27, μ 1, ρ [1, 1, 1], η 1, and L 3. The illustrations of the designed parameters are shown in Table 2. The comparison of the prediction error is shown in Figures 5, 6, from which it can be seen that there exists a fluctuation at the 200 th , 800 th , and 1400 th instant. The fluctuation is caused by the setpoint change. Therefore, both algorithms need to update the model dynamics based on the online process data. The comparison shows that the prediction of the proposed control strategy converges much faster than the PFDL-MFC algorithm. In addition, the fluctuation peaks of the proposed broad learning aided MPC is also much lower than the PFDL-MFC algorithm.
It is worth mentioning that the fluctuation of E p occurs before the setpoint changes at the 1400 th and 4800 th instant since the proposed predictive controller is based on the multi-step prediction. In this simulation, it holds that n c n p 50, which implies that the prediction of the output and the incremental of the input can be foreseen in the future 50 sample instants.
The control performance of the proposed broad learning aided MPC and the PFDL-MFC algorithm are demonstrated in Figures 7, 8, respectively. As the setpoint changes, the input signal converges after a few steps of adjustment. Both algorithms show the efficiency in the tracking control for nonlinear systems. However, the proposed algorithm Weighting factor 1 η Step size constant 1 ρ Step size  converges much faster than the PFDL-MFC algorithm though it requires to collect the data during the first several instants. In addition, the parameters of the PFDL-MFC algorithm have a great influence on the control performance. In contrast, only the weight λ 1 , λ 2 and the prediction horizon n p can be adjustable in the proposed predictive control strategy, which shows minor influence on the control performance.  This paper proposes a broad learning based data-driven predictive control strategy without any prior knowledge of the system model.
The online computation load of the proposed predictive controller is reduced by using the incremental broad learning algorithm. In addition, the prediction precision is enhanced by approximating the predictive model using a Jacobian matrix. The convergence of  the predictive control strategy is discussed theoretically, which demonstrates that the approximation has no influence on the convergence of the predictive control algorithm. The effectiveness of the proposed broad learning based MPC is demonstrated for temperature adjustment through the continuous stirred tank heater benchmark. Compared with other NN based predictive control strategies, the proposed strategy can update the predictive model online with less online computation burden. Meanwhile, the proposed predictive controller generates the local nonlinearity between the input and output data compared with the dynamic linearization based model free control strategy, which enhances the prediction precision and control performance. The simulation result shows that the proposed predictive control strategy shows faster convergence both in the control performance and the prediction than the PFDL-MFC algorithm. In the future, more work can be done to further reduce the online computation burden.

DATA AVAILABILITY STATEMENT
Publicly available datasets were analyzed in this study. This data can be found here: http://www.ps.ic.ac.uk/∼nina/CSTHSimulation/index.htm.

AUTHOR CONTRIBUTIONS
MT and TG: conceptualization, methodology, writing. XL: supervision. KL: writing. All authors contributed to the article and approved the submitted version.