Distributed Control of Discrete-Time Linear Multi-Agent Systems With Optimal Energy Performance

1 Introduction

Inspired by biological motion in nature, the cooperative motion of multi-agent systems (MASs) has been studied extensively in the past decade (Wang et al., 2017, 2019; Wang and Sun, 2018; Wang et al., 2020b; Koru et al., 2021; Wang and Sun, 2021). Compared to a single agent, networked MASs have the advantages of fast command response and robustness. Due to the distributed network computing system having the characteristics of strong scalability and fast computing speed, the study of distributed cooperative control problems for multi-agent systems has attracted increasing attention of control scientists and robotics engineers by virtue of its extensive applications in many cases, such as mobile robots (Mu et al., 2017; Zhao et al., 2019), autonomous underwater vehicles (AUVs) (Zuo et al., 2016; Li et al., 2019), and spacecrafts (Zhang et al., 2018; 2021a). A classical framework for the cooperative control of MASs with switching topologies is discussed in the study by Olfati-Saber and Murray (2004). Ren and Beard (2005) have further relaxed the conditions given by Olfati-Saber and Murray (2004), which present some new results with regard to the consensus of linear MASs.

In practice, it is necessary to investigate the control problem of multi-agent systems in discrete time with most computer systems being discrete structures. In the study by Liang et al. (2017), the cooperative containment control problem of a nonuniform discrete-time linear multi-agent is studied, and a novel internal mode compensator is designed to deal with the uncertain part of system dynamics. A solution method of the discrete-time MAS decentralized consensus problem based on linear matrix inequality (LMI) is given in the study by Liang et al. (2018). The problem of multi-agent consensus control based on the terminal iterative learning framework is discussed by Lv et al. (2018) where an adaptive control method based on time-varying control input is proposed to improve the control performance of the system. Su et al. (2019) proposed a distributed control algorithm based on the low-gain feedback method and the modified algebraic Riccati equation to achieve a semi-global consensus of discrete-time MASs under input saturation conditions. A multi-agent consensus framework based on the distributed model predictive controller is proposed by Li and Li (2020), while the self-triggering mechanism is adopted to reduce communication cost and solve the problem of asynchronous discrete-time information exchange. Liu et al. (2020) proposed a distributed state feedback control algorithm based on the Markov state compensator to solve the problem, that is, some followers cannot directly obtain the leader’s own state information. For MASs with unknown system parameters, a distributed adaptive control protocol containing local information was designed by Li et al. (2020) to ensure the inclusiveness of the system. In the study by Li et al. (2021), a class of discrete-time MASs adaptive fault-tolerant tracking problem based on reinforcement learning is studied, in which an adaptive auxiliary signal variable is designed to compensate the effect of actuator faults on the control system.

In practical applications, the energy cost performance of the designed protocols should be considered carefully, especially for the systems with low loadability, for example, autonomous underwater vehicles and spacecrafts. In the study by Zhang et al. (2017), the discrete-time MAS optimal consensus problem is discussed, and a data-driven adaptive dynamic programming method is proposed to solve the problem, that is, it is difficult to obtain an accurate mathematical model of the system. Wen et al. (2018) constructed a reinforced learning framework based on fuzzy logic system (FLS) approximators for the identifier–actor–critic system to achieve optimal tracking control of MASs. An optimal signal generator is presented in the study by Tang et al. (2018), where an embedded control scheme by embedding the generator in the feedback loop is adopted to realize the optimal output consensus of multi-agent networks. Tan (2020) transformed the distributed H_∞ optimal tracking problem of a class of physically interconnected large-scale systems with a strict feedback form and saturated actuators into the equivalent control problem of MASs; meanwhile, a feedback control algorithm is designed to learn the optimal control input of the system. In the study by Wang et al. (2020a), the optimal consensus problem of MASs is decomposed into three sub-problems: input optimization, consensus state optimization, and dual optimization, and a distributed control algorithm is proposed to achieve the optimal consensus of the system. The nonuniform MAS distributed optimal steady-state regulation is investigated in the study by Tang (2020), and the results are extended to the case where the system only has real-time gradient information using high gain control techniques. The single-agent goal representation heuristic dynamic programming (GrHDP) technique is extended to the multi-agent consensus control problem in by Zhong and He (2020), and an iterative learning algorithm based on GrHDP is designed to make the local performance indexes of the system converge to the optimal value. In the study by Xu et al. (2021), the optimal control problem with piecewise-constant controller gain in a random environment is solved, and an improved Hamilton–Jacobi –Bellman (HJB) partial differential equation is obtained by the splitting method and Feynman-KAC formula.

However, to the authors’ best knowledge, there are very few studies focusing on the optimal control of discrete-time MASs only containing a directed spanning tree. In this study, the leader-based distributed optimal control problem of discrete-time linear MASs on directed communication topologies is investigated. A distributed discrete-time consensus protocol based on the directed graph is designed, and it is proved that the optimization of energy cost performance can be satisfied with the presented consensus protocol. Furthermore, the optimal solution can be obtained by solving the algebraic Riccati equation (ARE), and the design of the protocol presented in this study does not require global communication topology information and relies on only the agent dynamics and relative states of neighboring agents, which means that every agent manages its protocol in a fully distributed way.

Notation. ${\mathfrak{R}}^N$ stands for the Euclidean space with N-dimension; I_n is an identity matrix of n-order; 1_N is a N-dimensional vector with all elements equaling 1, and 0_m×n denotes a zero matrix of order m × n; ‖x‖ stands for the Euclidean norm of the vector x; A ⊗ B denotes the Kronecker product between the matrices A and B; P > 0 represents the positive definiteness of the matrix P, and P ≥ 0 represents the positive semi-definiteness of P; P⁻¹ and P^T are the inverse matrix and transpose matrix of P, respectively.

2 Preliminaries

2.1 Algebraic Graph Theory

A digraph $\mathcal{G}=\{\mathcal{V},\mathcal{E}\}$ is used to describe the communication topology of MASs, where $\mathcal{V}=\{{\nu }_1,\dots ,{\nu }_N\}$ denotes the set of nodes. An edge (ν_j, ν_i) is included in the set $\mathcal{E}$ if the relative information can be transfered from ν_i to ν_j. A path from ν_i to ν_j is made up of a set of edges (ν_i, ν_l1), … (ν_ln, ν_j). A graph is supposed to be connected if a path from ν_i to ν_j for all pairs of (ν_i, ν_j) existed. An adjacency matrix $\mathcal{A}={[a_{ij}]}_{N\times N}$ is used to describe the digraph $\mathcal{G}$, where a_ii = 0, and a_ij = 1, i ≠ j, if $({\nu }_j,{\nu }_i)\in \mathcal{E}$, but 0 otherwise. Let $\mathcal{L}={[l_{ij}]}_{N\times N}$ denote the Laplacian matrix of $\mathcal{G}$ such that l_ij = −a_ij for i ≠ j and $l_{ii}={\sum }_{j=1}^N{a}_{ij}$.

2.2 Problem Formulation

Considering a group of N agents with the discrete-time system presented by the following equation.

$\left\{\begin{array}{c}\hfill x_i\left(k+1\right)=A{x}_i\left(k\right)+B{u}_i\left(k\right)\hfill \\ \hfill x_0\left(k+1\right)=A{x}_0\left(k\right)\hfill \end{array}i=\mathrm{1,2},\dots ,N\right.,(1)$

where $x_i(k)\in {\mathfrak{R}}^p$ denotes the state variable and $u_i(k)\in {\mathfrak{R}}^q$ denotes the control input; A and B are constant matrices with suitable dimensions. The purpose of this study is to design a protocol that guarantees the states of N agents in Eq. 1 to achieve an asymptotic consensus. i.e., ${\lim }_{k\to \infty }‖x_i(k)-x_0(k)‖=0$ and optimizes the cost function (which will be defined later).

Assumption 1. the leader agent’s index is defined as 0, and the leader agent’s and the follower agent’s index are defined as 1, … , N,. The digraph $\mathcal{G}$ contains a directed spanning tree with the leader as the root node.

Lemma 1. (Matrix Inversion Lemma (Horn and Johnson, 1996)): For any nonsingular matrices E ∈ C^N×N, G ∈ C^N×M and the general matrices F ∈ C^N×N, H ∈ C^N×M holds. Then, the inverse of the matrix(E + FGH) is as follows.

${\left(E+FGH\right)}^{-1}=E^{-1}-E^{-1}F{\left(H{E}^{-1}F+G^{-1}\right)}^{-1}H{E}^{-1}.$

3 Main Results

In this section, a distributed optimal controller is designed to solve the consensus of the system in Eq. 1, and the optimization of cost function is achieved with the presented protocol.

Since Assumption 1 holds, the Laplacian matrix $\mathcal{L}$ can be regraded as (Zhang and Lewis, 2012)

$\mathcal{L}=\left[\begin{array}{cc}\hfill 0\hfill & \hfill {\mathbf{0}}_{1\times N}\hfill \\ \hfill {\mathcal{L}}_2\hfill & \hfill {\mathcal{L}}_1\hfill \end{array}\right],(2)$

where ${\mathcal{L}}_1$ is a nonsingular matrix.

Let ${\xi }_i(k)={\sum }_{j=0}^N{a}_{ij}(x_i(k)-x_j(k))$, then we have

$\xi \left(k\right)=\left({\mathcal{L}}_1\otimes I_p\right)\left[x\left(k\right)-{\tilde{x}}_0\left(k\right)\right],(3)$

where ${\tilde{x}}_0(k)$ = ${\mathbf{1}}_N\otimes x_0(k)x(k)={\left[x_1^{\text{T}}(k),\dots ,x_N^{\text{T}}(k)\right]}^{\text{T}}$, and $\xi (k)={\left[{\xi }_1^{\text{T}}(k),\dots ,{\xi }_N^{\text{T}}(k)\right]}^{\text{T}}$; it implies that the leader-following consensus of the system in Eq. 1 can be achieved i.e., ${\lim }_{k\to \infty }‖x_i(k)-x_0(k)‖=0$ and $i\in \left\{1,\dots ,N\right\}$ can be achieved if and only if ${\lim }_{k\to \infty }‖\xi (k)‖=0$.

A distributed optimal controller is developed as follows.

$u_i\left(k\right)={\left(\sum _{j=0}^N{a}_{ij}\right)}^{-1}\left[\sum _{j=1}^N{a}_{ij}{u}_j\left(k\right)-cK{\xi }_i\left(k\right)\right],i=1,\dots ,N,j=0,\dots ,N(4)$

where c represents the coupling strength, and K denotes the control gain matrix.

The error system can be obtained by taking the difference of Eq. 3 as follows.

$\xi \left(k+1\right)=\left(I_N\otimes A\right)\xi \left(k\right)+\left({\mathcal{L}}_1\otimes B\right)U\left(k\right),(5)$

where $U(k)={\left[u_1^T(k),\dots ,u_N^T(k)\right]}^T$.

Inspired by reference given by Zhang et al. (2021b), the cost function is chosen to be

$L\left(k\right)=\frac{1}{2}{\xi }^{\text{T}}\left(k\right)\mathcal{Q}\xi \left(k\right)+\frac{1}{2}{U}^{\text{T}}\left(k\right)\mathcal{R}U\left(k\right),(6)$

where the matrices $\mathcal{Q}={\mathcal{Q}}^{\text{T}}>0$ and $\mathcal{R}={\mathcal{R}}^{\text{T}}>0$ denote the appropriate weighting matrices. In addition, the energy-cost function constraint performance for the system in Eq. 5 is considered as follows.

$J=\sum _{k=0}^{\infty }L\left(k\right)=\sum _{k=0}^{\infty }\left(\frac{1}{2}{\xi }^{\text{T}}\left(k\right)\mathcal{Q}\xi \left(k\right)+\frac{1}{2}{U}^{\text{T}}\left(k\right)\mathcal{R}U\left(k\right)\right).(7)$

In Eq. 7, $\frac{1}{2}{\xi }^{\text{T}}(k)\mathcal{Q}\xi (k)$ represents the process cost and $\frac{1}{2}{U}^{\text{T}}(k)\mathcal{R}U(k)$ represents the control cost. Therefore, J can be considered the goal of comprehensive optimization of control energy and error quantity. Furthermore, a Hamiltonian equation is utilized to optimize the cost function L(k) as

$H\left(k\right)=-L\left(k\right)+{\mathrm{\lambda }}^{\text{T}}\left(k+1\right)f\left(k\right),(8)$

where λ^T (k + 1) represents the costate variable and $f(k)=(I_N\otimes A)+({\mathcal{L}}_1\otimes B)U(K)$.

Next, the protocol presented in Eq. 4 is proved to guarantee the optimization of the energy cost performance and stability of system in Eq. 5.

Theorem 1. For the given matrices Q = Q^T > 0 and R = R^T > 0, the cost function L(k) is optimized and the stability of system in Eq. 5 can be achieved if and only if the following ARE holds:

$P=A^{\text{T}}PA-A^{\text{T}}PB{\left(R+B^{\text{T}}PB\right)}^{-1}{B}^{\text{T}}PA+Q.(9)$

where P is the positive definite solution of Eq. 9, and the control gain matrix is $K={(R+c{B}^{\text{T}}PB)}^{-1}{B}^{\text{T}}PA$; c is a constant value satisfying the condition c > 1.

Proof. i) Optimization of Cost Function

(i–i) Necessity

The optimal control input can be solved from the equation as follows.

$\frac{\partial H\left(k\right)}{\partial U\left(k\right)}=-\mathcal{R}U\left(k\right)+\left({\mathcal{L}}_1^{\text{T}}\otimes B^{\text{T}}\right)\mathrm{\lambda }\left(k+1\right)=0,(10)$

Let λ(k) = −(I_N ⊗ P)ξ(k), then the optimal controller can be obtained in Eq. 10 as follows.

$U^{*}={\mathcal{R}}^{-1}\left({\mathcal{L}}_1^{\text{T}}\otimes B^{\text{T}}\right)\mathrm{\lambda }\left(k+1\right)=-{\mathcal{R}}^{-1}\left({\mathcal{L}}_1^{\text{T}}\otimes B^{\text{T}}P\right)\xi \left(k+1\right),(11)$

Let $\mathcal{R}=\frac{1}{c}({\mathcal{L}}_1^{\text{T}}{\mathcal{L}}_1\otimes R)$, where R = R^T > 0. Since Eq. 11 holds true, Eq. 5 can be rewritten as

$\begin{array}{ll}\hfill \xi \left(k+1\right)& =\left(I_N\otimes A\right)\xi \left(k\right)-\left({\mathcal{L}}_1\otimes B\right){\mathcal{R}}^{-1}\times \left({\mathcal{L}}_1^{\text{T}}\otimes B^{\text{T}}P\right)\xi \left(k+1\right)\hfill \\ \hfill & =\left(I_N\otimes A\right)\xi \left(k\right)-c\left({\mathcal{L}}_1\otimes B\right)\left({\mathcal{L}}_1^{-1}{\mathcal{L}}_1^{-\mathrm{T}}\otimes R^{-1}\right)\times \left({\mathcal{L}}_1^{\text{T}}\otimes B^{\text{T}}P\right)\xi \left(k+1\right)\hfill \\ \hfill & =\left[I_N\otimes {\left(I_p+cB{R}^{-1}{B}^{\text{T}}P\right)}^{-1}A\right]\xi \left(k\right),\hfill \end{array}(12)$

According to Lemma 1, it indicates that the expression of U* can be rewritten as

$\begin{array}{ll}\hfill U^{*}& =-c\left({\mathcal{L}}_1^{-1}{\mathcal{L}}_1^{\text{T}}\otimes R^{-1}\right)\left({\mathcal{L}}_1^{\text{T}}\otimes B^{\text{T}}P\right)\xi \left(k+1\right)\hfill \\ \hfill & =-c\left[{\mathcal{L}}_1^{-1}\otimes R^{-1}{B}^{\text{T}}P{\left(I_p+cB{R}^{-1}{B}^{\text{T}}P\right)}^{-1}A\right]\xi \left(k\right)\hfill \\ \hfill & =-c\left({\mathcal{L}}_1^{-1}\otimes K\right)\xi \left(k\right),\hfill \end{array}(13)$

where $K={(R+c{B}^{\text{T}}PB)}^{-1}{B}^{\text{T}}PA$ is the control gain of the optimal controller in Eq. 4, which can guarantee the optimization of the cost function L(k).Considering the costate variable λ(k) = −(I_N ⊗ P)ξ(k), it can be obtained by the following equation.

$\begin{array}{ll}\hfill \mathrm{\lambda }\left(k\right)& =\frac{\partial H\left(k\right)}{\partial \xi \left(k\right)}\hfill \\ \hfill & =-\mathcal{Q}\xi \left(k\right)-\left[I_N\otimes A^{\text{T}}P{\left(I_p+cB{R}^{-1}{B}^{\text{T}}P\right)}^{-1}A\right]\xi \left(k\right),\hfill \end{array}(14)$

which indicates that

$I_N\otimes P=\mathcal{Q}+I_N\otimes A^{\text{T}}PB\left(c^{-1}R+B^{\text{T}}PB\right)B^{\text{T}}PA+I_N\otimes A^{\text{T}}PA.(15)$

Let $\mathcal{Q}=I_N\otimes A^{\text{T}}PB(c^{-1}R+B^{\text{T}}PB)B^{\text{T}}PA-I_N\otimes A^{\text{T}}PB(R+B^{\text{T}}PB)B^{\text{T}}PA+I_N\otimes Q$ be a positive definite matrix, and Q = Q^T > 0 holds true, then we have

$P=A^{\text{T}}PA-A^{\text{T}}PB{\left(R+B^{\text{T}}PB\right)}^{-1}{B}^{\text{T}}PA+Q,(16)$

which is identical to the ARE presented in Eq. 9.It is to be noted that if c ≥ 1, we have ${(c^{-1}R+B^{\text{T}}PB)}^{-1}\ge {(R+B^{\text{T}}PB)}^{-1}$, which implies that $\mathcal{Q}\ge 0$. Then, the positive definiteness of $\mathcal{Q}>0$ is achieved by c > 1.

(i–ii) Sufficiency

Considering the following equation:

$\begin{array}{ll}\hfill & \left(U^{\text{T}}\left(k\right)+{\xi }^{\text{T}}\left(k\right)c\left[{\mathcal{L}}_1^{-\mathrm{T}}\otimes A^{\text{T}}PB{\left(R+c{B}^{\text{T}}PB\right)}^{-1}\right]\right)\times \left[{\mathcal{L}}_1^{\text{T}}{\mathcal{L}}_1\otimes \left(\frac{R}{c}+B^{\text{T}}PB\right)\right]\hfill \\ \hfill & \times \left(U\left(k\right)+c\left[{\mathcal{L}}_1^{-1}\otimes {\left(R+c{B}^{\text{T}}PB\right)}^{-1}{B}^{\text{T}}PA\right]\xi \left(k\right)\right).\hfill \end{array}(17)$

Based on $K={(R+c{B}^{\text{T}}PB)}^{-1}{B}^{\text{T}}PA$ and $U(k)=U^{*}(k)=-c({\mathcal{L}}_1^{-1}\otimes K)\xi (k)$, the abovementioned Eq. 17 can be rewritten as

$\begin{array}{ll}\hfill & \left[U{\left(k\right)}^{\text{T}}+\xi {\left(k\right)}^{\text{T}}\left({\mathcal{L}}_1^{-\mathrm{T}}\otimes c{K}^{\text{T}}\right)\right]\left[{\mathcal{L}}_1^{\text{T}}{\mathcal{L}}_1\otimes \left(\frac{R}{c}+B^{\text{T}}PB\right)\right]\times \left[U\left(k\right)+\left({\mathcal{L}}_1^{-1}\otimes cK\right)\xi \left(k\right)\right]\hfill \\ \hfill =& U{\left(k\right)}^{\text{T}}\left({\mathcal{L}}_1^{-\mathrm{T}}{\mathcal{L}}_1\otimes \frac{R}{c}\right)U\left(k\right)+U{\left(k\right)}^{\text{T}}\left({\mathcal{L}}_1^{-\mathrm{T}}{\mathcal{L}}_1\otimes B^{\text{T}}PB\right)U\left(k\right)+\xi {\left(k\right)}^{\text{T}}\left[I_N\otimes c{K}^{\text{T}}\left(R+c{B}^{\text{T}}PB\right)K\right]\xi \left(k\right)\hfill \\ \hfill & +2\xi {\left(k\right)}^{\text{T}}\left[{\mathcal{L}}_1\otimes K^{\text{T}}\left(R+c{B}^{\text{T}}PB\right)\right]\hfill \\ \hfill =& U^{\text{T}}\left(k\right)\mathcal{R}U\left(k\right)+\xi {\left(k\right)}^{\text{T}}\left(I_N\otimes c^2{K}^{\text{T}}{B}^{\text{T}}PBK\right)+{\xi }^{\text{T}}\left(k\right)\left[I_N\otimes c{K}^{\text{T}}\left(R+c{B}^{\text{T}}PB\right)K\right]\xi \left(k\right)\hfill \\ \hfill & -2{\xi }^{\text{T}}\left(k\right)\left(I_N\otimes c{K}^{\text{T}}{B}^{\text{T}}PA\right)\xi \left(k\right).\hfill \end{array}(18)$

Since Eq. 5 and Eq. 15 hold true, we have

$\begin{array}{ll}\hfill & {\xi }^{\text{T}}\left(k+1\right)\left(I_N\otimes P\right)\xi \left(k+1\right)-{\xi }^{\text{T}}\left(k\right)\left(I_N\otimes P\right)\xi \left(k\right)+{\xi }^{\text{T}}\left(k\right)\mathcal{Q}\xi \left(k\right)\hfill \\ \hfill =& {\xi }^{\text{T}}\left(k\right)\left(I_N\otimes A^{\text{T}}PA\right)\xi \left(k\right)+{\xi }^{\text{T}}\left(I_N\otimes c^2{K}^{\text{T}}{B}^{\text{T}}PBK\right)\xi \left(k\right)-2{\xi }^{\text{T}}\left(k\right)\left(I_N\otimes c{K}^{\text{T}}{B}^{\text{T}}PA\right)\xi \left(k\right)\hfill \\ \hfill & -{\xi }^{\text{T}}\left(k\right)\left(I_N\otimes P\right)\xi \left(k\right)+{\xi }^{\text{T}}\left(k\right)\mathcal{Q}\xi \left(k\right)\hfill \\ \hfill =& {\xi }^{\text{T}}\left(k\right)\left(I_N\otimes c^2{K}^{\text{T}}{B}^{\text{T}}PBK\right)\xi \left(k\right)-2{\xi }^{\text{T}}\left(k\right)\left(I_N\otimes c{K}^{\text{T}}{B}^{\text{T}}PA\right)\hfill \\ \hfill & +{\xi }^{\text{T}}\left(k\right)\left[I_N\otimes A^{\text{T}}PB{\left(\frac{R}{c}+B^{\text{T}}PB\right)}^{-1}{B}^{\text{T}}PA\right]\xi \left(k\right).\hfill \end{array}(19)$

According to the following conditional equation

$\begin{array}{ll}\hfill & c{K}^{\text{T}}\left(R+c{B}^{\text{T}}PB\right)K\hfill \\ \hfill =& c{A}^{\text{T}}PB{\left(R+c{B}^{\text{T}}PB\right)}^{-1}\left(R+c{B}^{\text{T}}PB\right){\left(R+c{B}^{\text{T}}PB\right)}^{-1}{B}^{\text{T}}PA\hfill \\ \hfill =& c{A}^{\text{T}}PB{\left(R+c{B}^{\text{T}}PB\right)}^{-1}{B}^{\text{T}}PA\hfill \\ \hfill =& A^{\text{T}}PB{\left(\frac{R}{c}+B^{\text{T}}PB\right)}^{-1}{B}^{\text{T}}PA.\hfill \end{array}(20)$

Then, Eq. 19 can be regarded as

$\begin{array}{ll}\hfill & {\xi }^{\text{T}}\left(k\right)\left(I_N\otimes c^2{K}^{\text{T}}{B}^{\text{T}}PBK\right)\xi \left(k\right)-2{\xi }^{\text{T}}\left(k\right)\left(I_N\otimes c{K}^{\text{T}}{B}^{\text{T}}PA\right)\xi \left(k\right)\hfill \\ \hfill & +{\xi }^{\text{T}}\left(k\right)\left[I_N\otimes c{K}^{\text{T}}\left(R+c{B}^{\text{T}}PB\right)K\right]\hfill \\ \hfill =& {\xi }^{\text{T}}\left(k+1\right)\left(I_N\otimes P\right)\xi \left(k+1\right)-{\xi }^{\text{T}}\left(k\right)\left(I_N\otimes P\right)\xi \left(k\right)+{\xi }^{\text{T}}\left(k\right)\mathcal{Q}\xi \left(k\right).\hfill \end{array}(21)$

Let $V(k)=\frac{1}{2}{\xi }^{\text{T}}(k)(I_N\otimes P)\xi (k)$ denote the Lyapunov function, then, substituting Eq. 21 into Eq. 18, we have

$\begin{array}{ll}\hfill & \left[U^{\text{T}}\left(k\right)+{\xi }^{\text{T}}\left(k\right)\left({\mathcal{L}}_1^{\text{T}}\otimes c{K}^{\text{T}}\right)\right]\left[{\mathcal{L}}_1^{\text{T}}{\mathcal{L}}_1\otimes \left(\frac{R}{c}+B^{\text{T}}PB\right)\right]\times \left[U\left(k\right)+\left({\mathcal{L}}_1^{-1}\otimes cK\right)\xi \left(k\right)\right]\hfill \\ \hfill =& \begin{array}{c}\hfill \underbrace{U^{\text{T}}\left(k\right)\mathcal{R}U\left(k\right)+{\xi }^{\text{T}}\left(k\right)\mathcal{R}\xi \left(k\right)}\hfill \\ \hfill 2L\left(k\right)\hfill \end{array}+\begin{array}{c}\hfill \underbrace{{\xi }^{\text{T}}\left(k+1\right)\left(I_N\otimes P\right)\xi \left(k+1\right)-{\xi }^{\text{T}}\left(k\right)\left(I_N\otimes P\right)\xi \left(k\right)}\hfill \\ \hfill 2\mathrm{\Delta }V\left(k\right)\hfill \end{array}.\hfill \end{array}(22)$

Let $\varphi =[U^{\text{T}}(k)+{\xi }^{\text{T}}(k)({\mathcal{L}}_1^{\text{T}}\otimes c{K}^{\text{T}})][{\mathcal{L}}_1^{\text{T}}{\mathcal{L}}_1\otimes (\frac{R}{c})+B^{\text{T}}PB][U(k)+({\mathcal{L}}_1\otimes cK)\xi (k)]$, that is ϕ = 0 holds true if and only if U(k) = U*(k) holds, and the cost function L(k) can be rewritten as

$L\left(k\right)=-\mathrm{\Delta }V\left(k\right)+\frac{1}{2}\varphi .(23)$

Then, the cost function can be optimized, that is, L*(k) = −ΔV(k) with the controller $U^{*}(k)=-c({\mathcal{L}}_1^{-1}\otimes K)\xi (k)$.Hence, it indicates that the optimal performance index J* is derived as follows.

$J^{*}=\sum _{k=0}^{\infty }{L}^{*}\left(k\right)=-\sum _{k=0}^{\infty }\mathrm{\Delta }V\left(k\right)=-\underset{k\to \infty }{\lim }V\left(k\right)+V\left(0\right),(24)$

where V (0) represents the initial value of V(k).

(ii) The Stability of System

Based on the expressions of U*(k), we have

$\begin{array}{ll}\hfill \mathrm{\Delta }V\left(k\right)& =-L\left(k\right)\hfill \\ \hfill & =-\frac{1}{2}{\xi }^{\text{T}}\left(k\right)\mathcal{Q}\xi \left(k\right)-\frac{1}{2}{\xi }^{\text{T}}\left(k\right)\left({\mathcal{L}}_1^{-\mathrm{T}}\otimes c{K}^{\text{T}}\right)\times \left({\mathcal{L}}_1^{-\mathrm{T}}{\mathcal{L}}_1\otimes \frac{R}{c}\right)\left({\mathcal{L}}_1^{-1}\otimes cK\right)\xi \left(k\right)\hfill \\ \hfill & =-\frac{1}{2}{\xi }^{\text{T}}\left(k\right)\left(\mathcal{Q}+I_N\otimes c{K}^{\text{T}}RK\right)\xi \left(k\right)\le 0.\hfill \end{array}(25)$

It is inferred from Eq. 25 that − lim_k→∞V(k) = 0. Then, the optimal performance index J* can be rewritten as

$J^{*}=-\underset{k\to \infty }{\lim }V\left(k\right)+V\left(0\right)=V\left(0\right).(26)$

As a consequence, the conditions in Theorem 1 are all satisfied, which completes the proof.

Remark 1. Based on Theorem 1, it is obvious that the value of the control gain matrix K mainly depends on the matrix P and the coupling strength c, where the value of P is directly solved by Eq. 9, and c is a constant value satisfying the condition c > 1. Therefore, the design of the control protocol u_i(k) in Eq. 4 does not require global communication topology information and relies only on the agent dynamics and relative states of neighboring agents, that is, every agent manages its control protocol u_i(k) in a fully distributed way.

Remark 2. The topology considered in this study is a structure containing only one directed spanning tree, which means that the agent can only obtain the information of a single neighbor, and we prove the effectiveness of the proposed distributed optimal controller under the abovementioned conditions. In fact, the proposed controller is also suitable for the case with the general case, such as reference given by Wang et al. (2017), Wang et al. (2019).

4 Numerical Example

In this section, a numerical example is provided to demonstrate the effectiveness of the proposed controller.

Considering a network with seven agents, the communication topology is described by Figure 1. Moreover, the system parameters of each agent are given as follows (Xi et al., 2020).

$A=\left[\begin{array}{ccc}\hfill 1.0052\hfill & \hfill 0.0102\hfill & \hfill -0.0998\hfill \\ \hfill 0.0461\hfill & \hfill 1.0411\hfill & \hfill 0.0998\hfill \\ \hfill -0.1049\hfill & \hfill -0.2047\hfill & \hfill 0.9950\hfill \end{array}\right],B={\left[\begin{array}{ccc}\hfill -0.0677\hfill & \hfill -0.0246\hfill & \hfill 0.1559\hfill \end{array}\right]}^{\text{T}}.$

FIGURE 1

FIGURE 1. Communication topology among seven agents.

Let R = 10, Q = 10*I₃, and the coupling strength c = 2, then the matrix P and the control gain K can be calculated by Theorem 1. The initial conditions are given by x₀ (0) = [0.2–0.2 0.3]^T, x₁ (0) = [0.1 0.2 0.2]^T, x₂ (0) = [−0.15–0.1 0.1]^T, x₃ (0) = [0.3 0.2 0.1]^T, x₄ (0) = [−0.2 0.2–1.1]^T, x₅ (0) = [1.3 0.1–0.1]^T, and x₆ (0) = [1.0 0.5 1.5]^T. Then, the trajectories of the state norm and tracking error norm are shown in Figures 2, 3.

FIGURE 2

FIGURE 2. State norm of seven agents.

FIGURE 3

FIGURE 3. State tracking error norm.

It can be seen from Figures 2, 3 that six followers can track the leader successfully within about 11 s by using the proposed optimal controller, and the steady-state tracking error is less than 2.0. In addition, it is shown in Figure 4 that the control input of six agents will nearly reach zero at about 13 s.

FIGURE 4

FIGURE 4. Control input u_i of six agents.

Moreover, the trajectories of energy cost performance J are displayed in Figure 5, which shows that the optimal performance of J equals 924. It can be acquired from Theorem 1 that the theoretical value of the optimal performance is $J^{*}=V(0)=\frac{1}{2}{\xi }^{\text{T}}(0)(I_N\otimes P)\xi (0)=924.066$. Consequently, the simulated value of J* is consistent with its theoretical value, which proves that the controller proposed in this study satisfies the optimality requirements.

FIGURE 5

FIGURE 5. Trajectories of energy cost performance.

5 Conclusion

In this study, the leader-based distributed optimal control of discrete-time linear MASs only containing a directed spanning tree has been investigated. A distributed optimal consensus control protocol is presented to guarantee that multiple followers can successfully track the leader. It can be proved that the proposed protocol can ensure the optimization of the energy performance index with the optimal gain parameters which can be realized by solving the ARE. Moreover, the design of the protocol presented in this study is independent with the global information of topologies, which indicates that every agent manages its protocol in a fully distributed way. Finally, a numerical example which illustrates the effectiveness of the designed protocol is reported.

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.

Author Contributions

GH is responsible for the simulation and the writing of this manuscript. ZZ is responsible for the design idea of this study. WY is responsible for the revision of this manuscript.

Funding

This study was supported in part by the National Key R&D Program of China 2019YFB1310303, in part by the Key R&D program of Shaanxi Province, 2021GY-289, in part by the National Natural Science Foundation of China under Grant U21B2047, Grant U1813225, Grant 61733014, and Grant 51979228.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Horn, R., and Johnson, C. (1996). Matrix Analysis. Berlin: Springer.

Google Scholar

Koru, A. T., Sarslmaz, S. B., Yucelen, T., and Johnson, E. N. (2021). Cooperative Output Regulation of Heterogeneous Multiagent Systems: A Global Distributed Control Synthesis Approach. IEEE Trans. Automat. Contr. 66, 4289–4296. doi:10.1109/TAC.2020.3032496

CrossRef Full Text | Google Scholar

Li, H., and Li, X. (2020). Distributed Model Predictive Consensus of Heterogeneous Time-Varying Multi-Agent Systems: With and without Self-Triggered Mechanism. IEEE Trans. Circuits Syst. 67, 5358–5368. doi:10.1109/TCSI.2020.3008528

CrossRef Full Text | Google Scholar

Li, H., Wu, Y., and Chen, M. (2021). Adaptive Fault-Tolerant Tracking Control for Discrete-Time Multiagent Systems via Reinforcement Learning Algorithm. IEEE Trans. Cybern. 51, 1163–1174. doi:10.1109/TCYB.2020.2982168

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, J., Du, J., and Chang, W.-J. (2019). Robust Time-Varying Formation Control for Underactuated Autonomous Underwater Vehicles with Disturbances under Input Saturation. Ocean Eng. 179, 180–188. doi:10.1016/j.oceaneng.2019.03.017

CrossRef Full Text | Google Scholar

Li, N., Fei, Q., and Ma, H. (2020). Distributed Adaptive Containment Control for a Class of Discrete-Time Nonlinear Multi-Agent Systems with Unknown Parameters and Control Gains. J. Franklin Inst. 357, 8566–8590. doi:10.1016/j.jfranklin.2020.06.009

CrossRef Full Text | Google Scholar

Liang, H., Li, H., Yu, Z., Li, P., and Wang, W. (2017). Cooperative Robust Containment Control for General Discrete‐time Multi‐agent Systems with External Disturbance. IET Control. Theor. Appl. 11, 1928–1937. doi:10.1049/iet-cta.2016.1475

CrossRef Full Text | Google Scholar

Liu, Z., Yan, W., Li, H., and Zhang, S. (2020). Cooperative Output Regulation Problem of Discrete-Time Linear Multi-Agent Systems with Markov Switching Topologies. J. Franklin Inst. 357, 4795–4816. doi:10.1016/j.jfranklin.2020.02.020

CrossRef Full Text | Google Scholar

Lv, Y., Chi, R., and Feng, Y. (2018). Adaptive Estimation‐based TILC for the Finite‐time Consensus Control of Non‐linear Discrete‐time MASs under Directed Graph. IET Control. Theor. Appl. 12, 2516–2525. doi:10.1049/iet-cta.2018.5602

CrossRef Full Text | Google Scholar

Mahmoud, M. S., Khan, G. D., Yu, Z., Li, P., and Wang, W. (2018). Lmi Consensus Condition for Discrete-Time Multi-Agent Systems. Ieee/caa J. Autom. Sinica 5, 509–513. doi:10.1109/JAS.2016.7510016

CrossRef Full Text | Google Scholar

Mu, B., Chen, J., Shi, Y., and Chang, Y. (2017). Design and Implementation of Nonuniform Sampling Cooperative Control on a Group of Two-Wheeled mobile Robots. IEEE Trans. Ind. Electron. 64, 5035–5044. doi:10.1109/tie.2016.2638398

CrossRef Full Text | Google Scholar

Olfati-Saber, R., and Murray, R. M. (2004). Consensus Problems in Networks of Agents with Switching Topology and Time-Delays. IEEE Trans. Automat. Contr. 49, 1520–1533. doi:10.1109/TAC.2004.834113

CrossRef Full Text | Google Scholar

Su, H., Ye, Y., Qiu, Y., Cao, Y., and Chen, M. Z. Q. (2019). Semi-global Output Consensus for Discrete-Time Switching Networked Systems Subject to Input Saturation and External Disturbances. IEEE Trans. Cybern. 49, 3934–3945. doi:10.1109/TCYB.2018.2859436

PubMed Abstract | CrossRef Full Text | Google Scholar

Tan, L. N. (2020). Distributed H∞ Optimal Tracking Control for Strict-Feedback Nonlinear Large-Scale Systems with Disturbances and Saturating Actuators. IEEE Trans. Syst. Man. Cybern, Syst. 50, 4719–4731. doi:10.1109/TSMC.2018.2861470

CrossRef Full Text | Google Scholar

Tang, Y., Deng, Z., and Hong, Y. (2019). Optimal Output Consensus of High-Order Multiagent Systems with Embedded Technique. IEEE Trans. Cybern. 49, 1768–1779. doi:10.1109/TCYB.2018.2813431

PubMed Abstract | CrossRef Full Text | Google Scholar

Tang, Y. (2020). Distributed Optimal Steady-State Regulation for High-Order Multiagent Systems with External Disturbances. IEEE Trans. Syst. Man. Cybern, Syst. 50, 4828–4835. doi:10.1109/TSMC.2018.2866902

CrossRef Full Text | Google Scholar

Wang, B., Chen, W., and Zhang, B. (2019). Semi-global Robust Tracking Consensus for Multi-Agent Uncertain Systems with Input Saturation via Metamorphic Low-Gain Feedback. Automatica 103, 363–373. doi:10.1016/j.automatica.2019.02.002

CrossRef Full Text | Google Scholar

Wang, B., Wang, J., Zhang, B., and Li, X. (2017). Global Cooperative Control Framework for Multiagent Systems Subject to Actuator Saturation with Industrial Applications. IEEE Trans. Syst. Man. Cybern, Syst. 47, 1270–1283. doi:10.1109/TSMC.2016.2573584

CrossRef Full Text | Google Scholar

Wang, Q., Duan, Z., and Wang, J. (2020a). Distributed Optimal Consensus Control Algorithm for Continuous-Time Multi-Agent Systems. IEEE Trans. Circuits Syst. 67, 102–106. doi:10.1109/TCSII.2019.2900758

CrossRef Full Text | Google Scholar

Wang, Q., Psillakis, H. E., and Sun, C. (2020b). Adaptive Cooperative Control with Guaranteed Convergence in Time-Varying Networks of Nonlinear Dynamical Systems. IEEE Trans. Cybern. 50, 5035–5046. doi:10.1109/TCYB.2019.2916563

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, Q., and Sun, C. (2018). Adaptive Consensus of Multiagent Systems with Unknown High-Frequency Gain Signs under Directed Graphs. IEEE Trans. Syst. Man, Cybernetics: Syst. 50, 2181–2186. doi:10.1109/TSMC.2018.2810089

CrossRef Full Text | Google Scholar

Wang, Q., and Sun, C. (2021). Distributed Asymptotic Consensus in Directed Networks of Nonaffine Systems with Nonvanishing Disturbance. Ieee/caa J. Autom. Sinica 8, 1133–1140. doi:10.1109/JAS.2021.1004021

CrossRef Full Text | Google Scholar

Wei Ren, W., and Beard, R. W. (2005). Consensus Seeking in Multiagent Systems under Dynamically Changing Interaction Topologies. IEEE Trans. Automat. Contr. 50, 655–661. doi:10.1109/TAC.2005.846556

CrossRef Full Text | Google Scholar

Wen, G., Chen, C. L. P., Feng, J., and Zhou, N. (2018). Optimized Multi-Agent Formation Control Based on an Identifier-Actor-Critic Reinforcement Learning Algorithm. IEEE Trans. Fuzzy Syst. 26, 2719–2731. doi:10.1109/TFUZZ.2017.2787561

CrossRef Full Text | Google Scholar

Xi, J., Wang, L., Zheng, J., and Yang, X. (2020). Energy-constraint Formation for Multiagent Systems with Switching Interaction Topologies. IEEE Trans. Circuits Syst. 67, 2442–2454. doi:10.1109/tcsi.2020.2975383

CrossRef Full Text | Google Scholar

Xu, S., Cao, J., Liu, Q., and Rutkowski, L. (2021). Optimal Control on Finite-Time Consensus of the Leader-Following Stochastic Multiagent System with Heuristic Method. IEEE Trans. Syst. Man. Cybern, Syst. 51, 3617–3628. doi:10.1109/TSMC.2019.2930760

CrossRef Full Text | Google Scholar

Zhang, H., Jiang, H., Luo, Y., and Xiao, G. (2017). Data-driven Optimal Consensus Control for Discrete-Time Multi-Agent Systems with Unknown Dynamics Using Reinforcement Learning Method. IEEE Trans. Ind. Electron. 64, 4091–4100. doi:10.1109/TIE.2016.2542134

CrossRef Full Text | Google Scholar

Zhang, H., and Lewis, F. L. (2012). Adaptive Cooperative Tracking Control of Higher-Order Nonlinear Systems with Unknown Dynamics. Automatica 48, 1432–1439. doi:10.1016/j.automatica.2012.05.008

CrossRef Full Text | Google Scholar

Zhang, Z., Shi, Y., and Yan, W. (2021a). A Novel Attitude-Tracking Control for Spacecraft Networks with Input Delays. IEEE Trans. Contr. Syst. Technol. 29, 1035–1047. doi:10.1109/TCST.2020.2990532

CrossRef Full Text | Google Scholar

Zhang, Z., Shi, Y., Zhang, Z., Zhang, H., and Bi, S. (2018). Modified Order-Reduction Method for Distributed Control of Multi-Spacecraft Networks with Time-Varying Delays. IEEE Trans. Control. Netw. Syst. 5, 79–92. doi:10.1109/TCNS.2016.2578046

CrossRef Full Text | Google Scholar

Zhang, Z., Yan, W., and Li, H. (2021b). Distributed Optimal Control for Linear Multiagent Systems on General Digraphs. IEEE Trans. Automat. Contr. 66, 322–328. doi:10.1109/TAC.2020.2974424

CrossRef Full Text | Google Scholar

Zhao, S., Li, Z., and Ding, Z. (2019). Bearing-Only Formation Tracking Control of Multiagent Systems. IEEE Trans. Automat. Contr. 64, 4541–4554. doi:10.1109/TAC.2019.2903290

CrossRef Full Text | Google Scholar

Zhong, X., and He, H. (2020). Grhdp Solution for Optimal Consensus Control of Multiagent Discrete-Time Systems. IEEE Trans. Syst. Man. Cybern, Syst. 50, 2362–2374. doi:10.1109/TSMC.2018.2814018

CrossRef Full Text | Google Scholar

Zuo, L., Yan, W., Cui, R., and Gao, J. (2016). A Coverage Algorithm for Multiple Autonomous Surface Vehicles in Flowing Environments. Int. J. Control. Autom. Syst. 14, 540–548. doi:10.1007/s12555-014-0454-0

CrossRef Full Text | Google Scholar