Containment control of multiple unmanned surface vessels with NN control via reconfigurable hierarchical topology

This paper investigates the containment control of multiple unmanned surface vessels with nonlinear dynamics. To solve the leader-follower synchronization problem in a containment control system, a hierarchical control framework with a topology reconfiguration mechanism is proposed, and the process of containment control is converted into the tracking of a reference signal for each vessel on its respective target heading by means of the light-of-sight (LOS) guidance. In a control system, the neural networks (NNs) are adopted to consider the uncertainty. In the follower layer, a connectivity controller with a topology reconfiguration mechanism is embedded, to change the converging positions of followers so as to avoid collision within the system, and to maintain the system connectivity when the communication equality is poor. The effectiveness of the hierarchical control framework proposed in this paper is valid by simulation.


. Introduction
Containment control (Cao and Ren, 2009;Meng et al., 2010) refers that, with the framework of multi-leader-multi-follower system, all the followers are driven by a control protocol so as to converge to the convex hull spanned by all the leaders.A significance of studying the containment control problem is that agents equipped with various advanced sensors can detect a variety of environmental information, including obstacles, etc., during the movement process, and complete the information fusion through distributed algorithms, whereby a dynamic security area is formed.Under the containment control protocol, the followers are able to converge to the safe zone and keep synchronized movement with the leader, achieving cooperation and guaranteeing system safety at the same time.In research aimed at multi-agent systems, containment control has yielded very rich research results (Li et al., 2013;Ma et al., 2015;Wen et al., 2016;Wang et al., 2020).
However, when the application background of the problem changes to marine, many theoretical results in multi-agent systems cannot be well applied.The reasons behind this phenomenon mainly result from the dynamics of the marine crafts, one is that the dynamic is complex, and it is difficult to deal with the strong nonlinearity and strong coupling, and the other is that the dynamic uncertainty is difficult to be known and describe.In motion control problems, there are two commonly used models for the dynamics of vessels: the model derived by Fossen (1994Fossen ( , 2002) ) and the Nomoto model (Banazadeh and Ghorbani, 2013).The three-degree-of-freedom model derived by Fossen is widely used in various types of problems because it includes kinematics-related state variables.However, the controller form is more complicated and often with higher-order derivative terms.The Nomoto model is usually applied to problems related to heading tracking because it establishes the transfer function relationship between rudder angle and heading.The controller form is simple and has fewer parameters to be determined, but the model accuracy is lower than that of the model derived by Fossen.When used in combination with guidance methods such as line-of-sight(LOS) guidance (Fossen et al., 2015), predictorbased line-of-sight(PLOS) guidance (Liu et al., 2016), adaptive line-of-sight(ALOS) guidance (Gu et al., 2017), etc., the controller form designed based on the Nomoto model is the simplest one.
For the uncertainty, the neural networks(NNs) have been applied effectively due to their distinct advantages in recognizing unknown nonlinearity (Li et al., 2020(Li et al., , 2023;;Ding and Wang, 2022;Ding et al., 2022;Shao et al., 2022;Teng et al., 2023;Zhou et al., 2023).In (Li et al., 2010), the radial basis function neural network (RBFNN) is used to consider the uncertainty of the system, which is combined with the dynamic surface control (DSC) technique to avoid the problem of complexity explosion, and its effectiveness is verified in the application of autopilot on marine crafts.In Peng et al. (2014), a distributed adaptive synchronization controller is proposed to provide good decoupling between the observer and controller design of a nonlinear multi-agent system, and to ensure that the state of each agent has bounded residuals.In Li et al. (2022), an RBFNN approximator was adopted to estimate the uncertainties, and by introducing the minimum learning parameter technique to minimize the computational burden associated with neural network weight updates.The NNs were combined with the described performance control when the dynamics were unmodeled (Shen et al., 2020), to guarantee all the followers asymptotically synchronized to the leader, and the synchronization errors within a prescribed level.In Chen et al. (2020), NNs were employed to approximate the unknown external disturbances and uncertain hydrodynamics of unmanned surface vessels, and an adaptive trajectory tracking controller with guaranteed transient performance was developed.
Besides the above model-related factors affecting the performance of the controller, safety is another important factor to be considered in practical application problems (Yang et al., 2022).In containment control problems, the topology of the system determines the convergence positions of followers (Wang et al., 2014).In practical problems, the follower is an entity with a certain volume rather than an idealized prime model, and the resulting collision problem within the system also constrains the translation of theory into practice.The collision avoidance and obstacle avoidance problem of a single vessel can be solved by the artificial potential field method (Liu et al., 2020;Mu and Peng, 2022) or other collision avoidance algorithm (Lu et al., 2022), but for the large-scale cooperative system, due to the network communication load and computational capacity constraints, this method is not very effective, and may even cause the system to be stuck due to repeatedly triggering the collision avoidance conditions.Therefore, considering system safety, it is necessary to seek a reasonable way to reconfigure the system topology from the perspective of the followers' convergence positions.That is to say, through some reasonable topology reconfiguration method, these topologies that may adversely affect the system are reconfigured to achieve specific control objectives such as collision avoidance, accelerating or decelerating the system convergence rate, and maintaining the system communication continuity (Mikulski et al., 2012;Haus et al., 2014;Griparic et al., 2022).
Inspired by the aforementioned studies, in this paper, based on the controller designed in Peng et al. (2014) which takes into account disturbances and model uncertainties with NNs, a synchronized control framework for a containment control system with topology reconfiguration mechanism is proposed, and the process of containment control is converted into the tracking of a reference signal for each vessel on its respective target heading by means of the LOS guidance method.With the proposed hierarchical control framework, the communication topology of the follower layer can be reconfigured to enable the transformation of follower formation and dispersion Securing the system throughout its operation.
. Materials and methods . .Graph theory denotes the set of edges, representing the link between every two vertices.The link between every two vertices in the graph is defined by the adjacency matrix A ∈ R n×n , where a ij = 1 means that there is an edge directed to v i from v j , otherwise a ij = 0.The degree matrix a ij denotes the degree of vertex i.Another adjacency matrix A 0 ∈ R n×n is defined to describe the relationship between the virtual leader and the other agents, denoted A 0 = diag(a i0 ), where a i0 = 1 indicates that there is an edge directed from the virtual leader to v i , otherwise a i0 = 0.In our study, a matrix denotes the whole system, which is given as The Laplacian matrix is defined where In ascendental order, the eigenvalues of the Laplace matrix are as follows The second minimal eigenvalue of the Laplacian matrix, λ 2 , is also known as the algebraic connectivity.For undirected graph, when and only when λ 2 > 0, the graph is connected (Fiedler, 1973).

. . Containment control
For a containment control system consisting m leaders as well as n − m followers, the Laplacian matrix can be expressed as where To analyze the containment control of a system, Lemma 1 is given: Lemma 1. Assume that the communication digraph G has a directed spanning forest.The sum of each row of −L −1 2 L 1 is 1 and the element of −L −1 2 L 1 is positive if and only if the ith leader has a directed path to the jth follower (Shan et al., 2021).
Then, the position vectors of leaders as well as followers are denoted as where .

. System modeling
The kinematic model of the ith unmanned surface vessel is given as where [X i , Y i ] T is the position vector within X E − Y E coordinate system, ψ i denotes the heading, [u i , v i , r i ] T ∈ R 3 denotes the linear velocity vector and the angular velocity vector of the ith unmanned surface vessel.
To make the controller design simpler, Assumption 1 is given.Assumption 1.For each vessel in control system, the linear velocity u is a positive constant and the angular velocity v = 0.
For marine crafts, which usually have separate speed controllers for speed control, it is a reasonable assumption that u is a constant.During manipulation, the motion in the direction of the linear velocity is small relative to the motion in the other directions so it can be ignored (Li et al., 2009).
Considering a containment control system consisting of several unmanned surface vessels, the dynamics of each is stated as (Li et al., 2010) ψi ψi = 0 1 0 0 where ψ i , ψi , and ω i representatively denote the heading, the rate of heading, and the heading moment of ith vessel, K and T are parameters that represent the maneuverability of marine crafts.
F ψi is an unknown nonlinear function of ψi , which is estimated by following equation .

. Control methods
In the context of the cooperative containment control problem guided by a virtual leader, the system in terms of topology could be segmented into three layers, as shown in Figure 1.Layer 0 is the virtual leader layer, which gives the reference tracking signal.Layer 1 is the real leader layer, where all the real leaders track the reference signal through the virtual leader.Layer 2 is the follower layer, according to the real-time position of the real leaders, the respective reference trajectory point can be calculated for each follower, and the path-following process is converted as a heading tracking process through the LOS guidance, so as to achieve the cooperative containment control.
A topology reconfiguration strategy based on communication measurements is also incorporated in the second layer, to reconfigure the communication topology in time so as to maintain the performance level of the system.
The control method of the system proposed in this paper is described in three steps: (i) the real leader's tracking of the reference heading, (ii) the follower's tracking of the respective reference trajectory under containment control, (iii) the follower's tracking of the new reference trajectory after a change of the system's communication topology.The hierarchical control framework proposed in this paper is shown in Figure 2. The controller for Layer1 and Layer 2 are the same in form, as shown in Figure 3, wherein the controller for Layer 1, ψ denotes ψ r , and in the controller for Layer 2, ψ denotes ψ iLOS .

. . . Design of LOS guidance law based on containment control system
By the real leaders' tracking of the reference heading, combined with the kinematic equations given as Eq. ( 5), and with Eq. ( 4), the followers' desired positions could be calculated timely.For the goal of achieving containment control, the path-following process is converted as a time-varying heading tracking process by means of LOS guidance.

FIGURE
The hierarchical control framework in this paper.

FIGURE
The controller for Layer and Layer .
To design the tracking controller, the Serret-Frenet coordinate system is usually used, which is established by taking the current position of the marine craft (X, Y) projected on the reference path (X d (ω), Y d (ω)) as the origin, where ω is the path parameter variable.The reference heading angle is where Ẏp (ω) = dY p (ω) dω, Ẋp (ω) = dX p (ω) dω, the changing rate of the path parameter variable satisfying where U = √ u 2 + v 2 denotes the speed of navigation.The schematic of LOS guidance is given in Figure 4, the LOS guidance law based on forward-looking distance is expressed as

FIGURE
The schematic of LOS guidance.
where is the forward-looking distance.Thus the reference heading angle of followers is where β = arctan v u reprensents the angle of drift.From geometrical relations in Figure 1, the error of path tracking is obtained as follows where Y e is the position tracking error, and the desired position φ ir = [X r , Y r ] T can be calculated by Eq. ( 4).
For path-following problems where the path parameters are known, the LOS guidance law is stated as the above equation.In this paper, the reference heading is known while the global path is unknown, also, with Assumption 1, the effect of drift angle is not considered.Thus the guidance law for the ith follower marine craft in the system can be designed as . . .Controller design in which K r is a feedback matrix, then the model can be written as where is Hurwitz, and L v , L r and F denote the virtual leaders, the real leaders, and followers respectively, let L v = {0}, L r = {1, 2, ..., m}, F = {m + 1, m + 2, ..., n}.ψ r (t) is the reference input, u i is the control input, f i (x i ) is the unknown uncertainty and ω i (t) is disturbance which is unknown but bounded.
According to Stone (1948), f i (x i ) is approximated with the neural networks, which is as follows where W i ∈ R s denotes the ∈ R s×m satisfying W i ≤ W M is a constant real matrix which denotes the ideal neural network (NN) weight matrix, ϕ i (•) is a known basis function, ε i is the neural network approximation error satisfying ε i ≤ ε M .
(1) Controller for vessels in Layer 1 As given in Zhang et al. (2011), the following control protocol is considered: where c ∈ R is a coupling gain, K = −B T P −1 is the feedback, of which P is a positive definite solution of linear matrix inequality (LMI) where Q ∈ R n×n is positive definite.
Considering the uncertain nonlinear dynamical terms in the model, an adaptive control term u iad is given as where Ŵi is the estimation of W i .Thus the control law can be written as Combined with Figure 1 and the aforementioned, the object of the control law (13) implementation is a directed graph.According the results given in Peng et al. (2014), the update law of Ŵi should be chosen as where Ŵ W i , k W are constant to be designed, e i and τ i is respectively given as follows then, all signals in the closed-loop network can be uniformly ultimately bounded.
(2) Controller for vessels in Layer 2 From the former analysis, in the follower layer (i.e.Layer 2) the process of tracking the reference trajectory by each follower could be considered as the heading tracking process led by the virtual leader.Thus, in Layer 2, each follower can be also represented as follows, as shown in Figure 5, where x 0i denotes the virtual leader for the ith follower.With this form, the follower receives control signals not directly from the leader, but from its corresponding virtual leader and its neighbors.
Thus, in Layer 2, the controller is designed as where the update law of Ŵi is 25) where Ŵ W Fi , k FW are constant to be designed.According to the proof in Peng et al. (2014), it can be also obtained that all signals in the closed-loop network can be uniformly ultimately bounded.
(3) Connectivity control The autopilot of the vessel, as an assistant tool in the process of navigation, if timely changes can be made based on the current or possible conditions when cooperating, the safety performance of the system and the efficiency of cooperation will be greatly improved.Several classical cases of topology reconfiguration are to change the converging positions of followers so as to avoid collision Frontiers in Computational Neuroscience frontiersin.org

FIGURE
The framework of connectivity control in Layer .
within the system, or to maintain the system connectivity when the communication equality is poor.A topology reconfiguration algorithm was given by Griparic et al. (2022), which enabled the algebraic connectivity of the system to reach the desired value through its proposed method of adding/removing links.The inputs of the topology reconfiguration algorithm are the initial local adjacency matrix of each marine craft A l F (0)(F means that the marine craft is a follower and l = 1, 2, 3, ..., n − m represents the index of marine craft), desired level of system performance λ 2ref , and user-defined value K λ 2 .The connectivity controller can be stated as follows the parameter K λ 2 > 0 is to be determined so as to have Combined with the RNN-based system performance identification model proposed by Liu et al. (2023), the features of the containment control system can be extracted as the following matrix, which is considered as the input of the RNN-based performance identification model where Input represents the input to the NN where the first row composed of ones and zeros describes the connection relationship between the leader and follower agents in the system, indicates that the leader is connected to the ith follower while 0 denotes no connection.The remaining three rows are vectors transformed from the node degree relativity matrix, d i denotes the degree of the ith follower, d N i denotes the sum of degrees of the neighbor set of the ith follower, and d N N i denotes the sum of degrees of the neighbor set nodes of the ith follower's neighbors.
The output is the performance indicators of the containment control system, given as where λ 2 is the algebraic connectivity of the network in Layer 2, and σ (X F P ) is used to describe the dispersion of converging positions of followers in the convex hull spanned by all leader.
Through this RNN-based performance identification model, the system topology with good collision-avoiding performance can be selected.Thus the reconfiguration of topology is finished.
To integrate topology reconfiguration with system control, it is assumed that the communication between each marine craft is continuous.Thus, topology reconfiguration can be done within a very short period of time which is represented as a time-varying Laplacian matrix in the control system, expressed as where L c0 denotes the initial topology of Layer 2, and L cR denotes the reconfigured topology of Layer 2.
Based on the premise of the containment control problem, only the topology of Layer 2 is reconfigured in this paper.The framework of connectivity control is shown in Figure 6, where the λ l 2 denotes the algebraic connectivity after each processing, and the identification object of system performance is whether the dispersion of followers is large enough so as to keep a safe distance for each two followers.The system performance identification module is a determination of the reconfigured topology output by the connectivity controller.If the determination is 'Yes", the reconfigured topology satisfying the system performance requirements is obtained and output.If not, the control signal will be transmitted back to the first step which is to input λ 2ref and start another round of topology reconfiguration.

. Simulation results and discussions
The simulation is implemented to validate the effect of the control law ( 21) and ( 24), LOS guidance law (13), as well as the connectivity control algorithm designed in Section 3.2.3.Considering a multiple unmanned surface vessel system (6) with 1 virtual leader, 3 real leaders, and 9 followers, whose initial communication topology is shown in Figure 7A.
The initial position vectors of real leaders are given as φ 1 = [0, 100] T , φ 2 = [0, −100] T and φ 3 = [100, 0] T .The initial position vectors of followers are given as φ The parameters are given as b i1 = 0.1256, b i2 = 0.3576, b i3 = 0.0278.The coupling gain c = 30, the feedback matrix K r = [−1 − 40] T , and the ordered reference input ψ r is given as When using the neural network for estimation, to make the estimation better, the adaptive parameters of the NN should take larger values.Thus taking Ŵ W i = 100 and Ŵ W Fi = 100.k W and k FW are taking as k Figures 8, 9 give the state profile of real leaders and followers, in which the real leaders track well on the reference signal given by the virtual leader, and with the LOS guidance law, the followers also achieve tracking of the targeted heading, thus converging into the convex hull spanned by leaders.
The path profile is given in Figure 10, in which can be observed that with the initial system communication topology, the converging positions of the followers are very tight.Most of the marine crafts are not distant enough from each other to meet the safety distance requirement, in which case the risk of collision within the system will be greatly increased.Thus, combined with the connectivity control method and the system performance identification model stated in Section 3.2.3, a reference value λ 2ref = 0.4 is chosen as the desired performance index of Layer 2 and K λ 2 is taken as K λ 2 = 0.1, which is used to reconfigure the communication topology of Layer 2 so as to disperse the converging positions of followers.The reconfigured system topology is shown in Figure 7B, accordingly, λ 2 = 0.4131.
The process of topology reconfiguration is performed as t = 1, 200s.It is presented in Figure 9 that the response of the control system when the reference path tracked by the followers is changed after the topology reconfiguration is completed.

. Conclusion
This paper investigated the containment control of multiple unmanned surface vessels with NN control via reconfigurable hierarchical communication topology.A hierarchical control framework was proposed, so as to transform the containment control problem of followers to the synchronization of reference  heading tracking, which is realized with the LOS guidance law.In the control system, the NNs are adopted to consider the uncertainty.In the follower layer, a connectivity controller with a topology reconfiguration mechanism was embedded to reconfigure the communication topology, which is used to improve the safety of the system when operating.It was shown in the simulation results that with this hierarchical control framework, the real leader and the virtual leader, and the follower and the real leader all achieved well-tracking, and the controller also achieved the tracking of the new reference signal without significant oscillations of control output when the communication topology of the follower layer was changed.In this case, the converging positions of the followers were dispersed throughout the system when tracking the reference heading during containment control of multiple marine crafts, which greatly reduced the risk of collisions within the system.

FIGURE
FIGURE Communication topology of followers in Layer .(A) System topology, (B) communication topology of follower , and (C) communication topology of follower .

FIGURE
FIGURESystem communication topology.(A) Initial system topology and (B) reconfigured system topology.

FIGURE
FIGUREState profile of leaders.

FIGURE
FIGUREState profile of followers.

FIGURE
FIGUREPath profile of leaders and followers.