Adaptive formation learning control for cooperative AUVs under complete uncertainty

Abstract

Introduction:

This paper addresses the critical need for adaptive formation control in Autonomous Underwater Vehicles (AUVs) without requiring knowledge of system dynamics or environmental data. Current methods, often assuming partial knowledge like known mass matrices, limit adaptability in varied settings.

Methods:

We proposed two-layer framework treats all system dynamics, including the mass matrix, as entirely unknown, achieving configuration-agnostic control applicable to multiple underwater scenarios. The first layer features a cooperative estimator for inter-agent communication independent of global data, while the second employs a decentralized deterministic learning (DDL) controller using local feedback for precise trajectory control. The framework's radial basis function neural networks (RBFNN) store dynamic information, eliminating the need for relearning after system restarts.

Results:

This robust approach addresses uncertainties from unknown parametric values and unmodeled interactions internally, as well as external disturbances such as varying water currents and pressures, enhancing adaptability across diverse environments.

Discussion:

Comprehensive and rigorous mathematical proofs are provided to confirm the stability of the proposed controller, while simulation results validate each agent’s control accuracy and signal boundedness, confirming the framework’s stability and resilience in complex scenarios.

1 Introduction

Robotics and autonomous systems have a wide range of applications, spanning from manufacturing and surgical procedures to exploration in challenging environments (Ghafoori et al., 2024; Jandaghi et al., 2023). However, controlling robots in such settings, especially in space and underwater, presents significant difficulties due to unpredictable dynamics. In the context of underwater exploration, AUVs have become essential tools, offering cost-effective, reliable, and versatile solutions for adapting to dynamic conditions. Effective use of AUVs is critical for unlocking the mysteries of marine environments, making advancements in their control and operation essential. As the demand for efficient underwater exploration increases and the complexity of tasks assigned to AUVs grows, there is a pressing need to enhance their operational capabilities. This includes developing sophisticated formation control strategies that allow multiple AUVs to operate in coordination, drawing inspiration from natural behaviors observed in fish schools and bird flocks (Zhou et al., 2023; Yang et al., 2021). By leveraging multi-agent systems, AUVs can work in coordinated groups, enhancing efficiency, stability, and coverage while navigating dynamic and complex underwater environments. These strategies are essential for ensuring precise operations in varied underwater tasks, ranging from pipeline inspections and seafloor mapping to environmental monitoring (Yan et al., 2023).

Despite challenges from intricate nonlinear dynamics, complex interactions among AUVs, and the uncertain dynamic nature of underwater environments, effective multi-AUV formation control is increasingly critical in modern ocean industries (Yan et al., 2018; Hou and Cheah, 2009). Historically, formation control research has predominantly utilized the behavioral approach (Balch and Arkin, 1998; Lawton, 2000), which divides the overall control design into subproblems, with each vehicle’s action determined by a weighted average of solutions, though selecting appropriate weighting parameters can be challenging. The leader-following approach (Cui et al., 2010; Rout and Subudhi, 2016) designates one vehicle as the leader while others follow, maintaining predefined geometric relationships, and controlling formation behavior by designing specific motions for the leader. Alternatively, the virtual structure approach (Millán et al., 2013).

Despite advancements in formation control and path planning for multi-AUV systems, challenges such as environmental disturbances, complex underwater dynamics, and communication limitations continue to pose difficulties (Hadi et al., 2021). To address these challenges, there is a critical need for controllers that are independent of both robot dynamics and environmental disturbances. Developing such controllers would enhance formation control by allowing for decentralized application, which increases flexibility in formation structures and improves robustness against communication constraints. Addressing these gaps is essential for advancing the capabilities and reliability of multi-AUV systems. On the other hand, communication constraints in underwater environments make decentralized control with a virtual leader-following topology ideal for AUVs, enabling coordination using local information despite communication delays or interruptions (Yan et al., 2023).

Reinforcement learning (RL) has also been extensively applied in robotic control (Christen et al., 2021; Cao et al., 2022). RL approaches, such as deep reinforcement learning (DRL), offer advantages in learning complex, non-linear control policies directly from data. However, RL methods generally lack the ability to provide mathematical stability proofs and guarantees for the controller’s behavior, making it challenging to ensure safety and reliability, especially in critical applications. Besides, while Zhang et al. (2018) developed various direct neural adaptive laws that lead to increased oscillations with higher adaptation gains, indirect neural adaptive laws using prediction error methods were proposed to mitigate this issue, though they could not guarantee parameter convergence. However, NN-based learning control methods, such as those utilizing adaptive neural networks or deterministic learning frameworks Jandaghi et al. (2024), can incorporate stability analysis and provide rigorous mathematical proofs for parameter convergence. These methods enable researchers to establish theoretical guarantees for the stability and robustness of the controller, which is essential for deploying controllers in real-world applications where safety and reliability are critical. Most recently, Tutsoy et al. (2024) proposed an optimization-based approach for path planning in Unmanned Air Vehicles (UAVs) with actuator failures using particle swarm optimization and genetic algorithms. Their method focuses on minimizing both time and distance by optimizing predefined cost functions through heuristic methods, while incorporating system constraints such as actuator limits, kinematic, and dynamic constraints, as well as parametric uncertainties.

Despite extensive literature in the field, to the best of our knowledge, existing researches assume homogeneous dynamics and certain system parameters for all AUV agents, which is unrealistic in unpredictable underwater environments. Factors such as buoyancy, drag, and varying water viscosity significantly alter system dynamics and behavior. Additionally, AUVs may change shape during tasks like underwater sampling or when equipped with robotic arms, further complicating control. Typically, designing multi-AUV formation control involves planning desired formation paths and developing tracking controllers for each AUV. However, accurately tracking these paths is challenging due to the complex nonlinear dynamics of AUVs, especially when precise models are unavailable. Implementing a fully distributed and decentralized formation control system is also difficult, as centralized control designs become exceedingly complex with larger AUV groups. To address these challenges previous work, such as Yuan et al. (2017) and Dong et al. (2019), developed adaptive learning controllers that relied on the assumption of a known mass matrix, which is not practical in real-world applications. These controllers relied on known system parameters that can fail due to varying internal forces caused by varying external environmental conditions. The solution is to develop environment-independent controllers that do not rely on any specific system dynamical parameters.

The rest of the paper is organized as follows: Section 2 provides an initial overview of graph theory, RBFNN, and the problem statement. The design of the distributed cooperative estimator and the decentralized deterministic learning controller are discussed in Section 3. The formation adaptive control and formation control using pre-learned dynamics are explored in Section 4 and Section 5, respectively. Simulation studies are presented in Section 6, and Section 7 concludes the paper.

2 Preliminaries and problem statement

2.1 Notation and graph theory

Denoting the set of real numbers as , we define as the set of real matrices, and as the set of real vectors. The identity matrix is symbolized as . The vector with all elements being 1 in an -dimensional space is represented as . The sets and stand for real symmetric and positive definite matrices, respectively. A block diagonal matrix with matrices on its main diagonal is denoted by . signifies the Kronecker product of matrices and . For a matrix , is the vectorization of by stacking its columns on top of each other. For a series of column vectors , represents a column vector formed by stacking them together. Given two integers and with , . For a vector , its norm is defined as . For a square matrix , denotes its -th eigenvalue, while and represent its minimum and maximum eigenvalues, respectively.

A directed graph comprises nodes in the set and edges in . An edge from node to node is represented as , with as the parent node and as the child node. Node is also termed a neighbor of node . is considered as the subset of consisting of the neighbors of node . A sequence of edges in , , is called a path from node to node . Node is reachable from node . A directed tree is a graph where each node, except for a root node, has exactly one parent. The root node is reachable from all other nodes. A directed graph contains a directed spanning tree if at least one node can reach all other nodes. The weighted adjacency matrix of is a non-negative matrix , where and . The Laplacian of is denoted as , where and if . It is established that has at least one eigenvalue at the origin, and all nonzero eigenvalues of have positive real parts. From Ren and Beard (2005), has one zero eigenvalue and remaining eigenvalues with positive real parts if and only if has a directed spanning tree.

2.2 Radial basis function neural networks (RBFNN)

The RBFNN Networks can be described as , where and as input and weight vectors respectively (Park and Sandberg, 1991). indicates the number of NN nodes, with is a radial basis function, and is distinct points in the state space. The Gaussian function is generally used for radial basis function, where is the center and is the width of the receptive field. The Gaussian function categorized by localized radial basis function in the sense that as . Moreover, for any bounded trajectory within the compact set , can be approximated using a limited number of neurons located in a local region along the trajectory . denotes the indices of active RBFNN nodes where , based on the state . is the approximation error, with , , , , and the integers are defined by ( is a small positive constant) for some . This holds if for . The following lemma regarding the persistent excitation (PE) condition for RBFNN is recalled from Wang and Hill (2018).

2.3 Problem statement

A multi-agent system comprising AUVs with heterogeneous nonlinear uncertain dynamics is considered. The dynamics of each AUV can be expressed as Fossen (1999):

In this study, the subscript identifies each AUV within the multi-agent system. For every , the vector represents the -th AUV’s position coordinates and heading in the global coordinate frame, while denotes its linear velocities and angular rate of heading relative to a body-fixed frame. The positive definite inertial matrix , Coriolis and centripetal matrix , and damping matrix characterize the AUV’s dynamic response to motion. The vector accounts for the restoring forces and moments due to gravity and buoyancy. The term , with , describes the vector of generalized deterministic unmodeled uncertain dynamics for each AUV.

The vector represents the control inputs for each AUV. The associated rotation matrix is given by:

Unlike previous work Yuan et al. (2017), which assumed known values for the AUV’s inertia and rotation matrices, this study considers all matrix coefficients, including , , , and , as well as the inertia matrix, as completely unknown. The adaptive estimation process inherently addresses the effects of external forces and disturbances on system dynamics. This eliminates the need for explicit parameter estimation of these forces, as disturbances like water flow, varying currents, or depth variations are directly incorporated into the control input through adaptive estimation. This makes the controller universally applicable to any AUV, regardless of its design, weight, or environmental conditions by addressing both internal and external dynamic variation at the same time.

Internally, it handles unknown parameters such as mass and damping coefficients, as well as unmodeled nonlinear interactions and couplings. Externally, it accounts for unpredictable disturbances, including fluctuating water currents, depth-dependent pressures, and changes in hydrodynamic forces.

By avoiding reliance on predefined models, the proposed approach is robust and adaptable to diverse mission scenarios and unexpected environmental changes, ensuring reliable performance even in highly uncertain conditions.

In the context of leader-following formation tracking control, the following virtual leader dynamics generates the tracking reference signals:with “0” marking the leader node, the leader state with and , is a constant matrix available only to the leader’s neighboring AUV agents.

Considering the system dynamics of multiple AUVs (Equation 1) along with the leader dynamics (Equation 2), we establish a non-negative matrix , where such that for each , if and only if agent has access to the reference signals and . All remaining elements of are arbitrary non-negative values, such that for all . Correspondingly, we establish as a directed graph derived from , where designates node 0 as the leader, and the remaining nodes correspond to the AUV agents. We proceed under the following assumptions:

Assumption 1 is crucial for ensuring that the leader dynamics produce stable, meaningful reference trajectories for formation control. It ensures that all states of the leader, represented by , remain within the bounds of a compact set for all . The trajectory of the system, starting from and denoted by , generates periodic signal. This periodicity is essential for maintaining the Persistent Excitation (PE) condition, which is pivotal for achieving parameter convergence in Distributed Adaptive (DA) control systems. Modifications to the eigenvalue constraints on mentioned in Assumption 1 may be considered when focusing primarily on formation tracking control performance, as discussed later.

Additionally, Assumption 2 reveals key insights into the structure of the Laplacian matrix of the network graph . Let be an non-negative diagonal matrix where each -th diagonal element is for . The Laplacian is formulated as:where if and otherwise. This results in since . As cited in Su and Huang (2011), all nonzero eigenvalues of , if present, exhibit positive real parts, confirming as nonsingular under Assumption 2.

As shown in Figure 1, a two-layer hierarchical design approach is proposed to address the aforementioned challenges. The first layer, the Cooperative Estimator, enables information exchange among neighboring agents. The second layer, known as the Decentralized Deterministic Learning (DDL) controller, processes only local data from each individual AUV. The development and formulation of the first layer are discussed in detail in Section 3.1, while the DDL control strategy, along with its corresponding controller design and analysis, is provided in Section 3.2.

FIGURE 1

3 Two-layer distributed controller architecture

3.1 First layer: cooperative estimator

In the context of leader-following formation control, not all AUV agents may have direct access to the leader’s information, including tracking reference signals and the system matrix . This necessitates collaborative interactions among the AUV agents to estimate the leader’s information effectively. Drawing on principles from multiagent consensus and graph theories Ren and Beard (2008), we propose to develop a distributed adaptive observer for the AUV systems as:

The observer states for each -th AUV, denoted by , aim to estimate the leader’s state, . As , these estimates are expected to converge, such that approaches and approaches , representing the leader’s position and velocity, respectively. Equation 3 accounts for the communication graph by including the adjacency matrix information through the term . Note that represents an estimate computed by agent i of the leader’s matrix dynamics which is also not available to the agents. Therefore, each agent estimates such matrix using a cooperative adaptation law:which borrowed from Ren and Beard (2008) as well. The constants and are all positive numbers and are subject to design.

To verify the convergence properties, we need to compute the error dynamics. Now we define the estimation error for the state and the system matrix for agent as and , and then we derive the error dynamics:

Define the collective error states and adaptation matrices: for the state errors, for the adaptive parameter errors, representing the block diagonal of adaptive parameters, and for the diagonal matrices of design constants. With these definitions, the network-wide error dynamics can be expressed as:

This convergence is facilitated by the independent adaptation of each agent’s parameters within their respective error dynamics, represented by the block diagonal structure of and control gains and . These matrices ensure that each agent’s parameter updates are governed by local interactions and error feedback, consistent with the decentralized control framework.

Proof: We begin by examining the estimation error dynamics for as presented in Equation 5. This can be rewritten in the vector form:

Under Assumption 2, all eigenvalues of possess positive real parts according to Su and Huang (2011). Consequently, for any positive , the matrix is guaranteed to be Hurwitz, Which implies the exponential stability of system (Equation 6). Hence, it follows that exponentially, leading to exponentially for all . Now, we analyze the error dynamics for in Equation 5. Based on the previous discussions, We have exponentially, and the term will similarly decay to zero exponentially. Based on Cai et al. (2015), if the system defined byis exponentially stable, then exponentially. With Assumption 1, knowing that all eigenvalues of have zero real parts, and since as nonsingular with all eigenvalues in the right-half plane, system (Equation 7) is exponentially stable for any positive . Consequently, this ensures that , i.e., exponentially for all .

Now, each individual agent can accurately estimate both the state and the system matrix of the leader through cooperative observer estimation Equations 3, 4. This information will be utilized in the DDL controller design for each agent’s second layer, which will be discussed in the following subsection.

3.2 Second layer: decentralized deterministic learning controller

To fulfill the overall formation learning control objectives, in this section, we develop the DDL control law for the multi-AUV system defined in Equation 1. We use to denote the desired distance between the position of the -th AUV agent and the virtual leader’s position . Then, the formation control problem is framed as a position tracking control task, where each local AUV agent’s position is required to track the reference signal . Besides, due to the inaccessibility of the leader’s state information for all AUV agents, the tracking reference signal is employed instead of the reference signal . As established in Theorem 1, is autonomously generated by each local agent and will exponentially converge to . This ensures that the DDL controller is feasible and the formation control objectives are achievable for all using .

To design the DDL control law that addresses the formation tracking control and the precise learning of the AUVs’ complete nonlinear uncertain dynamics at the same time, we will integrate renowned backstepping adaptive control design method outlined in Krstic et al. (1995) along with techniques from Wang and Hill (2018) and Yuan et al. (2017) for deterministic learning using RBFNN. Specifically, for the -th AUV agent described in system (Equation 1), we define the position tracking error as for all . Considering for all , we proceed to:

To frame the problem in a more tractable way, we assume as a virtual control input and as a desired virtual control input in our control strategy design, and by implementing them in the above system we have:

A positive definite gain matrix is used for tuning the performance. Substituting into Equation 8 yields:

Now we derive the first derivatives of the virtual control input and the desired control input as follows:

As previously discussed, unlike earlier research that only identified the matrix coefficients , , , and as unknown system nonlinearities while assuming the mass matrix to be known, this work advances significantly by also considering as unknown. Consequently, all system dynamic parameters are treated as completely unknown, making the controller fully independent of the robot’s configuration such as its dimensions, mass, or any appendages and the uncertain environmental conditions it encounters, like depth, water flow, and viscosity. This independence is critical as it ensures that the controller does not rely on predefined assumptions about the dynamics, aligning with the main goal of this research. To address these challenges, we define a unique nonlinear function that encapsulates all nonlinear uncertainties as follows:where and , with being a bounded compact set. We then employ the following RBFNN to approximate the model dynamics in (Equation 11) expressed by nonlinear functions with for all and as follows:where is the ideal constant NN weights, and is the approximation error for all and , which satisfies . This error can be made arbitrarily small given a sufficient number of neurons in the network. A self-adaptation law is designed to estimate the unknown online. We aim to estimate with by constructing the DDL feedback control law as follows: is a feedback gain matrix that can be tuned to achieve the desired performance. To approximate the unknown nonlinear function vector in (Equation 11) along the trajectory within the compact set , we use:

Then, from Equations 1, 13 we have:

By subtracting from both sides and considering Equations 9, 11, we define , leading to:

For updating online, a robust self-adaptation law is constructed using the -modification technique Ioannou and Sun (1996) as follows:where , , and are free parameters to be designed for all and . Integrating Equations 9, 13, 14 yields the following closed-loop system:where, for all and , , and .

Evaluating the derivative of along the trajectory of Equation 15 for all yields:

Choose such that . Using the completion of squares, we have:where . Then, we obtain:

It follows that is negative definite whenever:

The derivative of is:

Similar to the proof of part one, we let with . According to Wang and Hill (2018), the Gaussian RBFNN regressor is bounded by for any and for all with some positive number . Through completion of squares, we have:

Also . This leads to:where and , . Solving the inequality Equation 17 yields:which together with Equation 16 implies that:also

Consequently, it is straightforward that given , there exists a finite time for all such that for all , both and satisfy and , where can be made arbitrarily small by choosing sufficiently large and for all . This ends the proof.

By integrating the outcomes of Theorems 1, 2, the following theorem is established, which can be presented without additional proof:

4 Accurate learning from formation control

It is necessary to demonstrate the convergence of the RBFNN weights in Equations 13, 14 to their optimal values for accurate learning and identification. The main result of this section is summarized in the following theorem.

Proof: From Theorem 3, we have shown that for all , will closely track the periodic signal in finite time . In addition, (Equation 9) implies that will also closely track the signal since both and will converge to a small neighborhood around zero according to Theorem 2. Moreover, since will converge to according to Theorem 1, and is a bounded rotation matrix, will also be a periodic signal after finite time , because is periodic under Assumption 1. Consequently, since the RBFNN input becomes a periodic signal for all , the PE condition of some internal closed-loop signals, i.e., the RBFNN regression subvector , is satisfied according to Lemma 1. As mentioned in Section 2.2, represents the subset of RBFNN nodes and weights that are specifically utilized along the recurrent trajectory of the system state . This subset focuses on the active neural components required for approximating the system’s nonlinear dynamics locally, ensuring that the learning and adaptation processes are efficient and accurate within the compact region where the trajectory resides. It should be noted that the periodicity of leads to the PE of the regression subvector , but not necessarily the PE of the whole regression vector . Thus, we term this as a partial PE condition, and we will show the convergence of the associated local estimated neural weights , rather than .

Thus, to prove accurate convergence of local neural weights associated with the regression subvector under the satisfaction of the partial PE condition, we first rewrite the closed-loop dynamics of and along the periodic tracking orbit by using the localization property of the Gaussian RBFNN:where with and being the approximation error. Additionally, with subscripts and denoting the regions close to and far away from the periodic trajectory , respectively. According to Wang and Hill (2018), is small, and the NN local approximation error with is also a small number. Thus, the overall closed-loop adaptive learning system can be described by:andwherefor all . The exponential stability property of the nominal part of subsystem (Equation 19) has been well-studied in Wang and Hill (2018), Yuan and Wang (2011), and Yuan and Wang (2012), where it is stated that PE of will guarantee exponential convergence of for all and . Based on this, since , and can be made small by choosing sufficiently small for all , , both the state error signals and the local parameter error signals will converge exponentially to small neighborhoods of zero, with the sizes of the neighborhoods determined by the RBFNN ideal approximation error as in Equation 12 and . The convergence of implies that along the periodic trajectory , we havewhere for all , , due to the convergence of . The last equality is obtained according to the definition of (Equation 18) with being the corresponding subvector of along the periodic trajectory , and being an approximation error using . Apparently, after the transient process, we will have , , . Conversely, for the neurons whose centers are distant from the trajectory , the values of will be very small due to the localization property of Gaussian RBFNN. From the adaptation law (Equation 13) with , it can be observed that these small values of will only minimally activate the adaptation of the associated neural weights . As a result, both and , as well as and , will remain very small for all , along the periodic trajectory . This indicates that the entire RBFNN and can be used to accurately approximate the unknown function locally along the periodic trajectory , meaning that.with the approximation accuracy level of and for all , . This ends the proof.

5 Formation control with pre-learned dynamics

In this section, we will further address objective 2 of Problem 1, which involves achieving formation control without readapting to the AUV’s nonlinear uncertain dynamics. To this end, consider the multiple AUV systems (Equation 1) and the virtual leader dynamics (Equation 2). We employ the estimator observer Equations 3, 4 to cooperatively estimate the leader’s state information. Instead of using the DDL feedback control law (Equation 13), and self-adaptation law (Equation 4), we introduce the following constant RBFNN controller, which does not require online adaptation of the NN weights:where is obtained from Equation 18. The term represents the locally accurate RBFNN approximation of the nonlinear uncertain function along the trajectory , and the associated constant neural weights are obtained from the formation learning control process as discussed in Remark 8.

Proof: The closed-loop system for each local AUV agent can be established by integrating the controller (Equation 20) with the AUV dynamics (Equation 1).where . Consider the Lyapunov function candidate , whose derivative along the closed-loop system described is given by:

Selecting where , we can utilize the method of completing squares to obtain:which implies that:where and . Using similar reasoning to that in the proof of Theorem 2, it is evident from the derived inequality that all signals within the closed-loop system remain bounded. Additionally, will converge to a small neighborhood around zero within a finite period. The magnitude of this neighborhood can be minimized by appropriately choosing large values for and across all . In line with Theorem 1, under Assumptions 1, 2, the implementation of the DA observer DA observer Equations 3, 4 facilitates the exponential convergence of towards . This conjunction of factors assures that rapidly aligns with , achieving the objectives set out for formation control.

6 Simulation

We consider a multi-AUV heterogeneous system composed of 5 AUVs for the simulation. The dynamics of these AUVs are described in THE system (Equation 1). The system parameters for each AUV are specified as follows:where the mass and damping matrix components for each AUV are defined as:

According to the notations in Prestero (2001) and Skjetne et al. (2005) the coefficients are hydrodynamic parameters. For the associated system parameters are borrowed from Skjetne et al. (2005) (with slight modifications for different AUV agents) and simulation purposes and listed in Table 1. For all , we set and . Model uncertainties are given by:

Figure 2 illustrates the communication topology and the spanning tree where agent 0 is the virtual leader and is considered as the root, in accordance with Assumption 2. The desired formation pattern requires each AUV, , to track a periodic signal generated by the virtual leader . The dynamics of the leader are defined as follows:

FIGURE 2

The initial conditions and system matrix are structured to ensure all eigenvalues of lie on the imaginary axis, thus satisfying Assumption 1. The reference trajectory for is defined as . The predefined offsets , which determine the relative positions of the AUVs to the leader, are specified as follows:

Each AUV tracks its respective position in the formation by adjusting its location to .

6.1 DDL formation learning control simulation

The estimated virtual leader’s state, derived from the cooperative estimator in the first layer (see Equations 3, 4), is utilized to estimate each agent’s complete uncertain dynamics within the DDL controller (second layer) using Equations 13, 14. The uncertain nonlinear functions for each agent are approximated using RBFNN, as described in Equation 11. Specifically, for each agent , the nonlinear uncertain functions , dependent on , are modeled. The input to the NN, , allows the construction of Gaussian RBFNN, represented by , utilizing 4,096 neurons arranged in an grid. The centers of these neurons are evenly distributed over the state space , and each has a width , ensuring bounded and structured parameter optimization for all and .

The observer and controller parameters are chosen as , and the diagonal matrices and , with and for all and . The initial conditions for the agents are set as , , , , and . Zero initial conditions are assumed for all the distributed observer states and the DDL controller states for all and . Time-domain simulation is carried out using the DDL formation learning control laws as specified in Equations 13, 14, along with Equations 3, 4.

Figure 3 displays the simulation results of the cooperative estimator (first layer) for all five agents. It illustrates how each agent’s estimated states,, converge perfectly to the leader’s states thorough Equations 3, 4. Figure 4 presents the position tracking control responses of all agents. Figures 4A–C illustrate the tracking performance of AUVs along the x-axis, y-axis, and vehicle heading, respectively, demonstrating effective tracking of the leader’s position signal. While the first AUV exactly tracks the leader’s states, agents 2 through 5 are shown to successfully follow agent 1, maintaining prescribed distances and alignment along the x and y axes, and matching the same heading angle. These results underscore the robustness of the real-time tracking control system, which enforces a predefined formation pattern, initially depicted in Figure 2. Additionally, Figure 5 highlights the real-time control performance for all agents, showcasing the effectiveness of the tracking strategy in maintaining the formation pattern.

FIGURE 3

FIGURE 4

FIGURE 5

The sum of the absolute values of the neural network weights in Figure 6. This convergence reflects the network’s ability to maintain consistent performance, as further adjustments to the weights become minimal. Also, updating neural network weights and their convergence throughout the learning process into their optimal valies depicted in Figure 7. This convergence of all neural network weights to their optimal values during the training process, aligns with Theorem 4 as well. This leads to achieving accurate function approximation in the second layer. Figure 8 represents the successful function approximation results for the unknown system dynamics as defined in Equation 11 for the third AUV, using RBFNN. The approximations are plotted for both and for all which defined in Theorem 4. The results confirm that locally accurate approximations of the AUV’s nonlinear dynamics were achieved. Moreover, this learned knowledge about the dynamics is effectively stored and represented using localized constant RBFNN.

FIGURE 6

FIGURE 7

FIGURE 8

6.2 Simulation for formation control with pre-learned dynamics

To evaluate the distributed control performance of the multi-AUV system, we implemented the pre-learned distributed formation control law. This strategy integrates the estimator observer Equations 3, 4, this time coupled with the constant RBFNN controller (Equation 21). We employed the virtual leader dynamics described in Equation 21 to generate consistent position tracking reference signals, as previously discussed in Section 6.1. To ensure a fair comparison, identical initial conditions and control gains and inputs were used across all simulations. Figure 9 illustrates the comparison of the tracking control results from Equations 13, 14 with the results using pre-trained weights in Equation 20.

FIGURE 9

The control experiments and simulation results presented demonstrate that the constant RBFNN control law (Equation 4) can achieve satisfactory tracking control performance comparable to that of the adaptive control laws (Equations 13, 14), but with no computational demand. The elimination of online recalculations or readaptations of the NN weights under this control strategy significantly reduces the computational load whenever system restarts without needing to retrain again. This reduction is particularly advantageous in scenarios involving extensive neural networks with a large number of neurons, thereby conserving system energy and enhancing operational efficiency in real-time applications.

7 Conclusion

In conclusion, this paper has introduced a novel two-layer control framework designed for Autonomous Underwater Vehicles (AUVs), aimed at universal applicability across various AUV configurations and environmental conditions. This framework assumes all system dynamics to be unknown, thereby enabling the controller to operate independently of specific dynamic parameters and effectively handle any environmental challenges, including hydrodynamic forces and torques. The framework consists of a first-layer distributed observer estimator that captures the leader’s dynamics using information from adjacent agents, and a second-layer decentralized deterministic learning controller. Each AUV utilizes the estimated signals from the first layer to determine the desired trajectory, simultaneously training its own dynamics using Radial Basis Function Neural Networks (RBFNN). This innovative approach not only sustains stability and performance in dynamic and unpredictable environments but also allows AUVs to efficiently utilize previously learned dynamics after system restarts, facilitating rapid resumption of optimal operations. The robustness and versatility of this framework have been rigorously confirmed through comprehensive simulations, demonstrating its potential to significantly enhance the adaptability and resilience of AUV systems. By embracing total uncertainty in system dynamics, this framework establishes a new benchmark in autonomous underwater vehicle control and lays a solid groundwork for future developments aimed at minimizing energy use and maximizing system flexibility. We plan to expand this framework by accommodating more general leader dynamics and conducting experimental applications to validate its performance in real-world settings. Moreover, a more accurate model of some source of uncertainty could improve performance which we will address in our future research. The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

• 1

  BalchT.ArkinR. (1998). Behavior-based formation control for multirobot teams. IEEE Trans. Robotics Automation14, 926–939. 10.1109/70.736776

  • CrossRef
  • Google Scholar

• 2

  CaiH.LewisF. L.HuG.HuangJ. (2015). “Cooperative output regulation of linear multi-agent systems by the adaptive distributed observer,” in 2015 54th IEEE Conference on Decision and Control (CDC) (IEEE), 5432–5437.

  • Google Scholar

• 3

  CaoX.RenL.SunC. (2022). Dynamic target tracking control of autonomous underwater vehicle based on trajectory prediction. IEEE Trans. Cybern.53, 1968–1981. 10.1109/tcyb.2022.3189688

  • CrossRef
  • Google Scholar

• 4

  ChristenS.JendeleL.AksanE.HilligesO. (2021). Learning functionally decomposed hierarchies for continuous control tasks with path planning. IEEE Robotics Automation Lett.6, 3623–3630. 10.1109/lra.2021.3060403

  • CrossRef
  • Google Scholar

• 5

  CuiR.GeS. S.HowB. V. E.ChooY. S. (2010). Leader–follower formation control of underactuated autonomous underwater vehicles. Ocean. Eng.37, 1491–1502. 10.1016/j.oceaneng.2010.07.006

  • CrossRef
  • Google Scholar

• 6

  DongX.YuanC.StegagnoP.ZengW.WangC. (2019). Composite cooperative synchronization and decentralized learning of multi-robot manipulators with heterogeneous nonlinear uncertain dynamics. J. Frankl. Inst.356, 5049–5072. 10.1016/j.jfranklin.2019.04.028

  • CrossRef
  • Google Scholar

• 7

  FossenT. I. (1999). “Guidance and control of ocean vehicles,”. Norway: University of Trondheim. Doctors Thesis.

  • Google Scholar

• 8

  GhafooriS.RabieeA.CeteraA.AbiriR. (2024). Bispectrum analysis of noninvasive eeg signals discriminates complex and natural grasp types. arXiv Prepr. arXiv:2402.01026, 1–5. 10.1109/embc53108.2024.10782163

  • CrossRef
  • Google Scholar

• 9

  HadiB.KhosraviA.SarhadiP. (2021). A review of the path planning and formation control for multiple autonomous underwater vehicles. J. Intelligent and Robotic Syst.101, 67–26. 10.1007/s10846-021-01330-4

  • CrossRef
  • Google Scholar

• 10

  HouS. P.CheahC. C. (2009). “Coordinated control of multiple autonomous underwater vehicles for pipeline inspection,” in Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference (IEEE), 3167–3172.

  • Google Scholar

• 11

  IoannouP. A.SunJ. (1996). Robust adaptive control, 1. Upper Saddle River, NJ: PTR Prentice-Hall.

  • Google Scholar

• 12

  JandaghiE.ChenX.YuanC. (2023). “Motion dynamics modeling and fault detection of a soft trunk robot,” in 2023 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM) (IEEE), 1324–1329.

  • Google Scholar

• 13

  JandaghiE.SteinD. L.HoburgA.StegagnoP.ZhouM.YuanC. (2024). “Composite distributed learning and synchronization of nonlinear multi-agent systems with complete uncertain dynamics,” in 2024 IEEE International Conference on Advanced Intelligent Mechatronics (AIM), 1367–1372. 10.1109/aim55361.2024.10637197

  • CrossRef
  • Google Scholar

• 14

  KrsticM.KokotovicP. V.KanellakopoulosI. (1995). Nonlinear and adaptive control design. John Wiley and Sons, Inc.

  • Google Scholar

• 15

  LawtonJ. R. T. (2000). A Behavior-Based Approach to Multiple Spacecraft Formation Flying (Ph.D. thesis). Brigham Young University, Provo, UT, United States.

  • Google Scholar

• 16

  MillánP.OrihuelaL.JuradoI.RubioF. R. (2013). Formation control of autonomous underwater vehicles subject to communication delays. IEEE Trans. Control Syst. Technol.22, 770–777. 10.1109/tcst.2013.2262768

  • CrossRef
  • Google Scholar

• 17

  ParkJ.SandbergI. W. (1991). Universal approximation using radial-basis-function networks. Neural Comput.3, 246–257. 10.1162/neco.1991.3.2.246

  • CrossRef
  • Google Scholar

• 18

  PengZ.WangD.ShiY.WangH.WangW. (2015). Containment control of networked autonomous underwater vehicles with model uncertainty and ocean disturbances guided by multiple leaders. Inf. Sci.316, 163–179. 10.1016/j.ins.2015.04.025

  • CrossRef
  • Google Scholar

• 19

  PengZ.WangJ.WangD. (2017). Distributed maneuvering of autonomous surface vehicles based on neurodynamic optimization and fuzzy approximation. IEEE Trans. Control Syst. Technol.26, 1083–1090. 10.1109/tcst.2017.2699167

  • CrossRef
  • Google Scholar

• 20

  PresteroT. (2001). “Development of a six-degree of freedom simulation model for the remus autonomous underwater vehicle,” in MTS/IEEE Oceans 2001. An Ocean Odyssey. Conference Proceedings (IEEE Cat. No.01CH37295), Honolulu, HI, USA, 2001, pp.450–455 vol.1, 10.1109/OCEANS.2001.968766

  • CrossRef
  • Google Scholar

• 21

  RenW.BeardR. W. (2005). Consensus seeking in multiagent systems under dynamically changing interaction topologies. IEEE Trans. automatic control50, 655–661. 10.1109/tac.2005.846556

  • CrossRef
  • Google Scholar

• 22

  RenW.BeardR. W. (2008). Distributed consensus in multi-vehicle cooperative control, 27. Springer.

  • Google Scholar

• 23

  RoutR.SubudhiB. (2016). A backstepping approach for the formation control of multiple autonomous underwater vehicles using a leader–follower strategy. J. Mar. Eng. and Technol.15, 38–46. 10.1080/20464177.2016.1173268

  • CrossRef
  • Google Scholar

• 24

  SkjetneR.FossenT. I.KokotovićP. V. (2005). Adaptive maneuvering, with experiments, for a model ship in a marine control laboratory. Automatica41, 289–298. 10.1016/j.automatica.2004.10.006

  • CrossRef
  • Google Scholar

• 25

  SuY.HuangJ. (2011). Cooperative output regulation of linear multi-agent systems. IEEE Trans. Automatic Control57, 1062–1066. 10.1109/TAC.2011.2169618

  • CrossRef
  • Google Scholar

• 26

  TutsoyO.AsadiD.AhmadiK.Nabavi-ChashmiS. Y.IqbalJ. (2024). Minimum distance and minimum time optimal path planning with bioinspired machine learning algorithms for faulty unmanned air vehicles. IEEE Trans. Intelligent Transp. Syst.25, 9069–9077. 10.1109/tits.2024.3367769

  • CrossRef
  • Google Scholar

• 27

  WangC.HillD. J. (2018). Deterministic learning theory for identification, recognition, and control. CRC Press. 10.1201/9781315221755

  • CrossRef
  • Google Scholar

• 28

  YanT.XuZ.YangS. X.GadsdenS. A. (2023). Formation control of multiple autonomous underwater vehicles: a review. Intell. and Robotics3, 1–22. 10.20517/ir.2023.01

  • CrossRef
  • Google Scholar

• 29

  YanZ.LiuX.ZhouJ.WuD. (2018). Coordinated target tracking strategy for multiple unmanned underwater vehicles with time delays. IEEE Access6, 10348–10357. 10.1109/access.2018.2793338

  • CrossRef
  • Google Scholar

• 30

  YangY.XiaoY.LiT. (2021). A survey of autonomous underwater vehicle formation: performance, formation control, and communication capability. IEEE Commun. Surv. and Tutorials23, 815–841. 10.1109/comst.2021.3059998

  • CrossRef
  • Google Scholar

• 31

  YuanC. (2017). Leader-following consensus of parameter-dependent networks via distributed gain-scheduling control. Int. J. Syst. Sci.48, 2013–2022. 10.1080/00207721.2017.1309597

  • CrossRef
  • Google Scholar

• 32

  YuanC.LichtS.HeH. (2017). Formation learning control of multiple autonomous underwater vehicles with heterogeneous nonlinear uncertain dynamics. IEEE Trans. Cybern.48, 2920–2934. 10.1109/tcyb.2017.2752458

  • CrossRef
  • Google Scholar

• 33

  YuanC.WangC. (2011). Persistency of excitation and performance of deterministic learning. Syst. and control Lett.60, 952–959. 10.1016/j.sysconle.2011.08.002

  • CrossRef
  • Google Scholar

• 34

  YuanC.WangC. (2012). Performance of deterministic learning in noisy environments. Neurocomputing78, 72–82. 10.1016/j.neucom.2011.05.037

  • CrossRef
  • Google Scholar

• 35

  ZhangY.LiS.LiuX. (2018). Neural network-based model-free adaptive near-optimal tracking control for a class of nonlinear systems. IEEE Trans. neural Netw. Learn. Syst.29, 6227–6241. 10.1109/tnnls.2018.2828114

  • CrossRef
  • Google Scholar

• 36

  ZhouJ.SiY.ChenY. (2023). A review of subsea auv technology. J. Mar. Sci. Eng.11, 1119. 10.3390/jmse11061119

  • CrossRef
  • Google Scholar