Consensus control for distributed power tracking by device-level digital twin agents

Abstract: The development of electronics and software has resulted in the ability of twin agents to act as digital counterparts for the optimization, control, and monitoring of real power grids. When increasing regulation resources are connected to the power grid, is challenging for independent system operators to use a centralized controller to achieve power tracking and balance. Therefore, the present study applied device-level-based digital twins to monitor physical signals for computer-aided design for power tracking. Moreover, a consensus control-based distributed power tracking system is proposed for the physical-model simulation of the power grid. A communication network was also designed for realistic signal exchange. The combination of the proposed distributed power tracking method and communication network can accelerate computational efficiency and protect the privacy of the regulation resources. Finally, the performance of the proposed distributed power tracking method is validated in a simulation system with 10 regulation resources.


Introduction
With the new energy development, increasing regulation resources participate in power tracking (Wu et al., 2021a;Zang et al., 2021;Wu et al., 2022). Traditional power tracking models only concern a centralized controller (Ray et al., 1999) with optimal algorithms for independent system operators (Zhang et al., 2020). However, with the increase in regulation resources, the computational costs for the optimization of dynamic power tracking also increase. Thus, high-quality dispatch schemes are hard to obtain using traditional centralized controller-based power tracking methods. Among, the recent advancements in electronics and software, digital twin agents can be viewed as digital counterparts for the optimization, control, and monitoring (Zhao and Lin, 2022) of real power grids (Doherty et al., 2010). Meanwhile, distributed-based power tracking frameworks (Kakran and Chanana, 2018) have been proposed to reduce computational costs and increase the optimal speed.
The key component of the digital concept is that it represents the connection between the physical and digital worlds, which enables an improved physical quantity model by combining the virtual and actual worlds. NASA and Michael Greaves defined a digital twin as a digital counterpart of a virtual physical thing or system (Schleich et al., 2017). Consequently, the digital twin technique is a powerful tool to integrate the real and the virtual to more effectively manage intelligence. Previous studies have proposed the OPEN ACCESS EDITED BY integration of the power grid with the digital twin technique. A novel structure based on a deep neural network technique was introduced by Zhou et al. (2019) to further provide a short time delay in the power grid for real-time online optimization. Moreover, Pan et al. (2020) developed a life cycle pattern with control, fault detection, and simulation applications. This work formulated a digital twin-based framework of communication between the virtual smart grid and the real power system. Additionally, Wang et al. (2021) described a datadriven-based methodology for feature extraction of the operational state for the power grid, which implemented a recursive state and provided decreased performance evaluation times.
Power tracking in power grids is a non-linear (Lakshmanan et al., 2016), multiple-constraint , and time-series problem (Sadeghi-Mobarakeh and Mohsenian-Rad, 2017). For these applications, centralized-based power tracking approaches, such as mathematical programming (Xie et al., 2000) or heuristic algorithms (Deb et al., 2002) are often used to obtain high-quality dispatch schemes. With the decentralization of the power grid and the expansion of regulation resources, it is hard for conventional centralized approaches to quickly determine a global optimum within the scheduled time (Wu et al., 2021b;Cheng et al., 2023). Compared to conventional centralized power tracking approaches such as mathematical programming or heuristic algorithms, distributed-based algorithms (Erdeljan et al., 2014) can more quickly provide optimal solutions for power grids. The two basic types of distributed methodologies for optimization are leaderless and leader-based, which are formulated for power tracking in power grids. Cui et al. (2018) presented an auction algorithm-based economic dispatch with leaderless regulation resources and evaluated the performance of various auction algorithms in various communication scenarios and clarified the impact of swap operations and communication networks of the regulation resources. Zhang et al. (2018) developed a framework for cyber, resource, and social systems and presented a human communication-based distributed consensus algorithm for regulation resources for a combined heat and power system. Furthermore, for the high connection of new energy resources, Zhang et al. (2021), proposed an adaptive parameter for the swap operation. Their study executed two simulation systems with various seniors for communication networks and verified the high performance of the proposed adaptive auction method. They also implemented a novel random forest base-auction method for the power tracking of a photovoltaic station (PV). Lastly, Zhang et al. (2022) developed a hydrogen regulation resource to store PV power output and formulate a negotiate-based distributed method to exploit the reconfiguration and provide the maximum profit for a PV-coordinated hydrogen system.
Among leader-based algorithms, the commonly used methodology is the cooperative consensus technique. Zhang and Chow (2012) used a consensus algorithm to quickly obtain an economic dispatch scheme by selecting a leader of a regulation resource and reaching the incremental cost. Bidram et al. (2013a) described a technique that facilitated the linearization of distributed generation for voltage control. The authors also developed an approach for cooperative consensus of regulation voltage (Bidram et al., 2013b). This study demonstrated that the proposed communication network provided a more reliable solution than the traditional centralized controller. Zhang et al. (2016a) formulated a distributed robust consensus method for transmission delay and noise. The simulation results showed that the algorithm could search for high-quality optimums at outage disturbances for economic dispatch. Subsequently, Zhang et al. (2016b) proposed a framework of virtual regulation resource tribe for frequency control with transfer learning-based consensus for regulation cost and ramp time.
Based on the fast computation of the distributed consensus method, communication network-based consensus algorithms will more be appropriate for communication between regulation resources. The present work employs a consensus control algorithm to rapidly obtain a high-performance scheme for power tracking with digital twin agents. The network communication between the agents is presented and the gain function is formulated for fast optimization. The main contributions of this work are as follows.
1) In contrast to previous work, this study proposes a digital twin agents-based power tracking model. It includes the life cycle of power tracking, from disturbance collection to controller strategy optimization, regulation resource response, and dynamic control performance visualization. Previous studies have focused on improving the computation time and optimal strategy but seldom apply the combination of digital twin agents and distributed power tracking with compensation payment (Papalexopoulos and Andrianesis, 2014). In particular, the influence of load disturbance on the dynamic control performance standard (CPS) can be rapidly evaluated for power grids. 2) A distributed consensus control-based algorithm (DCCA) for device-level digital twin agents is formulated to quickly obtain an optimal dispatch scheme for real-time power tracking. A distributed communication network to accelerate the communication speed and protect the privacy of the regulation resources is also presented. The proposed DDCCA can cooperate with the regulation resources, effectively utilize the fast regulation performance of renewable energy resources, and improve the control performance standard for digital twin-based distributed power tracking (DT-DPT).
The remaining sections of this work are organized as follows: Section 2 describes the mathematical model of DT-DPT that coordinates the device-level digital twin agents. Section 3 illustrates the specific implementation of DCCA for DT-DPT. Section 4 executes the simulation experiments and discusses the results. Finally, Section 5 concludes this paper.

DT-DPT framework
The DT-DPT framework is shown in Figure 1. DT-DPT includes physical information, algorithm optimization, and framework evaluation. In the physical control process, a load disturbance is exerted on the power grid. The frequency and tieline power are then measured for real-time power tracking. Next, the controller will receive the signal and distribute the power command

Constraints
To better imitate the connection between the physical reality and simulation model, some constraints should be considered in DT-DPT, including change dynamic due to the rapid regulation of power grid features. During power tracking, the constraints primarily include the regulation direction, power balance, generation capacity, and generation ramp (Yu et al., 2011), as follows.
1) Regulation direction constraint: To fully utilize the regulation resources, the regulation direction of the resources' power command should be equal to the direction of total power command, as follows: where ΔP in i (k) is the regulation command assigned to the ith DT-DPT regulation resource at kth control interval and ΔP C (k) denotes the total regulation command from the power grid to DT-DPT.
2) Power balance constraint: To confirm that the optimal solution satisfies the power grid regulation requirements, the accumulation of power regulation input commands received by all DT-DPT regulation resources should be accurately equivalent to the total power regulation command issued by the power grid, as follows: 3) Generation capacity constraint: To guarantee that the regulation resource has a good regulation performance and a secure environment, the power regulation commands obtained by the DT-DPT regulation resources should not exceed their capacities, as follows: where P i min (k) and P i max (k) represent the minimum and maximum of the ith regulation resource at the kth time control interval, respectively. 4) Generation ramp constraint: For the new energy regulation resources, the regulation of these resources can change rapidly due to control by the electrical switch. However, for traditional regulation resources, they should consider the generation ramp constraint, as follows: Frontiers in Energy Research frontiersin.org where ΔP out i (k) denotes the output power command received by the ith DT-DPT regulation resource from the controller at the kth control interval, ΔT represents the regulation time at a control interval, and ΔP R i is the maximum ramp rate of the ith regulation resource.

Objective functions
As the main focus of this work is the minimum frequency mileage-based (Chen et al., 2015) compensation of the DT-DPT regulation resources, one optimization goal is to reduce the total compensation payment of the regulation resources for the individual system operator. The regulation compensation payment can be computed by the comprehensive price coefficient and regulation power deviation (Wang et al., 2017). The comprehensive payment coefficient depends on the ramp performance and time delay of the regulation resources (Ariyo et al., 2014), as follows: where R i represents the compensation payment of the ith regulation resource, C i denotes the comprehensive payment coefficient, λ is the price coefficient of the mileage deviation, M i (k) denotes the power tracking mileage at the kth time control interval, S r i and S d i represent the rate performance score and the time delay of the ith regulation resource, w 1 and w 2 represent the corresponding weight value for the ith regulation resource (w 1 + w 2 1), ΔP a i is the average ramp of the ith regulation resource, and T r i denotes the regulation time delay of the ith regulation resource.
3 Design of the distributed DCCA for DT-DPT

Graph theory for the communication network
Since DT-DPT balances the power disturbance, the DCCA can be used to quickly obtain a power scheme. The regulation resources can be seen as nodes or agents; the corresponding communication network framework ids are shown in Figure 2, in which the graph G (V, E, W) denotes the communication network, the node set V V 1 , V 2 , . . . , V n { } is the set of the regulation resources or digital agents, the edges set E ⊆ V × V represents the relationship between the two nodes, and the weighted adjacency matrix W [w ij ] ∈ R n×n denotes the connection weight of the corresponding edge. Then, based on the proposed network setting, a Laplacian matrix L [l ij ] ∈ R n×n can be acquired, as follows: The following row stochastic matrix D [d ij ] ∈ R n×n can be determined by the Laplacian matrix, as follows:

DCCA design
Generally, based on the proposed communication network and matrix, DCCA can be formulated by several operations, as follows.
1) Payment compensation consensus: Based on the computation of the compensation payment in Eqs. 5-8, the payment compensation can be updated by the row stochastic matrix, as follows: where d ij represents the aggregation of the other jth node to the ith node and R j [t] denotes the corresponding compensation payment of the jth regulation resource at the tth optimal iteration.
2) Updating of the optimal leader: Firstly, the optimal leader can be updated by incrementing or decrementing the tracking power. Thus, the regulation resource with the highest R i performance is chosen as the optimal leader. Based on the power balance, the optimal leader should track the corresponding power responsible for the last iteration and the current power increment, as follows: where R q i [t + 1] represents the corresponding compensation payment of the optimal leader at the tth optimal iteration, μ represents the coefficient of the power tracking error, P e [t] denotes the power tracking error at the tth optimal iteration, and ΔP C (k) is the total power tracking value at the kth time control interval.
3) Modification of the power scheme: To ensure that the optimal solution does not violate the power capacity, the regulation resources that disobey the power constraints should be modified to the feasible zone of the solution. The feasible zone of the compensation payment should also be considered, as follows: 4) Consensus optimal process: For the presented computation of leader payment compensation and consensus computation, the power tracking error will decrease to nearly zero after a certain optimal iteration. The optimal process will terminate when the power error is a small value P e [t] ≤  or when the current iteration exceeds the maximum iteration N i ≤ t. The corresponding optimal power scheme can then be obtained based on the proposed compensation payment consensus.

Calculation flow
The optimal process of DCCA for DT-DPT is presented in Table 1.

DT-DPT system parameters
In the simulation test, a DT-DPT system with ten regulation resources was designed to verify the performance of the proposed algorithm. Table 2 lists the parameters of the DT-DPT system with five types of regulation resources. Table 3 lists the parameters of the regulation resources. The control interval period ΔT was set to 4 s, the total time control interval in one service period N s was set to 900 (1 h), and the mileage coefficient for compensation payment was 2 $/MW.
To set the DCCA parameters, the termination parameter for optimal process  was set to 0.001 MW. Three types of connected network topology are shown in Figure 3. The fourth regulation resource was set as the optimal leader, the coefficient of the power tracking error was set to 0.1, and the maximum optimal iteration N i was set to 500. Figure 3 shows the three types of topology networks to illustrate the performance of the proposed DCCA. The following section focuses on the proposed local connected network.

Statistical tests 4.2.1 Influence of power command
This section constructed two power commands (ΔP C −80 MW; ΔP C 120 MW) to illustrate the convergence of the DCAA and the consensus process in the DT-DPT system subject to load disturbance. First, the acquired regulation resources power commands were stable after 30 iterations for ΔP C −80 MW (Figure 4). Then, the consensus compensation payment was obtained during the optimal process. Among the ten regulation resources, the fourth regulation resource (leader agents) rapidly increased its power, then reached the maximum power command before decreasing to a stable value. The leader agent showed a similar regulation compensation payment tendency as that for the regulation power command during the optimal process. Besides,  Figure 5A shows that the proposed method can help acquire a stable power command in 40 steps. The corresponding compensation payment consensus can also be reached quickly ( Figure 5B).

Influence of the connected network topology
The proposed three topology networks ( Figure 3) were used to analyze the influence of various connected agents for DCCA convergence. The networks included the ring, local connected, and fully connected networks. These three networks assist in the optimization of the DCCA in the two power command scenarios (ΔP C −80 MW; ΔP C 120 MW). Figure 6 shows that the proposed local connected network can rapidly balance the power error; thus, the local network-assisted DCCA has the optimal performance compared to the other two connected networks. The optimization speed is approximately one time faster than the fully connected network and about two times faster than the ring network ( Figure 6). As shown in Figure 6A, the proposed network can acquire a high-quality solution within 18 steps, while the fully connected and ring networks require 35 and 55 iterations, respectively. Moreover, the proposed DCCA with the local connected network can reach high convergence in 15 iterations, compared to 30 and 45 steps for the fully connected-based and ring networks, respectively. Therefore, the proposed local connected methods reduced the communication pressure of digital agents and the convergence performance of the DCCA.

Real-time simulations 4.3.1 Load step disturbance
This section compares the performance of DCCA to a proportional method (PROP) (Papalexopoulos and

Regulation resource type Parameters (s)
Coal-fired        Figure 7 and Figure 8. The PROP is an engineering application approach that only distributes the power command to the regulation resources according to their regulation power capacities. Although this method can rapidly provide a power scheme, it lacks an optimal technique to improve the economic benefit or the regulation performance for the independent system operator. Generally, the power command for regulation resources obtained by PROP can be formulated as follows: First, the tracking processes of the two approaches for a DT-DPT system at ΔP D −100 MW are shown in Figure 7A, in whichDCCA shows an optimal power solution with a relatively lower amplitude of power output. Thus, the proposed approach can provide a solution with a higher power tracking performance compared to the proportional technique. Figure 7C further exemplifies the superiority of the proposed DCCA in decreasing the power deviation between the power input command and power output by effectively coordinating the power command of the various regulation resources ( Figure 7B). Three indices of dynamic regulation performance; namely, area control error (ACE), CPS, and frequency deviation, are used to depict the real-time power tracking ability of the approach. The DCAA effectively reduced the amplitude of the ACE, CPS1, and frequency deviation curves compared to the traditional PROP ( Figure 6D-F).
Likewise, the tracking process between the regulation power command and the real-time power output acquired by the two algorithms under a load step disturbance ΔP D 100 MW is given in Figure 8A. During the optimal process obtained by DCCA, the power output of all ten regulation resources increased from zeros to a maximum value ( Figure 8B). As shown in Figure 8C, the local connected network-based DCCA method showed a series of power schemes with lower power deviations compared to the proportional-based method. This occurred primarily because the distributed consensus-based approach more effectively coordinated the various regulation resource compared to PROP. Figure 8D-F shows that the proposed DCCA provides a higher power tracking performance of dynamic ACE, CPS1, and frequency deviation.

Stochastic load disturbance
To further verify the superiority of the proposed method, a stochastic load disturbance given in Figure 9A is constructed to be executed to the DT-DPT system. Additionally, the CPS and power deviation are used to exemplify the dynamic regulation performance with the local connected network-based DCCA and proportional-based approach. First, Figure 9A shows the real-time power tracking of the two algorithms, in which the lower power tracking power is acquired by the proposed DCCA rather than the PROP. Moreover, the proposed method effectively reduced the power deviation for the DT-DPT system under stochastic load disturbance ( Figure 9C). This may be due to the high coordinate ability and real-time regulation performance of the DCCA for the various regulation resources ( Figure 9B) while the proportionalbased method lacks the optimal mechanism for compensation payment or dynamic regulation. Finally, the dynamic CPS curve is shown in Figure 9D, in which the proposed method helps to reduce CPS decrement, thus demonstrating that the proposed DCCA can acquire a power scheme with a higher regulation performance than the traditional PROP.
Lastly, the optimal results of online optimization under various disturbance scenarios are shown in Table 4. Indexes including the ACE maximum and minimum, frequency deviation, and CPS are used to compare the tracking performance of CCA and PROP. The power deviation of the agent power tracking is represented by the approximate degree between the real-time output and input commands. The results demonstrated that the suggested strategy could significantly minimize power variation while simultaneously improving tracing accuracy. In the ninth simulation experiments of step load disturbances, the suggested approach effectively reduced the power deviation by 21.7%, 10.1%, 9.8%, 7.0%, 7.0%, 11.7%,16.1%, 10.1%, respectively, and improved the CPS scores by 23%, 3.6%, 0.90%, 0.16%, 0.16%, 0.9%, 3.6%, 23.2%, and 4.7%, respectively. Thus, with increased power amplitude, the optimization performance of the algorithm becomes more significant.

Conclusion
The three contributions made in this work are as follows.
1) The proposed DT-DPT demonstrated a power tracking process with disturbance collection, controller strategy optimization, regulation resource response, and visualization of dynamic control performance. 2) The proposed local connected network can help accelerate the convergence of the DCCA in the DT-DPT system and reduce the communication pressure between digital agents in power tracking systems. 3) The load disturbance tests of the DT-DPT demonstrated that the proposed DCCA effectively and efficiently provided a highquality solution for the DT-DPT system. This method effectively increased the CPS sconce, reduced the power and frequency deviations in the power tracking area, and reduced the ACE for the DT-DPT system.

Data availability statement
The original contributions presented in the study are included in the article/supplementary material. Further inquiries can be directed to the corresponding author.

Frontiers in Energy Research
frontiersin.org