Load frequency optimal control of the hydropower-photovoltaic hybrid microgrid system based on the off-policy integral reinforcement learning algorithm

Abstract

With the promotion and development of clean energy, it is challenging to ensure the optimization of control performance in frequency control of the hydropower-photovoltaic hybrid microgrid system caused by the output power fluctuation of photovoltaic power generation. In this study, an optimal load frequency controller (LFC) for a hydropower-photovoltaic hybrid microgrid system was designed to improve the dynamic response when the load and photovoltaic output power are perturbed based on the off-policy integral reinforcement learning algorithm. First, a mechanism model of the hydropower-photovoltaic hybrid microgrid system was established. Next, the LFC problem was transformed into a zero-sum game control problem based on the characteristics of the power system. Subsequently, three neural networks were employed to approximate the Nash equilibrium solution of the zero-sum game with historical input and output data when the system dynamics are completely unknown. Finally, simulation experiments were conducted to verify the effectiveness and optimality of the proposed method. The introduction of this method provides a new perspective for frequency control for the hydropower-photovoltaic hybrid microgrid system.

1 Introduction

With the development of the national economy and society, the contradiction between increasing energy demand and energy shortages has become increasingly obvious (Gilani et al., 2020; Patnaik et al., 2020; Zhang and Kong, 2022). Traditional thermal power generation causes problems such as the consumption of nonrenewable energy and excessive carbon emissions (Ahmad et al., 2018; Cowie et al., 2020; Olabi and Abdelkareem, 2022). Hydropower and solar energy have attracted the attention of researchers owing to their renewable and environment-friendly nature (Gielen et al., 2019; Zepter et al., 2019).

However, photovoltaic (PV) power generation is intermittent, leading to unstable output power and microgrid frequency oscillations (Thirunavukkarasu and Sawle, 2021; Chen et al., 2022; Wu and Yang, 2023). To ensure the frequency stability of a microgrid, it is necessary to supplement controllable power sources, such as hydroelectric units or energy storage devices, to fill the power deficit, which can effectively maintain the microgrid frequency stability (Coban et al., 2022). The power quality of PV power systems can be improved by utilizing a control algorithm for controllable power sources, which is applied to obtain an optimal load frequency controller (LFC) system (Papaefthymiou et al., 2010; Ma et al., 2014; Dhundhara and Verma, 2020). Some researchers focus on the suppression of local load fluctuations and their interactions with the distribution system (Khalid et al., 2022). Additionally, the role of ancillary services and the integration of renewable energy should also be addressed upon introduction to minimize fluctuations and cover intermittency (Khalid et al., 2022; Rehman et al., 2024; Osman et al., 2022).

Owing to their simple structure and ease of implementation, proportional-integral-derivative (PID) control methods are widely used in microgrid LFC (Mohamed et al., 2020; Nisha and Jamuna, 2022). Ray et al. (2011) utilized a PI controller to regulate the frequency of a microgrid and achieve the required frequency ratings. Guha et al. (2021) designed a fractional-order PID method to solve the frequency stabilization problem of microgrid systems with uncertain parameters. Huang et al. (2021) used fuzzy reasoning in PID to improve the control performance of a hydraulic turbine regulation system.

Many practical power systems can only be partially modeled, and models of unknown parts are unavailable (Ganguly et al., 2018; Li et al., 2023; Wu and Yang, 2023). Dynamic characteristics of droop-controlled inverters are evaluated by a reduce-order small-signal transfer function model, which is designed on the basis of the Jordan continued-fraction expansion to provide a preprocessing method for real-time power system simulation (Wang et al., 2020). Therefore, owing to the insensitivity to the dynamics of the unmodeled parts of the controlled object, adaptive control methods have been proposed by continuously identifying system parameters to achieve the ideal control effect. Adaptive control methods can be used to resolve problems arising from parameter variations in the LFC of a power system. Zeng et al. (2015) designed a port-controlled Hamiltonian system that decomposed nonlinear control into stabilizing control with a given equilibrium point and proposed L-2 adaptive control for application to a hydroelectric generator unit. Fang et al. (2011) effectively improved the dynamic performance of the hydraulic turbine regulation process using an improved particle swarm optimization algorithm, which was applied to the optimal design of the parameters of a hydraulic turbine regulation system to achieve an optimal positive setting of the parameters. Tran et al. (2021) used a combination of second-order sliding film control and a state estimator for frequency regulation to reduce the number of overtones. Although these methods can achieve better control performance, they have not been widely popularized in practical power systems owing to their complexity and difficulty.

The adaptive dynamic programming (ADP) algorithm is an emerging intelligent control algorithm that solves the problem of dimensional disasters caused by the traditional dynamic programming (DP) method (Werbos, 1992; Vamvoudakis and Lewis, 2010; Lewis et al., 2012; Bellman and Dreyfus, 2015) and is suitable for systems with a high degree of nonlinearity. Shuai et al. (2020) used a hybrid ADP algorithm to achieve optimal operation of gas and electric systems. Xue et al. (2022) used ADP for the real-time scheduling of battery heat storage tank integrated heat and power systems, providing optimal economic operation strategies. The off-policy integral reinforcement learning (IRL) algorithm is proposed based on the theory of the ADP algorithm, which can explore system information with historical input and output data, thereby overcoming the difficulty of traditional ADP relays on neural network weights in the training process to find the continuous excitation function. Chai et al. (2017) used the game theory to solve multi-objective trajectory optimization problems for aerial vehicles. Song et al. (2019) proposed an off-policy IRL algorithm to solve an optimal control problem with partially known system dynamics. Based on the ADP algorithm, this paper proposes an integral reinforcement learning method that requires only the historical input-output data of the system, allowing for optimal solutions even when the system dynamics are completely unknown.

2 Problem statement

The hydropower-photovoltaic microgrid power system effectively exploits the inherent frequency regulation advantages of hydropower units while integrating solar energy generation resources within the same regional grid. This hybrid system aims to enhance the overall frequency quality of the microgrid by balancing both renewable energy inputs and electrical load demand. However, such integration significantly increases the operational requirements for the Load Frequency Control (LFC) controller. In this study, Figure 1 outlines the core structure of the system: a power busbar connects hydropower units (HP), photovoltaic generation units (PV), and electrical loads. Specifically, the PV units are connected to the alternating current (AC) microgrid through direct current (DC) to alternating current (AC) conversion using DC/AC inverters.

FIGURE 1

The frequency stability of this isolated microgrid relies heavily on maintaining an active power balance within the network. Variations in electrical load and the intermittent, fluctuating output from photovoltaic sources can disturb this balance, leading to changes in system frequency. A central feature of the hydropower-photovoltaic microgrid system is the hydro-turbine generator, which is responsible for providing rotational reserves that help regulate frequency by adjusting the mechanical input to the turbine. This response compensates for any mismatch between generation and demand, ensuring system stability.

The hydropower units play a critical role in Load Frequency Control (LFC) tasks. The primary function of the LFC system is to regulate water flow into the turbines of the hydropower generators. It achieves this by dynamically adjusting the active power output of the hydropower units in real-time, depending on the load and intermittent power output from the solar resources. This real-time control is vital for compensating fluctuations in both solar power production and load changes, stabilizing the generator speed, and ultimately controlling the frequency of the microgrid.

To improve the effectiveness of the system and minimize control costs, an advanced optimal load frequency controller was designed, utilizing an off-policy Inverse Reinforcement Learning (IRL) algorithm. This controller ensures the stability of the grid-connected voltage in the hydropower-photovoltaic microgrid by optimizing the dynamic allocation of power resources. In essence, it manages the trade-offs between ensuring grid frequency stability and maintaining operational cost efficiency, leading to a robust, reliable, and sustainable microgrid power system.

3 Materials and methods

3.1 Establishment of the hydro turbine group model

The hydro turbine group consisted of hydro turbines, governors, and generators. A turbine group model was established for each part.

The equations of moment and flow of the hydro turbine are expressed as Equation 1:where and represent the increments in torque and flow, respectively; , , and indicate the relative deviation variables of the guide vane opening, water head, and hydro turbine speed, respectively; and represent the transfer coefficients of the hydro turbine torque to the speed, guide vane opening and water head, respectively; and are transfer coefficients of the hydro turbine flow to the speed, guide vane opening and the water head, respectively. Under stable conditions, each transfer coefficient is regarded as a constant.

When , the transfer function from the guide vane increment to the torque increment of the hydro turbine is as follows:where indicates the inertia time constant of the water flow.

The hydro turbine governor can be simplified as a first-order inertial link by ignoring the nonlinear factors, as Equation 3:where represents the response time constant of the hydro-turbine governor. The corresponding differential equation is as follows:

Equation 2 is substituted into Equation 4 after the Laplace transformation. The differential equation is obtained by the Laplace transformation of Equation 2, and Equation 4 is substituted into the differential equation to obtain the hydro turbine differential equation as follows:

The second-order model of the generator includes the rotor rotation motion equation and the equation that characterizes the relationship between the power angle and speed, as follows:where indicates the generator power angle, which remains constant under stable operating conditions, and indicates the electromagnetic power increment, which is equivalent to the relative value of the load increment indicates the synchronous electrical angular velocity, D is the damping coefficient, and indicates the inertia time constant of the generator. is the sum of the incremental relative values of power supplied by all sources within the microgrid. From the Equation 6, we get . represents the equivalent inertia time coefficient of the system, which is equal to the weighted sum of the inertia coefficients of all the generators in the system. represents the equivalent load-damping factor of the system.

By combining Equations 4–6, the following mathematical model of the hydro-turbine group can be obtained:

3.2 Establishment of the photovoltaic model

PV panels convert solar energy into electrical energy based on PV effects. The main body of the frequency control in this study was the hydropower unit. Therefore, in this subsection, a first-order model with time constant was used to express the frequency characteristics of the PV power as follows:where represents the time constant of the PV power system, represents the output power of the PV power system, and represents the solar power. Solar power has obvious volatility, as PV panels are easily affected by light intensity and ambient temperature. Therefore, solar power is regarded as a disturbance term. A block diagram of the model is shown in the Figure 2.

FIGURE 2

3.3 Establishment of the hydropower-photovoltaic microgrid power system model

The transient changes in the voltage and power angle of the system can be ignored in the frequency control analysis. Therefore, in the analysis process of LFC, is the incremental relative value of the output power of the hydro turbine group, the relative value of the electromagnetic power increment is replaced by the relative value of the load increment , and the relative value of the speed deviation is equal to the relative value of the frequency deviation .

By combining Equations 7, 8, the hydro-photovoltaic microgrid power system can be derived as Equation 9:

Here, = are state variable. The load frequency and solar energy of the input system are considered as the elements in the disturbance vector, and the PV output power fluctuation and load power change cause the power supply and demand of microgrid to lose balance. The system disturbance vector can be obtained as follows: ; is the control vector signal. Therefore, the load frequency model of the hydropower-photovoltaic system can be obtained as follows:where the system state variable the system control variable and is the disturbance variable . The system matrix is expressed as follows:

Thus far, the load frequency control problem of the hydropower-photovoltaic microgrid power system has been transformed into a zero-sum game optimal control problem, wherein the input of the governor was taken as the control variable and the load frequency and solar power were taken as the disturbance variables.

4 Results of the optimal controller based on off-policy IRL algorithm

The hydropower-photovoltaic microgrid power system model was established using Equation 10, where and are the state, control input, and disturbance input of the system, respectively. is the equilibrium point of the hydropower-photovoltaic microgrid power system. The infinite-horizon performance index function can be designed as follows:where in the utility function can be described as Equation 12:where the coefficient matrices , and are real symmetric positive definite matrices. Please note that this performance indicator has clear economic significance, in which is the penalty cost when the quality of power supply at the node i deviates from the system’s stable value and is the control costs and disturbance costs incurred by node i to reduce penalty costs.

The purpose of the zero-sum game is to solve for an optimal control that satisfies Equation 13.

The zero-sum game selects to minimize the player set and maximize the player set , the saddle point and must satisfy the following inequality Equation 14:

When there is a unique set of solutions that satisfy the following Nash equilibrium condition Equation 15:the cost function of every player can be written as Equation 16:

Using the Leibniz formula and differentiating Equation 6, the Bellman equation of the zero-sum game can be obtained as follows:where . The control and disturbance inputs can be obtained as Equations 18, 19:

The optimal control policy and the optimal disturbance can be derived as follows:

The Hamilton-Jacobi-Bellman equation can be obtained by substituting Equations 20, 21 into Equation 17 as follows:

The following equations were used to update the control and disturbance policies as Equations 23, 24:where the superscript represents the number of steps in the iteration process.

The Equation 11 can be transformed as Equation 25:

The Equation 22 can be rewritten as follows:

By deriving Equation 23, we get and ; upon substituting these parameters into Equation 26, the following equation was obtained:

From Equations 26, 27, the system dynamic matrices A, B, and F are replaced. Equation 27 overcomes the difficulty of obtaining the dynamic information of the system in practical applications. ) is a unique solution for the off-policy IRL algorithm. Three neural networks were employed to solve the solution ); the expressions are as follows: and satisfy , and , respectively, and are linearly independent. is approached by a critical neural network (CNN), and and are approached by an action neural network (ANN) and a disturbance neural network (DNN), respectively. Here, , , and indicate the weights of CNN, ANN, and DNN, respectively.

According to Equation 28, the residual can be written as follows:

Substituting Equations 28–30 into Equation 31 yields

In order to simplify Equation 32, the following parameters are defined as Equations 33–39:

Then, Equation 32 can be written as Equation 40:

The Equations 41–43 are then generated to obtain the optimal solutions:

Finally, Equation 32 can be written as follows:

In order to solve weight , the residual error is approximated to zero. The inner product is applied to solve using Equation 45 as follows:upon substituting Equation 45 is substituted in Equations 44, 46 is obtained as follows:

can be calculated as follows:

Various numerical integrals in domain D were acquired to calculate and . The Monte Carlo integration was used to resolve this calculation. When and all of them are the sets sampled on domain D. M indicates the dimensions of the , which should be as wide as possible to ensure that the sample set provides full coverage of D. Therefore, can be obtained as follows:in which, where .

Upon substituting Equations 48, 49 in Equation 47, the following equation is obtained:

The zero-sum problem can be solved using Algorithm 1 as follows:

It is worth mentioning that the input and output data of the hydropower-photovoltaic microgrid power system are necessary to solve the zero-sum problem when the system dynamics are completely unknown.

5 Discussion

The hydropower-photovoltaic microgrid power system model was established, the proposed Algorithm 1 was utilized to solve the LFC control, and the simulation was realized in the MATLAB platform. Simulation results verified that the microgrid can maintain frequency stability despite local load and PV disturbances. The control and disturbance curves eventually approach to 0, as shown in Figure 3. The Figure 3 illustrates the behavior of two variables over a period of 10 time steps, designated on the x-axis. The y-axis represents the Control Value ranging from −0.5 to 1.0. The graph features two sets of trajectories for the control u and ω, each represented by both initial estimated values and adjusted values obtained using the Algorithm 1. The dashed and solid lines indicate the approximation curves under initial admissive control and Algorithm 1, respectively. It can be seen that the convergence speed of Algorithm 1 is better than that of the initial admissible control method. The frequency finally stabilized. For variable ω, it starts from a lower value and similarly converge towards zero. Overall, the obtained trajectories using Algorithm 1 exhibit a more rapid convergence towards 0 for both u and ω compared to their respective initial trajectories. Demonstrating the enhanced performance of Algorithm 1 over the initial admissible control method.

FIGURE 3

The weight convergence curves of the three networks are shown in Figures 4–6. These three figures illustrates the convergence of weights for every seven different networks over 15 iterations, represented on the x-axis. The y-axis denotes the Weight of Value Function, with values ranging from −10 to 15. At the beginning of the iterations, the weights of these networks start from various initial values. Some weights exhibit significant fluctuations, particularly up to around the 5th iteration, where noticeable oscillations are apparent. As iterations proceed, the weights for all networks gradually stabilize and converge to constant values between 0 and 1, indicating that the values no longer change significantly with additional iterations. By the 15th step, all weights have reached their steady-state values. The convergence of these weights within 15 iterations suggests that the optimization process is effective, allowing the system to arrive at an optimal control that approximates the Nash equilibrium point for the zero-sum game problem.

FIGURE 4

FIGURE 5

FIGURE 6

Compared to traditional Dynamic Programming methods, the proposed method effectively overcomes the “curse of dimensionality,” significantly reducing the computational burden when solving high-dimensional matrices. In contrast to previous reinforcement learning approaches for controlling the optimal frequency of hydropower-photovoltaic microgrid power systems, this method incorporates the consideration of disturbance factors, providing a robust theoretical basis for the grid integration of hybrid power generation systems.

The desired voltage and current is 50 hz sinusoidal waves, such that the systemis dynamic with high frequency. Yet the IRL method depends a process to collect the control and states data from the system under a quasi-optimal control, which may lead to the power oscillation and need more time to turn the system from the transient state to steady state. Therefore, the limitation of this method is that it is currently only applicable to offline systems.

6 Conclusion

This paper focused on the hydropower-photovoltaic hybrid microgrid system and designed an optimal LFC using the IRL algorithm. First, the mechanism models of the hydro turbine generator and the photovoltaic generator were established, respectively. Second, a state-space model of the hydropower-photovoltaic hybrid microgrid system was developed, and based on the power generation characteristics, it was transformed in solving a zero-sum game problem. Third, the IRL algorithm was employed to approximate the Nash equilibrium point of the zero-sum game problem using three neural networks. Finally, the simulation experiments were conducted to verify the effectiveness of the proposed method.

References

• 1

  AhmadS.AhmadA.YaqubR. (2018). Optimized energy consumption and demand side management in smart grid. Smart Grid as a Solut. Renew. Effic. Energy, 1–25. 10.4018/978-1-5225-0072-8.ch001

  • CrossRef
  • Google Scholar

• 2

  BellmanR. E.DreyfusS. E. (2015). Applied dynamic programming. Princeton, New Jersey: Princeton University Press.

  • Google Scholar

• 3

  ChaiR.SavvarisA.TsourdosA.ChaiS. (2017). Multi-objective trajectory optimization of space manoeuvre vehicle using adaptive differential evolution and modified game theory. Acta Astronaut.136, 273–280. 10.1016/j.actaastro.2017.02.023

  • CrossRef
  • Google Scholar

• 4

  ChenZ.ChenJ.FuK.XueL. (2022). Power coordination control strategy of microgrid based on photovoltaic generation. MATEC Web Conf.355, 03065. 10.1051/matecconf/202235503065

  • CrossRef
  • Google Scholar

• 5

  CobanH. H.RehmanA.MousaM. (2022). Load frequency control of microgrid system by battery and pumped-hydro energy storage. Water14, 1818. 10.3390/w14111818

  • CrossRef
  • Google Scholar

• 6

  CowieP.TownsendL.SaleminkK. (2020). Smart rural futures: will rural areas be left behind in the 4th Industrial Revolution?. J. Rural. Stud.79, 169–176. 10.1016/j.jrurstud.2020.08.042

  • CrossRef
  • Google Scholar

• 7

  DhundharaS.VermaY. P. (2020). Application of micro pump hydro energy storage for reliable operation of microgrid system. IET Renew. Power Gener.14, 1368–1378. 10.1049/iet-rpg.2019.0822

  • CrossRef
  • Google Scholar

• 8

  FangH.ChenL.ShenZ. (2011). Application of an improved PSO algorithm to optimal tuning of PID gains for water turbine governor. Energy Convers. Manag.52, 1763–1770. 10.1016/j.enconman.2010.11.005

  • CrossRef
  • Google Scholar

• 9

  GangulyS.ShivaC. K.MukherjeeV. (2018). Frequency stabilization of isolated and grid connected hybrid power system models. J. Energy Storage.19, 145–159. 10.1016/j.est.2018.07.014

  • CrossRef
  • Google Scholar

• 10

  GielenD.BoshellF.SayginD.BazilianM. D.WagnerN.GoriniR. (2019). The role of renewable energy in the global energy transformation. Energy Strategy Rev.24, 38–50. 10.1016/j.esr.2019.01.006

  • CrossRef
  • Google Scholar

• 11

  GilaniM. A.KazemiA.GhasemiM. (2020). Distribution system resilience enhancement by microgrid formation considering distributed energy resources. Energy191, 116442. 10.1016/j.energy.2019.116442

  • CrossRef
  • Google Scholar

• 12

  GuhaD.RoyP. K.BanerjeeS. (2021). Equilibrium optimizer‐tuned cascade fractional‐order 3DOF−PID controller in load frequency control of power system having renewable energy resource integrated. Int. Trans. Electr. Energy Syst.31, e12702. 10.1002/2050-7038.12702

  • CrossRef
  • Google Scholar

• 13

  HuangZ.LiuX.FuH.DuZ. (2021). A novel parameter optimisation method of hydraulic turbine regulating system based on fuzzy differential evolution algorithm and fuzzy PID controller. Int. J. Bio Inspired Comput.18, 153–164. 10.1504/IJBIC.2021.119203

  • CrossRef
  • Google Scholar

• 14

  KhalidH. M.MuyeenS. M.KamwaI. (2022). An improved decentralized finite-time approach for excitation control of multi-area power systems. Sustain. Energy, Grids Netw.31, 100692. 10.1016/j.segan.2022.100692

  • CrossRef
  • Google Scholar

• 15

  LewisF. L.VrabieD.SyrmosV. L. (2012). Optimal control. Hoboken, New Jersey: John Wiley & Sons.

  • Google Scholar

• 16

  LiJ.GuoW.LiuY. (2023). Nonlinear state feedback-synergetic control for low frequency oscillation suppression in grid-connected pumped storage-wind power interconnection system. J. Energy Storage.73, 109281. 10.1016/j.est.2023.109281

  • CrossRef
  • Google Scholar

• 17

  MaT.YangH.LuL.PengJ. (2014). Technical feasibility study on a standalone hybrid solar-wind system with pumped hydro storage for a remote island in Hong Kong. Renew. Energy.69, 7–15. 10.1016/j.renene.2014.03.028

  • CrossRef
  • Google Scholar

• 18

  MohamedR.HelaimiM.TalebR.GabbarH. A.OthmanA. M. (2020). Frequency control of microgrid system based renewable generation using fractional PID controller. IJEECS19, 745–755. 10.11591/ijeecs.v19.i2.pp745-755

  • CrossRef
  • Google Scholar

• 19

  NishaG.JamunaK. (2022). Frequency stabilization of stand-alone microgrid with tuned PID controller. ECS Trans.107, 773–782. 10.1149/10701.0773ecst

  • CrossRef
  • Google Scholar

• 20

  OlabiA. G.AbdelkareemM. A. (2022). Renewable energy and climate change. Renew. Sustain. Energy Rev.158, 112111. 10.1016/j.rser.2022.112111

  • CrossRef
  • Google Scholar

• 21

  OsmanN.KhalidH. M.Tha’erO. S.AbuashourM. I.MuyeenS. M. (2022). A PV powered DC shunt motor: study of dynamic analysis using maximum power Point-Based fuzzy logic controller. Energy Convers. Manag. X15, 100253. 10.1016/j.ecmx.2022.100253

  • CrossRef
  • Google Scholar

• 22

  PapaefthymiouS. V.KaramanouE. G.PapathanassiouS. A.PapadopoulosM. P. (2010). A wind-hydro-pumped storage station leading to high RES penetration in the autonomous island system of Ikaria. IEEE Trans. Sustain. Energy.1, 163–172. 10.1109/TSTE.2010.2059053

  • CrossRef
  • Google Scholar

• 23

  PatnaikB.MishraM.BansalR. C.JenaR. K. (2020). AC microgrid protection–A review: current and future prospective. Appl. Energy.271, 115210. 10.1016/j.apenergy.2020.115210

  • CrossRef
  • Google Scholar

• 24

  RayP. K.MohantyS. R.KishorN. (2011). Proportional–integral controller based small-signal analysis of hybrid distributed generation systems. Energy Convers. Manag.52, 1943–1954. 10.1016/j.enconman.2010.11.011

  • CrossRef
  • Google Scholar

• 25

  RehmanA. U.UllahZ.QaziH. S.HasanienH. M.KhalidH. M. (2024). Reinforcement learning-driven proximal policy optimization-based voltage control for PV and WT integrated power system. Renew. Energy227, 120590. 10.1016/j.renene.2024.120590

  • CrossRef
  • Google Scholar

• 26

  ShuaiH.AiX.FangJ.DingT.ChenZ.WenJ. (2020). Real-time optimization of the integrated gas and power systems using hybrid approximate dynamic programming. Int. J. Electr. Power Energy Syst.118, 105776. 10.1016/j.ijepes.2019.105776

  • CrossRef
  • Google Scholar

• 27

  SongR.WeiQ.LiQ. (2019). Off-policy integral reinforcement learning method for multi-player non-zero-sum games. Adapt. Dyn. Program. Single Multiple Control., 227–249. 10.1007/978-981-13-1712-5_12

  • CrossRef
  • Google Scholar

• 28

  ThirunavukkarasuM.SawleY. (2021). Smart microgrid integration and optimization. Act. Electr. Distrib. Netw. Smart Approach, 201–235. 10.1002/9781119599593.ch11

  • CrossRef
  • Google Scholar

• 29

  TranA. T.MinhB. L. N.HuynhV. V.TranP. T.AmaefuleE. N.PhanV. D.et al (2021). Load frequency regulator in interconnected power system using second-order sliding mode control combined with state estimator. Energies14, 863. 10.3390/en14040863

  • CrossRef
  • Google Scholar

• 30

  VamvoudakisK. G.LewisF. L. (2010). Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica46, 878–888. 10.1016/j.automatica.2010.02.018

  • CrossRef
  • Google Scholar

• 31

  WangR.SunQ.PinjiaZ.YonghaoG.DehaoQ.PengW. (2020). Reduced-order transfer function model of the droop-controlled inverter via Jordan continued-fraction expansion. IEEE Trans. Energy35, 1585–1595. 10.1109/TEC.2020.2980033

  • CrossRef
  • Google Scholar

• 32

  WerbosP. (1992). Approximate dynamic programming for real-Time control and neural modeling. Handb. Intelligent Control.

  • Google Scholar

• 33

  WuJ.YangF. (2023). A dual-driven predictive control for photovoltaic-diesel microgrid secondary frequency regulation. Appl. Energy.334, 120652. 10.1016/j.apenergy.2023.120652

  • CrossRef
  • Google Scholar

• 34

  XueX.AiX.FangJ.YaoW.WenJ. (2022). Real-time schedule of integrated heat and power system: a multi-dimensional stochastic approximate dynamic programming approach. Int. J. Electr. Power Energy Syst.134, 107427. 10.1016/j.ijepes.2021.107427

  • CrossRef
  • Google Scholar

• 35

  ZengY.ZhangL. X.GuoY. K.QianJ. (2015). Hamiltonian stabilization additional L 2 adaptive control and its application to hydro turbine generating sets. Int. J. Control Autom. Syst.13, 867–876. 10.1007/s12555-013-0460-7

  • CrossRef
  • Google Scholar

• 36

  ZepterJ. M.LüthA.Crespo del GranadoP. C.EggingR. (2019). Prosumer integration in wholesale electricity markets: synergies of peer-to-peer trade and residential storage. Energy Build.184, 163–176. 10.1016/j.enbuild.2018.12.003

  • CrossRef
  • Google Scholar

• 37

  ZhangD.KongQ. (2022). Green energy transition and sustainable development of energy firms: an assessment of renewable energy policy. Energy Econ.111, 106060. 10.1016/j.eneco.2022.106060

  • CrossRef
  • Google Scholar