Integrated photovoltaic storage joint smoothing strategy based on simultaneous perturbation stochastic approximation algorithm

In order to realize the real-time control of photovoltaic power generation smoothly connected to the grid under the condition that the energy storage equipment can operate safely, a control strategy combining the simultaneous perturbation stochastic approximation (SPSA) algorithm with rule control is designed. Firstly, the photovoltaic data are processed to extract the data characteristics of the power ramp, and then the grid-connected reference power is obtained through SPSA algorithm. Secondly, considering the state of charge (SOC) of energy storage equipment and the safe operating power range of hydrogen storage equipment, 24 hybrid energy storage power allocation rules are formulated by using the rule control method. Finally, according to the sampling data of every 10 s interval in typical day, the simulation is carried out on MATLAB/simulink platform. The results show that, compared with the first-order low-pass filtering algorithm and recursive fuzzy neural network (RFNN) algorithm, the SPSA algorithm has stronger smoothing ability, and the rule control can also complete the allocation according to the characteristics of the hybrid storage device while ensuring its normal operation.


Introduction
China's government has clearly put forward the target of carbon peak in 2030 and carbon neutrality by 2060, and the proportion of non-fossil energy in primary energy consumption has increased significantly. Solar energy is a kind of renewable energy with abundant resources, wide distribution and no pollution (Nigam and Sharma, 2021). In 2020, the cumulative installed photovoltaic capacity in the world was 767.2 GW, which was significantly higher than that of 39.2 GW in 2010. Therefore, it is imperative to continue to develop ultra-large scale and ultra-high proportion of photovoltaic renewable clean energy. However, photovoltaic power generation depends on weather parameters, such as solar irradiance, so it has strong intermittence and randomness in nature (Shaner et al., 2018). The conflict between the controllability of the non-sourced side and the high dynamics load is more prominent (Bharti et al., 2022). In order to solve this problem, experts and scholars have proposed a more flexible solution, which combines photovoltaic power generation with energy storage systems to eliminate the fluctuations of active power (Akbari et al., 2019). Energy storage systems is a widely accepted solution because most of them are geographically independent and controllable in different time frames (Castillo and Dennice 2014). The power conditioning system composed of hydrogen storage and super-capacitor is one of the ways (Bharti et al., 2022). This energy storage system belongs to the category of hybrid energy storage, which can reduce environmental costs, improve response time, energy conversion efficiency and durability. Combining a large-scale photovoltaic power generation systems with hybrid energy storage systems can significantly alleviate the uncertain random power generation challenges in power grid operation and planning (Puranen et al., 2021).
The rapid growth of photovoltaic power generation in the past few years has promoted the demand for robust energy management strategies, which take into account the specific technical features and limitations of different storage devices . Most of the previous off-line power management strategies assumed perfect forecast of renewable energy, demand and markets, which is difficult to achieve in practice (Li 2021). Appropriate on-line energy management strategy can balance the power of a hybrid energy system, improving system reliability and system efficiency, reducing production costs, prolonging the life of system components (Y. Wu et al., 2018). As far as technical, economic and social standards are concerned, the stochastic strategy surpasses the deterministic method, and it has high potential for real-time power management strategies (S. Yin et al., 2021). In (P. Li et al., 2020), the Lyapunov drift-plus-penalty function is applied to formulate a relaxed form of the energy management problem, more effectively taking into account the uncertainty of energy storage and power load. Tabar et al. (2019) presents the application of stochastic linear programming method in the energy management of hybrid micro-gridncertain systems considering multiple markets and rereal-timeemand response. Using this method can effectively reduce the cost, pollution and demand payment of micro-grid.
Common stochastic methods include stochastic game theory algorithm, stochastic linear programming algorithm, Lyapunov optimization algorithm, and SPSA algorithm (Ciupageanu, Barelli et al., 2020). Among them, Liapunov optimization algorithm and SPSA algorithm are gradient-based stochastic algorithm, can adapt to the time-varying environment without mandatory uncertainty modeling in the process of real-time energy management (Neely 2010). In the past, most of the articles used real-time energy management methods based on Liapunov optimization algorithm, while the smoothing strategy of photovoltaic power generation based on SPSA algorithm is not common. What's more, most of the previous articles are aimed at the single-objective or multi-objective optimization of a single online control algorithm. few articles combine the online control based on optimization algorithm with the control strategy based on rules, and apply them to the energy distribution process of hybrid energy storage, which can smooth the fluctuations and considering the characteristics of the equipment. These articles can not consider the operation characteristics of the energy storage equipment more flexibly. For example (Barelli et al., 2020), smoothes the fluctuation of wind power generation based on the stochastic approximation of simultaneous disturbance combined with a flywheel-battery storage system, but its constraint target is not suitable for the safe operation conditions of hydrogen storage. Hydrogen storage devices can realize medium and long-term storage, and the hybrid storage system composed of this devices and super-capacitors has more unique requirements on operational safety than the previous hybrid storage system composed of batteries and supercapacitors.
Therefore, a strategy of photovoltaic output fluctuation smoothing and power distribution based on SPSA algorithm and rule control is proposed. Firstly, the SPSA algorithm is applied to smooth the photovoltaic output at the system level, the result is compared with the smoothing results of the first-order low-pass filtering algorithm and the RFNN algorithm during the result verification, the smoothness drop to 0.3418 in the verification, which effectively suppresses the fluctuations. Secondly, in the energy allocation of the hybrid energy storage system, considering the SOC of the energy storage system and the power constraint of the hydrogen storage equipment, 24 rules are established for the energy allocation of the hybrid energy storage system, so that the hydrogen storage equipment can operate within the safe power constraint on the basis of giving full play to the respective advantages of energybased and power-based equipment, so as to avoid the explosion accident caused by the excess hydrogen in oxygen.
2 Grid-connected photovoltaic hydrogen hybrid storage coupling system topology diagram The combination of integrated photovoltaic systems and hybrid energy storage devices can reduce the active power imbalance of power grid regulation, improve the stability of power grid and increase the penetration rate of new energy. In this paper, the photovoltaic power station adopts a common AC bus topology, and the energy storage system consists of alkaline electrolyzed water (AEW), hydrogen storage tank, proton exchange membrane fuel cell (PEMFC), and super capacitor, as shown in Figure 1.

Frontiers in Energy Research
frontiersin.org 02 The energy balance equation is shown below. P pv (t) P grid (t) + P HESS (1) Where: P pv (t) is the active power emitted by the photovoltaic array at the moment t, P grid (t) is the power received by the grid at the moment t, P HESS (t) is the power allocated to the energy storage system at the moment t. P c (t) and P H (t) is the power allocated by the energy storage system to the super-capacitor and hydrogen storage system respectively, and the units of the above power are kW.

Modeling of hybrid energy storage equipment
This paper only considers the charging and discharging behavior and capacity limitation of energy storage devices, so it only discuss the relationship between charging and discharging power and SOC.

Operating characteristics and constraints of super-capacitor
In order to make the capacitor work in a normal state, the output power of the supercapacitor needs to be corrected with the SOC (Zhang et al., 2018). The expression is shown as follows: SOC(i + 1) and SOC(i) denotes the SOC at moment t and moment t+1 respectively; E c max denotes the maximum capacity of super-capacitor; η C is the charging efficiency, which is taken as 0.95, and η D is the discharging efficiency, which is taken as 0.95. P c (t) is the power allocated to supercapacitor at moment t, which is taken as positive when charging and negative when discharging. Δt is the sampling time.

Operational characteristics and constraints of hydrogen storage system
In this paper, the pressure of the high-pressure hydrogen storage tank is used to describe the SOC of the hydrogen storage system (Wang and Lin, 2019), The expressions of hydrogen flow rate in electrolyzer and fuel cell are as follows.
N C is the hydrogen flow rate during charging, and N D is the hydrogen flow rate during discharging. η AEW is the charging

FIGURE 1
Topology of photovoltaic hydrogen hybrid energy storage system.

Frontiers in Energy
Research frontiersin.org 03 efficiency and η FC is the discharging efficiency. P AEW (t) is the charging power of electrolytic tank and P FC (t) is the discharging power of fuel cell. F is the Faraday constant. U AEW (t) is the voltage of the electrolyzer and U FC (t) is the voltage of the fuel cell.
The equation for the SOC is shown as follows.
is the increase in the pressure of the hydrogen storage tank and ΔP D (t) is the decrease in the pressure of the hydrogen storage tank. R is the universal time constant, T is the gas temperature, and V is the volume of the hydrogen storage tank. ΔP G (t) is the change in the total pressure of the hydrogen storage tank and P G (t − 1) is the total pressure of the tank at the previous moment. SOC H (t) is the SOC of the hydrogen storage tank. P H max is the maximum pressure that the hydrogen storage tank can withstand.
3 Photovoltaic power fluctuation smoothing strategy 3.1 Photovoltaic power data processing In this paper, we take the of historical photovoltaic data of a certain place in China for example to analyze. This data includes two datasets: the first dataset is the sampling data of a centralized 2 MW photovoltaic power generation from January to July 4th at 5 min intervals. The second set of data is the sampling data every 10 s from 4:00 to 19:00 on a certain day in June.
The power slope ΔP is a measure of the photovoltaic power fluctuation, which can be expressed as the difference between the current power and the previous power. In order to use the algorithm for online control, it is necessary to use mathematical statistics and reasoning methods to find the data characteristics of the power ramp, so as to predict the value of the next second during real-time control. We averaged the daily power ramps for in the sample, and obtained the confidence intervals for the average and standard deviation for all sample points by fitting a normal distribution curve, as shown in Figure 2.
According the fitted results, the average value of power ramp is 29.7353 kW (26.6837,32.787) at 95% confidence interval, and the standard deviation is 20.6894 kW (18.7452,23.087) at 95% confidence interval. This data feature will be described in the following SPSA algorithm by the cost function.

Fluctuation smoothing strategy
In the process of smoothing the fluctuation of photovoltaic processing, this paper divides the control system into two parts: system level and equipment level, and the upper level is the lower level to output reference signals, as shown in Figure 3.
In this paper, we use the SPSA algorithm at the system level to calculate the power P grid (t) that satisfies the grid connection criteria after smoothing and the power P HESS (t) of the hybrid energy storage. The power of the hybrid energy storage device is allocated at the device level using the rule control algorithm.

Description of the simultaneous perturbation stochastic approximation algorithm
The input and output of the stochastic system are random signals. The stochastic approximation algorithm constructs stochastic recursive formulas based on the input and output data and uses the observed values to estimate the extreme values of the unknown function; this algorithm does not require a priori knowledge of the system model, i.e., it does not need to know the specific form of the cost function (Mokkadem and Pelletier, 2007).
The SPSA algorithm was put forward by Spall in 1992. In each gradient approximation, the SPSA algorithm only needs two measurements of the objective function, which is easy to implement and effictive compared to other stochastic approximation algorithms (Spall, 1997). Compared with intelligent optimization-seeking algorithms such as finite difference, simulated annealing, and genetic algorithms, the SPSA algorithm is more robust to stochastic signals in the Introduce a sequence of independent identical distributions with zero mean Calculate the value of the cost function g (10) at θ k ± c k Δ k , as shown in the following equation.
k is the measurement noise term. The gradient is estimated as ...
The form of the SPSA algorithm is as follows.
The gain coefficient a k , c k , which can be obtained from the following equation.

Application of simultaneous perturbation stochastic approximation algorithm at system level
In order to smooth the fluctuation of photovoltaic power, this paper adopts the method of minimizing the power ramp of the next second grid-connected power relative to the photovoltaic power output at the moment. The SPSA algorithm can calculate the optimal power share to the power grid at every sampling moment. The equation of the cost function is shown as follows.
Each variable in the vector of unknown parameter θ corresponds to the distribution coefficient of the power ramp ΔP allocated to the power grid, and the hybrid storage device, as shown in the following formula. θ q grid q Hess q grid is the distribution coefficient of the grid, and q Hess is the distribution coefficient of the mixed storage equipment. Multiply the power ramp by the distribution coefficient obtained by the algorithm seeking to calculate the real-time distribution value. Then, add the power ramp allocated to the power grid with the photovoltaic power at this time to obtain the grid-connected power, as shown in the following formula.
P grid P t grid + P t pv (22) P t grid is the power allocated to the grid at time t, P t HESS is the power allocated to the hybrid storage device at time t, and P t pv is the power output of the photovoltaic at time t. The parameters ina k and c k are set according to the principle of the algorithm and the process of simulation as shown in Table 1.  Starting from the initial value of θ, according to the principles described in Section 2.1.1, the value of distribution coefficient is obtained iteratively.

Application of rule control at the device level
The concentration of hydrogen in oxygen is an important indicator to measure the safety of electrolyzers in industrial production. If it is too high, it may cause explosion. For example, Fang and Liang (2019) put forward that when the rated power of electrolyzers drops to 50%, the hydrogen concentration in oxygen will drop to about 2% within 2 hours after the power drops, reaching the upper limit of safe operation of the hydrogen production system. Secondly, when the operating power is too low, the hydrogen production and power generation efficiency will be extremely low. Therefore, the power limits of hydrogen storage equipment is set within 50% to rated power.In addition, this paper constrains the SOC of the device (within 0.2-0.8 for hydrogen storage devices and 0.05-0.95 for super-capacitors), and when the SOC does not meet the constraint, the charge/discharge power is corrected to ensure that the device is within the safe operating range.
When SOC x (k + 1) > SOC x max , the correction formula is as follows.
x can represent hydrogen storage tank or super-capacitor. When SOC is less than the lower limit, the calculation method of the correction power is similar and will not be repeated. According to the above constraints this paper sets up 24 operation modes for the operation of hybrid energy storage system, when P HESS (t) > 0the allocation strategy is shown in Table 2, when P HESS (t) < 0 the principle of allocation is the same as when P HESS (t) > 0. Several of the typical control rules are highlighted.
Mode 1 and Mode 3: When the SOC of super-capacitor is lower than the upper limit and the total power of mixed storage is less than 200kW, the power of electrolytic cell power is set to 0 and the super-capacitor power is the total power of hybrid storage.
Photovoltaic power curve.

Frontiers in Energy Research frontiersin.org 06
Mode 2: When the SOC of hydrogen storage is lower than the upper limit, the SOC of super-capacitor is higher than the upper limit and the total power of mixed storage is less than 200kW, set the power of electrolytic cell to P min and the power of supercapacitor to P HESS − P min .
Mode 5 and 6: When the SOC of hydrogen storage is below the upper limit and the total power of mixed storage is between 200 and 400kW, set the power of electrolytic cell to P HESS , and the power of super capacitor to 0.
Mode 9: When the SOC of hydrogen storage and supercapacitor is lower than the upper limit, and the total power of mixed storage is more than 400kw, set the power of electrolytic cell to P max , and the power of super-capacitor to P HESS − P max .
Mode 10: When only the SOC of hydrogen storage is below the upper limit and the total power of mixed storage is more than 400kw, set the power of electrolytic cell to P max , and the power of super-capacitor to 0. Mode 4,8,12: When the SOC of both hydrogen storage and super-capacitor are over-limited, the power of both the device electrolyzer and super-capacitor is 0.

Example analysis 4.1 Construction of simulation model
According to the control strategy proposed in Section 2, this paper uses dataset two to build a model on MATLAB/ simulink for simulation experiments. The photovoltaic power curve is shown in Figure 4. The charging and discharging efficiency of the super-capacitor is set to

Frontiers in Energy Research
frontiersin.org 95%, the charging and discharging efficiency of the electrolyzer and fuel cell is 80%, the initial SOC of the hybrid storage equipment is all 50%, and the volume of the hydrogen storage tank is 25m 2 . In order to simplify the smoothing process, the photovoltaic output curve is first processed by the smoothing filtering method. The simulation model of the smoothing strategy is shown in Figure 5.

Analysis of smoothing results
4.2.1 Analysis of system-level smoothing fluctuation results using the simultaneous perturbation stochastic approximation algorithm As shown in Figure 6, the simulation curve obtained by the SPSA algorithm with regular control is compared with the photovoltaic power generation. First-order low-pass filtering is an algorithm that can realize the function of hardware RC lowpass filtering by software programming, and it is widely used in practical projects. The principle is to adjust the ratio of the grid power at the previous time to the current time by adjusting the filter coefficient. RFNN is a recursive multilayer connected neural network that uses dynamic fuzzy rules to achieve fuzzy inference with online training and approximation capabilities (Teng and Lee, 2000). To verify the superiority of the algorithm, the smoothing result obtained by the algorithm, the low-pass filtering algorithm and RFNN algorithm are compared in three dimensions: the smoothness of the grid-connected power, the maximum power of the energy storage system and the maximum capacity of the hybrid energy storage system.
The stability of power network is used to measure the fluctuation range of power network. The lower the smoothness, the smaller the fluctuation of the grid power and the closer to the ideal power. The formula for calculating the smoothness is as follows.

FIGURE 6
Comparison between grid-connected power calculated by SPSA and original output.

FIGURE 7
Comparison of grid-connected power calculated by lowpass filtering and original output.

FIGURE 8
Comparison of grid-connected power calculated by RFNN and original output.
Frontiers in Energy Research frontiersin.org 08 where D represents the grid-connected power smoothness and P pv max represents the maximum photovoltaic power.
The greater the maximum charging/discharging power of the energy storage system, the higher the investment cost. The maximum power of the energy storage system is calculated as follows.
P HESS max represents the maximum power of the energy storage system. The maximum capacity of the hybrid energy storage system is calculated as follows.
E HESS (k) represents the total capacity of mixed storage at the kth sampling point, and, E HESS max (k) represents the maximum capacity of the hybrid energy storage device. The results of the first-order low-pass filtering method to eliminate photovoltaic output fluctuation are shown in Figure 7, in which the filter coefficient is 0.7.
The results of smoothing photovoltaic output fluctuations using the RFNN algorithm are shown in Figure 8.
A comparison of the smoothing results of the three algorithms is shown in Table 3.
It can be seen from the table that the SPSA algorithm can greatly improve the smoothness and reduce the gridconnected power fluctuation. Compared with the first-order low-pass filtering algorithm, its maximum power fluctuation is also relatively small, but its storage capacity requirement for energy storage equipment is higher and the high energy density of hydrogen is more desirable. Second, the first-order low-pass filtering algorithm has a large time-delay characteristic because it does not predict the power at the next moment (Shi et al., 2021). Compared with the RFNN algorithm, which has at least four state parameters, its initial value setting is more complicated and easy to fall into local optimum, but the initial value setting of the SPSA algorithm is limited to the distribution coefficient, which is easy to set.

FIGURE 9
Simulation results of SOC.

FIGURE 10
Simulation results of charge and discharge power.
Frontiers in Energy Research frontiersin.org 09

Analysis of device level power distribution results
Set the upper limit and the lower limit of power of electrolytic cell and fuel cell to 400 and 200 kW, respectively. The maximum pressure of the hydrogen storage tank is 10 kPa and the maximum capacity of the super-capacitor is 166 kWh. The SOC simulation results of two kinds of energy storage devices are shown in Figure 8. SOC H is the SOC of hydrogen storage, and SOC c is the SOC of super capacitor. From the figure, it can be seen that the SOC of super-capacitor and hydrogen storage are complementary in some time periods. The simulation results of charging and discharging power are as shown in Figure 9.
It can be seen that the rule control can keep the charging and discharging power of the hydrogen storage device within the range of rated power and safe operation as much as possible, and because the rule is fixed, the hydrogen storage device can avoid frequent start and stop and rapid high-frequency fluctuations mainly caused by super capacitor.

Conclusion
1) The SPSA algorithm with real-time control is used to smooth the fluctuation of photovoltaic power output. Compared with the first-order low-pass filtering algorithm and RFNN algorithm, SPSA algorithm has more remarkable ability to smooth photovoltaic power, and has greater robustness. 2) The power of hybrid energy storage is distributed by the method of rule control, so that the fuel cell and electrolytic cell can work between the rated power and the minimum working power as much as possible, and the SOC of the hybrid energy storage equipment is taken into account at the same time, so that the energy storage equipment can work in a safe range. 3) The model is built and simulated using simulation software, which verifies the effectiveness of the proposed strategy and has some practical significance.

Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.