Abstract
Wind power forecasting (WPF) is imperative to the control and dispatch of the power grid. Firstly, an ultra-short-term prediction method based on multilayer bidirectional gated recurrent unit (Bi-GRU) and fully connected (FC) layer is proposed. The layers of Bi-GRU extract the temporal feature information of wind power and meteorological data, and the FC layer predicts wind power by changing dimensions to match the output vector. Furthermore, a transfer learning (TL) strategy is utilized to establish the prediction model of a target wind farm with fewer data and less training time based on the source wind farm. The proposed method is validated on two wind farms located in China and the results prove its superior prediction performance compared with other approaches.
Introduction
The renewable energy problem is the focus of the 21st century (; ). The transformation of the power grid is the key to solving this problem. The new form of the power grid with renewable energy as the main bulk is the ruling development trend of the future power grid (; ). The Global Wind Energy Development Report 2019 shows that the newly installed capacity of global wind turbines in 2019 is 60.4Ā GW (). However, the uncertainty existing in new energy, such as wind power, is not conducive to the safe and stable operation of the power grid. Therefore, accurate WPF is beneficial for enhancing system reliability ().
There are three types of WPF methods including the physical method, statistical method, and artificial intelligence method. The first establishes a physical model that reflects the relationship between the wind power and numerical weather forecast (NWP) (), which is difficult to model and calculate. Yang proposes an expanded sequence-to-sequence (E-Seq2Seq) based data-driven SCUC expert system for dynamic multiple-sequence mapping samples, which is a pioneer study for SCUC problems (). The second (statistical method) is suitable for wind farms that have been built for a long time because it needs enough historical data. The representative algorithms of this method are Auto-Regression (AR) (; ), Bayesian approach (), and Kalman filter (). The final, AI method, such as support vector machine (), artificial neural network (), extreme learning machine (), can deal with the complex nonlinear relationship between input and output and extract the deep features of input information, which has been widely used in recent years.
The ultra-short-term prediction of wind power is essentially a multi-variable time series prediction problem. In recent years, recurrent neural network (RNN) has developed rapidly. As the improved versions of RNN, long short-term memory (LSTM) network and gated recurrent unit (GRU) (; ) can efficiently extract the temporal correlation characteristics of wind power, and also mine the relationship between power and weather, which improves the performance of WPF. But there is a timing delay in actual prediction.
In addition, all deep learning approaches rely on a sufficient sample of data. However, newly built wind farms may not provide enough data, which makes WPF difficult. However, TL is a new method that breaks through traditional machine learning and is widely used in computer vision, text classification, and other fields (; ; ; ; ). It can finish pre-training of a model in the source domain with sufficient data and then transfer the pre-training model to the target domain after fine-tuning. On the one hand, TL can overcome the problem of few data, on the other hand, it can reduce the training time (; ). At present, there are few studies on the applications of TL in WPF.
In order to improve the prediction performance of RNN, the Bi-GRU method is proposed in this paper to enhance the and capacity and forecasting accuracy of the model by bidirectionality of the structure. The Bi-GRU enables the GRU to process the data in two directions including forward (future) and backward (past). Moreover, the TL strategy is used to forecast the wind power of newly built wind farms with few training data. The TL combined with Bi-GRU is used to ensure the power prediction accuracy and reduce the training time, which can guarantee model performance and reduce computational costs at the same time.
The rest of this paper is organized as follows. In The Proposed BI-GRU Model and Transfer Learning Method, the Bi-GRU model and transfer learning method are explained. Case studies and discussion are shown in Case Studies. Conclusion concludes this study by summarizing the key findings and contributions of this paper.
The Proposed BI-GRU Model and Transfer Learning Method
The Bi-GRU Prediction Model
RNN is widely used in time series prediction, but it has problems of gradients vanishing and exploding, and its memory ability for long series is limited (). As the improved version of RNN, LSTM, and GRU effectively solve these problems and determine the sequential information to be forgotten and remembered through the gating mechanism. The gating mechanism of GRU is simpler than that of LSTM because it combines the forget gate and input gate of LSTM and reduces the computation while ensuring the prediction ability of the neural network. In addition, Bi-GRU is able to extract long-term dependencies before and after the current state, which means that Bi-GRU can extract more temporal features from sequential data, so Bi-GRU performs better than GRU. The structure diagram of Bi-GRU is shown in Figure 1.
FIGURE 1
The GRU cell has only two gates (an update gate and a reset gate ). The update gate controls the extent to which the state information at the previous moment is retained into the current state, and the reset gate determines the extent to which the current state is combined with the previous information. The information flow is shown as follows in a GRU cell.Where xt, ht are the input data and current state (also used as the output of a cell) at time t, respectively. ht-1 is the previous state. is the candidate state. Wr, Ur, Wz, Uz, W, U, and br, bz represent weights and bias parameters, respectively. , tanh are activation functions and denotes an element-wise product. But in Bi-GRU, the output ht is concatenated by the outputs in two directions.Where, , represent the outputs in two directions, Wt, Ut, and bt represent weights and bias parameters, respectively. In addition, FC neural network is used after Bi-GRU to fit the learned features to labels, which means achieving prediction by matching dimensions between inputs and outputs. The Rectified Linear Unit (ReLU) activation function is utilized in the FC layer.
The Transfer Learning Method
The TL method is a machine learning concept that TL is used to improve the performance of target tasks on target domains by transferring the knowledge contained in different but similar source domains (). Usually, the model is pre-trained in the source domain with sufficient data. Then the pre-trained model is fine-tuned in the target domain with small data, which makes full use of the source domain data to improve the performance of the model on the target domain data. TL methods can be divided into instance-based approach, feature-based approach, and parameter-based approach. The historical data of wind farms with short construction time or imperfect detection devices may not be enough to support the training of prediction models. In this paper, the parameter-based TL approach is used. The pre-training model trained by wind farms with sufficient data is fine-tuned by the target domain with insufficient data to accomplish the target tasks more efficiently. The basic idea of transfer learning can be expressed as follows.Where DSs, DSt represent the data space of the source domain and target domain, respectively. Fs, Ls are the features of the source domain and target domain data spaces, respectively. Ft, Lt are the labels of the source domain and target domain data spaces, respectively. The tasks of the source domain and target domain are to find the optimal parameters Ws and Wt, to make that the predicted values Ps and Pt are as close as possible to the labels Ls and Lt. TL is to fine-tune the source domain model parameter Ws to make the target domain parameter as close as possible to the optimal target domain parameter Wt.
The prediction framework diagram of the method proposed in this paper is shown in Figure 1, and the processing flowchart is shown in Figure 2. The prediction process is mainly divided into two parts. In the first part, the wind farm power prediction model in the source domain is established. In the data pre-processing stage, the original data in the source domain are normalized to eliminate the scale difference of features and facilitate the use of gradient descent of loss function. First, the pre-processed data is fed into the three-layer Bi-GRU neural network. Then the FC layer matches the output dimension to achieve WPF to get the source-domain prediction results. The second part is to build the wind farm power prediction model in the target domain, and the data pre-processing is the same as the first part. The pre-trained source domain model is loaded and the parameters in the pre-trained model are transferred to the target domain as the initial parameters. Using a small amount of target-domain data to train the network, a fine-tuned target domain prediction model is obtained.
FIGURE 2
Case Studies
In order to verify the effectiveness and superiority of the proposed prediction model and TL method, the experiment is divided into two parts. The first part compares the Bi-GRU with the AR, LSTM, and GRU. The second part uses the Bi-GRU prediction model and TL method to predict the power of wind farms in the target domain. The programming language used is Python3.8. The deep learning framework is PyTorch1.8.1.
Data Description
Two wind farms from Zhejiang Province in China are named ZJFD01 and ZJFD02 respectively. Each wind farm contains measured active power and meteorological data. The meteorological data contains wind speed, direction measured (sine and cosine of wind direction) at the hub, and air density. The time interval is 15Ā min. Since the running time of the two wind farms is different, the amount of historical data recorded is different. The wind farm ZJFD01 has recorded a large amount of data (including July 1, 2019āAugust 30, 2021) with an installed capacity of 90Ā MW, which is taken as the source-domain wind farm. The wind farm ZJFD02 (including January 1, 2021āAugust 25, 2021) has recorded a small amount of data with an installed capacity of 200Ā MW, which is taken as the target-domain wind farm. The relationship between input and output of samples in the target domain and source domain in these datasets is similar because the relationship of wind power and meteorological variables in different wind farms is semblable. Therefore, data domains can be positively transferred.
Evaluation Metrics
In order to evaluate the prediction performance of the prediction model, the root mean square error (RMSE), mean absolute error (MAE), and accuracy (Cr) are taken as evaluation metrics according to international standards. They are defined as follows. In addition, training time is introduced as a new evaluation index in the experiment of the target-domain wind farm.Where Preal,i, Ppred,i, and Ci are real output power, predicted output power, and capability of wind farm respectively. n is the total number of predicted samples.
The Experiment of Source-Domain Wind Farm
The source-domain prediction model, the Bi-GRU method, is established for the ZJFD01 wind farm. In the data pre-processing stage, the supervised learning dataset is constructed. The output power of the current time step yt is selected as the label. The previous four timesteps (xt-1,xt-2, ⦠, xt-4) are selected as features. A total of 70% of the dataset was used as the training set and the last 30% as the verification set.
For hyperparameters, set input size to 4, hidden size to 8, and the number of layers to 3. Then, the FC layer is connected to tag dimension matching, and the number of neuron nodes in the input layer is 64 (since the output of Bi-GRU is flattened), the hidden layer is 32, and the output layer is 1. In the model training stage, mean-squared loss is used as the loss function to measure the error between predicted power and actual output power, and the Adam optimization algorithm is used as the optimizer. In order to evaluate the superiority of the proposed method in wind farm prediction in the source domain, RMSE, MAE, and Cr are used as evaluation metrics.
The 400 sampling points of the test dataset are taken to verify the prediction effects of various methods. The power prediction results are shown in Figure 3. Compared with other methods, the power prediction curve of Bi-GRU is closer to the actual power output curve trend. As can be seen from Figure 3, the RMSE and MAE of the proposed method are significantly lower than those of other methods, and the accuracy is improved. Compared with LSTM and GRU, RMSE and MAE are reduced by 4.73 and 3.17% respectively. The prediction effect of GRU is better than that of LSTM because the same iteration times are set, but GRU has a simpler structure and fewer parameters to be optimized, so it has higher accuracy. There are two reasons why the proposed method is superior: 1) The Bi-GRU can excavate the relationship between historical meteorological data and current power data layer by layer through various gating mechanisms, and can also excavate the local and long-term correlation before and after the power data series; 2) The characteristics of both the forward and reverse time sequence of power and meteorological data are taken into account by the bidirectional mechanism, so it can effectively improve the accuracy of prediction. As seen from the local amplification figure, the forecasting curve trend of all methods is close to the actual power curve, but there are different levels of phase difference. However, the bidirectional mechanism of Bi-GRU solved this problem, making the prediction curve more closely fit the actual power curve, which is the important reason for its better prediction performance.
FIGURE 3
The Experiment of Target-Domain Wind Farm
In order to ensure the prediction accuracy and reduce the training time, the power prediction of the ZJFD02 wind farm in the target domain is based on transfer learning. The parameters and structure of the pre-trained model from ZJFD01 are migrated to the ZJFD02. The preprocessing method of the dataset is the same as that of source wind farm. In order to explore and verify the advantages of using transfer learning to predict power, the following cases are compared:
a) The pre-trained model in the source domain is directly loaded, and denoted as NO_fine-tunning (NO_FT);
b) The pre-trained model in the source domain is loaded and the parameters in Bi-GRU layers are frozen and the parameters of the FC layer are fine-tuned with target-domain data, which is named Fixed_Bi-GRU;
c) The pre-trained model in the source domain is loaded and the parameters in the FC layer are frozen and the parameters of Bi-GRU layers are fine-tuned with target-domain data, which is named Fixed_FC;
d) Redefine a prediction model whose structure is the same as that of the source-domain model but whose parameters are not trained at all. Then train it with target-domain data, which is named NO_TL.
In addition to RMSE and MAE, training time is added to the evaluation metrics to measure the improvement of computing speed caused by TL. Except for case (a), the number of iterations in other cases is set to 200.
By taking 400 sampling points, the prediction results and performance of the above cases can be compared, as shown in Figure 4. From the perspective of prediction accuracy, the prediction accuracy of case (a) and case (b) is lower than that of case (c) and case (d). RMSE and MAE of case (c) are 4.705 and 4.607%, respectively, lower than that of case (a), because there are still differences in the dataset of the source domain and target domain. If there is no parameter fine-tuning, it will cause a large prediction error. Case (b) and (c) fixed different parameters of the network layer, RMSE, and MAE were reduced by 1.533 and 1.404% respectively compared with case (b), because the number of three-layer network parameters of Bi-GRU was much more than that of FC layer. After fine-tuning in the target domain, case (b) changed the parameters of the model to a greater extent than case (b), so it is closer to the optimal target domain model; the prediction accuracy of case (c) and case (d) was similar, RMSE is 2.397 and 2.484%, MAE is 1.295%, and 1.298%, respectively. From the perspective of time-consuming, cases (b), (c), and (d) are compared. it is obvious that the training time of case (b) is less than that of (c). Most parameters of this prediction model are still Bi-GRU layers, so it saves training time to fine-tune FC layer parameters. The accuracy of (c) is similar to that of (d), but the training time of (c) is 9.9% shorter than that of (d). Therefore, using the transfer learning fine-tuning the pre-trained model can guarantee the prediction accuracy and save the training time to a certain extent compared with the training model starting from the beginning.
FIGURE 4
Conclusion
In this paper, a Bi-GRU prediction model based on the transfer learning method is presented for the ultra-short-term of wind power. According to the results of case studies, some conclusions are summed up as follows. On one hand, the Bi-GRU prediction model can extract the temporal features of wind power sequential data in two directions, which learns deeper historical information and realize higher accuracy of WPF than GRU and LSTM. On the other hand, the prediction model combined with the TL method saves training time and reduces the requirement for abundant data. In the future, more detailed research about how to balance training time and accuracy of prediction using TL will be completed. Moreover, more comprehensive evaluation metrics aimed at evaluating the TL method in WPF will be established ().
Statements
Data availability statement
The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.
Author contributions
WC: Data provision, Project administration, Methodology. WQ: Data Processing, Project administration, Supervision. YL: Methodology, Data Processing, Writing- Original draft preparation, Software. JZ: Conceptualization, Writing- Reviewing and Editing, Supervision. FZ: Validation, Investigation, Supervision. DX: Visualization, Supervision. WR: Investigation, Supervision. GL: Data Processing, Supervision. MS: Data Processing, Supervision. FT: Supervision, Writing-Reviewing and revising grammar and correct expression.
Funding
This study received funding from the Science and Technology Project of State Grid Zhejiang Electric Power Co., LTD. (5211SX2000ZM).
Conflict of interest
Author WC and JZ is employed by State Grid Zhejiang Electric Power Company, Ltd. Author WQ, FZ, DX, WR, GL, and MS are employed by State Grid Shaoxing Power Supply Company. This study received funding from the Science and Technology Project of State Grid Zhejiang Electric Power Co., LTD (5211SX2000ZM). The funder had the following involvement with the study: data collection and analysis.
The remaining author declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisherās note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1
AliM.PrasadR. (2019). Significant Wave Height Forecasting via an Extreme Learning Machine Model Integrated with Improved Complete Ensemble Empirical Mode Decomposition. Renew. Sustain. Energ. Rev.104, 281ā295. 10.1016/j.rser.2019.01.014
2
ChenX.ZhangX.DongM.HuangL.GuoY.HeS. (2021). Deep Learning-Based Prediction of Wind Power for Multi-Turbines in a Wind Farm. Front. Energ. Res.9, 723775. 10.3389/fenrg.2021.723775
3
DeoR. C.WenX.QiF. (2016). A Wavelet-Coupled Support Vector Machine Model for Forecasting Global Incident Solar Radiation Using Limited Meteorological Dataset. Appl. Energ.168, 568ā593. 10.1016/j.apenergy.2016.01.130
4
LiZ.JiangW.Abu-SiadaA.LiZ.XuY.LiuS. (2020). Research on a Composite Voltage and Current Measurement Device for HVDC Networks. IEEE Trans. Ind. Electron.68 (9), 8930ā8941. 10.1109/TIE.2020.3013772
5
LiZ.YeL.ZhaoY.SongX.TengJ.JinJ. (2016). Short-term Wind Power Prediction Based on Extreme Learning Machine with Error Correction. Prot. Control. Mod. Power Syst.1 (1), 1. 10.1186/s41601-016-0016-y
6
LinZ.LiuX. (2020). Wind Power Forecasting of an Offshore Wind Turbine Based on High-Frequency SCADA Data and Deep Learning Neural Network. Energy201, 117693. 10.1016/j.energy.2020.117693
7
LiuX.LinZ.FengZ. (2021). Short-term Offshore Wind Speed Forecast by Seasonal ARIMA - A Comparison against GRU and LSTM. Energy227, 120492. 10.1016/J.ENERGY.2021.120492
8
QureshiA. S.KhanA.ZameerA.UsmanA. (2017). Wind Power Prediction Using Deep Neural Network Based Meta Regression and Transfer Learning. Appl. Soft Comput.58, 742ā755. 10.1016/j.asoc.2017.05.031
9
ShenX.OuyangT.KhajorntraidetC.LiY.LiS.ZhuangJ. (2021a). Mixture Density Networks-Based Knock Simulator. Ieee/asme Trans. Mechatron., 1. 10.1109/TMECH.2021.3059775
10
ShenX.OuyangT.YangN.ZhuangJ. (2021b). Sample-based Neural Approximation Approach for Probabilistic Constrained Programs. IEEE Trans. Neural Netw. Learn. Syst.1, 8. 10.1109/TNNLS.2021.3102323
11
ShenX.RaksincharoensakP. (2021a). Pedestrian-aware Statistical Risk Assessment. IEEE Trans. Intell. Transport. Syst., 1ā9. 10.1109/TITS.2021.3074522
12
ShenX.RaksincharoensakP. (2021b). Statistical Models of Near-Accident Event and Pedestrian Behavior at Non-signalized Intersections. J. Appl. Stat.1, 21. 10.1080/02664763.2021.1962263
13
ShiJ.DingZ.LeeW.-J.YangY.LiuY.ZhangM. (2014). Hybrid Forecasting Model for Very-Short Term Wind Power Forecasting Based on Grey Relational Analysis and Wind Speed Distribution Features. IEEE Trans. Smart Grid.5 (1), 521ā526. 10.1109/tsg.2013.2283269
14
WangJ.ZhangN.LuH. (2019a). A Novel System Based on Neural Networks with Linear Combination Framework for Wind Speed Forecasting. Energ. Convers. Manage.181, 425ā442. 10.1016/j.enconman.2018.12.020
15
WangY.WangH.SrinivasanD.HuQ. (2019b). Robust Functional Regression for Wind Speed Forecasting Based on Sparse Bayesian Learning. Renew. Energ.132, 43ā60. 10.1016/j.renene.2018.07.083
16
WangZ.ZhangJ.ZhangY.HuangC.WangL. (2020). Short-term Wind Speed Forecasting Based on Information of Neighboring Wind Farms. IEEE Access8, 16760ā16770. 10.1109/access.2020.2966268
17
WuB.SongM.ChenK.HeZ.ZhangX. (2014). Wind Power Prediction System for Wind Farm Based on Auto Regressive Statistical Model and Physical Model. J. Renew. Sustain. Energ.6 (1), 013101. 10.1063/1.4861063
18
YangD. (2019). On post-processing Day-Ahead NWP Forecasts Using Kalman Filtering. Solar Energy182, 179ā181. 10.1016/j.solener.2019.02.044
19
YangN.HuangY.HouD.LiuS.YeD.DongB.et al (2019). Adaptive Nonparametric Kernel Density Estimation Approach for Joint Probability Density Function Modeling of Multiple Wind Farms. Energies12 (7), 1356. 10.3390/en12071356
20
YangN.LiuS.DengY.XingC. (2021a). An Improved Robust SCUC Approach Considering Multiple Uncertainty and Correlation. IEEJ Trans. Elec Electron. Eng.16 (1), 21ā34. 10.1002/tee.23265
21
YangN.YangC.WuL.ShenX.JiaJ.LiZ.et al (2021b). Intelligent Data-Driven Decision-Making Method for Dynamic Multi-Sequence: An E-Seq2Seq Based SCUC Expert System. IEEE Trans. Ind. Inf., 1. 10.1109/TII.2021.3107406
22
YangN.YangC.XingC.YeD.JiaJ.ChenD.et al (2021c). Deep Learningābased SCUC Decisionāmaking: An Intelligent Dataādriven Approach with Selfālearning Capabilities. IET Gener. Transm. Distrib.. 10.1049/gtd2.12315
23
ZhangL.XieY.YeJ.XueT.ChengJ.LiZ.et al (2021). Intelligent Frequency Control Strategy Based on Reinforcement Learning of Multi-Objective Collaborative Reward Function. Front. Energ. Res., 1ā12. 10.3389/fenrg.2021.760525
24
ZhaoX.LiuJ.YuD.ChangJ. (2018). One-day-ahead Probabilistic Wind Speed Forecast Based on Optimized Numerical Weather Prediction Data. Energ. Convers. Manage.164, 560ā569. 10.1016/j.enconman.2018.03.030
25
ZhengD.EseyeA. T.ZhangJ.LiH. (2017). Short-term Wind Power Forecasting Using a Double-Stage Hierarchical ANFIS Approach for Energy Management in Microgrids. Prot. Control. Mod. Power Syst.2 (1), 13. 10.1186/s41601-017-0041-5
26
ZhuangF.QiZ.DuanK.XiD.ZhuY.ZhuH.et al (2021). A Comprehensive Survey on Transfer Learning. Proc. IEEE109 (1), 43ā76. 10.1109/JPROC.2020.3004555
Summary
Keywords
bidirectional gated recurrent unit, transfer learning, target domain, wind power, wind power forecasting
Citation
Chen W, Qi W, Li Y, Zhang J, Zhu F, Xie D, Ru W, Luo G, Song M and Tang F (2021) Ultra-Short-Term Wind Power Prediction Based on Bidirectional Gated Recurrent Unit and Transfer Learning. Front. Energy Res. 9:808116. doi: 10.3389/fenrg.2021.808116
Received
03 November 2021
Accepted
12 November 2021
Published
17 December 2021
Volume
9 - 2021
Edited by
Xun Shen, Tokyo Institute of Technology, Japan
Reviewed by
Aihong Tang, Wuhan University of Technology, China
Yunyun Xie, Nanjing University of Science and Technology, China
Updates
Copyright
Ā© 2021 Chen, Qi, Li, Zhang, Zhu, Xie, Ru, Luo, Song and Tang.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Yu Li, 2877630621@qq.com
This article was submitted to Smart Grids, a section of the journal Frontiers in Energy Research
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.