The common sampling-based generative models have three types: Variational Auto-encoder (VAE), Generative Adversarial Network (GAN), and Flow-Based Models (Zhang et al., 2021). VAE can naturally generate low-dimensional space, approximate the sample distribution, and generate new samples easily. However, the gradient is difficult to calculate in the VAE model, and it can only optimize the lower boundary line of the edge likelihood function. The performance is generally inferior to that of GAN. GAN is good at generating samples, sampling to get new samples is easy, but the training of the model is difficult and the stability is poor. GAN cannot be used to get the sample distribution, and it is easy to have model collapse. Compared with the above two models, Flow-based Models can get the sample distribution and also the useful hidden space, and the training is very easy.

Flow Based Models are probabilistic generative models based on reversible transformations in which both sampling and density evaluation can be valid and accurate. The probabilistic generative model is a model used to learn the distribution of random variable X from a pile of sample data x i i = 1 N .

There are real sample data X and random variable Z, where Z obeys the known simple prior distribution π(z), and the sample data X obeys the complex distribution p(x). If there is a transformation function f that satisfies the mapping from Z to X. f : Z → X

Every sampling point in π z have a corresponding sample point in p x in order to obtain generated sample. Although most distributions in the real world are more complex than Gaussian distributions, Gaussian distributions are often used in latent variable generation models to calculate derivatives more easily and effectively.

The flow model generation process can be defined by the following formula (Dinh et al., 2014): z ∼ π z x = g θ z

In the above equation, z is the hidden variable; π z is the sample distribution of the hidden variable z; g θ is an invertible function, so the hidden variable z can be interpreted as z = f θ x = g θ − 1 x , where f θ consists of a series of transformed functions: f θ = f 1 ∘ f 2 ∘ … ∘ f K . The relationship between x and z can be expressed as: x → f 1 h 1 → f 2 h 2 ⋯ → f K z

This sequence of reversible transformations is known as the Normalizing Flow (Rezende and Mohamed, 2015). In Normalizing Flow, the prior distribution π z of random variables Z usually chooses Gaussian distribution, so that the model can be used for better and more powerful distribution approximation. Normalizing Flow can transform simple distribution into complex distribution by using a series of reversible transformation functions, and then the new variables are repeatedly replaced based on the variable substitution theorem, and finally we can get the probability distribution of the target variable. From the above equation, the model probability density function for a given sample data x can be expressed as: log ⁡ p θ x = ⁡ log ⁡ p θ z + ⁡ log det d z d x = ⁡ log ⁡ p θ z + ∑ i = 1 K log det d H i d H i − 1

It is easier to obtain the exact log-likelihood of the input through the Normalizing Flow. The training loss function of the generation model is the negative log-likelihood on the input data set: L = − 1 D ∑ i = 1 D log ⁡ p X i

The generator of the flow model is subject to some mathematical constraints, which leads to its own expression ability may be insufficient. Therefore, it is generally necessary to stack multi-layer networks to get a generator, which is also the origin of the name ' flow model '. Because it requires the same dimension of input and output, the calculation is very large.

The Glow model is a simple generative normalized flow model using reversible 1 × 1 convolution, improving on NICE (Dinh et al., 2014) and Real NVP (Dinh et al., 2016). The Glow model consists of a series of repetitive layers named scale. Each scale consists of a squeeze function and a flow step, and each flow step contains three parts, Activation Normalization Layer (ActNorm Layer), 1 × 1 Convolution Layer and Affine Coupling Layer (van de Schaft and van Sloun, 2021), and the flow step is followed by a splitting function. Figure 1 shows one step of Glow.

Figure 2 shows the multi-scale structure. The splitting function divides the input into two equal parts in the channel dimension. One-half goes to the subsequent layer and the other half goes to the loss function. The splitting is done to reduce the effect of gradient disappearance, which occurs when the model is trained in an end-to-end mode.

The ActNorm Layer is used for activation normalization, similar to batch normalization, which uses the scale and deviation parameters of each channel to affine transform the activation. Initializing these parameters makes the posterior behavioral actions of each channel have zero mean and unit variance for a given initial small batch of data. This is a form of the data-dependent initialization (Salimans and Kingma, 2016). After initialization, the scales and biases are considered as regular trainable parameters independent of the data.

The 1 × 1 invertible convolution layer is used to invert the ordering of the channels, where the weight matrix is initialized to a random rotation matrix and the number of input and output channels of the convolution layer is the same. The overall computation can be simplified by the simplified computation of the matrix.

The affine coupling layer creates a flexible and easy-to-handle bijection function by accumulating a series of simple bijections. In each simple bijection, some input vector is updated by utilizing a simple inverse function, but it rests with the other input vector in a complex way. The affine coupling layer can be divided into three parts: Zero initialization, Split and concatenation, and Permutation.

Compared to other deep learning generative models, the Glow model has the following advantages:

(1) Exact potential variable inference and log-likelihood evaluation. In reversible generative models like Glow, the exact inference of potential variables can be achieved without approximation, and the exact log-likelihood of the data can also be optimized.

The wind generator bearing mainly consists of outer ring, inner ring, rolling element and cage. One end of the bearing is connected to the wind turbine blade, and the other end is connected to the wind turbine drive system. The inner ring is connected to the shaft, and the outer ring is connected to the cage. The rolling element is the key component of the bearing rotation. Therefore, the inner ring, outer ring and sphere may fail. During operation, the main bearing area is easily affected by external factors and causes failure. This experiment data is from of the bearing data center of Case Western Reserve University (CWRU) (Loparo, 2012). The experimental sample set was selected from the drive end and fan end bearing failure data at 12 K sampling frequency with a motor load of 2 hp and an approximate motor speed of about 1750 r/min, and the collected data contained fan and drive end vibration data. The faults were classified into 19 categories according to the fault category (normal, fan end fault, drive end fault), fault location (inner ring, rolling element, outer ring) and damage diameter (0.007 inches, 0.014 inches, 0.021 inches), as shown in Table 1.

Because the design of one-dimensional convolution network is difficult and easy to lead to over-fitting (Xiao et al., 2019), this paper chooses to convert the one-dimensional timing signals from the fan end and the drive end of the wind turbine bearing into twodimensional image signals for fault sample generation and fault diagnosis through a two-dimensional convolutional network, respectively. Based on the bearing rotation speed and sampling frequency, approximately 411 points can be sampled for each rotation of the bearing. In order to ensure the integrity of information and the effectiveness of fault features and to mine more detailed features, 784 sampling points in about 2 rotation cycles are selected as the pixel points of the 2D grayscale image with the image format of 28 × 28. The original 1D time-domain signal needs to be normalized before the sample construction. The data processing is shown in Figure 3.

For the purpose of realizing multi-category bearing fault diagnosis, this paper presents a new fault diagnosis model for wind turbine bearings with multiple image inputs based on convolutional neural networks. The convolution layer of the convolutional neural network can extract the local part of the image, which greatly improves the accuracy of the model. For adding some noise to the data set or disturbing the input to a certain extent, the convolutional neural network can still perform good recognition and analysis. Moreover, the convolutional neural network can effectively reduce the over-fitting of the model, thereby improving the generalization ability of the model. Convolutional neural network has the advantage of fast convergence of back propagation algorithm, and can train the depth model in a short time. In general, convolutional neural networks have many advantages. For image, audio and other fields, the advantages of convolutional neural networks are more obvious. However, convolutional neural networks also have their limitations, such as large amount of model calculation, complex training data, etc., so in practical applications, it is necessary to select the appropriate model according to the actual situation.

The single input fault diagnosis model can only diagnose one position of the wind turbine bearing fault. However, the multi-signal input fault diagnosis model can learn the fault features of multiple image inputs to achieve the function of fault diagnosis. The model can diagnose the faults of multiple positions of the wind turbine bearing according to the vibration data of multiple different positions. In contrast, the multi-signal input fault diagnosis model can help improve the convenience of wind turbine operation and maintenance, effectively save operation and maintenance costs, and improve the economic benefits of wind farms.

Figure 4 depicts the structure of the multi-image input fault diagnosis model. The model input is 2 grayscale images of size 28 × 28, each input is convolved by multiple convolution layers, and the output is connected to a fully connected layer. The model uses a phase multiplication layer to multiply the inputs from two fully connected layers. The model output is a classification layer with softmax activation function.

In this study, the two-dimensional images of vibration signals at the fan end and the drive end of the wind turbine bearing are input into the multi-input fault diagnosis model. Considering different scenarios in the wind generator operation, the fault sample sets under different scenarios are constructed, and the multi-input fault diagnosis model is used to achieve multi-category bearing fault diagnosis. Figure 5.

The process of the multi-signal input fault diagnosis based on Glow model is as follows:

(1) Convert 2 sets of normalized original 1D signals into 2D grayscale image.

The study was complemented by the Glow model to generate some new fault samples in different scenarios. For the purpose of demonstrating the work of the generated model, we used image quality metrics such as maximum mean difference (MMD) (Xu et al., 2018), peak signal-to-noise ratio (PSNR) (Wang et al., 2008), and feature similarity (FSIM) (Zhang et al., 2011) to value the validity of the generated fault samples. MMD can be used to judge the distribution difference between two images by using the size of the summation obtained from each image projection, and the smaller value indicates the smaller image distribution difference. PSNR is based on the relative value of the mean square error of the original sample and the generated sample and the square of the possible maximum signal value of the sample, and a higher value of PSNR indicates a higher quality of the generated sample. FSIM can be used to evaluate the image quality by using feature similarity, and higher values indicate better similarity.

Table 2 shows the quality evaluation of the generated fault samples with different noise intensities. Based on the magnitude of the 3 metrics, the original fault samples generated by the Glow model have better image quality than the generated noisy fault samples. The generated drive end fault samples have better image quality compared to the wind turbine bearing fan end fault samples. In the comparison of different image quality metrics, we find no significant degradation in the generated sample quality in the presence of noise interference. The experimental effect demonstrates that the proposed method has good performance to generate wind turbine bearing fault samples.

The study supplemented the number of wind turbine bearing fault samples in different scenarios by VAE, GAN, and Glow models, respectively. To verify the performance of the generated models, we used the image quality metrics above to evaluate the authenticity of the generated fault samples.

Table 3 shows the evaluation of the quality of the generated fault samples with different noise intensities. According to the magnitude of the three metrics, the Glow model supplemented generated fault samples have smaller differences and distortions compared to the fault samples generated by other models, and the samples are not as similar as several other models. The image quality of the drive-side fault samples generated by each deep learning generation model is better compared to the wind turbine bearing fan end fault samples.

Because the real fault probability is relatively low, the probability of each type of fault of the wind turbine bearing is also different, and the number of samples obtained for each fault category is also different. Therefore, in the actual fault diagnosis of the fan spindle, there will be a problem of unbalanced number of fault category samples. To address this problem, this paper uses Glow model to generate new samples based on 2 sets of wind turbine bearing vibration data respectively to supplement the sample number of unbalanced fault categories, and the fault sample sets with balanced sample number are used to get the diagnostic results through the multiple input bearing fault diagnosis model. The unbalanced fault sample set is shown in Tabel 4, where the sample number of balanced category is 2000 per category and the sample number of unbalanced category is 1,000 per category.

Figure 6 demonstrates the diagnostic results of different sample sets under sample imbalance scenario. When the sample imbalance is relatively low, the model can 100% accurately diagnose the faults. As the sample imbalance increases, the diagnostic effect of the model can decrease slightly. However, the model can still effectively diagnose wind turbine bearing faults, indicating that the new model has good bearing fault diagnosis ability under sample imbalance scenario.

In the actual operation of the wind turbine, the collection of fault samples is very difficult, and the limited number of fault samples may affect the effect of fault diagnosis. For the purpose of confirming the fault diagnosis effect of the new method in complex scenarios, we construct sample sets based on different sample number and supplement the sample number through the Glow model until the sample number in the sample set is the same. The diagnostic effect of the proposed method is tested on the same sample set. The sample number in different sample sets is shown in Table 4.

Figure 7 shows the diagnostic results of the original sample set and the supplemental sample set in the scenario of insufficient samples. As shown in the figure, insufficient samples will lead to a decrease in the fault diagnosis effect of the sample set. The diagnostic result is better when the fault sample set is supplemented by the generative model. Although the diagnostic effect of the sample set with deep learning generation model supplemented with fault samples is not good enough, it also verifies that the new fault diagnosis method has certain effect when the samples are insufficient, and the fault samples generated by Glow model can effectively enhance the diagnostic accuracy.

The study considers that the acquisition of vibration data in the wind generator operation may be disturbed by noise. For the purpose of proving the validity of the proposed fault diagnosis method in practical applications, the study simulates the noise scenarios in the actual operation by adding Gaussian white noise with different intensities to the sample data. The sample sets with different noise intensities are set by the original fault samples and the fault samples with different noise intensities according to Tables 3, 5.

Figure 8 is the diagnostic result of different sample sets under noise interference. When the fault samples are balanced and sufficient, the diagnostic result of the sample set is significantly better than that of other sample sets. When the samples are insufficient and unbalanced, the fault diagnosis effect of the sample set will decrease. The diagnosis accuracy of each sample set under special scenarios is relatively low, but the diagnosis accuracy of each sample set is above 97.5%. The study shows that the proposed model has good performance of the bearing fault diagnosis under the noise scenario.

The combination of deep generative models and fault classification methods has the ability to solve unbalanced sample problems and small sample problems. In order to verify the ability of different deep learning generative models in complex scenarios fault recognition, this section compares the accuracy of fault diagnosis by supplementing the fault samples generated by different deep generative models, trained with the same sample set in the wind turbine bearing multi-signal input model.

Figure 9 shows the fault diagnosis accuracies of different deep generative models on the same sample set. In the unbalanced sample set, the fault diagnosis accuracy of the supplemental sample set of different deep learning generative models is basically the same. However, in the small sample set, the diagnostic accuracy of the fault sample set supplemented by the GLOW model is significantly higher than the diagnostic accuracy of the fault sample set supplemented by other deep learning generation models. In order to more intuitively reflect the superiority of GLOW, we have provided the average fault diagnosis accuracy of different models in the last column of Figure 9. The results show that the average diagnostic accuracy of GLOW model is significantly higher than that of the other two models. The experiments prove that the GLOW model has good performance in solving the unbalanced sample problem and the small sample problem in the multi-input fault diagnosis of wind turbine bearings.