BRIEF RESEARCH REPORT article

Front. Energy Res., 30 August 2022

Sec. Smart Grids

Volume 10 - 2022 | https://doi.org/10.3389/fenrg.2022.1000459

State identification of transfer learning based Yolov4 network for isolation switches used in substations

  • XH

    Xuyong Huang 1

  • BT

    Biao Tang 1*

  • MZ

    Mengmeng Zhu 1

  • LS

    Long Shen 1

  • YM

    Yutang Ma 1

  • XW

    Xin Wang 2

  • ZZ

    Zhou Zhang 3

  • DX

    Dengyu Xiao 3

  • 1. Electric Power Research Institute, Yunnan Power Grid Co., Ltd., China Southern Power Grid, Kunming, China

  • 2. Yunnan Power Grid Co., Ltd., China Southern Power Grid, Kunming, China

  • 3. Qujing Power Supply Bureau, Yunnan Power Grid Co., Ltd., China Southern Power Grid, Qujing, China

Abstract

With the steady progress of the intelligent development of power systems, as well as the higher demand for power supply reliability. It is essential to achieve the effective monitoring of substations 24 h a day. The vigorous development of deep learning network brings strong theoretical and technical support to the unmanned and intelligent construction of the substation. To identify the on/off state of the isolation switch in the substation robot inspection image, this paper proposes a method for identifying the isolation switch state of YOLOv4 (You Only Look Once V4) network based on transfer learning. Firstly, for the insufficient number of samples, transfer learning is introduced, and the network feature extraction layer is pre-trained by using public data sets. Secondly, images of isolation switch are obtained by a fixed camera and inspection robot in the substation, and data set of isolation switch is constructed. Finally, the isolation switch data set is used to train the YOLOv4 network. The test results show that compared with YOLOv3 and YOLOv4, the network can improve the identification precision of the isolation switch.

1 Introduction

As the social demand for electricity increases year by year, the reliable operation of power grid is faced with higher requirements (; ; ; ). Meanwhile, the construction of smart power grid further promotes the intelligent construction of substation inspection. The isolation switch is an important part of power equipment in the substation. It will be helpful to the stable operation of the substation to know the working state of the isolation switch in real time (; ; ). At present, most substations often use manual inspection, which is time-consuming and laborious, unable to realize real-time observation, and the observation process is easy to be affected by the subjectivity of inspection personnel. In addition, manual inspection is difficult due to the influence of ice disasters, rain and fog in some remote areas. Therefore, the monitoring and identification of typical power equipment in a substation has become an essential link in the process of intelligent substation construction. Accurate detection and identification of electrical equipment through deep learning technology will contribute to future research of unmanned substation monitoring and fault diagnosis (; ).

With the continuous advance and maturation of deep learning technology, it has been paid attention to by many scholars in the construction of the smart substation. The operation and maintenance of electrical equipment in the smart substation are being transitioned from traditional preventive maintenance based on time cycle to more targeted state maintenance mode (). Because of the particularity and confidentiality of the power industry, the different image resolution, complex scenes, multiple interference sources and other reasons for unmanned aerial vehicle and ground cameras, there is no good public data set for power equipment images, so few research are used on recognition of electrical equipment based on deep learning model. In recent years, computer vision based on deep learning and image processing and recognition has developed by leaps and bounds and has been widely used in the inspection system. In feature extraction, the current mainstream convolutional neural network has been able to provide better representation ability and alleviate gradient disappearance (; ). Therefore, the state identification technology of the deep learning network model lays a solid foundation for the electrical equipment identification research of substations.

At present, image recognition methods of electrical equipment based on deep learning can be divided into: One is the dual-order method, firstly, a deep convolution neural network is constructed to extract the feature of the target, and then the traditional image segmentation and location method is used to complete the target detection, including scale-invariant feature transform, SIFT), Fast Region-Convolution Neural Network (RCNN), Faster-RCNN, etc. The other is the single-order method, which integrates feature extraction, border location and target classification into a network to build an end-to-end training mode, to effectively reduce the repeated calculation in the process of image feature extraction, including You Only Look Once (YOLO) and Single ShotMultiBox Detector (SSD) and so on (; ). Based on YOLOv4, reference () improved the Mosaic data expansion algorithm, and studied its parameter tuning method, so as to achieve the recognition of major electrical equipment. In reference (), pretreatment infrared imaging data such as random rotation angle, saturation and exposure were studied, and a fault detection method of insulator infrared image based on YOLOv3 algorithm was proposed. In reference (), the Hough transform was used to detect the edge of transmission lines, and the thickness of the ice was calculated by comparing the thickness difference before and after icing. Reference () studied equipment identification and state detection tasks in the power room through Faster R-CNN. In reference (; ), YOLOv4 and Convolution Neural Network (CNN) were respectively used to locate and diagnose the fault heating area of the equipment through infrared image data of electrical equipment. Most of the above studies are mainly devoted to the monitoring of the fault state of power equipment, and there is little recognition of the working state of power equipment.

Therefore, based on the picture of the isolation switch of a 220 kV substation in southwest China, this paper adopts YOLOv4 deep-learning network model to achieve the identification of the working state of the isolation switch. Meanwhile, for the problem of insufficient sample setting, this paper introduced transfer learning to preprocess the network feature extraction layer, so that it can improve the identification precision.

The content of this paper is as follows: The second section introduces the YOLOv4 network model based on transfer learning, including the framework of the network model, activation function and loss function, etc. The third section carries on the calculation example analysis, including the experiment environment and the evaluation system, the realization result and so on. The fourth section gives the conclusion of this paper.

2 YOLOv4 network model based on transfer learning

2.1 YOLOv4 network

YOLO series model is an object recognition and location model based on a deep neural network. Because of its fast running speed, YOLO model is mostly used in real-time systems. YOLOv4 is a newer version of the YOLO series models proposed by Alexey Bochkovskiy et al., in 2020 (; ). Compared with YOLOv3 model, YOLOv4 developed an efficient and powerful small target detection model by optimizing the trunk network, network training, activation function and other aspects combined with the previous series of models, which has a great improvement in speed and accuracy (). The YOLOv4 network model is shown in Figure 1 (; ; ; ; ).

FIGURE 1

2.1.1 Network structure of YOLOv4

The backbone network is CSPDarkNet53, which is used to extract object features. The input image is processed by residual blocks, which can make the model lighter and ensure detection accuracy.

The receptive field of YOLOv4 can be enlarged by using Spatial Pyramid Pooling (SPP), as a variety of characteristics will be obtained by using three pools of different proportions. Path Aggregation Network (PANet) can solve the multi-scale problem in target detection. After the output of the PANet, the header module converts the features into predicted results to obtain the target and its type.

2.1.2 Activation function

Activation function is an important part of deep learning, which has an important influence on target detection. Activation functions are key to achieving high performance in a wide range of tasks. In general, activation functions can be divided into two main categories. One is the saturation activation function; The other is the non-saturated activation function. Saturation activation functions mainly include sigmoid and tanh function, and non-saturated functions include Leaky ReLU, Mish and Swish function and so on. Compared with the saturation function, the advantage of the non-saturated function is that it can be used to solve the problem of “gradient disappearance”, and the convergence speed of the unsaturated function is faster.

The swish function has attracted much attention due to its good performance on various challenging data sets under popular deep learning libraries. The non-monotone nature of the swish function distinguishes it from other common activation functions. In addition, the smoothness of the swish function is beneficial to model generation. The swish function is shown in Eq. 1 below, and the curve of the swish function is given in Figure 2B.where represents sigmoid function.

FIGURE 2

The YOLOv4 network model mainly uses mish activation functions. Meanwhile, mish functions are favoured by YOLOv4 because of their advantages of low cost, non-monotonic, upper and lower limits to ensure continuity of information and improve performance. The mish functions are shown in Eq. 2 and Figure 2A.

2.1.3 Loss function

The loss function is an operational function to calculate the difference between the predicted value

f

(x) and the true value

y

of the model. It represents by

L

(

y

,

f

(x)). The size of the loss function is related to the robustness of the model.

  • a) General loss function

The loss function often appears in the training process of model. After the trained data enters the model, the predicted value is output by forwarding propagation. Then, the difference between the

f

(x) and the

y

calculated by loss function is the loss value. Finally, the model updates the parameters by back propagation to make the predicted values as close to the real values as possible.

  • b) CIoU-loss

The loss function of YOLOv4 mainly cares about three factors: overlapping area, centre point distance and aspect ratio, as follows:where v resprents the similarity of aspect ratio, c resprents the diagonal distance in the figure below, α resprents the weight parameter.

2.2 Transfer learning

The isolation switch image set is small due to the confidentiality of the substation and other scenes in the field of power system, as well as the differences in equipment in different levels of the substation (). Therefore, because of the insufficient number of samples in the training process, this paper introduces transfer learning, uses imagenet and other public data sets to pre-train the network feature extraction layer, and migrates it to power equipment identification through model fine-tuning ().

As a special machine learning technology, transfer learning has attracted extensive attention in many research fields with the increasing demand for accuracy, data scale and efficiency of target task (; ). Inspired by human knowledge transfer ability, the purpose of transfer learning is to minimize the required training set size, as shown in Figure 3 (). Transfer learning has a strong adaptive ability. It can process other complex tasks by using labelled data in the existing related task domain, try to learn the source task in the source domain, apply the knowledge obtained to the target domain, and solve the task in the target domain.

FIGURE 3

Transfer learning mainly includes domain and task. Tasks are the object of learning and are mainly composed of labels and corresponding functions of labels. In particular, the transfer learning process corresponds to the source domain and the target domain. The transfer process is the transfer of knowledge from source to target ().

Transfer learning only needs to use the trained model to extract primary features and train the last few layers of neurons to recognize the on-off state of the isolation switch. Transfer learning can effectively reduce the training time and improve the precision of identification. Firstly, imagenet and other public data sets are used to pre-train the network feature extraction layer. Then, the features of the source domain image are transferred to the isolation switch image, the common features of the two kinds of images are compared and analyzed, and the parameters of the source domain model are transferred to the model. Finally, the model is trained according to the isolation switch data set.

2.3 Improved YOLOv4 network framework

According to the above theoretical knowledge, the YOLOv4 network model framework based on transfer learning is shown in Figure 4.

FIGURE 4

3 Results and discussion

3.1 Experimental platform and computer configuration

This paper adopts high-definition image data of a 220 kV substation of China Southern Power Grid, including 1,331 high-definition images of isolation switches. The pictures of the unmanned inspection robot and some pictures of the isolation switches are shown in Figure 5. The data set is divided into the training set and test set in a ratio of 9:1 for network training and testing. For the preprocessing of the data set, lableimg annotation training set is used to generate. xml file containing the status, size and location information of the isolation switch. All the training environments in this paper are Intel(R) Core (TM) i5-10400F @ 2.9 GHz, NVIDIA RTX 3080 Ti GPU, and PyTorch is used to build the mathematical experiment platform.

FIGURE 5

3.2 Evaluation indicators

To objectively evaluate the advantages and disadvantages of the recognition algorithm proposed in this paper, the most representative precision (Pre), recall (Rec), Average precision (AP) of each type of target, harmonic mean (F1) and mean average precision (mAP) were selected as the evaluation indexes of the method. Among them, Pre is used to measure the precision of the model to find samples; Rec is used to measure the ability of the algorithm to find a certain type of sample in the data set. The calculation formula is as follows:where, Tp: The number of samples correctly identified as positive, FP: The number of samples incorrectly identified as positive, and FN: The number of samples wrongly identified as negative.

AP value is defined as the area enclosed by Pre-Rec curve and coordinate axis, which also measures Pre and Rec of the network model in detecting a certain type of target. The calculation formula of AP is shown in Eq. 8. The value of AP reflects the detection effect of the network model.

In addition, mAP and F1 are widely accepted evaluation indicators. The larger the mAP value is, the better the overall detection effect of the model is. F1 represents the average harmonic value of Pre and Rec. The closer F1 is to 1, the better the optimization effect of the network model is. The calculation formulas for mAP and F1 are as follows.where n is the number of categories.

3.3 Experimental results

To improve the identification accuracy of isolation switches in substations, an improved YOLOv4 network model is proposed, and the overall performance of the network model is improved by using transfer learning. After sample base training of the improved model, the final realization of the isolation switch equipment identification effect is shown in Figure 6. It can be seen that the improved YOLOv4 network can accurately identify the on/off state of the isolation switch.

FIGURE 6

In terms of the identification precision of the isolation switch, the identification precision of the network model used in this paper is shown in Figure 7. As can be seen from Figure 7, the precision of the network model in identifying the closed state is 89.62%, and the precision of the network model in identifying the open state of the isolation switch is 90.79%, indicating a high overall identification accuracy. Meanwhile, the Rec curve of this network model in the identification process of isolation the switch is shown in Figure 8.

FIGURE 7

FIGURE 8

The AP value of the network model used in this paper in the isolation switch identification test is shown in Figure 9. As shown in Figure 9, the AP value of this network model in the recognition of the closing state of the isolation switch is 92.56%, and that in the recognition of the opening state of the isolation switch is 86.80%. mAP and F1 are widely recognized indicators to evaluate the quality of network models. During the experiment in this paper, mAP values are shown in Figure 10A. The mAP value is 89.67%. The overall detection quality of the network model is excellent. In terms of F1 value, in the process of network model detection, the F1 value of the closed state is 0.9, and the F1 value of the open state is 0.8. From the perspective of F1 value, the network model has better detection and recognition effect for closed state.

FIGURE 9

FIGURE 10

In order to further verify the recognition effect of the improved YOLOv4 network proposed in this paper, the test results were compared with YOLOv4 and YOLOv3, as shown in Table 1. The YOLOv4 network model was improved by transfer learning in this paper. After minor changes, the overall recognition effect of the model in the isolation switch of the substation was improved.

TABLE 1

MethodAP/%mAP/%F1
CloseOpenCloseOpen
YOLOv391.3085.8988.560.80.8
YOLOv492.5086.1089.200.90.8
Improved YOLOv492.5486.8089.670.90.8

Comparison of identification effects of different network models.

4 Conclusion

This paper presents method for detecting the open and close state of isolation switch based on YOLOv4 network model. Compared with the previous methods, YOLOv4 network model has the advantages of faster speed and higher precision. To minimize the number of samples in the training process, transfer learning method is introduced to pre-train the network feature extraction layer through the open data set, thus improving the overall performance of the model. The test results show that the mAP value reaches 89%, and the F1 value of closed state and open state recognition reaches 0.9 and 0.8 respectively. The overall performance of the proposed method is better than that of previous YOLOv3 and YOLOv4 network models.

Therefore, the method will promote the theoretical and technical research of deep learning in substation inspection. The team plans to further study the application and performance improvement under the conditions of insufficient training sets and rain and fog images in the future.

Statements

Data availability statement

The data analyzed in this study is subject to the following licenses/restrictions: Data sets are confidential content of the company. Requests to access these datasets should be directed to Biao_Tang@outlook.com.

Author contributions

XH was responsible for writing the script and debugging the program; BT was responsible for debugging the program; MZ, LS, and YM mainly assisted in writing the manuscript; XW was responsible for debugging the program; ZZ and DX provides a photo collection of related substations.

Acknowledgments

The authors gratefully acknowledge the support of Research and innovative application of key technologies of image processing and knowledge atlas for complex scenes in Smart Grid (202202AD080004), Research and innovative application of key technologies of image processing and knowledge atlas for complex scenes in Smart Grid (YNKJXM20220019), Practical maintenance of substation inspection robot master station management platform of Yunnan Electric Power Research Institute in 2022–2023 (056200MS62210005), Yunnan technological innovation talent training object project (No. 202205AD160005).

Conflict of interest

Authors XH, BT, MZ, LS, and YM are employed by Electric Power Research Institute of Yunnan Power Grid Co., Ltd., XW is employed by Yunnan Power Grid Co., Ltd., ZZ and DX are employed by Qujing Power Supply Bureau, Yunnan Power Grid Co., Ltd.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Abbreviations

AP, average precision; CNN, convolution Neural Network; F1, harmonic mean; mAP, mean average precision; PANet, path aggregation network; R-CNN, region-convolution neural network; SPP, spatial pyramid pooling; SSD, single shotMultiBox detector; YOLO, you only look once; Pre, precision; Rec, recall.

References

Summary

Keywords

YOLOv4, transfer learning, deep learning, isolation switch, smart grid

Citation

Huang X, Tang B, Zhu M, Shen L, Ma Y, Wang X, Zhang Z and Xiao D (2022) State identification of transfer learning based Yolov4 network for isolation switches used in substations. Front. Energy Res. 10:1000459. doi: 10.3389/fenrg.2022.1000459

Received

22 July 2022

Accepted

09 August 2022

Published

30 August 2022

Volume

10 - 2022

Edited by

Xiaoshun Zhang, Northeastern University, China

Reviewed by

Da Xu, China University of Geosciences Wuhan, China

Feixiong Chen, Fuzhou University, China

Updates

Copyright

*Correspondence: Biao Tang,

This article was submitted to Smart Grids, a section of the journal Frontiers in Energy Research

Disclaimer

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Outline

Figures

Cite article

Copy to clipboard


Export citation file


Share article

Article metrics