Intelligent detect for substation insulator defects based on CenterMask

With the development of intelligent operation and maintenance of substations, the daily inspection of substations needs to process massive video and image data. This puts forward higher requirements on the processing speed and accuracy of defect detection. Based on the end-to-end learning paradigm, this paper proposes an intelligent detection method for substation insulator defects based on CenterMask. First, the backbone network VoVNet is improved according to the residual connection and eSE module, which effectively solves the problems of deep network saturation and gradient information loss. On this basis, an insulator mask generation method based on a spatial attentiondirected mechanism is proposed. Insulators with complex image backgrounds are accurately segmented. Then, three strategies of pixel-wise regression prediction, multi-scale features and centerness are introduced. The anchor-free single-stage target detector accurately locates the defect points of insulators. Finally, an example analysis is carried out with the substation inspection image of a power supply company in a certain area to verify the effectiveness and robustness of the proposed method.


Introduction
In China, the "State Grid Corporation Artificial Intelligence Technology Application 2021 Work Plan" has made clear instructions. To build and operate two levels of AI "two libraries and one platform", and form a number of high-precision and high-value power-specific models. As an important and indispensable part of the power system, the safe and stable operation of substations is the basis for building a strong smart grid (Han et al., 2021, Li et al. 2022a. Today, rising renewable energy uncertainty (Li et al. 2022b, Li et al. 2022c) and concerns about cyber security and privacy issues (Li et al. 2019, Li et al. 2022d) pose new challenges for substation operations. Therefore, there are strict standards for operation and maintenance (Q&M) of substation equipment (Huang et al., 2017).
Insulators are key equipment in substations. Common types include: pillar insulators, bushing insulators and suspension insulators. Its operational performance is directly related to the stable and continuous operation of the regional power grid. However, insulator fouling and breakage are the main factors affecting its operational performance (He et al., 2017). In order to meet the needs of Q&M, the substation is equipped with inspection robots and deployed a large number of high-In summary, the application of computer vision and deep learning methods to realize defect detection of substation insulators has become a research hotspot. However, the existing methods are not ideal for the detection of massive high-definition image data in substations, and there is a common problem of low real-time detection of insulator defects. Therefore, based on an end-to-end learning paradigm, this work adopts a one-stage instance segmentation algorithm, and an end-to-end agile detection method for substation insulator defects is proposed.
The main novelties of this work are: (i) The end-to-end design idea revolutionizes the staged and step-by-step segmentation and detection methods in traditional insulator detection research. This strategy effectively reduces the impact of step-by-step error iteration on model performance.
(ii) Insulator mask generation and defect detection can be performed in parallel. This method greatly improves the fault identification efficiency of the equipment Q&M in the substation.
(iii) The one-stage instance segmentation algorithm has been successfully applied in the detection of insulator defect points. By introducing pixel-by-pixel regression prediction, multi-scale features and centerness three strategies to accurately output the bounding box label of the defect point location.
This paper is organized as follows: Section 2 briefly describes the architecture for agile detection of insulator defect points in substations. The method of insulator mask generation and defect point detection is described in detail in Section 3. Section 4 verifies the effectiveness of the proposed method based on the built experimental environment. Finally, conclusions and outlook are presented in Section 5.
The architecture of insulator defect intelligent detection method Due to the close connection between the equipment in the substation, the insulator image taken by the inspection robot is mixed with more other transformer equipment (Wang et al., 2019;Kou et al., 2022). This makes it necessary to step through the images during insulator defect detection. However, the result of step-by-step processing will certainly increase the detection time and reduce the accuracy of detection (Zhao et al., 2019;Chen et al., 2019). At the same time, the characteristics of the massive amount of inspection images are different to ignore. Therefore, this study comprehensively measures the Q&M requirements of substation insulators and the training complexity of massive inspection images. A defect detection method based on CenterMask is designed. The overall implementation plan of this work is shown in Figure 1. Based on the one-stage target detection algorithm, a parallelized image segmentation technique is incorporated to obtain the insulator mask (Yuan et al., 2021). At the same time, the separate extraction of masks for insulator defect points is also incorporated into the training. By counting the insulator defect detection task points, the main tasks are: insulator detection, insulator mask segmentation, insulator defect point detection, and insulator defect point location.

Beginning End
Insulator The mask of the original image.
The corresponding BoundingBox label of the defect point.

Figure 1
The overall framework of the intelligent detection method for insulator defects As shown in Figure 1, this end-to-end work can be decomposed into three parts. First, based on the accumulated original inspection images, the initial image database is constructed. Then, the defect detection model structure is designed. This part is also the core of the whole method. 1)The backbone network adopts improved VoVNet to achieve efficient extraction of target features.
2)The spatial attention-guided mask branch (SAG-Mask) generates target masks. 3)Using fully convolutional onestage (FCOS) to detect defect points of insulators. Finally, the end realizes the output of the insulator mask and the bounding box label of the defect point.

Agile Detection Method for Substation Insulator defects
CenterMask is a newer instance segmentation algorithm proposed by Youngwan Lee and Jongyoul Park of the Korea Institute of Electronics and Communications in 2019 (Lee et al., 2020). This algorithm adds the SAG-Mask module invented by scholars to the famous one-stage anchor-free target detection algorithm FCOS, which makes its target segmentation accuracy and speed superior to traditional algorithms. The two scholars made the algorithm open source for scholars in the field of 4 image and vision to jointly explore the application research of the algorithm. The application of this algorithm in substation insulator defect detection further improves the intelligence and accuracy of insulator defect detection.

Insulator feature extraction based on improved VoVNet
The inspection robot in the smart substation will collect images of insulators from different angles during the operation. The background of the insulator in the image is complex and diverse, which has serious noise interference to the accurate detection of insulator defect points (Tao et al., 2018;Kou et al., 2020). Therefore, our first task is to achieve accurate extraction of insulator target features. It is the backbone network of the model that accomplishes this task in the study.
At present, various target detection models rely on different backbone networks (Du et al., 2021). The core module used by the mainstream target detection model DenseNet is the dense block. All previous layers are aggregated through dense connections, resulting in a linear increase in the number of input pipelines for each subsequent layer. This strategy makes the memory access cost and energy consumption extremely high, and the computing speed is seriously hindered (Xu et al. 2020). When the input is a higher-quality insulator image, object extraction tends to consume more memory and inference time. This problem makes the detection effect of insulator defects far from meeting the standard of massive high-definition insulator images in substations. The CenterMask backbone network proposed by the research utilizes an improved VoVNet. The short board of feature redundancy caused by dense connections in DenseNet is effectively solved, and the core module of one-shot Aggregation (OSA) is adopted. In the strategy, all the previous layers are unified and aggregated at the end.
The improved VoVNet addresses the performance saturation problem and the information loss problem in standard VoVNet, respectively. 1)By adjusting the input of the OSA module and adding it to the output, the residual link of ResNet is introduced to ensure that VoVNet can train a deeper network. It can effectively process insulator images with high pixel ratio.
2)The eSE module of SENet is added to the last feature layer of VoVNet, and the two fully connected layers used in the original SE module are directly replaced by one, which effectively reduces the loss of channel information. This method further enhances the feature extraction capability of the insulator target. The improved connection structure of OSA modules in VoVNet according to the idea is shown in the insulator feature extraction area in Figure 1. The performance of the target detection models based on the improved VoVNet surpasses the models based on other target detection algorithms. The algorithm greatly improves the GPU computing efficiency and meets the needs of high-speed processing for intelligent identification of hidden dangers of insulators.

Insulator mask extraction based on SAG-Mask
Insulator image datasets mostly suffer from the problems of complex backgrounds, multiple strings in a single image and insulator overlap (Wang et al., 2020b). Section 3.1 Insulator feature extraction realizes insulator target detection. The position of the insulator in the image is accurately framed, and only completing this link cannot quickly detect the defect points of the insulator. Therefore, it is necessary to further realize the precise segmentation of insulator beads on this basis. Adding the SAG-Mask branch to the target result of FCOS detection in the CenterMask algorithm satisfactorily completes this task.
The most famous ones in instance segmentation are Mask RCNN and its related improved algorithms, as well as the YOLACT algorithm which focuses on speed (Wang et al., 2020b). 5 However, a spatial attention-oriented mechanism is proposed in CenterMask. The mechanism is able to utilize the spatial attention map of the input image to predict segmentation masks for each boxed instance. The added SAG-Mask branch can be performed in parallel with the insulator fault point identification of the target detector, which greatly improves the identification efficiency of intelligent hidden dangers of insulators.
In this paper, the instance segmentation of insulators is decomposed into two simple and parallel subtasks, as shown in the Mask generation area in Figure 1. Task 1 achieves the goal of constraining the local area of each insulator and naturally distinguishing the instance. A rough shape prediction is made around the center point of each insulator. Task 2 achieves precise segmentation of insulators while retaining the spatial position of precise segmentation. The branch is predicted using the insulator saliency pixels in the entire inspection image. The core idea is to focus on specific block features in the feature map by using an attention mechanism. Finally, a mask for each insulator's corresponding image position is constructed by multiplying the outputs of the two parallel branches.
The mathematical process of instance segmentation process can be described as: the input insulator feature map is marked as Then, saliency pixels are used in task 2 to enhance the input features of the original insulator image. As shown in equation (2). (2)

Identification and location of insulator defects based on FCOS
The goal of intelligent hidden danger identification of insulators is to quickly locate various common faults of insulators through intelligent algorithms, such as bead damage, excessive pollution, aging cracks, etc (Wang et al., 2020b). The accurate extraction of the insulator mask can only ensure that the location of the insulator is identified from the inspection image, and the insulator is accurately segmented. However, specific fault points require accurate location detection of the corresponding model. This work uses the anchor-free FCOS algorithm to complete the task in the target detection part of the overall CenterMask structure. FCOS is a fully convolutional one-stage object detection algorithm . It solves the object detection problem in a pixel-by-pixel prediction manner. At the same time, in the detection process, the anchor-free box of FOCS is a huge advantage. The hyperparameters associated with anchor boxes and all the complex computations associated with anchor boxes are effectively avoided. The efficiency of insulator defect identification and location has been greatly improved. The output of the entire defect identification and localization results includes three aspects: pixel-by-pixel regression prediction, multi-scale features and center-ness.
The specific process framework is shown in the defect point labeling area in Figure 1. The first output of the 3 output layers is the classification branch. H*W represents the size of the feature. C represents the number of categories. A position in the input inspection image can be mapped to a position on the feature, and the mapping function is: where the coordinates ( , ) pq denote a specific location in the inspection image. r denotes the scaling between the feature map and the original inspection image. This function can represent the relationship between the position of the point on the feature map and the position of the point on the output image. This lays the groundwork for computing the classification and regression objectives for each point on the feature map.
The second output is the center-ness policy. The FCOS algorithm is applied to achieve high efficiency of one-stage anchor-free box calculation and high recall rate of pixel-by-pixel regression strategy. It also brings many low-quality prediction bounding boxes with too large deviation from the center point. By introducing this strategy, the distance between each point and the target center can be calculated, thereby suppressing some predicted bounding boxes that are far away from the target center. But this strategy does not introduce any excessive hyperparameters.
The centerness strategy adds a branch to each prediction layer of the feature pyramid. And this branch is parallel to the classification branch, which is equivalent to adding a loss function to the network. During the training process of the image dataset, the loss function constrains the predicted bounding box to be as close to the center point as possible, so that the auxiliary non-maximum suppression (NMS) can filter out the low-quality bounding box prediction. The loss function is as follows: Loss is the value of the loss function, and the more it tends to zero, the better the prediction can be constrained.
The third output is the return branch. When performing the target frame regression on all the points in the target frame of defect points in the inspection image, the distance to each edge is used as the measurement standard. This part is the main difference between the target detection algorithm without anchor frame and the target detection algorithm based on anchor frame, where 4 in 4 HW  represents 4 values related to regression. Calculated as follows:

Case Study
In order to verify the effectiveness and robustness of the insulator defect agile detection method proposed in this paper. Taking the substation inspection image of a power supply company in a certain area as an example, an example analysis is carried out. There are two main aspects to build the experimental environment in DELL graphics workstation. Hardware: Intel(R) Xeon(R) Gold5118 CPU @ 2.30GHz, NVIDIA Quadro P5000, 128G RAM; Software: Python 3.7.6, CUDA 10.1, 7 PyTorch 1.4.0, Detectron2 0.1.1, CV2 4.2.0. The model output results are evaluated by a number of index combinations to further realize the optimization of the defect point identification model.

Insulator mask generation and defect detection
The insulator images captured by the inspection are divided into training set and test set. In the above experimental environment, the analysis of training set samples, the selection and optimization of the model backbone network are carried out. This model was trained over 200,000 iterations. During the training process, each part of the network loss is visualized to achieve accurate tuning of model parameters. The complete flow of insulator mask generation is shown in Figure 2.
First, the training set as shown in Figure 2(a) is manually annotated. It is used to assist the initial learning of the model. Figure 2(b) shows the manual annotation of the original inspection image using the labeling tool Labelme. Insulators are marked with the "Insulator" label, and insulator defect points are marked with the "Insulator error" label. Then, the CenterMask network model is iteratively learned and adjusted using the effectively labeled training set. This process takes a long time. Finally, the trained insulator defect detection model is applied. Input test set for model insulator mask extraction branch validity verification. The extracted rendering is shown in Figure 2(c). As mentioned in Section 2, defect point identification and mask extraction in the insulator defect detection model are two independent branches, and the two branches can be executed in parallel. The identification of defect points does not have to wait for mask generation or be disturbed by mask generation results. Therefore, the defect point identification and positioning is more flexible and the effect is better. By adopting the same multi-scale training strategy as insulators, the model parameters are set for random scaling within the range of input scales. The insulator explosion point training dataset is enriched according to the maximum and minimum input size constraints. When outputting the defect point xml file, the visual output display of single image insulator defect point identification and identification accuracy is realized. For the test set model insulator defect point detection visualization diagram and the corresponding positioning label stored in the xml file effect shown in

Example index evaluation
For the model test results, the average precision (AP) and average recall (AR) evaluation metrics are set. Under the combination of different IOUs, areas under inspection (AREA) and number of objects under inspection (MaxDets), model performance analysis is performed on all test results. The specific evaluation results are shown in Table 1. Where IOU represents the degree of overlap between the generated candidate frame and the original marked frame. It is the ratio of intersection and union, and the calculation equation is as follows: Area of Overlap IOU Area of Union  (6) 32~96 . AP l represents the AP measurement of the target box whose pixel area is larger than 2 96 . Value1 represents the AP/AR generated by the insulator mask under different 9 evaluation methods. Value2 represents the index result value of defect point detection in different situations. Table 1 shows that when 0.5 is used as the IOU threshold for segmentation, the model mask generation effect is the best. The accuracy of the model is 93.4%, indicating that the model satisfies the extraction of insulators in most inspection image scenarios. However, the AP and AR of target boxes with a pixel area smaller than are significantly lower, indicating that the model still needs to be improved in the details of mask generation. The performance of the model in defect point detection is further analyzed, and the overall Q&M requirements of the substation are met, and the detection effect is good. In the detection of objects with a pixel area greater than 2 96 , AP and AR are 97.6% and 94.2%, respectively. The model performs best. It can be seen that the model has a high detection accuracy rate for large insulator defect points, and has a good recall rate.

Conclusion
This work takes the intelligent Q&M of the equipment in the substation as the background, and comprehensively analyzes the characteristics of the massive images of inspection insulators and the complex background of defect point detection. An intelligent detection method for substation insulator defects based on CenterMask is proposed. The proposal of this technology improves the efficiency and accuracy of insulator defect detection to a certain extent. The results of the example analysis verify the following main conclusions: 1) An end-to-end insulator defect point detection architecture is designed. The entire learning process does not perform artificial sub-problem division, but the deep learning model directly learns the mapping from the original input to the desired output.
2) It is only necessary to input the insulator field image captured by the inspection robot into the defect detection model. The three key tasks of target feature extraction, mask segmentation and defect point target detection of insulators can be realized at one time.
3) A new agile detection method for insulator defects in substations is proposed. In the manual inspection mode, the waste of human resources and the potential safety hazards caused by the operation of the power grid are effectively reduced.
The transfer application research of this method will be the next exploration direction. It will greatly promote the efficiency and intelligence of substation equipment Q&M, and has important research significance and engineering application value.

Data Availability Statement
The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Author Contributions
BY: designed this study. FL: contributed to the architecture of insulator defect intelligent detection method. ML: contributed to the insulator feature extraction based on improved VoVNet. PY: contributed to the insulator mask extraction based on SAG-Mask. HY: performed the identification and location of insulator defects based on FCOS. LW: collected and cleansed the data. All authors contributed to the writing of the article and all agreed to the submitted version of the article.