ORIGINAL RESEARCH article

Front. Energy Res., 12 January 2023

Sec. Process and Energy Systems Engineering

Volume 10 - 2022 | https://doi.org/10.3389/fenrg.2022.1091322

Research on the identification method of safety wearing of electric power workers based on deep learning

  • Tianhe Power Supply Bureau of Guangzhou Power Supply Bureau, Guangdong Power Co., Ltd., Guangzhou, China

Abstract

Aiming at the difficulties of manual monitoring and compliance with the current wear identification of electric power workers, the detection and identification of safety helmets, work clothes, and insulating gloves are used to carry out normative identification and warning, and a deep learning-based power worker safety wear identification method is proposed in this paper. The AlexNet and Inception are introduced to increase the width and depth of the artificial neural network. At the same time, the ReLU activation function with better performance is used to reduce the amount of network computation, and the Global Average Pooling layer is used to replace the fully connected layer with more parameters. The improved convolution neural network model has a total of 13 layers. In order to prevent the network from overfitting, the Early-stopping mechanism and the L2 regularization method are used to improve the performance of the network model. The experimental results show that the algorithm can achieve a good recognition effect on the staff who do not wear safety according to the regulations in the video, and the feasibility and effectiveness of the algorithm in practical application are verified.

1 Introduction

As computer performance steadily improves, deep learning contributes to the development of society (; ). The environment of power construction sites is complex, and the types of tasks are diverse, so compliance with tooling is the basic requirement for safe production (; ). Correctly wearing safety helmets and tooling can protect the personal safety of operators to a large extent (). However, due to the slack and negligence of the operators themselves and the relaxed vigilance of the management personnel, safety risks in the construction process have occurred from time to time (). To this end, a deep learning-based power worker safety-wearing recognition method is proposed to identify operators who do not wear tooling correctly and remind them in time, which can improve the effectiveness of supervision, enhance the safety awareness of operators, reduce potential safety risks, and ensure that construction safety is of great significance.

At present, the safety wear detection based on deep learning method is in its infancy, and few scholars have studied it. Literature () proposed a parallel two-way convolution neural network method to identify human body by improving LeNet 5, and then recognized helmet by color features, which basically met the demand. Literature () realized pedestrian detection by constructing a multi-layer convolutional neural network (CNN), and then recognized helmets through both color and HOG features. In literature (), OpenPose was used to locate the head and neck of the human body and automatically intercept the small enclosure sub images around it, and then Faster R-CNN was used to detect the safety helmet in the sub images. This kind of method still recognizes the human body first and then the safety wear. There are two parts of errors, so the defects of traditional detection methods still exist. Literature () uses the improved YOLO V3 network to detect helmets with the whole human body as the detection target, but the detection accuracy is not very high because there are many features in the human body.

With the great contribution of deep learning to the field of target detection, relevant researchers are committed to combining deep learning with target detection of substation equipment. Literature () proposed a multi-target positioning method for infrared image of power equipment based on improved FAsT Match algorithm. This method overcomes the shortcomings of previous algorithms that are not suitable for infrared image target location and can only achieve single target location. However, the interference of complex background on target location is not considered, and changing different scenes may lead to poor recognition accuracy. The author of the literature () proposes a power equipment image recognition approach for the problem that the traditional methods are not clear in the classification of image features of power equipment, resulting in poor image recognition effect and difficulty in ensuring safe operation. The method can complete the effective identification of the collected images within 30s, which has a good practical application effect. According to (), the Faster R-CNN was able to reduce the complexity of the RPN network by optimizing its convolution kernel. Reference (), based on the recognition of the importance of safety helmet detection in construction site management, and considering practical issues such as cost control of hardware facilities in engineering projects, proposed a lightweight and improved version LT based on the deep learning network Tiny-YOLO v3 helmet detection technology method. Reference () designed a deep learning-based safety helmet and mask detection system in power construction scenarios by improving the CenterNet algorithm. Combined with system functions, it can effectively adapt to the detection of safety helmets and masks in power construction scenarios identification and violation management. Reference () proposes an improved YOLO-v3 network for the problems of occlusion, variable illumination, and different target sizes in helmet detection under the complex background of the construction work surface. Due to the huge amount of computation and parameters, convolutional neural networks usually rely on hardware with strong computing power, such as GPU, to complete the training and inference process, but they often do not have high-performance computing hardware in construction sites. Additional purchases will bring unnecessary economic burdens to production enterprises.

Whether it is deep learning or traditional methods, the research on safety wear detection at home and abroad is still at the initial stage, and the accuracy of good and bad cannot evaluate the quality of each detection method. In addition, the following problems still exist in helmet wearing detection: 1) Single scene. Most of the detection environments studied are single and ideal, which are not close to the actual application scenarios, making their practicability greatly reduced. 2) The detection method of first detecting pedestrians and then locating the head. Most detection methods adopt this two-step detection method, which will lead to failure to give accurate warning information on whether to wear a helmet once the person is missed. Aiming at the difficulties of manual monitoring and compliance with the current wear identification of electric power workers, the detection and identification of safety helmets, work clothes, and insulating gloves are used to carry out normative identification and warning, and a deep learning-based power worker safety wear identification method is proposed. The main contributions of this paper are summarized as follows.

  • 1) The AlexNet and Inception are introduced to increase the width and depth of the artificial neural network. At the same time, the ReLU activation function with better performance is used to reduce the amount of network computation.

  • 2) The Global Average Pooling layer is used to replace the fully connected layer with more parameters. The improved convolution neural network model has a total of 13 layers.

  • 3) In order to prevent the network from overfitting, the Early-stopping mechanism and the L2 regularization method are used to improve the performance of the network model.

This paper is organized as follows: The first section establishes the design framework for the identification of safety wearing of electric power workers; the second section establishes a deep learning-based safety wearing identification model and evaluation index for electric power workers; the third section is experiment verification and analysis; the last section is the conclusion.

2 Design framework for safety wearing identification of electric power workers

The safety wear identification process of electric power workers is divided into five steps. Firstly, select the frame of the video, convert the intercepted single-frame picture into a JPG format picture that the model can process, and input it into the pedestrian detection model to determine whether a pedestrian is detected. If a pedestrian is detected, proceed to the next stage of identification; The image after format conversion is preprocessed to make it meet the requirements of the model for image recognition; then, the model parameters are fine-tuned based on the training and test results, and finally realize image classification to meet the requirements of recognition accuracy, and return the wear recognition result. Figure 1 shows the safety wear identification process of electric power workers.

FIGURE 1

3 Safety wears recognition approach for electric power workers

3.1 Data preprocessing

In this paper, 4,300 site photos were collected at different construction sites and stages, and the samples were expanded to 8,600 by means of horizontal mirror image data enhancement. A data set containing 24,650 safety helmets of different scales, different light intensities and different shielding conditions was made. The ratio of training set and verification set during the training process was 8:2. Based on the samples in the pilot electric room scenario. The number of markings for pedestrians, work clothes, work caps, gloves, hand-held operations, and poles is shown in

Table 1

. The data preprocessing steps are as follows.

  • 1) Resampling the training set data.

  • 2) For the purpose of training and testing neural networks more easily, the size of the pictures is normalized, and the sizes of all pictures are normalized to 32*32.

  • 3) Image enhancement using histogram equalization. Histogram equalization enhances contrast by transforming pixel intensities, turning the histogram distribution of an image into an approximately uniform distribution.

  • 4) The image pixels are normalized to the [−1, 1] interval.

  • 5) Set the random flip angle of the image to 10°, the random horizontal or vertical offset of the image to .08, and the random zoom parameter of the image to .2.

TABLE 1

Pilot electric room sceneMain categorySubcategoryNumber of samples
Wear normative identificationHelmetRed280
Blue439
White184
Work clothes (jacket)Dark blue509
Light blue288
Blue307
Work pantsDark blue405
Light blue267
Blue229
Insulated glovesBlack-brown244
Orange306
Light yellow260
Work Behavior Status RecognitionHand-held joystickHandheld joystick without gloves295
Wear gloves while holding the joystick287

Annotation of training samples.

3.2 Model improvement strategies

At present, although the CNN algorithm has achieved a high recognition rate, the computational load is relatively large and does not meet the real-time requirements (

). In practical application scenarios, it is not only necessary to take into account the accuracy of the identification of electric power workers’ safety work, but also to consider the real-time nature of the identification, to inform the electric power workers in time and prevent the occurrence of safety accidents (

;

). Therefore, to improve the real-time performance of the CNN algorithm, we need to prune the original network model and propose a lightweight network model. Under the condition of ensuring the same accuracy, the recognition speed of the network model is accelerated, and the real-time performance of the network model is improved. This paper improves the CNN algorithm in the following aspects.

  • 1) Replace the convolution kernels of all convolutional layers with 3*3 convolution kernels. Two 3*3 convolutional layers are equivalent to a 5*5 convolutional layer, and three 3*3 convolutional layers are equivalent to a 7*7 convolutional layer, in the case of the same field of view, the network level is deepened, the non-linear transformation is added, the feature learning ability of the network is stronger, and the network capacity is larger. Compared with the large convolution kernels of 5*5 and 7*7, the number of parameters of the small convolution kernel of 3*3 is significantly reduced.

  • 2) The convolutional neural network is widened and deepened through the introduction of AlexNet and Inception. The Inception module combines convolutions of different scales on the same layer of convolution, and uses a 1*1 convolution kernel for feature dimensionality reduction. In the case of the same parameters, the network uses the Inception module to calculate more efficiently, extract more features, and train better.

  • 3) Use the batch normalization method to process the input batch samples (). In order to unify the data distribution of each layer of the network, batch normalization is introduced after each convolutional layer, and the data of each layer is normalized to a mean of 0 and a variance of 1. The formula for batch normalization is:

where,

is the average value of each batch of training data neurons;

is the standard deviation of the activation degrees of each batch of data neurons.

To protect the feature distribution learned by the network, the network changes are reconstructed and learnable parameters

and

are introduced to inverse normalization, so

. The batch normalization algorithm can prevent gradient disappearance or gradient explosion to some extent.

  • 4) Use the ReLU activation function with better performance instead of the Sigmoid activation function, it can be expressed as:

When the Sigmoid activation function is used, there are the following three obvious disadvantages: 1) The network input is too large or small, the neuron gradient will tend to zero, and the neuron gradient will disappear during backpropagation, which will cause the neural network to fail to train; 2) The output means of sigmoid activation function is non-zero, and the non-zero mean signal output by the neurons in the previous layer will be used as the input signal of neurons in the next layer. When the input data is positive, the gradient will always be updated in the positive direction; 3) The calculation of the sigmoid activation function is more complicated, which will increase the network training time for large-scale deep networks (). The formula of the ReLU activation function is:

Compared with the Sigmoid activation function, the ReLU activation function performs better and helps in the propagation of gradients. The ReLU activation function has a relatively small amount of calculation, and only needs to do one arithmetic operation. The ReLU activation function is always 1 for the part greater than 0, and the gradient will not be saturated (the gradient will not be too small). During the backpropagation process, the gradient can be better propagated to the previous network, and the network will converge faster. The improved structure parameters of the CNN model are shown in Table 2.

TABLE 2

Network layerLayer typeKernel sizeStrideFeature map
0Input layer32*32*3
1Convolutional layer3*3132*32*64
2Pooling layer2*2216*16*64
3Inception_v116*16*128
4Pooling layer2*228*8*128
5Inception_v28*8*256
6Pooling layer2*224*4*256
7Convolutional layer3*314*4*256
8GlobalAvg_pool256
9Softmax43

Improved structure parameters of convolution neural network.

About the improved CNN model, the input layer is a 32*32 work picture of electric power workers. There are four convolutional layers, the activation functions are all ReLU, the stride is 1, the padding equal to the same, and zeros are filled around the input picture. The size of the convolution kernels of the first and seventh layers of convolutional layers are both 3*3, and the number of convolutional kernels is 64 and 256 respectively. The third and fifth layers of convolutional layers are Inception modules, which consist of four parts.

3.3 Selection of the last convolutional layer

The number of neurons in the last convolutional layer is set to 128, 256 and 512 respectively, and the experimental results under different numbers of convolutional neurons are compared and analyzed, as shown in Table 3. According to the results, when the number of neurons in the last convolutional layer is 256, the recognition rate of the model is the highest, so the number of neurons in the last convolutional layer is selected as 256.

TABLE 3

Last convolutional layerTraining time/sRecognition rate/%
128133397.94
256160998.59
512198598.06

Comparison of the number of neurons in the last convolution layer.

3.4 Optimizer selection

To select the better optimizer, the optimizers of the stochastic gradient descent algorithm (SGD) (), Momentum algorithm (), Adagrad algorithm (), RMS prop algorithm (), and Adam algorithm () are compared with the gradient descent algorithm. Table 4 shows the performance of the network model in different optimization algorithms.

TABLE 4

OptimizerParametersTraining time/sRecognition rate/%
SGDMomentum = 0.9238994.74
MomentumRho = .95239197.87
Adagrad245097.07
RMSpropRho = 0.91,88798.47
AdamBeta1 = .9, Beta2 = .999160698.59

Experimental results of different optimizers.

In Table 4, when the network model adopts the Adam optimizer, the training time of the network model is the shortest and the recognition rate is the highest, so the Adam optimizer is used to identify the safety wear of electric power workers.

The loss function in the algorithm of safe wear recognition is designed as follows:

In 4), y is the label value; s is the predicted value of the network forward propagation.

The back-propagation process is as follows:

3.5 Evaluation indicators

To evaluate the recognition effect, the average accuracy index is used to measure the matching accuracy of the detection frame to the target object, it can be expressed as follows ():where, is the actual quantity of pictures at ith category; is the quantity of all objects at the data set category i; is the number of images containing objects in the category i.

The average precision metric is:

4 Experimental verification and analysis

4.1 Parameter settings

During the network training process, the early-stopping mechanism is used to prevent the network from overfitting, and the parameter is set to 15. Training is stopped when the loss on the training set drops while the loss on the validation set remains the same for 15 consecutive epochs. To avoid the network from overfitting, the weight of the network model is only taken as a small value to limit the complexity of the network model and make the weight distribution more regular. This is weight regularization, adding the cost associated with larger weights to the network loss function to make the absolute value of the weight coefficients small enough. Based on the L2 regularization method (; ; ), the formula is as follows:

The training parameters are set as shown in Table 5.

TABLE 5

ParameterParameter value
Enter image size32*32
Dynamic learning rateThe initial learning rate is .001, monitor = “val_loss”, min_1r = 10–6, factor = .1, patience = 10
Mini-batch64
Epoch40
Weight decay term for L2 regularization10–5
Early-stopping early stop mechanismmonitor = “val_loss”, patience = 50, verbose = 2
Loss functionCross-drop loss function

Setting of network training parameters.

4.2 Experimental results

The training time of the improved CNN network model is 1531s. The accuracy curve of the network training curve is shown in Figure 2 and the loss change curve of the network training curve is shown in Figure 3.

FIGURE 2

FIGURE 3

In Figures 2, 3, we can see that the accuracy in the early stage of training gradually increased, and the loss of the training set and the validation set also gradually decreased. There was a slight oscillation in the middle, and it gradually became stable with iterations. The network accuracy eventually tends to 100%, and the network loss eventually tends to 0. When the epoch is equal to 14, the loss of the training set is still decreasing, the loss of the validation set tends to remain unchanged, and when the loss of the validation set does not change for 10 consecutive times, the early-stopping mechanism will end the training of the network in advance, so the final training is taken. The network model was obtained 25 times. The safety wear recognition results of the electric staff working are shown in Table 6.

TABLE 6

Correct numberIncorrect numberAccuracy (%)
Pedestrian detection box95297.94
Helmet72396.00
Work clothes96198.97
Work pants71494.67
Insulated Gloves75692.00
Joystick70593.33

Safety wears identification results.

In Table 6, the safety wear recognition has high accuracy in each category.

The result shows that the improved CNN has the advantages of high accuracy and good real-time performance.

To verify the superiority of the improved CNN algorithm, the network model proposed in this paper is compared with the other network model under the same parameters. The performance comparison result is shown in Table 7.

TABLE 7

ModelmAPRecognition rate/%Running time/s
Proposed algorithm.88496.2126
SSD.86392.5334
Faster-RCNN.88689.6258
YOLOv3.85890.8361
EMD-SVM.84786.5389
BPNN algorithm.85980.3184
DBN.85282.8159

Performance comparison under different models.

In Table 7, the recognition method used in this paper has the highest recognition accuracy, reaching 96%. The highest recognition rate of other algorithms is 92.5%. Although these recognition methods solve the problem of image recognition to some extent, when the image features are too mixed, these algorithms cannot well complete the mapping from feature extraction to state recognition, which is not conducive to model recognition. Compared with other traditional algorithms, the improved deep learning algorithm has a higher recognition rate and significantly improves the recognition rate of safe wearing of electric power workers. Compared with other algorithms, the proposed method has faster running speed and can meet the requirements of real-time computing. The proposed network model has strong feature expression ability. The advantages of good generalization performance and strong robustness.

5 Conclusion

In this paper, through the detection and identification of safety helmets, work clothes, and insulating gloves to carry out normative identification and warning, a deep learning-based safety wear identification method for electric power workers is proposed. The experimental results show that: compared with other traditional algorithms, the improved deep learning algorithm proposed in this paper has a higher recognition rate, significantly improves the recognition rate of safe wearing of electric power workers, and has the advantages of high accuracy and good real-time performance, thus providing the practical work provides guidance.

Statements

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Author contributions

All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.

Funding

I and my fellow co-authors are fully aware of and agree with the payment of the listed article processing fee should the manuscript be accepted for publication.

Conflict of interest

Authors ZC, CM, JR, FH, and ZW were employed by the company Guangdong Power Co., Ltd.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Summary

Keywords

deep learning, convolutional neural network, electric power work, safety wear recognition, safety

Citation

Chen Z, Ma C, Ren J, Hao F and Wang Z (2023) Research on the identification method of safety wearing of electric power workers based on deep learning. Front. Energy Res. 10:1091322. doi: 10.3389/fenrg.2022.1091322

Received

06 November 2022

Accepted

30 December 2022

Published

12 January 2023

Volume

10 - 2022

Edited by

Cong Qi, China University of Mining and Technology, China

Reviewed by

Zhekang Dong, Hangzhou Dianzi University, China

Tiantian Chen, China University of Mining and Technology, China

Updates

Copyright

*Correspondence: Zengyu Wang,

This article was submitted to Process and Energy Systems Engineering, a section of the journal Frontiers in Energy Research

Disclaimer

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Outline

Figures

Cite article

Copy to clipboard


Export citation file


Share article

Article metrics