ORIGINAL RESEARCH article
Sec. Process and Energy Systems Engineering
Volume 10 - 2022 | https://doi.org/10.3389/fenrg.2022.1091322
Research on the identification method of safety wearing of electric power workers based on deep learning
- Tianhe Power Supply Bureau of Guangzhou Power Supply Bureau, Guangdong Power Co., Ltd., Guangzhou, China
Aiming at the difficulties of manual monitoring and compliance with the current wear identification of electric power workers, the detection and identification of safety helmets, work clothes, and insulating gloves are used to carry out normative identification and warning, and a deep learning-based power worker safety wear identification method is proposed in this paper. The AlexNet and Inception are introduced to increase the width and depth of the artificial neural network. At the same time, the ReLU activation function with better performance is used to reduce the amount of network computation, and the Global Average Pooling layer is used to replace the fully connected layer with more parameters. The improved convolution neural network model has a total of 13 layers. In order to prevent the network from overfitting, the Early-stopping mechanism and the L2 regularization method are used to improve the performance of the network model. The experimental results show that the algorithm can achieve a good recognition effect on the staff who do not wear safety according to the regulations in the video, and the feasibility and effectiveness of the algorithm in practical application are verified.
As computer performance steadily improves, deep learning contributes to the development of society (Dourado et al., 2020; Liu et al., 2022). The environment of power construction sites is complex, and the types of tasks are diverse, so compliance with tooling is the basic requirement for safe production (Liu et al., 2020; Postalcıoğlu, 2020). Correctly wearing safety helmets and tooling can protect the personal safety of operators to a large extent (Jacob and Darney, 2021). However, due to the slack and negligence of the operators themselves and the relaxed vigilance of the management personnel, safety risks in the construction process have occurred from time to time (Yuan et al., 2022). To this end, a deep learning-based power worker safety-wearing recognition method is proposed to identify operators who do not wear tooling correctly and remind them in time, which can improve the effectiveness of supervision, enhance the safety awareness of operators, reduce potential safety risks, and ensure that construction safety is of great significance.
At present, the safety wear detection based on deep learning method is in its infancy, and few scholars have studied it. Literature (Gangolells et al., 2010) proposed a parallel two-way convolution neural network method to identify human body by improving LeNet 5, and then recognized helmet by color features, which basically met the demand. Literature (Mroszczyk, 2015) realized pedestrian detection by constructing a multi-layer convolutional neural network (CNN), and then recognized helmets through both color and HOG features. In literature (Anastasiadou et al., 2021), OpenPose was used to locate the head and neck of the human body and automatically intercept the small enclosure sub images around it, and then Faster R-CNN was used to detect the safety helmet in the sub images. This kind of method still recognizes the human body first and then the safety wear. There are two parts of errors, so the defects of traditional detection methods still exist. Literature (Chen et al., 2022a) uses the improved YOLO V3 network to detect helmets with the whole human body as the detection target, but the detection accuracy is not very high because there are many features in the human body.
With the great contribution of deep learning to the field of target detection, relevant researchers are committed to combining deep learning with target detection of substation equipment. Literature (Yu et al., 2021) proposed a multi-target positioning method for infrared image of power equipment based on improved FAsT Match algorithm. This method overcomes the shortcomings of previous algorithms that are not suitable for infrared image target location and can only achieve single target location. However, the interference of complex background on target location is not considered, and changing different scenes may lead to poor recognition accuracy. The author of the literature (Qin et al., 2022) proposes a power equipment image recognition approach for the problem that the traditional methods are not clear in the classification of image features of power equipment, resulting in poor image recognition effect and difficulty in ensuring safe operation. The method can complete the effective identification of the collected images within 30s, which has a good practical application effect. According to (Yang et al., 2022), the Faster R-CNN was able to reduce the complexity of the RPN network by optimizing its convolution kernel. Reference (Yang et al., 2020), based on the recognition of the importance of safety helmet detection in construction site management, and considering practical issues such as cost control of hardware facilities in engineering projects, proposed a lightweight and improved version LT based on the deep learning network Tiny-YOLO v3 helmet detection technology method. Reference (Tang et al., 2020) designed a deep learning-based safety helmet and mask detection system in power construction scenarios by improving the CenterNet algorithm. Combined with system functions, it can effectively adapt to the detection of safety helmets and masks in power construction scenarios identification and violation management. Reference (Li et al., 2019) proposes an improved YOLO-v3 network for the problems of occlusion, variable illumination, and different target sizes in helmet detection under the complex background of the construction work surface. Due to the huge amount of computation and parameters, convolutional neural networks usually rely on hardware with strong computing power, such as GPU, to complete the training and inference process, but they often do not have high-performance computing hardware in construction sites. Additional purchases will bring unnecessary economic burdens to production enterprises.
Whether it is deep learning or traditional methods, the research on safety wear detection at home and abroad is still at the initial stage, and the accuracy of good and bad cannot evaluate the quality of each detection method. In addition, the following problems still exist in helmet wearing detection: 1) Single scene. Most of the detection environments studied are single and ideal, which are not close to the actual application scenarios, making their practicability greatly reduced. 2) The detection method of first detecting pedestrians and then locating the head. Most detection methods adopt this two-step detection method, which will lead to failure to give accurate warning information on whether to wear a helmet once the person is missed. Aiming at the difficulties of manual monitoring and compliance with the current wear identification of electric power workers, the detection and identification of safety helmets, work clothes, and insulating gloves are used to carry out normative identification and warning, and a deep learning-based power worker safety wear identification method is proposed. The main contributions of this paper are summarized as follows.
1) The AlexNet and Inception are introduced to increase the width and depth of the artificial neural network. At the same time, the ReLU activation function with better performance is used to reduce the amount of network computation.
2) The Global Average Pooling layer is used to replace the fully connected layer with more parameters. The improved convolution neural network model has a total of 13 layers.
3) In order to prevent the network from overfitting, the Early-stopping mechanism and the L2 regularization method are used to improve the performance of the network model.
This paper is organized as follows: The first section establishes the design framework for the identification of safety wearing of electric power workers; the second section establishes a deep learning-based safety wearing identification model and evaluation index for electric power workers; the third section is experiment verification and analysis; the last section is the conclusion.
2 Design framework for safety wearing identification of electric power workers
The safety wear identification process of electric power workers is divided into five steps. Firstly, select the frame of the video, convert the intercepted single-frame picture into a JPG format picture that the model can process, and input it into the pedestrian detection model to determine whether a pedestrian is detected. If a pedestrian is detected, proceed to the next stage of identification; The image after format conversion is preprocessed to make it meet the requirements of the model for image recognition; then, the model parameters are fine-tuned based on the training and test results, and finally realize image classification to meet the requirements of recognition accuracy, and return the wear recognition result. Figure 1 shows the safety wear identification process of electric power workers.
3 Safety wears recognition approach for electric power workers
3.1 Data preprocessing
In this paper, 4,300 site photos were collected at different construction sites and stages, and the samples were expanded to 8,600 by means of horizontal mirror image data enhancement. A data set containing 24,650 safety helmets of different scales, different light intensities and different shielding conditions was made. The ratio of training set and verification set during the training process was 8:2. Based on the samples in the pilot electric room scenario. The number of markings for pedestrians, work clothes, work caps, gloves, hand-held operations, and poles is shown in Table 1. The data preprocessing steps are as follows.
1) Resampling the training set data.
2) For the purpose of training and testing neural networks more easily, the size of the pictures is normalized, and the sizes of all pictures are normalized to 32*32.
3) Image enhancement using histogram equalization. Histogram equalization enhances contrast by transforming pixel intensities, turning the histogram distribution of an image into an approximately uniform distribution.
4) The image pixels are normalized to the [−1, 1] interval.
5) Set the random flip angle of the image to 10°, the random horizontal or vertical offset of the image to .08, and the random zoom parameter of the image to .2.
3.2 Model improvement strategies
At present, although the CNN algorithm has achieved a high recognition rate, the computational load is relatively large and does not meet the real-time requirements (Ramcharan et al., 2017). In practical application scenarios, it is not only necessary to take into account the accuracy of the identification of electric power workers’ safety work, but also to consider the real-time nature of the identification, to inform the electric power workers in time and prevent the occurrence of safety accidents (Ker et al., 2017; Mezgec et al., 2019). Therefore, to improve the real-time performance of the CNN algorithm, we need to prune the original network model and propose a lightweight network model. Under the condition of ensuring the same accuracy, the recognition speed of the network model is accelerated, and the real-time performance of the network model is improved. This paper improves the CNN algorithm in the following aspects.
1) Replace the convolution kernels of all convolutional layers with 3*3 convolution kernels. Two 3*3 convolutional layers are equivalent to a 5*5 convolutional layer, and three 3*3 convolutional layers are equivalent to a 7*7 convolutional layer, in the case of the same field of view, the network level is deepened, the non-linear transformation is added, the feature learning ability of the network is stronger, and the network capacity is larger. Compared with the large convolution kernels of 5*5 and 7*7, the number of parameters of the small convolution kernel of 3*3 is significantly reduced.
2) The convolutional neural network is widened and deepened through the introduction of AlexNet and Inception. The Inception module combines convolutions of different scales on the same layer of convolution, and uses a 1*1 convolution kernel for feature dimensionality reduction. In the case of the same parameters, the network uses the Inception module to calculate more efficiently, extract more features, and train better.
3) Use the batch normalization method to process the input batch samples (Bashar, 2019). In order to unify the data distribution of each layer of the network, batch normalization is introduced after each convolutional layer, and the data of each layer is normalized to a mean of 0 and a variance of 1. The formula for batch normalization is:
To protect the feature distribution learned by the network, the network changes are reconstructed and learnable parameters
4) Use the ReLU activation function with better performance instead of the Sigmoid activation function, it can be expressed as:
When the Sigmoid activation function is used, there are the following three obvious disadvantages: 1) The network input is too large or small, the neuron gradient will tend to zero, and the neuron gradient will disappear during backpropagation, which will cause the neural network to fail to train; 2) The output means of sigmoid activation function is non-zero, and the non-zero mean signal output by the neurons in the previous layer will be used as the input signal of neurons in the next layer. When the input data is positive, the gradient will always be updated in the positive direction; 3) The calculation of the sigmoid activation function is more complicated, which will increase the network training time for large-scale deep networks (Aggarwal, 2019). The formula of the ReLU activation function is:
Compared with the Sigmoid activation function, the ReLU activation function performs better and helps in the propagation of gradients. The ReLU activation function has a relatively small amount of calculation, and only needs to do one arithmetic operation. The ReLU activation function is always 1 for the part greater than 0, and the gradient will not be saturated (the gradient will not be too small). During the backpropagation process, the gradient can be better propagated to the previous network, and the network will converge faster. The improved structure parameters of the CNN model are shown in Table 2.
About the improved CNN model, the input layer is a 32*32 work picture of electric power workers. There are four convolutional layers, the activation functions are all ReLU, the stride is 1, the padding equal to the same, and zeros are filled around the input picture. The size of the convolution kernels of the first and seventh layers of convolutional layers are both 3*3, and the number of convolutional kernels is 64 and 256 respectively. The third and fifth layers of convolutional layers are Inception modules, which consist of four parts.
3.3 Selection of the last convolutional layer
The number of neurons in the last convolutional layer is set to 128, 256 and 512 respectively, and the experimental results under different numbers of convolutional neurons are compared and analyzed, as shown in Table 3. According to the results, when the number of neurons in the last convolutional layer is 256, the recognition rate of the model is the highest, so the number of neurons in the last convolutional layer is selected as 256.
3.4 Optimizer selection
To select the better optimizer, the optimizers of the stochastic gradient descent algorithm (SGD) (Sharma, 2018), Momentum algorithm (Li et al., 2021), Adagrad algorithm (Traoré and Pauwels, 2021), RMS prop algorithm (Xu et al., 2021), and Adam algorithm (Jais et al., 2019) are compared with the gradient descent algorithm. Table 4 shows the performance of the network model in different optimization algorithms.
In Table 4, when the network model adopts the Adam optimizer, the training time of the network model is the shortest and the recognition rate is the highest, so the Adam optimizer is used to identify the safety wear of electric power workers.
The loss function in the algorithm of safe wear recognition is designed as follows:
In 4), y is the label value; s is the predicted value of the network forward propagation.
The back-propagation process is as follows:
3.5 Evaluation indicators
To evaluate the recognition effect, the average accuracy index is used to measure the matching accuracy of the detection frame to the target object, it can be expressed as follows (Chen et al., 2022b):
The average precision metric is:
4 Experimental verification and analysis
4.1 Parameter settings
During the network training process, the early-stopping mechanism is used to prevent the network from overfitting, and the parameter is set to 15. Training is stopped when the loss on the training set drops while the loss on the validation set remains the same for 15 consecutive epochs. To avoid the network from overfitting, the weight of the network model is only taken as a small value to limit the complexity of the network model and make the weight distribution more regular. This is weight regularization, adding the cost associated with larger weights to the network loss function to make the absolute value of the weight coefficients small enough. Based on the L2 regularization method (Chen and Zhao, 2021; Wang et al., 2021; Liu et al., 2023), the formula is as follows:
The training parameters are set as shown in Table 5.
4.2 Experimental results
The training time of the improved CNN network model is 1531s. The accuracy curve of the network training curve is shown in Figure 2 and the loss change curve of the network training curve is shown in Figure 3.
In Figures 2, 3, we can see that the accuracy in the early stage of training gradually increased, and the loss of the training set and the validation set also gradually decreased. There was a slight oscillation in the middle, and it gradually became stable with iterations. The network accuracy eventually tends to 100%, and the network loss eventually tends to 0. When the epoch is equal to 14, the loss of the training set is still decreasing, the loss of the validation set tends to remain unchanged, and when the loss of the validation set does not change for 10 consecutive times, the early-stopping mechanism will end the training of the network in advance, so the final training is taken. The network model was obtained 25 times. The safety wear recognition results of the electric staff working are shown in Table 6.
To verify the superiority of the improved CNN algorithm, the network model proposed in this paper is compared with the other network model under the same parameters. The performance comparison result is shown in Table 7.
In Table 7, the recognition method used in this paper has the highest recognition accuracy, reaching 96%. The highest recognition rate of other algorithms is 92.5%. Although these recognition methods solve the problem of image recognition to some extent, when the image features are too mixed, these algorithms cannot well complete the mapping from feature extraction to state recognition, which is not conducive to model recognition. Compared with other traditional algorithms, the improved deep learning algorithm has a higher recognition rate and significantly improves the recognition rate of safe wearing of electric power workers. Compared with other algorithms, the proposed method has faster running speed and can meet the requirements of real-time computing. The proposed network model has strong feature expression ability. The advantages of good generalization performance and strong robustness.
In this paper, through the detection and identification of safety helmets, work clothes, and insulating gloves to carry out normative identification and warning, a deep learning-based safety wear identification method for electric power workers is proposed. The experimental results show that: compared with other traditional algorithms, the improved deep learning algorithm proposed in this paper has a higher recognition rate, significantly improves the recognition rate of safe wearing of electric power workers, and has the advantages of high accuracy and good real-time performance, thus providing the practical work provides guidance.
Data availability statement
The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.
All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.
I and my fellow co-authors are fully aware of and agree with the payment of the listed article processing fee should the manuscript be accepted for publication.
Conflict of interest
Authors ZC, CM, JR, FH, and ZW were employed by the company Guangdong Power Co., Ltd.
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Anastasiadou, K., Gavanas, N., Pitsiava-Latinopoulou, M., and Bekiaris, E. (2021). Infrastructure planning for autonomous electric vehicles, integrating safety and sustainability aspects: A multi-criteria analysis approach. Energies 14 (17), 5269–5274. doi:10.3390/en14175269
Chen, Y., and Zhao, Q. (2021). Mineral exploration targeting by combination of recursive indicator elimination with the ℓ2-regularization logistic regression based on geochemical data. Ore Geol. Rev. 135 (2), 104213. doi:10.1016/j.oregeorev.2021.104213
Chen, Z., Jin, T., Zheng, X., Liu, Y., Zhuang, Z., and Mohamed, M. A. (2022). An innovative method-based CEEMDAN–IGWO–GRU hybrid algorithm for short-term load forecasting. Electr. Eng. 104, 3137–3156. doi:10.1007/s00202-022-01533-4
Dourado, C. M. J. M., da Silva, S. P. P., da Nobrega, R. V. M., Reboucas Filho, P. P., Muhammad, K., and de Albuquerque, V. H. C. (2020). An open IoHT-based deep learning framework for online medical image recognition. IEEE J. Sel. Areas Commun. 39 (2), 541–548. doi:10.1109/jsac.2020.3020598
Gangolells, M., Casals, M., Forcada, N., Roca, X., and Fuertes, A. (2010). Mitigating construction safety risks using prevention through design. J. Saf. Res. 41 (2), 107–122. doi:10.1016/j.jsr.2009.10.007
Liu, J., Jia, R., Li, W., Ma, F., Abdullah, H. M., Ma, H., et al. (2020). High precision detection algorithm based on improved RetinaNet for defect recognition of transmission lines. Energy Rep. 6, 2430–2440. doi:10.1016/j.egyr.2020.09.002
Liu, L., Wang, B., Ma, F., Zheng, Q., Yao, L., Zhang, C., et al. (2022). A concurrent fault diagnosis method of transformer based on graph convolutional network and knowledge graph. Front. Energy Res. 10, 127. doi:10.3389/fenrg.2022.837553
Liu, Y., Yuan, D., Gong, Z., Jin, T., and Mohamed, M. A. (2023). Adaptive spectral trend based optimized EWT for monitoring the parameters of multiple power quality disturbances. Int. J. Electr. Power & Energy Syst. 146 (2023), 108797. doi:10.1016/j.ijepes.2022.108797
Mezgec, S., Eftimov, T., Bucher, T., and Korousic Seljak, B. (2019). Mixed deep learning and natural language processing method for fake-food image recognition and standardization to help automated dietary assessment. Public health Nutr. 22 (7), 1193–1202. doi:10.1017/S1368980018000708
Postalcıoğlu, S. (2020). Performance analysis of different optimizers for deep learning-based image recognition. Int. J. Pattern Recognit. Artif. Intell. 34 (02), 2051003–2051213. doi:10.1142/s0218001420510039
Ramcharan, A., Baranowski, K., McCloskey, P., Ahmed, B., Legg, J., and Hughes, D. P. (2017). Deep learning for image-based cassava disease detection. Front. plant Sci. 8 (14), 1852–2201. doi:10.3389/fpls.2017.01852
Tang, H., Liu, H., Xiao, W., and Sebe, N. (2020). When dictionary learning meets deep learning: Deep dictionary learning and coding network for image recognition with limited data. IEEE Trans. neural Netw. Learn. Syst. 32 (5), 2129–2141. doi:10.1109/tnnls.2020.2997289
Wang, H., Wang, B., Luo, P., Ma, F., Zhou, Y., Mohamed, M. A., et al. (2021). State evaluation based-feature identification of measurement data for resilient power system. CSEE J. Power Energy Syst. 8, 983–992. 10.17775/CSEEJPES.2021.01270.
Xu, D., Zhang, S., Zhang, H., and Mandic, D. P. (2021). Convergence of the RMSProp deep learning method with penalty for nonconvex optimization. Neural Netw. 139, 17–23. doi:10.1016/j.neunet.2021.02.011
Keywords: deep learning, convolutional neural network, electric power work, safety wear recognition, safety
Citation: Chen Z, Ma C, Ren J, Hao F and Wang Z (2023) Research on the identification method of safety wearing of electric power workers based on deep learning. Front. Energy Res. 10:1091322. doi: 10.3389/fenrg.2022.1091322
Received: 06 November 2022; Accepted: 30 December 2022;
Published: 12 January 2023.
Edited by:Cong Qi, China University of Mining and Technology, China
Reviewed by:Zhekang Dong, Hangzhou Dianzi University, China
Tiantian Chen, China University of Mining and Technology, China
Copyright © 2023 Chen, Ma, Ren, Hao and Wang. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Zengyu Wang, firstname.lastname@example.org