Many research studies have reported the application of deep learning for image processing, mainly for concrete structures and road condition assessment. This section presents a review of these applications starting with road condition assessment, followed by concrete structure condition assessment, and then wrapping up the review with the sparse application for roof assessment necessitating the need for this study. The first subsection below will summarize the work that has been done in the area of road assessment using Artificial Intelligence methods.
Road condition assessment
Many researchers have been working to optimize road conditions assessment, an important research area worthy of exploration for instance due to the high cost associated with road maintenance and the fact that road infrastructure globally is always not in its best shape. Ukhwah et al. (2019) proposed a novel approach for asphalt pavement road detection using YOLO neural network. Different YOLO versions such as Yolo v3, Yolo v3 Tiny, and Yolo v3 SPP are evaluated based on accuracy and mean average precision. The results show that mAP of Yolo v3, Yolo v3 Tiny, and Yolo v3 SPP were 83.43%, 79.33%, and 88.93%, with the accuracy of 64.45%, 53.26%, and 72.10%, respectively. It takes only 0.04 s to detect the image, which signifies the robustness of their proposed models. In a recent study, (Mubashshira et al., 2020), proposed a new model for crack detection on roads using unsupervised learning of deep learning technology. Color histograms of roads are used for the analysis of road pavements in their study. Afterward, K clustering and OTSU thresholding were employed for segmentation in order to detect the cracks. Results showed that the model performed well in detecting and localizing the damage in the image. In another study, (Sizyakin et al., 2020) proposed a new model for crack detection on roads using deep learning. In their work, they used the publicly available dataset CRACK500 and trained them on the state-of-the-art object detection algorithm U-net. Moreover, the morphological fitting is used to increase the binary map so the model can detect cracks with higher accuracy. Aravindkumar et al. (2021) proposed a model for automatic crack detection on roads using the deep learning technique. A total of 3,533 road images were taken as a dataset of roads of Tamil Nadu, India. They used the transfer learning technique using ResNet-152 classifier with the incorporation of a faster region-based convolutional neural network (Faster R-CNN). Images were trained based on a variety of damages and cracks and then notifying the relevant authority about the locations of these damages. From the results, it was concluded that their model worked effectively.
Bhat et al. (2020) proposed a model for crack detection on roads using deep learning technique. In their work, they exploited various models from traditional CNN and image processing to segmentation. A variety of classifiers are also analyzed, however, based on the result and findings CNN outperformed all other models due to its high accuracy. Elghaish et al. (2021) proposed a model for the classification and detection of highway cracks using deep learning. A total of 4,663 images were taken as dataset which were classified into subparts as “horizontal cracks”, “horizontal and vertical cracks” and diagonal cracks. State of the art object detection algorithms were used and a CNN model was develop in order to increase the accuracy of the proposed model. From the results it was concluded that the pre-trained model googlenet has higher accuracy of 89.08%. However, the new created CNN model exceeded the accuracy of all other algorithms with 97.62% accuracy rate with the use of adam’s optimizer.
Bhavya et al. (2021) proposed a model for pothole detection using deep learning using classification techniques. Images that were further divided into roads with potholes and roads without potholes were taken. They used a state-of-the-art pre-trained model Resnet-50 as a classifier. After the results it was concluded that the model can accurately classify between different kinds of roads and can replace external manpower in assessing road conditions. Ping et al. (2020) proposed an efficient deep learning model for detecting potholes using state-of- the-art objects detection models such as YOLO, SSD, HOG with SVM, and Faster-RCNN. The images were pre-processed and labeled accordingly. Their results showed that YOLO performed better than other object detecting for its higher accuracy and faster computation. Pan et al. (2018) proposed a new model for pothole detection on roads using deep learning and UAVs. In their study, spectral images were taken using UAVs and by the use of machine learning algorithms such as support vector machine (SVM) and random forest. Data is classified into normal pavements and pavements with potholes and cracks. They evaluated different models in their research and concluded that came to the conclusion that remote sensing via UAVs can offer an alternative to traditional assessment of roads.
Arman et al. (2020) presented a model for classification and detecting of road damage using deep learning models. Images were taken from a smartphone camera in the city of Dhaka. Damages such as a pothole, crack, and revealing were taken into account for this model. Their model’s images were fed into Faster RCNN, and RCNN for object detection and support vector machine algorithm of machine learning is used for classification purposes. The result demonstrated that they achieved 98.88% damage detection and classification accuracy. Wang et al. (2018) proposed a deep learning model for damage detection and classification in road networks. They used the SSD and Faster R-CNN in their work for detection and classification. The results were demonstrated in the IEEE BigData Road Damage Detection Challenge, and according to the results, their model performed better. Alfarrarjeh et al. (2018) suggested a deep learning model for road damage detection using images taken from mobile. According to their model, they trained their object detector algorithm on various images of damages as defined by Japan Road Association. Results show that their model performed well despite low imagery resolution and achieved an F1 (Accuracy) score up to 0.62. Seydi et al. (2020) proposed a model for the assessment of roads network damages caused by an earthquake. They used LIDAR point cloud in their model. The proposed model is based on three steps. In the first step, features of LIDAR data were extracted using CNN. In the second step, another neural network: Multilayer perceptron (MLP), is used to detect the debris in the data (images). The output from MLP was fed into another neural network to classify the road segments into blocked and unblocked. Their model performed well and achieved an accuracy of 97%.
Liu et al. (2020) proposed a novel approach for detecting damages inroads using a deep learning model. At first, they used segmentation technique to detect roads areas, then data was fed into object detection models You look Only Once (YOLOv4) and Faster Regional Convolutional Neural Network (F-RCNN) for detecting the damages. The result shows that the proposed model achieved good object detection capability in the IEEE Global Road Damage Detection Challenge 2020. Ale et al. (2018) proposed a faster and more accurate deep learning model for damages to the roads. In their model, classification and bounding box regression are done in a single stage as it is much faster than two stages, which are classification in one step and bounding box regression in the second step. They have trained the images on several one-stage models and inferred that a model RetinaNet can detect road damages with high accuracy. Bojarczak et al. (2021) proposed a model damage diagnosis of railways using a deep learning model and UAVs. They used a fully convoluted network (I.e., semantic segmentation) to locate the railhead’s defects. Their model is based on an OpenCV python library’s tensor flow work environment. The results showed that their model could accurately locate defects in the railway with an efficiency rate of 81%.
To summarize, this section shows that deep learning can be used successfully to diagnose and identify damage (including potholes and cracks) on asphalt roads and highways. The next subsection will discuss the assessment of concrete structures for automated crack detection using deep learning and other techniques.
Concrete structures condition assessment
Concrete structure condition assessment has been an interest to the Federal Department of Transportation for many years, especially the assessment of the bridge’s conditions. Otero et al. (2018) proposed a model for remote sensing of concrete bridge inspection using deep learning technology. In their work unsupervised learning approach is used to extract the features, i.e., cracks from the images. The dataset consisted of different kind of damages along with noise factor. After testing the model, they evaluated that algorithm can successfully detect cracks in different kind of images. Chen et al. (2020) proposed a model for damage detection in concrete bridges using a deep learning model. Their model is based on the technique of transfer learning, taking an existing state-of-the-art object detection algorithm YOLOv3 is used for object localizing. For the extraction of small features, deformable convolution is used. The tests showed that their model is more effective and takes less time in computation.
Kim et al. (2020) proposed a deep learning model for the detection of multiple cracks in concrete. Their model used Masked-RCNN, images of cracks, efflorescence, rebar exposure, and spalling were trained using Masked-RCNN. According to the results, their model achieved a precision of 90.4%, which promises the applicability of Masked-RCNN in damage detection. Rajadurai et al. (2021) proposed a model for using deep learning for automated crack detection in concrete. Transfer learning technique is used, and images (with cracks and no cracks) were fed into Alexnet model. Stochastic gradient descent was used as an optimizer to prevent high loss. Their model showed an accuracy of 99.99% when tested with test images. Cha et al. (2017) came up with a novel approach to detecting cracks in concrete automatically, and they used a convolutional neural network for their model. The model was trained on 40,000 images with a 256 × 256 pixels resolution. Their model outperformed all previous work on image processing and achieved 98% accuracy. Furthermore, they validated their result by testing data of 50 images of 5,888 × 3,584-pixel resolution. The result showed that their model could work in real-time situations.
Desilva et al. (2018) proposed a model for automatic detection of concrete cracks via deep learning and UAVs. A dataset of 3500 images was taken with different conditions such as daylight intensity, surface finishes, and humidity. The dataset was divided into training and testing with a ratio of 80/20. They used VGG-16 on their dataset, on which the model performed well. The overall accuracy of this model is 92.27% which shows the potential of deep learning in the detection of concrete cracks. Kumar et al. (2022) suggested a model for real-time monitoring of cracks in high rises concrete building. A total of 800 RGB pixel images (480 × 480 pixels) were fed into the You Look Only Once (YOLO-3) algorithm as input collected using UAVs. Images were annotated manually using open-source software. Their model outperformed all other previous work achieving an accuracy of 94.24%, and their model can process an image in 0.033 s.
All in all, detecting cracks in concrete automatically has been investigated by many researchers. Deep learning has been used successfully for this purpose. The next subsection will discuss the automation of roof condition assessment using deep learning and other Artificial Intelligent techniques.