A Novel Piezoceramic-Based Sensing Technology Combined With Visual Domain Networks for Timber Damage Quantification

Early detection of timber damage is essential for the safety of timber structures. In recent decades, wave-based approaches have shown great potential for structural damage assessment. Current damage assessment accuracy based on sensing signals in the time domain is highly affected by the varied boundary conditions and environmental factors in practical applications. In this research, a novel piezoceramic-based sensing technology combined with a visual domain network was developed to quantitatively evaluate timber damage conditions. Numerical and experimental studies reveal the stress wave propagation properties in different cases of timber crack depths. Through the spectrogram visualization process, all sensing signals in the time domain were transferred to images which contain both time and frequency features of signals collected from different crack conditions. A deep neural network (DNN) was adopted for image training, testing, and classification. The classification results show high efficiency and accuracy for identifying crack conditions for timber structures. The proposed technology can be further integrated with a fielding sensing system to provide real-time monitoring of timber damage in field applications.


INTRODUCTION
Timber structures have been increasingly utilized in building construction due to their excellent seismic performance and environmental friendliness, which mean flexible building function, especially with the improvement of engineering wood products (Cao et al., 2019;Chen et al., 2020;Sun et al., 2020). A variety of novel structural systems have been designed and built into landmarks. Wood cracking is an inevitable issue for timber structures because of the variation of temperature and humidity (Li et al., 2013;Dietsch and Winter 2018). Wood cracks could cause potential safety hazards and increase maintenance costs. Therefore, a reliable crack detection technology for extensive in-service and newly constructed timber structures is of great importance to ensure the safety of timber structures.
Existing approaches for timber cracks detection mainly include resistance drilling measurement and scanning image technique (Wei et al., 2011;Brites et al., 2012;Tannert et al., 2014;Kloiber et al., 2015;Zhang et al., 2015;Mol et al., 2020). Drilling measurement is a conventional method to assess the internal condition of timber members (Mol et al., 2020). However, this method is semidestructive, and the results are highly affected by the test personnel. The scanning image method utilizes different levels of absorbance between crack and health wood for external waves, such as microwaves (Baradit et al., 2009), X-rays (Wei et al., 2011;Pease et al., 2012;Sandak et al., 2015a), and infrared rays (Sandak et al., 2010;Sandak et al., 2015b), to obtain visualized information of the scanning section. However, this method is a qualitative assessment and limited to relatively high cost for real projects.
Compared to the existing wave-based inspection technology, the stress wave shows potential for timber crack detection since stress wave-based inspection is easy to perform with portable devices for in situ assessment. According to frequency bands, stress wave-based approaches can be divided into acoustic tomography (low frequency) (Dackermann et al., 2014a;Riggio et al., 2014;EI-Hadad, 2017) and ultrasonic echo method (high frequency exceeds 20 kHz) (Puaad et al., 2014;Koca et al., 2018;Linke et al., 2019). The detection principles, equipment, and applications have been well demonstrated in the literature (Dackermann et al., 2014b;Krause et al., 2015). Conventional data processing for stress wave-based crack detection measures the change of wave velocity in timber structures. However, wave velocity is highly affected by various factors, such as moisture content, wood species, and orientation of growth rings. All mentioned factors bring uncertainties to detection results. Furthermore, wave velocity is normally obtained by the measurement of time-of-flight of the timedomain probing signal. The influences of damage on the frequency domain of the probing signal cannot be investigated (Lee and Shin, 2002).
For obtaining frequency features to achieve high accuracy in damage identification, time-frequency technologies have been widely applied in radar signal recognition (Ahmad et al., 2020), equipment fault detection (Ulloa and Barbieri, 2018), and nondestructive testing (Obuchowski et al., 2014;Dorafshan and Azari, 2020;Li et al., 2020). Gong et al. (Gong et al., 2020) proposed an algorithm for the automatic extraction of the stress wave reflection period based on image processing to measure the different lengths of buried metal piles in soil. Le et al.  adopted time-frequency analysis to extract the instantaneous frequency of a vehicle-bridge interaction system and evaluated the bridge' state. Hui et al. (Bao et al., 2018) used the deep learning neural network to identify different time-series images of signal faults in bridge health monitoring. However, there is still no study using time-frequency analysis combined with visual domain networks to detect the timber cracks.
In this paper, the authors propose a novel piezoceramic-based sensing technology combined with the vision classification algorithm for timber crack detection. The proposed visual domain method replaces the conventional way (i.e., damage index) of quantifying the variation of the stress wave signal in the time domain. The received stress wave signal was transformed into spectrograms using the short-time Fourier transform (STFT). Then, the dataset of each case with the label of crack depths was fed into a deep learning network for classification. Numerical simulation was performed to illustrate the stress wave propagation properties when passing through timber cracks in different depth cases. A series of experimental investigations, including a total of three timber beam specimens with seven crack depth conditions in each, were conducted to validate the feasibility and accuracy of the proposed technology.

METHODOLOGY
The proposed approach consists of three major steps, as detailed in Figure 1: 1) data pre-processing by conducting joint time-frequency analysis; 2) data augmentation for expanding the dataset and simulating environmental uncertainties in operation; and 3) deep neural network (DNN) training for crack depth classification. Spectrograms of different crack depths are obtained in data pre-processing using the shorttime Fourier transform (STFT). Three data augmentation methods include adding white noise to wave data in the time series, adding Gaussian noise to the spectrum, and jittering spectrums, which will not change the main characteristics of signals from the statistical perspective (Tanner, 2014). After data augmentation, seven spectrum image datasets with the label of crack depths are fed into the proposed DNN to train a model and consequently identify damage severity.

Stress Wave-Based Sensing Technology Using Piezoelectric Transducers
Piezoelectric materials, such as lead zirconate titanate (PZT), have attracted increasing attention in real-time monitoring of structural damage for their high stability, small size, great linearity, and piezoelectric effects (Yang et al., 2018;Yuan et al., 2020). The feasibility, applicability, and reliability of PZT sensors have been widely validated in the past decade (Dansheng et al., 2016;Zhang et al., 2018a;Zhang et al., 2018b;Han et al., 2019). PZT-based detection mainly includes stress wave sensing and electro-mechanical impedance (EMI) method (Huo et al., 2017;Zhang et al., 2021), while the high-cost equipment requirement impedes practical applications of the EMI method. Furthermore, because of the larger range of perception and the indirect and the direct piezoelectricity of the PZT material, stress wave sensing is adopted in this research, in which PZT-based transducers can function as both actuators and sensors.
In this study, a pair of PZT patches (an actuator and a sensor) is mounted on the timber surface to generate and detect stress waves, respectively. Figure 2 shows the stress wave propagation mechanism when a crack occurs on timber. The propagation property is affected by the internal condition in the wave path. The existence of cracks in the timber beam could transform the propagation path and wave pattern. When a crack is present, stress waves will reflect and diffract at the damage interface, causing the energy drop-down on the other side of the crack. In Figure 2, it is obvious that the stress wave energy attenuates due to reflection and diffraction at crack interfaces.

Generation of Spectrum Image Data
The time-series signal received by piezoelectric transducers only contains information of the time domain, which does not consider the influence of cracks on frequency components.   Thus, time-frequency analysis including both time and frequency information of the received signal is utilized in this research to obtain spectrum images of the received signal.
In this generation process of image data, time series from tests are transformed into frequency spectrums by using short-time Fourier transform (STFT). For an original discrete signal sequence x(n), a pre-determined window function is used to divide the time series into many segments, and it is assumed the signal is pseudo-stationary over a short interval, and then Fourier transform is carried out on each window length (Bendory et al., 2017;Rashid and Louis, 2020). The transform process of STFT is shown in Figure 3. Both time-and frequency-domain characteristics of timber cracks are contained in STFT spectrums; thus, more damage features can be learned in the training process of the neural network. The calculation of STFT is shown in the following equation: where F(mΔt, f ) is a two-dimensional complex matrix representing STFT results, g(n) is the length of the window function (chosen as the Hamming window), and Δt is the hop size in samples between successive discrete Fourier transforms.

Process of Classification Training
STFT spectrum images contain multi-domain feature information of each damage case. Since the DNN has validated significant performance in the classification of images (Shukla and Piratla, 2020), the timber crack detection is conducted using a transferred DNN. The AlexNet network is transferred in this study to imitate the human decision-making process for a deep understanding of the characteristics of STFT spectrums in different crack conditions. The AlexNet network consists of eight layers, including five convolution (Conv) layers and three fully connected (FC) layers (Krizhevsky et al., 2012), as shown in Figure 4. As shown in Figure 4, damage spectrums are firstly normalized to the size of 227 × 227 × 3 pixels in the input layer, and then 96 kernels are used to extract edge features in the first convolutional layer. The feature map obtained from the former convolutional layer is then put into the ReLU layer and is mapped non-linearly using activation function (i.e., rectified linear unit, ReLU). The ReLU layer's output then gets into the pooling layer, in which salient features are extracted by the maxpooling algorithm. These steps are repeatedly conducted in the following four convolutional layers with different kernels and padding sizes. With the increase of convolutional layers, the extracted feature number is significantly increased. After that, feature maps are connected to FC layers. The dropout layer can avoid the occurrence of overfitting. The classification results can be obtained in the output layer using softmax function to normalize possibility in each class.

Model Setup
Numerical simulations were performed in Abaqus. The mechanical properties of timber were set as orthotropic, and detailed constants are listed in Table 1. The parameter E represents the elastic modulus, G is the shear modulus, and µ is Poisson's ratio. The indexes L, R, and T represent the longitudinal direction, radial direction, and tangential direction, respectively.
The material properties of PZT patches were adopted values in official documents of Abaqus 1 . The dielectric constant, engineering constants, and piezoelectric coupling matrix are listed in Table 2.
The interfaces between timber and PZT patches were set as the connection type of tie. The solid element C3D8R was adopted for the timber beam, and the solid element C3D8E with piezoelectric effects was used for PZT patches. The excitation signal was produced by setting different electrical potentials at the top and bottom surfaces of PZT patches. In particular, the electrical potential at the bottom surface was set as zero, and at the top surface, ten voltages with a pre-determined amplitude were set. The excitation load is a one-cycle sine wave with a frequency of 190 kHz. The implicit dynamic solver was used with a time period of 0.0001 s, and the time iteration was 1e-7. Figure 5 shows specimen dimensions and PZT locations in the established model. Figure 6 depicts a three-dimensional propagation path of stress waves.

Effects of the Crack on Stress Wave Propagation
To investigate the effect of cracks on stress wave propagation, four crack depths were performed. Figure 7 shows wavefronts of four damage cases in the 220th iteration step. Reflections are observed at the left and side boundaries, and wavefronts show the same pattern before passing through cracks, which means incident waves into the crack interface have same properties and differences as received signals that are only caused by cracks. Figure 8 shows strain contours when stress waves are passing through the crack interface in the 400th iteration step. Reflection waves caused by the crack are obvious in all damage cases. The transmission wave is only observed at the crack depth of 4 mm, which indicates wave velocity decays when the stress wave passes through cracks.
After passing through cracks, the characteristics of stress waves show extreme differences (as depicted in Figure 9). At the same time in the 470th iteration step, the wavefronts drop down rapidly with the increase of crack depths, representing the attenuation of wave velocities. At the same position of the transmission wave, the stress wave intensity reduces significantly with the increase of crack depths.

EXPERIMENTAL VERIFICATION Laboratory Test Specimen
Three timber specimens (pine wood) with the same dimensions (300 mm × 100 mm × 50 mm) were prepared at the State Key Laboratory of Tongji University. For simulating timber cracks, 2-mm-width cracks within seven different depths, listed in Table 3, were cut off on the top surface perpendicular to the Note: numbers 1, 2, 3 refers to the direction. Direction 1 represents the length direction of PZT patch, direction 2 refers to the width direction of PZT patch, and direction 3 refers to the thickness direction of PZT patch.
FIGURE 5 | Assembly model. grain direction. Two PZT patches were mounted at predetermined locations using adhesive (Comix super glue B2695), and the distance is 50 mm from two adjacent ends of the timber specimen, as shown in Figure 10. The dimensions of PZT patches are 10 mm, 10 mm, and 2 mm in length, width, and depth, respectively.

Experimental Setup
The experimental apparatus includes timber specimens, a data acquisition system, and a monitoring visualization system, as depicted in Figure 11. Timber specimens are fixed by two fixtures for simulating the boundary conditions in real cases. On the top surface of specimens, two PZT patches were used for transmitting and receiving signals. One PZT patch, excited by a one-cycle sine wave with a frequency of 190 kHz, is connected to the acquisition system with 2 MS/s sampling frequency. The data acquisition system consists of an acquisition card (NI USB 6366) and a signal power amplifier (Trek model 2100HF). The other PZT connected to the acquisition system is used to receive stress waves and displace the signal response in the visualization system.

Time-Domain Signal Collection
The time-domain probing signals for three specimens are shown in Figure 12. The time duration of each specimen is 0.5 ms. A general trend of all the test specimens shows two significant wave packets in time response. The first wave packet is the signal directly received from the probing signal, and the second wave packet is the signal received from boundary reflections. The signal amplitudes decrease with the increase of crack depth in the first wave packet. The second wave packet has no similar trend due to the multi-boundary reflection effects. Because of the large material dispersion of timber, time-domain signals are very difficult to process and classify the damage cases. Thus, time-frequency spectrums in terms of images are established from each time-domain signal for training the developed DNN.

Data Training Process
Since original signals from tests are limited to training neural networks, three data augmentation methods are used to expand datasets in both the time domain and the frequency domain. Gaussian noises with certain signal-to-noise ratios were added to raw data, and all raw data were transformed into frequency spectrums by STFT. In addition, salt and pepper noises were added into spectrum images of original signals, and spectrum images were jittered into 8 channels and 16 channels, respectively. Each original signal was expanded to 27 images after data augmentation. Spectrums from different specimens but for the same crack depth were mixed into a separate dataset. A total of 567 images were obtained in this research, as shown in Figure 13.

Classification Results
Visual domain network-based image identification for the spectrum dataset was conducted. Seven datasets with 567 images are mixed and fed into the transferred AlexNet network. The learning rate is set as 0.0001, and a total of 200 epochs are set. In each epoch, images are randomly split into the training set and validation set. As shown in Figure 14, the training dataset accounts for 70% of the total images, and the validation dataset accounts for 30%. As shown in Figure 15, the   predicted accuracy rapidly increases with the addition of iterations. When the number of iterations exceeds 2,000, the prediction accuracy approaches a constant value of 95.83%. Figure 16 illustrates the confusion matrix regarding the classification accuracy of the validation dataset. The average accuracy rate is 95.8%. The crack depth of 8 mm has the highest accuracy of 100%. The crack depths of 0 mm, 2 mm, 4 mm, and 10 mm have the same accuracy with a 4.3% probability of wrong predictions, and the crack depth of 20 mm has a lower accuracy of 91.7%.

Discussions
The classification results from the transferred DNN show the feasibility of the proposed technology to identify timber crack severity. Compared with traditional signal processing of the stress wave method, the proposed method based on STFT spectrums and computer vision technique shows the advantage in generality and the extensibility of the dataset. However, adopting this approach in field applications still faces a few challenges. In particular, four aspects should be considered in the future work: 1) The detection accuracy is highly limited to the size of the training dataset, which requires continuously expending the current dataset to improve the stability and adaptability of the developed DNN-based classification. 2) Only timber beam elements are investigated in the experimental study, whereas the boundary conditions and dimensional features may be different from other timber structural components, such as columns, wood slab, and timber connections. All mentioned factors may influence the reflection pattern of stress waves.
3) The performance of the proposed method in identifying cracks in propagation is not clear. Therefore, one possible solution is to collect stress wave signals over the whole loading process and then divide these raw data and their corresponding STFT spectrums into different stages according to crack propagating severity and then use the computer vision approach to identify crack propagation. 4) The form of sensor used in this research is surface-mounted patches, whose performance is easily affected by environmental variations and service life. Proper sensor protection should be considered to ensure its long-time service for structural health monitoring of timber structures.

CONCLUSIONS
This research proposes a novel piezoceramic-based sensing technology combined with visual domain networks for timber crack detection. Surface-mounted piezoelectric transducers were utilized to transmit and receive stress waves in both numerical simulations and experiments. The numerical and experimental results reveal the wave propagation properties in different cases of crack depths. A total of 567 images were reconstructed by transferring the time-domain signal to time-frequency spectrums. Respective labels from the dataset were trained through the transferred DNN, and the predicted

DATA AVAILABILITY STATEMENT
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

AUTHOR CONTRIBUTIONS
HX involved in data analysis. LC wrote the paper. CY contributed to FEM modeling. QK provided the idea and revised the paper.