Machine learning-based classification of seismic events: a case study of seismic events in Jilin Province, NE China

Introduction: The rapid and accurate identification of natural and non-natural seismic events is crucial for compiling comprehensive earthquake catalogs and assessing regional seismic risk.

Methods: This study utilized waveform data from seismic events (1.5 ≤ ML ≤ 3.5) recorded in Jilin Province between 2013 and 2024. Multiple physical features were extracted from the waveforms, and Support Vector Machine (SVM), Extreme Gradient Boosting (XGBoost), and Backpropagation Neural Network (BPNN) algorithms were employed to classify artificial blasts, mining collapses, and tectonic earthquakes.

Results: The study shows that all three models achieved over 94% accuracy in regional tests, with the SVM model performing best. In cross-regional validation, SVM maintained superior generalization capability, achieving an accuracy of 84%. Feature importance analysis confirmed that P-/S-wave spectrum amplitude and maximum P/S amplitude ratio served as the most critical features for event discrimination.

Discussion: This study validates the effectiveness of the selected features and machine learning methods for identifying low-magnitude multi-type seismic events; however, for events with strong regional characteristics, further development of cross-regional datasets or region-specific models is necessary to enhance classification accuracy.

1 Introduction

Rapid classification of seismic events is essential in seismic monitoring operations. Seismic networks record events originating not only from natural sources (such as tectonic earthquakes, volcanic activity, and meteors) but also from artificial sources, including explosions, collapses, and human-induced sonic booms (Alvizuri et al., 2021). Such events may attract public attention, and in severe cases, they could have adverse effects on social and economic stability as well as public safety.The increasing detection of non-natural earthquake events makes manual differentiation methods insufficient to meet the timeliness requirements for response (Zhou et al., 2021). Furthermore, most such events are of small magnitude, complicating detection and identification. Therefore, developing rapid classification methodologies is critical to enhance the output quality and operational efficiency of seismic monitoring and catalog compilation.

Research on seismic event classification originated from nuclear explosion monitoring and identification. Early studies by domestic and international scholars typically discriminated between explosions and earthquakes using waveform characteristics from different sources, including first-motion polarity, mb/MS ratio, hypocentral location, P/S amplitude ratio, and signal duration (Taylor et al., 1989; Bian, 2005; Wang and Bian, 2011). With the continuous evolution of industrial activities, monitored data from blasts, collapses, and geohazard events have substantially increased, prompting researchers to explore new discriminative features through single or combined characteristic analyses. Yang et al. (2020) introduced the second-order moment of spectrograms based on time-frequency spectral differences to distinguish earthquakes from explosions. Wang et al. (2020) established a discriminant relationship between the P/S amplitude ratio and the difference between local magnitude (ML) and coda magnitude (MC) to differentiate blasts from natural earthquakes.

Entering the era of intelligence, classification methods based on artificial intelligence have gradually become mainstream. In the field of supervised learning, Techniques such as Backpropagation Neural Network, Convolutional Neural Network (CNN), SVM, and Adaptive Boosting (AdaBoost) have been applied for this purpose (Bian, 2002; Bi et al., 2011; Zhao et al., 2019; Zhao et al., 2017). Research indicates that neural network approaches often achieve high identification performance. For instance, Yildirim et al. (2011) attained identification accuracy exceeding 97% using feedforward and probabilistic neural network. Shang et al. (2017) confirmed artificial neural networks (ANN) as optimal classifiers through testing of 1,600 events. Kong et al. (2022) integrated physical features with CNN to obtain high precision with robust generalization capabilities. However, It is also postulated by certain scholars that the network structure is related to the number of data samples. In comparison with neural network methods, support vector machines have been shown to offer distinct advantages in the context of nonlinear and high-dimensional pattern recognition with limited sample sizes. Saad et al. (2019) employed SVM to discriminate earthquakes from quarry blasts, attaining 98.5% accuracy on 900 events. Liang et al. (2023) attaining 99.2% accuracy for Northeast China events. Alternatively, ensemble learning algorithms have also yielded promising results. For instance, Ren et al. (2019) applied Bagging to achieve accuracy surpassing 85% within constrained epicentral distances. Li et al. (2025) applied K-Nearest Neighbor (KNN), AdaBoost and Light Gradient Boosting Machine (LGBM) algorithms to identify tectonic earthquakes, blasts, and collapses in North China, with all models exceeding 90% accuracy.

Semi-supervised and unsupervised learning are becoming increasingly valuable in seismic event classification, particularly when dealing with continuous data with limited or no labels. Unsupervised learning serves as an exploratory tool to identify potential clusters and anomalies through data clustering, guiding the discovery of new event categories. For example, Mousavi et al. (2019) employed a deep generative model (Variational Autoencoder) to perform unsupervised clustering of seismic signals, automatically revealing distinct patterns and sources from continuous data without relying on prior labels. The clusters generated through unsupervised learning can be used to create pseudo-labels, which then guide semi-supervised algorithms. Semi-supervised learning utilizes both a small number of expert-verified labels and these pseudo-labels to classify larger datasets. Murray et al. (2023) introduced a semi-supervised dual-network approach for clustering seismic events in remote Scottish hillsides, effectively identifying earthquakes, rockfalls, and microseismic events. These strategies enable efficient labeling of large datasets by learning from limited expert-confirmed examples, significantly reducing reliance on fully annotated datasets.

While existing methods demonstrate high classification performance, challenges remain in cross-regional generalization. These are primarily attributed to variations in data processing methods, event selection criteria, and region-specific characteristics of non-tectonic event recordings. Tang et al. (2020) observed that propagation path effects cause regional differences in the time-frequency features of non-tectonic events, degrading model generalization. Similarly, Wang et al. (2024) corroborated when applying Central-Eastern China classification models to North Korean data, advocating for region-specific model development.

This study focuses on classifying small-magnitude seismic events in Jilin Province, China. Building upon previous research, we evaluate machine learning classifiers using waveform data from local blasts, mining-induced seismicity, and tectonic earthquakes. Considering that a certain number of class samples have been clearly labeled through manual review and background checks in this study, we extracted features from the time, frequency, and time-frequency domains and applied three methods, including Support Vector Machine (SVM), Extreme Gradient Boosting (XGBoost), and Backpropagation Neural Network (BPNN), for event classification to evaluate their performance. This work can provide reference for the automatic identification of seismic event types in operational applications in Jilin Province.

2 Methodology

2.1 Support vector machine

The Support Vector Machine (Vapnik, 1995) is a supervised learning method grounded in structural risk minimization principles. Its core objective is to identify an optimal hyperplane within the feature space that maximizes classification margin while minimizing empirical error. This hyperplane determination is formulated as a convex optimization problem. SVM solve both linear and nonlinear classification tasks through kernel-induced transformations.

The training dataset of M points can be expressed as $T=\left\{\left(x_1,y_1\right),\dots ,\left(x_m,y_m\right)\right\}$, where $x_i\in R^n$, $x_m$ represents the mth training sample and $y_m$ denotes its corresponding label, the goal is to find a separating hyperplane $w\phi \left(x\right)+b=0$ within the feature space. This hyperplane, capable of partitioning the training set, leads to a quadratic programming problem, which can be expressed as Equation 1:

$\left\{\begin{array}{l}\underset{w,b}{\min }\frac{1}{2}{‖w‖}^{2}s.t.\\ y_i\left\{\left[w·\phi \left(x_i\right)\right]+b\right\}\ge 1,i=1,\dots ,m\end{array}\right.(1)$

When the dataset is linearly inseparable, $K\left(x_i,x_j\right)=\left(\varphi \left(x_i\right)·\varphi \left(x_j\right)\right)$ is introduced to map the samples into a higher-dimensional space, where an optimal hyperplane is sought to partition the dataset. By selecting optimal kernel functions and parameters, the optimization problem is formulated and solved, expressed as Equation 2:

$\min \left\{-\frac{1}{2}{\displaystyle \sum _{i=1}^m}{\displaystyle \sum _{j=1}^m}{y}_i{y}_j{a}_i{a}_{j}K\left(x_i,x_j\right)\right\}(2)$

From the equation above, the optimal solution $a^{*}=\left[a_1^{*},\dots ,a_m^{*}\right]$ is obtained. Thus constructing the decision function (Devos et al., 2014; Li et al., 2016), given by Equation 3:

$f\left(x\right)=sgn\left({\displaystyle \sum _{i=1}^m}{y}_i{a}_i^{*}K\left(x_i,x\right)+b^{*}\right)(3)$

The present study implemented hyperparameter optimization for the Support Vector Machine (SVM) algorithm using GridSearchCV from scikit-learn. Evaluating the performance of RBF kernel configurations across C values [10, 100], gamma parameters ['auto’, 0.01, 0.1, 1]. The final selection prioritized C = 100 and gamma = 'auto'.

2.2 Extreme Gradient Boosting

The Gradient Boosting Decision Tree (GBDT), introduced by Friedman (2001), iteratively constructs weak classifiers (decision trees) and optimizes model weights through gradient descent minimization of the loss function. This process generates a robust predictive model via weighted ensemble aggregation. Building on this foundation, Chen and Guestrin (2016) developed XGBoost, a distinctive boosting algorithm that employs second-order Taylor expansion of the loss function and incorporates L2 regularization. These innovations effectively mitigate overfitting while accelerating convergence rates, thereby enhancing predictive accuracy. The objective function of XGBoost incorporates a regularization penalty term and consists of two components (Chen and Guestrin, 2016; Nguyen et al., 2019), as expressed in Equations 4, 5:

$l^{\left(t\right)}=\sum _{i=1}^{n}l\left(y_i,{\widehat{y}}_i\right)+\sum _{i=1}^t\mathrm{\Omega }\left(f_i\right)(4)$

$\mathrm{\Omega }\left(f_t\right)=\gamma T+\frac{1}{2}\lambda {‖w‖}^2(5)$

Where $l^{\left(t\right)}$ represents the prediction error across n samples, as defined in Equation 4, $\mathrm{\Omega }\left(f_t\right)$ denotes the regularization penalty term for the tth tree, given in Equation 5. Here, γ and λ are penalty coefficients that prevent the decision tree model from becoming overly complex, T indicates the number of leaf nodes, and w signifies the vector of leaf weights. Applying second-order Taylor expansion to this objective function yields the analytically optimized expression, presented in Equation 6:

$l=-\frac{1}{2}\sum _{j=1}^T\frac{G_j^2}{\left(H_j+\lambda \right)}+\gamma T(6)$

Where $G_j^2$ and $H_j$ denote the first-order and second-order derivatives of the objective function with respect to the model predictions, respectively. XGBoost searches for the optimal parameter settings, which are learning rate (0.1), maximum depth (5), subsample ratio (0.8), and column sampling ratio (0.8).

2.3 Backpropagation Neural Network

The model under discussion is a multilayer network based on gradient optimisation (Rumelhart et al., 1986), employing error backpropagation to iteratively adjust connection weights. This architecture comprises input, hidden, and output layers, operating under the principle of mean squared error minimization.

The forward propagation of information is achieved through the application of a weighting function to the input data, in conjunction with the utilisation of a rectified linear unit for the purpose of nonlinear transformation. This process results in the determination of the output value of each neuron (Glorot et al., 2011). The output of the l-th network layer is expressed as Equation 7:

$z^{\left(l\right)}=f\left(w^{\left(l\right)}{a}^{\left(l-1\right)}+b^{\left(l\right)}\right)(7)$

Where w^(l) is the connection weights of layer l and layer (l-1), b^(l) is the bias vector, and z^(l) is the output value of layer l calculated by the activation function.

In the field of machine learning, backpropagation is a crucial concept in the analysis of neural networks. This method involves the calculation of error contributions through the chain rule, which is then used to update the weights and biases of the network. The gradient descent method is employed to achieve this update, with a specific learning rate determining the rate at which the network weights and biases are modified. The parameter updating process is calculated as follows, shown in Equations 8, 9:

$w^{\left(l\right)}=w^{\left(l\right)}-\eta \frac{\mathrm{\partial }E}{\mathrm{\partial }{w}^{\left(l\right)}}(8)$

$b^{\left(l\right)}=b^{\left(l\right)}-\eta \frac{\mathrm{\partial }E}{\mathrm{\partial }{b}^{\left(l\right)}}(9)$

Where η is the learning rate and E is the output layer loss function.

For this study, the BP neural network employs a structured architecture with one hidden layer containing 32 fully-connected units, utilizing ReLU activation and batch normalization for stable training. The model incorporates dropout regularization (rate = 0.3) and L2 weight decay (factor = 0.1) to prevent overfitting. It is optimized using Adam with an initial learning rate of 0.001 and piecewise decay scheduling, trained for up to 100 epochs with early stopping based on validation patience of 15 epochs.

All the above algorithms introduce class weights to adjust the model’s attention to minority classes. The class weights for blast, mining subsidence, and earthquake are set to 0.55, 1.83, and 1.58 respectively. For SVM and XGBoost, we set class_weight = balanced, allowing the algorithm to automatically calculate weights based on class frequency. In BPNN, class weighting is implemented through sample weights.

3 Data preprocessing

3.1 Data description

We compiled earthquake data recorded by 37 seismic stations of Jilin Seismic Network from January 2013 to December 2024, which includes 1,245 earthquake events: 290 tectonic earthquakes, 721 blasts, and 234 mining-induced earthquakes. We manually reviewed the research data to verify events based on recorded characteristics, occurrence time, epicenter location, and historical waveform comparisons. Additionally, to ensure the integrity of data records, seismic records with a signal-to-noise ratio (SNR) below 2.0 were excluded, as well as earthquake events with fewer than three valid seismic recording stations. Furthermore, since non-natural events typically have smaller magnitudes and rarely exceed M_L 3.5, natural earthquakes with similar magnitudes were selected to maintain consistency in the magnitude distribution. Finally, 898 earthquake events with magnitudes between M_L 1.5 and 3.5 were selected as sample data: 190 tectonic earthquakes, 544 blasts, and 164 mining collapses. Spatial analysis (Figure 1) shows that blast events are concentrated in southeastern Jilin Province, while mining-induced earthquakes are mainly clustered in the central region.

Figure 1

Figure 1. Spatial distribution of seismic events (M_L1.5–3.5) from 2013 to 2024 in Jilin Province, China.

3.2 Feature selection

Waveform characteristics exhibit significant discriminative variations across seismic source types, enabling effective feature extraction for event classification. Spatial analysis (Figure 1) reveals frequent co-occurrence of blasts with both earthquakes and collapses in adjacent regions. Comparative examination of stations at similar azimuths but varying epicentral distances demonstrates: Blasts feature impulsive P-waves with higher amplitude and energy than S-waves and abbreviated signal durations (Figure 2A); Collapses display prolonged waveforms dominated by low-frequency energy throughout the bandwidth (Figure 2B); Tectonic earthquakes manifest predominant S-wave energy consistent with double-couple source mechanisms, contrasting with P-wave dominance in blasts (Figure 2C).

Figure 2

Figure 2. Waveform and time-frequency characteristics of (A) blasts, (B) collapses, and (C) tectonic earthquakes, showing representative waveform segments (upper panels) and corresponding spectrograms (lower panels).

This study comprehensively analyzes the waveforms and spectrograms of three types of seismic events, and refers to the feature extraction methods for seismic events in central, eastern and northeastern China proposed by Wang et al. (2022) and Liang et al. (2023), to comprehensively extract 87-dimensional features, perform max normalization on their training samples to form a feature dataset. These features include spectrum amplitude (A_p(f), A_s(f)), maximum P/S amplitude ratio (A_p/A_s), waveform duration (D_t), the high- and low-frequency energy ratio (HL_r), waveform complexity (C_y), the zero-crossing rate (Z_r), the corner frequency (C_f) and the instantaneous frequency complexity (I_f).

The meaning of the features is detailed in Table 1, and their distribution is shown in Figure 3. As shown in Figures 3A,B, the amplitude characteristics of the three event types exhibit significantly different distributions: within the low-frequency band (0.2–2 Hz), the amplitudes of blasts and collapses are higher than those of tectonic earthquakes. In the 5–10 Hz, tectonic earthquakes have the highest average amplitude, while collapses have the lowest. Additionally, Figure 3C shows that above 6 Hz, A_p/A_s of blasts tends to be the largest. Conversely, A_p/A_s of tectonic earthquakes tends to be the smallest. Figure 3D presents the feature distributions of waveform duration (D_t), high-low frequency energy ratio (HL_r), waveform complexity (C_y), zero-crossing rate (Z_r), corner frequency (C_f), and instantaneous frequency complexity (I_f) for all events in this study’s dataset.

Table 1

Table 1. Definition of seismic waveform feature.

Figure 3

Figure 3. Distribution of characteristic data for three types of seismic events in Jilin Province. Red, yellow, and green represent blasts, mining collapses, and tectonic earthquakes, respectively. The feature values are the maximum normalized values of each feature, with specific details as follows: (A) shows 27 P-wave spectral amplitudes; (B) shows 27 S-wave spectral amplitudes; (C) Pg/Sg amplitude ratio corresponds to the 1–20 Hz frequency; (D) feature distribution includes: 1. Duration; 2, 3. High-low frequency energy ratio; 4, 5: waveform Complexity; 6, 7: Zero-crossing rate; 8, 9: Pg corner frequency; 10, 11: Sg corner frequency; 12, 13: Instantaneous frequency.

4 Testing and analysis of seismic event type identification capability

4.1 Training results of the classification model

This study employed SVM, XGBoost, and BPNN classifiers, trained on the 87-dimensional feature dataset derived from blasts, collapses, and tectonic earthquakes waveforms in Jilin Province. The dataset is divided into a training set and a test set in an 8:2 ratio. Events are labeled as: Category 0 (Blast) with 109 test events, Category 1 (mining Collapse) with 33 test events, and Category 2 (Earthquake) with 38 test events. All three methods were repeated with 5-fold cross-validation using different random seeds. As shown in the sample learning curve Figure 4, the overall performance of the three models gradually improves with the increase in training data. The learning curves indicate that when the number of training samples exceeds 600, the model performance tends to saturate. As shown in Table 2, All three models demonstrate strong classification capabilities, with an average training time of less than 5 s and memory usage not exceeding 300 MB. In the multi-classification task, the three models perform best in distinguishing between discriminant collapse and seismic events. Overall performance-wise, the SVM classifier performs optimally, standing out in terms of accuracy, macro F1 score, and stability.

Figure 4

Figure 4. Learning curves of three models.

Table 2

Table 2. Average classification accuracy of comparative models.

4.2 Model performance evaluation

Confusion matrices and four evaluation metrics (accuracy, precision, recall, F1-score) were generated for the test results of each classifier. As shown in Figure 5 with the confusion matrix results, the accuracy of all models on the test set exceeds 94%. Among the three algorithms, they perform best in distinguishing mining collapse events from construction seismic activities but perform poorly in distinguishing blasts from mining collapse. The discriminant accuracy for each category is calculated based on their results, as shown in Table 3. The discriminant accuracy of the three models for each event category reaches over 90%, with blast discrimination being superior to the others.

Figure 5

Figure 5. The confusion matrix of the model on the test dataset. The x-axis represents the input data type, and the y-axis represents the output data type of the classifier; red squares indicate the number of correctly classified samples for blasts, yellow squares for mine collapses, and green squares for earthquake events. Gray squares represent the number of misclassified samples. The percentage in the box is the proportion of the corresponding event count to the total number of test events.

Table 3

Table 3. Quantitative evaluation metrics for classification performance.

A reliability analysis of the predicted probabilities for each class in the test set is conducted, as shown in Figure 6 and Table 4. The three models exhibit different levels of prediction reliability. Among them, the SVM model performs best in both prediction accuracy (macro F1: 0.975) and probability calibration reliability (average ECE: 0.028), making it the model with the optimal overall performance. XGBoost has acceptable accuracy but slightly inferior calibration performance compared to SVM. While the BP neural network has acceptable prediction precision, its calibration error (average ECE: 0.098) is significantly higher than that of the other models, indicating that there is an issue of overconfidence in its predicted probabilities.

Figure 6

Figure 6. Reliability curve of event classification by different types.

Table 4

Table 4. Reliability indicators for the three models.

4.3 Analysis of factors impacting classification performance

To understand the importance of various criteria in model classification, we employed permutation importance to evaluate the feature importance of different classification models and conducted ablation tests on different feature groups. All three models used the same test set, with each experiment repeated 20 times. The analysis results are shown in Figure 7A, indicating that all extracted feature criteria contribute to some extent, with A_p/A_s, A_p(f) and A_s(f) standing out in importance. These three criteria ranked among the top three across all three classification models, demonstrating high consistency in classification models. As shown in Figure 7B, by removing different feature groups, the model performance decreased to varying degrees. Among them, the three features A_p/A_s, A_p(f) and A_s(f) had a relatively significant impact on model performance. Overall, the importance of multiple feature criteria exhibits a hierarchical contribution pattern, with A_p/A_s, A_p(f) and A_s(f) having higher contributions, followed by C_y, I_r, and Z_r, and D_t, C_f and HL_r having lower contributions.

Figure 7

Figure 7. (A) Shows the feature group classification importance ranking for three models, blue represents the feature group classification importance of the XGBoost model, orange represents that of the SVM model, green represents that of the BPNN model, and red represents the average classification importance of the feature groups; (B) Shows the performance changes of feature ablation for the three models, blue represents the XGBoost model, orange represents the SVM model, and green represents the BPNN model.

4.4 Validation with cross-regional datase

To evaluate regional generalization capability, the three trained algorithms were validated on seismic events from adjacent regions recorded by the Jilin Seismological Network (Heilongjiang, Inner Mongolia, Liaoning, and North Korea). The external test dataset comprised 30 blasts, 23 collapses, and 30 tectonic earthquakes. Classification results (confusion matrix in Figure 8 and metrics in Table 5) indicate that SVM exhibits significant performance advantages over XGBoost and BPNN. On the external test set, the Accuracy is 0.84 with a Binomial CI of [0.801, 0.912]. Additionally, the McNemar test shows that the SVM model is significantly superior to the BPNN model (p < 0.001), but no statistically significant difference is found between SVM and XGBoost (p = 0.118). This suggests that SVM maintains optimal performance while its advantages are reliably validated statistically.

Figure 8

Figure 8. Confusion matrices for SVM, XGBoost, and BPNN classifiers on external test dataset.

Table 5

Table 5. Confusion matrices for SVM, XGBoost, and BPNN classifiers on external test dataset.

5 Discussion

5.1 Discussion on typical misclassified events in JiLin

The main misjudgment events of natural earthquakes are minor magnitude events in the Songyuan area. Comparative waveform and time-frequency characteristics (Figure 9) reveal significant deviations between Songyuan events and typical tectonic earthquakes: (1) Slower waveform attenuation rates; (2) Longer dominant periods with enhanced surface wave development; (3) Elevated amplitude ratios.

Figure 9

Figure 9. Waveform and time-frequency characteristics of misclassified events: (A) Songyuan earthquake; (B) Tonghua earthquake. Upper panels: waveform components; lower panels: corresponding spectrograms.

Site analysis indicates the Songyuan source region primarily consists of unconsolidated sediments and soft rock, contrasting with the competent bedrock geology of eastern regions. This lithological contrast significantly modifies seismic wave propagation. Focal mechanism solutions (Gao et al., 2013; Liu et al., 2017; Liu, 2018; Liang et al., 2019) and our moment tensor inversions (Figure 10) consistently reveal substantial volumetric components in Songyuan events. These mechanisms exhibit dominant volumetric characteristics across magnitudes comparable to our dataset. Comprehensive analysis suggests Songyuan earthquakes are atypical tectonic events, with characteristics aligning closely with non-natural seismic sources (Yu et al., 2020).

Figure 10

Figure 10. Results of the seismic mechanism solution for Changling, Jilin.

Current understanding of Songyuan seismogenesis remains contested, with three primary hypotheses under consideration: (1) stress modulation from Pacific Plate subduction; (2) deep-seated magmatic processes or fluid migration; and (3) long-term hydrocarbon extraction effects. Through three-dimensional density structural modeling, Liu et al. (2024) demonstrate that Songyuan seismicity results from coupled interactions between regional stress, fluid dynamics, and petroleum operations. Analysis suggests Songyuan earthquakes relate to regional stress, fluid migration, and oil/gas extraction. Long-term water injection for oil extraction may be the primary trigger, potentially altering geological structures and stress conditions.

5.2 Analysis of cross-regional misclassification

Models trained on data from within Jilin Province show reduced classification accuracy for events in adjacent regions. Notably, seismic events near the North Korean nuclear test site are frequently misclassified. As shown by recorded waveforms (Figure 11), events in this region exhibit distinct P- and S-wave energy distributions. Figure 12 further indicates that the P/S amplitude ratio of misclassified events consistently falls between the characteristic curves established for explosion events and tectonic earthquakes within Jilin Province. Significantly, the average P/S amplitude ratio of misclassified collapse events exceeds that of both tectonic earthquakes and confirmed collapse events. Analysis of these misclassifications reveals region-specific seismic features crucial for improving North Korea earthquake identification, thereby reducing human error and enhancing model accuracy.

Figure 11

Figure 11. Waveform components of misclassified events: (A) Blast; (B) Collapse; (C) Tectonic earthquake.

Figure 12

Figure 12. P/S amplitude ratio characteristic curves. Blue: Average P/S amplitude ratio of blasts in Jilin Province; Red: Average P/S amplitude ratio of tectonic earthquakes in Jilin Province; Green: Average P/S amplitude ratio of collapses in Jilin Province; Bright green: Average P/S amplitude ratio of collapses misclassified as blasts; Cyan blue: Average P/S amplitude ratio of blasts misclassified as collapses; Orange: Average P/S amplitude ratio of earthquakes misclassified as collapses; Pink: Average P/S amplitude ratio of earthquakes misclassified as blasts.

5.3 Comparison of physics-informed features and generalized deep learning

The features and methods extracted in this study perform well in the local region but do not demonstrate excellent generalization to other regions. Recent studies have focused on using generic deep learning approaches for earthquake activity prediction, aiming to develop a single powerful model applicable to various tectonic environments (Muhammad et al., 2023). However, in reality, such models require large sample sizes and rigorous model construction. By leveraging physics-informed features and combining them with classical machine learning models like support vector machines, our approach provides concrete insights into the model’s decision-making process, which is a key advantage compared to the black box nature often associated with deep neural networks. Additionally, our method can be conveniently applied to relatively small datasets, making it particularly suitable for areas with limited seismic records or constrained computational resources. We acknowledge that generic deep learning models trained on massive global data (Mousavi et al., 2020) may exhibit superior cross-regional generalization. Nevertheless, physics-guided classical machine learning methods offer more interpretable, resource-efficient, and thus more deployable solutions, establishing valuable benchmarks and practical alternatives for focused seismic analysis.

6 Conclusion

Based on the data of artificial explosions, mining-induced collapses, and tectonic earthquakes (M_L 1.5–3.5) in Jilin Province from 2013 to 2024, this study quantitatively extracted multiple discriminant features and constructed an 87-dimensional feature dataset. Using SVM, XGBoost, and BPNN algorithms, classification training and testing were conducted on seismic events in Jilin Province and surrounding areas, and the classification performance of these models was investigated. The classification tests revealed that all three models achieved accuracy rates exceeding 94% in the multi-classification task, with the best performance observed in distinguishing mining-induced collapses and tectonic earthquakes, while the classification between collapses and explosions was less effective. Statistical validation of the model classifications indicated that, compared to the other two methods, the SVM model demonstrated superior predictive accuracy and probability calibration reliability, and achieved the best generalization performance with 84% accuracy on a cross-regional dataset. Feature importance analysis showed that all extracted discriminant features contributed to some extent to the classification, with “maximum P/S amplitude ratio,” “S- and P-wave spectral amplitude” consistently identified as key features across all three models. Accordingly, these features can be prioritized as primary indicators in practical systems and implemented in a hierarchical or combined strategy with other features.

The features and methods proposed in this study still exhibit certain limitations when applied to regions beyond Jilin Province, particularly for small-magnitude earthquakes and events with strong regional characteristics (such as those in the Songyuan area and the surrounding regions of the Korean Peninsula test site). Model performance is susceptible to influences from data selection bias, regional site conditions, and differences in seismic wave propagation paths. Building on the existing research findings, future work will focus on enhancing the model’s generalization capability. This will be achieved through the integration of waveform data from multiple regions and standards to construct a more representative test set, enabling systematic evaluation of the model’s classification performance across different geological environments. Additionally, cross-regional data partitioning strategies will be explored to investigate characteristic differences in propagation paths, thereby improving the model’s classification robustness in new regions and facilitating its effective application in seismic monitoring practice.

Data availability statement

This dataset can be applied for from the China Earthquake Networks Center and the National Earthquake Data Center (http://data.earthquake.cn) or international Earthquake Science Data Center (http://esdc.ac.cn). The source code of the related algorithms can be obtained from https://github.com/xiaomi8023/svm-xgb-code and https://gitcode.com/Open-source-documentation-tutorial/34b38. For further inquiries, please contact the corresponding author.

Author contributions

FR: Formal Analysis, Methodology, Writing – original draft, Writing – review and editing. HL: Resources, Formal Analysis, Writing – review and editing. HZ: Data curation, Investigation, Writing – review and editing. TW: Software, Writing – review and editing. QR: Writing – review and editing, Project administration, Funding acquisition. FZ: Supervision, Writing – review and editing. YW: Writing – review and editing.

Funding

The authors declare that financial support was received for the research and/or publication of this article. This work was supported by: Jilin Provincial Seismological Bureau Youth Development Program (JZQ-202501); Earthquake Monitoring, Forecasting and Early Warning Project (CEA-JCYJ-202501018); Jilin Provincial Science and Technology Development Plan (20240304127SF).

Acknowledgements

We thank Dr. Tingting Wang for feature analysis support, Engineer Hao Liang for algorithm development contributions, and colleagues at the Jilin Seismological Network for technical assistance. Appreciation extends to the Liaoning and Heilongjiang Seismological Networks for data access.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The authors declare that no Generative AI was used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Alvizuri, C. R., Matoza, R. S., and Okubo, P. G. (2021). Earthquake collapse mechanisms and periodic, migrating seismicity during the 2018 summit collapse at Kilauea caldera. Earth Planet. Sci. Lett. 562, 116–819. doi:10.1016/j.epsl.2021.116819

CrossRef Full Text | Google Scholar

Bi, M. X., Huang, H. M., Border, S. C., Li, R., Chen, Y., and Zhao, J. (2011). A study on seismic signal HHT features extraction and SVM recognitionof earthquake and explosion. Prog. Geophys. 26 (4), 1157–1164. doi:10.3969/j.issn.1004-2903.2011.04.004

CrossRef Full Text | Google Scholar

Bian, Y. J. (2002). Application of genetic BP network to discriminating earthquakes and explosions. Learning. Acta Seismol. Sin. 24 (5), 516–524.

Google Scholar

Bian, Y. J. (2005). Application of Fisher method to discriminating earthquakes and explosions using criterion mb/MS. Acta Seismol. Sin. 27 (4), 414–422.

Google Scholar

Brune, J. N. (1970). Tectonic stress and the spectra of seismic shear waves from earthquakes. J. Geophys. Res. 75, 4997–5009. doi:10.1029/jb075i026p04997

CrossRef Full Text | Google Scholar

Chen, T., and Guestrin, C. (2016). XGBoost: a scalable tree boosting system. Proc. 22nd ACM SIGKDD Int. Conf. Knowl. Discov. Data Min., 785–794. doi:10.1145/2939672.2939785

CrossRef Full Text | Google Scholar

Devos, O., Downey, G., and Duponchel, L. (2014). Simultaneous data pre-processing and svm classification model selection based on a parallel genetic algorithm applied to spectroscopic data of olive oils. Food Chem. 148 (apr.1), 124–130. doi:10.1016/j.foodchem.2013.10.020

PubMed Abstract | CrossRef Full Text | Google Scholar

Friedman, J. H. (2001). Greedy function approximation: a gradient boosting machine. Ann. Statistics 29 (5), 1189–1232. doi:10.1214/aos/1013203451

CrossRef Full Text | Google Scholar

Gao, J. Z., Li, Z. W., Bao, F., and Chong, J. J. (2013). Resolving focal depth of the 31 march 2006 Jilin QianAn-QianGuo earthquakewith multiple datasets and its implication for seismogenesis. Prog. Geophys. 28 (5), 2328–2335. doi:10.6038/pg20130515

CrossRef Full Text | Google Scholar

Glorot, X., Bordes, A., and Bengio, Y. (2011). Deep sparse rectifier neural networks. J. Mach. Learn. Res. 15, 315–323.

Google Scholar

Jimenez, A., Garcia, J. M., and Romacho, M. D. (2005). Simultaneous inversion of source parameters and attenuation factor using genetic algorithms. Bull. Seismol. Soc. Am. 95 (4), 1401–1411. doi:10.1785/0120040116

CrossRef Full Text | Google Scholar

Kong, Q., Wang, R. J., Walter, W. R., Pyle, M., Koper, K., and Schmandt, B. (2022). Combining deep learning with physics-based features in explosion-earthquake discrimination. Geophys. Res. Lett. 49 (13), e2022GL098645. doi:10.1029/2022GL098645

CrossRef Full Text | Google Scholar

Li, M., Liu, X., and Liu, X. (2016). Infrasound signal classification based on spectral entropy and support vector machine. Appl. Acoust. 113 (dec.), 116–120. doi:10.1016/j.apacoust.2016.06.019

CrossRef Full Text | Google Scholar

Li, X. Y., Bian, Y. J., Hou, X. L., Wang, T. T., and Zhang, Y. X. (2025). Recognition of small magnitude seismic events type based on time-frequency features and machine learning. Acta Seismol. Sin. 47. doi:10.11939/jass.20240012

CrossRef Full Text | Google Scholar

Liang, S. S., Xu, Z. G., Zhang, G. W., Liang, J. H., Zhang, M. H., and An, X. Y. (2019). Full moment tensor inversion for the 2019 Ningjiang MS5.1 earthquake in Jilin province and study on its seismogenic mechanism. Earthq. Res. China 35 (3), 488–498.

Google Scholar

Liang, H., Sun, L., Chen, S. Q., and Zhi, M. (2023). Research on seismic event classification based on SVM algorithm: an application in Northeast China. Chin. J. Geophys. 66 (12), 5030–5040. doi:10.6038/cjg2022Q0829

CrossRef Full Text | Google Scholar

Liu, J. Q. (2018). The study of the typical seismicity in Western region of Jilin Province. China: Jilin University.

Google Scholar

Liu, J. Q., Liu, C., Lei, J. S., Gan, W. J., Yang, Q. F., and Zhang, C. X. (2017). The moment tensors of the 2013 Qianguo Ms5.8 seismic swarm. Chin. J. Geophys. 60 (9), 3418–3431. doi:10.6038/cjg20170912

CrossRef Full Text | Google Scholar

Liu, W. Y., Cheng, Z. P., Nian, X. Q., Chen, Y., Hu, Y. L., Qin, Z. J., et al. (2024). Genesis of Songyuan earthquakes based on 3D residual density structure. Seismol. Geol. 46 (2), 462–476. doi:10.3969/j.issn.0253-4967.2024.02.013

CrossRef Full Text | Google Scholar

Mousavi, S. M., Zhu, W., Ellsworth, W., and Beroza, G. C. (2019). Unsupervised clustering of seismic signals using deep convolutional autoencoders. IEEE Geoscience and Remote Sensing Letters 16 (11), 1693–1697. doi:10.1109/LGRS.2019.2909218

CrossRef Full Text | Google Scholar

Mousavi, S. M., Zhu, W., Sheng, Y., and Beroza, G. C. (2020). CRED: a deep residual network of convolutional and recurrent units for earthquake signal detection. Scientific Reports 10 (1), 1–14.

Google Scholar

Muhammad, D., Ahmad, I., Khalil, M. I., Khalil, W., and Ahmad, M. O. (2023). A generalized deep learning approach to seismic activity prediction. Appl. Sci. 13 (3), 1598. doi:10.3390/app13031598

CrossRef Full Text | Google Scholar

Murray, D., Stankovic, L., Pytharouli, S., and Stankovic, V. (2023). Semi-supervised seismic event detection using Siamese networks. Vienna, Austria: Poster session presented at EGU General Assembly 2023. doi:10.5194/egusphere-egu23-14184

CrossRef Full Text | Google Scholar

Nguyen, H., Bui, X. N., Bui, H. B., and Cuong, D. T. (2019). Developing an XGBoost model to predict blast-induced peak particle velocity in an open-pit mine: a case study. Acta Geophys 67, 477–490. doi:10.1007/s11600-019-00268-4

CrossRef Full Text | Google Scholar

Ren, T., Lin, M. N., Chen, H. F., Wang, R. R., Li, S. W., Liu, X. Y., et al. (2019). Seismic event classification based on bagging ensemble learning algorithm. Chin. J. Geophys. 62 (1), 383–392. doi:10.6038/cjg2019M0380

CrossRef Full Text | Google Scholar

Rumelhart, D. E., Hinton, G. E., and Williams, R. J. (1986). Learning internal representations by error propagation. Parallel distributed processing: explorations in the microstructure of cognition. MIT Press, 318–362. doi:10.7551/mitpress/4943.003.0128

CrossRef Full Text | Google Scholar

Saad, O. M., Shalaby, A., and Sayed, M. S. (2019). Automatic discrimination of earthquakes and quarry blasts using wavelet filter bank and support vector machine. J. Seismol. 23 (2), 357–371. doi:10.1007/s10950-018-9810-5

CrossRef Full Text | Google Scholar

Shang, X., Li, X., Morales-Esteban, A., and Chen, G. (2017). Improving microseismic event and quarry blast classification using artificial neural networks based on principal component analysis. Soil Dyn. Earthq. Eng. 99, 142–149. doi:10.1016/j.soildyn.2017.05.008

CrossRef Full Text | Google Scholar

Tang, L., Zhang, M., and Wen, L. (2020). Support vector machine classification of seismic events in the tianshan orogenic belt. J. Geophys. Res. Solid Earth 125, e2019JB018132. doi:10.1029/2019JB018132

CrossRef Full Text | Google Scholar

Taylor, S. R., Denny, M. D., Vergino, E. S., and Glaser, R. E. (1989). Regional discrimination between NTS explosions and Western U.S. earthquakes. Bull. Seismol. Soc. Am. 79 (4), 1142–1176. doi:10.1785/bssa0790041142

CrossRef Full Text | Google Scholar

Vapnik, V. N. (1995). The nature of statistical learning theory. New York: Springer-Verlag. doi:10.1007/978-1-4757-2440-0

CrossRef Full Text | Google Scholar

Wang, T. T., and Bian, Y. J. (2011). Criterion selection of earthquake and explosion recognition. Seismol. Geomagnetic Observation Res. 32 (6), 62–67. doi:10.3969/j.issn.1003-3246.2011.06.012

CrossRef Full Text | Google Scholar

Wang, R. J., Schmandt, B., and Kiser, E. (2020). Seismic discrimination of controlled explosions and earthquakes near Mount St. Helens using P/S ratios. J. Geophys. Res. Solid Earth 125 (10), e2020JB020338. doi:10.1029/2020JB020338

CrossRef Full Text | Google Scholar

Wang, T. T., Bian, Y. J., You, Q. M., Ren, M. Y., and Yang, Q. L. (2024). Classification study of earthquakes and explosions in North Korea and adjacent regions. Seismol. Res. Lett. 96 (1), 421–434. doi:10.1785/0220240143

CrossRef Full Text | Google Scholar

Wang, T. T., Bian, Y. J., Zhang, Y. X., and Hou, X. L. (2022). Using artificial intelligence methods to classify different seismic events. Seismol. Res. Lett. 94 (1), 1–16. doi:10.1785/0220220055

CrossRef Full Text | Google Scholar

Yang, Q. L., Wang, T. T., and Bian, Y. J. (2020). Recognition of earthquakes and explosions based on generalized S transform. Acta Seismol. Sin. 42 (5), 613–628. doi:10.11939/jass.20190173

CrossRef Full Text | Google Scholar

Yildirim, E., Güçlübağ, A., Horasan, G., and Doğan, E. (2011). Discrimination of quarry blasts and earthquakes in the vicinity of Istanbul using soft computing techniques. Comput. & Geosciences 37 (9), 1209–1217. doi:10.1016/j.cageo.2011.02.011

CrossRef Full Text | Google Scholar

Yu, G. C., Yao, Y. S., Zhang, L. F., Wang, Q. L., and Wu, H. B. (2020). Study on genesis of Ms≥5 songyuan earthquake swarm in Jilin Province in 2013. J. Geodesy Geodyn. 40 (05), 534–539. doi:10.14075/j.jgg.2020.05.018

CrossRef Full Text | Google Scholar

Zhao, G., Huang, H. M., Lu, X. X., Guo, S. H., and Chai, H. M. (2017). Research on the classification of seismic wave signalsof earthquakes and explosion events based on BP-Adaboost. China Earthq. Eng. J. 39 (3), 557–562.

Google Scholar

Zhao, M., Chen, S., and Yuen, D. (2019). Waveform classification and seismic recognition by convolution neural network. Chin. J. Geophys. 62 (1), 374–382. doi:10.6038/cjg2019M0151

CrossRef Full Text | Google Scholar

Zhou, S. H., Jiang, H. K., Li, J., Qu, J. H., Zheng, C. C., and Li, Y. J. (2021). Research on identification of seismic events based on deep learning: taking the records of Shandong seismic network as an example. Seismol. Geol. 43 (3), 663–676. doi:10.3969/j.issn.0253-4967.2021.03.012

CrossRef Full Text | Google Scholar