Hilbert-Huang Transform Embedded Self-Attention Neural Network for EEG-based major depressive disorder vs. healthy controls classification

This paper proposes a novel approach for distinguishing Major Depressive Disorder (MDD) patients from healthy controls (HC), namely depression screening, using EEG signals, where the Hilbert-Huang Transform (HHT) is integrated into a Self-Attention neural network (HHT-SANN). The incorporation of the HHT enhances the model’s time-frequency analysis capabilities and allows for more effective nonlinear processing of the EEG data. By embedding the HHT within the self-attention module, the model captures intricate temporal and spectral patterns that are critical for accurate depression classification. We evaluated our method on a clinical EEG dataset comprising 34 MDD patients and 30 healthy controls from the Hospital of Universiti Sains Malaysia. Experimental results indicate that the proposed method achieves an accuracy of 98.78%, sensitivity of 99.23%, and specificity of 98.27%, outperforming traditional models and offering a more robust solution for depression detection. This work contributes to advancing the field of neuroinformatics by providing a more interpretable and effective model for mental health diagnostics based on EEG data.

1 Introduction

Depression is a pervasive and debilitating psychiatric disorder affecting hundreds of millions worldwide, with those experiencing severe major depressive disorder (MDD) at particularly high risk of suicidal ideation. Objective and reliable screening methods are therefore critical for early intervention and improved outcomes (1). However, traditional diagnostics—relying on clinician assessments and self-report questionnaires—are vulnerable to bias and inconsistency, fueling the search for quantitative biomarkers. In this context, compared with fMRI (2, 3) and structural MRI (4), electroencephalography (EEG) has emerged as a noninvasive, cost-effective modality with exceptional temporal resolution, adept at capturing the rapid neural dynamics underlying depressive pathology.

The efficacy of EEG-based diagnosis, however, hinges critically on two interdependent pillars: the capability of the classifier and, just as importantly, the appropriateness of the feature extraction methodology for uncovering the neural mechanisms specific to the disorder. EEG signals are inherently non-linear and non-stationary, reflecting the complex, dynamic nature of brain activity in MDD. Consequently, the choice of feature extraction is paramount. A diverse array of approaches has been explored in the literature to capture these aberrant neural patterns. For instance, studies have investigated functional connectivity and network properties, such as spectral coherence (5), domain-specific connectivity (6), and global EEG connectivity (7), which aim to reveal disorganization in large-scale brain networks associated with MDD. Others have focused on complexity measures and nonlinear dynamics to quantify the irregularity of neural signals (8), or have utilized connectivity features derived from EEG to predict treatment outcomes (9, 10).

While these methods provide valuable insights, many conventional feature extraction techniques, including those based on predefined linear transforms or hand-crafted metrics, may not fully adapt to the non-stationary and nonlinear characteristics of EEG in depression. This underscores a pressing need for methods that can effectively and adaptively model the time-varying, nonlinear dynamics of brain activity in depressive individuals. For example, Mumtaz et al. (11) proposed a machine learning method using pre-treatment EEG to predict SSRI antidepressant outcomes in MDD. Subsequently, Ke et al. developed an AutoML-based dual-CNN model for real-time EEG classification in brain e-health, addressing static models and computational complexity with autonomously optimized hyperparameters, achieving high accuracy for MDD and significantly outperforming CapsuleNet and ResNet-16 (12). More recently, a novel TanhReLU-based CNN was proposed to address MDD diagnosis challenges using EEG data, with the hybrid activation function combining Tanh and ReLU properties to mitigate gradient vanishing and overfitting in EEG pattern recognition (13). Wang et al. (14) enhanced MDD/BD classification by integrating clinical and EEG data from 400 patients, using feature engineering and ML models to improve accuracy, address overfitting, and highlight EEG’s diagnostic value for precision psychiatry.

The primary motivation for integrating HHT into a deep learning framework stems from its inherent suitability for analyzing the non-linear, non-stationary brain dynamics characteristic of MDD. Unlike methods relying on predefined basis functions, HHT’s fully adaptive decomposition into Intrinsic Mode Functions (IMFs) offers a physiologically plausible representation of EEG signals, enabling direct extraction of instantaneous frequency and amplitude components beyond linear assumptions. Our core theoretical contribution lies in conceptualizing HHT not as a fixed pre-processor, but as an integrated layer within an end-to-end network. This allows the model to learn task-specific time-frequency representations directly from raw data, optimizing feature extraction for MDD identification. Thus, this work bridges theoretically-grounded signal processing for non-stationary data and deep learning’s pattern recognition capabilities.

While HHT provides meaningful time-frequency representations, it remains under-explored as an integrated component in deep learning frameworks (15). Prior implementations often treated HHT as a static preprocessing step, limiting end-to-end optimization. To address this gap and better link neural mechanism-informed feature extraction with classification power, we propose the novel Hilbert-Huang Transform Embedded Self-Attention Neural Network (HHT-SANN).

In HHT-SANN, the HHT is embedded as a network layer to learn intrinsic time-frequency components dynamically. It is complemented by a Squeeze-and-Excitation (SE) module that adaptively recalibrates feature channels, and a self-attention mechanism that captures global dependencies in time-frequency representations. These components collectively form an end-to-end system that unifies advanced feature extraction rooted in MDD’s neural substrate with high-performance classification for EEG-based screening.

To summarize, the main contributions of this study are as follows:

1. We propose the first deep learning model that integrates the Hilbert-Huang Transform as a layer for EEG-based depression classification.

2. We introduce a novel combination of HHT, self-attention, and channel-wise excitation, enabling the model to learn adaptive and interpretable time-frequency representations.

3. We validate our method on real-world EEG datasets and demonstrate its superiority over conventional and state-of-the-art approaches in terms of classification accuracy and robustness.

2 Methodology

2.1 Dataset

The EEG dataset used in this study was acquired at the Hospital of Universiti Sains Malaysia and comprises recordings from 34 patients with Major Depressive Disorder (MDD: 17 males, mean age 40.3 ± 12.9 years) and 30 healthy controls (21 males, mean age 38.2 ± 15.6 years) (11). All participants underwent 5 min of eyes-closed and 5 min of eyes-open resting-state EEG, recorded via 20 scalp electrodes (Fp1, Fp2, F3, F4, F7, T3, T5, C3, C4, Fz, Cz, Pz, F8, T4, T6, P3, P4, O1, O2, ECG) placed according to the international 10–20 system at 256 Hz. Exclusion criteria included psychotic symptoms, pregnancy, alcohol/substance use, smoking, or epilepsy; controls were screened to confirm the absence of neurological or psychiatric conditions. EEG data were preprocessed in BESA to remove artifacts, and two-minute artifact-free segments from each resting condition were extracted using a 1,024-sample (4 s) sliding window, yielding 18,442 total epochs (9,789 MDD, 8,653 HC).

2.2 Hilbert Huang Transform Layer

This section introduces our proposed model architecture, which integrates the Hilbert-Huang Transform (HHT) into a deep neural network for EEG-based depression screening. Our architecture consists of three main components: a Hilbert Transform Layer, and a Squeeze-and-Excitation (SE) block, which forms the Self-Attention module. The overall architecture illustrated in Figure 1 is designed to learn discriminative, non-linear time-frequency features from raw EEG inputs in an end-to-end fashion.

Figure 1

Figure 1. Model architecture.

In this approach, the HHT layer is implemented as a neural network layer that applies Empirical Mode Decomposition (EMD) followed by the Hilbert Transform to the input EEG signal. Each EEG signal x(t) for each channel (20, 32*32) is decomposed into a set of Intrinsic Mode Functions (IMFs) (Equation 1):

$\begin{array}{ll}x(t)=\sum _{i=1}^N{\text{IMF}}_i(t)+r(t)& (1)\end{array}$

where N is the number of IMFs and r(t) is the residual signal.

The Hilbert Transform is then applied to each IMF to obtain its instantaneous amplitude and frequency. The analytic signal h(IMF_i(t)) is defined as Equation 2:

$\begin{array}{ll}h({\text{IMF}}_i(t))={\text{IMF}}_i(t)·\text{exp }(j·{\theta }_i(t))& (2)\end{array}$

The instantaneous frequency $f_i(t)$ is computed as Equation 3:

$\begin{array}{ll}{f}_i(t)=\frac{1}{2\pi }\frac{d{\theta }_i(t)}{dt}& (3)\end{array}$

The output of the HHT layer is a time-frequency feature matrix $X_{\text{HHT}}$ that incorporates both channel and temporal information from the EEG signal.

2.3 Squeeze-and-Excitation block with HHT

To enhance channel-wise representations, a Squeeze-and-Excitation (SE) block is incorporated after the HHT layer. Let $F_{\text{HHT}}\in {ℝ}^{T\times C^{\prime }}$ denote the output from the HHT layer, where $C^{\prime }$ represents the expanded feature dimension after IMF concatenation, and $T$ is the number of time points.

Squeeze step: Global average pooling is applied across the temporal dimension to generate a channel descriptor (Equation 4):

$\begin{array}{ll}{z}_c=\frac{1}{T}\sum _{t=1}^T{F}_{\text{HHT}}(t,c)& (4)\end{array}$

Excitation step: The descriptor $z\in {ℝ}^{C^{\prime }}$ is passed through two fully connected layers with a ReLU activation followed by a sigmoid function (Equation 5):

$\begin{array}{ll}s=\sigma (W_2·\text{ReLU}(W_1·z))& (5)\end{array}$

where $W_1\in {ℝ}^{\frac{C^{\prime }}{r}\times C^{\prime }}$, $W_2\in {ℝ}^{C^{\prime }\times \frac{C^{\prime }}{r}}$, and r is the reduction ratio controlling the bottleneck compression. The sigmoid function $\sigma (·)$ ensures the output values lie in the range (0, 1).

Scale: The excitation vector $s$ is broadcast and multiplied element-wise with the original feature map $F_{\text{HHT}}$ to generate the recalibrated feature map ${\tilde{F}}_{\text{SE}}$ in Equation 6:

$\begin{array}{ll}{\tilde{F}}_{\text{SE}}(t,c)=s_c·F_{\text{HHT}}(t,c)& (6)\end{array}$

This operation allows the network to emphasize informative features and suppress less relevant ones by adaptively adjusting the importance of each channel.

3 Results

The experiments conducted in this section serve as a validation and assessment of the classification performance of the proposed model. Initially, we describe the experimental platform utilized for these assessments. Finally, the classification effectiveness of the TanhReLU-based Convolutional Neural Network (CNN) is evaluated using metrics such as accuracy, sensitivity, and specificity (see Section 3.2). The experiments were executed on a desktop equipped with an Intel i7 CPU operating at 3.33GHz, an Nvidia RTX 1080Ti GPU, 32GB RAM, and running Windows 10. This system configuration ensured consistent testing conditions throughout the experiments.

3.1 Feature importance analysis

To quantify the contribution of Hilbert-Huang Transform (HHT) features compared to original signal features to model decision-making, this study employed the SHAP (SHapley Additive exPlanations) method (16, 17) for feature importance analysis, which illustrated in Figure 2. The specific procedure is as follows: using the KernelSHAP explainer (applicable to any model), one training sample was randomly selected as the background, and calculations were performed on 10 independent test samples. Feature importance was quantified by calculating the mean absolute SHAP value (|SHAP|) for each channel, with specific separation of HHT feature channels (containing amplitude and frequency information) and original signal channels. Finally, the contributions were compared by calculating the importance ratio R (R = Mean HHT Feature Importance/Mean Original Feature Importance). The experimental results show that HHT features exhibit significantly higher importance than original features, with the importance ratio R reaching as high as 3.52 2. This fully demonstrates that HHT features play a dominant role in model decision-making.

Figure 2

Figure 2. Feature importance comparison.

3.2 Performance on identifying MDD

Figure 3 presents the model’s learning curves. As shown, the training accuracy (red line) rises rapidly within the first few epochs and approaches 1 (nearly 100%), indicating that the model has sufficient capacity to fit the training data well. Meanwhile, the validation accuracy (blue line) also increases quickly at an early stage and eventually becomes close to the training accuracy, demonstrating good generalization to unseen data. Both the training loss and validation loss (green and black dashed lines, respectively) decrease steadily and converge toward zero, with the training and validation accuracies remaining highly consistent. Therefore, the model does not exhibit any obvious signs of underfitting or overfitting.

Figure 3

Figure 3. Learning curve for classifying MDD.

Finally, the classification performance of the proposed model was assessed on the designated test set using a Leave-One-Subject-Out cross-validation protocol. It achieved an impressive accuracy of 98.78%, along with a sensitivity of 99.23% and a specificity of 98.27%, as summarized in Table 1. Among the baseline approaches, CapsuleNet demonstrates strong specificity (99.23%) but comparatively low sensitivity (89.01%), leading to a moderate overall accuracy of 94.42%. ResNet-16 yields the lowest performance across all metrics, particularly in specificity (74.79%), which significantly reduces its classification reliability (82.26% accuracy). The MLRW method shows balanced but suboptimal results with 95% sensitivity and 80% specificity.

Table 1

Table 1. Comparative analysis of the performance between the proposed methodology and existing state-of-the-art approaches.

Our proposed model, both with and without the Hilbert-Huang Transform (HHT), outperforms all baselines. The ablated version (without HHT) achieves a notable accuracy of 98.61% and the highest specificity (99.25%), indicating a strong ability to correctly identify negative cases. However, the full version of our model (with HHT) achieves the highest sensitivity (99.23%) and the best overall accuracy (98.78%), demonstrating improved detection of positive cases and overall classification robustness.

This comparison highlights the benefit of incorporating the HHT module, which effectively enhances the model’s sensitivity while maintaining high specificity. The trade-off between the two versions of our model reflects a subtle balance between reducing false negatives and false positives, which is crucial in clinical applications where missed detections can have serious consequences.

4 Discussions and conclusions

Importance of HHT: Quantitatively, our model achieved 98.78% test accuracy (sensitivity 99.23%, specificity 98.27%), outperforming conventional baselines (see Table 1). In particular, incorporating the HHT layer boosted sensitivity from 98.05% to 99.23%, at a minimal cost to specificity. The SHAP feature-importance analysis further confirms the impact of HHT-based features: the mean absolute SHAP value for HHT-derived channels was 3.52 times that of the original signal channels. In other words, HHT-driven time-frequency components dominate the model’s decision-making, highlighting that the adaptive nonlinear features extracted by HHT are far more informative than raw time-domain EEG samples. This dominance of nonlinear features is consistent with prior studies showing that nonlinear EEG characteristics (e.g. asymmetry, entropy measures) are powerful biomarkers of depression. By embedding HHT as a network layer, our network learns task-specific intrinsic mode functions (IMFs) directly in the time domain, which aligns with the known strength of HHT: it is explicitly designed for nonstationary, nonlinear data and preserves the instantaneous frequency content of the signal.

Limitations and future works: Nevertheless, there are limitations. The high training accuracy warrants caution about overfitting to the specific dataset, even though validation results were strong. Like most EEG-depression studies, our dataset size is modest, and model generalization must be tested on larger and more diverse cohorts. As noted in recent surveys, EEG-based models sometimes suffer from small sample sizes and heterogeneous protocols. Additionally, we used only EEG data; integrating other modalities (e.g. MRI, clinical surveys, genetic or demographic data) could further enhance accuracy and clinical applicability. Existing research demonstrates that sophisticated multi-path feature fusion (18) techniques enable deep integration of heterogeneous data (19). In future work, we plan to extend HHT-SANN on additional datasets and modalities. Furthermore, the mobile/wearable deployment aligns perfectly with our ongoing efforts to optimize the framework for real-time EEG applications, a priority for future translational research aimed at point-of-care utility.

Conclusions: In conclusion, this study presents HHT-SANN, a novel self-attention neural network that embeds the Hilbert–Huang Transform as a network layer for EEG-based depression screening. By leveraging the adaptive, nonlinear time-frequency decomposition capabilities of HHT alongside attention mechanisms, the proposed model achieves superior classification performance, with 98.78% accuracy, 99.23% sensitivity, and 98.27% specificity. SHAP-based analysis reveals that HHT-derived features contribute over three times more than original EEG signals, highlighting their critical role in decision-making. These results demonstrate that HHT-SANN not only improves diagnostic accuracy but also enhances interpretability, offering a powerful and practical tool for advancing mental health diagnostics through neuroinformatics.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material. Further inquiries can be directed to the corresponding authors.

Author contributions

JC: Conceptualization, Writing – original draft. KT: Methodology, Writing – review & editing. YY: Formal analysis, Writing – review & editing. JL: Software, Writing – review & editing.

Funding

The author(s) declare financial support was received for the research and/or publication of this article. This work was supported in part by the Practical Center for the Integration of Industry and Education in New Generation Information Technology (CJRH2024102); and in part by the Suzhou Industrial Park Institute of Service Outsourcing University-level Research Project (SISOKY-YB202203); and in part by the Hubei Provincial Natural Science Foundation Joint Fund Project (2025AFD018).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declare that no Generative AI was used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Lu Z, Wang J, Wang F, and Wu Z. Application of graph frequency attention convolutional neural networks in depression treatment response. Front Psychiatry. (2023) 14:1244208. doi: 10.3389/fpsyt.2023.1244208

PubMed Abstract | Crossref Full Text | Google Scholar

2. Wang F, Ke H, Ma H, and Tang Y. Deep wavelet temporal-frequency attention for nonlinear fmri factorization in asd. Pattern Recognition. (2025) 165:111543. doi: 10.1016/j.patcog.2025.111543

Crossref Full Text | Google Scholar

3. Wang F, Ke H, and Tang Y. Fusion of generative adversarial networks and non-negative tensor decomposition for depression fmri data analysis. Inf Process Manage. (2025) 62:103961. doi: 10.1016/j.ipm.2024.103961

Crossref Full Text | Google Scholar

4. Ke H, Chen D, Yao Q, Tang Y, Wu J, Monaghan J, et al. Deep factor learning for accurate brain neuroimaging data analysis on discrimination for structural mri and functional mri. IEEE/ACM Trans Comput Biol Bioinf. (2024) 21:582–95. doi: 10.1109/TCBB.2023.3252577

PubMed Abstract | Crossref Full Text | Google Scholar

5. Aydın S. Cross-validated Adaboost classification of emotion regulation strategies identified by spectral coherence in resting-state. Neuroinformatics. (2022) 20:627–39. doi: 10.1007/s12021-021-09542-7

PubMed Abstract | Crossref Full Text | Google Scholar

6. Aydin S, Çetin FH, Çikili Uytun M, Babadağı Z, Güven AS, and Işık Y. Comparison of domain specific connectivity metrics for estimation brain network indices in boys with ADHD-C. Biomed Signal Process Control. (2022) 76:103626. doi: 10.1016/j.bspc.2022.103626

Crossref Full Text | Google Scholar

7. Trambaiolli LR and Biazoli CE. Resting-state global EEG connectivity predicts depression and anxiety severity. Annu Int Conf IEEE Eng Med Biol Soc (EMBC). (2020), 3707–10. doi: 10.1109/EMBC44109.2020.9176161

PubMed Abstract | Crossref Full Text | Google Scholar

8. Aydın S and Akın B. Machine learning classification of maladaptive rumination and cognitive distraction in terms of frequency specific complexity. Biomed Signal Process Control. (2022) 77:103740. doi: 10.1016/j.bspc.2022.103740

Crossref Full Text | Google Scholar

9. Hasanzadeh F, Mohebbi M, and Rostami R. Analysis of EEG-derived brain networks for predicting rTMS treatment outcomes in MDD patients. Biomed Signal Process Control. (2024) 96:106613. doi: 10.1016/j.bspc.2024.106613

Crossref Full Text | Google Scholar

10. Ho CC, Peng SJ, Yu YH, Chu YR, Huang SS, and Kuo PH. In perspective of specific symptoms of major depressive disorder: Functional connectivity analysis of electroencephalography and potential biomarkers of treatment response. J Affect Disord. (2024) 367:944–50. doi: 10.1016/j.jad.2024.06.012

PubMed Abstract | Crossref Full Text | Google Scholar

11. Mumtaz W, Xia L, Mohd Yasin MA, Azhar Ali SS, and Malik AS. A wavelet-based technique to predict treatment outcome for major depressive disorder. PloS One. (2017) 12:1–30. doi: 10.1371/journal.pone.0171409

PubMed Abstract | Crossref Full Text | Google Scholar

12. Ke H, Chen D, Shi B, Zhang J, Liu X, Zhang X, et al. Improving brain e-health services via high-performance EEG classification with Grouping Bayesian Optimization. IEEE Trans Serv Computing. (2020) 13:696–708. doi: 10.1109/TSC.2019.2962673

Crossref Full Text | Google Scholar

13. Zhou Q, Sun S, Wang S, and Jiang P. Tanhrelu -based convolutional neural networks for mdd classification. Front Psychiatry. (2024) 15:1346838. doi: 10.3389/fpsyt.2024.1346838

PubMed Abstract | Crossref Full Text | Google Scholar

14. Wang Z, Chen B, Zhao X, Liu X, and Liu X. Research on the application of deep neural network in the classification of eeg data of major depressive disorder and bipolar disorder and optimization strategy. In: Liu W, Wang Q, Feng J, and Zhang W, editors. Proceedings of the 4th international conference on frontiers of electronics, information and computation technologies (ICFEICT 2024). Springer Nature Singapore, Singapore (2025). p. 263–70.

Google Scholar

15. Xiong M, Yang X, Chen H, Aly WHF, AlTameem A, Saudagar AKJ, et al. Cloth-changing person re-identification with invariant feature parsing for uavs applications. IEEE Trans Vehicular Technol. (2024) 73:12448–57. doi: 10.1109/TVT.2024.3388249

Crossref Full Text | Google Scholar

16. Li Q, Ji Y, Zhu M, Zhu X, and Sun L. Unsupervised feature selection using chronological fitting with shapley additive explanation (shap) for industrial time-series anomaly detection. Appl Soft Computing. (2024) 155:111426. doi: 10.1016/j.asoc.2024.111426

Crossref Full Text | Google Scholar

17. Dong H, Chen D, Chen Y, Tang Y, Yin D, and Li X. A multi-task learning model with reinforcement optimization for asd comorbidity discrimination. Comput Methods Programs Biomedicine. (2024) 243:107865. doi: 10.1016/j.cmpb.2023.107865

PubMed Abstract | Crossref Full Text | Google Scholar

18. Xiong M, Yang X, Sun Z, Hu X, Alzahrani AI, and Muhammad K. Rffr-net: Robust feature fusion and reconstruction network for clothing-change person re-identification. Inf Fusion. (2025) 118:102885. doi: 10.1016/j.inffus.2024.102885

Crossref Full Text | Google Scholar

19. Wang F, Ke H, and Cai C. Deep wavelet self-attention non-negative tensor factorization for non-linear analysis and classification of fmri data. Appl Soft Computing. (2025) 182:113522. doi: 10.1016/j.asoc.2025.113522

Crossref Full Text | Google Scholar