Abstract
Introduction:
Electroencephalogram (EEG)-based fatigue classification is important for vigilance monitoring. Reliable recognition remains challenging due to fatigue-related neural changes that involve both spectral-temporal dynamics and altered inter-channel interactions.
Methods:
This study develops a lightweight Dual-Branch Multidimensional Attention Fusion Network (DB-MDAFNet) that integrates spectral-temporal features and PCC-derived inter-channel dependency representations for EEG-based fatigue classification. The framework uses differential entropy features extracted from five canonical EEG frequency bands as input. A parameter-free multidimensional enhancement (PF-ME) module is applied for feature recalibration. It then extracts complementary representations via a Channel Attention-Multi-scale Temporal (CA-MT) branch and a Functional Connectivity-Topological Encoding (FC-TE) branch derived from Pearson correlation coefficients (PCC). The fused representation is used to classify EEG samples into alert, tired, and drowsy states. Experiments were conducted on the public SEED-VIG dataset and the self-constructed video game-based fatigue dataset (DVG).
Results:
Under identical subject-dependent settings, DB-MDAFNet achieved 85.58% accuracy on SEED-VIG and 97.12%–98.56% accuracy on DVG datasets. The model maintained a compact architecture, with 0.185 M parameters under the DVG input configuration. Ablation results indicated the complementary contribution of the two branches. Model interpretability analyses suggested that the learned representations contained spatially structured patterns across frontal, parietal, and occipital electrode regions.
Discussion:
The proposed framework, integrating multi-scale temporal learning with connectivity-derived topological encoding, provides a lightweight and interpretable architecture for EEG-based fatigue monitoring.
1 Introduction
Fatigue is a major factor affecting human performance and safety in transportation, aviation, and industrial work environments (; ). Prolonged cognitive demands and monotonous tasks can reduce attention, slow reaction times, and impair decision-making. Reliable fatigue monitoring is therefore important for safety management, human-machine interaction, and health informatics. Fatigue assessment generally relies on subjective reports or objective physiological and behavioral measurements (; Shahid et al., 2010; ). Among physiological measures, scalp EEG non-invasively records electrical activity with millisecond-scale temporal resolution (). Accordingly, EEG has been widely used to characterize affective states (), support neurorehabilitation (; ; ), and monitor fatigue in safety-critical environments (; Subasi et al., 2022; Xu et al., 2023).
Neurophysiological evidence indicates that fatigue and cognitive load are associated with frequency-specific changes in EEG activity (; ; ). Theta and alpha activity generally increase as fatigue develops (; ), whereas beta activity tends to decrease with reduced alertness (; Tran et al., 2020; ). Fatigue is also associated with alterations in large-scale functional networks, including the default mode network (). Together, these findings motivate EEG representations that retain both spectral-temporal activity and inter-channel relationships. However, EEG-based fatigue classification remains difficult because EEG signals are nonstationary, noise-sensitive, and highly variable across individuals (; Rudroff, 2026).
Deep learning () methods, including convolutional neural networks (CNNs) (Rajwal and Aggarwal, 2023), recurrent neural networks (RNNs) (), and graph neural networks (GNNs) (), have improved automatic EEG feature extraction. Subasi et al. (Subasi et al., 2022) combined a flexible analytic wavelet transform with multi-boosting to distinguish fatigue from alertness. Wang et al. (Wang et al., 2024) employed a deep residual shrinkage network to suppress noise and redundant EEG features recorded in a construction environment. To address inter-subject variability, Liu et al. () investigated transfer learning for cross-subject mental-fatigue recognition. Attention mechanisms () adaptively reweight feature representations and have been widely incorporated into deep neural networks. Representative approaches include squeeze-and-excitation blocks () for channel recalibration and transformer self-attention for sequence modeling (Vaswani et al., 2017). Neural processing depends on coordinated activity across distributed brain regions rather than isolated regional responses (; Uhlhaas and Singer, 2006). Functional brain networks have been shown to reconfigure across cognitive loads (Zhang et al., 2023). Qi et al. () examined coherence, phase-lag index, and partial-directed coherence-based networks to predict individual behavioral impairment induced by mental fatigue. Chen et al. () combined PCC and magnitude-squared coherence within a dual-stream GCN to distinguish alert and fatigued states.
Nevertheless, existing EEG fatigue classifiers differ in how they represent temporal dynamics, frequency-band information, and inter-channel dependencies. Moreover, flattening or compressing frequency-band information during feature extraction may obscure band-specific physiological patterns. The limited availability of dedicated EEG fatigue datasets restricts model development and independent evaluation.
To address these limitations, this study constructs an EEG fatigue dataset based on video game (DVG) and proposes a Dual-Branch Multidimensional Attention Fusion Network (DB-MDAFNet). The DVG paradigm combines action-video-game-based fatigue induction with self-reported fatigue assessment and resting-state EEG acquisition. DB-MDAFNet comprises a parameter-free multidimensional enhancement module (PF-ME), a channel-attention multi-scale temporal branch (CA-MT), and a functional-connectivity topological encoding branch (FC-TE). These components jointly capture band-specific temporal dynamics and inter-channel connectivity while limiting additional trainable parameters. The main contributions of this study are as follows:
An EEG fatigue dataset (DVG) was constructed using an action video-gaming task to induce mental fatigue, with concurrent collection of self-reported fatigue ratings and EEG data.
A dual-branch EEG fatigue classification model, DB-MDAFNet, was designed to combine multi-scale temporal feature learning with Pearson correlation coefficient (PCC)-based functional connectivity representation while preserving the channel, frequency-band, and temporal structure of differential entropy (DE) features.
The proposed method was evaluated on both the public SEED-VIG and DVG datasets under a subject-dependent protocol. Comparative tests and ablation analyses evaluated classification performance, and descriptive visualizations illustrated the learned feature weight distributions.
2 Materials and methods
2.1 Dataset
DB-MDAFNet was evaluated on the self-constructed DVG dataset and the public SEED-VIG dataset.
2.1.1 Self-constructed dataset (DVG)
Fourteen male students (mean age: 23 years) from the University of Electronic Science and Technology of China were recruited. Two participants withdrew for personal reasons, and data from the remaining twelve participants were analyzed. None of the participants reported extensive or long-term video-gaming experience. All participants were right-handed, had normal or corrected-to-normal vision, and reported no history of neurological disorders or substance misuse. Participants were instructed to avoid coffee, tea, alcohol, sleep deprivation, and strenuous exercise before each session. The protocol was approved by the Departmental Ethics Committee of the University of Electronic Science and Technology of China and complied with the Declaration of Helsinki. All participants had written informed consent before participation.
The DVG paradigm (Figure 1) combined video-game-based fatigue induction, eyes-closed resting-state EEG recording, and the modified Fatigue Self-Assessment Scale (FSAS) (Wang and Xue, 2009). EEG was recorded using a 32-channel BioSemi Active Two System, with electrodes positioned according to the extended international 10–20 system (). EEG signals were sampled at 2048 Hz. The mental fatigue subscale of the modified FSAS was administered at each EEG recording time point. Each item of the modified FSAS was rated from 0 to 9, with higher scores indicating greater agreement with the described condition.
Figure 1
The three DVG datasets corresponded to Co-op vs. AI (DVG1), Normal (DVG2), and Ranked (DVG3) game modes. Each participant completed one mode per session. The session order was counterbalanced. Sessions were separated by at least three days to reduce potential carryover effects. During each 4-h session, the mental fatigue subscale and 5-min eyes-closed EEG recording were obtained at baseline (T1), 2 h (T2), and 4 h (T3). According to the predefined experimental stages, EEG samples collected at T1, T2, and T3 were operationally labeled as alert, tired, and drowsy, respectively. Then, FSAS scores at each time point were used to validate the stage-based labeling for each participant. The session-averaged FSAS scores (Figure 2) increased significantly across the three time points (Friedman test: χ² (2) = 24, p < 0.0001). Furthermore, Dunn’s post hoc tests showed significant differences between T1 and T2 (adjusted p = 0.0429), T2 and T3 (adjusted p = 0.0429), and T1 and T3 (adjusted p < 0.0001).
Figure 2
2.1.2 Public SEED-VIG dataset
SEED-VIG (Zheng and Lu, 2017) is a public vigilance-estimation dataset collected during simulated driving. It includes 23 healthy adults (11 males, 12 females; age: 23.3 ± 1.4 years) who completed an approximately 2-h monotonous driving task using a virtual-reality driving system. EEG was acquired using a Neuroscan system at 1,000 Hz, and eye movements were recorded synchronously using SMI eye-tracking glasses. Vigilance was annotated using the percentage of eyelid closure (PERCLOS) derived from the eye-tracking recordings (Equation 1):
where blink, fixation, saccade, and CLOS denote the durations of blinks, fixations, saccades, and eye closures, respectively. For the present three-class experiments, PERCLOS (P) values of , , and were assigned to the alert, tired, and drowsy classes, respectively.
2.2 Data preprocessing
Initially, EEG signals were processed with band-pass filtering (0.5–50 Hz) to remove slow baseline drifts and high-frequency noise. In addition, a 50-Hz notch filter was applied to suppress power-line interference. EEG was re-referenced to the infinite reference via REST (Yao, 2001). Ocular, muscle, and cardiac artifacts were removed using independent component analysis (ICA) (). Noisy channels were interpolated when necessary. Subsequently, resampling and segmentation were performed. The signals were down-sampled to 1000 Hz (DVG datasets) and 200 Hz (SEED-VIG dataset), with corresponding epoch lengths of 3.5 s and 8 s, respectively. Epochs were generated using non-overlapping windows.
Multi-band filtering was conducted using a fifth-order Butterworth band-pass filter to decompose the EEG signals into five canonical bands, namely delta (0.5–4 Hz), theta (4–8 Hz), alpha (8–12 Hz), beta (12–30 Hz), and gamma (30–50 Hz). Finally, DE features were calculated for each EEG channel and frequency band by the following formulas (Equations 2, 3):
where denotes the number of sampling points within the sub-window, is the i-th sample of the EEG signal, and and are its mean and variance.
2.3 Model architecture
As shown in Figure 3, DB-MDAFNet consists of a shared parameter-free multidimensional enhancement (PF-ME) module, two parallel feature-extraction branches (CA-MT and FC-TE), and a fusion classifier. The input is the DE feature tensor , where B, C, F, and P (Equation 2, 3) denote the batch size, number of channels, frequency bands, and DE temporal points, respectively. PF-ME first performs element-wise recalibration without additional trainable parameters. The recalibrated tensor is then passed to two parallel branches. The CA-MT branch combines channel attention with multiscale one-dimensional convolutions to extract channel-time-aware representations from the concatenated band-time sequence. In parallel, the FC-TE branch uses PCC-derived attention and two-dimensional convolutions to encode inter-channel connectivity patterns. The two branch features are concatenated and passed to the classification head. The classifier produces predicted labels for alert, tired, and drowsy states.
Figure 3
2.3.1 Parameter-free multidimensional enhancement module
To enhance the model’s capacity to capture critical features in EEG signals, we incorporate a PF-ME module based on SimAM. For each feature map, the squared deviation of an element from the corresponding mean is (Equation 4):
where denotes the squared difference value for the c-th channel at the p-th time point in the f-th frequency band. denotes the frequency-wise global mean. The corresponding variance estimate is (Equation 5):
The element-wise weight c,f,p is calculated as (Equation 6):
where is a small constant (set to 10-4) to ensure numerical stability, and is the sigmoid function. The recalibrated tensor is obtained through an element-wise product (Equation 7):
The enhanced tensor is reshaped to a channel-centered layout, and the frequency and time dimensions are concatenated in a fixed order (Equation 8):
where represents the reshaped alignment matrix, with denoting the integrated sequence length. This operation effectively fuses spectral and temporal information into a unified spatial feature space.
2.3.2 Channel attention – multiscale temporal branch
The CA-MT branch learns channel-weighted features from . A channel descriptor is obtained by averaging across the sequence dimension (Equation 9):
Subsequently, batch normalization and sigmoid activation are used to generate the channel weights (Equation 10):
where denotes the weight assigned to channel in sample . The recalibrated feature is obtained via an element-wise Hadamard product (Equation 11):
The weighted tensor is then processed by three parallel Conv1D paths with kernel sizes (Equation 12):
where denotes global average pooling. The resulting multi-scale feature vectors are concatenated to form a unified representation (Equation 13):
Finally, a fully connected layer (FC), activation, and dropout operation are applied to produce a 128-dimensional feature (Equation 14).
2.3.3 Functional connectivity – topological encoding branch
FC-TE encodes PCC-based inter-channel connectivity patterns. An initial PCC matrix is computed across the concatenated band-time sequence (Equation 15):
where u and v denote distinct EEG channels, denotes the covariance calculation, and denotes the standard deviation. A small constant is added to ensure numerical stability. Learnable projections map each PCC row to query and key vectors (Equation 16):
The channel-mixing matrix is (Equation 17):
The attention matrix left-multiplies the channel features (Equation 18):
Following this enhancement, PCC is recomputed from (Equation 19):
A singleton feature-map dimension is added before convolution. Three Conv2D, batch-normalization, ReLU, and max-pooling blocks are then applied (Equation 20):
where . Two-dimensional global average pooling produces (Equation 21):
2.3.4 Feature fusion and classification
The two branch features are concatenated (Equation 22):
where denotes the concatenation operation, yielding a unified feature vector . Subsequently, the fused feature is fed into a fully connected classification layer (Equation 23):
where represents a fully connected linear transformation, and Z denotes the class-logit matrix.
The class probabilities are obtained by applying the softmax function to the logits (Equation 24):
where is the predicted probability that the i-th sample belongs to the k-th class. The predicted class is given by (Equation 25):
where denotes the predicted class label of the i-th sample.
The proposed DB-MDAFNet is optimized by minimizing the cross-entropy loss (Equation 26).
where denotes the batch size, denotes the number of target classes, is the ground-truth label of the i-th sample for the j-th class, and is the corresponding predicted probability. All trainable parameters are optimized end-to-end by backpropagation.
2.3.5 Algorithm
The forward classification procedure of the DB-MDAFNet algorithm is presented in Algorithm 1.
Algorithm 1
2.4 Experimental implementation details and evaluation metrics
DB-MDAFNet was implemented using PyTorch 2.1.0 () and trained on an NVIDIA GeForce RTX 2060 GPU. The model was optimized using Adam () with a batch size of 32 and an initial learning rate of 0.001. Batch normalization and dropout of 0.2 were applied to improve generalization performance and mitigate overfitting. The convolution kernel stride is 1.
A subject-dependent protocol was adopted for experimental validation. For both the SEED-VIG and DVG datasets, each subject’s data were partitioned into an 80% training allocation and a 20% independent testing set. This scheme ensured that the model was validated on entirely unseen data for each subject.
To ensure a fair comparison, all baseline models were independently retrained using the same preprocessed EEG data, input features, subject-specific data partitions, and evaluation protocol as the proposed model. Unless otherwise required by the original model architecture, the optimizer, learning rate, batch size, and predefined number of training epochs were kept consistent across models.
To rigorously evaluate the classifier’s performance, four core metrics were employed, including accuracy (Acc), macro-precision (Pre), macro-recall (Rec), and macro-F1-score (F1). These evaluation metrics were calculated based on the following equations (Equations 27–30):
where K denotes the number of classes, and N denotes the total number of samples. , , and represent true positives, false positives, and false negatives for class k, respectively.
3 Results
3.1 Within-subject experimental results
Table 1 presents the average subject-dependent performance on the SEED-VIG and DVG datasets. In the current experiments, DB-MDAFNet achieved a mean classification accuracy of 85.58% and an F1 of 74.50% on the SEED-VIG dataset. Mean accuracy and F1 exceeded 97% for the DVG datasets (DVG1–DVG3).
Table 1
| Dataset | Acc (%) | Pre (%) | Rec (%) | F1 (%) | |
|---|---|---|---|---|---|
| SEED-VIG | 85.58 ± 1.64 | 80.98 ± 2.81 | 73.08 ± 2.34 | 74.50 ± 2.49 | |
| DVG | DVG1 | 98.24 ± 0.62 | 98.40 ± 0.56 | 98.25 ± 0.61 | 98.22 ± 0.63 |
| DVG2 | 97.12 ± 1.17 | 97.47 ± 1.00 | 97.10 ± 1.18 | 97.06 ± 1.20 | |
| DVG3 | 98.56 ± 0.61 | 98.70 ± 0.53 | 98.54 ± 0.62 | 98.54 ± 0.31 | |
Subject-dependent classification performance (mean ± SEM).
3.2 Comparative experiments
DB-MDAFNet was compared with some representative EEG classification models. These models included HMS-TENet (Tang et al., 2024), T-A-MFFNet (), M-ShallowConvNet (), TSception (), LGGNet (), E-5-D-BCDRNet (), and SFT-Net (). Comparisons were performed on SEED-VIG and DVG2 using accuracy, precision, recall, and F1. To assess whether these improvements were statistically significant, two-sided paired Wilcoxon signed-rank tests were used to compare participant-level accuracies (SEED-VIG: n = 23; DVG2: n = 12). The unit of analysis was the individual participant. For each comparison, the subject-level classification accuracy of the proposed model was paired with that of the corresponding baseline model.
As shown in Table 2, DB-MDAFNet obtained the highest mean value for each reported metric on the SEED-VIG dataset. Its accuracy and F1 were 85.58% and 74.50%, exceeding those of the strongest baseline LGGNet by 1.40% and 3.21%, respectively.
Table 2
| Model | Acc (%) | Pre (%) | Rec (%) | F1 (%) | p-value (Acc) | Significant |
|---|---|---|---|---|---|---|
| HMS-TENet (Tang et al., 2024) | 79.37 ± 1.99 | 65.24 ± 2.57 | 62.46 ± 2.58 | 62.04 ± 2.74 | <0.0001 | √ |
| T-A-MFFNet () | 78.90 ± 2.11 | 60.94 ± 3.41 | 57.12 ± 3.02 | 55.23 ± 3.31 | <0.0001 | √ |
| M-ShallowConvNet () | 82.83 ± 1.78 | 72.79 ± 2.86 | 67.93 ± 2.25 | 67.37 ± 2.43 | <0.0001 | √ |
| TSception () | 83.64 ± 1.89 | 73.39 ± 3.33 | 68.82 ± 2.77 | 68.85 ± 2.88 | 0.0124 | √ |
| LGGNet () | 84.18 ± 1.79 | 77.49 ± 2.63 | 70.79 ± 2.74 | 71.29 ± 2.74 | 0.0112 | √ |
| DB-MDAFNet | 85.58 ± 1.64 | 80.98 ± 2.81 | 73.08 ± 2.34 | 74.50 ± 2.49 | - | - |
Comparative experimental results of the SEED-VIG dataset (mean ± SEM).
On the DVG2 dataset (Table 3), DB-MDAFNet surpassed all baselines with an accuracy of 97.12%, and Pre, Rec, and F1 reached 97.47%, 97.10%, and 97.06%, respectively. Relative to M-ShallowConvNet, these results represented gains of 2.41% in accuracy and 2.34% in F1.
Table 3
| Model | Acc (%) | Pre (%) | Rec (%) | F1 (%) | p-value (Acc) | Significant |
|---|---|---|---|---|---|---|
| E-5-D-BCDRNet () | 79.17 ± 3.60 | 80.16 ± 3.51 | 79.18 ± 3.59 | 78.23 ± 3.98 | <0.001 | √ |
| M-ShallowConvNet () | 94.71 ± 1.67 | 95.23 ± 1.54 | 94.72 ± 1.67 | 94.72 ± 1.67 | 0.0313 | √ |
| T-A-MFFNet () | 81.41 ± 3.00 | 83.52 ± 2.91 | 81.54 ± 3.04 | 80.59 ± 3.48 | <0.001 | √ |
| SFT-Net () | 83.17 ± 2.38 | 86.31 ± 1.78 | 83.10 ± 2.37 | 82.33 ± 2.64 | <0.001 | √ |
| LGGNet () | 92.63 ± 2.04 | 92.83 ± 2.02 | 92.66 ± 2.03 | 92.56 ± 2.04 | 0.0039 | √ |
| DB-MDAFNet | 97.12 ± 1.17 | 97.47 ± 1.00 | 97.10 ± 1.18 | 97.06 ± 1.20 | - | - |
Comparative experimental results of the self-constructed dataset (DVG2, mean ± SEM).
3.3 Ablation study
To quantify the contribution of each core component within DB-MDAFNet, a systematic ablation study was conducted using a within-subject design. We evaluated four model variants: (1) the full DB-MDAFNet, (2) without the PF-ME module, (3) without the CA-MT module, and (4) without the FC-TE module.
As shown in Table 4, the full DB-MDAFNet achieved an accuracy of 98.24% and an F1 of 98.22%. Removing PF-ME reduced accuracy by 0.53% and F1 by 0.55%. Removing the CA-MT and FC-TE modules reduced accuracy to 77.17% and 81.09%, respectively. These reductions indicate that both the CA-MT and the FC-TE are essential components of the architecture.
Table 4
| Dataset | Acc (%) | Pre (%) | Rec (%) | F1 (%) |
|---|---|---|---|---|
| Full | 98.24 | 98.40 | 98.25 | 98.22 |
| w/o PF-ME | 97.71 | 97.70 | 97.81 | 97.67 |
| w/o CA-MT | 77.17 | 76.30 | 76.34 | 73.17 |
| w/o FC-TE | 81.09 | 82.17 | 81.21 | 80.55 |
Ablation results on DVG1.
3.4 Computational complexity
The computational complexity of DB-MDAFNet was evaluated under the DVG input configuration and compared with four representative baseline models. As shown in Table 5, DB-MDAFNet contained 0.185 M trainable parameters, required approximately 0.044 G FLOPs per sample, and occupied 0.704 MiB. It had substantially fewer parameters and a smaller model size than T-A-MFFNet and E-5-D-BCDRNet, while requiring fewer FLOPs than E-5-D-BCDRNet and SFT-Net. Although SFT-Net and LGGNet contained fewer parameters, and T-A-MFFNet and LGGNet required slightly fewer FLOPs, DB-MDAFNet maintained fewer than 0.2 M parameters, less than 0.05 G FLOPs, and a model size below 1 MB. Taken together with the classification results, these results suggest that DB-MDAFNet achieves a favorable trade-off among classification performance, model compactness, and computational cost.
Table 5
| Dataset | Model | Params (M) | FLOPs (G) | Model size (MiB) |
|---|---|---|---|---|
| DVG | DB-MDAFNet | 0.185 | 0.044 | 0.704 |
| T-A-MFFNet () | 0.909 | 0.041 | 3.467 | |
| E-5-D-BCDRNet () | 1.199 | 24.139 | 4.575 | |
| SFT-Net () | 0.141 | 0.085 | 0.537 | |
| LGGNet () | 0.102 | 0.041 | 0.390 |
Computational complexity comparison of DB-MDAFNet and representative baseline models under the DVG input configuration.
3.5 Model visualization and interpretability
To examine the discriminative structure and internal feature-focus patterns learned by DB-MDAFNet, we visualized the fused feature representations and the channel-related weights generated by the two branches. Figure 4 illustrates the evolution of feature manifolds from raw DE features to fused DB-MDAFNet features across randomly selected subjects (sub5, sub7, and sub10) of DVG1, colored by the three states (alert, tired, and drowsy). As demonstrated in Figure 4a, the raw DE features exhibit a highly mixed distribution for all visualized subjects. After processing by DB-MDAFNet, the class overlap was reduced, yielding compact and well-separated clusters (Figure 4b).
Figure 4
To examine the internal spatial focus of the model, we visualized the attention weights from the CA-MT module and mapped them onto the scalp topography. Figure 5a illustrates the weights for three randomly selected subjects from the DVG1 dataset (sub2, sub7, and sub9). For these participants, relatively high weights appeared over frontal, parietal, and occipital electrodes. These weights indicate the relative contribution of the corresponding channel features to the model’s classification process.
Figure 5
Figure 5b presents the subject-specific PCC-guided channel-attention matrices generated by the FC-TE branch. Each element represents the normalized weight assigned to an input channel when updating the representation of a query channel. Several columns showed relatively high weights across multiple query channels, indicating that the corresponding input channels contributed more strongly to feature aggregation. These high-weight channels were mainly located over frontal and occipital scalp areas and partially overlapped with the channels receiving relatively high weights in the CA-MT branch. Importantly, these matrices represent learned channel-mixing patterns.
4 Discussion
The experimental results show that DB-MDAFNet achieves competitive performance in subject-dependent EEG-based fatigue classification. By jointly modeling spectral-temporal dynamics and functional connectivity patterns, the model provides a more comprehensive representation of fatigue-related EEG changes than methods relying on a single feature. The performance degradation observed after removing either the CA-MT or the FC-TE branch further supports the effectiveness of combining these two computational representations within the proposed classification framework.
The performance of DB-MDAFNet may be attributed to its joint use of band-specific temporal features and statistical relationships among EEG channels. Previous studies have reported that fatigue-related EEG changes involve both frequency-specific changes in neural oscillations and altered functional coordination among cortical regions (; ; Tran et al., 2020). Motivated by these findings, the CA-MT branch focuses on local spectral-temporal dynamics through multi-scale one-dimensional convolutions and channel attention, thereby capturing short-term rhythm fluctuations and band-specific variations during fatigue transitions. In parallel, the FC-TE branch models inter-channel dependencies using PCC-based inter-channel connectivity representations, which describe network-level coordination across brain regions. By integrating local oscillatory features with global connectivity information, DB-MDAFNet provides a more comprehensive representation of fatigue-related neural activity and enhances the discrimination of different fatigue states.
The PF-ME module further improves feature representation by introducing lightweight feature recalibration. Based on the parameter-free SimAM mechanism (Yang et al., 2021), PF-ME assigns adaptive importance weights to channel-frequency-temporal feature units without introducing additional trainable parameters. This design enables the model to emphasize features that are more relevant to fatigue detection while preserving a compact architecture. PF-ME serves as a feature-refinement module that enhances fatigue-related representations with limited computational overhead.
The visualization analyses provide descriptive insight into the internal representations and feature-focus patterns learned by DB-MDAFNet. The channel-weight topographic maps learned by the CA-MT module show relatively higher weights over the prefrontal, parietal, and occipital areas. This spatial attention pattern is consistent with prior neurophysiological evidence, as prefrontal and parietal areas are commonly implicated in executive control and sustained attention (), whereas occipital areas are primarily involved in visual information processing (Zeki et al., 1991) during vigilance-related tasks. The FC-TE channel-attention matrices further reveal patterns of inter-channel dependencies. Notably, stronger attention weights are observed over frontal, parietal, and occipital regions, suggesting that DB-MDAFNet may be sensitive to coordinated interactions among fatigue-related cortical areas. Interactions involving frontal and parietal channels may be related to attentional control and executive regulation, and interactions involving occipital channels may reflect the contribution of visual information processing during vigilance-related tasks. Importantly, the overlap between high-weight regions in the CA-MT topographic maps and FC-TE channel-attention matrices indicates that the two branches capture fatigue-related patterns from complementary perspectives. CA-MT emphasizes region-specific spectral-temporal activity, whereas FC-TE highlights connectivity-related inter-channel interactions. Taken together, these visualization results suggest that DB-MDAFNet may learn physiologically meaningful representations from both intra-channel and inter-channel interactions.
Several limitations should be considered. The current experiments adopted a subject-dependent evaluation protocol. Although the results demonstrate the ability of DB-MDAFNet to distinguish fatigue stages within individual participants, its performance under cross-session and cross-dataset conditions remains unknown. Future studies should therefore evaluate the model using leave-one-subject-out validation, independent external datasets, and domain-generalization or adaptation strategies. Despite its compact architecture, DB-MDAFNet requires further validation for real-time fatigue monitoring under online EEG acquisition, and artifact-contaminated recording conditions. Finally, larger-scale datasets involving more diverse subjects, multimodal physiological signals, and fatigue-inducing scenarios are needed to further assess the generalizability and ecological validity of the proposed framework.
Overall, DB-MDAFNet provides an effective and interpretable framework for subject-dependent EEG-based fatigue classification. By integrating multi-scale spectral-temporal modeling, PCC-derived inter-channel relation encoding, and lightweight multidimensional feature recalibration, the model characterizes complementary fatigue-related EEG patterns from both local and network-level perspectives. While further validation is still required in more challenging generalization and real-world application settings, the proposed dual-branch design offers a promising direction for EEG-based fatigue monitoring. It may provide useful methodological insights for broader cognitive state recognition tasks.
5 Conclusion
This study introduced DB-MDAFNet, a dual-branch framework that integrates channel-aware band-time features with PCC-derived inter-channel representations for EEG fatigue classification. Under subject-dependent evaluation, DB-MDAFNet achieved the highest mean performance among the evaluated methods on SEED-VIG and DVG2 and maintained strong performance across DVG1–DVG3. The ablation results indicated that PF-ME, CA-MT, and FC-TE each contributed to model performance. Visualization analyses suggested that the learned representations contained class-discriminative representations and spatially structured channel-weighting patterns. The proposed model provides a lightweight and interpretable architecture for EEG-based fatigue monitoring.
Statements
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
The studies involving humans were approved by the Departmental Ethics Committee of the University of Electronic Science and Technology of China and complied with the Declaration of Helsinki. The participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.
Author contributions
JL: Data curation, Investigation, Methodology, Writing – original draft, Writing – review & editing. SL: Methodology, Writing – review & editing. WL: Methodology, Writing – review & editing. CM: Data curation, Investigation, Writing – review & editing. DG: Supervision, Writing – review & editing. DY: Funding acquisition, Supervision, Writing – review & editing.
Funding
The author(s) declared that financial support was received for this work and/or its publication. This work was supported in part by Brain Science and Brain-like Intelligence Technology-National Science and Technology Major Project (STI 2030-Major Projects) No. 2022ZD0208500, and in part by Sichuan Science and Technology Program under Grant 2024ZDZX0014.
Conflict of interest
The author(s) declared that this work was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Generative AI statement
The author(s) declared that generative AI was not used in the creation of this manuscript.
Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1
AngK. K.GuanC. T. (2017). EEG-based strategies to detect motor imagery for control and rehabilitation. IEEE Trans. Neural Syst. Rehabil. Eng.25, 392–401. doi: 10.1109/TNSRE.2016.2646763
2
AntonenkoP.PaasF.GrabnerR.van GogT. (2010). Using electroencephalography to measure cognitive load. Educ. Psychol. Rev.22, 425–438. doi: 10.1007/s10648-010-9130-y
3
BendakS.RashidH. S. J. (2020). Fatigue in aviation: a systematic review of the literature. Int. J. Ind. Ergon.76, 102928. doi: 10.1016/j.ergon.2020.102928
4
BiasiucciA.FranceschielloB.MurrayM. M. (2019). Electroencephalography. Curr. Biol.29, R80–R85. doi: 10.1016/j.cub.2018.11.052
5
BorghiniG.AstolfiL.VecchiatoG.MattiaD.BabiloniF. (2014). Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neurosci. Biobehav. Rev.44, 58–75. doi: 10.1016/j.neubiorev.2012.10.003
6
ChaiR. F.NaikG. R.NguyenT. N.LingS. H.TranY.CraigA.et al. (2017). Driver fatigue classification with independent component by entropy rate bound minimization analysis in an EEG-based system. IEEE J. Biomed. Health Inf.21, 715–724. doi: 10.1109/Jbhi.2016.2532354
7
ChenK.ChaiS. L.XieT. L.LiuQ.MaL. (2024). EEG spatial inter-channel connectivity analysis: a GCN-based dual stream approach to distinguish mental fatigue status. Artif. Intell. Med.157, 102996. doi: 10.1016/j.artmed.2024.102996
8
CorbettaM.ShulmanG. L. (2002). Control of goal-directed and stimulus-driven attention in the brain. Nat. Rev. Neurosci.3, 201–215. doi: 10.1038/nrn755
9
Díaz-GarcíaJ.González-PonceI.Ponce-BordónJ. C.López-GajardoM. A.Ramírez-BravoI.Rubio-MoralesA.et al. (2022). Mental load and fatigue assessment instruments: a systematic review. Int. J. Environ. Res. Public Health19, 419. doi: 10.3390/ijerph19010419
10
DingY.RobinsonN.TongC.ZengQ.GuanC. (2024). LGGNet: learning from local-global-graph representations for brain–computer interface. IEEE Trans. Neural Networks Learn. Syst.35, 9773–9786. doi: 10.1109/TNNLS.2023.3236635
11
DingY.RobinsonN.ZhangS.ZengQ. H.GuanC. T. (2023). TSception: capturing temporal dynamics and spatial asymmetry from EEG for emotion recognition. IEEE Trans. Affect. Comput.14, 2238–2250. doi: 10.1109/TAFFC.2022.3169001
12
FriesP. (2005). A mechanism for cognitive dynamics: neuronal communication through neuronal coherence. Trends Cognit. Sci.9, 474–480. doi: 10.1016/j.tics.2005.08.011
13
GaoD. R.WangK. J.WangM. Q.ZhouJ. L.ZhangY. Q. (2024). SFT-Net: a network for detecting fatigue from EEG signals by combining 4D feature flow and attention mechanism. IEEE J. Biomed. Health Inf.28, 4444–4455. doi: 10.1109/Jbhi.2023.3285268
14
GergelyfiM.Sanz-ArigitaE. J.SolopchukO.DricotL.JacobB.ZénonA. (2021). Mental fatigue correlates with depression of task-related network and augmented DMN activity but spares the reward circuit. Neuroimage243, 118532. doi: 10.1016/j.neuroimage.2021.118532
15
GoodmanS. P.CollinsB.ShorterK.MorelandA. T.PapicC.HamlinA. S.et al. (2025). Approaches to inducing mental fatigue: a systematic review and meta-analysis of (neuro) physiologic indices. Behav. Res. Methods57, 102. doi: 10.3758/s13428-025-02620-7
16
GülerN. F.ÜbeyliE. D.GülerI. (2005). Recurrent neural networks employing Lyapunov exponents for EEG signals classification. Expert Syst. Appl.29, 506–514. doi: 10.1016/j.eswa.2005.04.011
17
HuJ.ShenL.SunG. (2018). “ Squeeze-and-excitation networks”, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Salt Lake City, UT, USA: IEEE), 7132–7141. doi: 10.1109/CVPR.2018.00745
18
JungT. P.MakeigS.HumphriesC.LeeT. W.McKeownM. J.IraguiV.et al. (2000). Removing electroencephalographic artifacts by blind source separation. Psychophysiology37, 163–178. doi: 10.1111/1469-8986.3720163
19
KimS. J.LeeD. H.LeeS. W. (2022). Rethinking CNN architecture for enhancing decoding performance of motor imagery-based EEG signals. IEEE Access10, 96984–96996. doi: 10.1109/ACCESS.2022.3204758
20
KingmaD. P.BaJ. (2015). “ Adam: a method for stochastic optimization,” in Proceedings of the 3rd International Conference on Learning Representations (ICLR 2015) (San Diego, CA).
21
KlemG. H.LudersH. O.JasperH. H.ElgerC. (1999). The ten-twenty electrode system of the International Federation. The International Federation of Clinical Neurophysiology. Electroencephalogr Clin. Neurophysiol. Suppl.52, 3–6.
22
KleplD.WuM.HeF. (2024). Graph neural network-based EEG classification: a survey. IEEE Trans. Neural Syst. Rehabil. Eng.32, 493–503. doi: 10.1109/TNSRE.2024.3355750
23
KlimeschW. (1999). EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis. Brain Res. Rev.29, 169–195. doi: 10.1016/S0165-0173(98)00056-3
24
LaChapelleD. L.FinlaysonM. A. J. (1998). An evaluation of subjective and objective measures of fatigue in patients with brain injury and healthy controls. Brain Injury12, 649–659. doi: 10.1080/026990598122214
25
LeamyD. J.KocijanJ.DomijanK.DuffinJ.RocheR. A. P.ComminsS.et al. (2014). An exploration of EEG features during recovery following stroke-implications for BCI-mediated neurorehabilitation therapy. J. NeuroEng. Rehabil.11, 9. doi: 10.1186/1743-0003-11-9
26
LeCunY.BengioY.HintonG. (2015). Deep learning. Nature521, 436–444. doi: 10.1038/nature14539
27
LiG.HuangS.XuW.JiaoW.JiangY.GaoZ.et al. (2020). The impact of mental fatigue on brain activity: a comparative study both in resting state and task state using EEG. BMC Neurosci.21, 20. doi: 10.1186/s12868-020-00569-1
28
LiP. R.ZhangY. Q.LiuS. H.LinL. Q.ZhangH. K.TangT.et al. (2023). An EEG-based brain cognitive dynamic recognition network for representations of brain fatigue. Appl. Soft Comput.146, 110613. doi: 10.1016/j.asoc.2023.110613
29
LiX.ZhangY. Z.TiwariP.SongD. W.HuB.YangM. H.et al. (2022). EEG based emotion recognition: a tutorial and review. ACM Comput. Surv.55, 1–57. doi: 10.1145/3524499
30
LiuY. S.LanZ. R.CuiJ.SourinaO.Müller-WittigW. (2020). Inter-subject transfer learning for EEG-based mental fatigue recognition. Adv. Eng. Inf.46, 101157. doi: 10.1016/j.aei.2020.101157
31
LiuZ. H.ZhouY. Q.HaoC.MaN. (2025). Alteration in neural oscillatory activity and phase-amplitude coupling after sleep deprivation: evidence for impairment and compensation effects. J. Sleep Res.34, e14264. doi: 10.1111/jsr.14264
32
LuoY.LiuW.LiH. Q.LuY.LuB. L. (2024). A cross-scenario and cross-subject domain adaptation method for driving fatigue detection. J. Neural Eng.21, 046004. doi: 10.1088/1741-2552/ad546d
33
MattaP. M.BaurèsR.DuclayJ.AlamiaA. (2025). Modulation of beta oscillatory dynamics in motor and frontal areas during physical fatigue. Commun. Biol.8, 687. doi: 10.1038/s42003-025-08122-8
34
McDermottE. J.MetsomaaJ.BelardinelliP.Grosse-WentrupM.ZiemannU.ZrennerC. (2023). Predicting motor behavior: an efficient EEG signal processing pipeline to detect brain states with potential therapeutic relevance for VR-based neurorehabilitation. Virtual Reality27, 347–369. doi: 10.1007/s10055-021-00538-x
35
NiuZ. Y.ZhongG. Q.YuH. (2021). A review on the attention mechanism of deep learning. Neurocomputing452, 48–62. doi: 10.1016/j.neucom.2021.03.091
36
PaszkeA.GrossS.MassaF.LererA.BradburyJ.ChananG.et al. (2019). “ Pytorch: an imperative style, high-performance deep learning library”, in: Advances in Neural Information Processing Systems (Red Hook, NY: Curran Associates), 32, 8024–8035.
37
PengB.ZhangY. Q.WangM. Q.ChenJ.GaoD. R. (2023). T-A-MFFNet: multi-feature fusion network for EEG analysis and driving fatigue detection based on time domain network and attention network. Comput. Biol. Chem.104, 107863. doi: 10.1016/j.compbiolchem.2023.107863
38
PhillipsR. O.KecklundG.AnundA.SallinenM. (2017). Fatigue in transport: a review of exposure, risks, checks and controls. Transport Rev.37, 742–766. doi: 10.1080/01441647.2017.1349844
39
QiP.HuH. Y.ZhuL.GaoL. Y.YuanJ. J.ThakorN.et al. (2020). EEG functional connectivity predicts individual behavioural impairment during mental fatigue. IEEE Trans. Neural Syst. Rehabil. Eng.28, 2080–2089. doi: 10.1109/TNSRE.2020.3007324
40
RajwalS.AggarwalS. (2023). Convolutional neural network-based EEG signal analysis: a systematic review. Arch. Comput. Methods Eng.30, 3585–3615. doi: 10.1007/s11831-023-09920-1
41
RudroffT. (2026). The dual diversity crisis in EEG biomarker research for cognitive fatigue. Psychol. Med.56, e79. doi: 10.1017/S0033291726103808
42
ShahidA.ShenJ. H.ShapiroC. M. (2010). Measurements of sleepiness and fatigue. J. Psychosomatic Res.69, 81–89. doi: 10.1016/j.jpsychores.2010.04.001
43
SubasiA.SaikiaA.BagedoK.SinghA.HazarikaA. (2022). EEG-based driver fatigue detection using FAWT and multiboosting approaches. IEEE Trans. Ind. Inf.18, 6602–6609. doi: 10.1109/Tii.2022.3167470
44
TangM.LiP.ZhangH.DengL.LiuS.ZhengQ.et al. (2024). HMS-TENet: a hierarchical multi-scale topological enhanced network based on EEG and EOG for driver vigilance estimation. Biomed. Technol.8, 92–103. doi: 10.1016/j.bmt.2024.10.003
45
TranY.CraigA.CraigR.ChaiR.NguyenH. (2020). The influence of mental fatigue on brain activity: evidence from a systematic review with meta-analyses. Psychophysiology57, e13554. doi: 10.1111/psyp.13554
46
UhlhaasP. J.SingerW. (2006). Neural synchrony in brain disorders: relevance for cognitive dysfunctions and pathophysiology. Neuron52, 155–168. doi: 10.1016/j.neuron.2006.09.020
47
VaswaniA.ShazeerN.ParmarN.UszkoreitJ.JonesL.GomezA. N.et al. (2017). “ Attention is all you need”, in: Advances in Neural Information Processing Systems (Red Hook, NY: Curran Associates), 30, 5998–6008.
48
WangF.ChenD.ZhangX. (2024). Mental fatigue detection of construction equipment operators based on EEG collected by a novel valve-type semidry electrode using deep residual shrinkage networks under real construction environment. IEEE Trans. Instrum. Meas.73, 1–14. doi: 10.1109/TIM.2024.3451583
49
WangT. F.XueX. L. (2009). Fatigue self-assessment scale (FSAS). China J. Traditional Chin. Med. Pharm.24, 348–349.
50
XuT.WangH. T.LuG. Y.WanF.DengM. Q.QiP.et al. (2023). E-Key: an EEG-based biometric authentication and driving fatigue detection system. IEEE Trans. Affect. Comput.14, 864–877. doi: 10.1109/Taffc.2021.3133443
51
YangL.ZhangR.-Y.LiL.XieX. (2021). “ SimAM: a simple, parameter-free attention module for convolutional neural networks”, in: Proceedings of the 38th International Conference on Machine Learning, vol. 139 ( PMLR), 11863–11874.
52
YaoD. Z. (2001). A method to standardize a reference of scalp EEG recordings to a point at infinity. Physiol. Meas.22, 693–711. doi: 10.1088/0967-3334/22/4/305
53
ZekiS.WatsonJ.LueckC.FristonK. J.KennardC.FrackowiakR. (1991). A direct demonstration of functional specialization in human visual cortex. J. Neurosci.11, 641–649. doi: 10.1523/JNEUROSCI.11-03-00641.1991
54
ZhangH.MengC.DiX.WuX.BiswalB. (2023). Static and dynamic functional connectome reveals reconfiguration profiles of whole-brain network across cognitive states. Network Neurosci.7, 1034–1050. doi: 10.1162/netn_a_00314
55
ZhengW. L.LuB. L. (2017). A multimodal approach to estimating vigilance using EEG and forehead EOG. J. Neural Eng.14, 026017. doi: 10.1088/1741-2552/aa5a98
Summary
Keywords
attention mechanism, deep learning, electroencephalography, fatigue classification, spatial-temporal features
Citation
Lu J, Liu S, Li W, Mu C, Gong D and Yao D (2026) A dual-branch multidimensional attention fusion network for EEG-based fatigue classification. Front. Physiol. 17:1923970. doi: 10.3389/fphys.2026.1923970
Received
02 July 2026
Revised
05 August 2026
Accepted
07 August 2026
Published
09 September 2026
Volume
17 - 2026
Edited by
Jinghua Zhang, Hohai University, China
Reviewed by
Xuenan Wang, Shandong First Medical University, China
Xingyan Chen, Dalian University of Technology, China
Updates
Copyright
© 2026 Lu, Liu, Li, Mu, Gong and Yao.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Diankun Gong, gongdiankun@uestc.edu.cn; Dezhong Yao, dyao@uestc.edu.cn
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.