To assure aviation safety: the pilot fatigue detection based on short-term multimodal physiological signals

Pilot fatigue detection based on physiological signals is practical for aviation safety. Current methods face challenges in balancing the high computational cost of deep learning models with robust accuracy, especially when integrating short-term multimodal physiological signals. To address these challenges, this paper proposes a framework for fast, accurate, and robust pilot fatigue detection by fusing features from electroencephalogram (EEG) and electrocardiogram (ECG) signals. The primary novelty of this work lies in a streamlined selection and classification strategy that overcomes the intrinsic limitations of Heart Rate Variability (HRV) analysis in short (2-s) segments while maintaining competitive accuracy at a drastically lower training cost. Specifically, by utilizing statistical ECG features, which are then integrated with EEG markers through a two-stage ANOVA-SVM feature selection process. The optimized, low-dimensional feature set is then classified using an XGBoost model. Evaluated on data from 32 pilots, the framework demonstrated robust generalization with an accuracy of 88.42% in rigorous cross-subject cross-validation, significantly outperforming our previous EEG-only ASFT-Transformer. While standard cross-clip validation yielded a higher accuracy of 98.36%, the cross-subject metric highlights the model's potential utility for unseen individuals. Crucially, the framework achieves this performance with an average training time of only 39.3 s, a drastic reduction compared to mainstream deep learning models. By striking a balance between accuracy, generalization, and efficiency, this study presents a promising and feasible approach for objective pilot fatigue management.

1 Introduction

A persistent challenge in aviation safety is pilot fatigue, which directly impairs an operator's reaction speed, information integration, and decision-making capabilities (Hamann and Carstengerdes, 2023). Although extensive research has identified the roots of fatigue in sleep deprivation, high operational workload, and circadian misalignment (Mannawaduge et al., 2024; Berberich and Leitner, 2017; Marcus and Rosekind, 2017; Wingelaar-Jagt et al., 2021), a fundamental problem remains: human physiology dictates that optimal functioning relies on adequate nightly sleep, meaning fatigue cannot be completely eradicated (International Civil Aviation Organization, 2020). In this context, establishing a scientific and effective pilot fatigue management system is of paramount importance for ensuring aviation safety.

The International Civil Aviation Organization (ICAO) has published the latest version of Doc 9966, offering a guiding framework for pilot fatigue management (International Civil Aviation Organization, 2020). This document's principles have been adopted by major aviation authorities (Federal Aviation Administration, 2013; European Union Aviation Safety Agency, 2012; Civil Aviation Administration of China, 2021). Specifically, operators are strongly advised to evaluate pilot states based on continuous monitoring and data analysis. For data collection, although self-reporting scales are often proposed (Maclean et al., 1992; Kaida et al., 2006), fatigue indicators derived from this self-reporting process are largely unreliable in practice. Pilots may struggle to accurately evaluate their own condition against multiple fatigue levels (Guo et al., 2025; Gläsener et al., 2023), and more seriously, some may conceal their fatigue for certain reasons (such as failing to meet the flight duration requirements), thereby endangering aviation safety (Jun-Ya and Rui-Shan, 2023; Kioulepoglou et al., 2024; Brezonakova, 2023). Consequently, Doc 9966 identifies the collection and analysis of physiological data as a superior approach for fatigue management.

Unlike subjective self-reports, physiological data, including Electroencephalogram (EEG) (van Weelden et al., 2022), Electrocardiogram (ECG) (Guo et al., 2025), Electromyogram (EMG) (Naim et al., 2022), and Electrooculogram (EOG) (Faust et al., 2018), provides objective metrics for assessing the functional state of the human body. Utilizing these modalities for fatigue detection in pilots and drivers is a well-established methodology. Research has demonstrated that EEG frequency bands (specifically δ, θ, α, and β) are strongly correlated with workload, fatigue levels, and other functional states (Eoh et al., 2005). Regarding cognitive load, increased task difficulty typically results in elevated θ power (especially in frontal regions) and suppressed α power across frontal, central, and parietal areas, whereas β power exhibits mixed responses (decreasing in parietal regions or generally increasing) (Chikhi et al., 2022). Conversely, the onset of mental fatigue triggers distinct spectral shifts: δ notably increases; θ and α power both rise in frontal and parietal regions (with α also increasing in occipital areas) (Borghini et al., 2014). Meanwhile, β power responses remain variable under fatigue, with studies reporting either overall power decreases or parietal increases (Hamann and Carstengerdes, 2022). For ECG, the Heart Rate Variability (HRV) component extracted from ECG is particularly notable, as it reflects autonomic nervous system activity, is highly correlated with fatigue (Wang et al., 2018; Chen et al., 2024; Wang F. et al., 2024; Gao et al., 2025) and has proven effective in tasks like detecting sleep states (Piotrowski and Szypulska, 2017) and classifying physical fatigue (Ni et al., 2022). Regarding EMG, muscle activity typically exhibits a decrease in signal amplitude and a spectral shift toward lower mean power frequencies (muscle fatigue) in postural muscles, reflecting the reduced muscle tone and motor coordination associated with drowsiness (Naim et al., 2022). Additionally, EOG indicators serve as reliable detectors of visual vigilance, where the onset of fatigue is typically marked by significantly increased blink duration and frequency, alongside slower saccadic movements due to reduced oculomotor control (Faust et al., 2018).

Notably, EEG is often favored for workload and fatigue evaluation (Blanco et al., 2018; Luo et al., 2019; Dehais et al., 2019; Wang Y. et al., 2024). Its prominence stems from its ability to directly reflect brain activity (Zhang et al., 2023), making it highly suitable for end-to-end detection of human functional states via artificial neural networks, which bypasses the need for prior knowledge (Huang et al., 2022). Consequently, several deep learning architectures are widely applied to EEG data. Convolutional Neural Networks (CNNs), for instance, are commonly used for cognitive workload assessment (Jiao et al., 2018), emotion recognition (Shen et al., 2023), and fatigue detection (Gao et al., 2019). In parallel, Recurrent Neural Networks (RNNs), notably the Long Short-Term Memory (LSTM) (Tang et al., 2021; Cao et al., 2024) and Bi-LSTM (Mehmood et al., 2023), are increasingly adopted for processing its temporal sequences. Furthermore, research aiming to enhance spatio-temporal feature extraction has yielded favorable results from hybrid structures, including the CNN-LSTM (Pan et al., 2023), LGNet (Wang Y. et al., 2024), and MATCNT (Jia et al., 2023).

Despite its promise, end-to-end deep learning for fatigue detection is computationally intensive due to the massive volume of raw EEG data (Zheng and Lu, 2017), carrying risks of prolonged training times and technical challenges like gradient instability and overfitting. To mitigate these issues, researchers often employ feature extraction to lessen data complexity and boost efficiency (Zhang et al., 2023). This has led to successful models using entropy features (Luo et al., 2019; Gao et al., 2024a,b) as well as various time-domain, frequency-domain, and brain-network features (Dehais et al., 2019; Zhang et al., 2023; Liu et al., 2021; Wang et al., 2025; Jebelli et al., 2018; Zhou et al., 2023; Wang et al., 2021; Fu et al., 2020). However, this introduces the curse of dimensionality. While feature selection is the standard solution, existing strategies often fail to strike a balance between computational speed and physiological validity. Wrapper methods, such as those based on recursive feature elimination or genetic algorithms, incur prohibitive computational costs due to iterative model retraining, making them unsuitable for rapid, deployment-ready screening (Wang et al., 2021). Conversely, simple filter methods are computationally efficient but often overlook complex dependencies between multimodal features, potentially discarding variables that are weak individually but predictive in combination (Liu et al., 2021). Furthermore, while emerging deep learning approaches perform implicit feature extraction, they typically operate as “black boxes” (Faust et al., 2018). They lack the physiological interpretability required to explicitly identify which specific channels and features drive the fatigue classification (Zhang et al., 2023). Consequently, a critical gap remains: how to efficiently identify a feature subset that is not only statistically discriminative but also biologically meaningful.

To bridge this gap, feature selection must be grounded in neuroergonomic principles rather than relying solely on data-driven metrics. Physiologically, pilot fatigue is characterized by distinct spectral and spatial alterations, specifically a global increase in slow-wave activity (δ and θ bands) and elevated Alpha power in occipital and parietal regions, reflecting diminished visual attention (Jing et al., 2020; Borghini et al., 2014). Unlike prior approaches that either obscure these mechanisms within opaque deep neural networks (Gao et al., 2019) or rely on rigid, pre-defined manual selection, our framework employs a transparent, data-driven strategy. By systematically screening features through the ANOVA-SVM pipeline, we objectively isolate the most fatigue-sensitive markers, specifically confirming the predominance of occipital channels without prior assumptions (as detailed in Section 3.2). This approach not only aligns engineering design decisions with established neurophysiological mechanisms but also ensures the model's robustness and generalization across different subjects.

Concurrently, a growing trend involves constructing multimodal features by integrating ECG with EEG (Mu et al., 2024; Gao et al., 2025), leveraging the ECG's own utility as a viable signal for assessing mental and physical states (Guo et al., 2025; Chen et al., 2024; Wang F. et al., 2024). While both modalities are informative individually, the most significant advantages emerge from their fusion. Research has consistently demonstrated that combining EEG and ECG signals enhances overall detection accuracy and robustness. For instance, (Awais et al. 2017) demonstrated improved accuracy using an SVM-based fusion of the two signals. Similarly, (Mu et al. 2024) achieved 94% accuracy by designing a novel gating method to adaptively fuse ECG and HRV features. (Du et al. 2023) further confirmed this, using a Product Fuzzy Convolutional Network (PFCN) with a dedicated subnetwork for fusion, which resulted in superior robustness and accuracy. These findings collectively validate that a multimodal EEG-ECG approach can overcome the limitations of single-signal analysis.

Although significant progress has been made, several key challenges persist for practical application in pilot fatigue management:

(1) Real-time fatigue monitoring of pilots based on physiological signals is difficult. Extended wear of standard EEG and ECG sensors may introduce physical discomfort or cognitive interference, potentially distracting pilots from critical flight tasks. Furthermore, the logistical complexity of equipping cockpits with real-time signal processing and transmission hardware presents a substantial barrier to scalable, daily deployment, effectively limiting the practical utility of such real-time systems.

(2) Short-term ECG signals have limitations in fatigue detection. In the context of (1), a practical and objective pilot fatigue management solution is to use physiological signals to determine whether pilots are already fatigued before flight, which means that the acquisition time of physiological signal data for each pilot will be short. However, a too short signal may not accurately capture complete physiological information, thus affecting the reliability of fatigue assessment. Crucially, traditional frequency-domain HRV analysis, which requires at least 5 min of data to reliably reflect autonomic nervous system (ANS) regulation, is inapplicable to short segments (Chen et al., 2024; Sun et al., 2024).

(3) It is challenging for fatigue detection based on bimodal EEG-ECG to satisfy the requirements of small input samples, fast model training and high recognition accuracy. Although many studies have been able to achieve more accurate fatigue detection based on EEG signals of a few seconds (Zheng and Lu, 2017; Gao et al., 2024a,b; Wang et al., 2025; Li et al., 2024), effectively fusing these with short-term ECG signals without inflating computational cost or overfitting remains difficult. Furthermore, the key to achieve fatigue detection with short model training time and high recognition accuracy lies in the co-optimization of both the feature selection and the classification model itself.

To avoid the practical limitations of in-flight monitoring, this study proposes a strategic framework tailored for pre-flight fatigue detection, offering a feasible alternative for daily safety management. Building upon our previous exploration of EEG-based pre-flight fatigue detection (Liu et al., 2025), this study optimizes the framework by incorporating short-term ECG features. Although the standard time period for HRV analysis is usually 5 min, research indicates that fatigue-induced shifts in sympathetic and parasympathetic balance manifest not only in heart rate intervals but also in the morphological complexity and statistical distribution of the raw ECG signal (Butkevičiūtė et al., 2022; Mu et al., 2024). Statistical features, such as variance, skewness, and kurtosis, quantify the signal's deviation from a normal distribution and its transient instability. These statistics serve as robust physiological proxies for the immediate autonomic perturbations and non-linear complexity changes associated with mental fatigue, even within restricted time windows (Guo et al., 2025). Therefore, this study integrates ECG statistical features with EEG data to train a computationally efficient XGBoost classifier. This approach offers a dual advantage: it enhances the accuracy and robustness of fatigue detection while significantly reducing model training time.

2 Methodology

2.1 Analytical framework overview

The analytical framework for pilot fatigue detection is depicted in Figure 1. This process is structured into three sequential stages: (1) signal preprocessing and segmentation, (2) pivotal feature and EEG channel selection, and (3) final fatigue classification. Each component is detailed in the subsequent sections.

Figure 1

Figure 1. Schematic overview of the proposed pilot fatigue detection framework. The pipeline consists of three sequential stages: (1) Signal preprocessing and segmentation of EEG/ECG data, (2) Pivotal feature selection using ANOVA and EEG channel selection using SVM-based AUC ranking, and (3) Final fatigue classification using the XGBoost model. With permission from Liu et al. (2025).

2.2 Experimental dataset and procedure

2.2.1 Participant characteristics

To rigorously mitigate potential confounds and ensure that the detected physiological shifts were primarily driven by task-induced fatigue rather than sleep deprivation or circadian misalignment, strict inclusion criteria were enforced. Specifically, all participants were required to: (1) maintain at least 8 h of nightly sleep for the two days preceding the experiment; (2) abstain from alcohol, caffeine, and any drowsiness-inducing medications for 48 h prior to the study; and (3) verify the absence of any neurological or psychiatric history. Regarding the timing of the experiment, simulation sessions were scheduled between 06:00 and 08:00 (local time) contingent on the airline's operational roster. While precise start times varied slightly due to scheduling logistics, strictly maintaining this consistent morning window minimized the impact of circadian phase differences across subjects. Following a complete briefing of the experimental procedure, written informed consent was obtained from each pilot. The study protocol adhered to the principles of the Declaration of Helsinki and received ethical approval from the Civil Aviation Shanghai Hospital, China.

2.2.2 Fatigue induction procedure

To procure physiological data representing alert and fatigue states, we collected EEG and ECG recordings from each pilot immediately before (pre-training) and after (post-training) their standard regular simulator training. This training, which lasted approximately 6 h, was executed in a Boeing 787 Level D full-flight simulator at the airline's Flight Training Center (Figure 2). The training procedure, grounded in Evidence-Based Training (EBT) principles (International Civil Aviation Organization, 2013), lasted a total of about 6 h. Each training session paired one captain with one first officer under the supervision of a flight instructor. The procedure was divided into four main components:

Figure 2

Figure 2. Illustration of the Boeing 787 simulator. This high-fidelity simulator was used to conduct the 6-h flight training mission. The realistic operational environment ensures that the induced fatigue reflects actual flight conditions rather than simple laboratory drowsiness.

(1) Pre-flight Briefing (1 h): The instructor outlined the training scenarios, maneuver checks, and evaluation criteria.

(2) Scenario-Based Training (SBT) (2 h): The crew managed 1–2 complete flights involving various anticipated and unanticipated challenges, such as engine failures or adverse weather conditions.

(3) Maneuver-Based Checks (MBC) (2 h): The crew performed specific maneuvers, including various subjects, such as take-off with crosswind, non-precision approach and landing with one engine inoperative.

(4) Post-flight Debriefing (1 h): The instructor provided objective feedback on crew performance to identify areas for improvement.

Figure 3 illustrates this entire training procedure. Physiological data were recorded continuously, and the labeling strategy was defined as follows: the data collected during the first 10 min of the session was labeled as the Alert State, while the data from the last 10 min was labeled as the Fatigue State. This labeling strategy was grounded in the physiological effects of prolonged cognitive exertion to avoid the potential variability and subjectivity inherent in self-reported ratings. Previous aviation physiology research suggests that continuous performance of demanding simulated flight tasks typically induces significant mental fatigue and EEG slow-wave activity after 2–4 h (Guo et al., 2025; Wang et al., 2025; Borghini et al., 2014; Caldwell et al., 2009; Mangin et al., 2022). In this study, the 6-h duration provided a substantial operational window beyond these reported thresholds. Consequently, we adopted the assumption that participants had transitioned into a fatigued state by the end of the session.

Figure 3

Figure 3. Schematic of the fatigue induction procedure. The 6-h procedure follows EBT, principles, including Pre-flight Briefing, SBT, MBC and Post-flight Debriefing. Physiological data are collected before (pre-training) and after (Post-training) the procedure to capture alert and fatigue states, respectively.

2.2.3 Physiological signal acquisition

Pilot EEG signals were collected using a 16-channel Delica AEEG-3202 electroencephalograph (Shenzhen Delica Medical Equipment Co., Ltd., China). This system, comprising a compact control box and 16 standard Ag/AgCl wet disc electrodes (Figure 4a), provided high-quality data at a 500 Hz sampling rate. The 16 electrodes were positioned on the scalp according to the standard international convention, as shown in Figure 4b (FP: Frontopolar, F: Frontal, C: Central, P: Parietal, T: Temporal, O: Occipital; odd/even numbers denote left/right hemispheres).

Figure 4

Figure 4. Illustration of the EEG device. (a) The 16-channel Delica AEEG-3202 device with standard Ag/AgCl wet electrodes. (b) The electrode distribution on the scalp follows the international 10–20 system, covering Frontal (F), Central (C), Parietal (P), Temporal (T), and Occipital (O) regions. Image adapted with permission from Liu et al. (2025).

Pilot ECG signals were collected using a single-channel Mi-Rhythm Holter ECG recorder (Shenzhen Ruikang Hongye Technology Development Co., Ltd., China) at a sampling rate of 250 Hz. The ECG device consists of a Holter recorder, silicone sleeve, and disposable electrode patches, as shown in Figure 5.

Figure 5

Figure 5. Illustration of the ECG device. The single-channel Mi-Rhythm Holter recorder includes (a) the host unit, (b) a protective silicone sleeve, and (c) disposable electrode patches. This compact design allows for unobtrusive recording of cardiac activity.

To minimize participant discomfort and facilitate rapid detection, synchronous EEG and ECG data were recorded for approximately 12 min during both the pre-training and post-training sessions (Figure 6). The data collection was conducted in a quiet, controlled environment where participants were seated comfortably. Crucially, to mitigate behavioral confounds and simulate the visual vigilance required in operational settings, all pilots were instructed to maintain an eyes-open resting state to ensure they remained awake and strictly adhered to the protocol, thereby ensuring the signals reflected mental fatigue rather than drowsiness-induced sleep onset.

Figure 6

Figure 6. Schematic diagram of collecting physiological signals for pilots. Data collection was conducted in a quiet, controlled room while pilots remained in a seated, eyes-open resting state. This setup minimizes movement artifacts and ensures the signals reflect mental fatigue rather than sleep onset. Image adapted from Figure 4, Liu et al. (2025).

For a balanced and standardized analysis, only the initial 10 min of each recording were retained for processing. Given that the EEG and ECG signals were acquired via independent systems, hardware-based synchronization was not employed. Instead, temporal alignment was strictly performed post-acquisition. Specifically, the data streams from both modalities were synchronized by manually aligning their absolute recording timestamps. This procedure ensured that the retained 10-min segments for both EEG and ECG corresponded to the exact same temporal window for each subject.

2.3 Signal preprocessing and segmentation

To evaluate the robustness of the proposed framework in realistic, resource-constrained aviation environments, the raw EEG and ECG recordings underwent a streamlined preprocessing pipeline designed for low latency. We deliberately avoid computationally intensive reconstruction techniques (e.g., Independent Component Analysis or channel interpolation) to preserve the original signal manifold and ensure the system's viability for real-time deployment. Crucially, regarding artifact handling, we explicitly chose not to remove ocular activities (e.g., blinks and saccades). In the specific context of fatigue detection, these physiological patterns serve as valuable biomarkers rather than mere noise, as extensive research has linked fatigue onset to distinct ocular signatures such as increased blink duration (Naim et al., 2022; Faust et al., 2018; Zheng and Lu, 2017). By retaining these signals, we enable the classifier to leverage inherent fatigue characteristics as informative features. This strategy not only preserves critical physiological information but also avoids the computational overhead of artifact removal algorithms, aligning with our objective of efficient pre-flight screening.

A 0.5–30 Hz bandpass filter is firstly applied to the raw EEG signals to attenuate low-frequency drifts (EOG < 0.5 Hz) and high-frequency muscular artifacts. Following filtration, the Fast Fourier Transform (FFT) is employed to decompose the signals into the four target frequency bands (δ, θ, α, and β). For subsequent feature extraction, to continuously capture the dynamic physiological changes associated with fatigue and maximize the utility of the dataset, a sliding window technique is employed for signal segmentation. Specifically, the EEG signals are segmented into 2-s epochs with a 50% overlap (step size of 1 s). This overlapping strategy is widely adopted in physiological computing to prevent the loss of transient information at window boundaries and to serve as an effective data augmentation technique for enhancing the robustness of machine learning models (Zhang et al., 2023; Mouazen et al., 2025; Falih et al., 2024).

The preprocessing of the ECG signals is relatively streamlined. Raw ECG signals are processed with the identical 0.5–30 Hz bandpass filter and then segmented by using the same 2-s windowing approach (with 50% overlap) as the EEG data. This synchronized segmentation is crucial, establishing each 2-s epoch as a single, combined-modality sample for the subsequent fatigue state classification, as illustrated in Figure 7.

Figure 7

Figure 7. Visualization of the EEG and ECG signal preprocessing and segmentation. Both EEG and ECG signals are filtered (0.5–30 Hz) and segmented into 2-s epochs with a 50% overlap using a sliding window technique. This synchronized segmentation ensures that each data sample contains aligned multimodal information.

To justify the segmentation strategy adopted in this framework, we consider the specific operational requirements of fast pre-flight fatigue detection. Recent studies indicate that EEG signals exhibit quasi-stationarity within short intervals (typically 1–4 s), making short-window analysis a robust approach for fatigue detection (Zhang et al., 2023; Mouazen et al., 2025). Consistent with this finding, our previous work also demonstrated that fatigue detection based on 2-s EEG segments can achieve satisfactory performance (Liu et al., 2025). In contrast, standard ECG analysis, particularly frequency-domain Heart Rate Variability (HRV), typically necessitates longer recordings (e.g., 5 min) to ensure metric stability (Chen et al., 2024; Sun et al., 2024). Therefore, rather than relying on traditional HRV measures which are unstable in short windows, our strategy introduces ECG specifically as an auxiliary modality. We focus on extracting statistical morphological features from these 2-s aligned ECG segments to complement the EEG data. This multimodal fusion strategy aims to leverage the complementary information of ECG to enhance detection accuracy, a hypothesis that is subsequently validated in Section 3.

2.4 Physiological feature extraction

Following data preprocessing, a comprehensive set of multidimensional features is extracted from the segmented EEG and ECG epochs to capture the physiological manifestations of fatigue.

For EEG signals, both time-domain and frequency-domain analyses are utilized to generate 8 distinct features for each 2-s EEG frequency band segment. In the time domain, the mean (MEA) $\bar{X}$, energy (ENE) X_e, variance (VAR) X_var and root mean square (RMS) X_rms are calculated to quantify signal amplitude and fluctuation intensity. In the frequency domain, based on the Fast Fourier Transform (FFT), the power spectrum density (PSD) F_psd, centroid frequency (CF) F_cf, frequency variance (FV) F_fv and mean square frequency (MSF) F_msf are extracted to characterize the spectral energy distribution. Previous studies have already demonstrated the effectiveness of these features (Luo et al., 2019; Wang Y. et al., 2024; Zhou et al., 2023; Wang et al., 2021).

For ECG signals, given the limitations of frequency-domain HRV analysis in short segments, seven statistical features are derived from the raw waveform to quantify autonomic instability. These features include the mean μ, standard deviation σ, variance σ², maximum Max, minimum Min, skewness γ, and kurtosis κ. As detailed in the Introduction, these statistical moments are considered robust proxies for the morphological complexity and non-Gaussian deviations induced by fatigue-related autonomic dysregulation.

Since the mathematical formulations for these standard features are well-documented in prior physiological computing literature, detailed equations are omitted here for brevity. Interested readers are referred to (Guo et al. 2025), (Luo et al. 2019), Wang Y. et al. (2024), (Zhang et al. 2023), (Zhou et al. 2023), (Wang et al. 2021), and (Butkevičiūtė et al. 2022) for the detailed derivations.

Finally, to mitigate the significant inter-subject variability inherent in physiological signals, a subject-wise normalization strategy was implemented. Specifically, all extracted EEG and ECG features underwent Z-score standardization (z = (x−u)/σ) individually for each pilot. This process ensures that the subsequent classification focuses on relative fatigue-induced shifts rather than absolute baseline differences between subjects.

2.5 Pivotal feature and EEG channel selection

Extracting multi-dimensional features from EEG and ECG signals enables a model to capture more comprehensive information about underlying neurophysiological states than the raw signals themselves (Cai et al., 2020). This transformation, however, converts the data from a time-series representation to a high-dimensional feature space, which often introduces significant redundancy. This redundancy is especially problematic in EEG, given the high spatial correlation between adjacent channels. To address this, we implemented a combined strategy for both feature selection and EEG channel selection. This dual approach is crucial for mitigating data redundancy, improving computational efficiency, and minimizing the risk of model overfitting.

2.5.1 Feature selection

A one-way analysis of variance (ANOVA) is employed as the feature selection strategy, as it directly tests and quantifies mean differences between groups with high interpretability. Moreover, its efficacy for physiological data analysis has been previously validated (Zhang et al., 2023). Consequently, using ANOVA, we systematically identify features capable of discriminating between the alert and fatigue states at a high level of statistical significance (p < 0.01). This selection process is crucial, as irrelevant or redundant features can severely degrade a model's generalization performance (Cai et al., 2020). Notably, to effectively capture the distinct physiological signatures of brain and cardiac activities while avoiding the interference of high-dimensional redundancy, an independent feature selection strategy is implemented for each modality.

For EEG, given its multi-channel nature, the selection is performed in a channel-frequency manner. The ANOVA is applied individually to each feature extracted from the 4 frequency bands across all 16 channels. This process is utilized to pinpoint specific channel-frequency pairs that are statistically sensitive to fatigue. Conversely, for the single-channel ECG, the ANOVA is directly applied to its statistical features. This rigorous independent selection strategy prevents the high-dimensional EEG data from dominating the feature selection process, ensuring that critical ECG indicators are preserved for effective feature fusion.

2.5.2 EEG channel selection

The primary goal of EEG channel selection is to pinpoint the most informative channels for fatigue detection. An effective selection strategy can significantly reduce model complexity and computational load while also simplifying the practical setup for data acquisition. In this study, we have implemented a method to rank each channel's relevance to fatigue. Specifically, a distinct Support Vector Machine (SVM) classifier is trained for each individual EEG channel, using only its extracted features.The resulting Area Under the Receiver Operating Characteristic Curve (AUC) score for each single-channel classifier served as a quantitative measure of that channel's individual contribution to discriminating fatigue. This methodology is well-supported: SVM is a proven classifier for EEG-based fatigue detection (Wang Y. et al., 2024; Fu et al., 2020; Wang et al., 2020; Subasi et al., 2022), and AUC is a robust and standard metric for binary classification (Lin et al., 2014; Tang et al., 2020). The procedure is detailed as:

(1) Training separate SVM model

The channel selection procedure involves training 16 distinct SVM classifiers, one for each channel. Each classifier is tasked with discriminating between the alert and fatigue states, utilizing the complete feature set (from all 4 frequency bands) associated with that single channel. As a supervised learning algorithm, SVM operates by constructing an optimal separating hyperplane. This boundary is determined by maximizing the margin to the nearest data points (the support vectors) and is mathematically defined as:

$\begin{array}{ll}\text{w}·\text{x}+b=0& (1)\end{array}$

where w is the weight vector normal to the hyperplane, x represents the input feature vector, and b denotes the bias term.

Since perfect linear separability is not expected in the feature data, slack variables (ξ_i) are introduced to permit some degree of misclassification. This transforms the task into a soft-margin SVM optimization. The optimal parameters (w and b) are then found by solving the convex quadratic programming problem:

$\begin{array}{ll}{\displaystyle \underset{\text{w},b,\xi }{min}}\frac{1}{2}{||\text{w}||}^2+C{\displaystyle \sum _{i=1}^N}{\xi }_i& (2)\end{array}$

subject to the constraints:

$\begin{array}{ll}{y}_i(\text{w}·{\text{x}}_i+b)\ge 1-{\xi }_i,{\xi }_i\ge 0& (3)\end{array}$

where N denotes the total number of feature vectors, x_i is the i^th feature vector, and y_i∈{−1, 1} represents its corresponding state label. Additionally, the hyperparameter C is a regularization term that governs the trade-off between margin maximization and error minimization, while ξ_i represent the slack variables that allow for a degree of misclassification.

Once trained, the classification is determined by the decision function f(x) = w·x+b. The SVM classifies a sample x_i based on the sign of f(x) (e.g., positive for alert, negative for fatigue), while the magnitude of this value indicates the confidence of the classification.

(2) Selecting valuable channels based on AUC

Following the training of each single-channel SVM, its performance is quantified using the AUC value. In this context, the AUC score acts as a direct measure of a channel's discriminative power. A higher AUC signifies that a channel provides more effective information for separating the alert and fatigue states. Ranging from 0 to 1, the AUC score represents the probability that the classifier will rank a randomly selected positive (alert) sample higher than a randomly selected negative (fatigue) sample. The calculation is performed as follows (Zhang et al., 2023):

$\begin{array}{ll}AUC=\frac{{\displaystyle \sum _{i\in PositiveClass}}ran{k}_i-\frac{T(1+F)}{2}}{T\times F}& (4)\end{array}$

where T and F denote the total counts of positive and negative samples, respectively, and rank_i signifies the ranking number assigned to the i^th positive sample features.

All EEG channels were subsequently ranked based on their respective AUC scores. Channels yielding high AUC values were identified as more discriminative for fatigue and thus selected for the final classification model. Conversely, those with low scores indicating minimal contribution were discarded. This systematic selection process ensures that the subsequent fatigue recognition model is built using only the most informative channels, thereby reducing data redundancy, lowering the computational burden, and enhancing the potential for robust classification performance.

2.6 Fatigue detection based on multimodal physiological features

For the final detection of pilot fatigue, we employed XGBoost, an optimized ensemble algorithm implemented under the Gradient Boosting framework (Chen and Guestrin, 2016). XGBoost iteratively constructs a series of decision trees using an additive strategy, merging them into a single, robust predictive model. Its state-of-the-art performance on tabular data makes it exceptionally well-suited for our complex classification task, which involves a fused set of features from multiple physiological signals.

Given a multimodal feature vector x_i (comprising the selected EEG and ECG features), the model's final prediction ŷ_i is an additive sum of the predictions from all K trees:

$\begin{array}{ll}{ŷ}_i=\varphi (x_i)={\displaystyle \sum _{k=1}^K}{f}_k(x_i),f_k\in F& (5)\end{array}$

where f_k represents the k^th decision tree, which maps an input feature vector x_i to a leaf weight, and $\mathcal{F}$ denotes the function space containing possible decision trees.

This set of trees is learned by minimizing a regularized objective function:

$\begin{array}{ll}L(\varphi )={\displaystyle \sum _{i=1}^n}l(y_i,{ŷ}_i)+{\displaystyle \sum _{k=1}^K}\text{Ω}(f_k)& (6)\end{array}$

This objective combines a loss function, l(y_i, ŷ_i), which quantifies prediction error, with a regularization term, Ω(f_k), which penalizes model complexity to prevent overfitting.

The regularization term is defined as:

$\begin{array}{ll}\text{Ω}(f)=\gamma T+\frac{1}{2}\text{λ}{\displaystyle \sum _{j=1}^T}{w}_j^2& (7)\end{array}$

where T denotes the number of leaves in the decision tree, w_j is the weight of the j^th leaf, and the hyperparameters γ and λ control the penalty on the number of leaves and the strength of the L2 regularization on the leaf weights, respectively.

Training is performed additively, where at each iteration t, a new tree f_t is added to fit the residuals from the (t−1)^th iteration:

$\begin{array}{ll}{ŷ}_i^{(t)}={ŷ}_i^{(t-1)}+f_t(x_i)& (8)\end{array}$

XGBoost optimizes this objective function efficiently using a second-order Taylor expansion. Leveraging its sophisticated optimization strategy, XGBoost can effectively learn the complex, non-linear relationships within these fused features, thereby achieving more robust and accurate fatigue recognition than would be possible with a single signal source. Our previous research has also shown the XGBoost's superiority in analyzing pilot physiological signals (Liu et al., 2025).

3 Results

3.1 Identification of statistically significant features

As described in Section 2.5.1, a one-way ANOVA was conducted on the initial set of eight EEG features and seven ECG features (described in Section 2.4) to isolate features significantly relevant to fatigue.

As shown in Figure 8, the ANOVA results for EEG features revealed varied correlations with fatigue. The majority of frequency-domain metrics (specifically the PSD in θ and β bands) and time-domain features reflecting signal fluctuation intensity (e.g. the ENE, VAR, RMS) exhibited strong statistical significance (p < 0.01). These findings are consistent with recent studies, which suggest that these indicators effectively capture the rhythmic neural synchronization associated with fatigue (Zhang et al., 2023; Wang et al., 2025). However, the MEA did not demonstrate statistically significant differences (p > 0.05) across most channels. This observation aligns with previous research (Zhang et al., 2023), indicating that the raw mean of EEG signals primarily reflects baseline drifts rather than fatigue-related cortical processing, and thus lacks a robust statistical correlation with fatigue state changes. The FV feature was also deemed unreliable, showing significance across fewer than 10 channels in each frequency band and being removed as well.

Figure 8

Figure 8. Diagram of ANOVA statistical results of features under alert and fatigue states. The heatmaps display the statistical significance of 8 extracted features across 16 EEG channels for 4 frequency bands: (a) δ, (b) θ, (c) α, (d) β. Red blocks indicate significant features (p < 0.01) retained for the model, while Blue blocks indicate non-significant features (p ≥ 0.01) that are discarded to reduce redundancy.

For the ECG features, the analysis mainly indicated the superiority of higher-order statistical features over simple central tendency metrics (Table 1). Features characterizing the waveform distribution, such as the σ, σ² and γ showed significant differences (p < 0.01). These metrics effectively capture the cardiac instability and autonomic dysregulation associated with mental fatigue. In contrast, the simple μ showed reduced sensitivity (p > 0.05), likely due to its susceptibility to individual baseline variations and low-frequency wandering (Guo et al., 2025; Borghini et al., 2014). Overall, only these 4 EEG features were retained: σ, σ², Max and γ.

Table 1

Table 1. ANOVA statistical results for ECG features.

These findings demonstrate that not all extracted features provide valid information for fatigue detection. It is necessary to eliminate these non-significant features and retain only the effective physiological indicators.

3.2 Evaluation of EEG channel discriminability

As described in Section 2.5.2, AUC was used as the primary metric to quantitatively evaluate each EEG channel's ability to distinguish between alert and fatigue states. The evaluation involved training 16 distinct SVM models, one for each channel, using the 6 salient EEG features identified in Section 3.1. A 10-fold cross-validation strategy was employed to ensure robust performance evaluation.

The AUC scores were normalized and plotted on brain topographic maps for each frequency band to visualize correlation between each channel and pilot fatigue, as shown in Figure 9. Notably, channels O1 and O2 (occipital region) consistently yielded the highest AUC values across all frequency bands. This finding is physiologically grounded in the nature of pilots, which places heavy demands on the primary visual cortex for instrument scanning and environmental monitoring. Fatigue-induced decline in visual vigilance typically manifests as distinct rhythmic alterations, particularly in alpha activity, within these occipital regions (Jing et al., 2020; Borghini et al., 2014).

Figure 9

Figure 9. Topographic maps of average AUC values for each EEG channel across 4 frequency bands: (a) δ, (b) θ, (c) α, (d) β. The color scale represents the normalized AUC value, ranging from 0 (blue) to 1 (red). Data are obtained based on 10-fold cross-clip cross-validation. With permission from Liu et al. (2025).

Complementing the occipital findings, channels in the frontal (FP1, FP2) and central (C3, C4) regions also demonstrated significant discriminative power. This aligns with the hypothesis that mental fatigue impairs executive functions and attention allocation centered in the prefrontal cortex (Dehais et al., 2019). The significance of the central channels likely reflects the degradation of sensorimotor control required for manipulating the flight stick during prolonged tasks (Jiao et al., 2018). Conversely, channels in the parietal and outer temporal regions consistently demonstrated lower AUC values in this specific experimental setup.

A quantitative summary (Table 2) confirmed these visual findings. A consistent set of eight channels (O1, O2, T3, T4, C3, C4, FP1, and FP2) ranked highest for average AUC, regardless of the frequency band. Since the remaining 8 channels failed to achieve an average AUC above 0.5, this high-performing 8-channel set was selected as the optimal EEG subset for the final model. It is important to note that the maximum mean AUC for a single channel did not exceed 0.65. This apparently low value was an expected consequence of the evaluation method, as each SVM was trained on a minimal feature set (six features from only one frequency band). We hypothesized that performance would improve given more information and this hypothesis was tested by training each channel's SVM model using the EEG features from 4 frequency bands.

Table 2

Table 2. AUC values for single-channel SVM classifiers.

As shown in Figure 10, the results confirmed this hypothesis: for nearly all 16 channels, the AUC achieved using EEG features from 4 frequency bands was significantly higher than that from any single band. The efficacy of this channel selection process is further validated in Section 3.3.3, where the final model's accuracy demonstrates significant improvement.

Figure 10

Figure 10. Comparison of single-channel SVM classification performance (AUC) utilizing features from single vs. fused frequency bands. The bar chart compares the AUC of single-channel SVM classifiers using features from individual frequency bands versus fused features from all 4 bands (red bars). Data are obtained based on 10-fold cross-clip cross-validation. Adapted with permission from Liu et al. (2025).

3.3 Performance evaluation

3.3.1 Feature dataset construction

Following the preprocessing (Section 2.3), feature extraction, and channel selection, we constructed the final feature vector for each 2-s epoch. This vector represents a single, independent sample for the classification model. Each vector has a total length of 196 features, composed of the selected EEG and ECG features (4 EEG frequency bands × 6 features × 8 channels + 4 ECG features). This segmentation and feature construction process yielded a final dataset comprising 38,336 samples (32 pilots × 599 epochs × 2 states).

3.3.2 Experimental setup

All experiments were performed in Python 3.10 with PyTorch 2.8.0. The workstation was equipped with an Intel i9-10900X CPU, dual NVIDIA A500 GPUs, and 64 GB of RAM. Hyperparameters for the proposed XGBoost model were set to 100 estimators, a maximum tree depth of 6, and a learning rate of 0.1. For the comparative deep learning models, parameters were set consistent with previous work (Liu et al., 2025): a learning rate of 0.001, a batch size of 64, and 100 training epochs.

All results were evaluated using four standard classification metrics: Accuracy, Precision, Recall, and F1-Score, calculated as follows:

$\begin{array}{ll}Accuracy=\frac{TP+TN}{TP+FP+FN+TN}& (9)\end{array}$

$\begin{array}{ll}Precision=\frac{TP}{TP+FP}& (10)\end{array}$

$\begin{array}{ll}Recall=\frac{TP}{TP+FN}& (11)\end{array}$

$\begin{array}{ll}F1\text{\_}score=\frac{2\times Precision\times Recall}{Precision+Recall}& (12)\end{array}$

where TP (True Positive) is the number of positive samples correctly predicted as positive, and TN (True Negative) is the number of negative samples correctly predicted as negative. Conversely, FP (False Positive) represents the number of negative samples incorrectly predicted as positive, and FN (False Negative) represents the number of positive samples incorrectly predicted as negative.

3.3.3 Detection performances

Model performance was assessed by using two 4-fold cross-validation schemes. The first was a standard cross-clip cross-validation, where all 38,336 samples were shuffled and partitioned into four folds. The second, more stringent scheme was a cross-subject cross-validation, designed to evaluate generalization to unseen individuals, where the samples were divided into four pilot-disjoint folds. To ensure a robust evaluation, this entire cross-subject process was repeated five times with different random grouping.

A systematic comparative experiment was first designed to evaluate the effectiveness of the proposed feature and EEG channel selection method, and verify the complementary role of short-term ECG signals in fatigue detection. Specifically, the XGBoost model's inputs were organized into 6 distinct conditions (Tables 3, 4):

Table 3

Table 3. Ablation study results for XGBoost under different feature input conditions based on 4-fold cross-clip cross-validation.

Table 4

Table 4. Ablation study results for XGBoost under different feature input conditions based on 4-fold cross-subject cross-validation.

(1) EEG Features (no selection): The complete set of EEG features extracted from all channels was utilized as the raw baseline, without the application of any dimensionality reduction techniques.

(2) EEG Features with feature selection only: The proposed feature selection algorithm was applied to the raw EEG dataset. This scenario is established to facilitate a comparison with Condition (1) to evaluate the impact of mitigating feature redundancy.

(3) EEG Features with both feature and channel selection: Both feature selection and the proposed channel selection strategy were implemented. This configuration is intended to isolate the specific contribution of channel optimization by comparing the model performance against Condition (2).

(4) ECG Features (no selection): All extracted ECG features were employed to establish a raw baseline for cardiac physiological activity.

(5) ECG Features with feature selection: The proposed feature selection algorithm was applied to the raw ECG dataset. This scenario is established to facilitate a comparison with Condition (4) to evaluate the effectiveness of the feature selection strategy in mitigating redundancy within the ECG signals.

(6) EEG Features with feature and channel selection, and ECG Features with feature selection: The optimized EEG subset (from Condition 3) was integrated with the optimized ECG subset (from Condition 5). This final scenario is configured to examine the synergistic effects of multimodal fusion and to determine whether the inclusion of ECG signals yields performance improvements relative to the optimal unimodal EEG baseline (Condition 3).

The results clearly demonstrate the efficacy of our proposed optimization methods. For EEG-only data, the fully optimized model (Condition 3) achieved the highest accuracy, while the unoptimized model (Condition 1) performed the worst, confirming our selection process enhances performance. A similar benefit was seen for ECG-only data (Condition 5 vs. 4). Most importantly, the model using the final fused feature set (Condition 6) achieved the highest overall performance, with average accuracies of 98.36% (cross-clip cross-validation) and 88.42% (cross-subject cross-validation). This result validates the effectiveness of our multimodal fusion and the entire proposed framework.

To benchmark this performance, we compared our optimized XGBoost model against several established classifiers for tabular data. This included deep learning models (FT-Transformer, ResNet, MLP), classic machine learning (SVM, LR, KNN), and our previous EEG-only framework [ASFT-Transformer (Liu et al., 2025)]. The FT-Transformer is widely regarded as a state-of-the-art model for tabular data (Gorishniy et al., 2021; Vaswani et al., 2017). Comparative results are shown in Table 5 (cross-clip cross-validation) and Table 6 (cross-subject cross-validation).

Table 5

Table 5. Performance comparison of classification models based on 4-fold cross-clip cross-validation.

Table 6

Table 6. Performance comparison of classification models based on 4-fold cross-subject cross-validation.

In the cross-clip cross-validation (Table 5), XGBoost model achieved an impressive average accuracy of 98.36%. While this indicates the model's strong capability to fit the data, it is important to note that this metric may be overestimated due to the correlation between overlapping segments from the same subject. Thus, this result should be interpreted as an upper bound of the model's theoretical performance.

Of greater practical significance is the cross-subject cross-validation (Table 6), which evaluates the model's ability to generalize to unseen pilots. In this rigorous testing scenario, the framework maintained a robust average accuracy of 88.42%. This performance is particularly notable as it surpasses our previous EEG-only ASFT-Transformer (87.72%), confirming that the fusion of ECG statistical features effectively compensates for individual variability. Among the classifiers compared, XGBoost demonstrated the best overall balance between accuracy and efficiency. Although the FT-Transformer achieved a marginally higher accuracy in some metrics, XGBoost provided a highly competitive performance with a drastically reduced computational cost (average training time of 39.3 s vs. about 10 min).

To provide a more detailed analysis of model stability, Figure 11 presents a boxplot comparing the accuracy distributions for each model under the cross-subject cross-validation scheme. This visualization moves beyond the average accuracy reported in Table 6, illustrating the median, interquartile range (IQR), and overall spread of the accuracy scores obtained across the 5 repeated validation runs.

Figure 11

Figure 11. Classification accuracy comparison among different models. The performance is evaluated using 5-time 4-fold cross-subject cross-validation to assess generalization. The central line in each box represents the median accuracy, while the box edges define the interquartile range (IQR). Whiskers extend to the max/min values.

As the boxplot illustrates, our proposed framework based on XGBoost not only achieves a high median accuracy but also exhibits a compact distribution. This indicates highly consistent and robust accuracy when tested on different groups of unseen subjects. While the FT-Transformer reaches a slightly higher median accuracy, our framework's narrow distribution demonstrates comparable stability, reinforcing its reliability. In contrast, several other models show wider distributions, signifying greater variability in their accuracy and less consistent generalization. This visual evidence further substantiates that our framework strikes a balance between achieving high accuracy and maintaining reliable performance.

In summary, while the FT-Transformer achieves the highest absolute accuracy, our proposed framework, by fusing optimized EEG and ECG features, provides an advanced solution. It outperforms our previous EEG-only framework and represents a significant breakthrough in training efficiency, validating it as a solution that strikes an optimal balance between high accuracy, generalization, and computational cost.

4 Discussion

This study addresses the need for a pilot fatigue detection framework that balances accuracy with computational efficiency. By transforming physiological signal analysis into a tabular classification task, we successfully integrated optimized EEG and ECG features. The results re-validate the ANOVA-SVM strategy for effective data reduction, which, as shown in Tables 2, 3, lays the foundation for high model performance by eliminating redundant information.

A critical finding is the superiority of multimodal fusion over single-modality approaches. However, regarding validation, a distinction must be made between metrics. While cross-clip cross-validation yielded 98.36% accuracy, this likely overestimates performance due to the correlation of overlapping segments from the same subject. Consequently, we emphasize the cross-subject accuracy of 88.42% as the primary benchmark. This result, derived from unseen pilots, confirms that our framework captures robust, subject-invariant physiological signatures, significantly outperforming the EEG-only ASFT-Transformer (87.72%). Furthermore, the XGBoost classifier reduced average training time to 39.3 s, which is a drastic improvement over the 8–10 min required by deep learning baselines, positioning the framework as a feasible solution for resource-constrained deployment.

5 Limitations

To ensure scientific rigor, several key limitations must be acknowledged. First, regarding the labeling strategy, we categorized pre-training and post-training data as alert and fatigue states respectively. This approach was adopted to generate objective labels consistent with literature on prolonged flight tasks and to avoid the variability inherent in subjective self-assessments. However, a limitation remains that without corroborating measures, the physiological state is inferred rather than explicitly validated. Second, the exclusive recruitment of male pilots and the use of a flight simulator limit the ecological validity and generalizability of findings to gender-diverse populations and real-world flight conditions. Third, the use of short-term (10-min) measurements assumes a brief snapshot captures the pilot's state, necessitating further validation in continuous monitoring scenarios. Additionally, regarding statistical analysis, feature selection was performed using independent ANOVAs without correction for multiple comparisons (e.g., Bonferroni or False Discovery Rate). While this heuristic filtering approach theoretically increases the risk of Type I errors, it was chosen to minimize Type II errors (excluding informative features) at the screening stage, with the final feature robustness confirmed by the cross-subject classification performance. In future studies, more rigorous statistical protocols, such as applying correction methods or utilizing multivariate approaches (e.g., MANOVA), are planned to be implemented to further substantiate these physiological findings.

6 Conclusion

This paper presents a computationally efficient framework for pilot fatigue detection by fusing multimodal physiological signals. By employing a two-stage optimization process, ANOVA-SVM feature selection followed by XGBoost classification, the study overcomes the computational bottlenecks of deep learning models.

Experimental results indicate the framework's potential for practical application. The system achieved 88.42% accuracy in rigorous cross-subject validation, suggesting that integrating ECG statistical features enhances generalization to new individuals. Coupled with a training time of only 39.3 s, this approach represents an important step toward developing objective fatigue management tools. Future work will focus on addressing the identified limitations by validating this method with diverse populations and subjective fatigue measures in operational environments.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by Ethics Committee of Civil Aviation Shanghai Hospital. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

KC: Conceptualization, Investigation, Methodology, Validation, Writing – original draft, Writing – review & editing. JL: Conceptualization, Methodology, Validation, Writing – review & editing. JZ: Conceptualization, Data curation, Writing – original draft, Writing – review & editing. YX: Conceptualization, Data curation, Writing – original draft, Writing – review & editing. LZ: Conceptualization, Data curation, Writing – review & editing. ZG: Conceptualization, Investigation, Writing – review & editing.

Funding

The author(s) declared that financial support was received for this work and/or its publication. This study was funded by the National Natural Science Foundation of China (U2333202 and 52272351) and the Civil Aviation Administration of China (ASSA2024/121).

Acknowledgments

The authors acknowledge the EEG data provided by the Civil Aviation Shanghai Hospital, China.

Conflict of interest

The author(s) declared that this work was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declared that generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Awais, M., Badruddin, N., and Drieberg, M. (2017). A hybrid approach to detect driver drowsiness utilizing physiological signals to improve system performance and wearability. Sensors. 17:1991. doi: 10.3390/s17091991

PubMed Abstract | Crossref Full Text | Google Scholar

Berberich, J., and Leitner, R. (2017). The look of tiredness: evaluation of pilot fatigue based on video recordings. Aviat. Psychol. Appl. Human Fact. 7, 86–94. doi: 10.1027/2192-0923/a000122

Crossref Full Text | Google Scholar

Blanco, J. A., Johnson, M. K., Jaquess, K. J., Oh, H., Lo, L.-C., Gentili, R. J., et al. (2018). Quantifying cognitive workload in simulated flight using passive, dry EEG measurements. IEEE Trans. Cognit. Dev. Syst. 10, 373–383. doi: 10.1109/TCDS.2016.2628702

Crossref Full Text | Google Scholar

Borghini, G., Astolfi, L., Vecchiato, G., Mattia, D., and Babiloni, F. (2014). Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neurosci. Biobehav. Rev. 44, 58–75. doi: 10.1016/j.neubiorev.2012.10.003

PubMed Abstract | Crossref Full Text | Google Scholar

Brezonakova, A. D. (2023). Pilot fatigue experience and fatigue reporting: Short-haul airline operation. Transp. Res. Procedia 74, 330–337. doi: 10.1016/j.trpro.2023.11.152

Crossref Full Text | Google Scholar

Butkevičiūtė, E., Michalkovič, A., and Bikulčienė, L. (2022). Ecg signal features classification for the mental fatigue recognition. Mathematics 10:3395. doi: 10.3390/math10183395

Crossref Full Text | Google Scholar

Cai, H., Qu, Z., Li, Z., Zhang, Y., Hu, X., and Hu, B. (2020). Feature-level fusion approaches based on multimodal EEG data for depression recognition. Inform. Fusion 59, 127–138. doi: 10.1016/j.inffus.2020.01.008

Crossref Full Text | Google Scholar

Caldwell, J. A., Mallis, M. M., Caldwell, J. L., Paul, M. A., Miller, J. C., and Neri, D. F. (2009). Fatigue countermeasures in aviation. Aviat. Space Environ. Med. 80, 29–59. doi: 10.3357/ASEM.2435.2009

PubMed Abstract | Crossref Full Text | Google Scholar

Cao, L., Wang, W., Dong, Y., and Fan, C. (2024). Advancing classroom fatigue recognition: a multimodal fusion approach using self-attention mechanism. Biomed. Signal Process. Control 89:105756. doi: 10.1016/j.bspc.2023.105756

Crossref Full Text | Google Scholar

Chen, T., and Guestrin, C. (2016). “Xgboost: a scalable tree boosting system,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '16 (New York, NY: Association for Computing Machinery).

Google Scholar

Chen, Y., Ge, H., Su, X., and Ma, X. (2024). Classification of exercise fatigue levels by multi-class svm from ecg and hrv. Med. Biol. Eng. Comput. 62, 2853–2865. doi: 10.1007/s11517-024-03116-w

PubMed Abstract | Crossref Full Text | Google Scholar

Chikhi, S., Matton, N., and Blanchet, S. (2022). Eeg power spectral measures of cognitive workload: a meta-analysis. Psychophysiology 59:e14009. doi: 10.1111/psyp.14009

PubMed Abstract | Crossref Full Text | Google Scholar

Civil Aviation Administration of China (2021). Rules on the Certification of Public Air Carriers of Large Aircraft, CCAR-121. Beijing: CAAC.

Google Scholar

Dehais, F., Duprès, A., Blum, S., Drougard, N., Scannella, S., Roy, R. N., et al. (2019). Monitoring pilot's mental workload using ERPs and spectral power with a six-dry-electrode EEG system in real flight conditions. Sensors 19:1324. doi: 10.3390/s19061324

PubMed Abstract | Crossref Full Text | Google Scholar

Du, G., Long, S., Li, C., Wang, Z., and Liu, P. X. (2023). A product fuzzy convolutional network for detecting driving fatigue. IEEE Trans. Cybern. 53, 4175–4188. doi: 10.1109/TCYB.2021.3123842

PubMed Abstract | Crossref Full Text | Google Scholar

Eoh, H. J., Chung, M. K., and Kim, S.-H. (2005). Electroencephalographic study of drowsiness in simulated driving with sleep deprivation. Int. J. Ind. Ergon. 35, 307–320. doi: 10.1016/j.ergon.2004.09.006

Crossref Full Text | Google Scholar

European Union Aviation Safety Agency (2012). Commission Regulation (EU) No 965/2012 of 5 October 2012-Laying Down Technical Requirements and Administrative Procedures Related to Air Operations Pursuant to Regulation (EC) No 216/2008 of the European Parliament and of the Council. Cologne: EASA.

Google Scholar

Falih, B. S., Sabir, M. K., and Aydın, A. (2024). Impact of sliding window overlap ratio on EEG-based asd diagnosis using brain hemisphere energy and machine learning. Appl. Sci. 14:11702. doi: 10.3390/app142411702

Crossref Full Text | Google Scholar

Faust, O., Hagiwara, Y., Hong, T. J., Lih, O. S., and Acharya, U. R. (2018). Deep learning for healthcare applications based on physiological signals: a review. Comput. Methods Programs Biomed. 161, 1–13. doi: 10.1016/j.cmpb.2018.04.005

PubMed Abstract | Crossref Full Text | Google Scholar

Federal Aviation Administration (2013). Fatigue Risk Management Systems for Aviation Safety. Washington, DC: FAA.

Google Scholar

Fu, R., Han, M., Yu, B., Shi, P., and Wen, J. (2020). Phase fluctuation analysis in functional brain networks of scaling EEG for driver fatigue detection. Promet-Traffic Transp. 32, 487–495. doi: 10.7307/ptt.v32i4.3303

Crossref Full Text | Google Scholar

Gao, D., Li, P., Wang, M., Liang, Y., Liu, S., Zhou, J., et al. (2024a). CSF-GTNET: a novel multi-dimensional feature fusion network based on convnext-gelu- bilstm for EEG-signals-enabled fatigue driving detection. IEEE J. Biomed. Health Inform. 28, 2558–2568. doi: 10.1109/JBHI.2023.3240891

PubMed Abstract | Crossref Full Text | Google Scholar

Gao, D., Wang, K., Wang, M., Zhou, J., and Zhang, Y. (2024b). Sft-net: A network for detecting fatigue from EEG signals by combining 4D feature flow and attention mechanism. IEEE J. Biomed. Health Inform. 28, 4444–4455. doi: 10.1109/JBHI.2023.3285268

PubMed Abstract | Crossref Full Text | Google Scholar

Gao, D., Zhou, Z., Peng, Z., Zhang, H., Liu, S., Wang, M., et al. (2025). A multimodal functional structure-based graph neural network for fatigue detection. Brain Res. Bull. 230:111420. doi: 10.1016/j.brainresbull.2025.111420

PubMed Abstract | Crossref Full Text | Google Scholar

Gao, Z., Wang, X., Yang, Y., Mu, C., Cai, Q., Dang, W., et al. (2019). EEG-based spatio-temporal convolutional neural network for driver fatigue evaluation. IEEE Trans. Neural Netw. Learn. Syst. 30, 2755–2763. doi: 10.1109/TNNLS.2018.2886414

PubMed Abstract | Crossref Full Text | Google Scholar

Gläsener, D., Post, J., Cyrol, D., and Sammito, S. (2023). Fatigue among air crews on (ultra)-long-range flights - a comparison of subjective fatigue with objective concentration ability. Heliyon 9:e21669. doi: 10.1016/j.heliyon.2023.e21669

PubMed Abstract | Crossref Full Text | Google Scholar

Gorishniy, Y., Rubachev, I., Khrulkov, V., and Babenko, A. (2021). “Revisiting deep learning models for tabular data,” in Advances in Neural Information Processing Systems, eds. M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan (Red Hook, NY: Curran Associates, Inc.), 18932–18943.

Google Scholar

Guo, D., Wang, C., Qin, Y., Shang, L., Gao, A., Tan, B., et al. (2025). Assessment of flight fatigue using heart rate variability and machine learning approaches. Front. Neurosci. 19:1621638. doi: 10.3389/fnins.2025.1621638

PubMed Abstract | Crossref Full Text | Google Scholar

Hamann, A., and Carstengerdes, N. (2022). Investigating mental workload-induced changes in cortical oxygenation and frontal theta activity during simulated flights. Sci. Rep. 12:6449. doi: 10.1038/s41598-022-10044-y

PubMed Abstract | Crossref Full Text | Google Scholar

Hamann, A., and Carstengerdes, N. (2023). Assessing the development of mental fatigue during simulated flights with concurrent EEG-fNIRS measurement. Sci. Rep. 13:4738. doi: 10.1038/s41598-023-31264-w

PubMed Abstract | Crossref Full Text | Google Scholar

Huang, J., Liu, Y., and Peng, X. (2022). Recognition of driver's mental workload based on physiological signals, a comparative study. Biomed. Signal Process. Control 71:103094. doi: 10.1016/j.bspc.2021.103094

Crossref Full Text | Google Scholar

International Civil Aviation Organization (2013). Manual of Evidence-Based Training, Doc 9995. Montréal, QC: ICAO.

Google Scholar

International Civil Aviation Organization (2020). Manual for the Oversight of Fatigue Management Approaches, Doc 9966. Montréal, QC: ICAO.

Google Scholar

Jebelli, H., Hwang, S., and Lee, S. (2018). Eeg-based workers' stress recognition at construction sites. Automat. Construct. 93:315–324. doi: 10.1016/j.autcon.2018.05.027

Crossref Full Text | Google Scholar

Jia, H., Xiao, Z., and Ji, P. (2023). End-to-end fatigue driving EEG signal detection model based on improved temporal-graph convolution network. Comput. Biol. Med. 152:106431. doi: 10.1016/j.compbiomed.2022.106431

PubMed Abstract | Crossref Full Text | Google Scholar

Jiao, Z., Gao, X., Wang, Y., Li, J., and Xu, H. (2018). Deep convolutional neural networks for mental load classification based on EEG data. Pattern Recognit. 76, 582–595. doi: 10.1016/j.patcog.2017.12.002

Crossref Full Text | Google Scholar

Jing, D., Liu, D., Zhang, S., and Guo, Z. (2020). Fatigue driving detection method based on EEG analysis in low-voltage and hypoxia plateau environment. Int. J. Transport. Sci. Technol. 9, 366–376. doi: 10.1016/j.ijtst.2020.03.008

Crossref Full Text | Google Scholar

Jun-Ya, S., and Rui-Shan, S. (2023). Pilot fatigue survey: A study of the mutual influence among fatigue factors in the “work” dimension. Front. Public Health 11:1014503. doi: 10.3389/fpubh.2023.1014503

PubMed Abstract | Crossref Full Text | Google Scholar

Kaida, K., Takahashi, M., Åkerstedt, T., Nakata, A., Otsuka, Y., Haratani, T., et al. (2006). Validation of the karolinska sleepiness scale against performance and EEG variables. Clini. Neurophysiol. 117, 1574–1581. doi: 10.1016/j.clinph.2006.03.011

PubMed Abstract | Crossref Full Text | Google Scholar

Kioulepoglou, P., David, K. M., Nikolaidis, V., and Makris, I. (2024). Fatigue reporting in hellenic skies: attitudes, barriers and insights from military, commercial and general aviation pilots. Int. J. Aviat. Aeronaut. Aerospace 11:1934. doi: 10.58940/2374-6793.1934

Crossref Full Text | Google Scholar

Li, X., Tang, J., Li, X., and Yang, Y. (2024). CWSTR-Net: a channel-weighted spatial-temporal residual network based on nonsmooth nonnegative matrix factorization for fatigue detection using EEG signals. Biomed. Signal Process. Control 97:106685. doi: 10.1016/j.bspc.2024.106685

Crossref Full Text | Google Scholar

Lin, C.-T., Chuang, C.-H., Huang, C.-S., Tsai, S.-F., Lu, S.-W., Chen, Y.-H., et al. (2014). Wireless and wearable EEG system for evaluating driver vigilance. IEEE Trans. Biomed. Circuits Syst. 8, 165–176. doi: 10.1109/TBCAS.2014.2316224

PubMed Abstract | Crossref Full Text | Google Scholar

Liu, J., Zhou, Y., He, Q., and Gao, Z. (2025). ASFT-transformer: a fast and accurate framework for EEG-based pilot fatigue recognition. Sensors 25:6256. doi: 10.3390/s25196256

PubMed Abstract | Crossref Full Text | Google Scholar

Liu, Q., Liu, Y., Chen, K., Wang, L., Li, Z., Ai, Q., et al. (2021). Research on channel selection and multi-feature fusion of EEG signals for mental fatigue detection. Entropy 23:e23040457. doi: 10.3390/e23040457

PubMed Abstract | Crossref Full Text | Google Scholar

Luo, H., Qiu, T., Liu, C., and Huang, P. (2019). Research on fatigue driving detection using forehead EEG based on adaptive multi-scale entropy. Biomed. Signal Process. Control 51:50–58. doi: 10.1016/j.bspc.2019.02.005

Crossref Full Text | Google Scholar

Maclean, A. W., Fekken, G. C., Saskin, P., and Knowles, J. B. (1992). Psychometric evaluation of the stanford sleepiness scale. J. Sleep Res. 1, 35–39. doi: 10.1111/j.1365-2869.1992.tb00006.x

PubMed Abstract | Crossref Full Text | Google Scholar

Mangin, T., Audiffren, M., Lorcery, A., Mirabelli, F., Benraiss, A., and André, N. (2022). A plausible link between the time-on-task effect and the sequential task effect. Front. Psychol. 13:998393. doi: 10.3389/fpsyg.2022.998393

PubMed Abstract | Crossref Full Text | Google Scholar

Mannawaduge, C. D., Pignata, S., Banks, S., and Dorrian, J. (2024). Evaluating fatigue management regulations for flight crew in australia using a new fatigue regulation evaluation framework (FREF). Transport Policy 151, 75–84. doi: 10.1016/j.tranpol.2024.02.002

Crossref Full Text | Google Scholar

Marcus, J. H., and Rosekind, M. R. (2017). Fatigue in transportation: NTSB investigations and safety recommendations. Injury Prevent. 23, 232–238. doi: 10.1136/injuryprev-2015-041791

PubMed Abstract | Crossref Full Text | Google Scholar

Mehmood, I., Li, H., Qarout, Y., Umer, W., Anwer, S., Wu, H., et al. (2023). Deep learning-based construction equipment operators' mental fatigue classification using wearable EEG sensor data. Adv. Eng. Inform. 56:101978. doi: 10.1016/j.aei.2023.101978

Crossref Full Text | Google Scholar

Mouazen, B., Bendaouia, A., Bellakhdar, O., Laghdaf, K., Ennair, A., Abdelwahed, E. H., et al. (2025). Transparent EEG analysis: leveraging autoencoders, bi-lstms, and shap for improved neurodegenerative diseases detection. Sensors 25:5690. doi: 10.3390/s25185690

PubMed Abstract | Crossref Full Text | Google Scholar

Mu, S., Liao, S., Tao, K., and Shen, Y. (2024). Intelligent fatigue detection based on hierarchical multi-scale ecg representations and hrv measures. Biomed. Signal Process. Control 92:106127. doi: 10.1016/j.bspc.2024.106127

Crossref Full Text | Google Scholar

Naim, F., Mustafa, M., Sulaiman, N., and Zahari, Z. L. (2022). Dual-layer ranking feature selection method based on statistical formula for driver fatigue detection of emg signals. Traitement du Signal 39, 1079–1088. doi: 10.18280/ts.390335

Crossref Full Text | Google Scholar

Ni, Z., Sun, F., and Li, Y. (2022). Heart rate variability-based subjective physical fatigue assessment. Sensors 22:3199. doi: 10.3390/s22093199

PubMed Abstract | Crossref Full Text | Google Scholar

Pan, Y., Steven Li, Z., Zhang, E., and Guo, Z. (2023). A vigilance estimation method for high-speed rail drivers using physiological signals with a two-level fusion framework. Biomed. Signal Process. Control 84:104831. doi: 10.1016/j.bspc.2023.104831

Crossref Full Text | Google Scholar

Piotrowski, Z., and Szypulska, M. (2017). Classification of falling asleep states using hrv analysis. Biocybernet. Biomed. Eng. 37, 290–301. doi: 10.1016/j.bbe.2017.02.003

Crossref Full Text | Google Scholar

Shen, X., Liu, X., Hu, X., Zhang, D., and Song, S. (2023). Contrastive learning of subject-invariant EEG representations for cross-subject emotion recognition. IEEE Trans. Affect. Comp. 14, 2496–2511. doi: 10.1109/TAFFC.2022.3164516

Crossref Full Text | Google Scholar

Subasi, A., Saikia, A., Bagedo, K., Singh, A., and Hazarika, A. (2022). EEG-based driver fatigue detection using fawt and multiboosting approaches. IEEE Trans. Indust. Inform. 18, 6602–6609. doi: 10.1109/TII.2022.3167470

Crossref Full Text | Google Scholar

Sun, W., Jiang, W., Li, C., Si, Y., Li, S., Guo, M., et al. (2024). Study on identification method of driver fatigue considering individual ecg differences. Cognit. Technol. Work 26, 301–312. doi: 10.1007/s10111-024-00755-9

Crossref Full Text | Google Scholar

Tang, J., Li, X., Yang, Y., and Zhang, W. (2021). Euclidean space data alignment approach for multi-channel lstm network in EEG based fatigue driving detection. Electron. Lett. 57, 836–838. doi: 10.1049/ell2.12275

Crossref Full Text | Google Scholar

Tang, X., Li, W., Li, X., Ma, W., and Dang, X. (2020). Motor imagery EEG recognition based on conditional optimization empirical mode decomposition and multi-scale convolutional neural network. Expert Syst. Appl. 149:113285. doi: 10.1016/j.eswa.2020.113285

Crossref Full Text | Google Scholar

van Weelden, E., Alimardani, M., Wiltshire, T. J., and Louwerse, M. M. (2022). Aviation and neurophysiology: a systematic review. Appl. Ergon. 105:103838. doi: 10.1016/j.apergo.2022.103838

PubMed Abstract | Crossref Full Text | Google Scholar

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., et al. (2017). “Attention is all you need,” in Advances in Neural Information Processing Systems, eds. I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Red Hook, NY: Curran Associates, Inc.).

Google Scholar

Wang, F., Wang, H., and Fu, R. (2018). Real-time ecg-based detection of fatigue driving using sample entropy. Entropy 20:e20030196. doi: 10.3390/e20030196

PubMed Abstract | Crossref Full Text | Google Scholar

Wang, F., Wu, S., Ping, J., Xu, Z., and Chu, H. (2021). Eeg driving fatigue detection with pdc-based brain functional network. IEEE Sens. J. 21, 10811–10823. doi: 10.1109/JSEN.2021.3058658

Crossref Full Text | Google Scholar

Wang, F., Wu, S., Zhang, W., Xu, Z., Zhang, Y., and Chu, H. (2020). Multiple nonlinear features fusion based driving fatigue detection. Biomed. Signal Process. Control 62:102075. doi: 10.1016/j.bspc.2020.102075

Crossref Full Text | Google Scholar

Wang, F., Yao, W., Lu, B., and Fu, R. (2024). Ecg-based real-time drivers' fatigue detection using a novel elastic dry electrode. IEEE Trans. Instrum. Meas. 73, 1–16. doi: 10.1109/TIM.2023.3343799

Crossref Full Text | Google Scholar

Wang, K., Mao, X., Song, Y., and Chen, Q. (2025). Eeg-based fatigue state evaluation by combining complex network and frequency-spatial features. J. Neurosci. Methods 416:110385. doi: 10.1016/j.jneumeth.2025.110385

PubMed Abstract | Crossref Full Text | Google Scholar

Wang, Y., Han, M., Peng, Y., Zhao, R., Fan, D., Meng, X., et al. (2024). Lgnet: Learning local-global EEG representations for cognitive workload classification in simulated flights. Biomed. Signal Process. Control 92:106046. doi: 10.1016/j.bspc.2024.106046

Crossref Full Text | Google Scholar

Wingelaar-Jagt, Y. Q., Wingelaar, T. T., Riedel, W. J., and Ramaekers, J. G. (2021). Fatigue in aviation: safety risks, preventive strategies and pharmacological interventions. Front. Physiol. 12:712628. doi: 10.3389/fphys.2021.712628

PubMed Abstract | Crossref Full Text | Google Scholar

Zhang, Y., Guo, H., Zhou, Y., Xu, C., and Liao, Y. (2023). Recognising drivers' mental fatigue based on EEG multi-dimensional feature selection and fusion. Biomed. Signal Process. Control 79:104237. doi: 10.1016/j.bspc.2022.104237

Crossref Full Text | Google Scholar

Zheng, W.-L., and Lu, B.-L. (2017). A multimodal approach to estimating vigilance using EEG and forehead EOG. J. Neural Eng. 14:026017. doi: 10.1088/1741-2552/aa5a98

PubMed Abstract | Crossref Full Text | Google Scholar

Zhou, Y., Zeng, C., and Mu, Z. (2023). Optimal feature-algorithm combination research for EEG fatigue driving detection based on functional brain network. IET Biometrics 12, 65–76. doi: 10.1049/bme2.12108

Crossref Full Text | Google Scholar