Electro-Encephalography and Electro-Oculography in Aeronautics: A Review Over the Last Decade (2010–2020)

Electro-encephalography (EEG) and electro-oculography (EOG) are methods of electrophysiological monitoring that have potentially fruitful applications in neuroscience, clinical exploration, the aeronautical industry, and other sectors. These methods are often the most straightforward way of evaluating brain oscillations and eye movements, as they use standard laboratory or mobile techniques. This review describes the potential of EEG and EOG systems and the application of these methods in aeronautics. For example, EEG and EOG signals can be used to design brain-computer interfaces (BCI) and to interpret brain activity, such as monitoring the mental state of a pilot in determining their workload. The main objectives of this review are to, (i) offer an in-depth review of literature on the basics of EEG and EOG and their application in aeronautics; (ii) to explore the methodology and trends of research in combined EEG-EOG studies over the last decade; and (iii) to provide methodological guidelines for beginners and experts when applying these methods in environments outside the laboratory, with a particular focus on human factors and aeronautics. The study used databases from scientific, clinical, and neural engineering fields. The review first introduces the characteristics and the application of both EEG and EOG in aeronautics, undertaking a large review of relevant literature, from early to more recent studies. We then built a novel taxonomy model that includes 150 combined EEG-EOG papers published in peer-reviewed scientific journals and conferences from January 2010 to March 2020. Several data elements were reviewed for each study (e.g., pre-processing, extracted features and performance metrics), which were then examined to uncover trends in aeronautics and summarize interesting methods from this important body of literature. Finally, the review considers the advantages and limitations of these methods as well as future challenges.


INTRODUCTION
Electro-encephalography (EEG) and electro-oculography (EOG) are methods of electrophysiological monitoring in neuroscience and clinical exploration. EEG and EOG signals can be used in the design of brain-computer interfaces (BCI) that interpret brain activity. Due to the fact that they are straightforward approaches to evaluating brain oscillations and eye movements, and because they use standard laboratory and/or mobile techniques, EEG and EOG have in recent years been applied to the aeronautical industry. This review describes the potential of these systems when applied in aeronautics. The main objectives are, (i) to offer an in-depth review of literature on the basics of EEG and EOG and their application in aeronautics; (ii) to explore the methodology and trends of research in combined EEG-EOG studies over the last decade; and (iii) to provide methodological guidelines for beginners and experts when applying these methods in environments outside the laboratory, with a particular focus on human factors and aeronautics.
The review is structured as follows: section Introduction first describe EEG and EOG techniques, the main approaches to acquiring signals, and the use of them in aeronautics, before concluding with discussions of the motivations for applying them to aeronautics, and a discussion of contributions to this field. Section Methodology of the Review describes the methodology used to construct the review and taxonomy table, respectively. Section Results then presents the results of the review, including the trend analyses. Finally, section Discussion discusses research over the last decade with a focus on combined approaches to EEG-EOG and the relevance of this approach to aeronautics.

Origin of EEG Response
EEG is one of the most important methods of evaluating brain disorders and monitoring the electrical behavior of the brain. The EEG also has the major advantage of excellent temporal resolution, which enables it to study neural activity at a millisecond scale, and best approximates the neural timing. EEG allows for the analysis of the various rhythms generated by different cortical regions. The current produced by the electrical activity of neurons reaches the surface of the scalp. EEG offers a non-invasive method of recording the difference between the potentials that are generated by neural sources and annoying non-neural artifacts. As the signal induces important temporal and spatial variations, the electrode positions are determined using multiple channels settled by the international 10-20 standard. Recommendations for the use of EEG equipment in assessments are provided by the International Federation of Clinical Neurophysiology (Babiloni et al., 2020). The electrical signal is diffused from electrodes placed on the scalp to an external amplifier that intensifies the potentials. EEG signals are commonly detected between 20 and 150 µV in the 0.5-60 Hz band (Binnie et al., 1982). The signal is constantly sampled to provide a relevant temporal resolution to explore eventrelated potentials (ERP) and the EEG power spectrum. In research focused on frequency-based analyses (such as prefrontal lateralization of alpha or beta bands), a sampling rate of 128 Hz can be sufficient. When the objective is high time precision measurements (such as language-related high gamma activity), the EEG should ideally collect data at a high sampling rate (>500 Hz).
EEG analysis could be conducted in the time domain, frequency domain, or time-frequency domain. Extracting temporal features (e.g., amplitude, power, average periodicity, and synchronization) provides useful qualitative information for the classification. However, all of these temporal characteristics do not describe the signal in its integrity. During a seizure, for instance, the signal is not stable, and it is necessary to separate the frequency components to classify the seizure. Therefore, in addition to time-domain features, frequency domain exploration is also needed to detect and classify all types of seizures. A timedomain analysis provides better spatial information, thus poor frequency content information is required for EEG classification. The frequency-domain can provide time information when the function is windowed. The choice of window size is the biggest challenge in frequency analysis. Time-frequency analysis solves these two problems. Some EEG investigations consider that wavelet analysis is the best method for time-frequency analysis. Generally, authors apply a series of transformations e.g., Fourier transform (Radha et al., 2014), Short Time Fourier Transform (Görür et al., 2003), Wavelet transform (Fraiwan et al., 2012), Hilbert-Hung transform (Li et al., 2009), and Empirical Mode Decomposition (Hassan and Bhuiyan, 2016). Spectral analyses, based on Fourier Transform, are then commonly used to convert the time function into different frequencies and to calculate the amplitude in each frequency band. The frequency bands are universally classified as the following: slow and sleep wave delta (2-4 Hz), arousal wave theta (4-8 Hz), relaxation wave alpha (8)(9)(10)(11)(12), and active wave beta (13)(14)(15)(16)(17)(18)(19)(20)(21)(22)(23)(24)(25)(26)(27)(28)(29)(30)(31)(32). Sensorimotor rhythm frequency bands (13)(14)(15) are related to the sensorimotor rhythm and entitled as low beta. Delta waves are commonly frontally located in adults and posteriorly in children. Theta waves are mainly recorded in frontal areas during low brain activities, sleep, or drowsiness and cognitive processing. Alpha waves are among the first rhythmic waves documented and are recorded during relaxed conditions at decreased attention levels and in a wakeful state. The alpha waves are located in the occipital area and can be induced by closing eyes. Beta waves are often recorded in frontal or central areas when the eyes are open and are related to consciousness, alertness, arousal, and motor behaviors (Barry and De Blasio, 2017). Cognitive processes such as attention, learning, and diverse types of memory occur during gamma frequencies (over 33 Hz). Unconventional classifications have also been analyzed in some studies (Caldwell et al., 2002;Gevins et al., 2003;Dahlstrom and Nahlinder, 2009;Holm et al., 2009;De Vico Fallani et al., 2012;Zhang et al., 2019a). It is worth mentioning that the frequency limits of specific waves are conventional, as there is no proper way of determining their exact values.
Many studies on oscillations in brain dynamics have indicated that during fatigue accumulation and sustained attention, increased EEG power is detected in theta frequency bands in the frontal, parietal, and central regions. Theta power increase has been detected during working memory load situations (Klimesch et al., 2007), visual tasks (Yamada, 1998), flight simulations (Smith-Jentsch et al., 2001;Dussault et al., 2004;Borghini et al., 2014), and air-traffic control simulations (Postma et al., 2005). A decrease in the alpha is known to occur during complex and cognitively demanding tasks. It has also been shown (Postma et al., 2005) that alpha and beta bands are different between the beginning and the end of a mental fatigue task. Increases in theta and decreases in alpha oscillations have also been associated with an increase in the accuracy of task performance (Klimesch et al., 1999). Interestingly, such an increase in EEG power in the theta band can be used to characterize a single task from a multitask activity performed by pilots (Borghini et al., 2014). Varied EEG approach montages are given in complex higher-order cognitive operations.

EEG Approaches
Although great importance has been placed on classic unmovable EEG montages, they have disadvantages in that they are much bulkier and more time and effort consuming than the new generation of wearable EEG. The concept of wearable EEG improves upon the bulky and limited mobility of classic montages, using small devices that can record EEG outside of laboratory conditions. These miniaturized devices have the advantage of being able to detect EEG signals for short or long periods, for example when sleeping, which significantly improves the brain-computer interface (BCI) monitoring method. The importance of using the wireless EEG in BCI monitoring is to maximize wearability, enabling unconstrained mobility, usability, and reliability in operational environments.
As well as involving long preparation time and bulky design, classical scalp-mounted EEG are not suited to situations that involve environmental artifacts (e.g., aviation and space operation, or patients with a cochlear implant or hearing aid; Nogueira et al., 2019). To address these issues, novel EEG technologies use tiny electrodes that are placed externally around the ear Bleichner and Debener, 2017) or involve in-ear electrodes (Looney et al., 2011). The cEEGrid electrode array  is a promising device with 10 electrodes printed onto a C-shaped flexible board, enabling it to fit around and measure EEG data behind the ear. It also uses a small amount of electrolyte electrode gel, which ensures low-impedance contact between the cEEGrid electrodes and the skin. The capacity is stable because the gel does not dryup. The cEEGrid electrode array is then connected to a micro wireless amplifier and uses signal recordings from a cellphone. This accessible design means that it has potential applications in clinical settings, aeronautics, and other research areas.
With a reduced number of electrodes, the difference between cEEGrid performance and scalp EEG performance can be explained by the position of the reference electrode. The cEEGrid uses a local reference, with ground and recording electrodes allocated around the ear, giving out small signal amplitudes. However, for a scalp EEG with a smaller number of electrodes around the ear, the tip of the nose is used as a reference, and its position is farther from the recording electrode, and therefore gives higher amplitudes. In line with conventional EEG results, Debener et al. (2015) identified ERPs and alpha frequencies during an auditory oddball task with open eyes using the cEEGrid. Advanced explorations proved that the cEEGrid can detect neural signals to select the voice of a speaker with high precision, and can be even used as a BCI monitor for hearing aids (Mirkovic et al., 2016). Recently, the cEEGrid was successfully used in decoding selective attention in normal hearing and cochlear implant patients (Nogueira et al., 2019). Looney et al. (2011) presented the in-ear EEG device. It records to the same standards as the conventional scalp electrodes. However, the device relies on custom-made hearing aid earplugs, which take a wax impression of the ears (outer ear and external ear canal). The important benefits of in-ear EEG includes easy set-up; time of installation; accessibility for people with hair that is incompatible with EEG recordings; and durability because they are fixed in the ear canal. They are comfortable to wear, discreet, resembling earphones, earbuds, and earplugs, and facilitating everyday use. Contrary to scalp EEG, the in-ear EEG devices are easy to place without the presence of experienced assistants. They are held firmly in place and thus diminish motion artifacts. Since the electrodes are fixed on the earpiece surface, they offer a precise spatial positioning which decreases the inter-experiment variability. The feasibility of in-ear EEG for cognitive assessment has been studied in a few exploratory papers (Stochholm et al., 2016;Von Rosenberg et al., 2016;Zibrandtsen et al., 2016), which claim that in-ear EEG is a promising candidate for forthcoming explorations based on human monitoring technology (BCI, aviation, and space).
Despite advances in mobile EEG systems, around and inear EEG have some drawbacks. Compared to conventional scalp EEGs, these alternative systems have fewer electrodes and cover much-reduced regions. Therefore, it is recognized that its brain source analysis is less accurate than that of conventional scalp EEG. Note that physiological artifacts caused by the electrical activity of the skin are unavoidable, but may be relatively easy to deal with by temporal filtering and other post-processing procedures (Reis et al., 2014). The quality of the scalp EEG signal depends on the connection between the amplifier input and the skin surface. Wet electrodes based on conductive gel guarantee low impedance levels (<10 kΩ). Given that dry electrodes are placed on the skin without any gel application, the dry EEG system typically results in larger impedance than wet systems (Brown et al., 2010;Chen et al., 2014). To date, few studies have directly compared the data quality between these two systems. A recent study (Hinrichs et al., 2020) found that the resting state EEG power and ERP were comparable between the two systems. Di Flumeri et al. (2019) evaluated three different dry electrode types when compared with wet electrodes in terms of signal spectral features, mental state classification, and usability. The dry electrodes included a gold-coated single pin, multiple pins, and solid-gel electrodes. The results confirmed the high quality achieved by dry electrodes. They offered the same levels as wet electrodes with significantly reduced times of montage and increased the comfort of users. Although the signal quality is inevitably reduced, the dry electrodes are a reliable system for non-clinical and goal-oriented investigations, such as a comparison between two different mental states during real flight conditions.

EEG in Aeronautics
Since the late 1950s, many studies have investigated changes in EEG rhythms during flight conditions (Carl et al., 1959) despite the noisy environment (e.g., vibration, wind, acoustic noise, physiological artifacts, and important pilot physical movements). The recorded EEG signals show specific features and changes in the power spectrum of the various frequency bands associated with flight performance (Carl et al., 1959). One EEG experiment (Callan et al., 2015) conducted under real flight conditions, showed that the neural signature of inattentional deafness (e.g., inadvertently missing an auditory stimulus) was revealed by a reduction in phase resetting in the alpha and theta band frequencies. Sauvet et al. (2014) have used a single EEG channel during real long-haul flights to detect low states of vigilance. Another EEG study involved a critical scenario in a flight simulator (Dehais et al., 2016), which detected the existence of an early and unconscious gating mechanism based on the association between the auditory alarm and the N100 and P300 amplitude. Di Stasi et al. (2016) analyzed the inflight EEG activity of military helicopter pilots during real flights. They found that highly demanding procedures related to takeoff and landing were associated with higher power EEG frequency bands, whereas less demanding procedures were associated with lower EEG power over the same frequency bands. Several EEG investigations have focused on the mental workload of the operator during a flight task execution. For example, EEG revealed variations in alertness and could predict lower performance caused by increased mental workload during flight operations (Borghini et al., 2014). Theta changes were observed over the frontal brain areas when comparing the training improvements of novice pilots in flight simulation tasks (Borghini et al., 2014).
Workload perception was shown to be dependent on the level of experience, the abilities, or just the individual differences between pilots. The EEG was sensitive to workload level variance between novice and expert aircraft pilots during the execution of the identical task (Doppelmayr et al., 2008;Parasuraman and Jiang, 2012).
Recent studies exploring Air Traffic Controllers (Bernhardt et al., 2019) have computed an EEG-based workload index that could differentiate between task workload requirements exploring front-parietal brain function. Yet, EEG exploration has achieved traction in aviation and space operations, current studies face challenges related to the intrusive and bulky nature of the equipment (Caldwell et al., 2002), the discomfort of long preparation time, and dependence on gel electrodes (e.g., wet electrodes). In a recent study, Dehais et al. (2019) tested a dry EEG system on pilots during the low and high load traffic patterns associated with the passive auditory oddball. Analysis of ERPs and frequency characteristics has confirmed that dry EEG can be used to study cognition during highly ecological and noisy conditions. Recent developments in dry electrodes (Liao et al., 2012) allow them to reduce preparation time by eliminating the conductive gel or saline patch and preparing the skin to reduce the contact impedance. Some dry electrodes use a ring model with pins to touch the scalp (Hairston et al., 2014), while other dry electrodes use foam-based supplies covered in conductive textile materials. Other promising non-contact electrode techniques enable weak biopotentials to be amplified using a contact-free electrode-skin.

Origin of EOG Response
The EOG method recognizes the differences in potential changes induced by eye movements between two electrodes placed either horizontally or vertically around the eyes. Early eye movement studies revealed that the human eye is an electric dipole, comprising a positively charged cornea and a negatively charged retina (Anderson, 1937). When the eyeball moves in the direction of the electrode, the electrical potential increases and it decreases when the eye moves in the other direction. The voltage variation between the poles is known as corneoretinal potential and varies between 0.4 and 1 mV. The acquired potential varies based on the viewing angle, up to an angle of 30 degrees (Anderson, 1937). When the eyes do not move and are fixed, the potential does not vary. When the eye moves in the direction of the sensor, the potential greatly increases. Importantly, for people who are blind, or when people have their eyes are closed, the electrical changes remain. There are many configurations for electrode placement. Commonly, the horizontal electrodes are placed at the external borders of the eyes while the vertical ones are positioned above and below the eye (Singh and Singh, 2012). The horizontal EOG signal is the difference of the voltage between horizontal electrodes and the vertical EOG signal is the subtraction of the voltage between the vertical electrodes. Usually, the reference electrode is placed on the forehead (Barea et al., 2002) in the middle of the eyes (Yamagishi et al., 2006) or on the mastoids (Pettersson et al., 2013). Other atypical electrode placements have been extensively tested such as six electrodes positioned above and below both the eyes (Pettersson et al., 2013). Some typical and atypical electrode placements are presented in Figure 1.
Other studies have used a headband to place electrodes close to the skin and minimize noise, but without measuring the vertical eye movements (Chang, 2019). Adopting this technique, Yan et al. (2013) positioned five electrodes, one horizontal and one vertical around each eye in addition to the reference. Kanoh et al. (2015) placed electrodes near nasion and on both sides of the rhinion, which correspond to the bridge and nose pads of eyewear. Another remarkable placement was proposed by Manabe et al. (2013), who pioneered in-ear EOG signal detection by testing different material-electrode combinations that are suitable for daily use. They highlighted the relation between the in-ear electrical signal and eye movement but did not estimate the accuracy of the eye position. Favre-Felix et al. (2017) proposes a novel fitted ear EOG device that uses a single model taken from the participant's ear canals. Interestingly, they found a strong correlation between conventional EOG and ear EOG signals. Hence, ear EOG can be used to accurately estimate eye gaze in real-time situations, which is particularly relevant for aeronautics and the hearing aid industry. For example, a visually directed hearing aid could be faster and easier to manipulate than a hearing aid piloted by other conventional tools like a pointer or remote control (Hart et al., 2009).
In addition to these techniques, numerous tests have explored the placement options for mobile electrodes that measure changes in EOG voltage. JinsMeme (JINS Inc., Japan) implanted an EOG amplifier into eyeglasses. BioPac (CA, USA) connected a mobile EOG device to a stationary system (MP160) with wireless communication. Some other EOG systems have been connected to a wheelchair (Rajesh, 2014) or fixed into goggles that can be used in everyday environments (Bulling et al., 2009). One study (Acuna et al., 2014) has shown that a low-cost EOG system (<50 euros) can give good results for eye tracking. The performance achieved by this system is different depending on whether one considers vertical or horizontal movements; the monitoring of the latter being much more precise. Thus, an EOG system gives an accuracy of <3 • on the horizontal plane (with movements ranging from −50 • to +50 • ), and an accuracy of <4 • on the vertical plane (movements ranging from −10 • to +10 • ). For larger vertical movements (from −30 • to +30 • ), the imprecision increases to 11 • . Other systems could be used as an alternative way of measuring combined EEG and EOG including Muse (Toronto, Canada), Melomind (Paris, France), Open BCI (Brooklyn, NY, USA), and Emotiv system (San Francisco, USA). They present several advantages over conventional wired, bulky EEG devices in that they are economical, portable, and easy to use. In summary, the main reason for such diversity in electrode placement is due to the specificity of each experimental paradigm and whether other equipment is associated with EOG. Largely, these systems recognize eye movements with accuracy and further investigation of electrode placement and the quality of the EOG signal will be the subject of future research.

EOG Approaches
The main eye movements captured with EOG are fixations and saccades (Singh and Singh, 2012). Fixational movement is an attempt to capture the steady image on the retina. Depending on the quality of the processed information and the current cognitive load, fixation can last between 100 and 1,000 ms, with a mean range of 200-500 ms. The saccade movement changes the eye direction around the field of view and brings the object of interest into the foveal region. Saccades are characterized by a simultaneous and rapid change of the eye position between two fixation points. The duration of saccades is determined by the angular distance the eye travels during the movement. The saccades involve distinct patterns in the EOG signal. They are relatively easy to identify as the deflected amplitudes are above the common high-frequency noise level, and they are short (in duration). When the target is slowly moving, smooth pursuit movement allows us to maintain focus.
Smooth pursuit eye movements are typically initiated with a delay of 100-180 ms relative to the onset of an unpredictable motion (Lisberger and Westbrook, 1985). It is difficult to distinguish these movements on EOG recordings because it can be confounded with linear signal drift. Other movements (such as vestibulo-ocular movements and optokinetic nystagmus) allow for the gaze to stabilize during head and body movement. Nystagmus is a rhythmic, often rapid, involuntary eye movement that takes place when the head moves rapidly and the eyes move in response to the moving fluid in the vestibular system (Kang and Shaikh, 2017). Physiological nystagmus occurs during the motions of the head (vestibular nystagmus), or in the presence of patterns in the visual field (optokinetic nystagmus). It is often accompanied by a feeling of disorientation or vertigo and can be a reliable indicator of vestibular pathology. Vergence movements rotate the eyes inwards and outwards, with slow 10 • /s disconjugate movements. They allow the visual system to incorporate deep targets, permitting the perception of the world across three dimensions (Alvarez et al., 2005). Even if the eyeballs participate in the contraction of medial and lateral rectal muscles until the paired images are projected onto the fovea, they will activate shared and independent nerve regions. While saccadic and vergence eye movements rely on different muscles to restore the globes, they will activate both shared and distinct neural areas (Semmlow et al., 1998).
Numerous algorithms exist for detecting and modeling oculomotor parameters including eye blink, saccade direction, and fixation. Nolan et al. (2010) developed a fully automated statistical thresholding method that detects and classifies a portion of the signal as an eye blink if the potential of the portion exceeds the threshold. Niemenlehto (2009) analyzed EOG signals by exploring a detection method of a constant false alarm rate that detects saccades. Pettersson et al. (2013) detected eye blinks and saccades by identifying the threshold of the temporal derivative of the EOG signal. Behrens et al. (2010) explored the deviation of the eye-movement acceleration of the EOG signals. Toivanen et al. (2015) computed a realtime algorithm EOG signal that automatically detects blinks, saccades, and fixations and analyzes the temporal features of these reflexes. Although EOG signals have a good signal-to-noise ratio, due to their large amplitude and the relative ease with which they can detect saccades and blinks (Skotte et al., 2007), they are continuously contaminated by physiological artifacts such as electromyography (EMG). EMG artifacts are detected when the participant moves their facial muscles or body during EOG recording, such as jaw clenching, raising an eyebrow, or smiling. The forms and amplitudes of these artifacts depend on the types of movement, the position of the electrode, and sampling rates. High-frequency noise and smooth waveforms are beyond the potential range of eye movement-related signals and are usually differentiated. Median and low-pass filters are the most commonly used techniques to preserve these noises while retaining edge. The filters are based on split windows with varied cut-off frequencies and can be used for either online and/or offline analysis.
Besides EOG, other eye-tracking techniques include photoand video-oculography, or scleral research lenses (Duchowski, 2017). These techniques allow the capture of eye movements with greater precision than EOG but involve complex set-ups (flight deck video-camera integration, for instance) and involve processing pipelines that demand a large amount of power (computer vision algorithms). Alternatively, EOG is a technique associated with simple electrodes that can be embedded in a headset, providing good insights into the mental state and brain activity of a participant with lower consumption of energy and simpler processing pipelines, which are useful in aeronautics.

EOG in Aeronautics
The assessment of eye movements is particularly relevant for aeronautic (Peysakhovich et al., 2018) and neuro-ergonomic applications (Peysakhovich et al., 2019). Given the EOG's capacity to detect eye movements, it is an excellent candidate to be embedded into aeronautical systems. A number of important studies have related eye movement analysis to fatigue, mental workload, and cognitive performance in pilots. One of the most important causes of aviation accidents is provoked by human errors e.g., drowsiness or fatigue (Velazquez, 2018). Pilot fatigue and loss of control are considered by the Federal Aviation Administration as one of four common aviation hazards (Federal Aviation Administration, "Risk Management Handbook: U.S. Department of Transportation, " 2016). For example, a study described how two pilots from a commercial aircraft missed the target airport because both were sleeping (Borghini et al., 2014). Thus, EOG is considered to be an effective and predictive tool in detecting drowsiness markers such as a reduction in performance, and changes in the frequency of eye blinking (Oken et al., 2006). Interestingly, eye movement analysis can serve as a reference for the mental workload and state of pilots (Di Nocera et al., 2007). It has been shown that the workload reflected by eye fixation distribution varies according to the phases of the flight. The highest workload was noted during takeoff and landing, and the lowest during the cruise phase (Di Nocera et al., 2007). Brams et al. (2018) discuss how the gaze behavior of expert pilots differs from that of novices, suggesting relevant information about basic processes that explain the successful performance of expert pilots in flight. The authors explain that the expert pilots have expanded visual range that analyzes the global scene using the field of view next to the fovea, and that they then shift their attention to the pertinent area (Gegenfurtner et al., 2011). Experts have larger saccades that cover more areas and spend less time focusing on task-related regions. In addition to the important capacity to process information from multiple sources, this ability is also related to advanced cognitive performances. The gaze behavior of fighter pilots during flight conditions also varies according to altitude and speed (Svensson et al., 1997). An augmented workload can induce a lower percentage of eye fixation distribution outside and a higher percentage of fixation on the tactical display.
In addition to investigating the eye movement patterns underlying cognitive functions, alternative EOG (around the ear) may be a promising tool in real aeronautic environments. In numerous activities such as air traffic control or piloting an aircraft, the operators are equipped with peripherals (typically headsets). As the EOG requires only a few electrodes, it does not obstruct the visual field nor does it unnecessarily illuminate the eyes with infrared light. Therefore, this technique is convenient for head-mounted peripherals such as the audio headset used by pilots. Moreover, exploring EOG integration in control and communication peripherals may enhance human-system interaction and make psycho-physiological monitoring possible (e.g., based on blink rate or saccades). Such an approach could have numerous applications in aeronautics (fighters, helicopters, and unmanned aerial vehicle operation), naval systems, and control-command centers.

Motivations and Contributions
Although a large number of surveys are published for EEG and EOG in several fields, to the best of our knowledge, we have not identified any exhaustive review that summarizes a taxonomy of combined EEG-EOG studies. To bridge this gap, this work analyzes and categorizes developments from the last decade, examining published literature on combined EEG-EOG in different applications related to aeronautics.
This non-exhaustive review analyzes literature from the oldest to the most recent relevant studies. Section Introduction presents EEG and EOG signals, their methodological approaches, and a summary of their application in aeronautics. This section could be particularly beneficial for beginners in the field of electrophysiology.
The review presents a novel taxonomy model for classifying different approaches to EEG and EOG by emphasizing the extracted features, the applied pre-processing treatment, and the performance metrics. The studies belonging to each main category are sub-categorized as per the corresponding domain of application. Sections Methodology of the Review and Results are particularly beneficial for expert researchers in the field. Sections Discussion and Limitations and Conclusions Outline Insights and research directions as a means of providing guidelines for beginner and expert researchers who are interested in combined EEG-EOG studies in the future.
This review is useful for both beginners and experts in this field. It is intended to be a time-and resource-saving guide for those searching for exploration of neuro-physiological correlates in aeronautics. On the one hand, this review is beneficial for beginners who may easily explore specific literature on EEG and EOG from basic approaches to aeronautics applications in a single document and investigate it by spending less effort. On the other hand, it is useful for expert researchers who may explore the literature to discover trends and methodologies for exploring brain and eye signals. Experts may explore these approaches as valuable tools and could be useful in building and analyzing experimental paradigms.

METHODOLOGY OF THE REVIEW
Peer-reviewed journal articles in the English language and conference papers published on PubMed for the decade from 2010 to 2020 (January 2010-March 2020) were identified as targets for our review. The database was last queried on March 23, 2020. Search items related only to studies combining EEG and EOG methods in experiments. This filtered selection resulted in 249 studies that were then included or excluded according to subsequent criteria. The inclusion criteria were: (i) combined EEG-EOG studies; (ii) original research papers; and, (iii) that the experiments involved human subjects. The exclusion criteria were: (i) review, methodology, and proof-of-concept papers; (ii) dataset publications; (iii) animal model experiments; and, (iv) studies focusing solely on EEG or EOG.
Article titles were examined to evaluate the relevance of a selected article. If the title did not noticeably specify whether it corresponds to the inclusion and exclusion criteria, the abstract was then considered. Lastly, during the full paper examination and data gathering process, an article that did not meet the criteria was excluded. Non-peer reviewed papers, such as arXiv or BiorXiv electronic preprints were considered as a possible source of bias. Thus, preprints that had not been peer-reviewed were not selected. Finally, 150 studies were included in our final database. We constructed a data extraction table containing several data items related to our investigation question, according to previous comments with a similar scope and the author's past work in the field.
A taxonomy table was used to classify and describe the included items for each selected study. The first section is the justification category that emphasizes the domains of application of the included studies (e.g., sleep, BCI, signal processing, cognition, and driving). This valuable information allows an understanding of the scope of the research and also enabled us to determine trends in the analysis. The second section shows the bibliographic reference attributed to each study. The third section shows the name of the first author, giving a specific identification for each paper in the database. The fourth section outlines the year of publication, which is relevant for our selection criteria and statistical analysis. The fifth section regroups the number of subjects in each study to give a quick overview of the coherence of the relative main findings. The sixth section covers the publication category of the article, such as whether it was a journal article or a conference publication. These first six sections indicate the types of included papers and the main selected items. The seventh section includes all relevant information about the EEG and EOG data. This comprises the category of EEG and EOG equipment used, in addition to the different extracted features.
Standard EEG and standard EOG sections refer to classic wet electrode gels that are not mobile, while non-standard refers to dry wearable electrodes and any other equipment that is different from the standard category. Features Classification (Table 1) refers to the properties of the analyzed EEG and EOG signal. The content of this dataset includes the support of the statistical analysis relevant to the critical component of our discussion. The EEG features an extraction section and covers the signal processing methodology that was analyzed in each study, including the waveband frequency Delta (2-4 Hz), Theta (4-8 Hz), Alpha (9-13 Hz), Beta (14-32 Hz), Gamma (over 33 Hz), the ERP, or the basic raw signal. The EOG feature extractions consist of the explored signal processing for eye movements including blinks, saccades, fixations, raw signal, or whether the EOG was used as an artifact handling methodology. Polysomnography is a multiparameter test based on several different types of physiological signals called polysomnograms used in sleep diagnosis. Here, Huang Daly Min 2016 20 Frontiers in Neuroergonomics | www.frontiersin.org Signal processing Klein and Skrandies (2013) Pettersson Frontiers in Neuroergonomics | www.frontiersin.org Zeng and Song (2014) Barry 2014 20 Verma and Tiwary (2014) MacDonald MacDonald and Barry (2014) Lee Gordon et al. (2015) Signal processing Barthélemy et al. (2017) Delisle-Rodriguez Olsen et al. (2017) Korkalainen Kempfner 2011 20 Scarlatelli-Lima Frontiers in Neuroergonomics | www.frontiersin.org List of all references from 2010 to 2020 that used combined EEG-EOG methods. The references are classified by domain of application.
Frontiers in Neuroergonomics | www.frontiersin.org polysomnography was included in the taxonomy as an extracted feature from sleeping studies that analyzed both EEG and EOG signals.

RESULTS
A total of 150 papers were selected for inclusion in this review. Our search methodology returned 121 journal papers and 29 conference papers that met our criteria. We noted that the article papers were published in different journals; however, all the conference papers were only published in numerous years of the International Conferences of the Institute of Electrical and Electronics Engineers (IEEE). The included papers have combined EEG-EOG in various domains of application (see Figure 2). Most studies focused notably on sleep, signal processing, and BCI categories, respectively 34, 33, and 21% of the total selected studies. Sleep category concerns EEG and EOG data for sleep classification in healthy and clinic patients as well as using deep learning for classification of sleep stages and disorders.
The signal processing category regroups the development of tools, such as analyzing, modifying, and synthesizing signals, handling artifacts, learning features, and training models. BCI category groups the manipulation of hybrid EEG-EOG systems as a valuable communication tool for monitoring computers, wheelchairs, or a robotic exoskeleton. Seven percent of the selected studies belong to the cognition category. They were particularly related to analyzing the performance of attention and cognition. The remaining papers (5%) explored different analyses of driving with conditions such as sleep deprivation or fatigue. Figure 3 indicates the evolution of EEG-EOG investigations since 2010 in each domain of application. We did not observe any clear tendency apart from a growing concern for BCI. The first 3 months of 2020 associated with the years 2019 and 2018 alone account for 42% of the total selected publications. Nevertheless, given the relatively small number of publications to date, it is too early to draw conclusions about trends.
The number of subjects included in each study varies expressively across the different domains of application (see Figure 4). Seventy-five percent of the included datasets contained fewer than 30 participants. Some studies have datasets with a higher number of participants with at least 600 subjects (Singh et al., 2014;Olsen et al., 2017;Korkalainen et al., 2019), while others included studies used datasets with <5 subjects, particularly in BCI and signal processing categories.
When reviewing our included studies, we regrouped some of the common pre-processing steps employed ( Figure 5A). The pre-processing methodology presented some routine steps, such as bandpass filtering, downsampling, windowing, interpolating the bad channels, computing the average reference, or removing line noise. Eighty-seven percent of the selected studies used band-pass filters and notch filters that allow the extraction of characteristic signals located in the stimulus frequency and that remove noise and artifacts. Forty-seven percent of studies describe artifact handling, which consisted of eliminating certain types of noise, such as eyes and muscle artifacts. Seventeen percent of the studies downsampled the signal acquired at a higher sampling rate to 256 Hz or less. Even when deleting the noise might be essential to achieving relevant EEG decoding efficiency, 9% of the included studies did not explicitly mention pre-processing steps. The distribution of the selected preprocessing steps according to each domain of application is shown in Figure 6A. In particular, the filtering process was employed by studies in BCI (n = 26), sleep (n = 43), and signal processing (n = 45). Artifact handling was mostly used in signal processing (n = 34) and sleep (n = 18) domains. Downsampling was used in BCI, sleep (n = 7), and signal processing (n = 7) categories.
When assessing the performance of signal processing in the selected studies (Figure 5B), we were not surprised to find that most of them applied common metrics that are derived from confusion matrices. We classified the performance metrics into the component that emerged from the included studies: accuracy estimation (54%), sensitivity and specificity values (23%), false positive and/or false negative detection (22%), classification score (F1-score, Kappa score or Support Vector Machine score) (18%), an approximation of error correction (15%), and other noncommon metrics (31%).
The distribution of the selected performance metrics according to each domain of application is shown in Figure 6B. As expected, we noted an important evaluation of accuracy in the BCI (n = 23), signal processing (n = 21), and sleep (n = 33) domains. Figure 5C details the performed analysis on the registered EEG signals from the included studies. The evaluation revealed the following extracted features: raw signal (40%), alpha frequency (27%), polysomnography (20%), beta frequency (18%), theta frequency (17%), ERPs (17%), gamma frequency (10%), and delta frequency (6%).
The distribution of the EEG extracted features according to each domain of application is shown in Figure 6C. Particularly, alpha, beta, and theta frequencies were investigated in sleep and signal processing domains. ERPs were analyzed in BCI (n = 10), cognition (n = 5), and signal processing domains (n = 11). The raw signal was used in the majority of the domains BCI (n = 16), signal processing (n = 29), and sleep (n = 12). As expected, The classification of the extracted features from the registered EOG signals showed the following results: blinks (33%), raw signal (30%), artifact removal (27%), polysomnography (20%), saccades (7%) and fixation (3%) (Figure 5D). The analysis of blinks movements was particularly present in BCI (n = 17) and algorithms (n = 21) studies. The artifact removal was majority used in algorithms (n = 24) and equally used in BCI and sleep (n = 6). The raw signal was analyzed in BCI, (n = 11), algorithms (n = 16) and sleep (n = 13).

DISCUSSION
The objective of this work is to present a comprehensive summarized review of combined EEG-EOG studies and a taxonomy model of the last decade. Our taxonomy model presents an overview of classification by domains of application: sleep, BCI, signal processing, cognition, and driving. It is worth mentioning that these identified categories are significantly relevant to research in the exploration of human factors in the aeronautical industry and neural engineering. Figures 2-5 present the distribution of combined EEG-EOG papers of the taxonomy according to domains of application, while Figure 6 illustrates the different methodological approaches. The motivation behind this is to derive trends about how each feature type and domain of application have been investigated. This may assist researchers in exploring human factors and the identification of feature types in future combined EEG-EOG work. Here, we discuss the most relevant outcomes from our results section and review the involvement of the different findings highlighted above in aeronautics. We also provide recommendations for combined EEG-EOG studies to facilitate extension in the field. Finally, we present some limitations in our work and future perspectives.
Sleep and fatigue prediction are the main challenges explored in investigations on human factors. In this context, one of the outcomes of this review was the identification of EEG and EOG features that may help to detect unintentional sleep in airline pilots. Our results highlight relevant features that help to study aspects related to the sleep of pilots, aiming to improve safety conditions. For example, typical markers of sleep are the disappearance of the alpha rhythm and the appearance of roving eye movements. Fatigue, irregular, and long working hours, and time zone crossings can change sleepwake cycles, alert levels, and affect pilot decisions during a flight. These issues cause excessive sleepiness, unintentional sleep, and increase the risk of accidents. Some investigations with airline pilots observed that this working environment typically exposes several sleep and health issues, particularly extreme sleepiness, unintentional naps, and fatigue (Petrie and Dawson, 1997;Wright and McGown, 2001). However, such studies are still incipient, and knowing the characteristics of sleep prediction is necessary in supporting safety policies and working practices. The tasks performed by pilots are complex and involve several functions, including cognitive, technical, and relationship skills. They involve concentration, the ability to work under pressure, adaptation to operational modifications, teamwork, the prediction of the consequences, the interpretation of signals, and quick decisions (Itani, 2009). When excessive sleepiness occurs, such skills may be impaired, thereby affecting flight safety. Ingre et al. (2014) have observed that unintentional sleep in pilots may compromise the safety of flights. Therefore, the ability to observe and predict fatigue provides a significant benefit in avoiding incidents and accidents. Fatigue and sleepiness decrease task-related activity in the frontal and parietal regions and also reduce activity in and connectivity with, the extrastriate visual cortex during tasks that require visuospatial attention (Chee, 2010). These neural changes affect the behavior of pilots, for example, they might miss specific visual or auditory stimulus. This disturbance alternates the top-down allocation of attentional resources, such as the attentional orientation of a target. Sleep disorders during flights impair sustained attention because of the decreased activity of the dorsolateral prefrontal cortex and parietal sulcus (Kong and Soon, 2012). According to these neural alterations, our review suggests that combined EEG-EOG analysis may provide useful information in evaluating sleep and sleep-related disorders in pilots during flights. Future research could focus on improving EOG models and conduct a comprehensive evaluation of the content of various EOG results, studying the proportion of slow eye movement, rapid eye movements, and no eye movements, the time distribution, and mixtures of slow/fast/no eye movements. Figure 3 revealed the distribution of combined EEG-EOG works each year since 2010. Although the absolute number of combined EEG-EOG studies is relatively smaller compared to other single EEG (Bonanni et al., 2016) or eye movement applications (Hodgson et al., 2019), there is an important amount of interest in combined EEG-EOG studies. Unexpectedly, we were unable to draw conclusions about trends in the number of studies. We only observed that the first 3 months of 2020 associated with the years 2019 and 2018 alone account for 42% of total BCI publications. Due to the relatively small number of publications to date, it is too early to make assumptions about trends, but a possible explanation for this could be that BCI technology is rapidly gaining attention from scientists, engineers, clinicians, and the general public. The initial applications of BCI aimed to rehabilitate patients with neuromuscular diseases such as amyotrophic lateral sclerosis and injuries to the spinal cord. Here, we observed various patterns in BCI techniques, including EEG based spelling system (Lee et al., 2018), control of cursors , robotic arms (Zhang et al., 2019b), prostheses (Soekadar et al., 2015), wheelchairs (Huang et al., 2019), and other complex devices.
In aeronautics, passive BCIs have been successfully used to detect and characterize several operator mental states such as workload and fatigue (Zander et al., 2010;Khan and Hong, 2015;Roy and Frey, 2016). Functional Near-InfraRed Spectroscopy (fNIRS) connectivity based on passive BCI metrics has been explored to detect a pilot's engagement when undertaking automated and manual landing scenarios . Analysis confirmed that these two situations contrast, as manual landing led to significantly higher subjective NASA-TLX scores than automated landing. Designing a system capable of measuring continuous monitoring based on eyes and brain signal features or detecting an operator's degraded states would enhance both safety and performance. The achievements of BCI tend to be that it improves aeronautic performance in time-critical situations by decoding an operator's neural activity associated with the act. As brain activity precedes motor performance (Belkhiria et al., 2019), the decoded output in real time could ameliorate the operator's action. Our review showed that some EEG-EOG studies used online BCI to detect error-related potentials and reduce error-rate, improving overall performance. While these methods are promising, they have not been tested in aeronautics. An interesting study (Callan et al., 2016) used magnetoencephalography (MEG) and BCI to explore neuroadaptive automation to reduce a pilot's time response to a hazardous event by decoding their perceptual-motor intentions.
The BCI system succeeded in decoding motor intention faster than manual control in response to a change in attitude while ignoring ongoing motor and visual induced activity related to piloting the airplane. While this system used 400 channels, the authors expressed the possibility of making a dedicated real-time system working with mobile EEG. Such signal processing handling methods could separate artifacts from brain-related activity in flight even in an open cockpit biplane (Callan et al., 2016). In the future, research in combined EEG-EOG systems could use a BCI-decoder to distinguish between brain activity that responds to changes in the visual field and motor intention in a flight simulator or even in real aircraft.
Our analysis shows that the number of subjects included in each study varies across the different domains of application. It should also be highlighted that database availability is different from one domain to another. We noted that the most important datasets come from clinical investigations of conditions such as epilepsy, Parkinson's Disease, and sleep disorders. However, in other fields with more exploratory goals, studies rely on data registered in laboratory settings with a limited number of participants (e.g., 30 participants). Some studies explained that the reduced number of subjects is due to experimental conditions, time, and device limitations (Al-Hudhud et al., 2019). Further studies should use more strict inclusion and exclusion criteria for their datasets. This may generate a more robust statistical result. Figures 5, 6 show methodologies of data pre-processing, various extracted features from EEG and EOG signals, and performance metrics. The literature shows that performance metrics, including accuracies classification, determine computerbased analysis in various applications (Acharya et al., 2011). The objective is to construct algorithms that overcome recognized methods (Faust et al., 2010). Here, most of the selected studies used an offline or an online system. Commonly, the system involves three consecutive processing steps: (i) pre-processing (Kalayci and Özdamar, 1995;Rao and Derakhshani, 2005); (ii) feature extraction (Tyagi and Nehra, 2018); and (iii) classification (Wang et al., 2014b). All of the selected studies explored at least one parameter from the processing pipeline steps ( Table 1). Recent advances in signal processing analysis provide a powerful tool for modeling complex probability distributions by automatically discovering intermediate abstractions from a huge amount of basic features. Deep machine learning and artificial intelligence have shown great promise in helping make sense of EEG signals due to their capacity to learn good feature representations from raw data . In that sense, the number of aeronautics publications applying these techniques to EEG signal processing has seen an exponential increase in recent years. The increasing interest in methodologies for processing EEG-EOG signal data (e.g., deep learning, machine learning, and artificial intelligence) in relation to human factors noticeably reflects an emergent interest in these kinds of approaches. Research in aeronautic sensors and signal processing systems (e.g., EEG and EOG) is exploring revolutionary improvements, potentially enhancing civil and military applications in fighter planes, helicopters, and the remote operation of drones. Yet another trend in sensor and signal processing involves blending artificial intelligence and machine learning into system designs and the incorporation of electrophysiological sensors and associated signal processing methods into equipment, such as aviation headsets, may enhance operational safety (Wilson et al., 2020).

LIMITATIONS AND CONCLUSIONS
The review presented state-of-the-art research on the characteristics and applications of both EEG and EOG signals in aeronautics. Our taxonomy and statistical analysis described a number of operational mechanisms, such as extracted features, pre-processing treatment, and performance metrics. We discussed how these methodologies could be adopted by researchers examining human factors and aeronautics. The effectiveness of combining EEG and EOG as a psycho-physiological tool is unequivocal. However, numerous challenges still need to be resolved. An exciting and unprecedented approach would be the assessment of both eye movement and brain activity during novel paradigms using dry electrodes that are integrated into existing control and communication peripherals. An equipped headset measuring real-time EEG and EOG may pose a great challenge in terms of applicability and generalizability to both commercial and scientific research into mobile EEG devices. This method would improve human-system interaction by making it possible to follow up on data about eye movements and use it to determine the psycho-physiological state of a person. Studies combining both EEG and EOG technologies and a review of the application of these in several fields, including in laboratory and real-world situations, are of particular value, as these technologies could be of interest in areas using both visual and auditory information simultaneously through headphones and gaze inputs, including in aeronautics, helicopters, teleoperation drones, naval systems, and control-command centers.
Despite the clear value of data collected and presented here, this review has some limitations. First, despite the use of a well-founded methodology to identify pertinent studies on the topic; the review did not cover all existing papers. It is also important to stress the error risk that could occur as a result of the inclusion methodology. Even though we examined 150 studies among a wider pool of 255 from Pubmed research, this is an exhaustive list nor should we undervalue the relevance of the studies that were not included. Second, in managing the length of the review, we limited our analysis to the main domains of applications. However, some studies could involve two or more application domains. Some topics could overlap, such as BCI and signal processing (Daly et al., 2015;Ivorra et al., 2018). We did consider categorizing another field of application related to clinical applications. However, this would have moved the focus away from aeronautics. Finally, as with any literature review, new articles are being published and new trends are being established and future studies should eventually be added to the analysis.

AUTHOR CONTRIBUTIONS
CB: conceptualization, methodology, data curation, and writingoriginal draft preparation. VP: conceptualization, methodology, writing, reviewing, and editing. Both authors contributed to the article and approved the submitted version.