Toward Predicting Motion Sickness Using Virtual Reality and a Moving Platform Assessing Brain, Muscles, and Heart Signals

Introduction

Postural control (PC) is a central nervous system (CNS) feedback control system that governs human upright stance and gives a platform for locomotion and task-driven behavior, as well as several autonomic responses. The PC system works on a subconscious level and is based on continuous CNS input from visual, vestibular, proprioceptive, and somatosensory receptors Massion (1994). The CNS then processes this information to direct (efferent signals) both somatic (muscular) and autonomic (blood pressure etc.) responses. The PC system can be disturbed in two ways: the first one is a disease disruption (lost function) at all levels, and the second is a physiological “overstimulation” (increased function), which gives rise to motion sickness (MS).

State of the Art

Motion sickness is experienced by those who passively travel and is more common in women and at a young age. Although there are great individual differences, sex and age are both predictors of MS and motion sickness susceptibility (MSS) in general populations, probably due to gene–environment interaction Golding (2006a).

In addition, MS and MSS also fluctuate across age, i.e., in general, humans from 2 years of age begin to feel motion sick during traveling, peaking at 13 years of age and declining postpubertal (Bos et al., 2007; Huppert et al., 2019).

One of the best-known manifestations of MS is seasickness Petersen (2012). Due to modern technology, humans have faced new MS situations such as spaceflights [space sickness Crampton (1990)] or when playing computer games, including the phenomenon of “cybersickness” in virtual reality (VR) environments LaViola (2000). MS is a polysymptomatic disorder, where the primary symptoms are nausea and vomiting, but sweating, facial pallor, increased salivation, drowsiness, and dizziness are also frequent Golding (2006a). There is varying susceptibility among the general population, but all those with a fully or partially functional vestibular system can experience

MS. Females report higher incidence in MS history (higher frequency and severity of symptoms) and are more susceptible to seasickness, simulator sickness, and visually induced MS than males of the same age (Flanagan et al., 2005; Lawther and Griffin, 1988; Turner, 1999).

Two main theories regarding the pathogenesis of MS exist. The “sensory conflict theory” (SCT) Reason (1978) states that MS is caused by conflict between visual, vestibular, and/or somatosensory inputs. In the case of passive travel, such as being a passenger in a car or on a ship, the physical motion perceived by the vestibular system does not match the expected signals from the visual system. Sensory conflict can also occur due to a purely visual stimulus, as can be experienced by people during VR simulations who may perceive a visual movement, but vestibular signals do not match this. Recent studies report possible “sensory conflict neurons” in the brainstem and cerebellum (Oman and Cullen, 2014; Cohen et al., 2019) and also brain networks that mediate nausea and vomiting Yates et al. (2014b), which appear to further support the sensory conflict theory. A second theory of pathogenesis in MS is the “postural control theory” Riccio and Stoffregen (1991). It states that prolonged postural instability precedes the subjective symptoms of MS, i.e., that MS is directly brought on by an inability to control the posture during motion rather than a detection of any sensory conflict. The ability to remain bipedal/upright is crucial to human survival and MS appears to be closely linked to postural instability; studies have shown that greater postural instability or increased body sway correlates with greater MSS (Cobb, 1999; Tal et al., 2010).

Regardless of the underlying pathophysiology, CNS adaptive signals as well as efferent signals involved in the corrective processes, preceding and during MS, are measurable via various methods. Some studies have looked into possible correlations between MS levels and physiological biosignals such as electroencephalography (EEG), electrogastrography (EGG), electro-oculography (EOG), skin conductivity, heart rate (HR), blood pressure, body temperature, and cerebral blood oxygen demand visualized in functional magnetic resonance imaging. Relationships between the levels of MS experienced by subjects have been demonstrated in various EEG, EGG, and eye movement studies. Koohestani et al. (2019) give an overview of objective biosignal measures in MS research. Objective kinematic measures such as center of pressure (COP) are also documented as having relationships to MS levels in the literature (Thurrell and Bronstein, 2002; Weech et al., 2018) as well as spectral characteristics of spontaneous sway, which have been measured as a possible objective measurement for a predictive MS parameter Laboissière et al. (2015).

Objective measures can be useful for tracking the onset of MS, as it may be possible to use biosignals to predict the likelihood of the subject experiencing MS. Therefore, a subject’s MSS can be linked to measurable physiological or kinematic parameters in some cases by correlating the objective measurements with a standardized subjective MSS test/experienced MS level test. This is crucial for all further genetic studies on MS.

Motion sickness susceptibility is generally assessed by means of a questionnaire: subjective reporting of experienced levels of typical MS symptoms during biosignal measurement is a method used extensively in recent experimental studies (Kennedy et al., 1993; Golding, 2016; Mazloumi Gavgani et al., 2018). Correlation of various biosignals and subjective reporting of MS levels is a task to which machine learning (ML) is actively contributing. EEG has been used as a technique to correlate biosignal measurements with MS levels in multiple subjects using ML for VR-related MS Li et al. (2020). Ko et al. (2011) used neural network ML algorithms to estimate patient’s MS level based on the EEG power spectra from possible stimulated brain areas. Li et al. (2019) also studied EEG, COP, and head and waist motion markers correlated to a subjective MS questionnaire using ML following visually induced MS. Wang et al. (2019) used postural difference measures pre- and post-visually induced MS calculated with a deep long short term memory model. These studies used visually induced MS exclusively for estimating physiological response in virtual environments. Hell and Argyriou (2018) also used ML to predict MS using a VR rollercoaster simulation tool and a neural network architecture predicting MS and the intensity of roller coasters in order to improve the gaming experience.

Scientific Goal and Proposed Experiment

In this paper, we present the results from the first study using a new seasickness measurement platform called BioVRSea. This system is a sailing simulator that records, in synchronized fashion, heart, muscle, and brain signals (Figure 1). The participants wear the VR goggles showing a rough sea scenario. The movement of the ship on the waves in the VR scenario is coupled to the moving platform and the frequency and amplitude of the VR wave motion is synchronized with the platform motion. Subjective and objective MS levels are assessed by a questionnaire while biosensors measure EEG, electromyography (EMG), and HR of the subject. The creation of a database allows the implementation of various statistical and ML algorithms with the aim of correlating the biometric results with new indexes that combine the various symptoms of MS, having as main novelty the EEG application and interpretation in association with VR and moving platform inducing MS, linked to other biosignals.

FIGURE 1

Figure 1. BioVRSea structure: the moving platform, shown in a photo with a subject on the left, is combined with a rough sea VR scenario and with EEG, EMG, and HR bio-signal acquisition.

Materials and Methods

The biosensors used in this study are 64-channel EEG, 2-channel EMG, and HR chest monitor.

This first study is based on data acquired from 28 subjects (age: 23.8 ± 1.2), 22 women and 6 men (ethic approval by the Icelandic Bioethics Commission – Number: VSN-20-101 – May 2020). Each participant is measured three times (except one subject who underwent only two protocols) using different protocols based on the amplitude and frequency of the simulated waves. From each protocol, we extract 19 parameters associated with HR, EEG, and EMG signals. Moreover, after every protocol, the subject answers the Motion Sickness Susceptibility Questionnaire (MSSQ) Golding (2006b) based on the self-evaluation of 13 different neurophysiological conditions. A total of 83 datasets [(28 × 3) - 1] constitute the final database of our study.

BioVRSea Measurement’s Protocols

The VR software (Virtualis, VR, France) dynamically visualizes a virtual environment as if the subject is out on the open sea on a little boat. A moving platform (Virtualis VR, France) mimics the waves according to the simulated environment (Figure 1). The operator can set frequency (between 0.5 and 3 Hz) and amplitude of the waves (from 0 to 2). During the simulation, we vary the amplitude of the platform movements from 0% up to 100%. The platform allows fast (tailored) movements in 0°−45°−90°−135°−180°−225°−270°−315°−360° (linear acc. not available) coupled to synchronized visual VR movements.

Three different protocols are implemented in this study (Figure 2):

FIGURE 2

Figure 2. The three acquisition protocols that each patient has been subjected to.

• 0 Hz, null wave amplitude. Sea simulation is not performed during this protocol. The subject remains in an upright position on the platform for 60 s, observing mountains surrounded by lights in a dark environment through the goggles.

• 1 Hz, and wave amplitude = 0.6. This sea simulation protocol is divided into four parts, 30 s each with different platform movement amplitudes: 25, 50, 75, and again 25%. Total time: 120 s.

• 3 Hz, and wave amplitude = 0.5. This sea simulation protocol is divided into four parts, 30 s each with different platform movement amplitudes: 25, 50, 75, and again 25%. Total time: 120 s.

The first protocol mentioned (0 Hz) is the non-movable (platform stable) pre-test (baseline) sampling where the subjects can relax. This is done before the other two protocols (1 and 3 Hz) where the subjects during the movements have to grab onto the protection bars that they have in front. The eyes must be opened during all the three protocols.

The selection of these frequencies was based on two main reasons. The first reason is to only act upon one of MS etiologic theory: multiple theories have been listed to explain MS, and the SCT is easily the leading perspective. Frequencies below 1 Hz are not considered because they might act upon the additional Postural Instability Theory, which is rooted in perception of lower <0.5 Hz frequencies Riccio and Stoffregen (1991). The second reason is to ensure that an easy scenario (1 Hz) is available to reduce the risk of falling, as well as a harder one (3 Hz) to ensure sufficient movement to trigger MS.

Data Acquisition

During each protocol, heart, muscle, and brain data are acquired using the following technologies:

• HR is measured using a heart chest sensor (Polar Electro, Kempele, Finland, sampling frequency of 1,000 Hz).

• Muscle electrical activities from the lower limbs are acquired using two wireless EMG sensors (sampling frequency of 1,600 Hz) placed on the gastrocnemius of each leg (Kiso ehf, Reykjavik, Iceland).

• Brain electrical activity is measured using a 64-channel dry electrode cap (sampling frequency of 500 Hz) from AntNeuro, Hengelo, Netherlands.

Feature Extraction

Electromyography data processing was performed using Matlab_2020b (MathWorks Inc., Natick, Massachusetts, United States). The EMG signal was filtered with a 50th-order FIR bandpass filter with cutoff frequencies at 40 and 500 Hz. A fast Fourier transform was then used to obtain the frequency spectrum. The relative power spectral density (PSD) was then calculated for five frequency bands, equally distributed from 40 to 500 Hz, resulting in five parameters per leg. Finally, the integral of the rectified EMG signal for each leg was calculated and divided by the sample size, resulting in one parameter per leg. A total of 12 EMG-related features is thus computed.

The EEG was recorded using a 64-dry electrode channel system with an EOG electrode placed below the right eye and a ground electrode placed on the left side of the neck. Data pre-processing and analysis were performed with Brainstorm Tadel et al. (2011) and Matlab_2020b.

The data were re-referenced using the common average reference. A high-pass and a low-pass filter were set, respectively, from 0.1 and 40 Hz. Bad channels were manually removed when EEG voltage was higher than 300 μV. If more than 20% of the channels showed too much noise or incorrect signal, the whole trial was rejected. The signals were digitized in segments of 30 s, within 1-Hz and 3-Hz protocols. DC offset correction was performed, and baseline correction was applied using the 0-Hz segment. Channels marked as bad were removed and interpolated. Individual trials were visually inspected and rejected when indicative of excessive muscle activity, eye movements, or other artifacts.

The PSD was computed for each epoch with Welch’s method, with the following frequency bands: delta (0.5–4 Hz), theta (4–8 Hz), alpha (8–13 Hz), beta (13–35 Hz), and low gamma (LG) (35–40 Hz). The relative power of each band was then computed and averaged across all channels, obtaining a total of five EEG-related features.

Finally, from the HR signal, we calculate two features: HR average and standard deviation.

This results in a total of 19 biometric features for each acquisition protocol (Table 1).

TABLE 1

Table 1. Description of the 19 biometric parameters that compose the database.

Table 2 shows the objective physiological measurement differences for all the subjects between the first static protocol and the other two, the light one (1 Hz) in green and the hard one (3 Hz) in red. The arrows show how the values of the single EEG, EMG, and HR data rise or fall during the protocols. For example, it is possible to notice how the EMG values for both legs at low frequencies increase for the most patients, while they decrease at high frequencies. On the opposite, the EEG values do not follow such a regular trend.

TABLE 2

Table 2. Difference of the objective brain, muscle, and health bio measurements between the first static protocol and the light (1 Hz – green) and the hard one (3 Hz – red) for all the patients (the one that did not perform the 3-Hz protocol is not included).

Motion Sickness Questionnaire and Relative Indexes

At the end of every protocol the subjects were asked to fill out a questionnaire regarding their MS symptoms. The questionnaire is based on the MSSQ proposed by Golding (2006b). The subjects must give a score between zero and two for 13 typical MS symptoms: general discomfort, dizziness and vertigo, stomach awareness, sweating, nausea, salivation, burping, headache, fullness of head, blurred vision, fatigue, eye strain, and difficulty focusing.

We define a total of eight binary indexes considering the MSSQ answers.

General discomfort (I_GenDis, 1st) and Dizziness and Vertigo (I_Dizz, 2nd) are considered as independent and individual indexes. Stomach awareness, sweating, nausea, salivation, and burping are considered together as stomach-related to create the Stomach-related Index (I_Stom, 3rd). Headache, fullness of head, and blurred vision together produce the Head Index (I_Head, 4th) while fatigue, eye strain, and difficulty focusing contribute to the Fatigue Index (I_Fatig, 5th). I_Stom, I_Fatig, and I_Head are computed as binary indexes following these steps: first, we compute the average from the individual responses of each index; second, we calculate the maximum among the averages; and third, we divide the cohort into two groups (below and above 1/3 of the maximum). For I_GenDis and I_Dizz, we apply only steps 2 and 3 using the direct response instead of the average.

Moreover, we established two more indexes, Physiological/Vegetative Index (I_PV, 6th) and Neurological/Muscle Strain Index (I_NM, 7th). I_PV is based on the previously outlined steps from the responses from sweating, salivation, nausea, burping, stomach awareness, and general discomfort conditions. Similarly, the I_NM is based on fatigue, eye strain, difficulty focusing, headache, fullness of head, blurred vision, and again general discomfort conditions.

The last index is called Motion Sickness Index (I_MS, 8th), and it is based on the weighted sum SumMS of all the MSSQ answers (Eq. 1) and steps 2 and 3.

$$SumMS=(0,2\cdot GenDisc+0,2\cdot Dizz\&Vert+0,2\cdot$$

$$\sum (StomAwe,Nausea,Sweat,Saliv,Burp)+0,2\cdot$$

$$\sum (Fatigue,EyeStr,DiffFocus)+0,2\cdot$$

$$\sum (Headache,FullHead,BlurrVis))(1)$$

In Table 3, it is possible to see the percentage of MSSQ answers and indexes for the entire cohort. It is possible to identify general discomfort, sweating, nausea, and vertigo as the most significant indexes with over 20% of responses being 2, which is the highest possible value. Salivation and burping, conversely, are the least significant with a percentage lower than 5% providing a response of value 2.

TABLE 3

Table 3. Percentage of the MSSQ answers for each symptom, and percentage of zeros and ones for the eight computed indexes.

Table 4, similarly to Table 2, shows the increase or decrease of the value of the subjective given answer to the questionnaire using the colored arrows. It is possible to see that some patients, like numbers 10, 11, 14, and 22, have an increase of the symptom for both the 1-Hz and the 3-Hz protocols while others do not show any significant difference. Subject number 16 shows an increase only with the 3-Hz protocol, confirming the strong influence of the wave frequency on the body.

TABLE 4

Table 4. Difference of the subjective MS symptoms between the first static protocol and the light (1 Hz – green) and the hard (3 Hz – red) for all the patients (the one that did not perform the 3-Hz protocol is not included).

Statistical Analysis

All the parameters extracted from EEG, EMG, and HR underwent a non-parametric statistical univariate explorative analysis in order to understand whether there was a statistically different grouping by I_GenDis, I_Dizz, I_Stom, I_Head, I_Fatig, I_PV, I_NM, and I_Ms. All the indexes underwent univariate statistical analysis through the Mann–Whitney test.

ML Tool and Algorithms

The ML analysis was performed by using KNIME Analytics Platform (v. 4.2.0), which is a well-known platform in the field of biomedical studies, as it is considered the best choice for advanced users of ML Tougui et al. (2020) Several studies have been performed in clinical settings: for radiomics studies in oncology (Ricciardi et al., 2019; Romeo et al., 2020), for fetal monitoring (Improta et al., 2019; Ricciardi et al., 2020b), for investigating some relationships in ophthalmology (D’Addio et al., 2019; Improta et al., 2020), and in cardiology Ricciardi et al. (2020a).

The following algorithms were implemented through the platform: Random Forests (RF), Gradient Boosting tree (GB), Ada-Boosting of decision tree (ADA-B), Support Vector Machine (SVM), K Nearest Neighbor (KNN), and Multilayer Perceptron (MLP). The first three are based on a structure made up of nodes (starting point of the tree, which indicates an attribute), leaves (the question to be answered, which is the final label), and branches (connecting the nodes) and showed good results in different studies (Recenti et al., 2020; Ricciardi et al., 2020c). There are many criteria for splitting up the records: the gain ratio was used in this study Safavian and Landgrebe (1991). RF and GB are two empowered versions of the decision tree; they apply the ensemble learning methods of randomization, bagging, and boosting to make the weak learner stronger (Breiman, 2001; Friedman, 2001). SVM and KNN are two instance-based algorithms (Keller et al., 1985; Suykens and Vandewalle, 1999); the former assigns the class to the test data based on their distance from similar training data while the latter is capable of solving problems that have to do with overfitting, small datasets, and non-linear and/or high-dimensional data; it can also be used for both classification and regression. It aims to find the best hyperplane that divides the dataset into two classes. MLP consists of a form of neural network with an input layer, one or more hidden layers, and an output layer. The training is usually achieved by using the algorithm backpropagation of errors or some of its variants Riedmiller and Braun (1993).

The most employed evaluation metrics were used to assess the performance of the algorithms into the classifications tasks: accuracy, sensitivity, specificity, and Area Under the Curve Receiver Operating Characteristics (AUCROC) Hossin and Sulaiman (2015). All these metrics were computed using the K-Fold Cross Validation Kohavi (1995) with k = 10 using 10 different seeds. This means that the database is divided into 10 groups and each of them is used in turn as the test group while the other nine are used for the training of the model. Using 10 different seeds allows the creation of different 10-fold divisions, which allows a better exploration of the database and achieving the best results.

Results

Statistical Analysis Results

Table 5 shows the results of the statistical tests that assess the significance of the 19 parameters with the eight binary MSSQ indexes. Interestingly, only 4 out of 19 parameters never show a significance.

TABLE 5

Table 5. Significance of the 19 biometric parameters calculated with the univariate statistical analysis (Mann–Whitney test) for all the eight indexes.

The EEG Beta and LG showed significance only for the individuals suffering from headache, fullness of head, and blurred vision (I_Head), while no other significances were found for an EEG parameter.

The amplitude/area of EMG on both sides achieved a significance for all the conditions except for General Discomfort (I_GenDis). Similarly, excluding a few cases, the power spectrum of the EMG obtained a significance for almost all the conditions except in the band of 132–224 for the left side, which is never significant.

The HR Average was significant according to all indexes excluding I_Stom, I_Fatig, and I_Head while HR std showed statistical significance only according to I_GenDis.

The I_GenDis index was the index that showed the least number of significances for the analyzed parameters; only EMG – R 40–132 and 132–4 Hz and HR parameters achieved significant results according to this index. On the other hand, I_Head, I_Dizz, and I_NM were the indexes according to which the biometric parameters show the greatest number of significant results (respectively, 13 and 12).

Finally, 11 out of 19 parameters show a significant result according to the overall I_MS: 10 EMG-related features, 1 HR-related feature, and no EEG feature.

ML Results

The ML analysis focuses on the binary classification of physiological, neurological, and general MS conditions based on the MSSQ responses. We performed the classification of the following index previously defined:

(1) The Physiological/Vegetative Index (I_PV),

(2) The Neurological/Muscle Strain Index (I_NM),

(3) The MS Index (I_MS).

We assessed these conditions using six different algorithms, finding RF to yield the best results (Table 6).

TABLE 6

Table 6. Evaluation metrics after the classification ML analysis for I_PV, I_NM, and I_MS.

As regards I_PV and I_NM, the RF was the best algorithm for classifying both indexes; an accuracy of 75.9% with an AUCROC of 0.815 was achieved for I_PV while an accuracy of 79.5% with an AUCROC of 0.832 was obtained. The highest sensitivity (85.0%) was obtained by the ADA-B for the physiological index, while the highest sensitivity was achieved by RF (74.4%). As regards the neurological index, the best sensitivity (80.0%) was achieved by the ADA-B while the best specificity (83.3%) was obtained by RF. The MLP was the worst algorithm to perform both the classifications since it reached the lowest accuracy (respectively, 49.4 and 45.8%) while the lowest AUCROC was reached by KNN for the physiological index (0.573) and by SVM for the neurological index (0.590).

The feature importance analysis (Figure 3) shows that parameters extracted from EMG were the most important ones by far for the classification of both indexes. The first EEG-based features can be found in the 5th place in the ranking while the first HR-based features can be found after the 10th place. Moreover, it has to be highlighted that the top three features in Figure 3 for these indexes are all statistically significant also in the previous univariate analysis (Table 5).

FIGURE 3

Figure 3. Brain, muscle, and heart feature importance for I_PV, I_NM, and I_MS using Random Forest algorithm.

Concerning I_MS, the overall model for the indexes is good enough considering accuracies greater than 70.0%, AUCROC greater than 0.800, and the number of trials (equal to 83) that does not allow us to analyze a large dataset; indeed, a greater number of subjects would allow the improvement of the evaluation metrics of the models.

The feature importance analysis (Figure 3 and Table 7) highlighted novel results for I_MS: the seasickness can be strongly linked to features extracted from EMG (the top two were area and frequency analysis in the range 40–132) and HR-based (average and standard deviation were at the third and fourth place). On the other hand, another important and surprising result is the low importance of all the features extracted from the EEG, they were below the seventh place in the final ranking (this also for the other indexes except EEG-Beta which is quite relevant for I_PV and I_NM). This can be explained by the fact that a dry cap EEG was used for the acquisition. More noise was detected and led to a lower signal quality. Channels had to be rejected and could not be interpolated, leading to an averaged PSD on less channels. This can be one of the reasons of the low significance related to EEG features.

TABLE 7

Table 7. Feature importance (%) for IPV, INM, and IMS using Random Forest algorithm.

Discussion

Postural control is central in governing upright posture in humans. PC failure is dual, firstly pathological disruption leading to clinical difficulties where symptoms of vertigo, dizziness, imbalance, and falling are prominent Dakin and Bolton (2018). Secondly, an overstimulation of the PC system may precipitate a series of symptoms of discomfort known as MS Golding (2016). As in PC diseases, there are many objective measurements to be used in the diagnosis of these diseases. On the other hand, there are limited ways to objectively measure MS. Questionnaires are used to evaluate the incidence of subjective symptoms associated with MS, most often nausea, pallor, vomiting, sweating, headache, lightheadedness, and body discomfort Balk et al. (2013). There is an urgent need for objective measurement to evaluate MS, as it threatens human well-being when one is situated in a motion-rich environment. It is also critical to objectively distinguish people into MS-prone and non-MS-prone individuals. This is possible using questionnaires Golding (2006b), but having an objective way to discriminate these two groups is of great value when comes to genetic research.

In this study, we use the BioVRSea research setup and focus on EEG, EMG, and HR bio-signals associated with subjective MS symptoms. Our EEG-coupled results show significant difference in brain neural networking in individuals indicating subjective symptoms of headache, fullness of the head, and blurred vision (I_Head). In an earlier study, we showed that open eyes trials reflect a greater number of significant differences in EEG absolute spectral power across all bands during both adaptation and habituation. This suggests that following both acute and prolonged proprioceptive perturbation, cortical activity may be up-regulated with the availability of visual feedback Barollo et al. (2020). These results generally support our prior hypothesis that the visual recognition of instability may play a critical role in governing cortical processes requisite for PC Edmunds et al. (2019). These results underline the importance of visual information in PC and simultaneously open up the VR afferent link in PC perturbations. Being able to couple these subjective symptoms, i.e., headache, fullness of the head, and blurred vision, to objective intracranial activity is crucial in clinical context and opens up ways for VR-coupled biosignal evaluation of PC pathologies Maire et al. (2017). This is in keeping with many other studies performed on motion and CNS triggers of head-related symptoms. Jang et al. (2020) identified that the alpha band was linked to VR sickness, with a decrease of the absolute power during the experiment, followed by an increase during the recovery, highlighting a negative correlation with the MSSQ score. Kim et al. (2005) detected that in the case of cybersickness, the severity of the symptoms was positively correlated with the delta wave, and negatively with the beta waves. It is interesting to see that, in our study, despite the low significance of EEG regarding the different indexes chosen, the power associated to the beta band is the parameter presenting the most importance in I_PV and I_NM. This corroborates the fact that beta band is related to MS symptoms and is a feature that should be investigated in MS studies. Our results do not enable to drawing of hypotheses regarding the other power bands.

Our observations have indicated that definite vection does not necessarily result in visually induced MS (you can have very compelling vection but no visually induced MS), but at the same time, most participants who get sick also report vection. Our intentions are to verify the relationship between MS and visually induced MS, although there are some participants that get sick on the platform but never experience MS in the real world. They probably have not experienced enough rough waters and therefore we do not expect false positive/negative.

To be able to evaluate the relationship between BioVRSea biosignals and subjective MS symptoms, the use of ML was necessary. This study clearly shows the benefit of ML; indeed, it allowed us to achieve two aims: first, the possibility to model several biometric parameters extracted from three types of signals (EEG, EMG, and HR) in order to be able to classify/distinguish patients suffering from seasickness according to these features; second, the feature importance analysis allowed us to further confirm the statistical results by ranking the features according to their contribution to the classification task. Moreover, as regards the ML models, the RF was the most reliable among all the implemented ones (Table 6).

The amplitude of the EMG signals in both legs showed significant difference regarding all conditions except general discomfort (I_GenDis). That indicates that almost all subjective symptoms of MS showed correlation with changes in EMG. This is in context with the fact that all human efforts initiated to prevent falls, i.e., acute or long-term vertigo and dizziness, are mediated through postural stabilizing muscles Perrin et al. (2018). Some studies used EMG measurements to analyze the behavior related to MS, with sensors placed on the abdominal muscles Shafeie et al. (2013) and EMG combined with cervical vestibular myogenic potential to study the effect of scopolamine for the seasickness treatment Tal et al. (2016). As far as we know, no study found significance related to EMG in the lower limbs to quantify MS. This is an important piece of information regarding our BioVRSea research setup and is a promising single tool to objectively extract MS sufferers. On the other hand, this is not surprising as the prime effector in PC is aimed at muscles maintaining the upright posture and simultaneously avoiding falls. Our BioVRSea research setup might answer several clinical questions related to strategies used to prevent falls in patients with PC pathologies.

The HR parameters were significantly associated with the symptom of General Discomfort (I_GenDis). The General Discomfort symptom is general in its nature and does not specifically point to MS. On the other hand, the triggered MS discomfort relates to an escalated sense of generalized panic in severe MS conditions, which is well capable of creating extreme cardiovascular deviations Yates et al. (2014a). This is expected as HR is probably the best-known biosignal associated with numerous physical as well as pathological conditions, particularly of a PC nature.

These results can be used to have quite a whole vision of the body reaction to induced MS. This total vision can be used to help the pathological patients and the people that are more prone to MS planning an eventual rehabilitative therapy. Future ideas are to use more physiological measurements like blood oxygenation, skin sweating, and force used on the legs for the equilibrium. All these actual and future body parameters coupled with the BioVRSea system and ML are of value in further evaluation of PC disruptions, which are probably the most disturbing and costly health conditions affecting humans.

Limitations

Of course, the study has some limitations. The first is the small number of subjects that limits also the possibility of obtaining higher evaluation metrics in the ML analysis. The second is the type of population that has been analyzed in this research because it was limited regarding age and health status; all the subjects were young and healthy. Further studies could increase the number of subjects, which would allow improvements in the performance of ML and include in the population more diverse subjects. We use dry electrodes for the EEG acquisition resulting in high noise signal, which we believe has limited the value of the associated EEG parameters in both statistical significance and ML. The use of a wet cap EEG for further acquisitions is a potential improvement.

Of course, the VR view that the investigated individual visualizes standing on a virtual small vessel is not a true scenario of working environment at sea but is nevertheless capable of creating MS sensation at least in experienced sailors (verbal statements after being on platform).

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by the Icelandic Bioethics Committee – Number: VSN-20-101. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

MR, CR, RA, HP, and PG wrote the manuscript. MR and CR conceived the acquisition protocols and conducted the acquisitions with HS, SA, and GK. MR and CR conceived and computed the eight indexes with the support of PG. RA and GK performed the EEG analysis. HS, SA, and VB performed the EMG analysis. CR performed the machine learning analysis. IP performed the univariate statistical analysis. MR designed the tables and the figures and formatted the manuscript; Figure 1 is created by VB and MR. DJ reviewed and made additions to the manuscript. HP contributed to the medical knowledge. HP and PG conceived the original idea of quantifying motion sickness using bionsensors and virtual reality. PG coordinated the work. All authors contributed to the article and approved the submitted version.

Funding

This study was funded by Landspitali University Hospital, Reykjavik (Number: A-2020-054).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors would like to thank the students of the course “Medical Technology and Bio-signals 2020” of the BA studies in Biomedical Engineering in Reykjavik University who contributed to the acquisition of the database and inspiring the present work. The authors would also like to thank Samherji hf. Akureyri Iceland for support and Virtualis VR, France and his president Frank Assaban for the precious collaboration.

Abbreviations

ADA-B, ada-boosting; AUCROC, area under the curve receiver operating characteristics; CNS, central nervous system; EEG, electroencephalography; EGG, electrogastrography; EMG, electromyography; EOG, electro-oculography; GB, gradient boosting tree; HR, heart rate; I_Dizz, dizziness and vertigo index; I_Fatig, fatigue index; I_GenDis, general discomfort index; I_Head, head index; I_MS, motion sickness index; I_NM, neurological/muscle strain index; I_PV, physiological/vegetative index; I_Stom, stomach-related index; KNN, K nearest neighbor; LG, low gamma; ML, machine learning; MLP, multilayer perceptron; MS, motion sickness; MSS, motion sickness susceptibility; MSSQ, motion sickness susceptibility questionnaire; PC, postural control; PSD, power spectral density; RF, random forests; SCT, sensory conflict theory; SVM, support vector machine; VR, virtual reality.

References

Balk, S. A., Bertola, M. A., and Inman, V. W. (2013). “Simulator sickness questionnaire: twenty years later,” in Proceedings of the 7th International Driving Symposium on Human Factors in Driver Assessment, Training, and Vehicle Design: Driving Assessment, New York, NY, 257–263.

Google Scholar

Barollo, F., Frioriksdottir, R., Edmunds, K. J., Karlsson, G. H., Svansson, H. A., Hassan, M., et al. (2020). Postural control adaptation and habituation during vibratory proprioceptive stimulation: an HD-EEG investigation of cortical recruitment and kinematics. IEEE Trans. Neural. Syst. Rehabil. Eng. 28, 1381–1388. doi: 10.1109/TNSRE.2020.2988585

PubMed Abstract | CrossRef Full Text | Google Scholar

Bos, J. E., Damala, D., Lewis, C., Ganguly, A., and Turan, O. (2007). Susceptibility to seasickness. Ergonomics 50, 890–901. doi: 10.1080/00140130701245512

PubMed Abstract | CrossRef Full Text | Google Scholar

Breiman, L. (2001). Random forests. Mach. Learn. 45, 5–32.

Google Scholar

Cobb, S. V. G. (1999). Measurement of postural stability before and after immersion in a virtual environment. Appl. Ergon. 30, 47–57. doi: 10.1016/s0003-6870(98)00038-6

CrossRef Full Text | Google Scholar

Cohen, B., Dai, M., Yakushin, S. B., and Cho, C. (2019). The neural basis of motion sickness. J. Neurophysiol. 121, 973–982. doi: 10.1152/jn.00674.2018

PubMed Abstract | CrossRef Full Text | Google Scholar

Crampton, G. H. (1990). Motion and Space Sickness. Boca Raton, FL: CRC Press.

Google Scholar

D’Addio, G., Ricciardi, C., Improta, G., Bifulco, P., and Cesarelli, M. (2019). “Feasibility of machine learning in predicting features related to congenital nystagmus,” in XV Mediterranean Conference on Medical and Biological Engineering and Computing – MEDICON 2019. MEDICON 2019. IFMBE Proceedings, Vol. 76, eds J. Henriques, N. Neves, and P. de Carvalho (Cham: Springer).

Google Scholar

Dakin, C. J., and Bolton, D. A. (2018). Forecast or fall: prediction’s importance to postural control. Front. Neurol. 9:924. doi: 10.3389/fneur.2018.00924

PubMed Abstract | CrossRef Full Text | Google Scholar

Edmunds, K. J., Petersen, H., Hassan, M., Yassine, S., Olivieri, A., Barollo, F., et al. (2019). Cortical recruitment and functional dynamics in postural control adaptation and habituation during vibratory proprioceptive stimulation. J. Neural Eng. 16:026037. doi: 10.1088/1741-2552/ab0678

PubMed Abstract | CrossRef Full Text | Google Scholar

Flanagan, M. B., May, J. G., and Dobie, T. G. (2005). Sex differences in tolerance to visually-induced motion sickness. Aviat. Space Environ. Med. 76, 642–646.

Google Scholar

Friedman, J. H. (2001). Greedy function approximation: a gradient boosting machine. Ann. Stat. 29:1189–1232.

Google Scholar

Golding, J. F. (2006a). Motion sickness susceptibility. Auton. Neurosci. 129, 67–76. doi: 10.1016/j.autneu.2006.07.019

PubMed Abstract | CrossRef Full Text | Google Scholar

Golding, J. F. (2006b). Predicting individual differences in motion sickness susceptibility by questionnaire. Pers. Individ. Diff. 41, 237–248. doi: 10.1016/j.paid.2006.01.012

CrossRef Full Text | Google Scholar

Golding, J. F. (2016). Motion Sickness. In Handbook of Clinical Neurology, Vol. 137. Amsterdam: Elsevier, 371–390.

Google Scholar

Hell, S., and Argyriou, V. (2018). “Machine learning architectures to predict motion sickness using a virtual reality rollercoaster simulation tool,” in Proceedings of the 2018 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR), (Piscataway, NJ: IEEE), 153–156.

Google Scholar

Hossin, M., and Sulaiman, M. N. (2015). A review on evaluation metrics for data classification evaluations. Int. J. Data Mining Knowl. Manag. Process 5:1.

Google Scholar

Huppert, D., Grill, E., and Brandt, T. (2019). Survey of motion sickness susceptibility in children and adolescents aged 3 months to 18 years. J. Neurol. 266, 65–73. doi: 10.1007/s00415-019-09333-w

PubMed Abstract | CrossRef Full Text | Google Scholar

Improta, G., Ricciardi, C., Amato, F., D’Addio, G., Cesarelli, M., and Romano, M. (2019). “Efficacy of machine learning in predicting the kind of delivery by cardiotocography,” in Proceedings of the Mediterranean Conference on Medical and Biological Engineering and Computing, (Cham: Springer), 793–799. doi: 10.1007/978-3-030-31635-8_95

CrossRef Full Text | Google Scholar

Improta, G., Ricciardi, C., Cesarelli, G., D’Addio, G., Bifulco, P., and Cesarelli, M. (2020). Machine learning models for the prediction of acuity and variability of eye-positioning using features extracted from oculography. Health Technol. 10, 961–968. doi: 10.1007/s12553-020-00449-y

CrossRef Full Text | Google Scholar

Jang, K. M., Shin Woo, Y., and Kyoon Lim, H. (2020). “Electrophysiological changes in the virtual reality sickness: EEG in the VR sickness,” in Proceedings of the 25th International Conference on 3D Web Technology, (Cham: Springer), 1–4.

Google Scholar

Keller, J. M., Gray, M. R., and Givens, J. A. (1985). A fuzzy k-nearest neighbor algorithm. IEEE Trans. Syst. Man Cybernet. 15, 580–585.

Google Scholar

Kennedy, R. S., Lane, N. E., Berbaum, K. S., and Lilienthal, M. G. (1993). Simulator sickness questionnaire: an enhanced method for quantifying simulator sickness. Int. J. Aviation Psychol. 3, 203–220. doi: 10.1207/s15327108ijap0303_3

CrossRef Full Text | Google Scholar

Kim, Y. Y., Kim, H. J., Kim, E. N., Ko, H. D., and Kim, H. T. (2005). Characteristic changes in the physiological components of cybersickness. Psychophysiology 42, 616–625.

Google Scholar

Ko, L. W., Wei, C. S., Jung, T. P., and Lin, C. T. (2011). “Estimating the level of motion sickness based on EEG spectra,” in Proceedings of the International Conference on Foundations of Augmented Cognition, (Berlin: Springer), 169–176. doi: 10.1007/978-3-642-21852-1_21

CrossRef Full Text | Google Scholar

Kohavi, R. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. Ijcai 14, 1137–1145.

Google Scholar

Koohestani, A., Nahavandi, D., Asadi, H., Kebria, P. M., Khosravi, A., Alizadehsani, R., et al. (2019). A knowledge discovery in motion sickness: a comprehensive literature review. IEEE access 7, 85755–85770. doi: 10.1109/access.2019.2922993

CrossRef Full Text | Google Scholar

Laboissière, R., Letievant, J. C., Ionescu, E., Barraud, P. A., Mazzuca, M., and Cian, C. (2015). Relationship between spectral characteristics of spontaneous postural sway and motion sickness susceptibility. PLoS One 10:e0144466. doi: 10.1371/journal.pone.0144466

PubMed Abstract | CrossRef Full Text | Google Scholar

LaViola, J. J. Jr. (2000). A discussion of cybersickness in virtual environments. ACM Sigchi Bull. 32, 47–56. doi: 10.1145/333329.333344

CrossRef Full Text | Google Scholar

Lawther, A., and Griffin, M. J. (1988). A survey of the occurrence of motion sickness amongst passengers at sea. Aviat. Space Environ. Med. 59, 399–406.

Google Scholar

Li, X., Zhu, C., Xu, C., Zhu, J., Li, Y., and Wu, S. (2020). VR motion sickness recognition by using EEG rhythm energy ratio based on wavelet packet transform. Comput. Methods Programs 188:105266. doi: 10.1016/j.cmpb.2019.105266

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, Y., Liu, A., and Ding, L. (2019). Machine learning assessment of visually induced motion sickness levels based on multiple biosignals. Biomed. Signal Process. Control 49, 202–211. doi: 10.1016/j.bspc.2018.12.007

CrossRef Full Text | Google Scholar

Maire, R., Mallinson, A., Ceyte, H., Caudron, S., Van Nechel, C., Bisdorff, A., et al. (2017). Discussion about visual dependence in balance control: european society for clinical evaluation of balance disorders. J. Int. Adv. Otol. 13, 404–406. doi: 10.5152/iao.2017.4344

PubMed Abstract | CrossRef Full Text | Google Scholar

Massion, J. (1994). Postural control system. Curr. Opin. Neurobiol. 4, 877–887.

Google Scholar

Mazloumi Gavgani, A., Walker, F. R., Hodgson, D. M., and Nalivaiko, E. (2018). A comparative study of cybersickness during exposure to virtual reality and “classic” motion sickness: are they different? J. Appl. Physiol. 125, 1670–1680. doi: 10.1152/japplphysiol.00338.2018

PubMed Abstract | CrossRef Full Text | Google Scholar

Oman, C. M., and Cullen, K. E. (2014). Brainstem processing of vestibular sensory exafference: implications for motion sickness etiology. Exp. Brain Res. 232, 2483–2492. doi: 10.1007/s00221-014-3973-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Perrin, P., Mallinson, A., Van Nechel, C., Peultier-Celli, L., Petersen, H., Magnusson, M., et al. (2018). Defining clinical-posturographic and intra-posturographic discordances: what do these two concepts mean? J. Int. Adv. Otol. 14, 127–129. doi: 10.4324/9780429484803-8

CrossRef Full Text | Google Scholar

Petersen, H. (2012). Seasickness. Laeknabladid 98, 653–659.

Google Scholar

Reason, J. T. (1978). Motion sickness adaptation: a neural mismatch model. J. R. Soc. Med. 71, 819–829. doi: 10.1177/014107687807101109

CrossRef Full Text | Google Scholar

Recenti, M., Ricciardi, C., Monet, A., Jacob, D., Ramos, J., Gìslason, M., et al. (2020). Predicting body mass index and isometric leg strength using soft tissue distributions from computed tomography scans. Health Technol. 11, 239–249. doi: 10.1007/s12553-020-00498-3

CrossRef Full Text | Google Scholar

Ricciardi, C., Edmunds, K. J., Recenti, M., Sigurdsson, S., Gudnason, V., Carraro, U., et al. (2020a). Assessing cardiovascular risks from a mid-thigh CT image: a tree-based machine learning approach using radiodensitometric distributions. Sci. Rep. 10:2863.

Google Scholar

Ricciardi, C., Improta, G., Amato, F., Cesarelli, G., and Romano, M. (2020b). Classifying the type of delivery from cardiotocographic signals: a machine learning approach. Comput. Methods Programs 196:105712. doi: 10.1016/j.cmpb.2020.105712

PubMed Abstract | CrossRef Full Text | Google Scholar

Ricciardi, C., Jónsson, H. Jr., Jacob, D., Improta, G., Recenti, M., Gíslason, M. K., et al. (2020c). Improving prosthetic selection and predicting BMD from biometric measurements in patients receiving total hip arthroplasty. Diagnostics 10:815. doi: 10.3390/diagnostics10100815

PubMed Abstract | CrossRef Full Text | Google Scholar

Ricciardi, C., Renato, C., Giuseppe, C., Lorenzo, U., Giovanni, I., Domenico, S., et al. (2019). “Distinguishing Functional from Non-functional Pituitary Macroadenomas with a Machine Learning Analysis,” in Proceedings of the XV Mediterranean Conference on Medical and Biological Engineering and Computing – MEDICON 2019. MEDICON 2019. IFMBE Proceedings, Vol. 76, eds J. Henriques, N. Neves, and P. de Carvalho (Cham: Springer).

Google Scholar

Riccio, G. E., and Stoffregen, T. A. (1991). An ecological theory of motion sickness and postural instability. Ecol. Psychol. 3, 195–240. doi: 10.1207/s15326969eco0303_2

CrossRef Full Text | Google Scholar

Riedmiller, M., and Braun, H. (1993). “A direct adaptive method for faster backpropagation learning: the RPROP algorithm,” in Proceedings of the IEEE International Conference on Neural Networks, (Piscataway, NJ: IEEE), 586–591.

Google Scholar

Romeo, V., Cuocolo, R., Ricciardi, C., Ugga, L., Cocozza, S., Verde, F., et al. (2020). Prediction of tumor grade and nodal status in oropharyngeal and oral cavity squamous-cell carcinoma using a radiomic approach. Anticancer. Res. 40, 271–280. doi: 10.21873/anticanres.13949

PubMed Abstract | CrossRef Full Text | Google Scholar

Safavian, S. R., and Landgrebe, D. (1991). A survey of decision tree classifier methodology. IEEE Trans. Syst. Man Cybernet. 21, 660–674. doi: 10.1109/21.97458

CrossRef Full Text | Google Scholar

Shafeie, M., Zolfaghari, N., and McConville, K. M. V. (2013). “Abdominal muscle behavior and motion sickness during paired visual input with roll motion,” in Proceedings of the 2013 6th International Conference on Human System Interactions (HSI), (Piscataway, NJ: IEEE), 247–251.

Google Scholar

Suykens, J. A., and Vandewalle, J. (1999). Least squares support vector machine classifiers. Neural Process. Lett. 9, 293–300.

Google Scholar

Tadel, F., Baillet, S., Mosher, J. C., Pantazis, D., and Leahy, R. M. (2011). Brainstorm: a user-friendly application for MEG/EEG analysis. Comput. Intellig. Neurosci. 2011:879716.

Google Scholar

Tal, D., Bar, R., Nachum, Z., Gil, A., and Shupak, A. (2010). Postural dynamics and habituation to seasickness. Neurosci. Lett. 479, 134–137. doi: 10.1016/j.neulet.2010.05.044

PubMed Abstract | CrossRef Full Text | Google Scholar

Tal, D., Shemy, S., Kaminski-Graif, G., Wiener, G., and Hershkovitz, D. (2016). Vestibular evoked myogenic potentials and motion sickness medications. Clin. Neurophysiol. 127, 2350–2354. doi: 10.1016/j.clinph.2016.03.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Thurrell, A., and Bronstein, A. (2002). Vection increases the magnitude and accuracy of visually evoked postural responses. Exp. Brain Res. 147, 558–560. doi: 10.1007/s00221-002-1296-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Tougui, I., Jilbab, A., and El Mhamdi, J. (2020). Heart disease classification using data mining tools and machine learning techniques. Health Technol. 10, 1137–1144. doi: 10.1007/s12553-020-00438-1

CrossRef Full Text | Google Scholar

Turner, M. (1999). Motion sickness in public road transport: passenger behaviour and susceptibility. Ergonomics 42, 444–461. doi: 10.1080/001401399185586

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, Y., Chardonnet, J. R., and Merienne, F. (2019). “VR sickness prediction for navigation in immersive virtual environments using a deep long short term memory model,” in Proceedings of the 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), (Piscataway, NJ: IEEE), 1874–1881.

Google Scholar

Weech, S., Moon, J., and Troje, N. F. (2018). Influence of bone-conducted vibration on simulator sickness in virtual reality. PLoS One 13:e0194137. doi: 10.1371/journal.pone.0194137

PubMed Abstract | CrossRef Full Text | Google Scholar

Yates, B. J., Bolton, P. S., and Macefield, V. G. (2014a). Vestibulo-sympathetic responses. Compr. Physiol. 4, 851–887. doi: 10.1002/cphy.c130041

PubMed Abstract | CrossRef Full Text | Google Scholar

Yates, B. J., Catanzaro, M. F., Miller, D. J., and McCall, A. A. (2014b). Integration of vestibular and emetic gastrointestinal signals that produce nausea and vomiting: potential contributions to motion sickness. Exp. Brain Res. 232, 2455–2469. doi: 10.1007/s00221-014-3937-6

PubMed Abstract | CrossRef Full Text | Google Scholar