Pre-Exposure Cybersickness Assessment Within a Chronic Pain Population in Virtual Reality

Introduction

Virtual Reality (VR) is being used more often in medical and scientific research, for a variety of applications (Riva, 2005; Malloy and Milling, 2010; Valmaggia et al., 2016; Vaughan et al., 2016), and has been demonstrated as a powerful and flexible technology which is also affordable and relatively easy to use.

However, in spite of the rich potential of this technology for use in healthcare, it is common for persons to prematurely exit a VR experience because of symptoms associated with cybersickness (McCauley and Sharkey, 1992; Garrett et al., 2017). Cybersickness is defined as onset of nausea, oculomotor, and/or disorientation while experiencing virtual environments (Rebenitsch and Owen, 2016). This can cause problems for VR users as discomfort caused as a result of cybersickness prevents interaction longevity (Davis et al., 2015).

Symptoms of cybersickness can include nausea, headaches, dizziness, eyestrain, sweating, and disorientation (LaViola, 2000). It has been reported that as many as 80% of participants experience an increase in symptoms within 10 min of being exposed to VR (Kim et al., 2005; Cobb et al., 1999), and although these studies pre-date consumer VR, recent research indicates that this issue is still prevalent (Yildirim, 2020).

Although it’s clear that side effects from VR exposure exist, there is a lack of consistency in the literature regarding the precise definitions. Rebenitsch and Owen (2016) describe the symptoms of cybersickness produced in users of VR systems to “mimic motion sickness, but due to the absence of actual physical motion this affliction is considered a distinct condition referred to as cybersickness.” Cybersickness has been referred to also as visually induced motion sickness (VIMS), virtual simulation sickness, virtual reality-induced symptoms and effects, amongst other terms, as well as commonly being misinterpreted as simulator sickness. Cybersickness is distinctly separate from simulator sickness by the characteristics of its symptom profile, and the apparent disparity in symptom intensity (Stanney et al., 1997).

There is some discussion in the literature regarding other terms for these effects, for example ‘virtual reality-induced symptoms and effect’ (Cobb et al., 1999). However, for clarity, in this paper we will refer to the side effects of VR exposure as ‘Cybersickness,’ as this is the term most commonly used in the literature under discussion. We acknowledge that future work in this field should be considering updated terminology in order to describe the symptoms.

The safety of a device or intervention should be paramount when determining whether it is suitable for its intended audience, especially when developing novel applications for the purpose of medical interventions, rehabilitation, or training.

For clinical VR research, it is common to evaluate whether the VR system causes cybersickness symptoms, and thus determining whether it is safe to implement compared to an alternative intervention. For example, VR is being used more commonly within military environments where retention of information and task performance is vital, and thus information inhibition caused by cybersickness symptoms is an important consideration (Stanney et al., 2020).

Aside from the safety considerations, cybersickness may have implications for other factors in immersive systems. It has been suggested that individuals who report greater sickness symptoms in VR could be expected to report less presence (Witmer and Singer, 1998; Weech et al., 2019), which may have unwanted effects on desired outcomes. For example, when VR is used for the purposes of pain distraction, presence is considered a major contributor toward pain alleviation being achieved (Hoffman et al., 2004; Wiederhold et al., 2014). It is therefore important to test for factors, such as, cybersickness, which could potentially affect treatment outcomes. It is recommended that applications should be tested for cybersickness, and evaluated at the feasibility stage (Lubetzky et al., 2018; Davis, Nesbitt, and Nalivaiko, 2015).

Cybersickness is traditionally measured using self-report questionnaires, with the most commonly used being the Simulator Sickness Questionnaire (SSQ) (Kennedy et al., 1993). The SSQ was developed for use with simulators, and was adapted from Kennedy’s work in developing the Pensacola Motion Sickness Questionnaire (Kennedy et al., 1965), however it has been adopted widely for use with Virtual Environments (VE), as their symptom profiles and sickness characteristics are similar (Stanney et al., 1997).

A number of other self-assessment questionnaires have been devised for monitoring cybersickness, such as the Virtual Reality Symptom Questionnaire (Ames et al., 2005), and the Virtual Reality Sickness Questionnaire (Kim et al., 2018)–an adaption of Kennedy’s SSQ, have been used sporadically. A common criticism of these self-report questionnaires has been that they take too long to administer, therefore shorter single question measures have been used also (Nalivaiko et al., 2015; Keshavarz and Hecht, 2011). Aside from questionnaires, assessment of cybersickness by means of postural instability has been used more recently (Risi and Palmisano, 2019), as postural stability has been suggested to be a cause of the experience of cybersickness (as it has similarly been hypothesised to be a contributing factor in the cause of simulator sickness) (Stoffregen et al., 2000), although conflicting opinions exist (Dennison and D’Zmura, 2017).

There is a surprisingly limited discussion in the literature regarding what is considered a ‘normal’ score for cybersickness amongst healthy or non-healthy populations. In relation to the SSQ, amongst healthy participants, it is not usually necessary to perform sickness questionnaires pre-exposure as a baseline SSQ score could reasonably be assumed to be 0 (indicating no symptoms). However, participants from clinical populations may exhibit symptoms similar to cybersickness pre-exposure, and thus for these populations, the assumed zero baseline may be incorrect. For example, Bouchard et al. (2009) reported non-zero pre-exposure scores amongst participants with selected anxieties.

Kennedy et al. (1993) did suggest that pre-exposure screening of participants should be administered, but went onto recommend that individuals in a state other than their usual fitness (who score a non-zero pre-exposure score) should be eliminated from further participation, and thus only post-exposure assessment should be scored. However, if we only test on healthy participants, we can never test with clinical populations (such as people with chronic pain). Likewise, if we removed the participants who answered as anything other than ‘well,’ we would be removing the target population we are trying to study, which in turn would not facilitate clinical work being conducted. Amongst clinical populations such as chronic pain patients, it would certainly be counter-productive to eliminate individuals in this capacity as it has been suggested that confounders between cybersickness and medication exists (McCauley and Sharkey, 1992). Furthermore, understanding the pre-exposure state is important, as any pre-exposure symptoms could influence the interpretation of post-exposure scoring (Kennedy et al., 1993). Thus we suggest that it would be more informative to assess cybersickness pre-exposure, and observe changes which may occur between pre and post-exposure assessment. Without pre-exposure assessment, incorrect conclusions about the effect of a VR intervention could be formulated.

In lieu of pre-exposure assessment via SSQ, instruments such as, the Motion Sickness Susceptibility Questionnaire (Golding, 1998) may be administered to assess susceptibility to symptoms. However, susceptibility questioning alone does not reflect the current state of patients, but rather previous experiences within motion sickness-inducing situations, which is not indicative of determining the effect of a VR intervention.

To date, much of the pain research concerned with pain populations does not include any type of sickness assessment as part of their study protocols–including post-exposure sickness questionnaires. However, there are a few which do measure or discuss pre-exposure baseline or pre-exposure symptoms (e.g., Sarig Bahat et al., 2015; Wiederhold et al., 2014; Bouchard et al., 2009). Kennedy et al. (1993) suggested that sickness susceptibility questionnaires could be used as an alternative to pre-exposure baseline testing. However, in the majority of pain research in VR, neither this nor other pre-exposure baseline symptom testing measure is used, nor is the potential need for them discussed. Furthermore, susceptibility questionnaires do not elicit data regarding current symptoms, and therefore do not address the issue of a non-zero baseline score.

As the SSQ is currently the most widely use measure for cybersickness, we use this measure pre and post-exposure in order to facilitate comparisons with other work. The most common approach for assessing results of the SSQ is Kennedy et al.’s. weighted scoring (1993), although this has been criticised for scores being inflated by counting items multiple times in the total score calculation (Bouchard et al., 2007). Alternative scoring methodologies have been proposed, such as Bouchard et al. (2007) revised factor structure which proposes assessment with raw scores, rather than Kennedy’s weighted score calculation.

We suggest that it is important to understand whether some clinical populations may present pre-existing symptoms similar to symptoms of cybersickness (H1). Furthermore, it has been observed previously that cybersickness symptom scores may decrease rather than increase as the result of a VR intervention, (e.g., Bouchard et al. , 2009). A decrease in cybersickness-like symptoms in such a population may still give a post-exposure score greater than the zero baseline (H2). It could be hypothesised that a direct comparison of post-exposure SSQ scores between healthy and pain populations cybersickness scorings post-VR intervention may indicate that an intervention has made the pain population sicker than the healthy population. However, if pre-exposure (baseline) cybersickness scores were taken into account, then it may be that the any difference is due to a baseline difference, and not caused by the intervention itself (H3).

H1—The pain population will have significantly higher pre-exposure SSQ scores than the normal population assumed baseline.

H2—The pain population will have significantly higher post-exposure SSQ scores than the normal population assumed baseline of zero.

H3—The difference in SSQ scores from pre-exposure to post-exposure will be significantly less than the difference between the assumed baseline score and the post-exposure score.

Methods

The participants for this study were drawn from a population of Chronic Pain patients, as this group has been identified as one which may present pre-existing symptoms (McCauley and Sharkey, 1992). In order to reduce the burden on the patient population, this SSQ study was conducted alongside a study observing the effect of VR on experimentally induced pain in Chronic Pain patients, which describes the study methods and procedure summarised here in more depth.

Participants

Twelve participants aged 39–70 (M = 56 ± 9.36) (Table 1) were recruited from a United Kingdom pain support group and networks. All participants had been experiencing chronic pain (defined as a period lasting 3 months or greater). Participants also completed pre-study screening questionnaires to exclude any factors which would prevent them from participating in a VR study. Factors for exclusion included health issues which could prevent someone from using a visual display for an extended period of time.

TABLE 1

TABLE 1. Participant demographical information.

Design

A within-subjects, repeated measures study design was used. Pre-exposure SSQ was recorded before participants were randomised to receive either an active or passive VR distraction in a counterbalanced order. The passive intervention was part of the parallel study and is not considered further in this paper. In line with previous literature relating to SSQ scores, in this study we are only considering post-active SSQ results referred to hereon as post-exposure, unless explicitly stated otherwise.

Hardware and Software

The software interventions used were 1) Bananaland. An active intervention and a proprietary VR experience, in which the user traverses through a jungle environment with ambient music accompanying the visuals. 2) A passive intervention which consisted of grey lines on the screen. In this condition users could look around however no dynamic visual feedback was present. This was designed to be neutral and non-engaging.

Both interventions were presented using an Oculus Rift CV1 Head Mounted Display (HMD).

Procedure

Participants were asked to sign a consent, and were asked to confirm that no changes had occurred since registering that might be applicable to the studies medical exclusion criteria.

Before any VR intervention, participants completed a pre-exposure (baseline) SSQ. As per the protocol for the accompanying study, participants were induced with experimental pain by means of a pressure cuff inflated to and sustained at 200 mmHg, and applied to their non-dominant arm. This procedure was conducted in accordance with the Submaximal Tourniquet Effort Test (SMET) (Moore et al., 1979).

Participants were then exposed to the VR intervention for a maximum of 5 min, or until the participant asked to exit, which they were able to do at any point. After the completion of each VR session, participants completed a post-exposure SSQ. After each session, the participant was given time to rest before continuing.

The study was approved by the University of Portsmouth institutional review board.

Data Analysis

A Shapiro–Wilk test of normality indicated that pre-test sickness scoring was not normally distributed (p = 0.001). A Shapiro-wilk test of normality indicated that the post-test sickness scoring was not normally distributed (p = 0.014).

As this work is concerned with being comparable in the relevant literature, we will be calculating our results using Kennedy’s traditional approach, although will also apply Bouchard et al. (2007) revised factor structure and scoring where applicable.

A Wilcoxon signed-rank test was performed to assess whether our study sample had a significantly higher pre-exposure SSQ scores than the normal population assumed baseline. A Wilcoxon signed-rank test was performed to assess whether our study sample had a significantly higher post-exposure SSQ scores than the normal population assumed baseline. A Mann-Whitney U test was performed to assess whether the difference in SSQ scores from pre and post-exposure of our study sample will be significantly less than the difference the assumed baseline score and the post-exposure score of our study sample.

Results

Descriptive statistics for the pre-exposure and both post-exposure SSQ scores are shown in Table 2, and Figures 1 and 2.

TABLE 2

TABLE 2. Pre and post exposure SSQ scores for pain participants.

FIGURE 1

FIGURE 1. Weighted total severity SSQ scores pre-exposure and post-exposure of VR interventions.

FIGURE 2

FIGURE 2. Non-weighted total severity SSQ scores pre-exposure and post-exposure.

All scores referred to as weighted have been calculated using Kennedy et al. 1993) method. References to non-weighted scores are applying Bouchard et al. (2007) scoring approach.

(1) Pre-exposure SSQ scores compared to the assumed baseline (zero) of healthy participants.

A Wilcoxon signed-rank test indicated that the pre-exposure scores were significantly greater than the assumed baseline (zero) (z = 55, p = 0.005).

(2) Post-exposure SSQ scores compared to the assumed baseline (zero) of healthy participants.

A Wilcoxon signed-rank test indicated that a significantly higher post-exposure score than the assumed baseline (zero) was reported (z = 66, p = 0.003).

The mean post-exposure score obtained from our participants was 18.39, with high variability as the SD for the post-exposure was 16.19 (Table 1).

(3) Comparing post-exposure differences between actual and assumed baseline SSQ scores

A Mann–Whitney U test indicated that the difference between the pre and post-exposure SSQ scores was significantly less than the difference between the assumed baseline and post-exposure SSQ score (z = −2.297, p = 0.020) (Figure 3).

FIGURE 3

FIGURE 3. Post exposure differences between actual and assumed baseline SSQ scores.

A Mann-Whitney U test performed using Bouchard et al. (2007) non-weighting scoring approach indicated that the difference between the pre and post-exposure SSQ scores was significantly less than the difference between the assumed baseline and post-exposure SSQ score (z = −2.297, p = 0.020).

Individual items of the SSQ are categorised into subscales which are distinct symptom clusters (Kennedy et al., 1993). We therefore also scored each symptom cluster independently (Table 3).

TABLE 3

TABLE 3. SSQ symptom sub category scores.

There was an approximately 10 point increase in mean weighted SSQ scores in each of the sub categories, with the greatest variability being observed in the disorientation sub category. Whilst increases in mean SSQ scores were noted in all three categories, the symptom profile remained the same pre and post exposure.

Discussion

In this study, we theorised that using post-exposure SSQ scores in VR studies may lead to misinterpretation of results in some clinical populations such as those with chronic pain. To investigate this, we formulated three hypotheses regarding how pain participants cybersickness scores would compare to an assumed baseline scoring of the healthy population, and whether the pre-exposure state of pain participants differs from healthy participants assumed baseline.

We first hypothesised that a chronic pain population will have a higher than zero baseline SSQ score than the normal population. Our results indicate that a significant difference between the assumed (zero) baseline, and the measured pre-exposure SSQ scores exists, supporting the hypothesis. Much of the existing literature relies on post-exposure SSQ scores only, and few studies to date are concerned with determining the pre-exposure condition (Sarig Bahat et al., 2015; Wiederhold et al., 2014). Whilst this may be of limited importance if we are only interested in absolute SSQ score post-exposure, it becomes highly relevant when comparing the effect of VR exposure between different populations. We cannot assume that all population sub-groups will be starting studies at the same baseline, and this could affect the interpretation of post-exposure scores.

Secondly, we hypothesised that a chronic pain population will have greater post-exposure SSQ scores than the assumed (zero) baseline. Our results indicate that a significant difference does indeed exist. We expect with a VR intervention that participants will report some, potentially significant, cybersickness post-exposure, as it is unlikely to be zero. However, when assuming the comparison against a zero baseline, the result is not necessarily indicative of the VR interventions causation. This highlights potential disparities when using assumed baselines amongst groups such as chronic pain participants (who likely enter studies with elevated baselines). Furthermore, one participants cybersickness symptoms actually decreased between pre and post-VR intervention (Figure 1), supporting similar observations made by Bouchard et al. (2009). This result also supports our first hypothesis that assuming baselines can affect the interpretation of post-exposure scoring.

Thirdly, we hypothesised that the difference between SSQ scores of the pre and post-exposure would be significantly less than the difference between the assumed baseline and the post-exposure SSQ scores. Our results demonstrated that the difference in SSQ scores measured pre to post was significantly less than the difference in SSQ scores measured between the assumed baseline and post-exposure (Figure 3). If we aim to determine whether an intervention has caused cybersickness, by just collecting and observing post-exposure scores we may conclude that it does (when compared to zero). However, if we are able to look at our populations incoming SSQ score, it may be that the intervention has little or no negative effect by comparison. Furthermore, this result demonstrates how if these results were analysed without a known baseline and just the assumed (zero) baseline, that a false positive effect of the intervention on SSQ would have been reported. In the absence of pre-exposure or susceptibility testing (Golding, 1998), just post-exposure SSQ scoring could indicate that the pain population would become sicker than the healthy population, and could ultimately conclude that an application is unsuitable for the non-healthy population. However, if pre-exposure testing is conducted, it’s possible that the rate of change in both populations is comparable, and therefore the application is equally suitable for both.

This is of particular importance for pain populations, as the literature highlights how cybersickness may be negatively correlated with presence (Witmer and Singer, 1998; Weech et al.,, 2019), which is considered a key reason for VR’s efficacy of providing pain alleviation via distraction, as it’s suggested to be positively correlated with presence (Hoffman et al., 2004; Wiederhold et al., 2014).

While difference scoring alone is not generally considered a valid measure due to potentially poor reliability (Cronbach and Furby, 1970; Young et al., 2006), our result nevertheless demonstrates the importance of understanding pre-exposure symptoms of a clinical population when examining the effect of cybersickness When considering whether the symptoms of cybersickness become exacerbated in a VR application, basing our assumptions on purely on the post-exposure SSQ scores would be misleading.

The SSQ categorises symptoms into three distinct clusters which are Oculomotor, Nausea, and Disorientation (Kennedy et al., 1993). Spectral profiles of sickness exist to define cybersickness from the often misinterpreted simulator sickness (Stanney et al., 1997), with simulator sickness following the symptom profile of Oculomotor, Nausea, and Disorientation; in order of perceived symptom severity. Cybersickness however is suggested to follows the symptom profile of Disorientation, Nausea, and Oculomotor (Stanney et al., 1997) (Table 4). Although examining the differences between the SSQ subcategories was not a primary aim of this study, and is thus purely observational, we noted that the reported SSQ symptoms did not follow the symptom profiles which traditionally defines cybersickness uniquely (Figure 4). We observed that the symptom profile of participants was similar to that of simulator sickness (Table 4), rather than that of cybersickness. The literature has previously highlighted discrepancies in cybersickness/simulator sickness spectral profile definitions, suggesting that users affected with cybersickness have followed symptom profiles different to that reported by Stanney et al. (1997) and Gavgani et al. (2018). Although this cannot be generalised for the population as this is a small sample, this effect should be looked into further with a larger sample.

TABLE 4

TABLE 4. Severity of symptoms in spectral profiles.

FIGURE 4

FIGURE 4. SSQ total severity and symptom category scores.

This could be because rather than following symptom profiles devised to categorise cybersickness, pain participants are exhibiting symptoms associated with chronic pain symptom profiles. For example, at pre and post-exposure our participants scored highest in Oculomotor and Nausea (Table 3). Oculomotor elements include fatigue, headache, eye strain, and difficulty focusing, similar to some somatic symptoms associated with chronic pain which includes fatigue, unrefreshing sleep, and dyscognition (difficulty concentrating and thinking) (Wolfe et al., 2010; Crofford, 2015). Nausea elements include feeling nauseous and dizziness, which are also common side effects of opioid based medication, such as Hydrocodone and Fentanyl, commonly prescribed for the treatment of chronic pain (Benyamin et al., 2008). However, when observing the rate of increase between pre and post-exposure, the rate of which symptoms have increased is comparable to the traditionally proposed symptom severity of cybersickness (Figure 4). This would indicate that participants entered with symptoms of their illness, which is comparable to Simulator Sickness and pain symptom profiles, however their increase is representative of cybersickness. Future work could further investigate the symptom profiles of clinical sub-populations pre and post-exposure, which may give some indication as to whether post-exposure SSQ scores are more attributable to the VR intervention or the pre-existing clinical condition.

It’s also possible that the experimentally induced painful stimuli experienced during the parallel study may have caused an increase in the pain-related symptoms, which overlap with the SSQ symptom list. Further studies using a VR exposure for this group without the experimentally induced pain would control for this possible confounding factor. It has be argued that pre-exposure may influence post-exposure results (Kim et al., 2005; Young et al., 2006). In these previous studies they show a significant increase when given pre-exposure questionnaires, an effect which we did not observe. Future work could explore pre-exposure bias when administering the SSQ further.

Two participants reported a pre-exposure SSQ score comparable to the normal populations assumed baseline (zero), with the mean pre-exposure SSQ for the pain participants being 10.29. A high SD of 11.44 was reported, indicating that reported pre-exposure SSQ scores were highly variable (Table 1), which is representative of pain populations variability and individuality of symptom exhibition (Allen et al., 2009; Bartley et al., 2018). Of the 12 participants, 9 had a post-exposure score greater than 10, only 2 of these showed a substantial increase on pre-exposure scores, and all 9 had a non-zero pre-exposure score. However, 1 participant did show a decrease is symptoms as oppose to an increase (Figure 1).

Using Bouchard et al. (2007) scoring approach, a comparable Mann-Witney U test was performed which returned the same confidence interval as we observed when using Kennedy’s scoring. Similar differences were also observed when scoring the Nausea and Oculomotor sub-scales respectively (Figure 5). Therefore, although in other contexts Bouchard’s scoring may give rise to different conclusions, for the purposes of this study both Bouchard’s and Kennedy’s approaches lead to the same conclusions.

FIGURE 5

FIGURE 5. SSQ oculomotor and nausea non-weighted scores.

We would like to acknowledge also that although the focus of this work was to highlight that certain populations may not be as adversely affected as SSQ scores might indicate, it is possible that starting with a non-zero pre-score might mean that when participants do become sick because of VR, the rate which their sickness increases may be greater than someone who entered the study with a zero baseline score. Alternatively, entering with a non-zero baseline score could also provide greater resilience to symptoms when interacting within VR. Kruk,(1992) suggests that medication increases the susceptibility to simulator sickness. Although little work exists as to whether this same interaction exists for cybersickness, McCauley and Sharkey 1992) suggest that problems with cybersickness symptoms may be exacerbated by medication. Further work could be warranted to explore these points.

This study has highlighted that some sub-populations cannot be assumed to have a zero baseline. 10 out of 12 participants enrolled in this study entered with a non-zero baseline, confirming that assumed baselines should not be an indicator for informed research concerned with measuring cybersickness amongst non-healthy populations. and that considerations should be taken when evaluating the usability of VR systems or interventions for participants from different demographics.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by University of Portsmouth CCi. The participants provided their written informed consent to participate in this study.

Author Contributions

The study was designed by PB and WP. The data acquisition was conducted by PB. Formal analysis was conducted by PB with statistical advice provided by WP. Supervision for this study was completed by WP. The author who took lead on writing the manuscript was PB, and main feedback and edits were provided by WP. All authors contributed feedback to the final version of the manuscript.

Funding

This work was funded by the Faculty of Creative and Cultural Industries at the University of Portsmouth.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors would like to acknowledge the participants who took part in the study, with specific gratitude toward the Keeping Pace with Pain support group in Southampton, United Kingdom.

References

Allen, K. D., Coffman, C. J., Golightly, Y. M., Stechuchak, K. M., and Keefe, F. J. (2009). Daily Pain Variations Among Patients with Hand, Hip, and Knee Osteoarthritis. Osteoarthritis Cartilage 17, 1275–1282. doi:10.1016/j.joca.2009.03.021

PubMed Abstract | CrossRef Full Text | Google Scholar

Ames, S. L., Wolffsohn, J. S., and Mcbrien, N. A. (2005). The Development of a Symptom Questionnaire for Assessing Virtual Reality Viewing Using a Head-Mounted Display. Optom. Vis. Sci. 82, 168–176. doi:10.1097/01.OPX.0000156307.95086.6

PubMed Abstract | CrossRef Full Text | Google Scholar

Bartley, E. J., Robinson, M. E., and Staud, R. (2018). Pain and Fatigue Variability Patterns Distinguish Subgroups of Fibromyalgia Patients. The J. Pain 19, 372–381. doi:10.1016/j.jpain.2017.11.014

PubMed Abstract | CrossRef Full Text | Google Scholar

Benyamin, R., Trescot, A. M., Datta, S., Buenaventura, R., Adlaka, R., Sehgal, N., et al. (2008). Opioid Complications and Side Effects. Pain Physician. 19 (2 Suppl), S105–20. doi:10.1007/springerreference_120558

CrossRef Full Text | Google Scholar

Bouchard, S., Geneviève, R., and Patrice, R. (2007). Revising the Factor Structure of the Simulator Sickness Questionnaire. Annu. Rev. Cybetherapy Telemed. 5, 128–137. doi:10.31274/etd-180810-5050

CrossRef Full Text | Google Scholar

Bouchard, S., St-Jacques, J., Renaud, P., Brenda, K., and Wiederhold, (2009). Side Effects of Immersions in Virtual Reality for People Suffering from Anxiety Disorders. J. Cyber Ther. Rehabil. 2 (2), 127–137. doi:10.1037/10858-005

CrossRef Full Text | Google Scholar

Cobb, S. V. G., Nichols, S., Ramsey, A., and Wilson, J. R. (1999). Virtual Reality-Induced Symptoms and Effects (VRISE). Presence: Teleoperators Virtual Environ. 8, 169–186. doi:10.1162/105474699566152

CrossRef Full Text | Google Scholar

Crofford, L. J. (2015). Chronic Pain: Where the Body Meets the Brain. Trans. Am. Clin. Climatol. Assoc. 126, 167–83. doi:10.25250/thescbr.brk404

PubMed Abstract | CrossRef Full Text | Google Scholar

Cronbach, L. J., and Furby., L. (1970). How We Should Measure "change": Or Should We?. Psychol. Bull. 74, 68–80. doi:10.1037/h0029382

CrossRef Full Text | Google Scholar

Davis, S., Nesbitt, K., and Nalivaiko, E. (2015). “Comparing the Onset of Cybersickness Using the Oculus Rift and Two Virtual Roller Coasters,” in 11th Australasian Conference on Interactive Entertainment (IE 2015)

Google Scholar

Dennison, M. S., and D’Zmura, M. (2017). Cybersickness without the Wobble: Experimental Results Speak against Postural Instability Theory. Appl. Ergon. 58, 215–223. doi:10.1016/j.apergo.2016.06.014

PubMed Abstract | CrossRef Full Text | Google Scholar

Garrett, B., Taverner, T., and McDade, P. (2017). Virtual Reality as an Adjunct Home Therapy in Chronic Pain Management: An Exploratory Study. JMIR Med. Inform. 5, e11. doi:10.2196/medinform.7271

PubMed Abstract | CrossRef Full Text | Google Scholar

Gavgani, A. M., Walker, F. R., Hodgson, D. M., and Nalivaiko, E. (2018). “A Comparative Study of Cybersickness during Exposure to Virtual Reality and ‘Classic’ Motion Sickness: Are They Different?. J. Appl. Physiol. 125, 1670–1680. doi:10.1152/japplphysiol.00338.2018

CrossRef Full Text | Google Scholar

Golding, J. F. (1998). Motion Sickness Susceptibility Questionnaire Revised and its Relationship to Other Forms of Sickness. Brain Res. Bull. 47, 507–516. doi:10.1016/S0361-9230(98)00091-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Hoffman, H. G., Sharar, S. R., Coda, B., Everett, J. J., Ciol, M., Richards, T., et al. (2004). Manipulating Presence Influences the Magnitude of Virtual Reality Analgesia. Pain 111, 162–168. doi:10.1016/j.pain.2004.06.013

PubMed Abstract | CrossRef Full Text | Google Scholar

Kennedy, R. S., Tolhurst, G. C., and Graybiel, A. (1965). The Effects of Visual Deprivation on Adaptation to a Rotating Environment. London: Bureau of Medicine and Surgery. doi:10.1037/e430802004-001

CrossRef Full Text

Kennedy, R. S., Lane, N. E., Berbaum, K. S., and Lilienthal, M. G. (1993). Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness. Int. J. Aviation Psychol. 3, 203–220. doi:10.1207/s15327108ijap0303_3

CrossRef Full Text | Google Scholar

Keshavarz, B., and Hecht, H. (2011). Validating an Efficient Method to Quantify Motion Sickness. Hum. Factors 53, 415–426. doi:10.1177/0018720811403736

PubMed Abstract | CrossRef Full Text | Google Scholar

Kim, Y. Y., Kim, H. J., Kim, E. N., Ko, H. D., and Kim, H. T. (2005). Characteristic Changes in the Physiological Components of Cybersickness. Psychophysiology 42, 616–625. doi:10.1111/j.1469-8986.2005.00349.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Kim, H. K., Park, J., Choi, Y., and Choe, M. (2018). Virtual Reality Sickness Questionnaire (VRSQ): Motion Sickness Measurement Index in a Virtual Reality Environment. Appl. Ergon. 69, 66–73. doi:10.1016/j.apergo.2017.12.016

PubMed Abstract | CrossRef Full Text | Google Scholar

Kruk, R. V. (1992). “Simulator Sickness Experience in Simulators Equipped with Fiber Optic Helmet Mounted Display Systems,” in Flight Simulation Technologies Conference. doi:10.2514/6.1992-4135

CrossRef Full Text | Google Scholar

LaViola, J. J. (2000). A Discussion of Cybersickness in Virtual Environments. ACM SIGCHI Bull. 32, 47–56. doi:10.1145/333329.333344

CrossRef Full Text | Google Scholar

Lubetzky, A. V., Kary, E. E., Harel, D., Bryan, H., and Perlin, K. (2018). Feasibility and Reliability of a Virtual Reality Oculus Platform to Measure Sensory Integration for Postural Control in Young Adults. Physiother. Theor. Pract. 34, 935–950. doi:10.1080/09593985.2018.1431344

CrossRef Full Text | Google Scholar

Malloy, K. M., and Milling, L. S. (2010). The Effectiveness of Virtual Reality Distraction for Pain Reduction: A Systematic Review. Clin. Psychol. Rev. 30, 1011–1018. doi:10.1016/j.cpr.2010.07.001

PubMed Abstract | CrossRef Full Text | Google Scholar

McCauley, M. E., and Sharkey, T. J. (1992). “Cybersickness: Perception of Self-Motion in Virtual Environments,” in Presence: Teleoperators and Virtual Environments. Berlin: Springer. doi:10.1162/pres.1992.1.3.311

CrossRef Full Text | Google Scholar

Moore, P. A., Duncan, G. H., Scott, D. S., Gregg, J. M., and Ghia, J. N. (1979). The Submaximal Effort Tourniquet Test: Its Use in Evaluating Experimental and Chronic Pain. Pain 6, 375–382. doi:10.1016/0304-3959(79)90055-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Nalivaiko, E., Davis, S. L., Blackmore, K. L., Vakulin, A., and Nesbitt, K. V. (2015). Cybersickness Provoked by Head-Mounted Display Affects Cutaneous Vascular Tone, Heart Rate and Reaction Time. Physiol. Behav. 151, 583–590. doi:10.1016/j.physbeh.2015.08.043

PubMed Abstract | CrossRef Full Text | Google Scholar

Rebenitsch, L., and Owen, C. (2016). Review on Cybersickness in Applications and Visual Displays. Virtual Reality 20, 101–125. doi:10.1007/s10055-016-0285-9

CrossRef Full Text | Google Scholar

Risi, D., and Palmisano, S. (2019). Effects of Postural Stability, Active Control, Exposure Duration and Repeated Exposures on HMD Induced Cybersickness. Displays 60, 9–17. doi:10.1016/j.displa.2019.08.003

CrossRef Full Text | Google Scholar

Riva, G. (2005). Virtual Reality in Psychotherapy: Review. Cyberpsychology Behav. 8, 220–30. doi:10.1089/cpb.2005.8.220

CrossRef Full Text | Google Scholar

Sarig Bahat, H., Takasaki, H., Chen, X., Bet-Or, Y., and Julia Treleaven, J. (2015). Cervical Kinematic Training with and without Interactive VR Training for Chronic Neck Pain - a Randomized Clinical Trial. Man. Ther. 19, 372–381. doi:10.1016/j.math.2014.06.008

CrossRef Full Text | Google Scholar

Stanney, K. M., Kennedy, R. S., and Drexler, J. M. (1997). Cybersickness Is Not Simulator Sickness. Proc. Hum. Factors Ergon. Soc. 7, 112–184. doi:10.1177/107118139704100292

CrossRef Full Text | Google Scholar

Stanney, K., Lawson, B. D., Rokers, B., Dennison, M., Fidopiastis, C., Stoffregen, T., et al. (2020). Identifying Causes of and Solutions for Cybersickness in Immersive Technology: Reformulation of a Research and Development Agenda. Int. J. Human-Computer Interactions 36, 1783–1803. doi:10.1080/10447318.2020.1828535

CrossRef Full Text | Google Scholar

Stoffregen, T. A., Hettinger, L. J., Haas, M. W., Roe, M. M., and Smart, L. J. (2000). Postural Instability and Motion Sickness in a Fixed-Base Flight Simulator. Hum. Factors 42, 458–469. doi:10.1518/001872000779698097

PubMed Abstract | CrossRef Full Text | Google Scholar

Valmaggia, L. R., Latif, Leila., Kempton, M. J., and Rus-Calafell, M. (2016). Virtual Reality in the Psychological Treatment for Mental Health Problems: An Systematic Review of Recent Evidence. Psychiatry Res. 236, 189–195. doi:10.1016/j.psychres.2016.01.015

PubMed Abstract | CrossRef Full Text | Google Scholar

Vaughan, N, Dubey, V. N., Wainwright, T. W., and Middleton, R. G. (2016). A Review of Virtual Reality Based Training Simulators for Orthopaedic Surgery. Med. Eng. Phys. 38 (2), 59–71. doi:10.1016/j.medengphy.2015.11.021

PubMed Abstract | CrossRef Full Text | Google Scholar

Weech, S., Kenny, S., and Barnett-Cowan, M. (2019). Presence and Cybersickness in Virtual Reality Are Negatively Related: A Review. Front. Psychol. 10, 158. doi:10.3389/fpsyg.2019.00158

PubMed Abstract | CrossRef Full Text | Google Scholar

Wiederhold, B. K., Gao, K., Sulea, C., and Wiederhold, M. D. (2014). Virtual Reality as a Distraction Technique in Chronic Pain Patients. Cyberpsychology, Behav. Soc. Netw 17 (6), 346–352. doi:10.1089/cyber.2014.0207

CrossRef Full Text | Google Scholar

Witmer, B G., and Singer, M. J. (1998). “Measuring Presence in Virtual Environments: A Presence Questionnaire.” in Presence: Teleoperators and Virtual Environments. Berlin: Springer. doi:10.1162/105474698565686

CrossRef Full Text | Google Scholar

Wolfe, F., Clauw, D. J., Fitzcharles, M. A., Goldenberg, D. L., Katz, R. S., Mease, P., et al. (2010). The American College of Rheumatology Preliminary Diagnostic Criteria for Fibromyalgia and Measurement of Symptom Severity. Arthritis Care Res. 62 (5), 600–610. doi:10.1002/acr.20140

PubMed Abstract | CrossRef Full Text | Google Scholar

Yildirim, C. (2020). “Don’t Make Me Sick: Investigating the Incidence of Cybersickness in Commercial Virtual Reality Headsets. Virtual Reality 24 (2), 231–239. doi:10.1007/s10055-019-00401-0

CrossRef Full Text | Google Scholar

Young, S. D., Adelstein, B. D., and Ellis, S. R. (2006). “Demand Characteristics of a Questionnaire Used to Assess Motion Sickness in a Virtual Environment,” In Proceedings - IEEE Virtual Reality. doi:10.1109/VR.2006.44

CrossRef Full Text | Google Scholar