Reduction of cybersickness in head mounted displays use: A systematic review and taxonomy of current strategies

This literature review examines the existing research into cybersickness reduction with regards to head mounted display use. Cybersickness refers to a collection of negative symptoms sometimes experienced as the result of being immersed in a virtual environment, such as nausea, dizziness, or eye strain. These symptoms can prevent individuals from utilizing virtual reality (VR) technologies, so discovering new methods of reducing them is critical. Our objective in this literature review is to provide a better picture of what cybersickness reduction techniques exist, the quantity of research demonstrating their effectiveness, and the virtual scenes testing has taken place in. This will help to direct researches towards promising avenues, and illuminate gaps in the literature. Following the preferred reporting items for systematic reviews and meta-analyses statement, we obtained a batch of 1,055 papers through the use of software aids. We selected 88 papers that examine potential cybersickness reduction approaches. Our acceptance criteria required that papers examined malleable conditions that could be conceivably modified for everyday use, examined techniques in conjunction with head mounted displays, and compared cybersickness levels between two or more user conditions. These papers were sorted into categories based on their general approach to combating cybersickness, and labeled based on the presence of statistically significant results, the use of virtual vehicles, the level of visual realism, and the virtual scene contents used in evaluation of their effectiveness. In doing this we have created a snapshot of the literature to date so that researchers may better understand what approaches are being researched, and the types of virtual experiences used in their evaluation. Keywords: Virtual reality cybersickness Simulator Sickness Visually induced motion sickness reduction Systematic review Head mounted display.


Introduction
Virtual reality (VR) has been described as technology which allows "humans to visualize, manipulate and interact with computers and extremely complex data" Aukstakalnis and Blatner (1992). VR technologies have been made more immersive through the use of head mounted displays (HMD), wearable devices that place a view port over a user's eyes, allowing them to peer into the virtual scene and often direct their gaze with physical head motions as if they were really there.
In the last decade, VR applications making use HMDs have been used in an increasingly diverse variety of applications. These applications include the treatment of phobias Botella et al. (2017), workplace training Gavish et al. (2015), the piloting of remote vehicles Carruth et al. (2019), and the enabling of traditionally difficult research fields such as evacuation pattern research Lin et al. (2020).
HMDs show promise in many settings, but the phenomenon known as cybersickness continues to hinder widespread use and acceptance. Cybersickness consists of several symptoms akin to motion sickness, such as nausea, vertigo, disorientation, and eye strain LaViola Jr (2000). HMDs have been shown to produce worse cybersickness outcomes than common desktop displays Yildirim (2019). For this reason, we tailored this review specifically towards cybersickness reduction approaches that have been evaluated with HMD devices. These approaches are intended to reduce symptom severity, or prevent symptoms from appearing altogether.
These symptoms can arise during or after HMD use, and can dramatically reduce user performance, and force sensitive users to end their VR usage prematurely even in controlled experimental settings Mittelstaedt et al. (2018); Farmani and Teather (2018). This problem is made more challenging by a lack of understanding regarding the fundamental causes of cybersickness, and the existence of competing theories to explain its occurrence Chang et al. (2020).
In light of this problem, researchers have been at work designing and evaluating new cybersickness reduction techniques to improve user comfort. By cybersickness reduction technique, we refer to any augmentation or design prescription for a VR application intended to reduce or eliminate cybersickness. This literature review aims to present a comprehensive overview of cybersickness reduction techniques developed and evaluated since 2013, with the release of the first developer kit for the Oculus Rift. Drawing from a collection of 1,055 papers, we discovered 88 papers that present potential techniques for reducing cybersickness and evaluate their performance against one or more additional conditions in user studies. We categorize the approaches of these papers into nine categories to develop a better understanding of how cybersickness research is distributed across different types of approaches. We also note whether the techniques in these papers produced statistically significant reductions in cybersickness, to better grasp which approaches show the most promise. Finally, because the design of the virtual scene contents can impact cybersickness Davis et al. (2015); Ihemedu-Steinke et al. (2017); Dorado and Figueroa (2014), we categorize these papers in terms of their virtual realism, their use of virtual vehicles, the virtual scenes in which testing took place, and the methods users are given to navigate the virtual environment.

Background on cybersickness 2.1 Symptoms and measures
Cybersickness refers to the visually induced form of motion sickness caused by virtual environments Kim et al. (2005). Cybersickness encompasses a variety of motion sickness like symptoms that include nausea, vertigo, disorientation, eye strain, sweating, and others LaViola Jr (2000). Researchers use a variety of strategies to detect and quantify these symptoms in test participants. Measuring cybersickness may involve the use of subjective reporting methods, such as the Simulator Sickness Questionnaire (SSQ) introduced by Kennedy et al. (1993), or dedicated Likert scales asking study participants to rate individual symptoms Keshavarz and Hecht (2011). Physiological signals have also been measured to estimate a user's level of cybersickness Dennison et al. (2016), such as Galvanic skin response, heart rate, and blink rate.

Theoretical causes
Three predominant theories for the underlying causes of cybersickness are the sensory conflict theory, poison theory, and postural instability theory LaViola Jr (2000). The sensory conflict theory posits that a mismatch between visual stimulus, and the body's vestibular system gives rise to the symptoms of cybersickness. The vestibular system is located within the inner ear, and provides information regarding the orientation of the body. If this theory is correct, minimizing the conflict between the vestibular system and visual input may reduce the levels of cybersickness experienced by users. The poison theory suggests that the consumption of certain poisons lead to distortions in visual input and the vestibular system, thus providing an evolutionary incentive to vomit when the interaction between the two are disrupted LaViola Jr (2000). The postural instability theory Riccio and Stoffregen (1991) suggests that the human body always tries to maintain postural stability, and that any failure to do so is an indication of danger in the environment. Reducing cybersickness is difficult in part because the causes are still uncertain. These theories are also general in nature, and do not specifically describe cybersickness experienced as the result of wearing and HMD. The focus of this literature review is not on the variety of cybersickness measurements and potential causes. A comprehensive review of these topics has been accomplished by Chang et al. (2020), and an analysis of impact of different factors was given by Saredakis et al. (2020).

Grand challenge
Cybersickness presents a significant challenge to the comfortable use of virtual reality. Upwards of 60 percent of users experience these symptoms while participating in VR applications Regan and Price (1994). Remedying cybersickness is especially pertinent for the development of HMD applications, as HMDs are significantly more conducive to cybersickness than common desktop displays Sharples et al. (2008). In this paper, we aim to perform a comprehensive review of the approaches researchers have taken to reduce cybersickness in modern HMD applications.

Methods
While conducting our review of the literature, we followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) proposed by Moher et al. (2009). The PRISMA statement provides a recommended structure for literature reviews, and a checklist of included information.

Inclusion
Accepted papers were required to match the following criteria. Firstly, the paper must measure and compare at least one symptom of cybersickness between two or more user conditions. At least two of these conditions must involve an HMD. Papers must examine the impact of a mutable factor on cybersickness. We define a mutable condition as one that can be conceivably altered to improve the VR experience for users of HMDs. For example, a paper examining the effects of room temperature on cybersickness Arnold et al. (2019) would qualify, because the temperature can easily be altered in a lab or at home if it improves the user experience. A paper comparing user cybersickness exclusively based on age would not qualify, because it is not possible to lower the age of a user to improve their experience. These studies may examine the impacts of software Fernandes and Feiner (2016), additional hardware aids Aldaba and Moussavi (2020), or non-technological factors such as body positioning Marengo et al. (2019); Clifton and Palmisano (2020).

Exclusion
Papers that exclusively compared HMDs to alternative displays such as cave automatic virtual environments (CAVE) or common desktop displays were not accepted. Papers were additionally excluded if they were published prior to 2013 or later because this year marked the release of the Oculus Rift DK1. We believe that the release of the Oculus Rift marked the beginning of broader public adoption and commercial accessibility to HMD technology. In selecting this as a cutoff point, we aim to set a reasonable scope for this review, and prioritize cybersickness research that has been conducted alongside modern HMD devices.

Process of selection
We used the Publish or Perish (PoP) database software Harzing. (2014) to generate our initial list of candidate papers from Google Scholar. We requested papers using tags related to virtual reality (VR, virtual reality, HMD, head mounted display), and to cybersickness (cybersickness, motion sickness, simulator sickness, cybersickness reduction, cybersickness reduction technique), and limiting results between 2013 and December of 2020. The PoP software generated 997 results using our search terms. A separate, prior search by hand resulted in 58 papers, 13 of which were not found by the PoP software. We first removed papers based on factors that did not require a full read through. A total of 72 duplicates were removed, 45 of which came from the hand search. Next, 12 patents, 43 non-english, and 10 thesis results were removed from the total. Next we read the titles and abstracts of the remaining 917 papers. Of these 71 surveys, 10 papers with no relation to HMDs, and 17 papers comparing HMDs to other display types were filtered out. An additional 499 papers with seemingly no relation to reducing or examining cybersickness were filtered as well. The remaining 321 papers were evaluated based on a full text read through. Of these 18 were removed for examining nonmalleable conditions, such as age or health conditions. An additional 45 papers were removed for not measuring cybersickness symptoms. A single paper was removed for failing to include seemingly any information whatsoever regarding the testing conditions. Finally, 169 papers were removed from the pool for failing to ultimately compare cybersickness severity between two or more conditions. This left us with 88 papers that fit our criteria. An illustration of this process can be found in Figure 1.

Classification
We created a taxonomy for organizing the accepted papers. Our goal was to illustrate the distribution of research across different approaches. In doing this, we aimed to highlight what types of reduction techniques have received the most research, and what approaches may warrant further investigation. Additionally, our taxonomy is used to organize this review so that information is clearly compartmentalized, and so readers may return to the sections of this paper covering their specific topics of interest. Each paper was placed into one or more of the following nine categories.
• Virtual Camera Manipulation Approaches that manipulate the direction or movement pattern of the user's virtual gaze within the simulation. • Image Manipulation Techniques that utilize shaders, filters, or other forms of visual modification such as alterations to frame rate. • Field of View Techniques that modify the user's field of view (FOV) to reduce visual flow and information. FOV alteration may qualify as a form of image manipulation, but as a common and distinct approach to reducing cybersickness we placed it in a seperate category. • Virtual Companion Objects Strategies that use 3 dimensional virtual objects that follow or exist in close proximity to the user's avatar. • Movement Reduction approaches that alter the user's movement method or control of their movement. • Virtual Scene Design considerations with regards to graphical style or environment geometry. • Viewing Hardware Reducing cybersickness by switching to a different head mounted display. • External Hardware The use of hardware devices besides the display. • Non-Technological Approaches Reducing cybersickness through non-software or hardware solutions, such as medication, exercise, or physical positioning.
Papers have the potential to fall into multiple categories in the case they examine more than one approach to reducing cybersickness within a single or among multiple user studies.
We additionally categorize papers based on the visual realism of their virtual scene content. We broke realism into four categories: abstract, simple, realistic, video. Abstract virtual environments are those with little to no resemblance to the real world, Such as those made up of basic geometric shapes Ziegler et al. (2018) or those in which the user is suspended in a vast void Zielasko et al. (2018). We classify virtual scenes as visually "simple" if they satisfy at least one of three criteria. Simple virtual scenes may make wide use flat solid colored textures [example Risi and Palmisano (2019)], use two dimensional sprites to represent 3D objects [example Ragan et al. (2016)], or exist in a scene composed primarily of a flat plane with minimal geometry [example Nguyen-Vo et al. (2018)]. Realistic scenes are those that more closely resemble the real world, with complex textures. Video scenes are those that are captured directly from the real world with a recording device, and then played back to users for the duration of the experiment. Tables 1 and 2 include this information, and Figure 3 illustrates how many times environments with each level of realism appeared in papers. There are limitations to our classification of realism. We do not account for other factors impacting realism such as lighting. The "realistic" category is also broad, as we do not exclude scenes using cartoon or stylized aesthetics so long as textures are not solid colored.
We note whether the user traversed the environment controlling a virtual vehicle such as a car, bicycle, or roller coaster carriage. Our motivation for this was based on the usage of virtual rest frames to reduce cybersickness, and the hypothesis from Luks and Liarokapis (2019) that virtual vehicles could serve as effective rest frames.
Papers are also tagged based on the method by which the user could move through the virtual environment. Researchers have demonstrated that the selection movement methods can significantly impact reported cybersickness, such as teleportation Habgood et al. (2018), physical walking Llorach et al. (2014), and gaze directed movement Wang et al. (2018). We encountered 12 different movement methods throughout the literature.

FIGURE 1
The selection process of papers included in this review.

Frontiers in Virtual Reality
frontiersin.org 04 • Steering The user travels in a direction over time based on the input or orientation of a controller device. • Arm Swing The user swings their arms in physical space to control their virtual speed. • Teleport (TP) Movement to a targeted location is completed instantaneously or in a very short amount of time. • Leaning The user is propelled through the virtual space by physically leaning in the desired direction. • Node Similar to teleportation, but when movement is only possible between predefined points in the virtual environment. • Anchored The user does not move along X, Y, or Z-axes, but may retain control of their independent orientation. For example, a user sitting in a stationary chair but with the freedom to look around would qualify as anchored movement. • Passive The user moves along the X, Y, or Z-axes automatically without input, and at most controls their independent orientation. For example, a user riding in a roller coaster with no control over their global movement direction or speed, but with the freedom to look around would qualify as passive movement. Prior research has demonstrated that different aspects of virtual scene contents can produce a measurable impact on cybersickness, such as traversable geometry Dorado and Figueroa (2014), virtual object density Ihemedu-Steinke et al. (2017), propensity for sensory conflict Shafer et al. (2019), and intensity of experience Guna et al. (2019). Research still appears to be too limited to holistically understand how the different properties of a virtual scene impact cybersickness. We believed it worthwhile to collect what types of virtual scenes are receiving the most attention within the literature matching our criteria. A technique could conceivably work well in an virtual indoor setting, but not on a virtual forest, beach, or roller coaster. Because qualities such as object density, and experience intensity are sometimes difficult to determine, we sorted the virtual scenes used in these papers into eight easily recognizable categories: Natural, Rural, Urban, Indoors, Roller Coaster, Race Track, Void, and Test Area. While coasters and race tracks are often housed within natural outdoor environments, we felt that the content and experience was specific enough to warrant separate categories. Void environments are those in which the user is suspended in space, without a visible ceiling or floor even in the distance. The final category, Test Area, refers to utilitarian spaces with floors but no ceilings that are built to facilitate testing rather than to act as a standin for a real world location.

Results
Figure 2 provides a breakdown of how many papers featured each cybersickness reduction strategy, and whether that strategy produced statistically significant results. Our goal was to provide a snapshot of the literature to inform researchers of where research is being conducted, where it is not, and where it has produced promising results.
We provide a number of graphs to illustrate the nature of the environments cybersickness reduction techniques are being evaluated in. With regards to realism, realistic environments were by far the most  commonly in a total of 55 papers. This may be because, as mentioned previously, our realistic category does not account for aspects such as lightning, nor exclude environments based on cartoon or stylized aesthetics. Abstract virtual environments were the next most prominent, featured in 15 papers. Simple and video environments were less popular, featured in 9 and 8 papers respectively. Finally, 5 papers did not provide sufficient information to understand their level of realism. Consult Figure 3 for a visual representation of these findings.

Movement and use of virtual vehicles
When it came to the use of vehicles, 71.9% of papers (64 total) did not report the use of a drivable virtual vehicle. In 28.1% of the remaining papers (25 total), users moved through the environment on-board a vehicle, such as a roller coaster, bike, or car. In five of these papers, reduction was evaluated both with and without a vehicle. Figure 4 illustrates this breakdown of vehicle usage.

FIGURE 2
The number of papers with and without statistically significant findings for each category.
Frontiers in Virtual Reality frontiersin.org Steering as a movement method was by far the most popular movement, featured in 42 papers total. Other common movement techniques included passive movement (21 papers), anchored (17 papers), teleportation (10 papers), and physical walking (9 papers). Remaining movement techniques were only featured in one to two papers each. We believe this indicates that more research into alternative movement techniques warrants consideration. Figure 5 provides a visual representation of how commonly each movement method was featured.

Measurement of cybersickness
We labeled papers based on their method of measuring cybersickness. Of the twelve we identified, the simulator sickness questionnaire Kennedy et al. (1993) (SSQ) was by far the most common, used in 73 of the 88 papers in our review. Six papers utilized the Fast Motion Sickness Scale (FMS) Keshavarz and Hecht (2011). A Discomfort Score (DS) based on that of Fernandes and Feiner (2016), and physiological metrics such as heart rate and galvanic skin response (PHYS) were used in 5 papers each. Seven papers made use of basic likert scales to allow participants to report cybersickness symptoms. Two papers featured Yes/No (YN) questions, and three others featured the Misery Scale (MISC) Wertheim et al. (2001). The remaining methods including The Subjective Units of Distress Scale (SUDS) Milosevic and McCabe (2015), Comfort/Okay/Uncomfortable (COU) Mehrfard et al. (2019), and HMD data were each featured in only a single paper each. Figure 6 groups these remaining methods into a single "other" column.

Environment realism
We have also included a breakdown of virtual environment content. Prior research has demonstrated that different aspects of the virtual The portion of papers using vehicles, using none, or using both.

FIGURE 3
The number of papers featuring each level of realism.
Frontiers in Virtual Reality frontiersin.org environment contents can produce a measurable impact on cybersickness. Virtual outdoor environments were widely used, 28 Natural, 15 Urban, and 7 Rural. Indoor environments were also common, featured in 23 papers. Roller coasters were featured in 8 papers, and race tracks were featured in 4 papers. Void environments without ceilings or floors were featured in 6 papers. The final category consists of what we refer to as "test areas." These virtual environments consist of hybridized environments that sometimes have walls, but no ceilings, and serve as utilitarian spaces for housing experiments. These environments do not appear to replicate real world spaces, but still feature a floor and at least some level rudimentary geometry. This type of environment was featured in 7 papers. Please note that three environments were not included in our graph due to a lack of information Han et al. (2017); Kaufeld and Alexander (2019); Zaidi and Male (2018), each an outdoor environment without further details. Figure 7Below we provide a brief description of each accepted paper by category, and whether the authors demonstrated a statistically significant (i.e., labeled Significant or Non-Significant) reduction in symptoms of cybersickness. Tables 1 and 2 provide an overview of the results. Figure 2 illustrates the proportion of significant and insignificant findings for each reduction strategy. Please note that papers may appear in multiple categories should they examine multiple approaches to reducing cybersickness.

Virtual camera manipulation
Some cybersickness reduction techniques involve the manipulation of the user's viewing direction within the simulation. We found eleven papers utilizing this approach, seven of which demonstrated a statistically significant reduction in cybersickness symptoms.
Significant: Sargunam et al. (2017) evaluated the impact of amplified head rotation and a guided head rotation technique in a series of virtual rooms. The guided rotation technique, that significantly increased sickness compared to standard rotation, limits virtual gaze rotation, and slowly rotates the world around the user to prevent neck strain. Farmani and Teather (2018) tested a discrete viewpoint control method of reducing cybersickness in a first person video game. If a user rotated their head past a certain speed threshold, their vision was temporarily blacked out, and their gaze was snapped by a fixed angle within the simulation. Sargunam and Ragan (2018) compared discrete and continuous rotation in a VR gallery. Discrete rotation significantly decreased cybersickness compared to continuous rotation. Ueda et al. (2018) successfully reduced cybersickness in an interior environment with a reorientation method that rotated objects in the foreground but not the background. Palmisano et al. (2017) compared three different gaze simulation methods in VR. The user's simulated gaze was set to match physical head motions, move in the opposite direction of yaw, or ignore head movements and only match the user yaw orientation. The condition in which motion was opposite of head yaw produced significantly higher levels of cybersickness. Onuki and Kumazawa (2019) examined two reorientation methods. The first, called gaze turning, matches the user's virtual head orientation to their physical orientation after a user presses a button, and closes their eyes to return their head to a default position. The second, called face turning had users press and hold a button while moving their head to a desired center position. These methods resulted in significantly less cybersickness than common snap and smooth turning methods. Ziegler et al. (2018) found that manipulating the virtual position of the user's head during collisions with geometry significantly worsened disorientation.
Not Significant: Ryge et al. (2018) compared smooth and discrete rotation in a VR navigation task. Ragan et al. (2016) examined the effects of amplified head rotation in a VR search task. Monteiro et al. (2018) compared first-person and third-person view points in a racing game. Arcioni et al. (2019) had test participants move their heads side to side while their gaze within the scene either moved with motions or against their motions in the opposite direction.

Image manipulation
Image manipluation techniques involve the modification or distortion of the visual stimulus presented to the user. Ten papers

Field of view
FOV reduction techniques reduce optical flow and the amount of visual information available to users by obscuring the edges of their vision. Ten papers presented an FOV reduction approach to reducing cybersickness, four of which featured statistically significant results.
Significant: Kala et al. (2017) used FOV reduction in conjunction with a content analysis algorithm that estimates a user's level of cybersickness based on the images presented to them. The authors additionally compared different fixed FOV angles on users. Experiments were conducted within a virtual roller coaster simulation.

Companion objects
Many studies have explored the use of virtual objects that follow the user's avatar through the simulation. These virtual objects often serve as stationary rest frames, or serve to induce other effects such as reducing perceived motion. We found ten papers that utilized companion objects to reduce cybersickness. Of these, four produced statistically significant results. Significant

Movement
Movement methods of cybersickness reduction involve the changing of movement metaphors (e.g. steering to teleportation), or modifying other aspects of movement such as direction Mazloumi Gavgani et al. (2017), or degree of control Venkatakrishnan et al. (2020). Twenty-one papers examined the effects of movement reduction approaches and twelve demonstrated statistically significant reductions in cybersickness.
Significant: Habgood et al. (2018) compared steering movement, teleportation, and node based travel techniques within a virtual interior navigation exercise. Participants using the node and teleport experienced significantly less cybersickness than those using steering movement. Loup and Loup-Escande (2019) compared a teleportation movement technique to an arm swinging method, in which the user moves forward by rocking their arms back and forth. These movement methods were compared in a viking village simulation. Participants using the arm swinging method experienced significantly more nausea than those using teleportation. Another study from Clifton and Palmisano (2020) compared teleportation in VR to steering movement in a nature walking simulation. They found that the steering movement produced significantly more cybersickness than teleportation. Teleportation and steering movement were compared along with a sliding technique in a virtual urban setting by Lugrin et al. (2019). This sliding technique allows users to move by extending their arms to a location and pulling themselves towards it. The authors found that the

FIGURE 7
The number of papers featuring each environment type.

Frontiers in Virtual Reality
frontiersin.org teleportation technique resulted in significantly less cybersickness. Christou and Aristidou (2017) compared teleportation to two steering movement methods in a virtual desert town. One steering method had users pointing in the direction they wished to travel, while the other allowed users to travel based on the direction of their gaze. Participants using the teleportation method experienced significantly less cybersickness than those in the other two conditions. Llorach et al. (2014) compared physical motion tracking movement to gamepad based steering in a virtual museum. Participants using the motion tracking experienced significantly less cybersickness. Wang et al. (2018) compared joystick based steering movement to a gaze directed semi automatic movement technique in a virtual office interior. Participants using gaze directed movement reported significantly less cybersickness. One study examined the impact of affording users control over their movement Venkatakrishnan et al. (2020). Participants completed a driving simulation either steering the virtual car themselves, riding in an autonomous vehicle that drives itself, or riding in a "yoked pair" condition that made the car follow a trajectory created by another participant who drove the car. The authors found that participants doing the driving themselves experienced significantly more sickness than those in the yoked pair condition. The impacts of movement direction were examined by Mazloumi Gavgani et al. (2017) when users had no control. Participants were placed on a virtual roller coaster facing either forwards or backwards. Participants riding backwards demonstrated a significantly higher ride tolerance, allowing them to ride longer without feeling the effects of cybersickness. In a study from Kwok et al. (2018) participants were tasked with navigating an urban environment at speeds of 10 m per second and 24 m per second. Users moving in the faster condition experienced significantly more cybersickness. Cortes et al. (2019) examined the effects of virtual movement gain on users walking through the physical world. Increasing gain was found to significantly increase reported sickness. Mayor et al. (2019) compared four different virtual movement methods: point of interest (node), room scale (physical walking), steering, and teleportation. They found significant differences between each movement approach, and found that teleportation reduced total SSQ scores the most. Not Significant: Andersen et al. (2020) compared gamepad based steering movement to an organic physical world walking method in an outdoor virtual environment. Steering movement and teleportation were compared by Zaidi and Male (2018). Traveling with the teleportation method resulted in less reported cybersickness, but the statistical significance of the results was not specified. Dorado and Figueroa (2015) compared a gamepad steering method to a head shaking movement method that had users moving their head back and forth to propel themselves forward. Qian and Teather (2018) compared seven different movement techniques in a flying simulation where users had to pass through floating rings in the environment. Users had to control their movement in air using combinations of eye tracking, head movement, mouse, and joystick control. In their first experiment, they found that joystick-only, eye tracking-only, and head movement + joystick movement methods resulted in the highest levels of cybersickness. In a second experiment taking place on a virtual street, the authors compared head tracking, eye tracking, and joystick movement methods. Users in the eye tracking and joystick conditions reported more cybersickness, but statistical significance was not provided. Cmentowski et al. (2019) compared two teleportation movement techniques, one in that greatly enlarged the virtual size of the user, and one that kept user size constant. Nabioyuni and Bowman (2015) examined the effects of increasing the rate of virtual movement based on real world movement. The impact of affording control was studied by Risi and Palmisano (2019) in an futuristic rooftop environment. Users either controlled their locomotion or passively watched a prerecorded traversal. Widdowson et al. (2019) had test participants experience linear and angular motion in a virtual interior environment under three different speed profiles: constant, ramp, and polynomial. Dorado and Figueroa (2014) compared three separate speed mappings with a joystick. This included constant speed, direct mapping to joystick position, and a smoothed option.

Virtual scene design
Virtual Scene Design approaches to reducing cybersickness can involve the modification of lighting, geometry, or textures. We found eight papers that examined the impacts of scene design. Of these, five reported statistically significant effects on cybersickness.
Significant: Researchers have employed a strategy of reducing visual realism in order to reduce cybersickness. In a comparison of two virtual roller coasters, Davis et al. (2015) found that the coaster with increased realism and visual detail was significantly more likely to induce cybersickness. Modifications to virtual scene geometry has been used to improve cybersickness. Traversing up stairs has been shown to induce more cybersickness than walking up ramps in a virtual interior environment Dorado and Figueroa (2014). The density of virtual assets has also been shown to effect cybersickness by Ihemedu-Steinke et al.

Viewing hardware
Some studies have compared the effects of different viewing hardware conditions on cybersickness. This may involve the modification of existing HMD devices, or the comparison of different HMD devices. Seven papers examined viewing hardware approaches to reducing cybersickness, and two produces statistically significant results.
Significant: Geršak et al. (2020) compared the Oculus Rift DK1, Oculus Rift DK2, Oculus Rift CK1, and Samsung Galaxy Note 4. Participants used these different devices to view a simulated roller coaster ride. The authors found that the Oculus DK2 produced Frontiers in Virtual Reality frontiersin.org significantly more disorientation than the Oculus CK1. Guna et al. (2019) compared the effects of various HMD devices and video content. They found some significant differences in reported cybersickness between the Oculus DK2 and Oculus CV1, and between the Oculus CV1 and Samsung Gear VR. They found that the Oculus CV1 produced less disorientation. Not Significant: Mehrfard et al. (2019) compared the Oculus Rift S, HTC Vive, and Samsung Odyssey + HMD devices in a reading task. They found only minor differences in reported sickness across the three devices. One study Han et al. (2017) compared a mobile HMD setup (Samsung Galaxy 7) to a dedicated HMD device (LG G5) in virtual outdoor environments. Another study Gonçalves et al. (2020) compared wired and wireless VR setups. Participants navigated a virtual building interior using either a wireless HTC VIVE, a wired HTC Vive with the aid of a researcher managing the cable for them, or a wired HTC Vive on their own. Yildirim (2019) compared the Oculus Rift CV1 HMD to the HTC Vive in a racing game and a first person shooter. Shafer et al. (2019) compared the Oculus Rift DK2 to the Oculus CV1 to determine if more recent hardware produced better cybersickness outcomes in a variety of commercial VR games.

External hardware
Studies in this category examine the impacts of hardware devices besides the HMD itself. Sixteen papers made use of external hardware devices to reduce cybersickness, and ten of these found significant reductions in cybersickness.
Significant: Harrington and Headlend. (2019) used a standing fan calibrated to match the movement of the user within a desert driving simulation, which resulted in lower cybersickness. Wearable head mounted devices that stimulate the vestibular system were shown to reduce cybersickness by Weech et al. (2018) in a virtual grassy plain, and by Sra et al. (2019) in a virtual roller coaster context. Liu et al. (2019) developed a haptic feedback device that taps both sides of the user's face as they move through a virtual rural environment. Peng et al. (2020) reduced cybersickness with vibrating motors in a device that responded to the user's movements within an urban walking simulation. Langbehn et al. (2019) were able to reduce cybersickness in an abstract redirected walking task by attaching two electrodes to the user's head.
Motion platforms were used to reduce cybersickness in an virtual interior environment by in two studies from the same authors, when vibrating Ng et al. (2019) or when synchronized to the user's movement Ng et al. (2020). Aldaba and Moussavi (2020) examined the effects of joystick, omnidirectional treadmill, VRNChair and TiltChair movement methods in a virtual interior navigation task. Results from two experiments demonstrated that the TiltChair produced significantly less cybersickness than the VRNChair. Nabioyuni and Bowman (2015) examined the effects of wearing jump boots in VR. Users wearing the jump boots reported significantly more sweating.
Not Significant: Two studies using a fan to reduce cybersickness did not produce significant results. Paroz and Potter (2018) placed a small fan in front of users during their time in a robot piloting simulation. Narciso et al. (2020) produced wind stimuli using eight fans scattered around users as they watched 3D videos of an urban environment. In a separate condition, participants experienced olfactory stimuli via a "smell nozzle" device. Weech et al. (2020) used a wearable device to stimulate the user's vestibular system in both an outer space, and third person outdoor virtual environment. Skopp et al. (2014) compared a 12 foot spherical movement controller that users stood inside of to a gamepad controller in a virtual Iraq warzone environmnet. Mittelstaedt et al. (2018) compared a bike ergometer to a gamepad in an outdoor biking simulation. Kaufeld and Alexander (2019) used a motion platform to reduce cybersickness in a helicopter piloting simulation.

Non-technological papers
The remaining papers covered in this review did not involve the use of hardware or software. These papers instead use non-technological means of cybersickness reduction, such as body positioning, substance intake, or physical environment modification. Six papers examined nontechnological remedies to cybersickness, and two of these produced statistically significant results.
Significant: One paper investigated the effects of alcohol use prior to VR usage Iskenderova et al. (2017) in a virtual airplane hangar. Surprisingly, this study indicates that alcohol use can significantly reduce symptoms of cybersickness. Park et al. (2017) demonstrated that a 5 minute oculomotor exercise prior to using VR was shown to significantly reduce cybersickness in Samsung's VR Batman experience. Almeida et al. (2017) compared participants who were told to read a consent form carefully with those who were not. Those who read the consent form carefully experienced a significant increase in sweating and fatigue. Not Significant: Marengo et al. (2019) compared the cybersickness levels of sitting, and supine users in a maze navigation task. Sitting and standing were also compared by Clifton and Palmisano (2020) in a virtual nature walk environment. Risi and Palmisano (2019) used a postural restraint harness that forced users to maintain an upright position. Arnold et al. (2019) had test participants watch driving footage in VR at two different room temperatures, 22 and 35°C.

Discussion
In this section we discuss the key takeaways from our findings, both regarding approaches to reduce cybersickness and the circumstances of their study and evaluation.

External devices
Of the 88 papers in our survey, 16 involved the introduction of external hardware devices beyond the HMD itself. While the introduction of new hardware devices may be costly, inconvenient, or sometimes infeasible, many have shown promise in reducing cybersickness, particularly those that the user wears on their body. Of the six studies involving a wearable device, five produced statistically significant reductions in cybersickness. Studies involving wearable devices were also very diverse in terms of their virtual scene content. Wearable devices produced results in simple natural Weech et al. (2018), realistic rural Liu et al. (2019), and urban Peng et al. (2020) outdoor virtual environments, as well as abstract environments Langbehn et al. (2019), and a virtual roller coaster Sra et al. (2019). These types of devices also saw success with a variety of movement methods, including steering Weech et al. (2018), walking Langbehn et al. (2019), and passive movement Sra et al. (2019).

Frontiers in Virtual Reality
frontiersin.org

Image manipulation
Image manipulation effects are more diverse than FOV effects, but featured more statistically significant results overall in nine of the ten papers. Image blurring produced significant cybersickness reduction in all three of the papers we found Budhiraja et al. (2017); Nie et al. (2019); Qionghua et al. (2019). Image manipulation techniques such as depth of field Carnegie (2015), modified scene transitions Rahimi et al. (2018), and central vision blockers  were used to successfully reduce cybersickness, but were still not featured in a large number of papers. More research into these image manipulation strategies may be useful in confirming their usefulness.

Companion objects
We found several papers employing companion objects as a cybersickness reduction strategy. Results for this general approach are mixed, with four of the ten papers producing statistically significant results in natural and urban outdoor environments  Bala et al. (2018). Given the lack of significant results from rest frames, it may be interesting to conduct a study build from the notion that they are not effective.

Virtual scene design
Virtual scene design also had mixed results, with four of the seven papers in our survey producing significant cybersickness reductions. These papers demonstrated success in a variety of virtual environments, including roller coaster Davis et al. (2015), urban outdoors Ihemedu-Steinke et al. (2017), and building interior Dorado and Figueroa (2014). The approaches of Davis et al. (2015) involved reducing simulation realism to reduce cybersickness, but this general strategy did not produce significant results when employed by Choroś and Nippe (2019), or by Pouke et al. (2018). Reducing realism may continue to be proven effective in future research, however it also possible that this is not a reliable means of reducing cybersickness. The study by Dorado and Figueroa (2014) was unique in examining the effects of movement over different geometry. We believe these strong preliminary results warrant additional research.

Non-technological papers
Non-technological papers, utilizing neither software nor hardware, were very uncommon with only six in total. Three of these papers examined different physical positions of the head Arcioni et al. (2019) and body Clifton and Palmisano (2020); Marengo et al. (2019) but did not report statistically significant results. Alcohol intake Iskenderova et al. (2017) and pre-immersion oculomotor exercises Park et al. (2017) did produce significant reductions in cybersickness, but only one paper exists for each of these approaches. Non-technological papers were exceedingly rare, suggesting that there may wide be a gap in cybersickness research that warrants more investigation.

Realism
With regards to virtual realism, realistic papers as defined by our criteria were by far the most popular, featured in 55 papers. As mentioned before, this may be because our definition of realistic environments was overly broad. Considering the numerous VR applications involve real world video, such as sports spectating Ochi et al. (2016); Kim and Ko (2019) and remote drone operation, Xia et al. (2019), more research into the use of real world video may be warranted. Video footage was rarely used in the evaluation of reduction techniques, only featured in only 8 papers.

Vehicles
Of the 88 papers collected, only a small minority (25 papers, 24.7%) involved the user traveling in a personalized vehicle, such as car, bicycle, or roller coaster carriage. All approaches to reducing cybersickness contained at least one paper making use of a virtual vehicle. In the case of Virtual Camera Manipulation, there was exactly one paper Monteiro et al. (2018). Strangely, Virtual Scene Design papers were more likely to use virtual vehicles than not. Of the eight papers, only two did not feature a virtual vehicle Pouke et al. (2018); Dorado and Figueroa (2014). Only four papers made use of both onfoot and vehicle perspectives Yildirim (2019)

Field of view reduction
Visual FOV reduction was a common software-based approach to reducing cybersickness, but only four Kala et al. (2017); ; Fernandes and Feiner (2016); Al Zayer et al. (2019) of the ten papers produced statistically significant improvements. The lack of statistically significant results is surprising considering the number of papers using this approach, and the fact that FOV is a uniquely homogeneous category compared to broader approaches such as image manipulation. A small proportion of significant results is not enough to indicate that a technique is ineffective, only that more research may be warranted. Upon examining the individual papers, we further argue that these results do not indicate that FOV is a dead-end for cybersickness reduction research. Of the papers that did not produce significant results, one compared two variations of a FOV technique against one another rather than an FOV technique against a control condition Adhanom et al. (2020). Three of these papers had exceedingly small sample sizes of 12 Bala et al. (2018), 17 Ferdous et al. (2018), and 18 Norouzi et al. (2018); Buhler et al. (2018). The final paper evaluated FOV in a highly unique abstract environment Al Zayer et al. (2019).

Frontiers in Virtual Reality
frontiersin.org 6.9 Statistical significance and equivalence testing We found that many papers did not uncover a statistically significant difference in cybersickness between two or more study conditions. These papers still provide value through novel study designs, and other reported data. We wish to stress that these studies do not serve as evidence that a given technique is ineffective. It may be the case, however, that the lack of significant findings for some approaches may result from the technique being ineffective.
While statistical tests such as t-tests and ANOVA are designed to identify differences between groups, equivalence tests are used to confirm statistical similarity. To give an example, researchers investigating the impact of a particular FOV reduction method cannot conclude that the impact was equivalent between groups just because their results were not statistically significant. However, similarity could be confirmed through the use of an equivalence test, such as the two one-sided test (TOST) Schuirmann (1987). In our final collection of papers, we did not encounter a study designed around the use of equivalence testing.
Across most of the categories in our review, we observed a mix of both statistically significant and insignificant results as seen in 2. Insignificant results may be explained by a number of factors, including low sample size Bala et al. (2018), the use of subtle techniques intended to reduce cybersickness with minimal noticeable change Norouzi et al. (2018), and the use of highly specific virtual testing settings that may not reflect more common use cases Al Zayer et al. (2019). Given these explanations, some techniques may bear fruit in future research with adjustments, greater sample sizes, or different testing environments.
An alternative interpretation of these results is that many techniques are not in fact effective at reducing cybersickness, and this ineffectiveness is reflected in the mixed results of the research. As mentioned previously, we did not encounter any research attempting to demonstrate an equivalence between two conditions. Given the plethora of studies unable to measure a significant impact for particular cybersickness reduction strategies, we believe the field would benefit from studies explicitly designed around a starting hypothesis that the techniques in question are not effective. Such research was non-existent in our review, but could be used to determine if individual techniques are ineffective in certain environments. Virtual rest frames and changes in environmental realism stand out in particular as two noteworthy candidates for this type of research, given their mixed findings. In addition to uncovering dead-ends in the research, this approach could determine if separate techniques are comparable in their performance.

Comparing techniques
Research comparing reduction techniques against a control condition, or comparing variations of a technique was common among the papers found in our review. Far less common were studies comparing different techniques, such as a FOV reduction technique compared to a static rest frame, or changes in movement speed compared to changes in movement metaphor. Of the papers in our review, only four compared separate techniques in this way Ziegler et al. (2018); Bala et al. (2018); Buhler et al. (2018); Luks and Liarokapis (2019).
We strongly recommend that researchers consider designing studies around comparing separate techniques. This type of research appears to be largely absent from the literature, and could provide further insight into which techniques are preferable in various virtual environments.

Limitations
In this section we acknowledge many of the limitations of this review. Our initial batch of papers was mass generated from Google Scholar using the Publish or Perish (PoP) database software Harzing. (2014). As a result, papers exclusively found within databases not indexed by Google Scholar may not have had the opportunity to appear in our review.
This review was directed at reduction strategies and good-practice design principles that had been evaluated with a physical HMD. Some approaches may have been evaluated in part or exclusively with other virtual reality display types such as desktop displays or cave automatic virtual environments. Developers may wish to explore certain techniques that have not been specifically tested with an HMD for HMD based applications, but any such approaches have not been included in this review.
In an effort to set an achievable scope for this review and place greater emphasis on recent research, a cutoff year was set in 2013 with the release of the Oculus Rift DK1. We believed the release of this device marked a turning point for the commercial acceptance and viability of VR, and that the HMD's used in papers past this date would generally fall in line with the modern state of VR technology. It is important to note that plenty of relevant VR research exists prior to this point. This review should be considered a look into the modern state of research on cybersickness mitigation since 2013.

Conclusion
In this literature review, we have provided a high level understanding of the current state of cybersickness research with respect to HMDs, both in terms of how research is distributed across reduction strategies, and the types of virtual environments used in evaluating them. This involved the selection of 88 papers from an original total of 1,055 that examine the relationship between malleable conditions and cybersickness intensity. We categorized these papers based on their approach to reducing cybersickness, and whether effects on cybersickness were reported as statistically significant. We also labeled papers based on the virtual environments used in evaluation, with labels describing realism, virtual scene content, the presence of virtual vehicles, and user locomotion methods. We present several takeaways from our findings. Approaches involving virtual scene design, and viewing hardware, and nontechnological approaches were the least researched, and more research may be warranted to understand their effectiveness. Approaches involving Movement, External Hardware, an Virtual Camera Manipulation appear most promising, with a large number of papers each demonstrating significant reductions in cybersickness. In addition to being a narrow category with ten papers, only three papers utilizing FOV approaches demonstrated significant reductions in cybersickness. Non-technological solutions were rare, indicating that more research into reduction strategies beyond software and hardware may be warranted. Reducing cybersickness through the design of virtual spaces also appears to warrant more research, particularly in regards to virtual geometry. With regards to the virtual circumstances in which techniques were Frontiers in Virtual Reality frontiersin.org evaluated, realistic scenes as defined by this paper were by far the most utilized over abstract, simplistic, and video levels of realism. Natural, urban, and indoor environments were the mostly widely used in terms of virtual scene contents, and most (71.9%) papers did not report using virtual vehicle for transporting the user. Equivalence testing was completely absent from our findings, and studies comparing separate techniques were exceedingly rare. The SSQ Kennedy et al. (1993) completely dominated other measurements of cybersickness, featured in 73 papers. A great deal of research has been accomplished towards reducing cybersickness during HMD use, but there is plenty more to be done in confirming the effectiveness of existing techniques in a wider array of settings, and discovering novel reduction strategies to further increase the applications of VR technologies.

Author contributions
SA completed the initial review of 1,055 papers and elimination down to 88. JQ and SA collaborated on the creation of the taxonomy, and the writing of the final document. All authors contributed to the article and approved the submitted version.

Funding
This research was supported in part by a grant from the National Science Foundation (IIS-2007041, IIS-2211785). The NSF did not have a direct role in this research project.

Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.