Reducing Motion Sickness When Reading With Head-Mounted Displays By Using See-Through Background Images

With the development of virtual reality technology, head-mounted displays (HMDs) have been increasingly used to view or read 2D content while in vehicles. When 2D content is displayed in a fixed position on an HMD, the visual appearance does not change even with head movements, and body movements cannot be perceived while watching 2D content. This may cause visual-vestibular conflict and severe motion sickness. This study investigates whether motion sickness when reading 2D content fixed to the HMD coordinate system can be reduced by allowing readers to perceive body movements through video see-through images obtained from the front camera of the HMD in situations where the body vibrates. Twenty participants performed 20-minutes reading tasks in which they read a book fixed to the HMD coordinate while seated in a vibration device. Two background conditions of the book were explored: 1) the white background condition—not allowing participants to perceive their movements visually, and 2) the camera background condition—allowing participants to perceive their movements visually through see-through images. Evaluation of motion sickness using the Misery Scale, which is an 11-point questionnaire, showed that motion sickness after task completion was significantly lower in the camera background condition than in the white background condition. This result suggests that motion sickness can be relieved, provided that the users perceive their motion in the peripheral vision through the camera image, even if they gaze at 2D content fixed in the HMD coordinate system in the central vision. This study helps promote the use of HMDs by alleviating motion sickness.


INTRODUCTION
Motion sickness is a disease whose pinnacle symptom is vomiting. It is classified into carsickness, seasickness, airsickness, simulator sickness, and so on, depending on the scene in which it occurs. Regarding the onset mechanism of motion sickness, various theories, such as sensory conflict theory (Reason 1978), postural instability theory (Riccio and Stoffregen 1991), and eye-movement theory (Ebenholtz et al., 1994), have been proposed. For example, sensory conflict theory postulates that motion sickness is assumed to occur when there is a period of accumulation of conflicts between actual sensory information and sensory information expected from past experience, or conflicts between sensory signals, such as vision, vestibule, and somatosensory signals.
Carsickness is one of the most frequently observed motion disorders in daily life. Various causes and situations are related to carsickness. The physical motion of a vehicle causes motion sickness. Moreover, visual information affects motion sickness; for example, an obstruction of the forward vision when sitting in the back seat can cause motion sickness (Griffin and Newman 2004). In addition, motion sickness caused by reading, watching smartphones and tablets while moving, and watching in-car displays has increased, given recent advances in visual media devices (Morimoto et al., 2008a). For example, when watching an in-car display in a moving car, viewers perceive their body motion from the vestibular sensation, while they perceive it as stationary from their visual senses because the display is fixed to a part of the car. This conflict between the body movements perceived via visual and vestibular sensations is thought to increase motion sickness. Therefore, abundant research has been conducted on measures against motion sickness while viewing content when riding a vehicle Kitazaki 2006, 2008;Kitazaki 2006;Morimoto et al., 2008b;Isu et al., 2014;Karjanto et al., 2018;Hatayama and Sato 2021).
However, with the development of autonomous driving technology, opportunities for non-driving-related activities in cars are increasing, as drivers are released from driving. In addition to traditional in-car activities, such as smartphone operation, in-car display viewing, and reading books, the use of head-mounted displays (HMDs) and smart glasses have also been considered (Schwind et al., 2018;Li et al., 2020). For example, using an HMD allows us to be in a 3D virtual environment at any time and place while our body is in a vehicle. However, it has been pointed out that experiencing a virtual environment using an HMD while riding induces motion sickness (Krueger 2011;McGill and Brewster 2019). This is considered to be caused by conflicting visual and vestibular sensations, similar to motion sickness caused by gazing at a monitor in a car. Even when experiencing an HMD in a stationary state, motion sickness can occur because of the experience of vestibular and visual sensory conflicts (Laviola 2000). In addition, body movements associated with car motion add to the factors of motion sickness when experiencing HMD while riding a car. Therefore, further research on motion sickness, its mechanisms, and the measures to reduce it in such a 3D virtual environment is being actively pursued (Hock et al., 2017;McGill et al., 2017;Li et al., 2021).
The use of HMDs in a car is enabled not only for 3D content such as virtual environments but also for 2D content such as web pages and e-books. Similar to a virtual experience (3D) on an HMD and watching 2D content on an in-car display, watching 2D content on an HMD is considered to cause motion sickness. There are various possible positions of display for 2D content in HMDs and smart glasses, but one of the easiest setting methods is to fix it on the display (HMD coordinate system). However, when the 2D content is fixed on the HMD coordinate system, the view of the 2D content does not change, even if the viewer's head moves, and visual information does not convey body motion information. Motion sickness caused by watching 2D content on an HMD can be more severe than that due to watching content on an in-car display. The reason is that the viewer is able to perceive their body movements in the vehicle when using an in-car display; however, when the 2D content is fixed to the HMD coordinate system, the viewer is unable to perceive any motion from the visual system.
To enhance productivity when watching 2D content on an HMD in a moving vehicle, a method to reduce motion sickness is necessary. However, this topic has remained largely underexplored in the literature. An exception is Sato et al., 2022, who investigated the effect of the motion of a book as 2D content read by participants on motion sickness when viewed with an HMD in a moving environment. As the experimental variable, 2D content was either fixed to an earth-fixed coordinate system, so that the appearance of the content changed when the body moved, or to an HMD coordinate system, so that it remained constant. The results of their study revealed that the severity of discomfort caused by motion sickness, evaluated by an index called the relative evaluation questionnaire (REQ), was significantly reduced by fixing 2D content to the earth-fixed coordinate system. However, no significant differences were found in the other indicators related to sickness symptoms, such as MISC (Bos et al., 2005) and total symptom score (Golding et al., 1995). Furthermore, the technique, in which 2D content is fixed to the earth-fixed coordinate system cannot be applied to a moving vehicle. Therefore, it is necessary to find a method for reducing motion sickness in such situations.
Previous studies have shown that a visible background reduces motion sickness (e.g., Rolnick and Bles 1989;Griffin and Newman 2004;Bos et al., 2005;Butler and Griffin 2006). From these studies, we hypothesized that a visible background presented in the form of see-through video images would reduce sensory conflict and reduce motion sickness. Moreover, for non-HMD devices such as in-vehicle displays, presenting motion information in the background of 2D content has been shown to reduce motion sickness (Kuiper et al., 2018;Meschtscherjakov et al., 2019). In the case of HMD, as well, presenting motion information in the background of 2D content may reduce motion sickness. Therefore, this study investigated whether presenting body motion using video see-through images acquired from the front camera of the HMD in the background of 2D content can reduce motion sickness when watching 2D content fixed to the HMD coordinate system in a vibrating environment. One of the contributions of the present study is to verify whether the presentation of a visible background to reduce motion sickness can be expanded to providing this background information as see-though video images while viewing 2D content on an HMD.
The participants were seated on a vibrating device and performed the task of reading aloud a book displayed on the HMD. Because the book was fixed to the HMD coordinate system and the view of the book was constant, the user could not perceive their body movement from the movement of the characters on the book. Two experimental conditions were explored: the camera and white background conditions. In the camera background condition, a video see-through image obtained from the front camera of the HMD was presented on the background of the book. In the white background condition, the background of the book was completely white, and there was no movement. The camera background condition is considered to have less sensory conflict than the white background condition because the participants could perceive their body movements from their peripheral vision. Therefore, we tested the hypothesis that a camera background reduces motion sickness more than a white background.

Participants
Twenty adults (four females, age = 21.9 (M) ± 1.04 (SD) years, range 21-25 participated in this experiment. The Motion Sickness Susceptibility Questionnaire (MSSQ; Golding 2006) results of the participants are illustrated in Figure 1. This questionnaire expresses the susceptibility to motion sickness as a percentile based on information regarding how often motion sickness occurs in childhood (under 12 years of age) and the last 10 years for various vehicles. The inclusion criterion of participants was set at an MSSQ of over 30% to exclude people who have never experienced motion sickness. All participants provided informed consent before taking part in the experiment, and they were provided 4,000 yen as a reward. This study was approved by the Ritsumeikan University Ethics Review Committee for Medical and Health Research involving Human Subjects. All participants were uninformed about the study's purpose and were unfamiliar with the experimental task and rationale.

Vibration Device
The participants were seated in a chair attached to the vibrating device, as shown in Figure 2, and their bodies were secured with four-point seatbelts.

Head-Mounted Display
The HMD used in this experiment was a VIVE Pro Eye (HTC Corporation). It has a resolution of 1,440 × 1,600 pixels per eye and a 110°field of view, and was rendered at 90 Hz. The resolution of the pass-through camera image was 640 × 480 pixels per eye, with a 96°field of view horizontally, 80°field of view vertically, and rendered at 90 Hz; the latency was 200 ms.

HMD Reading Task Under Vibration
The participants were asked to read aloud a text on the HMD while seated on a chair attached to the vibration device. The initial chair position was perpendicular to the ground, as shown in Figure 2. The chair rotates by 0.15 rad in either of the forward or backward pitch directions and returns to its initial position. The time it took for the chair to rotate and return to its initial position was constant at 2.5 s. After stopping at the initial position for a few seconds, the same rotation was repeated. The direction of rotation from the initial position to the front or back was random. In addition, the stopping time at the initial position was set randomly by a uniformly distributed random number between 1  and 3 s. Therefore, the participants could not predict their future motion. The text was displayed as 2D at a point 1.0 m in front of the eye in the HMD. It was displayed in a white frame of 0.5 m in length and width and printed in a range of 0.3 m in length and width with a total of 44 characters-4 characters in length and 11 characters in width. The participants read the book using a controller to flip through the pages. Because the text was fixed to the HMD and the appearance of the text was always constant, the participants could not visually perceive their body motion by looking at the text.

Experimental Factor
The experimental factor was the background of the book, and two conditions were set up: one with a white background and another with a camera background. Figures 3, 4 show examples of images from the left eye for each condition. In both conditions, the text was fixed to the HMD coordinate system, but the appearance of the background was different. In the white background condition, the background was completely white, and the users could not perceive their body motion from visual sensations. In the camera background condition, the background of the text was a real-time video see-through image acquired from the front camera of the HMD. Therefore, they could perceive their body movements by looking at the peripheral areas of the book. The experiment was divided into 2 days: 1 day for the white background condition and the other day for the camera background condition. The content of the book was changed daily: Natsume Soseki's I Am a Cat on the first day and Botchan on the second day. The duration of the reading task was 20 min.

Motion Sickness
Three indices were used to measure motion sickness. The first is the Misery Scale (MISC; Bos et al., 2005). This index evaluates the severity of motion sickness on an 11-point scale ranging from 0 to 10. Here, MISC is evaluated every minute during the task. If the MISC reaches 6, the experiment is continued for 2 min and then terminated. If the MISC reaches 7 or more, or if the participants request to stop the experiment, the task is terminated immediately. The MISC after the participants' interruption is set to the maximum value before the interruption. The second is Total symptom scores (Golding et al., 1995). This index evaluates the severity of motion sickness as the total value of the various symptoms of motion sickness. Symptoms include dizziness, hot flashes, headache, sweating, stomach discomfort, saliva secretion, nausea, and pallor of the face (a maximum of 27 points). Here, total symptom score is evaluated after completion of the task.
The third is the Relative Evaluation Questionnaire (REQ). After completing the task under both conditions, the participants were asked to choose in which condition they experienced a more severe motion sickness and to assess the relative severity of motion sickness for both conditions. The scoring was as follows: the white background condition was more severe (2), the white background condition was a little more severe (1), neither changed (0), the camera background condition was a little more severe (-1), and the camera background condition was more severe (-2). This evaluation index was created by Sato et al. (2022).

Reading Performance
To measure readability in each condition, the participants were asked "Was the book easy to read?" They were asked to draw a line through continuous values ranging from 0 to 1. The easier it is to read, the closer it is to 1; the harder it is to read, the closer it is to 0. Additionally, the number of pages read for each condition was measured. However, participants were not forced to read the text quickly.

Procedures
The following instructions were given to the participants at least 1 day before the experiment.
1) Take care of yourself so that your health condition does not worsen.
2) The day before the experiment, get enough sleep (approximately 7 h) to avoid drowsiness. 3) Do not drink alcohol on the day of the experiment. As much as possible, avoid drinking alcohol on the day before the experiment so that no alcohol remains in your body on the day of the experiment. 4) Finish your meal at least 2 h before the start of the experiment, and do not be hungry or too full. In this way, factors related to the severity of motion sickness other than the experimental factors can be eliminated as much as possible.
On the day of the experiment, participants were interviewed about their physical condition, sleep duration, satiety, and alcohol consumption. If they were judged to be unwell, the experiment was terminated that day. On the day of the experiment, an explanation of the experiment and a questionnaire on the participant's physical condition were given first, and informed consent was obtained. The participants were seated on a chair attached to a vibration device, and they were restrained with a seatbelt and a head restraint. The HMD was then attached to them and they were provided with a controller. Subsequently, the participants performed the HMD reading task with the vibration device turned on. During the task, the bell rang every minute, and when the bell rang, the participants were asked to answer the MISC verbally. After completing the task, they were asked to fill out questionnaires on total symptom score and reading performance. At the end of the experiment on the second day, the participants were asked to answer the REQ. For counterbalancing purposes, on the first day of the experiment, the white background and camera background conditions were set for 10 participants each. The participants were not informed of the hypotheses of this experiment.

Motion Sickness
Note that one participant interrupted the experiment at 480 s, because the MISC reached 7.   FIGURE 7 | Average total symptom scores (TSS) for each condition immediately after the task was completed. The median TSS was significantly lower in the camera background condition than in the white background condition. Error bars indicate the standard error. Asterisk indicates statistically significant difference (*p < 0.05).

MISC
Frontiers in Virtual Reality | www.frontiersin.org June 2022 | Volume 3 | Article 910434 background condition was lower than that under the white background condition. Figure 6 shows the mean values and standard errors of the MISC for each condition 20 min (1,200 s) after the start of the task for all participants. The Wilcoxon signed-rank test showed that the median MISC at 20 min was significantly lower in the camera background condition than in the white background condition (p = 0.0240). Figure 7 illustrates the total symptom score (TSS) for each condition immediately after task completion. The Wilcoxon signed-rank test showed that the median TSS was significantly lower in the camera background condition than in the white background condition (p = 0.0227). Figure 8 shows the mean and standard error of the REQ for all the participants. The Wilcoxon signed-rank test showed that the median REQ was significantly greater than zero (p = 0.0046). This result indicates that motion sickness was more severe in the white background condition than in the camera background condition. Figure 9 shows the mean and standard error of book readability in the task for all the participants. The Wilcoxon signed-rank test showed that books were significantly easier to read in the camera background condition than in the white background condition (p = 0.0297). Figure 10 shows the mean and standard error for the number of pages read by the participants under each condition. The Wilcoxon's signed-rank test showed no significant difference between the two conditions (p = 0.6149). Note that the result did not include the value of one participant who interrupted the task.

DISCUSSION AND CONCLUSION
This study investigated the effect of presenting video see-through images in the background of 2D content on reducing motion sickness when 2D content fixed to the HMD coordinate system is viewed in a vibrating environment. Participants performed the task of reading aloud a book fixed to the HMD coordinate system while seated in a device that vibrated in the pitch direction. The results for all three motion sickness evaluation indices were significantly milder in the camera background condition than in the white background condition. The results also indicate that the readability of the book was significantly improved in the camera background condition compared with the white background condition. From the viewpoint of sensory conflict theory, it can be interpreted that the camera background condition likely reduced the conflict between visual and vestibular sensations, reducing motion sickness. It is also interpreted that the camera background condition likely improved postural stability, based on postural instability theory (Bles et al., 1980;Riccio and Stoffregen 1991;Stoffregen FIGURE 8 | Average REQ. The median REQ was significantly greater than zero. Error bars indicate the standard error. Asterisk indicates statistically significant difference (*p < 0.01).
FIGURE 9 | Average readability of the book for each condition immediately after the task was completed. Books were significantly easier to read in the camera background condition than in the white background condition Error bars indicate the standard error. Asterisk indicates statistically significant difference (*p < 0.05).
FIGURE 10 | Average number of pages of the book for each condition immediately after the task was completed. There was no significant difference between the two conditions. Error bars indicate the standard error.
Frontiers in Virtual Reality | www.frontiersin.org June 2022 | Volume 3 | Article 910434 6 et al. , 1999;Mayo et al., 2011;Stoffregen et al., 2013;Munafo et al., 2016). Therefore, it is crucial for future research to investigate whether postural control is stabilized by presenting the images from the front camera of the HMD in the background of the displayed book. Furthermore, the result that the camera background condition improves readability suggests that the motion sickness mitigation method of this study may be used to improve productivity in cars. This study helps promote the use of HMDs in a vehicle by alleviating motion sickness. Sato et al. (2022) proposed a method to reduce motion sickness when viewing 2D content on an HMD in a moving environment by fixing the content to the earth-fixed coordinate system. The results of their study revealed that the severity of motion sickness as evaluated by the REQ was significantly reduced. However, no significant reduction was found in the MISC (Bos et al., 2005) or in total symptom score (Golding et al., 1995), which evaluates motion sickness on the basis of symptoms. Expanding on the results of Sato et al. (2022), this study contributes to the literature as follows. First, it demonstrates that even when the 2D content is fixed on the HMD, motion sickness can be reduced by displaying the body motion using camera images as the background image. In Sato et al.'s (2022) method, the appearance of the 2D content changed according to the user's body movement; however, the background of the 2D content was white and motionless. Therefore, users perceived their body movement from the change in 2D content mainly in the central vision but not in the peripheral vision. Meanwhile, in the present study, because the 2D content was fixed to the HMD coordinate system, the participants could perceive their body movements from the camera image in the background of the book through their peripheral vision. It is known that visual stimulation in the periphery is more effective than in the central visual area for visually induced self-motion illusions, often referred to as vection (Telford and Frost 1993). The difference in whether the participants perceived their body movements from their central or peripheral vision might be a factor in the difference in the motion sickness reduction effect. Other than Sato et al.'s (2022) method, no motion sickness mitigation method has so far been proposed for watching 2D content on an HMD in a vibrating environment. Although not involving HMD, scholars have investigated the reduction of motion sickness by providing motion information of the body or car around the 2D content viewed on in-car displays, tablets, and smartphones. Our results show, for the first time, the motion sickness reduction effect suggested by previous research on in-car displays.
Previous findings on the reduction in motion sickness by displaying images that reflect body motion mainly on the peripheral vision are consistent with the tendency observed in the present study. It should be noted that a camera image was used in the present study, whereas real scenery was shown around the content in Kuiper et al. (2018) and computer graphics were used in Morimoto et al. (2008a). Therefore, the effect of this method on the reduction of motion sickness was not obvious. In fact, a study showed that motion sickness was not reduced by the presence of camera images (Griffin and Newman 2004). The study used a small display placed in front of passengers in the rear seat, where the view outside the car was completely blocked, to show a camera image of the front direction of the car to reduce the severity of passengers' motion sickness. However, there was no significant difference in the severity of motion sickness between the conditions with and without the display, and the presentation of the camera image did not alleviate motion sickness.
Differences exist between the study by Griffin and Newman (2004) and the present study. First, watching the camera image in the central vision differs from watching it in the peripheral vision. In the present study, because 2D content was viewed, most of the central vision was occupied by the 2D content, and the camera image had to be perceived in the peripheral vision. Meanwhile, in the study by Griffin and Newman (2004), as the small display was viewed in the central vision and the inside of the car was viewed in the peripheral vision, the movement of the car in the Earth coordinate system could not be perceived from the peripheral vision. As previously mentioned, peripheral vision is superior to central vision in terms of motion perception; therefore, motion sickness might have been alleviated in our study because the camera image was presented in the peripheral vision. Furthermore, in our study, by presenting the camera image in the peripheral vision, where the temporal and spatial resolution is lower than that of the central vision, it becomes difficult to recognize the time delay and the low resolution in the camera image. This is another reason that motion sickness was reduced in our study.
This study shows that placing a camera image in the background of 2D content can reduce motion sickness when watching 2D content with an HMD in a vibrating environment, even with camera images that have low resolution and time delay. Kuiper et al. (2018) showed that the wider the visual range of the outside view around the in-car display, the less severe the motion sickness. However, in a typical vehicle, only passengers in the front seat can see the outside with a wide field of view. The fact that the camera image can reduce motion sickness suggests that it may be possible to reduce the motion sickness of passengers in the rear seat-where it is physically difficult to see outside the vehicle because of the presence of obstacles-by presenting camera images taken outside the vehicle.
Furthermore, in this study, the camera background condition not only reduced the severity of motion sickness, but also improved the readability of the book. There was a concern that the focus on reading would be compromised, and the book would be difficult to read because the extra information from the camera image can be seen around the content; however, an opposite result was obtained. This result suggests that the motion sickness mitigation method used in this study can be used to improve people's productivity in cars.
Some limitations of the present research include the fact that only participants with an MSSQ of 30% or higher were recruited to eliminate those who have never experienced motion sickness. In addition, all participants were in their twenties, and only 4 of the 20 participants were female. Therefore, it is unclear whether this reduction method is effective for people of all motion sickness susceptibilities, ages, and genders. Another limitation was that the participants moved only in the pitch direction. Therefore, it is unclear whether the mitigation effect is effective when the body is moved by rotation in the roll or yaw directions. To increase the generalizability of the results, it is necessary to verify whether the method used in this study is effective for various participants and motion patterns. Future work may also investigate whether our proposed method is effective, even in a vehicle environment.

DATA AVAILABILITY STATEMENT
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

ETHICS STATEMENT
This study involving human participants was reviewed and approved by the Ritsumeikan University Ethics Review Committee for Medical and Health Research Involving Human Subjects. All participants provided written informed consent to participate in the study.

AUTHOR CONTRIBUTIONS
TS contributed to the data collection, data analysis, and wrote the first draft of the manuscript. YS and TW conceived the study, designed the experiment, critically contributed to interpretation of the results. All authors contributed to the manuscript revision and approved the submitted version.

FUNDING
This study was funded by Japan Science and Technology Agency (JPMJTR20RR).