ORIGINAL RESEARCH article
Sec. Virtual Reality and Human Behaviour
Granulated Rest Frames Outperform Field of View Restrictors on Visual Search Performance
- 1Department of Mechanical Engineering and Materials Science, Duke University, Durham, NC, United States
- 2Department of Computer Science, University of North Carolina at Greensboro, Greensboro, NC, United States
Dynamic field of view (FOV) restrictors have been successfully used to reduce visually induced motion sickness (VIMS) during continuous viewpoint motion control (virtual travel) in virtual reality (VR). This benefit, however, comes at the cost of losing peripheral awareness during provocative motion. Likewise, the use of visual references that are stable in relation to the physical environment, called rest frames (RFs), has also been shown to reduce discomfort during virtual travel tasks in VR. We propose a new RF-based design called Granulated Rest Frames (GRFs) with a soft-edged circular cutout in the center that leverages the rest frames’ benefits without completely blocking the user’s peripheral view. The GRF design is application-agnostic and does not rely on context-specific RFs, such as commonly used cockpits. We report on a within-subjects experiment with 20 participants. The results suggest that, by strategically applying GRFs during a visual search session in VR, we can achieve better item searching efficiency as compared to restricted FOV. The effect of GRFs on reducing VIMS remains to be determined by future work.
As part of the immersive experience in virtual reality (VR), navigation in the virtual environment (VE) is an essential action along with selection and manipulation. Several application areas, such as the military (Zyda, 2005; [Dataset] Baumann, J., 1993), medicine (Seymour et al., 2002; Seymour, 2008), athletics (Sorrentino et al., 2005), and manufacturing (Kozak et al., 1993) require long-distance or frequent virtual travel1. However, virtual travel has the common side effect of visually induced motion sickness (VIMS) (Keshavarz et al., 2014; Jerald, 2015), also known as cybersickness (LaViola, 2000), experienced when visual motion conflicts with motion provided by the vestibular system (Irwin, 1881; Reason, 1970; Reason and Brand, 1975; Johnson, 2005). VIMS generally provokes an unpleasant VR experience and is expressed by headaches, stomach awareness, nausea, and disorientation (Johnson, 2005).
As a way to mitigate VIMS, field of view (FOV) reduction techniques, such as dynamic FOV Restrictors (Fernandes and Feiner, 2016), manipulate the FOV to decrease the peripheral visual flow information presented to users during provocative (travel) motion (Figure 1C). However, FOV Restrictors carry the drawback of potentially leading to lower peripheral awareness, which may demand more effort and time to discern the surrounding scene (Jang et al., 2016). There is an open challenge in identifying a way to maintain the benefits from FOV reduction while maximizing the usage of peripheral FOV.
FIGURE 1. The Granulated Rest Frames design allows the visual recreation of most details from the original image by exploiting human’s ability to discern objects even in the presence of visual noise. (A) Reference Image: a virtual scene with no modifications; (B) Granulated Rest Frames: the scene with Granulated Rest Frames using a
Rest frames (RFs) have been proposed as an alternative to FOV Restrictors, where they were shown to mitigate VIMS by adding virtual elements that remain stable in relation to the physical environment, such as the use of cockpits in many commercial VR games (e.g., [Dataset] Astrofish Games Ltd, 2018; CCP Games, 2017). However, RFs are application-dependent and, similar to FOV reduction techniques, usually block large parts of the peripheral view (Cao et al., 2018).
Blocking the user’s peripheral view can have critical influences on performance. Previous studies have shown that the performance in search tasks depends mostly on FOV size, where users tend to adjust their search behavior (such as the visual scanning pattern) based on the size of the FOV (Ragan et al., 2015), and small FOVs cause performance deterioration (Hogervorst et al., 2013). Conversely, the human visual system can tolerate occluding noise to a great extent, even when processing time is very brief (Meng and Potter, 2008). The challenge, then, is to explore designs of RFs to leverage amodal completion–the ability humans have to detect objects as a whole even if they are partially occluded (Michotte et al., 1991)–and VIMS mitigation so that RFs’ adverse effects on the peripheral view are minimized.
In this work, we propose Granulated Rest Frames (GRFs), a new RF-based design that adds random noise-like grains to the periphery of the FOV (Figure 1B), in a similar way that happens with restricted FOV. However, rather than blocking much of the peripheral view as traditional RFs, it provides an optimized way to generate occluding noise-like RFs that allows amodal completion. Given that RFs reduce VIMS to a similar extent as FOV restrictors, we compared GRFs to restricted FOV on visual search, a common task in many VR applications. We evaluated different designs of GRFs, combining Density and Size to establish how they affect visual search performance. Thus, by comparing visual scanning performance (search time and amount of head rotation) under different conditions, we assessed the hypothesis that GRFs could increase peripheral awareness compared to restricted FOV. Our overarching research questions are, “can users have better visual scanning performance under GRFs, as compared to FOV restrictors?,” and “what is the optimal design of GRFs that can degrade user performance the least?” Specifically, we focus on the effects of FOV restrictors and GRFs on search time and the amount of head rotation during an item searching task in virtual reality.
The results provide a deep understanding of the distinction between GRFs and FOV restrictors on visual scanning task performance. Importantly, our results add to the growing body of literature on the generic generation of RFs (Frey et al., 2007; Lin et al., 2002a; Lin et al., 2004).
In this section, we review related literature on human tolerance to visual noise, and the impact of FOV restrictors and RFs.
2.1 Visually-Induced Motion Sickness
As early as the 1890s, visually induced disturbances have been reported in some devices like Haunted Swing (Wood, 1895) and eyeglasses with inverting prisms (Stratton, 1897). It results in sensations similar to motion sickness: nausea, dizziness, vertigo, and sweating, among other symptoms. In such situations, those symptoms caused by visual artifacts are referred to as Visually-induced Motion Sickness (VIMS) (McCauley and Sharkey, 1992; Kennedy et al., 2010). Aside from VIMS, it has also been found that visual stimuli can lead to perceived illusory self-motion (referred to as “vection”) (Henn et al., 1974; Dichgans and Brandt, 1978). Later work found that vection can also be triggered by auditory cues (Larsson et al., 2004; Valjamae et al., 2005).
The analysis of the associated occurrence of vection and VIMS provides a way to investigate the mechanism of VIMS. Hettinger et al. (1990) suggested that vection was a necessary precondition for VIMS. Nooij et al. (2017) also discovered that VIMS increases with vection strength. They also found that this relation was detected when pooling correlations across all conditions, but not for all conditions considered individually. In a later study, Nooij et al. (2018) hypothesized that strong vection enhanced the velocity storage, a central integrative network involved in motion sickness. However, some work contradicted the positive relation between vection and VIMS (Webb and Griffin, 2003; Weech and Troje, 2017; Weech et al., 2018; Keshavarz et al., 2019). Kuiper et al. (2019) also argued that vection is not a direct cause of VIMS, but a state that relies on other factors to cause VIMS. One striking part of studying the relation between vection and VIMS is the variability of the results in the literature. For example, Palmisano et al. (2018) failed to find a relation between vection and VIMS in a spontaneous postural activity measurement study. However, they affirmed the contribution of vection to VIMS in other work (Palmisano et al., 2017; Risi and Palmisano, 2019). What the existing research suggests is that the real mechanism of VIMS causation is still not entirely understood.
Despite the divergent conclusions regarding the contributors to VIMS, it is clear that vection relates to the visual stimuli, specifically the optical flow patterns induced by visual stimuli (Telford and Frost, 1993; Palmisano et al., 2000; Bubka et al., 2008; Fujii and Seno, 2020). Seya et al. (2014) described the participants felt stronger vection even if facing much smaller and slower-moving optical flow in the background space than in the foreground space. Other studies also have reported that the more distant stimulus causes vection when visual stimuli differ in depth (Ito and Shibata, 2005; Seno et al., 2009). Although there are studies that disagree with the direct correlation between vection and VIMS, considering the incoherent findings in the literature and the potential positive relationship between vection, VIMS, and optical flow, we hypothesize that certain optical flow types elicit vection and may contribute to VIMS. Our approach is to explore static references at the foreground, while maintaining peripheral awareness through the use of a particular implementation of rest frames, which have been demonstrated to alleviate symptoms of VIMS (Cao et al., 2018).
2.2 Peripheral Vision and Visual Search
Previous work has investigated the link between visual search and peripheral awareness. When performing visual searches, people rely on saccades interleaved with periods of fixation more often than smooth eye movements (Collewijn et al., 1988). The angular speed of eye movement can reach up to 600 deg/s during a saccade (Collewijn et al., 1988). Such high retinal speed requires peripheral vision to select potential targets and guide eyes in search tasks efficiently (Erkelens and Hooge, 1996; Rajashekar et al., 2002). Peripheral vision operates coarsely on patches containing multiple items rather than accurately on individual items (Rosenholtz et al., 2012). Geringswald and Pollmann (2015) confirmed that peripheral vision loss could prevent integrating local configurations with the global display layout. The integration failure led to reduced spatial configuration learning and impaired contextual cueing in visual search.
2.3 Effects of Visual Noise on Visual Perception
Amodal completion is a remarkable characteristic of the human visual system that helps people tolerate visual noise, reconstruct, and recognize partially occluded objects (Gerbino and Salmaso, 1987; Sekuler and Palmer, 1992). In three-dimensional (3D) scenes–those occurring in VEs–, Tse (1999) argued that “mergeable” 3D enclosures are crucial elements in amodal completion. Objects with “mergeable” 3D enclosures are partially occluded can be merged by the human visual system when their surfaces have content-related or similar patterns, even if they are geometrically separated. This concept has been validated by other researchers (Anderson et al., 2002; Nanay, 2010; Soska et al., 2010).
2.4 Effects of Field of View on User Experience in Virtual Environments
Several studies in the literature have pointed out the benefits and detriments to large FOVs in VR. Although large FOVs can increase the sense of presence (Howlett, 1990; Prothero et al., 1995), they have also been demonstrated to worsen VIMS (Jex, 1991; Fernandes and Feiner, 2016). A cue conflict generally explains VIMS: the contradictory information received by visual information and vestibular system triggers the nervous system’s reaction to motion sickness (Reason and Brand, 1975). Hence, the larger the FOV, the more apparent a possible conflict might be, which can result in more sickness. It has been observed that people experience more instability, sickness, and presence with larger FOV (Duh et al., 2001; Lin et al., 2002b).
Studies have also found critical influences of FOV over other aspects of the user experience. Low FOV appears to decrease the user’s sense of presence (Arthur and Brooks, 2000), widen the user’s search paths in a visual search task (Cunningham et al., 1995), increase reaction time in shape identification (Robinett and Holloway, 1992), and significantly impair virtual travel performance (Geruschat et al., 1998; Hassan et al., 2002).
2.5 Effects of Rest Frames on VIMS
Several theories have been proposed to explain VIMS. Among them, the Sensory Conflict Theory and the Postural Instability Theory (Irwin, 1881; Reason, 1970; Reason and Brand, 1975; Riccio and Stoffregen, 1991; Johnson, 2005) focus on the role of vestibular systems or postural balance. The Rest Frames Hypothesis (RFH), on the other hand, emphasizes the role of spatial-perceptual references on the effects of VIMS (Prothero, 1998; Prothero and Parker, 2003).
According to the Rest Frames Hypothesis (Prothero, 1998; Prothero and Parker, 2003), whose essential concepts were first introduced in the work by Steele (1961), when the brain selects rest frames for the body next-step motion, the preferable option to select rest frames is heavily influenced by what is perceived to be the visual background. The reason is that most visual cues in the environment with coherent motion status are in the visual background.
Studies on projection-based systems, such as CAVEs (Cruz-Neira et al., 1993), show that the seams between screens and elements of the real world visible beyond the screens acting as RFs on the foreground may induce lower VIMS (Cruz-Neira et al., 1993; Lin et al., 2004, Lin et al., 2002a). Lin et al. (2004) employed a Virtual Guiding Avatar to alleviate VIMS, where the avatar, a visual cue, provided a relatively stable cue. The results from a revised simulator sickness questionnaire (SSQ) (Kennedy et al., 1993) indicated that a Virtual Guiding Avatar with rotational cues alone or with translation could reduce VIMS (Lin et al., 2004). Cao et al. (2018) demonstrated that software-based RFs in HMD systems could also effectively achieve similar impacts.
As a whole, prior research points to a benefit in user comfort of restricting the FOV, under the cost of adverse effects to presence and other user experience factors. Rest frames can mitigate these side effects of FOVs while also reducing discomfort. Based on the human ability to amodal completion, we propose to use Granular RFs that look like visual noise. Our contribution takes advantage of the human visual system’s tolerance to noise in the RF design.
3 Granulated Rest Frames
The design of our proposed GRF technique was inspired by human tolerance to visual noise and the ability to mentally recreate the entire object despite parts of it being occluded by other objects (known as amodal completion, Gerbino and Salmaso, 1987; Sekuler and Palmer, 1992). Our solution uses granulation and random distribution of tiny black circles (grains) as RFs.
With GRFs, we combine the benefits of RFs and FOV restrictors into one visualization technique. RFs are always visible to the user, leveraging the benefits of reducing VIMS, as demonstrated by Cao et al. (2018). Furthermore, the central part of the view is always unrestricted, as is the case with FOV restrictors, (Fernandes and Feiner, 2016). Effectively, our GRF design emulates FOV restrictors without completely obstructing the user’s peripheral view, but, instead, offering stable references in that region of the FOV.
Previously, Cao et al., 2018 demonstrated that RFs effectively alleviate VIMS. However, their design used an application-specific prop–a metal net–as the RF, which, to some extent, restricted the FOV due to the metal net blocking parts of the environment. Notably, some parts of the FOV were continually blocked, making it impossible for a person to reconstruct objects beyond the net with amodal completion. Therefore, we proposed an improved design of RFs, which is application-agnostic. The design ensures that each RF grain can be made small enough to allow human amodal completion, and the random distribution minimizes the possibility of patterns that can continuously block the view. Figure 1B shows an example of the GRFs distribution and how it affects the surrounding environment.
Figure 2 summarizes the design of GRFs. Two parameters control their generation: Size and Density. Size represents the amount of FOV (in degrees) that every single grain (black circle) should cover (Figure 2A), which is measured by
where d is the distance between the user’s viewpoint and the GRFs, and
FIGURE 2. Granulated Rest Frames design. Image (B) illustrates the layout distribution of the RFs at d meters away from the user’s view. GRFs are created using two parameters: (A)size, which represents the amount of FOV one single particle covers and (C)density, which represents the coverage of RF per degree of FOV.
Density represents the number of GRFs per degree of FOV (Figure 2C). Its value can range from 0, representing no RF coverage (as in Figure 1A), to 4/(
In our implementation, the GRF is composed of separated same-size grains (2D circle). All grains are randomly distributed around the user’s head as a sphere with a radius of 1 m. Based on Size, we first calculated the spherical coordinates of all grains with an even distribution. In other words, there is no overlap or gap between any two grains. Then, according to the Density, for each
By controlling the Size and Density parameters, the GRF design can be more user-friendly and application-based. To allow the fovea to be fully available, we left a soft-edged circular cutout at the center of the user’s FOV, as Figure 1A shows. Same as Fernandes and Feiner (2016), the cutout’s opacity linearly increases from completely transparent within the
4 User Study
We conducted a user study to assess different GRF variations on item searching efficiency, specifically, search time and amount of head rotation. Moreover, we compared the effect of GRFs to that of FOV Restrictors (Fernandes and Feiner, 2016). The study was approved by Duke Campus Institutional Review Board.
4.1 Experimental Design
The experiment followed a within-subject design with repeated measures, where Size and Density are the independent variables (IVs) and Time, Accuracy, and Head Rotations are the dependent variables (DVs). A control condition with FOV restrictors recreating the Fernandes and Feiner (2016) technique was added for comparison with our GRF design.
The levels for each IV were:
– Size: 1S(
Note that the condition with
FIGURE 3. Representation of the 9 combinations of GRFs distribution used in our user study. Note that the two red spheres outside the cutout would not be visible by the FOV Restrictors technique.
The DVs are detailed as follows (all data were sampled at 50 Hz):
– Search Time: Time in seconds to identify a target, calculated by:
– Accuracy: The ratio between the targets counted by the participant and the real number of targets.
– Head rotation: The average amount of head rotations, in degrees, to identify one target
Twenty participants from the same institution participated in the experiment (6 females), with mean ages of 26.75
According to the RFH (Prothero, 1998) and its influence on peripheral awareness, we formulated four hypotheses for our study.
• H1: GRFs lead to faster search and lower head rotations as compared to FOV Restrictors.
• H2: Small grains lead to better performance than large grains.
• H3: Low-density GRFs lead to better performance than high-density GRFs.
• H4: Small grains in low-density GRFs are the optimal combination for the lowest search time and head rotation amount.
In order to make our design comparable with dynamic FOV restrictors, we configured the parameters of the circular cutouts with the same inside FOV (IFOV) (
4.4 Task and Stimuli
The visual search task is one of the most popular methods to investigate the attention spent in visual object recognition, in which an observer actively scans the visual environment for a particular target among an array of distractors (Treisman and Gelade, 1980; McElree and Carrasco, 1999). The visual search contains feature search and conjunction search. Searching for a target amongst distractors that differ from the target by a simple visual feature, such as color, shape, or orientation, is known as feature search (Treisman, 1998; McElree and Carrasco, 1999). The other one requires a process to distinguish targets from distractors possessing one or more common visual features, which involves bottom-up processes at an early stage to locate analogs to the target fast, and top-down process in later stages to eliminate distractors (McElree and Carrasco, 1999; Shen et al., 2003). It’s believed that visual search in the real world is more similar to conjunction search with less complexity (Alexander and Zelinsky, 2011, 2012; Hout and Goldinger, 2015).
However, either Fernandes and Feiner (2016) or us apply the visual modification to the periphery, which yields more impacts on the bottom-up process of fast locating targets, like feature search, rather than the top-down process that requires one’s previous knowledge (Zhaoping and Frith, 2011; Rosenholtz et al., 2012). Therefore, we designed a classic feature search experiment in VR. The task consisted of identifying and counting targets that were distributed in the distractors. We separated the field into
FIGURE 4. Representation of the search experiment setup. The virtual targets (red circles) and distractors (green circles and triangles) were placed at a 2-meter radius in a semi-sphere in front of the participant. The GRFs are rendered at a 1-meter radius around the participant’s head (for clarity, in this representation, we just show the front half-sphere of the GRFs). The pink area represents the FOV of the HMD. The point of view is changed by rotating the head and body. (Note. The figure was shot to explain the experimental design, and the person in the figure is the author.)
4.5 Formative Pilot Study
We ran a formative pilot study to determine the stimuli distribution layout around the user. We assessed three layouts: spherical target distribution (
The formative study aims to find the layout to yield the largest different search time between the restricted FOV and our design.
Results (Table 1) from the formative study show that the conditions that caused a variation in search time were the semi-sphere distribution layout and the complete sphere distribution layout. Thus, we chose the semi-sphere distribution layout since navigation rarely requires the users to look back when they are moving. The final task layout consisted of 8–13 targets (red circles) embedded among 187–192 distractors (half is circles, and the other half is triangles. If the distractor amount is odd, the extra one is a circle.) that were equally distributed along the 3D regions around the user. The target distribution is illustrated in Figure 4.
An Oculus Rift CV1 (about
The participants answered a biographical questionnaire and were instructed on the task goals and guided through the XBox controller’s input commands. They practiced in a scene without visual restrictions to get familiar with the task. Then, they experienced 10 conditions in a random sequence. One with fixed FOV restrictor, and 9 conditions with different GRF combinations of Size and Density. Each condition had five trials, followed by a 30-second rest, where the screen of the headset was blacked out, and participants could rest. For each trial, the number of targets randomly ranged from 8 to 13 to avoid a learning effect. When the participant finished the counting, they pressed the Xbox controller right trigger to finish the trial and stop the timer. The targets were deactivated, the time and head rotations data were automatically recorded, and the participant verbally informed the researcher about the number of identified targets and the answer was recorded by the researcher. We instructed the participants to be as accurate as possible, considering time. After the study, the participants were asked their preference on the GRFs and FOV restrictors via the post-experiment questionnaire. The question was “Among all 10 conditions, which one did you feel much easier and more comfortable to identify the target?”
In summary, the experiment consisted of: (3 Size
5.1 Data Analysis
We analyzed whether the Granulated Rest Frames (GRFs) IVs (Size and Density) and their interaction had a significant effect in the Search Time, Accuracy and Amount of Head Rotation with a 2-way repeated measure ANOVA test. We verified if the ANOVA assumption of normality of the residuals was violated with the Shapiro-Wilk test. Then, we used repeated-measures ANOVA to analyze the effect of the nine GRF combinations plus the FOV Restrictors (Fernandes and Feiner, 2016) conditions on Search Time and Amount of Head Rotation, and the non-parametric Friedman test to analyze the effect of the GRF variations plus and the FOV Restrictors condition on Accuracy. The alpha significance level was set to 0.05. We conducted a Post-hoc analysis if a variable was found statistically significant.
5.2 Search Time
We conducted a two-way ANOVA that examined the effect of Size and Density level. There was a statistically significant interaction between the effect of Size and Density on search time (
FIGURE 5. Search time for each level of GRF Size and Density. When Density is 75%, GRFs with Size 1S (
A single factor analysis was conduced that examined the effect of FOV Restrictors and Rest Frames (9 GRFs variations) conditions. The statistical analysis indicates that there was a statistically significant effect of the conditions on search time (
FIGURE 6. Search time performance. Each variation of GRFs is compared with FOV Restrictors. GRFs combinations 1S_50% (
In conclusion, regarding the search time per identified target, the results confirm our first hypothesis that GRFs help users obtain better performance compared to FOV Restrictors. Nevertheless, it only partially demonstrated our second and fourth assumptions that low Size only prevails large Size with large Density.
The Shapiro-Wilk test (
5.4 Amount of Head Rotations
A two-way ANOVA indicates a statistically significant interaction between the effect of Size and Density on search movements (
The single factor analysis examining the effect of FOV Restrictors and Granulated Rest Frames(9 Rest Frames variations) conditions on search movements revealed a statistically significant effect of the conditions on search movements (
FIGURE 7. Amount of head rotations to identify the targets. Each variation of GRFs is compared with FOV Restrictors. GRF combinations 1S_25% (
As with direct association with search time, the number of head rotations presented highly similar results to search time, where 1S_75%, 4S_25%, 4S_50%, 7S_25% and 7S_50% produced significantly fewer search movements than the FOV Restrictors condition. The only exception is the conditions 1S_25% and 1S_50%, where the former presented fewer head rotations than the reference condition, while under the 1S_50% condition, the search time was faster, even without a difference in head rotations. Likewise, the results confirmed the first hypothesis, and partially accepted the second and fourth hypotheses, but rejected the third.
5.5 Post-experiment Users Preference
We also collected the participants’ post-experiment preferences on the GRFs and FOV restrictors.
Among all 20 participants, 5 people preferred FOV restrictors, since GRFs distracted them. The other 15 participants preferred GRFs. They reported that it helped them concentrate on the target and use peripheral vision to search for items. “The viewport with GRFs helps in concentration while I can perceive the background as well.” (Subject 1 and 4). Subject 17 mentioned that with the FOV restrictors, it was hard to track the targets due to the limited view. She had to scan the whole graph line by line.
Of the subjects that preferred GRFs, eleven voted against the highest Density. According to their opinions, high Density blocked too much view, and they felt it difficult to see information on the peripheral vision. Size had the same results and responses.
Based on our understanding of peripheral awareness, visual searching, and previous work (Xiao and Benko, 2016), we hypothesized the granulated rest frames could accelerate people’s visual searching efficiency by contrast to FOV Restrictors (Fernandes and Feiner, 2016) due to it blocks the fewer peripheral view, and our visual systems can reconstruct the environment with amodal completion. As a result of this, we proposed four hypotheses regarding the influence of Size and Density of granulated rest frames over visual search performance.
We reported the results of a user study that compared combinations of size and density and the performance of each GRFs combination with FOV Restrictors. FOV Restrictors are considered the standard and have already been implemented in various applications that require virtual navigation. However, the reduction of FOV has a major limitation in peripheral awareness. As hypothesized, users have significantly better visual perception with GRFs compared to restricted FOV. Both the search time efficiency, accuracy and scene scanning (head rotations) are significantly lower when using the specific settings of size and density. Interestingly, the combinations 1S_75%, 4S_25%, 4S_50%, 7S_25% and 7S_50% achieved better time efficiency and lower head rotations. We analyzed the correlation of time and head rotations using the Pearson test and found a strong effect between the two factors (
The significantly faster search time with the combination of Size 1S and Density 75% compared to Size 4S and Density 75%, partially demonstrated our second and fourth hypotheses that only when the Density is very large, the small Size is better than large Size. Along with the fact that no significant main effect but significant interaction effect of Density or Size over the head rotation amount, we can conclude the GRFs’ influence is due to the combination of Density and Size. It might be due to the Density rather than Size causes a more continuous block of FOV. When the Density is low, regardless of the Size, it serves more like a regular FOV containing a more continuously perceived visible peripheral view, which weakens the utilization of amodal completion. That is why only medium Density associated with all Sizes achieved better performance than other conditions.
Therefore, if we want to take advantage of this technique with very high density, it would be best to use small grain sizes. Medium level density can work well with all sizes, but the low-level density performs worse with small grains concerning search time.
One concern we had was the user acceptance of the visual noise added by our technique. The post-experiment questionnaire results suggest that the participants quickly adapted to the GRFs in the peripheral vision, confirming that our implementation of RFs can be successfully tolerated without degrading the experience.
Overall, GRFs helped subjects achieve better peripheral awareness than restricted FOV. On the other hand, performance did not change significantly among conditions with different settings of GRFs. While we can’t make a strong conclusion on the lack of significance, this may suggest that different layouts of GRFs do not influence peripheral awareness. Considering the majority of participants voted against high Density, as well as considering the computational performance, we suggest choosing low-level density combined with medium size to set up the Granulated Rest Frames. Not only because the fundamentals of GRFs are maintaining the advantages of FOV modification, but also using the ability to amodal completion to avoid the loss of peripheral information.
Inspired by the Rest Frames Hypothesis (Prothero, 1998; Prothero and Parker, 2003), previous work applied rest frames in a cockpit design to moderate VIMS (Cao et al., 2018). Similar to Cao et al. (2018), we employed GRFs, a type of foreground visual stimuli that occupies the peripheral FOV, which, according to the Rest Frames Hypothesis, is sufficient to be selected as stationary references, even if there are conflicting cues from the background.
Unlike other types of rest frames, such as a cockpit (Cao et al., 2018) or a virtual nose (Wittinghinll et al., 2015), GRFs can be created as visual noise that is generic and content-irrelevant. With Size and Density, the format of GRFs can be more flexible, user-friendly and context-free, which allows this technique to be implemented at the graphics driver-level and seamlessly applied to various HMDs or applications. However, we still need to validate our proposed GRF design as effective to alleviate VIMS symptoms, which we plan to investigate in future work.
7 Conclusion and Future Work
We have discussed an improved design of rest frames–Granulated Rest Frames, summarized how we iterated on variations of its design, and performed a within-subject user study exploring seated users’ visual perception performance under granulated rest frames with different parameters to a condition with FOV restrictors. The data from the 20 participants we analyzed indicated that densely granulated rest frames might degrade visual perception efficiency. The comparisons among different conditions suggested users tend to perform more efficiently with GRFs as compared to FOV restrictors.
Spatial recognition ability will be an essential aspect to further exploration, which might be achieved, in part, by employing another counterbalanced study comparing FOV Restrictors and medium-size sparse GRFs. Since we limited our work on the effects of RFs’ peripheral awareness using a static scene, future work should validate the expected benefits of GRFs on VIMS reduction during virtual navigation tasks. Moreover, future improvements of the GRFs technique will require a real-time dynamic modification of the rest frames (size and density), considering the scene features and user locomotion speed and actions. Besides, the search performance highly depends on the specific task and stimulus parameters. For example, when the search task requires top-down processes involving previous knowledge, such as words or elaborate patterns, the performance will likely be different from the task involving bottom-up search processes. Also, the stimulus’s size plays an important role, especially when the GRFs size is similar or larger. In this study, we investigated the influence of GRFs on bottom-up search processes rather than the top-down processes, which only involves fast locating according to color and shapes. Besides, the GRFs size or density was set at a low level to avoid the impacts. Nevertheless, the potential influence of stimulus size and tasks’ specificity also should be studied in the future.
Data Availability Statement
The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.
The studies involving human participants were reviewed and approved by Duke Health Institutional Review Board. The patients/participants provided their written informed consent to participate in this study.
ZC performed the study design, examined patients, took samples, and wrote the manuscript draft. JG contributed to the data, crafted the illustrations, corrected the draft of the manuscript. RK contributed to study design, supervised and coordinated the project, and corrected the manuscript. All authors contributed to manuscript revision, read and approved the submitted version.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frvir.2021.604889/full#supplementary-material
1In this paper, we consider virtual travel as continuous motion control of the viewpoint, and distinguish it from other forms of virtual travel, such as teleportation.
Cao, Z., Jerald, J., and Kopper, R. (2018). Visually-induced Motion Sickness Reduction via Static and Dynamic Rest Frames. IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (IEEE), 105–112, Tuebingen/Reutlingen, March 18-22 2018, Germany, IEEE. doi:10.1109/vr.2018.8446210
CCP Games (2017). Eve: Valkyrie - Warzone. Online. Available at: https://store.steampowered.com/app/688480/[[accessed] November 23, 2019).
Cruz-Neira, C., Sandin, D. J., and DeFanti, T. A. (1993). Surround-screen Projection-Based Virtual Reality: the Design and Implementation of the Cave. Proc. 20th SIGGRAPH (Acm), 135–142. 10.1145/166117.166134.
Cunningham, J. A., Nelson, W. T., Hettinger, L. J., Haas, M. W., and Russell, C. A. (1995). Assessing target detection performance using head position data- A qualitative and quantitative analysis. In International Symposium on Aviation Psychology, 8 th, Columbus. OH, 158–163.
Dataset Astrofish Games Ltd (2018). Ground Runner: Trials. Online. Reading, UK, Available at: https://store.steampowered.com/app/759060/Ground_Runner_Trials/ [accessed] November 23, 2019).
[Dataset] Baumann, J. (1993). Military Applications of Virtual Reality. Online. doi:10.1007/978-3-663-10693-7 Available at: http://hitl.washington.edu/projects/knowledge_base/virtual-worlds/EVE/II.G.Military.html [accessed] July 23, 2019)
[Dataset] Doc-Ok.org (2016). Optical Properties of Current VR HMDs. Online. Available at: http://http://doc-ok.org/?p=1414 [accessed] July 14, 2020).
[Dataset] SIGTRAP Ltd (2019). VR Tunnelling Pro. Available at: http://assetstore.unity.com/packages/tools/camera/vr-tunnelling-pro-106782 [Online accessed] February 17, 2020).
Dichgans, J., and Brandt, T. (1978). Visual-vestibular Interaction: Effects on Self-Motion Perception and Postural Control. Perception. Berlin, Germany Springer, 755–804. doi:10.1007/978-3-642-46354-9_25
Duh, H.-L., Lin, J., Kenyon, R. V., Parker, D. E., and Furness, T. A. (2001). Effects of Field of View on Balance in an Immersive Environment. Proceedings IEEE Virtual Reality 2001. Yokohama, Japan, IEEE, 235–240. 10.1109/VR.2001.913791.
Frey, A., Hartig, J., Ketzel, A., Zinkernagel, A., and Moosbrugger, H. (2007). The Use of Virtual Environments Based on a Modification of the Computer Game Quake III Arena in Psychological Experimenting. Comput. Hum. Behav. 23, 2026–2039. doi:10.1016/j.chb.2006.02.010
Geringswald, F., and Pollmann, S. (2015). Central and Peripheral Vision Loss Differentially Affects Contextual Cueing in Visual Search. J. Exp. Psychol. Learn. Mem. Cogn. 41, 1485–1496. doi:10.1037/xlm0000117
Hassan, S. E., Lovie-Kitchin, J. E., and Woods, R. L. (2002). Vision and Mobility Performance of Subjects with Age-Related Macular Degeneration. Optom. Vis. Sci. 79, 697–707. doi:10.1097/00006324-200211000-00007
Hogervorst, M. A., Toet, A., and Bijl, P. (2013). Human Search with a Limited Field of View: Effects of Scan Speed, Aperture Size, and Target Conspicuity. Opt. Eng. 52, 041106. doi:10.1117/1.oe.52.4.041106
Hout, M. C., and Goldinger, S. D. (2015). Target Templates: The Precision of Mental Representations Affects Attentional Guidance and Decision-Making in Visual Search. Atten Percept Psychophys 77, 128–149. doi:10.3758/s13414-014-0764-6
Howlett, E. M. (1990). Wide-angle Orthostereo. Stereoscopic Displays and Applications, San Diego, CA: International Society for Optics and Photonics), 1256. 210–223. 10.1117/12.19915.
Jex, H. R. (1991). Some Criteria for Teleoperators and Virtual Environments from Experiences with Vehicle/operator Simulation. NASA. Ames Res. Cent. Hum. Machine Inter. Teleoperators Virtual Environments, 42–47.
Kennedy, R. S., Lane, N. E., Berbaum, K. S., and Lilienthal, M. G. (1993). Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness. Int. J. aviation Psychol. 3, 203–220. doi:10.1207/s15327108ijap0303_3
Keshavarz, B., Hecht, H., and Lawson, B. (2014). Visually Induced Motion Sickness: Characteristics, Causes, and Countermeasures. Handbook of Virtual Environments: Design, Implementation, and Applications. Boca Raton, FL: CRC Press, 648–697.
Keshavarz, B., Philipp-Muller, A. E., Hemmerich, W., Riecke, B. E., and Campos, J. L. (2019). The Effect of Visual Motion Stimulus Characteristics on Vection and Visually Induced Motion Sickness. Displays 58, 71–81. doi:10.1016/j.displa.2018.07.005
Larsson, P., Västfjäll, D., and Kleiner, M. (2004). Perception of Self-Motion and Presence in Auditory Virtual Environments. Proceedings of seventh annual workshop presence, Valencia, Spain: The International Society for Presence Research (ISPR), 2004, 252–258.
Lin, J.-W., Duh, H. B.-L., Parker, D. E., Abi-Rached, H., and Furness, T. A. (2002b). Effects of Field of View on Presence, Enjoyment, Memory, and Simulator Sickness in a Virtual Environment. Proceedings Ieee Virtual Reality 2002. Orlando, FL, USA, IEEE, 164–171. 10.1109/VR.2002.996519.
Lin, J. J.-W., Abi-Rached, H., Kim, D.-H., Parker, D. E., and Furness, T. A. (2002a). A “Natural” Independent Visual Background Reduced Simulator Sickness. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 46, 2124–2128. doi:10.1177/154193120204602605
Lin, J. J., Abi-Rached, H., and Lahav, M. (2004). Virtual Guiding Avatar: An Effective Procedure to Reduce Simulator Sickness in Virtual Environments. Proc. SIGCHI Conference on Human Factors in Computing Systems. Vienna, Austria: ACM), 719–726. 10.1145/985692.985783.
McElree, B., and Carrasco, M. (1999). The Temporal Dynamics of Visual Search: Evidence for Parallel Processing in Feature and Conjunction Searches. J. Exp. Psychol. Hum. Perception Perform. 25, 1517–1539. doi:10.1037/0096-15220.127.116.117
Nooij, S. A. E., Pretto, P., and Bülthoff, H. H. (2018). More Vection Means More Velocity Storage Activity: a Factor in Visually Induced Motion Sickness? Exp. Brain Res. 236, 3031–3041. doi:10.1007/s00221-018-5340-1
Nooij, S. A., Pretto, P., Oberfeld, D., Hecht, H., and Bülthoff, H. H. (2017). Vection Is the Main Contributor to Motion Sickness Induced by Visual Yaw Rotation: Implications for Conflict and Eye Movement Theories. PloS one 12, e0175305. doi:10.1371/journal.pone.0175305
Palmisano, S., Arcioni, B., and Stapley, P. J. (2018). Predicting Vection and Visually Induced Motion Sickness Based on Spontaneous Postural Activity. Exp. Brain Res. 236, 315–329. doi:10.1007/s00221-017-5130-1
Prothero, J. D., Hoffman, H. G., Parker, D. E., Furness, T. A., Furness, I., and Wells, M. J. (1995). Foreground/background Manipulations Affect Presence. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 39, 1410–1414. doi:10.1177/154193129503902111
Prothero, J. D., and Parker, D. E. (2003). A Unified Approach to Presence and Motion Sickness. Virtual and Adaptive Environments: Applications, Implications, and Human Performance Issues, Mahwah, NJ: Lawrence Erlbaum Associates, 47. 10.1201/9781410608888.ch3.
Ragan, E. D., Bowman, D. A., Kopper, R., Stinson, C., Scerbo, S., and McMahan, R. P. (2015). Effects of Field of View and Visual Complexity on Virtual Reality Training Effectiveness for a Visual Scanning Task. IEEE Trans. Vis. Comput. Graphics 21, 794–807. doi:10.1109/tvcg.2015.2403312
Rajashekar, U., Cormack, L. K., and Bovik, A. C. (2002). Visual Search: Structure from Noise. Proceedings of the 2002 symposium on Eye tracking research & applications, New Orleans, LA: Association for Computing Machinery (ACM), 119–123. 10.1145/507072.507096.
Risi, D., and Palmisano, S. (2019). Effects of Postural Stability, Active Control, Exposure Duration and Repeated Exposures on Hmd Induced Cybersickness. Displays 60, 9–17. doi:10.1016/j.displa.2019.08.003
Robinett, W., and Holloway, R. (1992). Implementation of Flying, Scaling and Grabbing in Virtual Worlds. Proceedings of the 1992 symposium on Interactive 3D graphics. Cambridge, MA: ACM), 189–192. 10.1145/147156.147201.
Seymour, N. E., Gallagher, A. G., Roman, S. A., O’Brien, M. K., Bansal, V. K., Andersen, D. K., et al. (2002). Virtual Reality Training Improves Operating Room Performance. Ann. Surg. 236, 458–464. doi:10.1097/00000658-200210000-00008
Shen, J., Reingold, E. M., and Pomplun, M. (2003). Guidance of Eye Movements during Conjunctive Visual Search: the Distractor-Ratio Effect. Can. J. Exp. Psychology/Revue canadienne de Psychol. expérimentale 57, 76–96. doi:10.1037/h0087415
Soska, K. C., Adolph, K. E., and Johnson, S. P. (2010). Systems in Development: Motor Skill Acquisition Facilitates Three-Dimensional Object Completion. Developmental Psychol. 46, 129–138. doi:10.1037/a0014618
Wittinghinll, D., Ziegler, B., Moore, J., and Case, T. (2015). Nasum Virtualis: A Simple Technique for Reducing Simulator Sickness. Game Developers Conference (GDC). San Francisco, CA: Game Developers Conference.
Xiao, R., and Benko, H. (2016). Augmenting the Field-Of-View of Head-Mounted Displays with Sparse Peripheral Displays. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. NY: ACM, 1221–1232. doi:10.1145/2858036.2858212
Zhaoping, L., and Frith, U. (2011). A Clash of Bottom-Up and Top-Down Processes in Visual Search: The Reversed Letter Effect Revisited. J. Exp. Psychol. Hum. Perception Perform. 37, 997–1006. doi:10.1037/a0023099
Keywords: human performance, visual search, rest frames, virtual reality, HCI
Citation: Cao Z, Grandi J and Kopper R (2021) Granulated Rest Frames Outperform Field of View Restrictors on Visual Search Performance. Front. Virtual Real. 2:604889. doi: 10.3389/frvir.2021.604889
Received: 10 September 2020; Accepted: 10 May 2021;
Published: 31 May 2021.
Edited by:Ben Lawson, Naval Submarine Medical Research Laboratory, United States
Reviewed by:Richard So, Hong Kong University of Science and Technology, China
Bernhard E. Riecke, Simon Fraser University, Canada
Copyright © 2021 Cao, Grandi and Kopper. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Regis Kopper, email@example.com