Proof-of-Concept of the Virtual Reality Comprehensive Balance Assessment and Training for Sensory Organization of Dynamic Postural Control

Introduction

Postural control requires the brain to integrate information from vision, somatosensory, and vestibular cues in order to respond timely and accurately to changes in the body’s alignment and tone with respect to visual surroundings, support surface, internal references, and gravity (Alcock et al., 2018; Ivanenko and Gurfinkel, 2018). These sensory systems sometimes provide conflicting information (Nashner and Peters, 1990). To illustrate, while sitting on a moving bus, the eyes inform the brain that the surroundings appear to be moving whereas the somatosensory system provides information that the body is stationary. In such cases, the brain needs to ignore or prioritize conflicting information to accomplish postural control (Gaerlan, 2010). Discrepancies in these systems become even more apparent in individuals with neurological conditions, resulting in increased postural imbalance, a higher risk of falls, and, in many cases, a higher fear of falling (Horak, 2006; Cronin et al., 2017). Accurate detection of the system that affects postural control in neurological conditions is key to tailor balance intervention and fall prevention programs to the needs of individuals at risk of falls.

Several computerized balance assessment tools have been developed, including dynamic posturography [e.g., Neurocom^® SMART EquiTest^®, Bertec^® computerized dynamic posturography (CDP/IVR), Biodex^® Balance] or wearable motion sensors (e.g., APDM Opal). These computerized balance assessment tools enable quantitative assessment of postural control (Mancini and Horak, 2010), reduce test performance variability, increase sensitivity to subtle changes (Visser et al., 2008), and determine the systems that may underly impaired postural control (Mancini and Horak, 2010). Of those, the Neurocom^® SMART EquiTest^® system (Natus, San Carlos, CA, United States) and Bertec computerized dynamic posturography (Bertec^®, Columbus, OH, United States) are considered the gold standard of dynamic posturography assessment. These computerized sensory organization tests (SOT) challenge the visual, somatosensory, and vestibular systems to provide a detailed assessment of the underlying sensory deficits affecting balance. Although effective in evaluating and treating balance disorders (Alahmari et al., 2014), the high cost ($80,000–$180,000), space needs for the equipment, lack of portability, and time needed for training, have limited the use of the computerized SOT in clinical practice (Visser et al., 2008). The Clinical Test of Sensory Interaction on Balance (CTSIB) was developed to discern the relative contributions of the visual, somatosensory, and vestibular systems to postural control (Shumway-Cook and Horak, 1986). This test includes six static balance conditions with eyes open, eyes closed, and the use of a dome to create visual conflict combining with feet on firm or foam surface. This test was later modified to only four conditions, excluding the visual dome from the test procedures as the balance tasks with the visual dome did not differ from those in the eyes closed conditions (Cohen et al., 1993). Although clinically useful, the scoring system is crude, semi-objective, and may not be sensitive enough to detect subtle changes in postural control in neurodegenerative conditions (Suttanon et al., 2011).

The advent of portable force platforms with head-mounted virtual reality (VR) technology may provide a clinical, cost-effective, and user-friendly dynamic posturography assessment without compromising the evaluative and rehabilitative effectiveness of current computerized SOT. Although VR technology has emerged in the rehabilitation realm as a promising intervention tool to improve balance and gait (de Rooij et al., 2016; Cano Porras et al., 2018, 2019; Massetti et al., 2018; Chen et al., 2020), only few studies utilized VR to assess the sensory organization of postural balance (Lubetzky et al., 2018, 2019; Trueblood et al., 2018; Wittstein et al., 2020). However, it is not clear whether the SOT administered using a head-mounted VR set is safe and easy to use, provides a realistic experience to the user, and discriminates between postural control conditions that test different balance systems.

Unlike the Equilibrium Score calculated from the force platform of the Equitest^® that evaluates postural balance through center of pressure (COP) displacement only in the anteroposterior (AP) direction, portable force platforms provide a multidimensional assessment of postural control. Quiet standing on a foam increased COP displacement in the AP direction by 8%, and in the mediolateral (ML) direction by 21% (Reynard et al., 2019). Adding assessment of ML sway and sway area has shown to quantify fall risk in neurological conditions, such as changes in postural sway characteristics while standing with eyes closed in Parkinson’s disease (Błaszczyk et al., 2007). Additionally, measuring COP frequency in AP and ML directions brings perspective of adopting different strategies to maintain balance. For example, evidence suggests that changes in perturbation conditions alter COP AP or ML frequency characteristics (Creath et al., 2005). Previous study also noted that postural sway in AP and ML directions can be attributed to ankle and hip control, respectively (Saffer et al., 2008).

The aim of this proof-of-concept study was to demonstrate the safety, sense of presence, system usability, and face validity of the Virtual Reality Comprehensive Balance Assessment and Training (VR-ComBAT) in healthy individuals. The overall hypothesis was that the VR-ComBAT will provide a safe and standardized VR-based balance testing environment in the healthy population. To test our hypothesis, we gauged sense of presence and system usability in the VR environment and satisfaction with our novel VR-based test. Face validity of the VR-ComBAT was evaluated by comparing multidirectional COP displacement and frequency measures across different postural control conditions.

Materials and Equipment

Hardware and Software Specifications

The VR-ComBAT consists of a computer that processes the VR input and output [Alienware (Intel^® Core^TM i7-7800X CPU @ 3.50GHz; 16.0 GB RAM), Dell USA Corporation, Round Rock, TX, United States], a commercially available, head-mounted device (HMD) with integrated VR (HTC VIVE Pro Eye, HTC, Taoyuan, Taiwan), and two VR tracking sensors (Steam VR Base Stations, HTC, Taoyuan, Taiwan). The HTC VIVE Pro Eye headset includes dual-OLED 3.5-inch displays with a combined resolution of 2,880 × 1,600 pixels. The refresh rate of the screen is 90 Hz. The field of view is 110 degrees. The Steam VR (version 1.13, Valve, Bellevue, WA, United States) was used to simultaneously link the computer and the VR headset.

The VR headset was integrated with a force plate (AMTI Optima, Watertown, MA, United States) to measure the displacement and velocity of COP. The sampling frequency of the force plate was 200 Hz. The force plate was manually synchronized with the VR system by simultaneously starting the VR conditions and force plate measurement at each trial. The experimental setup is shown in Figure 1.

FIGURE 1

Figure 1. (A) Virtual Reality Comprehensive Balance Assessment and Training (VR-ComBAT) setup; (B) A participant wears the VR headset and stands on the foam surface placed over the force plate.

Unity 3D (version 2019.3.0; San Francisco, CA, United States) was utilized to create the VR-ComBAT environment. The executable application is available at http://bit.ly/VR-ComBAT (github). The basic setup of the VR environment includes three panels (one front, and two side panels), positioned at 90-degree angle to each, with multi-colored triangular patterns to help participants with visual fixation on the VR environment (Figure 2).

FIGURE 2

Figure 2. Six VR-ComBAT conditions. (1) stable surface with fixed virtual reality (VR) surroundings; (2) stable surface with blacked out VR surroundings; (3) stable surface with VR visual conflict; (4) unstable surface with fixed VR surroundings; (5) unstable surface with blacked out VR surroundings; and (6) unstable surface with VR visual conflict.

Balance Conditions

The VR-ComBAT emulates six different conditions, that, in combination, test visual, somatosensory, and vestibular systems of balance (Figure 2). These conditions mimic the six conditions in the SOT of the Equitest^®: (1) stable surface with fixed VR surrounding; (2) stable surface with blacked out VR surroundings; (3) stable surface with VR visual conflict; (4) unstable surface with fixed VR surroundings; (5) unstable surface with blacked out VR surroundings; and (6) unstable surface with VR visual conflict.

Condition 1 is a baseline that measures static balance on a fixed surface. The surrounding panels in the VR environment remain stationary. Participants can use input from visual, somatosensory, and vestibular systems to maintain balance. In condition 2, participants cannot rely on vision to remain upright since the VR surroundings are blackened out. Condition 3 creates a conflict between normal input from the somatosensory and vestibular systems and the visual information from the moving VR panels. The surrounding panels are moving in the anteroposterior direction with a maximum of 20 degrees and a maximum velocity of 15 degrees/s. Conditions 4, 5, and 6 are identical to 1, 2, and 3, respectively. However, in conditions 4, 5, and 6, a foam (Amazon Basics Balance Pad for Exercise Training, 35 cm × 5 cm × 40 cm, density = 0.04 g/cm³) is placed between the feet and the force plate, thus challenging the somatosensory system.

Methods

Participants

This study was conducted in the Laboratory for Advanced Rehabilitation Research in Simulation, at the University of Kansas Medical Center. This proof-of-concept study recruited 23 healthy participants if they were (1) between 19 and 65 years of age; (2) had no walking or balance impairment [>45 in Berg Balance Scale (BBS; Berg et al., 1992) and >20 in Mini Balance Evaluation Systems Test (Mini-BESTest; Leddy et al., 2011)]; and (3) were able to understand and follow instructions in English. We excluded individuals who (1) had a history of neurological or vestibular conditions and (2) had visual acuity or visual field impairment that could not be resolved by corrective lenses. All participants signed written informed consent. The study ethics was approved by the Institutional Review Board at the University of Kansas Medical Center (#STUDY00145395).

Study Protocol

Following written consent, demographic information (age, sex, and education) was collected. The BBS (total score: 56, lower score indicating higher risk of balance problems) and Mini-BESTest (total score: 28, lower score indicating the higher risk of balance problems) were administered to confirm the absence of any balance impairments. Cognitive impairments were ruled out using the Montreal Cognitive Assessment (MOCA; Nasreddine et al., 2005). Next, participants were fitted with the VR headset and asked to step on the force plate. Then, participants stood on the force plate (conditions 1–3) or on the foam surface (conditions 4–6) with their feet shoulder width apart and hands naturally placed at their sides. Participants were asked to remain in the same position during each VR-ComBAT task (Figure 3). All participants were newly recruited and had no previous exposure to the VR-ComBAT system. There was no practice or learning trial before the recording, since all the conditions in VR-ComBAT were thoroughly explained by the lead researcher and were easy to perform for healthy young participants. Participants with glasses wore the VR headset without taking their glasses off, which allowed participants to experience the VR environment with their normal vision or corrected-to-normal vision. Participants were instructed to stand as still as possible. Each condition consisted of three trials that lasted 20 s per trial. Between each trial, a 5-s break was given. For conditions 2 and 4 (blacked out VR surroundings), the light of the testing room was also turned off to create a completely dark VR environment by removing any lights coming from the gaps of the VR headset. During the experiment, safety was assured by a gait belt and a research team member next to a participant in case of a fall or near-fall event. The experimental protocol is shown in Figure 4.

FIGURE 3

Figure 3. Six VR-ComBAT conditions in the virtual environment. Conditions 1 and 4, conditions 2 and 5, and conditions 3 and 6 share the same VR balance testing environments.

FIGURE 4

Figure 4. A flow chart of VR-ComBAT experimental procedure (BBS = Berg Balance Scale; iPQ = igroup Presence Questionnaire; Mini-BEST = Mini Balance Evaluation Systems Test; MOCA = Montreal Cognitive Assessment; SSQ = Simulator Sickness Questionnaire; SUS = System Usability Scale).

Outcome Measures

Safety

The number of adverse events (e.g., falls) was recorded. VR HMD is believed to increase simulator sickness (similar to motion sickness) compared to remote VR displays (Kim et al., 2014; Dennison et al., 2016). This increased proneness to simulator sickness is thought to stem from the high-fidelity, stereoscopic rendering of VR images. The realistic images displayed in VR HMD can create discrepancies between the perceived and expected visual sensory information, leading to increased symptoms of simulator sickness (Chang et al., 2020). We administered the Simulator Sickness Questionnaire (SSQ) after condition 6, which is the condition that was expected to induce most simulator sickness. The SSQ is regarded as the current gold standard in calculating simulator sickness in research (Kennedy et al., 1993; Jinjakam and Hamamoto, 2012). The SSQ is accurate and reliable to measure simulator sickness in high-fidelity VR environments such as driving simulators, flight simulators, and other VR systems (Kennedy et al., 1993). The score is comprised of three subsections, each with seven symptoms, in which there is some overlap: disorientation, nausea, and oculomotor. The symptoms include general discomfort, fatigue, headache, eyestrain, difficulty focusing, increased salivation, sweating, nausea, difficulty concentrating, the fullness of head, blurred vision, dizziness (eyes open and closed), vertigo, stomach awareness, and burping. The weighted score formula was used to calculate the global index, which reflects the total discomfort level, as well as the scores for the three subsections.

According to previous research (Kennedy et al., 1993), the three subscale scores were calculated by summing scores associated with each subscale and multiplying them by an appropriate weighting factor (9.54 for SSQ-Nausea, 7.58 for SSQ-Oculomotor, and 13.92 for SSQ-Disorientation). To calculate the total score of SSQ, the result was equal to the sum of the three unweighted subscale scores, multiplied by 3.74. A person with a total score of >100 is considered actively ill due to simulator sickness (Kennedy et al., 1993).

Sense of Presence

Sense of presence was evaluated using the iPQ. The iPQ is a multidimensional scale assessing the sense of presence in a VR environment. Sense of presence refers to the subjective feeling of being in a virtual environment. The reliability and validity of the iPQ have been established in previous work (Panahi et al., 2009). The 14 items of the questionnaire are scores on questionnaire contains 14 items, scored on an ordinal scale ranging from 1 to 5. One item reflects general sense of presence. The other items are categorized into sense of spatial presence (5 items), involvement (4 items), and experienced realism (4 items) (Hein et al., 2018).

System Usability

The SUS was administered to capture participants’ viewpoints on effectiveness, efficiency, and satisfaction levels of the VR-ComBAT (Brooke, 1996; Bangor et al., 2008). The questionnaire contains 10 items, each scored on a range from ranges from 0 to 100. The SUS total scores representing a composed measure of the system’s overall usability on a scale from 0 to 100.

Face Validity

Face validity evaluates the extent to which the measured variable appears to adequately measure the conceptual variable (Thomas et al., 1992). The face validity of VR-ComBAT to discriminate between different postural control conditions was evaluated using quantitative COP measures. These COP data were extracted from the force plate and processed using the MATLAB application (MathWorks, Natick, MA, United States) to calculate the following outcomes (Prieto et al., 1996):

• Mean distance in the AP (MeanAP) or ML (MeanML) direction: The average distance in the AP/ML direction from the mean COP.

• Mean velocity in the AP (VelAP) or ML (VelML) direction: The average velocity of COP in the AP/ML direction.

• 95% confidence ellipse area (95%Area): The area that encloses approximately 95% of the points on the COP path.

• Mean frequency in the AP (MfAP) or ML (MfML) direction: The frequency (Hz) of a sinusoidal oscillation with an average value of total path length of excursions in AP (or ML) over MeanAP (or MeanML).

Statistical Analysis

All statistical analyses were performed using SPSS (version 26, IBM Corporation, Armonk, NY, United States). The distribution of data was examined for normality using the Shapiro–Wilk test. Since the number of participants of this study was small and almost all data were not normally distributed, the Friedman test was conducted to examine the differences between conditions followed by post hoc Wilcoxon Signed Rank test. We identified a priori four pairwise comparisons to minimize committing type 1 error. We compared (1) condition 2 to condition 1 to tease out somatosensory system contributions; (2) condition 4 to condition 1 to tease out visual system contributions; and (3) condition 5 to condition 1 and (4) condition 6 to condition 5 to tease out vestibular system contributions (Shumway-Cook, 2011; Pletcher et al., 2017). The effect size ($r=\frac{Z}{\sqrt{n}}$) was adopted to indicate the strength of two conditions of VR-ComBAT. Of note, given that Wilcoxon Signed-rank test was conducted in this study, to more accurately reflect this non-parametric approach, we calculated the effect size (r) according to Rosenthal et al. instead of Cohen’s d effect size (Rosenthal et al., 1994). The significance levels of all analyses were set α = 0.05.

Results

Participants (13 women and 10 men) were on average 27.4 ± 8 years old and reported 19.2 ± 2.5 years of education. None of the participants showed any impairment in static or dynamic balance, as evidenced by maximum scores on the BBS (56 ± 0) and on the Mini-BESTest (28 ± 0). Participants scored on average 28.3 ± 1.4 on the MOCA, indicating no cognitive impairments (Table 1).

TABLE 1

Table 1. Demographic and clinical characteristics.

Safety

No adverse events were reported during or after the study visit. SSQ total scores were on average 33.3 ± 66.1. Average SSQ average subscores were 2.9 ± 6.1 for nausea, 3 ± 5.9 for oculomotor, and 3 ± 7.2 for disorientation. Participants either scored none or slight on each item of the SSQ.

Sense of Presence

The average iPQ score of total iPQ was 44.9 ± 9.6 with average subscores of 18.8 ± 3.7 for spatial presence, 11.3 ± 3.5 for involvement, and 10.7 ± 2.7 for experienced realism.

System Usability

The SUS demonstrated nearly a total score of 80 in system usability (79.7 ± 9.9).

Face Validity

The Friedman test revealed significant differences across the six conditions for all COP-based measures (p < 0.01). Post hoc analyses (Table 2) showed that participants exhibited worse performance on most COP measures in conditions 2, 4, and 5 compared to condition 1.

TABLE 2

Table 2. COP-based measures across the six VR-ComBAT conditions (N = 23).

In the frequency-based analysis, an inverse trend between MfAP and MfML was observed across the conditions. Pairwise comparisons showed that MfAP increased while MfML decreased with respect to increased task difficulty (Figure 5).

FIGURE 5

Figure 5. Mean rotational frequency of COP in anteroposterior (black) and mediolateral (gray) directions across the 6 VR-ComBAT conditions. *Significant differences (p < 0.05).

Discussion

The aim of this proof-of-concept study was to determine the safety, sense of presence, systems usability, and face validity of the newly developed VR-ComBAT in healthy young individuals. We gauged sense of presence in the VR environment and satisfaction with our novel VR-based system as important outcomes of sense of presence. We also included comprehensive quantitative comparisons of multidirectional COP outcomes across the six conditions. The results of this study support our hypotheses that the VR-ComBAT is safe, highly acceptable, and able to detect multidirectional changes of postural control in terms of COP amid different conditions from healthy young participants.

The VR-ComBAT provided a safe and feasible virtual balance testing environment. As Howarth and Costello noted (Howarth and Costello, 1997), VR environments can induce temporary side effects such as general discomfort, fatigue, headache, nausea, and irritating eyes. However, in our study, no adverse events were reported by participants. In addition, the results of SSQ suggested that the VR-ComBAT did not cause any adverse effects of VR throughout the experiment.

Our findings on the sense of presence of the VR-ComBAT measured by iPQ and SUS demonstrated that our novel VR system was well accepted by participants. The total iPQ (44.9) was 9.9-point above neutral presence (35 = 70/2), demonstrating that participants indicated being present in the VR-ComBAT environment. The means of the three subscales of iPQ all contributed to the averaged overall score [spatial presence (18.8 out of 25), involvement (11.3 out of 20), and experience realism (10.7 out of 20)]. The average total score of SUS was approximately 80 out of 100, which demonstrated high effectiveness, efficiency, and satisfaction levels of the VR-ComBAT. The high system usability of VR was also reported in a VR-based rehabilitation study (Meldrum et al., 2012).

The six conditions of the VR-ComBAT emulate an environment equivalent to the SOT. Combined, they are purported to evaluate the sensory systems involved in postural control. Comparing conditions 2, 4, and 5 versus condition 1, our findings are consistent with results from previous SOT studies in healthy younger and older adults (Cohen et al., 1996; Pletcher et al., 2017). When the somatosensory system is confronted with a challenging environment (e.g., conditions 4 and 5), healthy young participants reweighted appropriately to their vision and vestibular systems to maintain postural control. The large effect sizes observed in condition 4 vs. 1 and condition 5 vs. 1 support postural control reliance on vision and vestibular systems, respectively (Pletcher et al., 2017). When comparing condition 6 (unstable surface with VR visual conflict) to condition 5 (unstable surface with blacked out VR surroundings), the significant decreased mean velocity and mean frequency indicate that blacking out the VR surroundings compromised postural control more so than providing conflicting visual information through a moving VR surround. Our results demonstrate that the perturbed somatosensory system might be compensated for by the visual-guided scene through the VR-ComBAT. Vision contributes to balance during quiet standing, and a previous study showed that individuals with impaired vision failed to maintain their postural stability in challenging conditions (Ray et al., 2008). To this end, future studies identifying the changes of COP-based measures in patients with sensory deficits using VR-ComBAT are warranted.

In addition to the findings in the AP direction, the COP-based measures in the ML direction (e.g., MeanML, VelML, and MfML) demonstrated significant changes in this study (e.g., conditions 2, 4, 5 vs. condition 1). Specifically, the change of mean COP frequency of this study is direction-dependent where the mean frequency increases in the AP direction while it decreases in the ML direction with respect to increased task difficulty (Figure 5). While confronted with a challenging environment, healthy young adults seem to restrict ML movement to maintain postural control. This direction-dependent phenomenon was in line with current evidence that additional energy expenditure, such as muscular effort, is required to maintain lateral stability during walking (Kuo and Donelan, 2010). More evidence is required to elucidate whether there is a relationship between COP frequency changes in different directions and energy expenditure. Future studies using the VR-ComBAT will examine whether this direction-dependent phenomenon of COP frequency alters in older adults or patients with deficits in sensory integration.

A previous study that integrated VR HMD with posturography provided similar conditions as the Equitest^® SOT (Wittstein et al., 2020) and demonstrated a moderate reliability and a weak correlation between the developed VR SOT and the EquiTest^® (Wittstein et al., 2020). The VR-ComBAT of this study emulates conditions that are equivalent to those in the EquiTest^® as illustrated in Figure 2. In addition, our study provided a quantitative analysis of COP distance, velocity, area, and frequency in the anteroposterior and mediolateral direction. The importance of including multidirectional COP outcomes was illustrated in our direction-dependent phenomenon of frequency changes in the anteroposterior and mediolateral direction.

There are limitations to this proof-of-concept study. The current study included a relatively small number of healthy young adults. Therefore, we did not adjust for multiple comparisons in the post hoc analyses. Although we established the safety of the VR-ComBAT in healthy individuals, further studies are warranted to validate the VR-ComBAT in people with balance impairments. In addition, the balance testing conditions in the VR-ComBAT system were based on the testing conditions that originate from the EquiTest^®. Although the current study demonstrated the face validity of the novel system, our new test requires validation against the EquiTest^®. We list future directions of work to overcome the limitations observed in the current work and optimize the utilization of the VR-ComBAT. First, we plan to integrate the VR-ComBAT with a cognitive assessment to evaluate the effect of dual-tasking on reweighting for postural control. The eye-tracking integrated VR system such as HTC VIVE Pro Eye can be used to extract pupillary response, which is a valid measure of cognitive workload in demanding postural conditions (Orlosky et al., 2017). Second, we will employ the VR-ComBAT in older and/or patient populations to confirm the sensitivity of the VR-ComBAT in evaluating sensory organization of postural control. Third, not all clinical settings can afford a relatively expensive force plate system. One of the overarching goals of the VR-ComBAT development is to provide a cost-effective system. We developed the VR-ComBAT for research purposes. Thus, the cost of the current setup including force platform system ($10,000–$20,000) and VR HMD system ($500–$2,000) may be expensive for most clinics. However, we plan to create a clinically affordable VR-ComBAT by replacing the force plate with cost-effective COP measures such as Nintendo Wii Fit or gyroscope/accelerometers in smartphones. Lastly, our study found meaningful outcomes of COP changes in ML direction. Future VR-ComBAT conditions should include ML tilting in VR to further elucidate the effect of visual conflict on ML postural control.

In conclusion, this current proof-of-concept study demonstrates the safety, sense of presence, and face validity of the VR-ComBAT integrated with a COP measuring system.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by IRB at University of Kansas Medical Center. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

SM and HD designed the study. SM performed the experiments. SM, C-KH, MS, and HD analyzed the collected data. All authors wrote the manuscript, discussed the results, and commented on the manuscript.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Alahmari, K. A., Marchetti, G. F., Sparto, P. J., Furman, J. M., and Whitney, S. L. (2014). Estimating postural control with the balance rehabilitation unit: measurement consistency, accuracy, validity, and comparison with dynamic posturography. Arch. Phys. Med. Rehabil. 95, 65–73. doi: 10.1016/j.apmr.2013.09.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Alcock, L., O’brien, T. D., and Vanicek, N. (2018). Association between somatosensory, visual and vestibular contributions to postural control, reactive balance capacity and healthy ageing in older women. Health Care Women Int. 39, 1366–1380. doi: 10.1080/07399332.2018.1499106

PubMed Abstract | CrossRef Full Text | Google Scholar

Bangor, A., Kortum, P. T., and Miller, J. T. (2008). An empirical evaluation of the system usability scale. Int. J. Hum. Comp. Interact. 24, 574–594. doi: 10.1080/10447310802205776

CrossRef Full Text | Google Scholar

Berg, K. O., Wood-Dauphinee, S. L., Williams, J. I., and Maki, B. (1992). Measuring balance in the elderly: validation of an instrument. Can. J. Public Health 83(Suppl. 2), S7–S11.

Google Scholar

Błaszczyk, J. W., Orawiec, R., Duda-Kłodowska, D., and Opala, G. (2007). Assessment of postural instability in patients with Parkinson’s disease. Exp. Brain Res. 183, 107–114. doi: 10.1007/s00221-007-1024-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Brooke, J. (1996). “Sus: a “quick and dirty’usability,” in Usability Evaluation in Industry, eds P. W. Jordan, B. Thomas, I. McLelland, and B. A. Weerdmeester (London: Taylor and Francis), 189.

Google Scholar

Cano Porras, D., Sharon, H., Inzelberg, R., Ziv-Ner, Y., Zeilig, G., and Plotnik, M. (2019). Advanced virtual reality-based rehabilitation of balance and gait in clinical practice. Ther. Adv. Chronic Dis. 10:2040622319868379.

Google Scholar

Cano Porras, D., Siemonsma, P., Inzelberg, R., Zeilig, G., and Plotnik, M. (2018). Advantages of virtual reality in the rehabilitation of balance and gait. Syst. Rev. 90, 1017–1025. doi: 10.1212/wnl.0000000000005603

PubMed Abstract | CrossRef Full Text | Google Scholar

Chang, E., Kim, H. T., and Yoo, B. (2020). Virtual reality sickness: a review of causes and measurements. Int. J. Hum. Comput. Interact. 36, 1658–1682. doi: 10.1080/10447318.2020.1778351

CrossRef Full Text | Google Scholar

Chen, Y., Gao, Q., He, C. Q., and Bian, R. (2020). Effect of virtual reality on balance in individuals with Parkinson disease: a systematic review and meta-analysis of randomized controlled trials. Phys. Ther. 100, 933–945. doi: 10.1093/ptj/pzaa042

PubMed Abstract | CrossRef Full Text | Google Scholar

Cohen, H., Blatchly, C. A., and Gombash, L. L. (1993). A study of the clinical test of sensory interaction and balance. Phys. Ther. 73, 346–351; discussion 351–344.

Google Scholar

Cohen, H., Heaton, L. G., Congdon, S. L., and Jenkins, H. A. (1996). Changes in sensory organization test scores with age. Age Ageing 25, 39–44. doi: 10.1093/ageing/25.1.39

PubMed Abstract | CrossRef Full Text | Google Scholar

Creath, R., Kiemel, T., Horak, F., Peterka, R., and Jeka, J. (2005). A unified view of quiet and perturbed stance: simultaneous co-existing excitable modes. Neurosci. Lett. 377, 75–80. doi: 10.1016/j.neulet.2004.11.071

PubMed Abstract | CrossRef Full Text | Google Scholar

Cronin, T., Arshad, Q., and Seemungal, B. M. (2017). Vestibular deficits in neurodegenerative disorders: balance, dizziness, and spatial disorientation. Front. Neurol. 8:538. doi: 10.3389/fneur.2017.00538

PubMed Abstract | CrossRef Full Text | Google Scholar

de Rooij, I. J., Van De Port, I. G., and Meijer, J. G. (2016). Effect of virtual reality training on balance and gait ability in patients with stroke: systematic review and meta-analysis. Phys. Ther. 96, 1905–1918. doi: 10.2522/ptj.20160054

PubMed Abstract | CrossRef Full Text | Google Scholar

Dennison, M. S., Wisti, A. Z., and D’zmura, M. (2016). Use of physiological signals to predict cybersickness. Displays 44, 42–52. doi: 10.1016/j.displa.2016.07.002

CrossRef Full Text | Google Scholar

Gaerlan, M. G. (2010). The Role of Visual, Vestibular, and Somatosensory Systems in Postural Balance. Ph. D. thesis. Las Vegas, NV: University of Nevada.

Google Scholar

Hein, D., Mai, C., and Hußmann, H. (2018). “The usage of presence measurements in research: a review,” in Proceedings of the International Society for Presence Research Annual Conference (Presence), (Prague: The International Society for Presence Research), 21–22.

Google Scholar

Horak, F. B. (2006). Postural orientation and equilibrium: what do we need to know about neural control of balance to prevent falls? Age Ageing 35(Suppl. 2), ii7–ii11.

Google Scholar

Howarth, P. A., and Costello, P. J. (1997). The occurrence of virtual simulation sickness symptoms when an HMD was used as a personal viewing system. Displays 18, 107–116. doi: 10.1016/s0141-9382(97)00011-5

CrossRef Full Text | Google Scholar

Ivanenko, Y., and Gurfinkel, V. S. (2018). Human postural control. Front. Neurosci. 12:171. doi: 10.3389/fnins.2018.00171

PubMed Abstract | CrossRef Full Text | Google Scholar

Jinjakam, C., and Hamamoto, K. (2012). “Simulator sickness in immersive virtual environment,” in Proceedings of the 5th 2012 Biomedical Engineering International Conference, Muang, 1–4.

Google Scholar

Kennedy, R. S., Lane, N. E., Berbaum, K. S., and Lilienthal, M. G. (1993). Simulator sickness questionnaire: an enhanced method for quantifying simulator sickness. Int. J. Aviat. Psychol. 3, 203–220. doi: 10.1207/s15327108ijap0303_3

CrossRef Full Text | Google Scholar

Kim, K., Rosenthal, M. Z., Zielinski, D. J., and Brady, R. (2014). Effects of virtual environment platforms on emotional responses. Comput. Methods Programs Biomed. 113, 882–893. doi: 10.1016/j.cmpb.2013.12.024

PubMed Abstract | CrossRef Full Text | Google Scholar

Kuo, A. D., and Donelan, J. M. (2010). Dynamic principles of gait and their clinical implications. Phys. Ther. 90, 157–174. doi: 10.2522/ptj.20090125

PubMed Abstract | CrossRef Full Text | Google Scholar

Leddy, A. L., Crowner, B. E., and Earhart, G. M. (2011). Utility of the Mini-BESTest, BESTest, and BESTest sections for balance assessments in individuals with Parkinson disease. J. Neurol. Phys. Ther. 35, 90–97. doi: 10.1097/npt.0b013e31821a620c

PubMed Abstract | CrossRef Full Text | Google Scholar

Lubetzky, A. V., Harel, D., Kelly, J., Hujsak, B. D., and Perlin, K. (2019). Weighting and reweighting of visual input via head mounted display given unilateral peripheral vestibular dysfunction. Hum. Mov. Sci. 68:102526. doi: 10.1016/j.humov.2019.102526

PubMed Abstract | CrossRef Full Text | Google Scholar

Lubetzky, A. V., Kary, E. E., Harel, D., Hujsak, B., and Perlin, K. (2018). Feasibility and reliability of a virtual reality oculus platform to measure sensory integration for postural control in young adults. Physiother. Theory Pract. 34, 935–950. doi: 10.1080/09593985.2018.1431344

PubMed Abstract | CrossRef Full Text | Google Scholar

Mancini, M., and Horak, F. B. (2010). The relevance of clinical balance assessment tools to differentiate balance deficits. Eur. J. Phys. Rehabil. Med. 46, 239–248.

Google Scholar

Massetti, T., Da Silva, T. D., Crocetta, T. B., Guarnieri, R., De Freitas, B. L., Bianchi Lopes, P., et al. (2018). The clinical utility of virtual reality in neurorehabilitation: a systematic review. J. Cent. Nerv. Syst. Dis. 10:1179573518813541.

Google Scholar

Meldrum, D., Glennon, A., Herdman, S., Murray, D., and Mcconn-Walsh, R. (2012). Virtual reality rehabilitation of balance: assessment of the usability of the Nintendo Wii^® Fit Plus. Disabil. Rehabil. Assist. Technol. 7, 205–210. doi: 10.3109/17483107.2011.616922

PubMed Abstract | CrossRef Full Text | Google Scholar

Nashner, L. M., and Peters, J. F. (1990). Dynamic posturography in the diagnosis and management of dizziness and balance disorders. Neurol. Clin. 8, 331–349. doi: 10.1016/s0733-8619(18)30359-1

CrossRef Full Text | Google Scholar

Nasreddine, Z. S., Phillips, N. A., Bédirian, V., Charbonneau, S., Whitehead, V., Collin, I., et al. (2005). The montreal cognitive assessment, MoCA: a brief screening tool for mild cognitive impairment. J. Am. Geriatr. Soc. 53, 695–699. doi: 10.1111/j.1532-5415.2005.53221.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Orlosky, J., Itoh, Y., Ranchet, M., Kiyokawa, K., Morgan, J., and Devos, H. (2017). Emulation of physician tasks in eye-tracked virtual reality for remote diagnosis of neurodegenerative disease. IEEE Trans. Vis. Comput. Graph. 23, 1302–1311. doi: 10.1109/tvcg.2017.2657018

PubMed Abstract | CrossRef Full Text | Google Scholar

Panahi, S. M., Fathi, A. A., Azad, F. P., and Montazer, G. A. (2009). Reliability and validity of igroup presence questionnaire (ipq). J. Behav. Sci. 3, 27–34.

Google Scholar

Pletcher, E. R., Williams, V. J., Abt, J. P., Morgan, P. M., Parr, J. J., Wohleber, M. F., et al. (2017). Normative data for the NeuroCom sensory organization test in US military special operations forces. J. Athl. Train. 52, 129–136. doi: 10.4085/1062-6050-52.1.05

PubMed Abstract | CrossRef Full Text | Google Scholar

Prieto, T. E., Myklebust, J. B., Hoffmann, R. G., Lovett, E. G., and Myklebust, B. M. (1996). Measures of postural steadiness: differences between healthy young and elderly adults. IEEE Trans. Biomed. Eng. 43, 956–966. doi: 10.1109/10.532130

CrossRef Full Text | Google Scholar

Ray, C. T., Horvat, M., Croce, R., Mason, R. C., and Wolf, S. L. (2008). The impact of vision loss on postural stability and balance strategies in individuals with profound vision loss. Gait Posture 28, 58–61. doi: 10.1016/j.gaitpost.2007.09.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Reynard, F., Christe, D., and Terrier, P. (2019). Postural control in healthy adults: determinants of trunk sway assessed with a chest-worn accelerometer in 12 quiet standing tasks. PLoS One 14:e0211051. doi: 10.1371/journal.pone.0211051

PubMed Abstract | CrossRef Full Text | Google Scholar

Rosenthal, R., Cooper, H., and Hedges, L. (1994). Parametric measures of effect size. Handb. Res. Synth. 621, 231–244.

Google Scholar

Saffer, M., Kiemel, T., and Jeka, J. (2008). Coherence analysis of muscle activity during quiet stance. Exp. Brain Res. 185, 215–226.

Google Scholar

Shumway-Cook, A. (2011). Motor control. Alphen aan den Rijn: Wolters Kluwer press.

Google Scholar

Shumway-Cook, A., and Horak, F. B. (1986). Assessing the influence of sensory interaction of balance. Suggestion from the field. Phys. Ther. 66, 1548–1550. doi: 10.1093/ptj/66.10.1548

PubMed Abstract | CrossRef Full Text | Google Scholar

Suttanon, P., Hill, K. D., Dodd, K. J., and Said, C. M. (2011). Retest reliability of balance and mobility measurements in people with mild to moderate Alzheimer’s disease. Int. Psychogeriatr. 23, 1152–1159. doi: 10.1017/s1041610211000639

PubMed Abstract | CrossRef Full Text | Google Scholar

Thomas, S. D., Hathaway, D. K., and Arheart, K. L. (1992). Face validity. West. J. Nurs. Res. 14, 109–112.

Google Scholar

Trueblood, P. R., Rivera, M., Lopez, C., Bentley, C., and Wubenhorst, N. (2018). Age-based normative data for a computerized dynamic posturography system that uses a virtual visual surround environment. Acta Otolaryngol. 138, 597–602. doi: 10.1080/00016489.2018.1429653

PubMed Abstract | CrossRef Full Text | Google Scholar

Visser, J. E., Carpenter, M. G., Van Der Kooij, H., and Bloem, B. R. (2008). The clinical utility of posturography. Clin. Neurophysiol. 119, 2424–2436.

Google Scholar

Wittstein, M. W., Crider, A., Mastrocola, S., and Gonzalez, M. G. (2020). Use of virtual reality to assess dynamic posturography and sensory organization: instrument validation study. JMIR Serious Games 8:e19580. doi: 10.2196/19580

PubMed Abstract | CrossRef Full Text | Google Scholar