Ready Medic One: A Feasibility Study of a Semi-Autonomous Virtual Reality Trauma Simulator

Introduction

Immersive virtual reality (VR) can provide portable, high-fidelity training options for high-acuity, low-opportunity (HALO) medical scenarios. However, effective training for HALO clinical situations can be resource-intensive with barriers including prohibitive cost of high-fidelity equipment, lack of scenario reality, and need for significant faculty training and support. VR technologies have rapidly advanced, allowing the creation of high-fidelity immersive environments without the need for expensive mannequins and dedicated simulation space and faculty. VR systems can now utilize headsets paired with controllers that allow interaction with the environment and some haptic feedback. These systems are powered by a gaming laptop with minimal space requirements and do not require internet connectivity. Several barriers have historically limited the widespread utilization of VR in medical training, including developing the necessary VR training programs, the technical knowledge required for the use, cost-effectiveness, and the educational value of the scenarios.

The surgical subspecialties have demonstrated VR platforms are useful for honing procedurally focused skills (Grantcharov et al., 2004; Gallagher et al., 2013; Huber et al., 2017; Pulijala et al., 2018). Additionally, several studies have recently suggested that learner performance in virtual training environments is at least as effective as a traditional simulation in scenarios more applicable to Emergency Medicine, including Advanced Trauma Life Support (ATLS), team management, and oral board performance (Vincent et al., 2008; Luigi Ingrassia et al., 2015; McGrath et al., 2017; Pourmand et al., 2017). VR simulation seems particularly valuable in training for HALO events that military medical providers must prepare for, such as massive trauma and mass casualty events.

Both traditional mannequin-based and VR medical simulations are valuable for medical training. VR simulators appear to provide a promising learning strategy over time for refreshing clinical skills and training courses; given the flexibility and practicality, VR simulation requires less time for learners to complete the training and is more cost-effective as fewer resources are required for the setup such as space and faculty workload (Erlinger et al., 2019; Katz et al., 2020). Due to the feeling of immersion and realism of the environment and simulation, other studies also indicate that people training in VR improve cognitive learning and have lower performance errors and higher accuracy than other training platforms in different medical groups based on their individual and collective needs (Vincent et al., 2008; Pourmand et al., 2017; Samadbeik et al., 2018).

Almost every medical simulation needs at least one experienced facilitator, a simulation technologist, and various equipment. The facilitator develops learning objectives, identifies the best equipment or simulation modality to execute these objectives, develops a sense of realism through narration, and provides feedback. The simulation technologist assists in facilitating equipment, physiologic inputs, and simulation modalities to reach the learning objectives. For example, in an airway obstruction case, a facilitator could have a learning objective to “appropriately set initial ventilator settings,” which would require multiple simulation modalities to include a ventilator and a manikin. Sometimes, a ventilator or manikin may not be available, in which case the facilitator may be relegated to an “imagined” ventilator. The facilitator may smooth gaps in equipment availability through statements like, “You now have a ventilator. What settings would you like to enter?” The learner then has to imagine a ventilator is present. The instructor then has to integrate the learner’s response (ventilator settings) into their own experience and understanding of pathophysiology to provide vital signs and patient exam information back to the learner. Finally, the facilitator watches the learner’s performance and provides targeted learning points and feedback. A metaphor would require an experienced pilot to stand with a student pilot next to a flight simulator. The experienced pilot must describe the wind speed, current altitude, and what the clouds look like to the student. And the student has to tell the experienced pilot what they are doing with their rudder pedals. The experienced pilot then has to use their experience and understanding of physics to determine how that change would impact the plane’s course. While this facilitator-supported method of training still provides valuable training, it is unscalable. In our collective experiences, the facilitator’s role in immersive simulation events is significantly time-consuming, and the equipment needed to simulate the experience may be logistically challenging.

We collaboratively developed a virtual reality simulator that offers immediate feedback through real-time patient physiologic responses and overall grading in training scenarios without requiring an instructor or facilitator. This was completed through the support of the Telemedicine and Advanced Technology Research Center and our technical collaborators. We describe the collaborative development of this simulator in a separate article (Couperus et al., 2020). The available simulations are based on the highest mortality battlefield injuries: massive hemorrhage, tension pneumothorax, and airway obstruction (Couperus et al., 2020). In addition, multiple VR simulators are being utilized and studied; however, this VR simulator provides crucial training opportunities for military health professionals for complex procedures such as cricothyrotomy that are not encountered with high frequency throughout training education (Kaufmann and Liu, 2001; Davis and Warrington, 2021). Furthermore, to our knowledge, this is the only virtual reality trauma management simulator that integrates real-time physiology and feedback without an instructor on a platform that does not require internet connectivity. We are aware of others reportedly in active development, but none that we have seen in operation, used, or seen deployed at scale. Several studies suggested that virtual reality (VR) simulations may be associated with more headaches, dizziness, and blurred vision than augmented reality or tablet-based learning (Lewis and Griffin, 1997; Moro et al., 2017).

This pilot study evaluates the feasibility of using a novel VR trauma simulator that can function without an instructor, assessing potential challenges with the technology, perceived realism of the simulation, side effects experienced while completing the simulations, and overall perception of training utility from end-users.

Methods

This was a single-center prospective cohort study. The study protocol was reviewed and approved by the Madigan Army Medical Center Institutional Review Board. Emergency medicine residents, attendings, medical students, physician assistants, Army medics, and nurses were recruited to participate in this study by word of mouth, posters, and e-mail. Exclusion criteria for the study included any physical limitations preventing the use of VR (i.e., Inability to grasp controllers or significant visual impairment) as well as seizure disorders or severe recurrent vertigo.

Participants were informed of the risks of taking part in this study and that they may stop participation at any time before or during the simulation. Participants completed a presurvey (see Supplementary Appendix S1) including age, gender, years of training, the medical profession, and the amount of experience with video games (both traditional and VR-based), experience with 2D medical simulations (BLS, ACLS, etc.), and the participant’s perceived experience with medical simulation using a standard likert (1–10) scale. One of the investigators provided participants with standardized instructions on use of the VR headset and handheld controllers (approximately 5 min). The primary objectives of this orientation were to familiarize participants with use of the equipment and placement of buttons on the handheld controllers that would be needed to locomote and retrieve items and perform actions within the simulator. The investigator then monitored the participant to ensure safe use of the physical space while the participant independently completed first a 10-min virtual orientation to the trauma room and actions within the simulator followed by a 5-min, self-directed trauma simulation involving either a massive hemorrhage, tension pneumothorax, or airway obstruction case (Figure 7). While in the scenario, in response to physical exam findings and real time vital signs, participants could perform a variety of actions themselves or direct a team member (with direct gaze or use of virtual handheld tablet) to help them perform actions such as placing an IV and tourniquet, drawing labs, performing needle decompression or chest tube, intubating, and administering blood and medications (Table 1). In addition, participants could utilize the virtual handheld tablet to order and review lab results and chest x-ray images. If participants needed to stop the simulation due to side effects, they took the post-simulation survey at that point. Otherwise, the participants took the survey after completing the self-directed simulation. The follow-up survey (see Supplementary Appendix S1) asked questions regarding the time it took until participants experienced side effects, side effects experienced, and the participant’s perception and experience with the simulator using a standard likert scale.

TABLE 1

TABLE 1. Summary of actions available to participants while in the scenario(s). A total of 150 unique actions were available and tracked.

The participants completed the simulation in the first version of the Trauma Simulator (2019), programmed by Exonicus, Inc (Seattle, WA., Riga, Latvia). The simulations were presented using Microsoft Mixed Reality headsets/controllers shown in Figure 1A (Redmond, WA). The headset was paired with either an Alienware 17 Gaming Laptop (i-9 Series Processor, Nvidia GeForce GTX 10 Series (1080) graphics card), or a Razor Blade 15 seen in Figure 1B (Intel Core i7-8750H CPU @2.2GHz, 6 Core, NVIDIA GeForce RTX 2070 Max-Q). The simulator setup and screenshots from within the VR environment are demonstrated in Figures 2–7.

FIGURE 1

FIGURE 1. VR Simulator Setup: (A) Microsoft Mixed Reality headset and joysticks and (B) Razor Blade 15 laptop with Intel Core i7-8750H CPU @2.2GHz, 6 Core, NVIDIA GeForce RTX 2070 Max-Q (B).

FIGURE 2

FIGURE 2. The initial room the user enters, these doors act as a menu to enter a tutorial, learning mode, or unguided interactive simulation.

FIGURE 3

FIGURE 3. The virtual environment was created using a photogrammetry 3D scan of the Madigan Army Medical Center trauma bay, a representative image from that scan is shown in this panel.

FIGURE 4

FIGURE 4. A screenshot of the learners view in the massive external hemorrhage simulation. The items on the wall and table are used to interact with the patient to complete assessments, obtain vitals, and treat injuries.

FIGURE 5

FIGURE 5. A screenshot of a medic using a bag-valve-mask on a patient with internal hemorrhage. This is one of two non-player characters that are present. The learner gives commands to these characters through point and click menus and via a hand tablet interface.

FIGURE 6

FIGURE 6. The simulator works without an instructor and does not need an internet connection. The authors took a headset and laptop on a deployment to Iraq. In this panel, a combat medic can be seen using a beta version of the VR simulator in 2019.

FIGURE 7

FIGURE 7. Flowchart of participant experience.

Data was collected from the surveys and reviewed for accurate entry by two investigators. Results were analyzed using descriptive statistics, including subgroup analyses based on age range, gender, level of training/education, and degree of previous experience with VR technology. Categorical survey responses were summarized using percentages and analyzed using Chi-Squared tests or Fisher’s Exact test. Continuous variables were summarized with medians and interquartile ranges and were analyzed using Wilcoxon’s Test. The statistical tests selected were chosen based on standard use definitions for sample size and data categories. The raw data set may be made available upon request to the corresponding author. While the study’s main goal was to evaluate overall feasibility, we did complete significance testing evaluating participant characteristics associated with a likert score greater or less than (8/10) for the question. “I would support further use of this technology in my training.” (Figure 8) Significance for results was established when p-values were less than 0.05. We appreciate there is considerable debate around the use of Likert scores for inferential statistical analysis. In addition, given multiple statistical tests are performed, p-value significance correction methods, such as the Bonferroni correction, would change the significance threshold. Given this fact, and the study design, the data should be interpreted looking for useful correlations rather than causation. The statistical analysis was performed using JMP v13.2 (SAS Corp, Cary NC).

FIGURE 8

FIGURE 8. Range of participant responses (1–10 Likert scale) to survey statement “I would support further use of this technology in my training”.

Results

A total of 17 participants were enrolled, completed the simulations, and completed the pre and post-surveys. The participant demographics are summarized in Table 2. The participants were predominantly male (82%), representing all clinical roles from an emergency department, with nurses (35%) and medics (24%) serving as the majority of the respondents.

TABLE 2

TABLE 2. Demographics of 17 participants. Values are numbers (rounded percentages) of subjects.

The structured pre and post-survey data and analysis evaluating characteristics and responses of participants who indicated supporting further similar training (likert greater and less than 8/10) are presented in Table 3. The participants had a wide range of years (yrs) of experience, 2–33 years, with an average of 8.3 years. Individuals who supported further use of VR had an average (AV) of 6.3 years experience (SD 4.0, 95% CI 3.9, 8.7) versus those that did not support further use who maintained an average of 14.8years experience (SD 12.2, 95% CI -4.0, 34) (p-value 0.0306). Overall, participants indicated relatively little experience with VR (AV 3.4/10, SD 3.0, 95% CI 1.8, 4.9). Participants showed moderate experience using (and appreciation) of traditional mannequin-based medical simulation with average likert responses of 6.8 and 7.5, respectively. There were various prior experiences using computer-based medical training/teaching programs (AV 5.0, SD 2.1, CI 3.9, 6.1), those who support further use of VR (AV 5.7, SD 1.0, 95% CI 0.91, 4.1) versus individuals who did not support further use (AV 2.5, SD 1.7, 95% CI 4.7, 6.9) (p-value 0.0049). Interestingly, individuals who supported further use indicated they had found prior computer training more helpful (AV 6.8, SD 1.2, 95% CI 4.7, 6.8) versus those who did not support further use (AV 4.0, SD 2.0, 95% CI 0.91, 4.1) (p-value 0.0077).

TABLE 3

TABLE 3. Results and data analysis. The analysis evaluates responses for statistically different characteristics in participants that answered “I would support further use of this technology in my training” greater than 8 on a 1–10 likert versus less than 8. Greater than 8 was associated with (I support further use) and less than 8 was associated with (I do not support further use).

Participants indicated realistic changes in hemodynamics and response to clinical interventions (AV 7.4, SD 2.1, 95% CI 6.3, 8.5), in those who supported further use (AV 8.15, SD 1.46, 95% CI 7.3, 9.0) versus those who did not support further use (AV 4.33, SD 1.15, 95% CI 1.5, 7.2) (p-value 0.0080). Overall, participants indicated the training environment felt realistic (AV 8.3, SD 1.4, 95% CI 7.6, 8.0) in those who supported further use (AV 8.8, SD 0.90, 95% CI 8.3, 9.3) versus those who did not support further use (AV 6.5, SD 1.4 95% CI 7.6, 8.0) (p-value 0.0023). Some participants found the headset challenging to use (AV 2.9, SD 1.4, 95% CI 1.9, 3.3), in those who supported further use (AV 2.3, SD 1.4, 95% CI 1.4, 3.2) versus those who did not support further use (AV 3.5, SD 0.58, 95% CI 2.6, 4.4) (p-value 0.0487). Individuals who did not support further use of VR indicated that a PC/Phone based version would be better than virtual reality, while those who supported further use of the VR training modality indicated VR would be much better (p-value 0.0099) (our small sample size makes this calculation statistically fragile). Several participants experienced side effects (41%), the majority indicated mild nausea (35%), mild headache (6%), and mild dizziness (6%). Three participants (18%) stopped early due to side effects. One participant stopped 10 min into the scenario, and two participants after 20 min. The individual who stopped after 10 min did not support further use; however, the two individuals who stopped after 20 min supported further use of the training modality. Further information on side effects and associated demographics are represented in Table 4.

TABLE 4

TABLE 4. Side effect profile experienced by users.

Technical difficulties were evaluated through likert (Table 3) and free text questions (Tables 5, 6). Overall, participants indicated they had sufficient orientation to the technology and scenario (AV 9.2, SD 1.2, 95% CI 8.5, 9.8), experienced technical difficulties (AV 2.94.

TABLE 5

TABLE 5. Participant answers (post-survey) “please list any other technical difficulties or suggestions.” (direct quotes with spelling corrections only)

TABLE 6

TABLE 6. Participant answers (post-survey) “what is the one most difficult thing about using this simulator?” (direct quotes with spelling corrections only)

SD 2.05, 95% CI 1.9, 4.0), and had difficulty navigating with the joysticks (AV 3.0, SD 1.8, 95% CI 2.1, 3.9). Analysis of the free-text responses indicate some challenges learning the controls, lack of haptic feedback, and for participants with glasses challenges wearing the headset.

Overall, participants reported immersive virtual reality would be a useful addition to traditional mannequin-based medical simulation (AV 8.4, SD 1.8, 95% CI 7.4, 3.9), in those who supported further use (AV 8.9, SD 1.6, 95% CI 8.0, 9.9) versus those who did not support further use (AV 6.5, SD 1.0, 95% CI 4.9, 8.1) (p-value 0.0097). Participants indicated support for further use of this technology in training (AV 9.3, SD 0.99, 95% CI 8.8, 9.8). Free-text questions were used to evaluate any perceived positives about the simulation experience (Table 7). Participants indicated they appreciated the hands-on nature of the training, that it prompted critical thinking, was useful practice, and felt realistic. Lastly, participants were asked what other instructorless VR cases might be useful (Table 8) and they indicated future scenarios could include air/land transports, sick medical patients, prolonged field care, and individual procedures such as foley catheter placement and intubation.

TABLE 7

TABLE 7. Participant answers (post-survey) “what is one thing you found most useful/helpful about this simulation?” (direct quotes with spelling corrections only)

TABLE 8

TABLE 8. Participant answers (post-survey) “If you could have a virtual reality simulator for other medical scenarios, what would they be?” (direct quotes with spelling corrections only)

Discussion

The results of this pilot study indicate it may be possible to create realistic dynamic VR simulations that function without an instructor, and that medical personnel may be supportive of integrating VR technology into medical education, particularly those earlier in their career. Participants with fewer years of medical experience and those familiar with PC/Phone simulation-based training were more likely to favor using VR technology. They also tended to have a better experience with the VR system and found the VR room more accurately imitated the trauma bay and had fewer difficulties operating the headset. The reasons for these associations are not clear; however, a general comfort level with computer-based training and video games due to adoption at an earlier age may explain a more positive attitude toward VR among younger participants. The authors anticipate VR is more likely to be accepted by learners as it evolves, becoming more immersive and realistic.

This study found that older subjects were more likely to experience side effects and were less likely to support VR technology being implemented into medical education. This is consistent with other research that found that elderly users were more likely to experience nausea with immersive VR platforms (Plechatá et al., 2019). A screen-based PC version of the trauma simulator was developed/piloted to allow patients prone to side effects to still obtain the benefits of virtual medical simulation because of these findings (data was not collected on use of this system). Participants appeared to perceive the simulations as valuable and a useful adjunct. This is consistent with previously published virtual reality studies (Grantcharov et al., 2004; Gallagher et al., 2013; Huber et al., 2017; Pulijala et al., 2018; Plechatá et al., 2019).

Several participants had some sort of side effect (41%), while most were mild, and participants still supported the use of the training modality (even those that stopped due to side effects), this number is high. This could impact the general usability and scalability of this training platform. The side effects may be related to the scrolling locomotion method used in the simulator. With this method of movement, there is a continuous “scrolling motion” rather than a “teleportation.” Several other studies have shown that there is a disconnect between the users’ balance and the visual sense that occurs when using locomotion to move about a virtual environment (Boletsis and Cedergren, 2019; Saredakis et al., 2020). This method was required for the wired headset and the size of the environment in our study. Others have suggested using a wireless headset that allows self-movement in the virtual world or the teleportation locomotion method for a more comfortable experience (Boletsis and Cedergren, 2019). At the time of this study (2018–2019) there was no other backpack-sized portable system (other than the Microsoft Mixed Reality headset) that would allow inside out tracking and the computing power required to run the physiology engine (off the network). Since that time, new commercial off the shelf streaming technologies allowing a wireless presentation of the same content have been developed and may decrease the rate of side effects.

This study had several limitations. Participants were recruited using convenience sampling, so it is possible that more individuals interested in the technology agreed to be part of the study which led to more favorable responses. This is a limitation of most studies. The distribution of responses to “using gaming systems” (Table 3) may indicate there was an even distribution of people in this regard. The study was completed at a single military treatment facility which may limit generalizability. This limitation likely led to participant demographics being skewed towards mostly male gender with a higher representation of medics and nurses compared to other emergency department roles. The small sample size may have prevented other statistically significant associations from being identified, but we feel it was sufficient to answer the feasibility questions this study posed. This study was not designed to prove superiority, skills transfer, or broad learning outcomes, it was designed to provide incremental information on a developing technology. Information obtained during this study directly guided ongoing software requirement decisions in parallel research efforts. Based on our own experience, we believe this information will help other teams working on similar projects.

Future research could focus on how VR can best augment current training practices, while searching for specific use cases where VR may be superior to traditional learning techniques. We hypothesize that VR training has the potential to improve training efficiency, but to do so it will require an integrated curricular approach with clearly defined learning objectives directed at specific use cases. We also recommend focusing on leveraging the immersive contextual strengths of virtual reality, while acknowledging its limitations in things such as haptics.

Conclusion

Virtual reality is a rapidly developing technology that has the potential to revolutionize the way people learn. This study suggests that VR simulation is a viable way to expose learners to complex trauma cases, and that it may be possible to create realistic dynamic VR simulations that function without an instructor. It also indicates medical personnel may be supportive of integrating VR technology into medical training. This seems most likely for younger individuals, with less experience, who have found computer based medical training helpful in the past, and (for this particular simulation) are a medic, nurse, or resident physician. Future research could focus on methods to minimize side effects (such as teleportation locomotion instead of scrolling), and how VR technology can best augment current training techniques and curricula to maximize training outcomes.

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.

Ethics Statement

The studies involving human participants were reviewed and approved by Regional Health Command-Pacific (RHC-P) IRB. The study participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author Contributions

RL drafted and submitted the IRB protocol, participated in data collection and led data analysis. NW participated in data collection and manuscript development. SY, CG, ZS, CK contributed to the study design, reviewed/edited the manuscript, and provided reviews of the VR training platform. OT reviewed and edited the manuscript and assisted in the study administrative task. KC secure funding, facilitated VR training platform development, and assist with manuscript drafting.

Funding

This study was supported by AMEDD Advanced Medical Technology Initiative Rapid Innovation Funding (AAMTI RIF) (Award number: AAMTI#8509).

Author Disclaimer

The views expressed are those of the authors and do not reflect the official policy of the Department of the Army, the Department of Defense, or the US Government.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors, and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frvir.2021.719656/full#supplementary-material

References

Boletsis, C., and Cedergren, J. E. (2019). VR Locomotion in the New Era of Virtual Reality: An Empirical Comparison of Prevalent Techniques. Adv. Human-Computer Interaction 2019, 1–15. Available at: https://www.hindawi.com/journals/ahci/2019/7420781/. (Accessed May 11, 2021). doi:10.1155/2019/7420781

CrossRef Full Text | Google Scholar

Couperus, K., Young, S., Walsh, R., Kang, C., Skinner, C., Essendrop, R., et al. (2020). Immersive Virtual Reality Medical Simulation: Autonomous Trauma Training Simulator. Cureus 12 (5), e8062. doi:10.7759/cureus.8062

PubMed Abstract | CrossRef Full Text | Google Scholar

Davis, D., and Warrington, S. J. (2021). Simulation Training and Skill Assessment in Emergency Medicine. Treasure Island (FL): StatPearls [Internet]. U.S. National Library of Medicine. Available at: https://www.ncbi.nlm.nih.gov/books/NBK557695/. (Accessed September 28, 2021).

Google Scholar

Erlinger, L., Bartlett, A., and Perez, A. (2019). High-Fidelity Mannequin Simulation versus Virtual Simulation for Recognition of Critical Events by Student Registered Nurse Anesthetists.: Semantic Scholar. aana.com. Available at: https://www.aana.com/docs/default-source/aana-journal-web-documents-1/high-fidelity-mannequin-simulation-versus-virtual-simulation-for-recognition-of-critical-events-by-student-registered-nurse-anesthetists-april-2019.pdf?sfvrsn=39513ada_6. (Accessed September 28, 2021).

Google Scholar

Gallagher, A. G., Seymour, N. E., Jordan-Black, J.-A., Bunting, B. P., McGlade, K., and Satava, R. M. (2013). Prospective, Randomized Assessment of Transfer of Training (ToT) and Transfer Effectiveness Ratio (TER) of Virtual Reality Simulation Training for Laparoscopic Skill Acquisition. Ann. Surg. 257 (6), 1025–1031. doi:10.1097/sla.0b013e318284f658

PubMed Abstract | CrossRef Full Text | Google Scholar

Grantcharov, T. P., Kristiansen, V. B., Bendix, J., Bardram, L., Rosenberg, J., and Funch-Jensen, P. (2004). Randomized Clinical Trial of Virtual Reality Simulation for Laparoscopic Skills Training. Br. J. Surg. 91 (2), 146–150. doi:10.1002/bjs.4407

CrossRef Full Text | Google Scholar

Huber, T., Paschold, M., Hansen, C., Wunderling, T., Lang, H., and Kneist, W. (2017). New Dimensions in Surgical Training: Immersive Virtual Reality Laparoscopic Simulation Exhilarates Surgical Staff. Surg. Endosc. 31 (11), 4472–4477. doi:10.1007/s00464-017-5500-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Katz, D., Shah, R., Kim, E., Chang, P., Shah, A., Adam, L., et al. (2020). Utilization of a Voice-Based Virtual Reality Advanced Cardiac Life Support Team Leader Refresher: Prospective Observational Study. J. Med. Internet Res. 22 (3), e17425. JMIR Publications. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7099400/. (Accessed January 25, 2021). doi:10.2196/17425

PubMed Abstract | CrossRef Full Text | Google Scholar

Kaufmann, C., and Liu, A. (2001). Trauma Training: Virtual Reality Applications. Stud. Health Technol. Inform. 81, 236–241. PMID: 11317747.

PubMed Abstract | Google Scholar

Lewis, C. H., and Griffin, M. J. (1997). Human Factors Consideration in Clinical Applications of Virtual Reality. Stud. Health Technol. Inform. 44, 35–56.

PubMed Abstract | Google Scholar

Luigi Ingrassia, P., Ragazzoni, L., Carenzo, L., Colombo, D., Ripoll Gallardo, A., and Della Corte, F. (2015). Virtual Reality and Live Simulation. Eur. J. Emerg. Med. 22 (2), 121–127. doi:10.1097/mej.0000000000000132

CrossRef Full Text | Google Scholar

McGrath, J. L., Taekman, J. M., Dev, P., Danforth, D. R., Mohan, D., Kman, N., et al. (2017). Using Virtual Reality Simulation Environments to Assess Competence for Emergency Medicine Learners. Acad. Emerg. Med. 25 (2), 186–195. doi:10.1111/acem.13308

PubMed Abstract | CrossRef Full Text | Google Scholar

Moro, C., Štromberga, Z., Raikos, A., and Stirling, A. (2017). The Effectiveness of Virtual and Augmented Reality in Health Sciences and Medical Anatomy. Am. Assoc. Anatomists 10 (6), 549–559. doi:10.1002/ase.1696

CrossRef Full Text | Google Scholar

Plechatá, A., Sahula, V., Fayette, D., and Fajnerová, I. (2019). Age-Related Differences with Immersive and Non-Immersive Virtual Reality in Memory Assessment. Front. Psychol. 10, 1330. doi:10.3389/fpsyg.2019.01330

PubMed Abstract | CrossRef Full Text | Google Scholar

Pourmand, A., Davis, S., Lee, D., Barber, S., and Sikka, N. (2017). Emerging Utility of Virtual Reality as a Multidisciplinary Tool in Clinical Medicine. Games Health J. 6 (5), 263–270. doi:10.1089/g4h.2017.0046

PubMed Abstract | CrossRef Full Text | Google Scholar

Pulijala, Y., Ma, M., Pears, M., Peebles, D., and Ayoub, A. (2018). Effectiveness of Immersive Virtual Reality in Surgical Training-A Randomized Control Trial. J. Oral Maxillofac. Surg. 76 (5), 1065–1072. doi:10.1016/j.joms.2017.10.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Samadbeik, M., Yaaghobi, D., Bastani, P., Abhari, S., Rezaee, R., and Garavand, A. (2018). The Applications of Virtual Reality Technology in Medical Groups Teaching. J. Adv. Med. Educ. Prof. 6 (3), 123–129. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6039818/. (Accessed January 25, 2021).

PubMed Abstract | Google Scholar

Saredakis, D., Szpak, A., Birckhead, B., Keage, H. A. D., Rizzo, A., and Loetscher, T. (2020). Factors Associated with Virtual Reality Sickness in Head-Mounted Displays: A Systematic Review and Meta-Analysis. Front. Hum. Neurosci. 14, 96. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7145389/.(Accessed May 11, 2021). doi:10.3389/fnhum.2020.00096

CrossRef Full Text | Google Scholar

Vincent, D. S., Sherstyuk, A., Burgess, L., and Connolly, K. K. (2008). Teaching Mass Casualty Triage Skills Using Immersive Three-Dimensional Virtual Reality. Acad. Emerg. Med. 15 (11), 1160–1165. doi:10.1111/j.1553-2712.2008.00191.x

PubMed Abstract | CrossRef Full Text | Google Scholar