Comparisons Between First Person Point-of-View 180° Video Virtual Reality Head-Mounted Display and 3D Video Computer Display in Teaching Undergraduate Neuroscience Students Stereotaxic Surgeries

Introduction

Evaluating the Pedagogical Benefits and Limitations of Virtual Reality for Training in the Medical Fields

A human surgical procedure comprises completing consecutive technical steps and the ability to anticipate and to face both anticipated and unanticipated issues when implementing such critical steps, while also maintaining awareness of the patient’s history and characteristics. The pedagogical material and tools that are available for learning the practical parts/aspects of a given surgery were traditionally offered through textbooks, cadaver and animal dissections, and, more recently, video and/or hybrid simulators and virtual reality (VR). However, the VR experience can be presented in a number of ways that may make it more or less immersive to the user/student/learner. For example, true immersive VR applications can serve to emphasize the user’s/student’s/learner’s placement within the experience, its plausibility and sensorimotor contingencies, and the embodiment of illusions that elicit a sense of presence and immersion itself (for review, see Kourtesis et al., 2020; Slater and Sanchez-Vives, 2016; Slater, 2009). Moreover, the immersion can be considered a predominant spectrum that is proportional to the device and software potency (for review, see Kourtesis et al., 2020; Slater and Sanchez-Vives, 2016; Slater, 2009).

There are a number of benefits by which VR applications can enhance student learning, such as creating a stark contrast to the traditional teaching methods and being able to visualize multiple angles of an organ structure to better understand physiological and anatomical concepts (Alfalah et al., 2019), more sensitive feedback for the student learner through haptic feedback to simulate a more realistic or naturally occurring tactile sensorimotor experience near-identical to the real action (Coles et al., 2010; Hudson et al., 2019), and ensuring that consistent with the sensorimotor experience, the software features integrated within the VR serve to truly immerse the student/learner within the context of the learning experience and also provide ample ability to engage in as realistic as possible interactions within the VR environment (Kourtesis et al., 2020). The latter has brought about more cross-user experiences to (co)engage in a shared VR immersive experience with multiple users within the same context as part of the VR environment (Schild et al., 2018) as well as their spatial recognition of the place they occupy within that very VR environment (Slater, 2009).

However, despite these advantages of the VR experience when it comes to facilitating pedagogical instruction of particular medical training concepts, a key concern for both researchers and educators is to ensure the quality and calibration controls and the potential for the adverse side effects that some VR equipment may cause for the user, like cybersickness, that may obscure the ability to properly evaluate the user VR experience and hinder learning outcomes within the VR environment (Kourtesis et al., 2019). Thus, in order for the medical education field to advance both appropriately and meaningfully while limiting extraneous variables that might be evoked by cybersickness or inadequate hardware and software features that would influence learning within the VR environment and negatively influence the student’s VR experience and learning outcomes, the pedagogical design and methodological approach to teaching through VR experiences ought to be carefully considered (for review, see Pottle, 2019; Izard et al., 2018).

It is important to note that despite all of these VR pedagogical approaches, there still remains the practical training of shadowing of a teacher to complement what is to be learned from a given pedagogical approach. Ros et al. (2017) defined a new pedagogical approach with specific technology (Ros and Trives, 2020) to bring students/learners into the surgical experience to “live” in the procedural aspects of the task from the visual perspective of the expert. While this method has been assessed for healthcare professionals at the graduate/professional school levels (Ros et al., 2017; 2020a; 2021), it is yet to be tested on undergraduate students as a means to upskill or reskill the next generation of healthcare providers. Moreover, such pedagogical innovation appeared interesting in terms of teaching healthcare professionals human surgical techniques, and the next logical step was to evaluate whether this approach could also show promise for teaching the technical aspects of animal surgery.

Teaching Undergraduate Neuroscience and Biopsychology Students Stereotaxic Surgery: Textbooks vs. Practical Training

Animal research in the neurosciences has continued to benefit society with respect to the advancement of biomedical translational research that directly contributes to developing treatments for a wide range of developmental and/or acquired neuropathological disorders. However, these advancements also come with resistance from animal rights activists who seek more humane treatment of animals when they are used in and for research or to limit or eliminate their use in research entirely, and as such, the scientific community has continued to adhere to the recommendations from Russell and Burch (1959) for the last 60 years to consciously advance the field using the 3-Rs when conducting animal experimentation (i.e., to identify and use animal replacement, reduction, and refinement alternatives whenever possible to minimize the use of animals in biomedical research). This is consistent with all practices and oversight regulations followed by Institutional Animal Care and Use Committees (IACUCs) that are in accordance with the Guide for the Care and Use of Laboratory Animals (Committee for the Update of the Guide for the Care and Use of Laboratory Animals, Institute for Laboratory Animal Research, Division on Earth and Life Studies, and National Research Council, 2010), the United States Department of Agriculture (USDA) and its regulations that are consistent with the Animal Welfare Act (P.L. 98–544, 1966; P.L. 94–279, 1976), and Public Health Service (PHS) policy (Office of Laboratory Animal Welfare, 2015) as it is applied to the wide range of neuroscience and behavioral research techniques employed in this area of specialty (for review, see Cardon et al., 2012; National Research Council of The National Academies, 2003).

Moreover, veterinary (Pritchett-Corning et al., 2010) and other research–related procedural manuals (Sharp and La Regina, 1998; Bogdanske et al., 2011), and some packaged with digital video/versatile discs (DVDs) to further increase the comprehension of the techniques that are covered through step-by-step procedural demonstrations from trained experts (Bogdanske et al., 2011), have been developed to further reduce animal discomfort during such procedures, which has served as an invaluable set of resources to the respective disciplines. Given the sensitivity of these procedures within the animal sciences, on one level, it is understandable as to why undergraduate- and graduate-level textbooks (e.g., Martin, 1998; Ward, 2015; Prus, 2020) have restricted their explanation of the stereotaxic surgical procedures to an inset box, half a page to a few pages of a description, and/or a static image of the apparatus used to introduce the technique to the novel learner. Conversely, on another level, this limited information may equally hinder the learner’s comprehension and the full scope of the versatility of the neuroscience technique, and as such, its value to the field may be less understood and as a consequence, it might be less used in the field to advance future science. Notably, there was one textbook that tried to visually walk the learner through the stereotaxic technique procedures using photographs with captions, which was more successful (Cooley and Vanderwolf, 1978) than the previously mentioned textbooks. However, the textbook by Cooley and Vanderwolf (1978) is not referred to or used in every undergraduate or graduate neuroscience course when stereotaxic surgery is covered, thereby adding to the student learning gap within the neuroscience curriculum on the comprehension, utility, and value of stereotaxic surgical techniques.

Student’s Personal Views, the 3-Rs, and Academic Learning Gaps

Neuwirth et al. (2018a) reported that depending on students’ personal views as they relate to the 3-Rs and whether they were a traditional or nontraditional undergraduate student, they might approach the learning of animal dissections, neuroanatomy, anatomy and physiology, and other associated dissection laboratories differently. Furthermore, 5–10% of first-year medical students reported that they might experience transient symptoms of post-traumatic stress disorder (PTSD) when conducting cadaver dissections (Bergeron, 2005) and given these reports, the potential for the undergraduate nontraditional learner to experience similar undesired psychological effects of dissection-related courses was predicted to be higher (i.e., ∼20–30% of the undergraduate population; Bergeron, 2005). Moreover, given these issues, Neuwirth et al. (2018b) suggested that students may seek out alternative pedagogical approaches such as learning through the historical context of neuroanatomy. However, learning through a historical context is invaluable, but it may not facilitate the translation of applied learning into procedural skills to conduct neuroscience experimentation properly, skillfully, and with reliable precision. Furthermore, the diversity of nontraditional students (i.e., for review, see Mukherji et al., 2017) may lead to a wide range of prerequisite issues that may further contribute to learning gaps when they enter biopsychology/neuroscience classes from different majors, and one of the ways in which such learning gaps can be closed was proposed to be through online laboratory instruction of the course content (Neuwirth et al., 2018b; 2019). The current coronavirus (COVID-19) pandemic has further highlighted the digital divide amongst the nontraditional undergraduate student population, many of whom are technology insecure (i.e., lack of laptops, stable Wi-Fi, etc.), and in turn, this has further contributed to the widening of the academic learning gaps that have presented educators with both challenges and opportunities for developing innovative pedagogies to address this very issue (Neuwirth et al., 2020). To address such learning gaps, Ros et al. (2020b) has proposed the use of virtual reality stereoscopic 180° video-based first person point-of-view (FPV) technology that employs immersive virtual reality applications (IVRA) to help bring the student/learner closer to the material to shadow and simulate what the instructor is doing by activating their mirror neurons during their FPV-IVRA learning simulation (for review, see Ros et al., 2017; Ros et al., 2020b). It is important to note that this FPV-IVRA requires a certain technical adoption and Wi-Fi to download the Revinax^® Handbook prior to use and may not overcome all learning gaps that technology insecurity may present as an exclusion factor for some groups.

The Effectiveness of the IVRA Based on First Person Point-of-View Video

Considering that learning in the FPV decreases the users’ cognitive load, Fiorella et al. (2017) performed a study comparing two groups which learned technical skills through video in first-person vs. third-person views; they showed that the FPV helped to decrease errors when learning the material by up to 50%. Additionally, the immersive virtual reality (IVR) has also been shown to decrease the users’ cognitive load (Andersen et al., 2016) and further helps the learner to become more engaged. Andersen et al. (2016) did not record in FPV, but Ros et al. (2017) described a method to record the 3D FPV of an expert and subsequently displayed this experience within a VR environment, using a head-mounted display (HMD) to teach students/learners (Ros and Trives, 2020), thereby permitting the unique ability to add complementary pedagogical data within the rest of the environment (i.e., a blended learning environment). They showed that learners/students understood the material better, were able to answer practical questions more accurately, and could reproduce technical procedural skills with 65% less mistakes when using a VR HMD (Ros et al., 2021). They enhanced the student’s/learner’s experience as they were better able to answer practical questions over theoretical questions than a group learning from only a traditional lecture (Ros et al., 2021).

During the first COVID-19 wave, Ros and Neuwirth (2020) released a mobile application to be used with or without a VR HMD (i.e., cardboard versatile 3D glasses) based on the same FPV video approach with additional pedagogical data that was integrated within the environment. This app targeted healthcare professionals to help them learn (i.e., upskill) or refresh their skills (i.e., reskill) while taking care of patients in real time from various specialties, with the primary objective being to manage COVID-19. Subsequently, Ros and Neuwirth (2020) conducted a study to obtain the app users’ feedback. The app was downloaded 12,500 times in one month with a user/learner satisfaction rating of 95%. The results showed that the app users/learners understood the COVID-19 material better, felt more confident in implementing the healthcare procedures, and it further helped them to acquire on-the-job skills (i.e., upskill) to be ready to perform these procedures in the wake of a global pandemic (Ros and Neuwirth, 2020). They thought about it as a pedagogical continuum when building the app, to help people in refreshing/updating their skills (i.e., reskill), and due to the global resource demands to address this unprecedented emergency situation, the app also addressed a real practical issue in providing access to and, to a degree, consistency in on-the-job training for a large number of healthcare professionals (Ros and Neuwirth, 2020). This app further provided the users/learners with the ability to repeatedly view and retrain on any module they felt necessary, which, in turn, also had the advantage of increasing efficiency of a procedural skill and reducing errors when generalized into the real-world healthcare situation (Ros and Neuwirth, 2020). The IVR environment displayed in this way also exists on a desktop as a computer-displayed option (i.e., semi-immersive), but it is yet to be assessed through a systematic study with respect to being used by a teacher as a pedagogical tool for remote instruction.

These procedures have been used before in different contexts and have been shown to be effective in teaching a broad learning group to learn new procedural techniques (Ros et al., 2020a) and under unprecedented situations during the COVID-19 pandemic for quickly educating front-line responders (Ros and Neuwirth, 2020). Thus, the present study sought to use the 180° video VR HMD pedagogical method compared to a 3D video computer display pedagogical method to address the following hypotheses: 1) the 180° video VR HMD would be better for teaching a neuroscience stereotaxic surgical technique in the rat model to bring the student closer to the learning content; 2) both the 180° video VR HMD and the 3D video computer display would equally immerse the student/learner within the procedural technique while also adhering to the recommendations of the 3-Rs to find an alternative pedagogical method to use less animals in neuroscience research to demonstrate such procedures, and 3) both the 180° video VR HMD and the 3D video computer display would have equal value as an alternative pedagogical instructional learning method that may best address academic learning gaps experienced by the student learner in both in-person and distance/remote learning formats.

Methods

Participants

Students were recruited at random from the State University of New York College at Old Westbury (SUNY-OW), whereby participation was voluntary, and the students could terminate the study at any time if they so chose. The study was approved by the SUNY-OW Institutional Review Board (IRB) and the IACUC. The study comprised N = 58 students (n = 27 males and n = 30 females) who were randomly assigned to either the 180° video VR HMD (i.e., FPV through the HMD) group that was run prior to the COVID-19 pandemic (n = 26) or the 3D video computer display (i.e., FPV through the computer screen) group that was run during the COVID-19 pandemic (n = 32). The difference between the two groups was that the 180° video VR HMD IVRA group was presented within an HMD, whereas the 3D video computer display group was presented over the internet through a computer screen with the instructor providing explanations. The participants’ ages were in the range of 21–23 years (n = 4; 6.9%), 24–26 years (n = 30; 51.72%), 27–29 years (n = 13; 22.41%), 30–32 years (n = 0; 0%), 33–35 years (n = 0; 0%), and 36+ years (n = 2; 3.45%), respectively. Of the N = 58 participants, n = 11 (18.97%) openly opposed animal research. Moreover, of the N = 58 participants, n = 12 (20.69%) had never used any form of VR prior to this research study. Furthermore, of the N = 58 participants, n = 5 (8.62%) experienced vertigo more than twice a week prior to participating in the present study. Taken together, the participant sample population comprised a rather diverse and representative sample across a number of demographics typical of an undergraduate student sample population representative of learning about stereotaxic surgery, animal neuroscience, and VR. Additionally, one of the authors (M.R) consented to having their picture taken to demonstrate the student’s/learner’s 180° video VR HMD experience, consistent with prior reports (Ros et al., 2021).

Design and Procedure

The participants were randomly assigned to either the pre–COVID-19 180° video VR HMD group or the during–COVID-19 3D video computer display group, respectively. In the 180° video VR HMD group, the researchers provided the participants with an overview of the study in person and were given informed consent. Once the participants agreed to volunteer for the study, they were provided with an Oculus VR Headset Model MH-A64 (Oculus VR, LLC, Dublin, Ireland) as the HMD and shown how to wear it and understand their surroundings to ensure safety while operating. They were then given a short tutorial (i.e., ∼5 min) on the HMD and how to calibrate and use it. Then they were given access to an immersive experience in the VR platform through the Revinax^® Handbook. This experience was designed according to the Revinax^® method (i.e., using a device worn by the neuroscientist conducting the stereotaxic surgery to record their point of view [POV]; see Figure 1). After the short tutorial, participants were directed to open the stereotaxic surgery learning module and were instructed to turn left to view the Microsoft PowerPoint lecture slides, to turn right to view specific images with refined details for select content as they went through the learning module, to look up to view the different learning modules, and to look down to enter the 180° video VR HMD experience (see Figures 2A–D). Participants viewed the stereotaxic surgery learning module for approximately 20 min while seated in a chair. Once they finished viewing the stereotaxic surgery learning module, the participants completed an anonymous survey through Google Forms that would collect some basic demographic information and assess their perspective of the 180° video VR HMD group and their comprehension of a few questions regarding the learning modules they experienced. The survey comprised six demographic questions, four questions on how participants felt during the VR-experience, nine questions on the learning module outcomes to assess the student’s learning comprehension, four questions on how the VR experience compared to traditional and online lectures, reading the textbook, and watching videos, and four questions on how participants felt the VR experience influenced their learning of the material and whether participants would use the VR experience as an educational tool. For the 3D video computer display group, the overview of the study and the obtaining of informed consent were conducted through Zoom, consistent with remote teaching methods due to the COVID-19 pandemic and its evolving situation (see Figure 2E). Then the same procedures as those for the 180° video VR HMD group were implemented over Zoom (Zoom Video Communications, Inc., San Jose, CA, United States) for the 3D video computer display group; however, the researchers directed the viewing for the participants to ensure that the same experience for all participants could be achieved. In the 3D video computer display group, no participant lost connectivity during the study. Once the 3D video computer display group completed the learning modules, they similarly completed the survey immediately thereafter.

FIGURE 1

FIGURE 1. Actual first person point-of-view (FPV) perspective of the neuroscientist LSN (i.e., white arrow) wearing the point-of-view recording device technology setup (Ros and Trives, 2020). The stereotaxic surgery was recorded from this perspective and then integrated into the Revinax Handbook to be displayed in both the 180° video VR HMD and the 3D video computer display formats for the present study. LSN consented to having his picture taken to demonstrate the FPV.

FIGURE 2

FIGURE 2. 180° video VR HMD perspective of the POV of the student/learner. (A) Learning modules displayed when looking up. (B) 180° video VR HMD audio/video experience when looking down. (C) Microsoft PowerPoint slides displaying the lecture to the left. (D) Additional specific images with the refined details for teaching emphasis. (E) FPV of the student/learner for the comparison 3D video computer display group.

Survey Questionnaire

The survey questionnaire was deployed using Google Forms, and the survey was conducted anonymously and took less than 10 min to complete. The survey questionnaire comprised four specific areas: 1) how participants felt during the VR experience, 2) how participants felt the VR experience influenced their learning of the material, 3) how the VR experience compared to traditional and online lectures, reading the textbook, and watching videos, and 4) whether participants would use the VR experience as an educational tool. For the “How participants felt during the VR-experience” survey section, the following demographic questions were used to categorize the participant sample population: “What is your gender?,” “What is your age range?,” “Do your personal feelings align with opposing animal research?,” “Do you experience vertigo, motion sickness, or movement related nausea more than twice a week?,” and “Have you ever used VR technology before? If so, in what capacity.” For the Likert scale from “0” (not at all) to “10” (definitely), the following questions were asked: “Based on this VR-experience learning method, do you think the VR-experience was too short?,” “Based on the VR-experience learning method, do you think the font/text on the slides to the left side of the screen were informative?,” “Based on the VR-experience learning method, do you think the supplemental diagrams on the right side of the screen were informative?,” and “Based on the VR-experience learning method, do you think the audio and video recordings were informative?”

For the “How participants felt the VR-experience influenced their learning of the material” survey section, the following Likert scale multiple-choice questions were used to compare the two groups: “Which one of these sentences is correct regarding the difference between in-vivo and in-vitro neuroscience techniques?,” “In which quadrant do we give an inter-peritoneal (i.p.) injection to the rat?,” “What is the formula to administer drugs?,” “Why is aseptic preparation important prior to surgery?,” “What is a brain atlas?,” “What is the name and location of the skill used in this video as the surgical reference point?,” “How many electrodes are being used in the study?,” “How are seizures categorized by severity?,” and “If you were to see two seizures with high amplitude in a short amount of time, what seizure category would the animal’s behavior fall under?” For these questions, the percent correct responses were determined, and the percent difference from the 180° video VR HMD was used to determine between-group differences.

For the “How the VR-experience compares to traditional and online lectures, reading the textbook, and watching videos” survey section, the following Likert scale questions from “0” (it is worse/not faster than learning from a traditional lecture/not more attentive than learning from a traditional lecture) to “10” (it is better/much faster than learning from a traditional lecture/more attentive than learning from a traditional lecture) were used to compare the two groups: “How does this VR-experience learning method compare to a traditional in class lecture?,” “How does this VR-experience learning method compare to an online/hybrid class lecture?,” “How does this VR-experience learning method compare to reading from a textbook?,” “How does this VR-experience learning method compare to learning from watching a video from YouTube or other similar internet sources?,” “How did the VR-experience influence your learning the material being covered?,” and “How did the VR-experience influence your attention when learning the material being covered?”

For the “Whether you would use the VR-experience as an educational tool” survey section, the following Likert scale questions from “0” (not at all) to “10” (definitely) were used to compare the two groups: “If you were able to have access to this VR-experience outside of the lecture to study on your own, rate your likelihood of using it?,” “From your VR-experience, how likely would you recommend your university/college to invest into this novel learning resource?,” “From your VR-experience, would you like to have access to more content like this?,” “Do you think this VR-experience learning method, could influence your speed of learning the lecture material?,” “Do you think this VR-experience learning method could influence your ability to retain the lecture information?,” and “Do you think this VR-experience learning method could influence your ability to translate the lecture information into real-world experiences?”

Statistical Analyses

All statistical analyses were carried out using SPSS version 24 (IBM^®, Armonk, NY, United States). The criteria for statistical significance was set to α = ≤ 0.05 with a confidence interval of 95% (CI = 95%). To assess the normality of the participant sample distribution, a Levene’s test was used to assess the equality of variances. Where unequal variances were reported, the degree of freedom was adjusted accordingly to ensure appropriate statistical comparisons where applicable. This was followed by an independent samples t-test that was used to evaluate the differences between the two groups for all multiple-choice questions. For the remaining Likert scale questions, the mean, SD, median, and mode were reported for each group. Finally, a Cohen’s d for unequal n-sizes was used for appropriately determining the effect size of the participant’s rating differences between the two groups, and a Hedge’s g for correcting for effect sizes was used for sample sizes under N = 50, consistent with the statistical methods employed by Harpe (2015) and Lakens (2013).

Results

Participants Feelings During the VR Experience

The participants indicated that on a scale of “0” (not at all) to “10” (definitely), they reported no significant differences between the 180° video VR HMD and the 3D video computer display groups in terms of “Do you think the VR-experience was too short?” t₍₅₆₎ = −0.658, p = 0.513 n/s (Mean_180VRHMD = 3.35, SD_180VRHMD = 2.78, Median_180VRHMD = 3, and Mode_180VRHMD = 0; Mean_3DVCD = 3.88, SD_3DVCD = 3.24, Median_3DVCD = 2.5, and Mode_3DVCD = 2; Figure 3). Additionally, there were no significant differences between the 180° video VR HMD and the 3D video computer display groups for “Do you think the font/text on the slides to the left side of the screen were informative?” t₍₅₆₎ = −0.772, p = 0.444 n/s (Mean_180VRHMD = 8.23, SD_180VRHMD = 1.70, Median_180VRHMD = 8.5, and Mode_180VRHMD = 10; Mean_3DVCD = 8.56, SD_3DVCD = 1.56, Median_3DVCD = 9, and Mode_3DVCD = 10; Figure 3). Moreover, there were no significant differences between the 180° video VR HMD and the 3D video computer display groups for “Do you think the supplemental diagrams on the slides to the right side of the screen were informative?” t₍₅₆₎ = 0.699, p = 0.488 n/s (Mean_180VRHMD = 8.65, SD_180VRHMD = 1.72, Median_180VRHMD = 9, and Mode_180VRHMD = 10; Mean_3DVCD = 8.28, SD_3DVCD = 2.23, Median_3DVCD = 9, and Mode_3DVCD = 10; Figure 3). Furthermore, there were no significant differences between the 180° video VR HMD and the 3D video computer display groups for “Do you think the audio and video recordings were informative?” t₍₅₆₎ = −0.802, p = 0.426 n/s (Mean_180VRHMD = 9.12, SD_180VRHMD = 1.24, Median_180VRHMD = 10, and Mode_180VRHMD = 10; Mean_3DVCD = 9.38, SD_3DVCD = 1.21, Median_3DVCD = 10, and Mode_3DVCD = 10; Figure 3). Taken together, the participants equally valued the way that the left slides, the right slides, and the audio/video information were presented to both the 180° video VR HMD and the 3D video computer display groups, respectively (Figure 3). Interestingly, even though the VR experience lasted approximately 20 min, both the 180° video VR HMD and the 3D video computer display groups felt that the VR experience was not too short; thus, the participants desired longer learning modules through these IVRA pedagogical formats (Figure 3).

FIGURE 3

FIGURE 3. Between-groups participants’ feelings about the VR experience. Regardless of the group (dark gray bars = 180° video VR HMD and light gray bars = 3D video computer display), the participants felt similarly that the VR experience was not too short and that the slides to the left and the right and the audio/video that was provided were informative. Thus, the participants found the presentation of the VR experience in both the 180° video VR HMD and the 3D video computer display formats to be educationally informative. The data are illustrated as the mean ± SEM.

The Influence of the VR Experience on the Participant’s Ability to Learn the Material

The participants indicated that on the multiple-choice questions used to assess their learning of the material, they reported no significant differences between the 180° video VR HMD and the 3D video computer display groups in terms of: “Which one of these sentences is correct regarding the difference between in-vivo and in-vitro neuroscience techniques? t₍₅₆₎ = 1.844, p = 0.071 n/s but showed an emerging trend with a Cohen’s ds = 0.495 for the Effect Size (Figure 4). In contrast, there was a significant difference between the 180° Video VR HMD and the 3D Video Computer Display Groups in terms of “In which quadrant do we give an inter-peritoneal (i.p.) injection to the rat?” t_(41.579) = 3.224, p < 0.001***, a Cohen’s d = 0.903 for the Effect Size, and a Hedge’s g = 1.00 (Figure 4). Moreover, the participants indicated that on the multiple-choice questions used to assess their learning of the material, they reported significant differences between the 180° video VR HMD and the 3D video computer display groups in terms of “What is the formula to administer drugs?” t_(50.401) = −2.6531, p < 0.01**, a Cohen’s d = -0.675 for the Effect Size, and a Hedge’s g = −0.87.; “Why is aseptic preparation important prior to surgery?” t_(36.697) = −1.183, p = 0.244 n/s; “What is a brain atlas?” t₍₅₆₎ = −0.825, p = 0.413 n/s; and “What is the name of the location of the skull used in this video as the surgical point of reference?” t_(49.929) = 2.923, p < 0.01**, a Cohen’s d = 0.743 for the Effect Size, and a Hedge’s g = 0.94 (Figure 4). Furthermore, the participants indicated that on the multiple-choice questions used to assess their learning of the material, they reported significant differences between the 180° video VR HMD and the 3D video computer display groups in terms of “How many electrodes are being use in the study?” t_(55.999) = −1.176, p = 0.244 n/s; “How are seizures categorized by severity?” t₍₅₆₎ = −0.312, p = 0.757 n/s; and “If you were to see two seizures with high amplitude in short amount of time, what seizure category would the animal’s behavior fall under?” t_(25.000) = 2.132, p < 0.005*, a Cohen’s d = 0.637 for the Effect Size, and a Hedge’s g = 1.14 (Figure 4). Taken together, the participants from the 180° video VR HMD group reported higher educational learning outcomes for the following: In vivo vs. in vitro research advantages (24.04%), i.p. injection site (37.50%), skull reference point (29.81%), electrode number (25.96%), and seizure cluster type (15.38%; Figure 4). In contrast, the participants from the 3D video computer display group reported higher educational learning outcomes for the following: drug injection formula (26.68%) and surgical infection risks (8.41%; Figure 4). Interestingly, participants from both groups reported similar but not statistically significant learning outcomes for the following: brain atlas map (6.49% increased difference in the 3D video computer display group) and seizure severity category (2.88% increased difference in the 3D video computer display group; Figure 4).

FIGURE 4

FIGURE 4. Between-groups participants’ learning outcomes assessed. The data show that the 180° video VR HMD experience increased the student’s learning of the in-vivo research advantages, the i.p. injection site (p < 0.01**), the skull reference points (p < 0.01**), the electrode number, and the seizure cluster type (p < 0.05*). In contrast, the 3D video computer display increased the learning of the drug injection formula (p < 0.01**), the surgical infection risks, the brain atlas map, and the seizure severity type. The data are illustrated as the mean ± SEM and the statistical significance is denoted as p < 0.05*, p < 0.01**, and p < 0.001***.

Participants Valued the VR Experience Over Traditional and Online Lectures, Reading the Textbook, and Watching Videos

The participants indicated that on a scale of “0” (it is worse) to “10” (it is better), they reported no significant differences between the 180° video VR HMD and the 3D video computer display groups in terms of “How does this VR-experience learning method compare to a traditional in class lecture?” t₍₅₆₎ = 0.173, p = 0.863 n/s (Mean_180VRHMD = 7.35, SD_180VRHMD = 2.08, Median_180VRHMD = 7.5, and Mode_180VRHMD = 8; Mean_3DVCD = 7.25, SD_3DVCD = 2.13, Median_3DVCD = 7, and Mode_3DVCD = 10; Figure 5). Additionally, there were no significant differences between the 180° video VR HMD and the 3D video computer display groups for “How does this VR-experience learning method compare to an online/hybrid class lecture?” t_(52.410) = 1.383, p = 0.173 n/s (Mean_180VRHMD = 8.69, SD_180VRHMD = 1.44, Median_180VRHMD = 9, and Mode_180VRHMD = 10; Mean_3DVCD = 8.00, SD_3DVCD = 2.34, Median_3DVCD = 9, and Mode_3DVCD = 10; Figure 5). Moreover, there were no significant differences between the 180° video VR HMD and the 3D video computer display groups for “How does this VR-experience learning method compare to reading from a textbook?” t₍₅₆₎ = −1.876, p = 0.066 n/s (Mean_180VRHMD = 8.15, SD_180VRHMD = 1.71, Median_180VRHMD = 8, and Mode_180VRHMD = 10; Mean_3DVCD = 9.00, SD_3DVCD = 1.70, Median_3DVCD = 10, and Mode_3DVCD = 10; Figure 5). Furthermore, there were no significant differences between the 180° video VR HMD and the 3D video computer display groups for “How does this VR-experience learning method compare to learning from watching a video from YouTube or other similar internet sources?” t₍₅₆₎ = 0.271, p = 0.787 n/s (Mean_180VRHMD = 8.08, SD_180VRHMD = 2.31, Median_180VRHMD = 8.5, and Mode_180VRHMD = 10; Mean_3DVCD = 7.91, SD_3DVCD = 2.44, Median_3DVCD = 9, and Mode_3DVCD = 10; Figure 5). Taken together, the participants equally valued the IVRA, regardless of format (i.e., 180° video VR HMD or 3D video computer display over traditional in-class lectures, online/hybrid lectures, reading from a textbook, and watching a video/YouTube video, respectively) (Figure 5).

FIGURE 5

FIGURE 5. Between-groups participants’ reporting of the VR experience compared to other learning methods. The data show that both the 180° video VR HMD (dark gray bars) and the 3D video computer display (light gray bars) experiences were reported with equally high ratings by participants, but they did not differ significantly from one another. However, the overall finding was that the students equally preferred the 180° video VR HMD and the 3D video computer display over a traditional lecture, an online hybrid lecture, a textbook, and a video/YouTube. The data are illustrated as the mean ± SEM.

Participants Valued the VR Experience as an Educational Tool

The participants indicated that on a scale of “0” (not faster than learning from a traditional lecture) to “10” (much faster than learning from a traditional lecture), they reported no significant differences between the 180° video VR HMD and the 3D video computer display groups in terms of “How did the VR-experience influence your learning the material being covered?” t₍₅₆₎ = −0.789, p = 0.434 n/s (Mean_180VRHMD = 6.92, SD_180VRHMD = 2.71, Median_180VRHMD = 7, and Mode_180VRHMD = 8; Mean_3DVCD = 7.44, SD_3DVCD = 2.26, Median_3DVCD = 7, and Mode_3DVCD = 8; Figure 6). Additionally, the participants indicated that on a scale of “0” (not more attentive than learning from a traditional lecture) to “10” (more attentive than learning from a traditional lecture), there were no significant differences between the 180° video VR HMD and the 3D video computer display groups for “How did the VR-experience influence your attention when learning the material being covered?” t₍₅₆₎ = −0.026, p = 0.980 n/s (Mean_180VRHMD = 7.92, SD_180VRHMD = 2.13, Median_180VRHMD = 8, and Mode_180VRHMD = 10; Mean_3DVCD = 7.94, SD_3DVCD = 2.14, Median_3DVCD = 8.5, and Mode_3DVCD = 10; Figure 6). Moreover, the participants indicated that on a scale of “0” (not at all) to “10” (definitely), there were no significant differences between the 180° video VR HMD and the 3D video computer display groups for “If you were able to have access to this VR-experience outside of the lecture to study on your own, rate your likelihood of using it?” t₍₅₆₎ = 0.516, p = 0.608 n/s (Mean_180VRHMD = 8.27, SD_180VRHMD = 2.07, Median_180VRHMD = 9, and Mode_180VRHMD = 10; Mean_3DVCD = 7.97, SD_3DVCD = 2.31, Median_3DVCD = 8.5, and Mode_3DVCD = 10; Figure 6). Furthermore, the participants indicated that on a scale of “0” (not at all) to “10” (definitely), there were no significant differences between the 180° video VR HMD and the 3D video computer display groups for “From your VR-experience, how likely would you recommend your university/college to invest into this novel learning resource?” t_(54.762) = 0.486, p = 0.629 n/s (Mean_180VRHMD = 9.00, SD_180VRHMD = 1.20, Median_180VRHMD = 9.5, and Mode_180VRHMD = 10; Mean_3DVCD = 8.81, SD_3DVCD = 1.73, Median_3DVCD = 10, and Mode_3DVCD = 10; Figure 6). Taken together, the participants equally valued the IVRA, regardless of format (i.e., 180° video VR HMD or 3D video computer display), to influence their learning, to influence their attention, to motivate their self-study independent of in-class learning, and to recommend that their university/college invest in such pedagogical technology/teaching approaches, respectively (Figure 6).

FIGURE 6

FIGURE 6. Between-groups participants’ reporting of their desire to use the VR experience to enhance their learning. The data show that both the 180° video VR HMD (dark gray bars) and the 3D video computer display (light gray bars) experiences were reported with equally high ratings by participants, but they did not differ significantly from one another. However, the overall finding was that the students equally preferred the 180° video VR HMD and the 3D video computer display to increase their learning, attention, and self-study of the material, and they would recommend that their college/university use such a pedagogical tool for their education. The data are illustrated as the mean ± SEM.

Discussion

Students Self-Report That the 180° Video VR HMD and the 3D Video Computer Display Address the 3-Rs

Consistent with the 3-Rs proposed by Russell and Burch (1959), the 180° video VR HMD and the 3D video computer display experience was intended to reduce the use of animals to teach a neuroscience technique to undergraduate students. The ability to review and revisit specific components of the learning modules through the VR experience helped to address replacement- and refinement-related issues. Thus, the 180° video VR HMD and the 3D video computer display offer unique alternative solutions consistent with the 3-Rs in a modern technological format, with broad pedagogical options for the student/learner both through in-person and distance/remote learning formats. The self-report data from the 180° video VR HMD and the 3D video computer display groups equally indicated (i.e., average Likert scale ratings of 9.35 and 8.88, respectively) that the participants felt that the VR experience addressed their animal welfare concerns, as they saw the value that this pedagogical tool offered by reducing the need to use more animals to demonstrate the same kinds of research techniques in the neurosciences by displaying them as simulated VR experiences.

Moreover, the alternative learning textbooks regarding such surgeries (Cooley and Venderwolf, 1978; Lawrence, 1997; Perrett, 1997; Martin, 1998; Ward, 2015) and brain atlases (Lim et al., 1960; Snider and Niemer, 1961; Gergen and MacLean, 1962; DeLucchi et al., 1965; Karten and Hodos, 1967; Montemurro and Dukelow, 1972; Slotnick and Leonard, 1975; Paxinos, 1985a, 1985b; Zilles, 1985; Bradley and Bydder, 1989; DeArmond et al., 1989; Paxinos and Watson, 2005; Ramachandra, 2011) as well as neuroanatomy (Kent, 1965; Angevine and Cotman, 1981; Foster, 1988; Gilman and Winans-Newman, 1991; O’ Rahilly and Müller, 1994; Martin, 1996; Bayer and Altman, 2002; Gould and Brueckner, 2008; Haines, 2008; Woolsey et al., 2008; Jacobson et al., 2018) often fail to offer a very clear understanding of the photographs and precisely how they align with the surgical and stereotaxic procedures and furthermore, how they relate to neuroanatomical connections, thereby adding to the academic achievement gap.

Furthermore, these textbooks are often rather old (i.e., some were published half a century ago) and as such, are difficult to find as both a methodological and/or educational resource within the neuroscience/biopsychology field to facilitate this kind of student learning. The rarity of these textbooks may unfortunately create a need to repeat the type of work done on additional animals to produce a current and more readily available resource, unless the authors and publishers permit e-book versions of the original; thus, this very controversial situation may be in direct opposition to the 3-Rs and perhaps the 180° video VR HMD and the 3D video computer display could be used to display select content of key concepts to circumvent these issues in the field. Taken together, the 180° video VR HMD and the 3D video computer display pedagogical tools can help to bridge structure–function relationships, bring students into an immersive environment with respect to the learning content with a reduced number of animals (i.e., in this study’s example, a single rat), and furthermore, this may help to close the academic achievement gap by clarifying critical procedural aspects of key research techniques that help to facilitate the student’s/learner’s concept formation of the foundational skills required for achieving maximal learning comprehension of stereotaxic surgery.

Student Self-Report and Anecdotal Themes on the Value of the 180° Video VR HMD and the 3D Video Computer Display

The self-reports compiled from the representative undergraduate student sample showed a clear interest in the use of the 180° video VR HMD and the 3D video computer display experiences being integrated within the subjects that they learned within the neuroscience/biopsychology curriculum. The students felt that both the VR experiences were not too short and that the information displayed on the left of the VR environment (i.e., the Microsoft PowerPoint lecture slides), and the right of the VR environment (i.e., the emphasis slides that redirected students to key learning concepts from the learning modules), as well as the clarity of the narration and presentation of the VR experience were informative (Figure 3). This indicated that the VR experience that was designed for this study captured the attention of the target audience (i.e., the student/learner) and motivated them to engage in its usage as an alternative pedagogical learning format.

Moreover, the students indicated that when comparing the 180° video VR HMD and the 3D video computer display experiences with other learning formats that they had been taught (i.e., traditional lecture, online/hybrid lecture, textbook, and video/YouTube), they preferred both VR experiences over the other learning formats (Figure 5). These results were telling as online/hybrid lectures often use videos and other types of interactive learning to engage students through distance learning formats, and in traditional lectures, videos/YouTube channels are used sometimes to demonstrate and reinforce key learning concepts, which could have resulted in some students feeling that these VR experiences would be no different. Interestingly, it was observed that both the 180° video VR HMD and the 3D video computer display experiences were preferred over these other learning formats, and anecdotally, students thematically indicated that the immersive experience was most effective for them by stating the following: “It is a more interactive, engaging form of learning which requires the full attention of the user. It also allows for viewing real-life scenarios that are otherwise discussed during traditional lectures,” “I think the VR- Experience can be integrated with a class. As a professor teaches, he/she can demonstrate hard concepts and procedures in VR which would make it much easier for the students to understand. The students will be immersed into the subject matter and it can definitely help attention and memory retention as they are having a heightened sensory experience as they are learning,” “I feel that the VR experience will help with skills that require hands-on learning. Based on my experience, a traditional lecture will teach you a hands-on skill once and it is up to the student to memorize and take-notes in order to mimic the skill again. VR will allow students to have access to view and re-learn this skill properly and efficiently,” and “I believe that the software would be able to keep students interested in lecture and give them a similar real life experience to see everything up close which can help conveying information.”

In contrast, some students questioned whether these VR formats would replace other learning formats and felt that rather, they might better serve to further reinforce them, as the following statements indicate: “I’m not sure if it would be able to replace traditional lectures. However, it would make a good reinforcer or be a useful tool to go further in-depth on certain subjects/get hands on experience,” “Traditional lectures still hold better than this however, it’s a good additive measure,” “Much like a video the VR-experience provided real life examples of the material allowing for a better understanding of the material,” “I don’t think it could substitute a traditional lecture, but it could substitute a more hands on lecture,” and “The VR experience would be more like a hands on type of class, not actually my hands but feel like my hands.” These comments suggest that a subpopulation of students still prefer other learning formats but showed interest in adding the 180° video VR HMD and/or the 3D video computer display to them to further enhance, supplement, or expand their learning of key concepts of their curriculum.

Unique Teaching Approaches During and Post COVID-19: 180° Video VR HMD and 3D Video Computer Display Pedagogy Solutions to Upskill and Reskill Students

The challenges presented to both instructors and students during the evolving COVID-19 pandemic have truly challenged the ways in which pedagogy is used to regain, sustain, and engage students’ attention, given the inherent challenges associated with remote learning (Eadens and Eadens, 2021; Gelles et al., 2021; Lysenko and Tokareva, 2021; Mamun et al., 2021; McCormak et al., 2021; O’Sullivan et al., 2021). This has placed an enormous amount of pressure on instructors to find novel, unique, and creative ways to engage their students to maintain their interests and academic momentum from prior to COVID-19 until the situation becomes resolved. This was the impetus for the current study to similarly identify ways for instructors to think differently to upskill and reskill students; rather than bringing the curriculum to the students through remote learning, the intent was to bring the student into an immersive learning experience from wherever they were to obtain as high quality of a learning experience with as much realistic visual stimuli as possible (i.e., avoiding the typical avatar and cartooning of most VR technology on the market; Ai et al., 2002; Kurillo et al., 2011). Subsequently, this created a unique opportunity for the students, where they did find a substantial value for this type of pedagogical approach through both the 180° video VR HMD when used in person and the 3D video computer display when used remotely.

Students also indicated these VR experiences’ pedagogical value in distance learning formats as it relates to the current COVID-19 situation by stating the following: “Like it was used today, having this VR-learning software to enhance in distance learning in a laboratory settings, to learn techniques like stereotaxic surgeries, would be very beneficial to academia and research students as well as mentors,” “This could substitute with the more hands on classes and have a more safe space if not dealing with certain substances in real life,” “It would be beneficial for students who can’t perform task in class without feeling anxious, nervous, or sick,” “Specifically in labs, when there are large amounts of students in one classroom this could be beneficial so everyone could be following through instead of being on their phone all the way in the back. This ensures every student gets to try it as well without hurting an actual animal,” and “This experience gave a different perspective on surgery that we would not get in a traditional classroom, especially during the pandemic and everything being online. This was more informative, along with easier to understand and retain because of the visuals.” Taken together, the students suggest that these VR experiences and pedagogical approaches have value both in terms of remote education continuing through the COVID-19 situation and offering new and creative alternative learning solutions to enhancing remote learning, online learning, and hybrid learning post the COVID-19 situation.

Student’s Self-Report of the VR Value vs. Their Assessed VR Learning Outcomes

The students self-reported that they felt that the 180° video VR HMD and the 3D video computer display were equally valuable to their learning of the content, that both captured their attention, and furthermore, they would use such pedagogical technology to engage in self-study outside of the typical class time (Figure 6). Additionally, the students felt that their college/university should look into finding ways to integrate such VR experiences into their curriculum, which provided the most important feedback obtained from this study with respect to the translatability of the pedagogical approaches’ value to the student/learner (Figure 6). However, in order to assess whether these subjective student self-reports produced better learning outcomes, their responses to questions on the learning module were used as an objective dataset to determine learning comprehension of stereotaxic surgery.

The data from the student learning outcome assessment indicated that the 180° video VR HMD group produced better comprehension of in vivo research advantages (24.04% increase), i.p. injection site (37.50% increase), skull reference points (29.81% increase), electrode number (25.96% increase), and seizure cluster type (15.38% increase) than the 3D video computer display group. Interestingly, the 3D video computer display group produced a better learning comprehension of the drug injection formula (26.68% increase), the surgical infection risks (8.41% increase), the brain atlas map (6.49% increase), and the seizure severity category (2.88% increase). These data revealed that it is apparent that there are key learning concepts that, when presented in the 180° video VR HMD, are maximally learned, perhaps due to the immersion of the material directing and increasing the student’s/learner’s focus on the content in front of them. The 3D video computer display increased the learning of the content displayed on the left Microsoft PowerPoint slides and the right slides that emphasized the key concepts of the module. Although the researchers attempted to control for the experience in the 3D video computer display group to be equivalent to that in the 180° video VR HMD group, it is entirely possible that the 180° video VR HMD group actually spent less time on reviewing the information on the left and the right of them while in the VR experience, which could explain these between-group differences. However, when taken together, the students equally valued both types of VR experience (i.e., 180° video VR HMD and 3D video computer display) to learn the stereotaxic surgical content that they were immersed within.

The current study’s results corroborate the results of Ros et al. (2021), in which students answered better to theoretical questions when they were taught by a teacher/instructor. Here, the teacher/instructor explained the slides, whereas in the 180° video VR HMD, students/learners controlled their viewing of the slides and may have spent less time on some of the content. One way to overcome this issue may be to record a voice-over on each slide to create a unique learning experience for each slide that would serve to increase the attention of the student/learner. Such a modification may reinforce the student’s learning but might not ever replace a teacher. Notably, the student’s anecdotal reports from the pedagogy employed herein showed that it facilitated the student’s memory and learning of the animal surgical techniques. The way the environment was displayed made students feel immersed, even if they did not have a headset (i.e., 3D video computer display). Two points seem to be important for this self-reported student/learner experience: 1) the feeling of being within a circular room that is user centered and 2) the additional data that change in every chapter sequentially teach the animal surgery steps in more tangible modules that the student/learner can grasp easily without overwhelming their cognitive load. Furthermore, every pedagogical aspect of the animal surgery appeared/was displayed within the same VR environment that simulated the sequential nature of conducting the animal surgery in an accelerated form (i.e., approximately 20 min) of what would have been experiences in real time (i.e., approximately 4–5 h of surgery). Thus, such a learning experience can increase the studentʼs comprehension as it relates to upskilling and reskilling next-generation workers in these fields and can do so in an efficient manner compared to traditional animal surgery labs, thereby increasing student/learner focus on achieving the lesson-learning outcomes. Additionally, the 3D video computer display may serve as a flexible blended learning tool to compensate for students abroad, online, or in remote learning capacities who are otherwise unable to attend class in person or learn from a 180° video VR HMD due to economic and/or financial reasons. Thus, the 3D video computer display could leverage educational inequities and increase the reliability of the possibility that the same student learning outcomes can be achieved to reduce and prevent the widening of academic achievement gaps for such critical lessons. Ultimately, the combination of these VR tools could also reduce the need to repeat similar surgeries on animals unnecessarily, and the modules could be further refined to adjust to the student/learner needs and could be replayed/re-experienced as needed to achieve learning proficiency consistent with the 3-Rs.

Study Limitations and Future Implications

Finally, it is important to address some limitations of the present study. The present study compared a 180° video VR HMD and a remote display of the same content over Zoom (i.e., the 3D video computer display) with respect to a student’s ability to learn stereotaxic surgeries, and it could be argued that the absence of significant differences (i.e., participants in the present study favored the VR pedagogical methods regardless of the format in which the VR was presented) opposes the findings of other reports which implemented fully immersive, ergonomic, and interactive VR medical training applications that were able to elicit real-life behaviors and outcomes (Slater, 2009; Kourtesis et al., 2020). The difference in these findings could be attributed to the design of the VR experience through the Revinax^® Handbook as it related to the types of learning outcome questions asked and its accompanying content that is displayed within the experience. It may be the case that the study questions themselves, although confirming the value of the VR experience, may not have been technical and/or sensitive enough to parse out additional differences between the 180° video VR HMD and the 3D video computer display groups. Therefore, future studies should revisit the present study’s findings and build upon them to best address this issue as the VR HMD is quite favorable and creates a much richer learning experience for the student.

Additionally, the Oculus Go has the potential to stimulate cybersickness symptomatology, which in turn could negatively influence the cognitive and behavioral performance of the user’s/student’s/learner’s experience as described in other reports (Kourtesis et al., 2019; Somrak et al., 2021). When we surveyed our study participants following the VR experiences, it was identified that 57% in the 180° video VR HMD group and 12.5% in the 3D video computer display group self-reported a form of visual dizziness regarding their ability to, at times, follow along visually within the VR experience. The cybersickness symptoms reported by the 3D video computer display group may have been due to the lack of full control over visualizing the experience through the guided remote session, whereas in the 180° video VR HMD group, it is most likely due to the Oculus Go, and this situation could be circumvented by using newer VR technology such as the Oculus Quest 1 or 2 or the Pico G2 4K. Notably, despite the Oculus Go’s potential to increase cybersickness symptoms, the present study did not find any difference in learning outcomes between the two groups. Yet, if the present study had used the Oculus Quest 1 or 2, we may have seen an enhancement of responses from the students/learners, but since their survey responses were already at the top of the Likert scale, it is unlikely that they would have differed significantly. There may have been a more sensitive detection in the learning outcomes between the groups with the use of the Oculus Quest 1 or 2, but again, both groups equally valued the pedagogical intervention. Future studies that add this change in the HMD device would be predictive to further enhance the desire to use such a pedagogical approach.

Consistent with the aforementioned points, the present study found that students valued the VR experience over traditional in-person learning and other video instructional formats. This finding of students equally valuing the 180° video VR HMD and the 3D video computer display despite the differences in the type of immersion (i.e., fully immersive vs. not fully immersive) may, in part, be due to the student’s lack of in-person learning experiences over the last year and a half during the COVID-19 pandemic causing a form of an educational deprivation condition. This, in turn, may have increased the student’s sensitivity and receptivity to the 3D video computer display as though the material was presented in the 180° video VR HMD, which might explain why the present study’s findings opposed the findings of Kourtesis et al. (2020) and Slater (2009).

Furthermore, it should be noted that Kourtesis et al. (2020) focused their VR laboratory experience on cognitive neuroscience and neuropsychological test batteries and Slater (2009) focused their VR on more spatial and place-related concepts within the VR environment to assess plausibility and realism, both of which are quite different from the present study, as it used an FPV combined with VR experience to teach stereotaxic surgery. Thus, one should be cautious not to assume through a direct comparison that the VR experiences across these studies are similar but understand that they should rather be evaluated for what each study’s assessment and contribution of the elements of the individual study’s VR experiences offer to the field and advancement of developing future pedagogical VR laboratory methods for teaching students neuroscience content.

Moreover, the present study did not combine the use of eye-gaze analyses with the 3D video computer display, which could have provided an additional level of data-based evidence to corroborate the student’s self-reports and the amount of time spent on both correct and incorrect questions addressing student learning outcomes. Thus, future studies either comparing the 180° video VR HMD and the 3D video computer display or focusing on the use of the 3D video computer display alone should include a form of eye-gaze analysis with the Revinax^® Handbook to assess different perspectives of the end-user’s/student’s/learner’s usability and cognitive load through such eye-tracking measures as one possibility. Conversely, the present study also utilized self-report data, which, in and of itself, is a limitation. A future study should consider directly assessing the performance of the students with respect to either a task analysis of the stereotaxic surgery or an actual surgery to accurately test the reliability and generalizability of the learning outcomes. However, a main goal of this study design was to develop a teaching method that could assess learning outcomes while avoiding the use of additional animals to test such learning outcomes, while making the experience as real as possible to an authentic stereotaxic surgery.

It would have been interesting to use a cross-over test design with respect to the immersive feelings of each group. However, for practical and safety reasons due to COVID-19, this add-on experiment was not possible and serves as a limitation of the present study. For the participants in the 3D video computer display group, they felt more immersed than they would have during a traditional lecture, and they may have felt more immersed with a VR headset as immersion also depends on the hardware used (Fowler et al., 2015). Thinking about it as a pedagogical continuum, future studies could envision the following different steps: 1) the 180° video VR HMD could be used to learn and increase active engagement, 2) the 3D video computer display could be used by the teacher to debrief the student/learner about the experience and subsequently answer theoretical/concept questions, 3) the use of mobile-IVRA could be an ideal alternative for teaching on-the-job skills, refreshing needed skills, and building new practical skills/upskilling, and 4) it is also possible, but is yet to be studied, that the FPV-ARA (i.e., augmented reality) could be used to support the student/learner during the first experience of conducting the actual procedures (i.e., if the goal is to perform the technique thereafter). This latter point is consistent with the medical model—see one, do one, and teach one, but without harm or issue to both a human and animal—thereby making this approach more ethical and attractive to future generations of students/learners.

Conclusion

In summary, the present study shows that using novel, unique, and creative pedagogical approaches through VR may enhance student learning outcomes through the Revinax^® Handbook method displayed in both the 180° video VR HMD and the 3D video computer display for undergraduate students learning stereotaxic surgery. The results of this study are important and contribute to the field as they can be used to either upskill or reskill students in their early career paths to becoming medical professionals, doctors, physicians, surgeons, and/or basic research scientists who work in the neuroscience/biopsychology areas. Furthermore, the use of the 180° video VR HMD and the 3D video computer display has been shown to be a valuable pedagogical tool, as evidenced by the students’ self-reports and anecdotes, suggesting that such technology may help them to close and perhaps prevent the widening of the academic achievement gap during the COVID-19 pandemic era. Although the use of the 180° video VR HMD and the 3D video computer display may help to regain, sustain, and perhaps improve student’s learning of the curriculum, one must be cautious in that simply offering such technologies does not necessarily overcome the digital divide that is disproportionately affecting students of underrepresented minority populations and low-income populations. Thus, more support efforts in technological educational outreach, stability of and access to Wi-Fi, and other forms of educational/learning connectivity will continue to be the critical tethering point for further (re)defining of learning as the COVID-19 situation continues to evolve. Furthermore, the present study provides evidence of the value of the Revinax^® Handbook method through the 180° video VR HMD and the 3D video computer display in teaching undergraduate students stereotaxic surgery and may suggest the possibility that other procedural learning techniques in the neuroscience/biopsychology or other medical/practical fields and additional learning content that could be enhanced through immersive learning could be developed and deployed for a more modern pedagogy both during and following the COVID-19 pandemic to best address distance/remote learning. The most noteworthy of what has been learned from the COVID-19 pandemic is that instructors can find ways in which to “virtually” reach their students through the Revinax^® Handbook method through the 180° video VR HMD and the 3D video computer display, and perhaps such pedagogical formats may in the future serve as an adjunct to or even perhaps replace traditional educational textbooks, e-books, and/or video/YouTube learning altogether, but what is clear is that it will not have the capability to replace the instructor despite the learning format deployed within the traditional, online, and hybrid distance/remote classroom.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by the SUNY Old Westbury Institutional Review Board (IRB). The participants provided their written informed consent to participate in this study. The animal study was reviewed and approved by the SUNY Old Westbury Institutional Animal Care and Use Committee (IACUC). Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author Contributions

LSN and MR conceived of and completed the study, analyzed the data, and wrote and approved the final version of the manuscript. The authors would like to thank Mr. Brandon Rosenthal for his assistance with setting up the HMD FPV-IVRA technology to complete this study.

Funding

This study received no funding support.

Conflict of Interest

MR was practicing neurosurgery at the Montpellier University Hospital before founding REVINAX®, the company that developed the tutorial using the HMD-FPV IVRA and the CD-FPV SIVRA methodology.

The remaining author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Ai, Z., Dech, F., Silverstein, J., and Rasmussen, M. (2002). “Tele-immersive Medical Educational Environment,” in Medicine Meets Virtual Reality 02/10 Digital Upgrades: Applying Moore’s Law to Health. Editors J. D. Westwood, H. M. Hoffman, R. A. Robb, and D. Stredney (Washington, DC: IOS Press Ohmsha), 24–30.

Google Scholar

Alfalah, S. F. M., Falah, J. F. M., Alfalah, T., Elfalah, M., Muhaidat, N., and Falah, O. (2019). A Comparative Study Between a Virtual Reality Heart Anatomy System and Traditional Medical Teaching Modalities. Virtual Reality 23 (3), 229–234. doi:10.1007/s10055-018-0359-y

CrossRef Full Text | Google Scholar

Andersen, S. A. W., Mikkelsen, P. T., Konge, L., Cayé-Thomasen, P., and Sørensen, M. S. (2016). Cognitive Load in Mastoidectomy Skills Training: Virtual Reality Simulation and Traditional Dissection Compared. J. Surg. Educ. 73 (1), 45–50. doi:10.1016/j.jsurg.2015.09.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Angevine, J. B., and Cotman, C. W. (1981). Principles of Neuroanatomy. New York, NY: Oxford University Press.

Bayer, S. A., and Altman, J. (2002). The Spinal Cord from Gestational Week 4 to the 4^th Postnatal Month. New York, NY: CRC Press. doi:10.1201/9781420040180

CrossRef Full Text

Bergeron, L. (2005). “Rite-of-passage for First Year Medical Students: Meeting Their Cadavers.” Stanford Report. Stanford News (Retrieved March 15, 2021).

Google Scholar

Bogdanske, J. J., Hubbard-Van Stelle, S., Riley, M. R., and Schiffman, B. M. (2011). Laboratory Rat: Procedural Techniques Manual and DVD. Boca Raton, FL: CRC Press.

Bradley, W. G., and Bydder, G. (1989). MRI Atlas of the Brain. New York, NY: Raven Press.

Cardon, A. D., Bailey, M. R., and Bennett, B. T. (2012). The Animal Welfare Act: From Enactment to Enforcement. J. Am. Assoc. Lab. Anim. Sci. 51 (3), 301–305. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3358977/

Google Scholar

Coles, T. R., Meglan, D., and John, N. W. (2010). The Role of Haptics in Medical Training Simulators: A Survey of the State of the Art. IEEE Trans. Haptics 4 (1), 51–66. doi:10.1109/TOH.2010.19

PubMed Abstract | CrossRef Full Text | Google Scholar

Committee for the Update of the Guide for the Care and Use of Laboratory Animals, Institute for Laboratory Animal Research, Division on Earth and Life Studies, & National Research Council (2010). Guide for the Care and Use of Laboratory Animals. 8th Edn. Washington, DC: National Academies Press.

Cooley, R. K., and Vanderwolf, C. H. (1978). A Stereotaxic Surgery in the Rat: A Photographic Series. London, UK: A.J. Kirby Co.

DeArmond, S. J., Fusco, M. M., and Dewey, M. M. (1989). Structure of the Human Brain: A Photographic Atlas. 3rd Edn. New York, NY: Oxford University Press.

DeLucchi, M. R., Dennis, B. J., and Adey, W. R. (1965). A Stereotaxic Atlas of the Chimpanzee Brain (Pan Satyrus). Berkeley and Los Angeles, CA: University of California Press.

Eadens, D. M., and Eadens, D. W. (2021). “Pivoting to Deeper Experiences in Education,” in Handbook of Research on Lessons Learned from Transitioning to Virtual Classrooms during a Pandemic. Editors A. Thornburg, R. Ceglie, and D. Abernathy (Orlando, FL: College of Community Innovation and Education), 277–290.

CrossRef Full Text | Google Scholar

Fowler, C. (2015). Virtual Reality and Learning: Where is the Pedagogy? British J. Educat. Technol. 46 (2), 412–422. doi:10.1111/bjet/12135

CrossRef Full Text | Google Scholar

Fiorella, L., Van Gog, T., Hoogerheide, V., and Mayer, R. E. (2017). It’s All a Matter of Perspective: Viewing First-Person Video Modeling Examples Promotes Learning of an Assembly Task. J. Educ. Psychol. 109 (5), 653–665. doi:10.1037/edu0000161

CrossRef Full Text | Google Scholar

Foster, G. A. (1988). Chemical Neuroanatomy of the Prenatal Rat Brain: A Developmental Atlas. New York, NY: Oxford University Press.

Gelles, L. A., Lord, S. M., Hoople, G. D., Chen, D. A., and Mejia, J. A. (2021). Compassionate Flexibility and Self-Discipline: Student Adaptation to Emergency Remote Teaching in an Integrated Engineering Energy Course during COVID-19. Educ. Sci. 10 (11), 304. doi:10.3390/educsci10110304

CrossRef Full Text | Google Scholar

Gergen, J. A., and MacLean, P. D. (1962). A Stereotaxic Atlas of the Squirrel Monkey’s Brain (Saimiri sciureus). Bethesda, MD: U.S. Department of Health Education, and Welfare. doi:10.5962/bhl.title.114088

CrossRef Full Text

Gilman, S., and Winans-Newman, S. (1991). Manter and Gatz’s Essentials of Clinical Neuroanatomy and Neurophysiology (Essentials of Medical Education). 7th Edn. Philadelphia, PA: F.A. Davis Company.

Gould, D. G., and Brueckner, J. K. (2008). Sidman’s Neuroanatomy: A Programmed Learning Tool. 2nd Edn. New York, NY: Wolters Kluwer Health - Lippincott Williams & Wilkins.

Haines, D. E. (2008). Neuroanatomy: An Atlas of Structures, Sections, and Systems. 7th Edn. New York, NY: Wolters Kluwer Health - Lippincott Williams & Wilkins.

Harpe, S. E. (2015). How to Analyze Likert and Other Rating Scale Data. Currents Pharm. Teach. Learn. 7 (6), 836–850. doi:10.1016/j.cptl.2015.08.001

CrossRef Full Text | Google Scholar

Hudson, S., Matson-Barkat, S., Pallamin, N., and Jegou, G. (2019). With or without You? Interaction and Immersion in a Virtual Reality Experience. J. Business Res. 100, 459–468. doi:10.1016/j.jbusres.2018.10.062

CrossRef Full Text | Google Scholar

Izard, S. G., Juanes, J. A., Peñalvo, F. J. G., Estella, J. M. G., Ledesma, M. J. S., and Ruisoto, P. (2018). Virtual Reality as an Educational and Training Tool for Medicine. J. Med. Syst. 42 (3), 1–5. doi:10.1007/s10916-018-0900-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Jacobson, S., Marcus, E. M., and Pugsley, S. (2018). Neuroanatomy for the Neuroscientist. 3rd Edn. Cham, Switzerland: Springer International Publishing AG. doi:10.1007/978-3-319-60187-8

CrossRef Full Text

Karten, H. J., and Hodos, W. (1967). A Stereotaxic Atlas of the Brain of the pigeon (Columbia Livia). Baltimore, MD: The Johns Hopkins Press.

Kent, G. C. (1965). Comparative Anatomy of the Vertebrates. Saint Louis, MO: The C.V. Mosby Company.

Kourtesis, P., Collina, S., Doumas, L. A., and MacPherson, S. E. (2019). Technological Competence Is a Pre-condition for Effective Implementation of Virtual Reality Head Mounted Displays in Human Neuroscience: a Technological Review and Meta-Analysis. Front. Hum. Neurosci. 13, 342. doi:10.3389/fnhum.2019.00342

PubMed Abstract | CrossRef Full Text | Google Scholar

Kourtesis, P., Korre, D., Collina, S., Doumas, L. A., and MacPherson, S. E. (2020). Guidelines for the Development of Immersive Virtual Reality Software for Cognitive Neuroscience and Neuropsychology: the Development of Virtual Reality Everyday Assessment Lab (VR-EAL), A Neuropsychological Test Battery in Immersive Virtual Reality. Front. Comput. Sci. 1, 12. doi:10.3389/fcomp.2019.00012

CrossRef Full Text | Google Scholar

Kurillo, G., Koritnik, T., Bajd, T., and Bajcsy, R. (2011). “Real-time 3D Avatars for Tele-Rehabilitation in Virtual Reality,” in Medicine Meets Virtual Reality 18: Next Med. Editors J. D. Westwood, S. W. Westwood, L. Felländer-Tsai, R. S. Haluck, H. M. Hoffman, R. A. Robbet al. (Washington, DC: IOS Press), 290–303. doi:10.3233/978-1-60750-706-2-290Laboratory

CrossRef Full Text | Google Scholar

Lakens, D. (2013). Calculating and Reporting Effect Sizes to Facilitate Cumulative Science: a Practical Primer for T-Tests and ANOVAs. Front. Psychol. 4, 863. doi:10.3389/fpsyg.2013.00863

PubMed Abstract | CrossRef Full Text | Google Scholar

Lawrence, A. J. (1997). “In-vivo Microdialysis,” in Neuroscience Methods: A Guide for Advanced Students. Editor R. Martin (Amsterdam, Netherlands: Harwood Academic Publishers).

Google Scholar

Lim, R. K. S., Liu, C. N., and Moffitt, R. L. (1960). A Stereotaxic Atlas of the Dog’s Brain. Springfield, IL: Charles C. Thomas Publisher.

Lysenko, E., and Tokareva, J. (2021). Study of Satisfaction with the Organization of the Studying Process in Remote Mode during the COVID-19 Pandemic: Case UrFU Named after the First President of Russia B.N.Yeltsin. SHS Web of Conferences, Les Ulis, 92, 01026. doi:10.1051/shsconf/20219201026

CrossRef Full Text | Google Scholar

Mamun, M. A. A., Hossain, M. A., Salehin, S., Khan, M. S. H., and Hasan, M. (2021). Engineering Students’ Readiness for Online Learning amidst the COVID-19 Pandemic: Scale Validation and Lessons Learned from a Developing Country. Res. Square 1–28. doi:10.21203/rs.3.rs-374991/v1

CrossRef Full Text | Google Scholar

Martin, J. H. (1996). Neuroanatomy Text and Atlas. 2nd Edn. New York, NY: McGraw-Hill Health Professions Division.

Martin, R. (1998). Neuroscience Methods: A Guide for Advanced Students. Amsterdam, Netherlands: Harwood Academic Publishers.

McCormak, T. J., Lemoine, P. A., Waller, R. E., and Richardson, M. D. (2021). Global Higher Education: Examining Response to the COVID-19 Pandemic Using Agility and Adaptability. J. Educ. Develop. 5 (1), 10. doi:10.20849/jed.v5i1.848

CrossRef Full Text | Google Scholar

Montemurro, D. G., and Dukelow, R. H. (1972). A Stereotaxic Atlas of the Diencephalon and Related Structures of the Mouse. Mount Kisco, NY: Futura Publishing Company, Inc.

Mukherji, B. R., Neuwirth, L. S., and Limonic, L. (2017). Making the Case for Real Diversity: Redefining Underrepresented Minority Students in Public Universities. Sage Open. 7 (2), 215824401770779. doi:10.1177/2158244017707796

CrossRef Full Text | Google Scholar

National Research Council of The National Academies (2003). Guidelines for the Care and Use of Mammals in Neuroscience and Behavioral Research. Washington, DC: National Academies Press.

Neuwirth, L. S., Dacius, T. F., and Mukherji, B. R. (2018a). Teaching Neuroanatomy through a Historical Context. J. Undergraduate Neurosci. Educ. (June) 16 (2), E26–E31. Available at: http://www.funjournal.org/wp-content/uploads/2018/04/june-16-e26.pdf.

PubMed Abstract | Google Scholar

Neuwirth, L. S., Ebrahimi, A., Mukherji, B. R., and Park, L. (2018b). “Addressing Diverse College Students and Interdisciplinary Learning Experiences Through Online Virtual Laboratory Instruction,” in Visual Approaches to Cognitive Education with Technology Integration. Editor A. Ursyn(Hershey, PA: IGI Global), 283–303. Available at: https://www.igi-global.com/chapter/addressing-diverse-college-students-and-interdisciplinary-learning-experiences-through-online-virtual-laboratory-instruction/195070. doi:10.4018/978-1-5225-5332-8.ch012

CrossRef Full Text | Google Scholar

Neuwirth, L. S., Ebrahimi, A., Mukherji, B. R., and Park, L. (2019). “Addressing Diverse College Students and Interdisciplinary Learning Experiences through Online Virtual Laboratory Instruction: A Theoretical Approach to Error-Based Learning in Biopsychology,” in Information Resources Management Association. Virtual Reality in Education: Breakthroughs in Research and Practice (Hershey, PA: IGI Global), 511–531. Available at: https://www.igi-global.com/chapter/addressing-diverse-college-students-and-interdisciplinary-learning-experiences-through-online-virtual-laboratory-instruction/195070.

CrossRef Full Text | Google Scholar

Neuwirth, L. S., Jović, S., and Mukherji, B. R. (2020). Reimagining Higher Education during and post-COVID-19: Challenges and Opportunities. J. Adult Contin. Educ.. doi:10.1177/1477971420947738

CrossRef Full Text | Google Scholar

Office of Laboratory Animal Welfare (2015). PHS Policy on Humane Care and Use of Laboratory Animals. (Accessed March 29, 2021). Available at: Olaw.nih.gov/policies-laws/phs-policy.htm

Google Scholar

O’Rahilly, R., and Müller, F. (1994). The Embryonic Human Brain: An Atlas of Developmental Stages. New York, NY: A John Wiley & Sons, Inc., Publication.

O’Sullivan, S., Khraibi, A., Chen, W., and Corridon, P. R. (2021). Lessons Learned Transitioning from Traditional Premedical and Medical Education to E-Learning Platforms during the COVID-19 Pandemic. SSRN. doi:10.2139/ssrn.3787260 (February 17, 2021).

CrossRef Full Text | Google Scholar

Paxinos, G. (1985a). The Rat Nervous System: Volume 1 Forebrain and Midbrain. New York, NY: Academic Press.

Paxinos, G. (1985b). The Rat Nervous System: Volume 2 Hindbrain and Spinal Cord. New York, NY: Academic Press.

Paxinos, G., and Watson, C. (2005). The Rat Brain in Stereotaxic Coordinates. Burlington, MA: Elsevier Academic Press.

Perrett, S. P. (1997). “Stereotaxic Placement of Probes in Neurobiology,” in Neuroscience Methods: A Guide for Advanced Students. Editor R. Martin (Amsterdam, Netherlands: Harwood Academic Publishers).

Google Scholar

Pottle, J. (2019). Virtual Reality and the Transformation of Medical Education. Future Healthc. J. 6 (3), 181–185. doi:10.7861/fhj.2019-0036

PubMed Abstract | CrossRef Full Text | Google Scholar

Pritchett-Corning, K. R., Girod, A., Vellaneda, G., Fritz, P. E., Chou, S., and Brown, M. J. (2010). Handbook of Clinical Signs in Rodents and Rabbits. 1st Edn. Wilmington, MA: Charles River Laboratories.

Prus, A. (2020). Drugs and the Neuroscience of Behavior: An Introduction to Psychopharmacology. 3rd Edn. Thousand Oaks, CA: Sage Publications, Inc.

Ramachandra, R. (2011). Atlas of the Neonatal Rat Brain. Boca Raton, FL: CRC Press Taylor & Francis Group.

Ros, M., Debien, B., Cyteval, C., Molinari, N., Gatto, F., and Lonjon, N. (2020a). Applying an Immersive Tutorial in Virtual Reality to Learning a New Technique. Neurochirurgie 66 (4), 212–218. doi:10.1016/j.neuchi.2020.05.006

CrossRef Full Text | Google Scholar

Ros, M., and Neuwirth, L. S. (2020). Increasing Global Awareness of Timely COVID-19 Healthcare Guidelines through FPV Training Tutorials: Portable Public Health Crises Teaching Method. Nurse Educ. Today 91, 104479. doi:10.1016/j.nedt.2020.104479

PubMed Abstract | CrossRef Full Text | Google Scholar

Ros, M., Neuwirth, L. S., Ng, S., Debien, B., Molinari, N., Gatto, F., et al. (2021). The Effects of an Immersive Virtual Reality Application in First Person Point-of-View, Video-Based on the Learning and Generalized Performance of a Lumbar Puncture Medical Procedure. Educat. Technol. Res. Develop. 69, 129–1556. doi:10.1007/s11423-021-10003-w

CrossRef Full Text | Google Scholar

Ros, M., Trives, J.-V., and Lonjon, N. (2017). From Stereoscopic Recording to Virtual Reality Headsets: Designing a New Way to Learn Surgery. Neurochirurgie 63 (1), 1–5. doi:10.1016/j.neuchi.2016.08.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Ros, M., and Trives, J. V. (2020). Point-of-view Recording Device. US Patent App. 16/341,070.

Ros, M., Weaver, L., and Neuwirth, L. S. (2020b). “Virtual Reality Stereoscopic 180-degree Video-Based Immersive Environments: Applications for Training Surgeons and Other Medical Professionals,” in Cases on Instructional Design and Performance Outcomes in Medical Education. Editor J. E. Stefaniak (Hershey, PA: IGI Global).

Google Scholar

Russell, W. M. S., and Burch, R. L. (1959). The Principles of Humane Experimental Technique. London: Methuen.

Schild, J., Misztal, S., Roth, B., Flock, L., Luiz, T., Lerner, D., et al. (2018). “Applying Multi-User Virtual Reality to Collaborative Medical Training,” in 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Tuebingen, Germany, March 18–22, 2018 (Piscataway, NJ: IEEE), 775–776. doi:10.1109/VR.2018.8446160

CrossRef Full Text | Google Scholar

Sharp, P. E., and La Regina, M. C. (1998). The Laboratory Rat: A Volume in the Laboratory Animal Pocket Reference Series. Boca Raton, FL: CRC Press.

Slater, M. (2009). Place Illusion and Plausibility Can lead to Realistic Behaviour in Immersive Virtual Environments. Phil. Trans. R. Soc. B 364 (1535), 3549–3557. doi:10.1098/rstb.2009.0138

PubMed Abstract | CrossRef Full Text | Google Scholar

Slater, M., and Sanchez-Vives, M. V. (2016). Enhancing Our Lives with Immersive Virtual Reality. Front. Robot. AI 3, 19. doi:10.3389/frobt.2016.00074

CrossRef Full Text | Google Scholar

Slotnick, B. M., and Leonard, C. M. (1975). A Stereotaxic Atlas of the Albino Mouse Forebrain. Rockville, MD: U.S. Department of Health Education, and Welfare.

Snider, R. S., and Niemer, W. T. (1961). A Stereotaxic Atlas of the Cat Brain. Chicago, IL: The University of Chicago Press.

Somrak, A., Pogačnik, M., and Guna, J. (2021). Suitability and Comparison of Questionnaires Assessing Virtual Reality-Induced Symptoms and Effects and User Experience in Virtual Environments. Sensors 21 (4), 1185. doi:10.3390/s21041185

PubMed Abstract | CrossRef Full Text | Google Scholar

Ward, J. (2015). The Student’s Guide to Cognitive Neuroscience. 3rd Edn. New York, NY: CRC Press. doi:10.4324/9781315742397

CrossRef Full Text

Woolsey, T. A., Hanaway, J., and Gado, M. H. (2008). The Brain Atlas: A Visual Guide to the Human central Nervous System. 3rd Edn. Hoboken, NJ: A John Wiley & Sons, Inc., Publication.

Zilles, K. (1985). The Cortex of the Rat: A Stereotaxic Atlas. New York, NY: Springer-Verlag. doi:10.1007/978-3-642-70573-1

CrossRef Full Text