Enhancing nursing competency through virtual reality simulation among nursing students: a systematic review and meta-analysis

Abstract

Aim:

Studies on the effectiveness of virtual reality (VR) in nursing education have explored its impact on learning outcomes, emotional immersion and engagement, learner self-confidence, and satisfaction, generally showing positive aspects. However, there is a need for a systematic review to examine the specific influence of VR-based education on nursing students’ practical competency.

Method:

According to the PRISMA 2020 guidelines, 22 studies were selected based on inclusion criteria from 579 articles, published from January 1, 2018, to March 31, 2024, across nine major databases including PubMed and EMbase. The target population comprised nursing students, and the intervention focused on VR-based simulations aimed at enhancing competency, compared to control groups receiving either no intervention or conventional non-virtual simulation. The primary outcome, nursing competency, was analyzed using MIX 2.0 Pro (Ver. 2.0.1.6, BiostatXL, 2017) to calculate pooled effect sizes.

Result:

The pooled effect size for nursing competency was determined to be large, with Hedge’s g = 0.88 (95% CI, 0.47 to 1.29). Meta-regression analysis identified several factors associated with an increase in nursing competency. These included studies published after 2022, approval of an IRB, absence of funding, randomized controlled trials (RCTs), interventions reported as shorter than 4 weeks or not reported, sessions fewer than 4 or not reported, session duration under 1 h or not reported, and observational measurement methods. Additional factors enhancing nursing competency were the inclusion of a pre-briefing before simulations, the absence of a debriefing afterward, and the exclusion of other activities during the simulation.

Conclusion:

By combining the results of the included studies, the systematic review and meta-analysis accounted for variations in sample size, study methodology, and independent intervention effects, providing an overall evaluation of the effectiveness of simulation-based education in improving nursing students’ competency.

Limitation:

The selection criteria for the studies analyzed, which included only those published in English or Korean and reported precise means, standard deviations, and sample sizes, could lead to selection bias and limit the generalization of our study results.

Systematic review registration:

PROSPERO International Prospective Register of Systematic Reviews: http://www.crd.york.ac.uk/PROSPERO/, identifier CRD42023446348.

1 Introduction

Nursing education is an applied discipline in which theory and practical education are combined; prospective nurses prepare to become competent by applying the knowledge learned in theoretical education to the practical education process. The need for nursing education to train professionals who provide nursing and medical services to humans utilizing digital-based, non-face-to-face media such as artificial intelligence (AI) and big data has recently become more urgent (1). In nursing education, there has been an increasing interest in virtual-reality simulation (VRS) education as an alternative and complementary method to traditional simulation education, providing students with new learning experiences in a reproduced clinical environment and enhancing clinical adaptability (2). Virtual reality (VR) is defined as “the use of partial immersion through a digital learning environment (computer, tablet, phone, screen, etc.) to foster a perceived lived experience for an intended outcome (e.g., learning and entertainment)” (3). This study defines VRS to include VR and its derivatives, augmented reality (AR), and mixed reality (MR), using the terminology consistently. In VRS, learners can collaborate with other healthcare professionals to provide interventions, such as solving patients’ problems or practicing simple skills (4, 5). Improved clinical performance skills, knowledge, and metacognition, as well as enhanced learning satisfaction, communication, self-efficacy, confidence, and teamwork have been reported as effects of these VR programs (4, 6). In addition, studies on the effectiveness of nursing education using VR have been conducted on learning effectiveness, emotional engagement and immersion, learner confidence, and satisfaction (7, 8). Reportedly, VRS programs for nursing skills are effective in improving skills (9) and have the advantage of enabling safe and repetitive training without time and space constraints (10). Thus, learning through VRS has demonstrated improvement in various factors related to clinical nursing competency, albeit often assessed in a fragmented manner. As various forms of VRS are being applied in nursing education, and diverse elements contributing to nursing competency are considered, there is a need to comprehend the holistic outcomes of these studies. Consequently, this study aims to comprehensively review the results, considering nursing competency in a broader sense that encompasses collaboration, interpersonal relationships, communication, professional development, and the nursing process, skills, and education (11).

Moreover, a systematic review and analysis of nursing students’ outcomes are essential for determining specific factors that are deemed effective. Systematic reviews and meta-analyses can amalgamate the results of included studies, accounting for differences in sample size, variations in research approaches, and intervention effects among independent studies. We believe that the systematic review and meta-analysis in this study will enable an assessment of the overall effect of VRS-based education on nursing students’ nursing competency. Consequently, this study aims to provide foundational data on VRS by conducting a systematic literature review and meta-analysis, investigating the improvement effect of VRS on nursing students’ nursing competency as a primary outcome, and examining knowledge, self-efficacy, problem-solving skills, confidence, and satisfaction as secondary outcomes.

This study aims to acquire and analyze evidence regarding the enhancement of nursing students’ nursing competency through VRS. The primary outcome focuses on nursing students’ self-reported feelings and reactions, while the secondary outcome assesses nursing students’ nursing competency following exposure to VRS.

2 Materials and methods

2.1 Search strategy and data sources

The search was jointly conducted by two researchers, Cho, M.-K. and Kim, M.Y., across nine electronic databases or e-journals: PubMed, Cochrane, EMBASE-OVID, CINAHL, World of Science, SCOPUS, PQDT, APA PsycArticles, and Research Information Sharing Service. The primary search, conducted from July 18, 2023, to August 20, 2023, targeted articles published in English and Korean from January 1, 2003, to April 30, 2023. A secondary search was carried out from April 6, 2024, to April 9, 2024, focusing on articles published from May 1, 2023, to March 31, 2024, also in English and Korean. The search strategy and formula, following the PICO-SD framework (population, intervention, comparison, outcome, study design), are detailed in Table 1. The keywords employed in search terms across the nine databases included combinations and variations of “nursing students,” “virtual reality,” “augmented reality,” “extended reality,” “metaverse,” “competency-based education,” “clinical competence,” “competency,” and “controlled clinical trial.” These keywords were chosen to comprehensively capture studies relevant to the impact of virtual reality simulation on nursing competency.

2.2 Inclusion and exclusion criteria

The reporting of the results adhered to the PRISMA 2020 checklist. Inclusion criteria comprised nursing students aged 19 years or older (Population), interventions involving VRS (Intervention), with conventional learning methods or no intervention as the control (comparison). The primary outcome was nursing competency, and secondary outcomes included knowledge, self-efficacy, problem-solving, confidence, satisfaction, and other variables, which were concurrently measured. If multiple measurements were conducted post-intervention, the first measurement was used to calculate the effect size. Only studies presenting subject numbers, means, and standard deviations in the results were selected for precise effect-size calculation. The study designs included randomized controlled trials (RCTs) and quasi-experimental studies. Exclusion criteria included studies encompassing students from majors other than nursing, interventions using conventional simulation-learning methods instead of VRS, the absence of nursing competency as an outcome variable, studies not reported in Korean or English, studies with inaccessible original texts, and single-group studies lacking a control group.

2.3 Data extraction

Two researchers, Cho, M.-K. and Kim, M.Y., independently conducted searches and selected studies for analysis based on the predefined inclusion and exclusion criteria. The selected studies were extracted, incorporating information such as author, year of publication, country, publication language, number of schools, institutional review board (IRB) approval, funding details, number of participants, study design, intervention characteristics (type, facilitator, duration, session, time/session, pre-briefing, debriefing, other activities, outcome measurement time, and measurement method), quality assessment score, and dependent variables. This information was meticulously recorded in a coding book created using the Microsoft Excel spreadsheet software. Any disparities in coding were addressed by revisiting the original text to ascertain and input the final coding values (Table 2).

2.4 Quality assessment

The quality assessment of selected articles was independently performed by Cho, M.-K. and Kim, M.Y. using the Joanna Briggs Institute (JBI) Checklist for RCTs and the Checklist for Quasi-Experimental Studies. Five RCTs were assessed using the 13-question JBI Checklist; the average score was 8.40, and all five studies lacked clear reporting on “Q2. Was allocation to treatment groups concealed?” and “Q4. Were participants blind to treatment assignment?” Quasi-experimental studies comprised eight articles, and on evaluation using the 9-item JBI Checklist for Quasi-Experimental Studies (32), the average score was 8.50, with generally well-reported items (Table 3).

2.5 Statistical analyses

MIX 2.0 Pro (Ver. 2.0.1.6, BiostatXL, 2017) was used to calculate and merge effect sizes for both the primary outcome of nursing competency and secondary outcomes. The overall effect was determined using a random-effects model, considering between-subject variability and heterogeneity between studies. Hedge’s g was employed for effect-size calculation, and significance was assessed using 95% confidence intervals (CIs), Z tests, and p-values. The weight of each effect size was determined using the inverse of variance (33). Heterogeneity was evaluated using Higgin’s I² (34), with an I² of >50% indicating heterogeneity (35). Subgroup analysis, meta-regression, and exclusion-sensitivity analysis were conducted for nursing competency to identify factors contributing to heterogeneity. Publication bias was examined using funnel plots, trim-and-fill plots, Begg’s test, Egger’s regression, and the trim-and-fill method to correct for the overall effect (36).

3 Results

3.1 Characteristics of the included studies

A total of 579 articles were initially identified from 9 databases following the search strategy. After excluding duplicates, 373 articles were extracted. Following the application of the inclusion and exclusion criteria, 21 research articles were ultimately selected. The research by Sahin Karaduman and Basak (28) was designed using two experimental groups and was analyzed as two separate studies, resulting in 22 studies being analyzed (Figure 1). Of these, six studies were published before 2022; three were conducted in the United States (USA), twelve studies were published in English; nineteen were conducted at a single university; nineteen and nine studies had IRB approval and funding, respectively. The study designs included seven RCTs, fifteen quasi-experimental studies, and eight studies with fewer than 60 participants. Interventions comprised 18 VR/AR simulations and four metaverse. Eight studies had a professor as a facilitator, four had an intervention duration of more than 4 weeks, two had four or more intervention sessions, eight had an intervention time of more than 1 h per session, 19 had a pre-briefing, and nine had a debriefing. Dependent-variable measurements were taken immediately after the intervention in 20 studies, the measurement method was observational measurement in 12 studies, 19 studies had no additional activities, such as reflection, besides the simulation, and 14 studies had an above-average quality assessment score (Table 2).

Figure 1

3.2 Effect of VRS-based intervention on nursing competency

The overall effect of nursing competency, as the primary outcome for the 22 VRSs, was found to be Hedge’s g = 0.88 (95% CI: 0.47 to 1.29). This was interpreted as a large effect based on the criteria provided by Brydges (37) for interpreting effect sizes (Figure 2). The high degree of heterogeneity among the studies, indicated by Higgins’s I² of 91.8% in the heterogeneity test, prompted subgroup and meta-regression analyses to explore factors contributing to this heterogeneity.

Figure 2

In subgroup analyses, the characteristics of studies significantly associated with improvements in nursing competency IRB-approved studies (Hedge’s g = 1.02, 95% CI: 0.57, 1.48); interventions with a duration not reported or those with a duration of less than 4 weeks (Hedge’s g = 1.05, 95% CI: 0.56, 1.53); interventions with sessions not reported or those with less than 4 sessions (Hedge’s g = 0.95, 95% CI: 0.50, 1.39); those with outcome measurement immediately after the intervention (Hedge’s g = 0.93, 95% CI: 0.50, 1.37); those with pre-briefing before the simulation (Hedge’s g = 0.71, 95% CI: 0.23, 1.20); those with debriefing after the simulation (Hedge’s g = 1.02, 95% CI: 0.57, 1.48); and those with no other activities besides the simulation, such as keeping a reflective journal (Hedge’s g = 1.03, 95% CI: 0.55, 1.50). Publication year, Country, publication language, number of schools, funding status, research design, number of participants, intervention type, facilitator, intervention time per session, measurement method, debriefing, and quality assessment score also showed statistically significant effect sizes (Table 4).

Univariate meta-regression identified factors influencing the overall effect. Publication year after 2022 (Z = 2.68, p = 0.007); having an IRB (Z = 5.17, p < 0.001); having an fund (Z = −2.61, p = 0.009); RCT (Z = 2.02, p = 0.044); intervention duration over than 4 weeks (Z = −3.33, p < 0.001); intervention session over than 4 sessions (Z = −3.01, p < 0.001); intervention time/session over than 1 h (Z = −5.20, p < 0.001); observational measurement rather than self-reporting (Z = 3.21, p = 0.001); having a pre-briefing before the simulation (Z = 3.76, p < 0.001); having a debriefing after the simulation (Z = −4.41, p < 0.001); and having other activities besides the simulation (Z = −4.41, p < 0.001) had statistically significant effects on nursing competency (Table 5).

The exclusion-sensitivity test (38), excluding one study at a time, showed Hedge’s g ranging from 0.67 to 0.94, indicating a moderate to large effect size. The 95% CI (0.36 ~ 0.53, 0.98 ~ 1.36) did not include 0, signifying statistical significance. The effect sizes from the exclusion-sensitivity test were not significantly different from Hedge’s g = 0.88, which included all 22 studies (Table 6).

3.3 Effect of intervention program on secondary outcomes

The secondary outcomes in this study included knowledge, self-efficacy, problem-solving, confidence, and satisfaction. Among these, knowledge, self-efficacy, confidence, and satisfaction exhibited statistically significant changes. After VRS, knowledge and self-efficacy showed significant increases, with moderate effect sizes of Hedge’s g = 0.60 (95% CI: 0.07, 1.14) and Hedge’s g = 0.53 (95% CI: 0.09, 0.97), respectively. Confidence and satisfaction exhibited substantial increases, with large effect sizes of Hedge’s g = 1.02 (95% CI: 0.48, 1.57) and Hedge’s g = 1.35 (95% CI: 0.43, 2.28), respectively (Table 7).

3.4 Publication bias

To evaluate publication bias, funnel-plot and trim-and-fill plot analyses were conducted. Represented by the black circle, the individual effect sizes of the 22 studies included in the study were asymmetrical— slightly skewed to the right—indicating some potential publication bias (Figure 3). The trim-and-fill plot suggested the addition of one study, represented by the white circle, skewing to the left (Figure 4). Further analysis, The coefficient of the bias was 8.58, indicating the initial value (intercept) and the p-value was 0.001. Thus, the null hypothesis was rejected, and the existence of a publication bias could be confirmed. Unlike Egger’s regression test result, Begg’s test for rank correlation (Tau b = 0.27, ties = 0; Z = 1.75, p = 0.080) confirmed the absence of publication bias. Moreover, the trim-and-fill method suggested adding one article; the effect size of the 23 corrected articles was 0.60 (95% CI: 0.49, 0.72). Although the effect size of nursing competency was somewhat smaller after correction than before, it remained statistically significant. In conclusion, this study was deemed free of publication bias (Table 8).

Figure 3

Figure 4

4 Discussion

In this study, the impact of simulation-based programs on nursing competency demonstrated a significant effect size of 0.88. It’s notable that this simulation-based program yielded encouraging results by positively enhancing nursing competency. This is consistent with similar improvements observed in self-efficacy, a factor linked to nursing competency (19), enhanced knowledge, educational satisfaction, and academic achievement through VR in a hospital environment (39); and improved nursing-process performance (40), heightened critical thinking, clinical performance, and practice satisfaction through vSim for Nursing (41). Additionally, these results partially correlate with those in a study indicating that hands-on training utilizing scenario-based admission management in VR increased learning immersion, learner confidence, and learning satisfaction (7).

In the meta-regression analysis evaluating nursing competency, several factors emerged as influential. First, in cases where the publication year was 2022 or later, nursing competency was found to be significantly improved compared to studies that received IRB approval, compared to studies published before then. In the evolving landscape of clinical practice, recent emphasis on patient safety and rights has shifted the focus toward observing nursing behavior rather than direct patient care (42). This shift underscores the active implementation of simulation-based education, suggesting a more systematic adaptation of teaching methods and educational systems to enhance nursing competency compared to previous approaches. Moreover, studies with an intervention duration not reported or one of less than 4 weeks demonstrated a significant effect on nursing competency compared to those lasting more than 4 weeks. In cases of intervention with fewer than four sessions, competency was significantly improved compared to intervention sessions with four or more sessions. Similarly, interventions with time per session not reported or those lasting less than 1 h were associated with a significant improvement in nursing competency compared to those lasting more than 1 h. These findings suggest that shorter, more intensive interventions may be more effective in enhancing nursing competency through VRS. Establishing short-term intensive courses could thus be a meaningful approach. Even in the case of pre-briefings, which are recognized for their utility, the introduction and assignment of roles and expectations during pre-briefings may not be optimal. This is because simulation anxiety is linked to higher levels of extraneous cognitive load (43). Further investigation into the timing and temporal aspects of these activities is warranted to optimize their effectiveness. Therefore, further research specifically focusing on the temporal aspect is deemed necessary to comprehensively understand its implications.

Furthermore, pre-briefing before simulation emerged as a significant factor contributing to the improvement of nursing competency compared to that in the control group. This is consistent with the recognized importance of pre-briefing in face-to-face simulations, in which it influences simulation readiness (44). Given that most included studies conducted virtual pre-briefing activities individually, such as pre-briefing lessons and quizzes, the findings imply that virtual pre-briefing can be actively utilized with comparable effectiveness in face-to-face simulations. Various pre-briefing methods, including role rubrics, are currently under development (45). Further research will be necessary to ascertain the effectiveness of these diverse pre-briefing approaches.

Moreover, this study identified that post-simulation debriefing had a more significant effect of improving nursing competency compared to non-simulation debriefing. This could be attributed to the characteristic of VRS that enables repeated and reflective learning through debriefing with immediate feedback, thus providing learner-customized learning (46). The ability to facilitate individual improvement in nursing competency through immediate feedback is consistent with previous studies emphasizing the effectiveness and importance of debriefing in simulation (47). While debriefing in a virtual setting may differ from team interaction, reflection, and discussion in a face-to-face simulation, the results underscore the crucial role of debriefing in VRS situations.

Competency improved significantly when observation was measured rather than self-report. Role assignment in nursing simulation often elicits significant anxiety stemming from uncertainty, performing in front of faculty and peers, and social evaluation (45). Moreover, many individuals perceive themselves as lacking proficiency, particularly in terms of nursing competency. Consequently, self-reported improvements in nursing competency may underestimate actual progress observed through objective evaluation. Hence, effective communication and encouragement regarding the significance of simulation are vital when implementing simulation programs.

Nursing competency was statistically significantly improved when compared to those who did not engage in any other activities other than simulation. Other activities take as much time, which suggests that core simulation activities are important for improving nursing competency. Non-simulation activities, denoting the absence of activities other than simulation, exhibited a significant effect on nursing competency. While non-simulation activities may improve competencies such as team cooperation, communication, or empathy, they were not associated with improvements in nursing competency. This suggests that clear simulation content, along with pre-briefing and debriefing activities tailored to enhance nursing competency, directly influence this competency.

Meanwhile, several variables did not demonstrate a statistically significant effect of improving nursing competency. The country, number of centers, funding status, research design, and all the variables related to the operation of the intervention program (participants, intervention type, facilitator, intervention session, and outcome-measurement time), as well as the quality score, did not show significant differences in improving nursing competency. The inconsistency in trends observed across these variables can be attributed to the diverse definitions and measurements of nursing competency utilized in the included studies. This variability in research outcomes underscores the absence of a standardized measurement tool for nursing competency, potentially leading to increased heterogeneity in results.

Furthermore, the secondary outcomes measured alongside nursing competency in this study included knowledge, self-efficacy, problem-solving, confidence, and satisfaction. Among these, knowledge and confidence demonstrated statistically significant improvement. These variables, particularly knowledge and confidence, are closely related to nursing competency and can concurrently improve with it in VRS. Conversely, self-efficacy, problem-solving, and satisfaction did not show significant improvement. This is consistent with previous research indicating that VR nursing education improves knowledge (48) and increases learning satisfaction (49) but does not enhance technical skills (48) or significantly impact self-efficacy (49). This suggests that while VRS is effective in improving knowledge-related competencies, consistent improvements in self-efficacy, problem-solving, and satisfaction may depend on its design and utilization.

Given that learning immersion through simulation has been demonstrated to impact the development of clinical-nursing competence (50), and VR-based programs have been effective in improving cognitive performance, such as theoretical knowledge, through realism (51), VRS holds promise in nursing education. However, the results of this study underscore the need to carefully consider elements that are more challenging to implement in virtual situations than in face-to-face scenarios. Therefore, further research, such as systematic reviews and meta-analyses exploring other variables in VRS, is recommended for a more comprehensive understanding of its impact on nursing education.

VR-based nursing education represents an innovative field that has not been previously explored. These simulators offer a range of environments that transcend physical constraints, enabling participants to immerse themselves within the virtual space (52). It’s crucial for educators responsible for program development to grasp the distinctions between virtual reality and reality to facilitate effective education.

This study underscores the significance of pre-briefing and debriefing elements in VR-based simulation, highlighting the importance of their organization. Rather than focusing solely on operational time or the duration of the simulation itself, the key lies in how these elements are implemented for optimal educational outcomes. Additionally, when assessing effectiveness, we advocate for a combined approach utilizing both self-reported evaluations and objective evaluations through observation or assessment.

4.1 Limitations of the study

This study acknowledges several limitations. First, there is variability in reporting randomization methods among the included studies, with some providing comprehensive discussions on the topic while others lack detailed information on the methods employed. Second, the diverse interpretations and definitions of nursing competency across the included studies may introduce variability in the study outcomes. Third, the absence of a standardized measurement tool for nursing competency could contribute to increased heterogeneity. Fourth, the selection criteria for the studies analyzed, which included only those published in English or Korean and reported precise means, standard deviations, and sample sizes, could lead to selection bias and limit the generalization of our study results. Additionally, the studies encompass sample sizes from different countries, further contributing to overall heterogeneity. To enhance the robustness of future research and validate the effectiveness of interventions for nursing students, larger sample sizes and higher-quality studies are recommended.

5 Conclusion

The meta-analysis of nursing competency in VRS revealed the latter’s effectiveness in enhancing nursing competency. Notably, the incorporation of key elements from face-to-face simulation, such as pre-briefing and debriefing, significantly improved nursing competency compared to scenarios in which these elements were absent. This study suggests the importance of reflecting core simulation elements in virtual simulations and underscores the need to enhance the quality of pre-briefing and debriefing in virtual contexts. Moreover, the findings suggest that intensively operating VRS over a short period could be more effective in improving nursing competency. This implies the significance of considering the effectiveness of short-term intensive courses for nursing-competency improvement within virtual spaces. The study findings provide valuable insights for the design of VRS aimed at enhancing nursing competency.

References

• 1.

  Han D-L . Nursing students’ perception of virtual reality (VR) and needs assessment for virtual reality simulation in mental health nursing. J Digit Contents Soc. (2020) 21:1481–7. doi: 10.9728/dcs.2020.21.8.1481

  • CrossRef
  • Google Scholar

• 2.

  Foronda CL Fernandez-Burgos M Nadeau C Kelley CN Henry MN . Virtual simulation in nursing education: a systematic review spanning 1996–2018. Simul Healthc. (2020) 15:46–54. doi: 10.1097/SIH.0000000000000411

  • CrossRef
  • Google Scholar

• 3.

  Foronda CL . What is virtual simulation?Clin Simul Nurs. (2021) 52:8. doi: 10.1016/j.ecns.2020.12.004

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 4.

  Shin H Rim D Kim H Park S Shon S . Educational characteristics of virtual simulation in nursing: an integrative review. Clin Simul Nurs. (2019) 37:18–28. doi: 10.1016/j.ecns.2019.08.002

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 5.

  Foronda C Gattamorta K Snowden K Bauman EB . Use of virtual clinical simulation to improve communication skills of baccalaureate nursing students: a pilot study. Nurse Educ Today. (2014) 34:e53–7. doi: 10.1016/j.nedt.2013.10.007

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 6.

  Irwin P Coutts R . A systematic review of the experience of using second life in the education of undergraduate nurses. J Nurs Educ. (2015) 54:572–7. doi: 10.3928/01484834-20150916-05

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 7.

  Kim YJ . Development and application of scenario-based Admission Management VR contents for nursing students. J Korea Soc Comput Inf. (2021) 26:209–16. doi: 10.9708/jksci.2021.26.01.209

  • CrossRef
  • Google Scholar

• 8.

  Butt AL Kardong-Edgren SK Ellertson A . Using game-based virtual reality with haptics for skill acquisition. Clin Simul Nurs. (2018) 16:25–32. doi: 10.1016/j.ecns.2017.09.010

  • CrossRef
  • Google Scholar

• 9.

  Jung A Kwon E Seo J . Effects of nursing skills simulation program using virtual reality (VR) on learning flow, nursing skills confidence, nursing skills performance and usability verification. J Korea Acad-Ind Coop Soc. (2022) 23:127–35. doi: 10.5762/KAIS.2022.23.11.127

  • CrossRef
  • Google Scholar

• 10.

  Kim JW . Virtual reality (VR) based sustainable food education contents for elementary school students. Korean Assoc Pract Arts Edu. (2019) 32:45–63. doi: 10.24062/kpae.2019.32.4.45

  • CrossRef
  • Google Scholar

• 11.

  Ahn MK . The development and effects of metaverse-based core nursing skill contents of vital signs measurements and subcutaneous injections for nursing students. J Korean Acad Soc Nurs Educ. (2022) 28:378–88. doi: 10.5977/jkasne.2022.28.4.378

  • CrossRef
  • Google Scholar

• 12.

  Lee JS . Implementation and evaluation of a virtual reality simulation intravenous injection training system. Int J Environ Res Public Health. (2022) 19:5439. doi: 10.3390/ijerph19095439

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 13.

  Ahn MK Lee CM . Development and effects of head-mounted display-based home-visits virtual reality simulation program for nursing students. Korean Soc Nurs Sci. (2021) 51:465–77. doi: 10.4040/jkan.21051

  • CrossRef
  • Google Scholar

• 14.

  Rossler KL Sankaranarayanan G Duvall A . Acquisition of fire safety knowledge and skills with virtual reality simulation. Nurse Educ. (2019) 44:88–92. doi: 10.1097/NNE.0000000000000551

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 15.

  Aebersold M Voepel-Lewis T Cherara L Weber M Khouri C Levine MD et al . Interactive anatomy, augmented virtual simulation training. Clin Simul Nurs. (2018) 15:34–41. doi: 10.1016/j.ecns.2017.09.008

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 16.

  An J Oh J Park K . Self-regulated learning strategies for nursing students: a pilot randomized controlled trial. Int J Environ Res Public Health. (2022) 19:9058. doi: 10.3390/ijerph19159058

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 17.

  Chang CY Panjaburee P Chang SC . Effects of integrating maternity VR based situated learning into professional training on students’ learning performances. Interact Learn Environ. (2022) 2022:1–15. doi: 10.1080/10494820.2022.2141263

  • CrossRef
  • Google Scholar

• 18.

  Kim MS Jeong HC . The effects and adaptation of augmented reality–based intradermal injection practice education for nursing students. J Korean Soc Simul Nurs. (2022) 10:93–104. doi: 10.17333/JKSSN.2022.10.2.93

  • CrossRef
  • Google Scholar

• 19.

  Ha YO Kwon SJ Kim J Song JH . Effects of nursing skills practice using VR (virtual reality) on competency and confidence in nursing skills, learning self-efficacy, and satisfaction of nursing students. J Ind Converg. (2022) 20:47–55. doi: 10.22678/JIC.2022.20.4.047

  • CrossRef
  • Google Scholar

• 20.

  You H Yang B . The effects of virtual reality simulation scenario application on clinical competency, problem solving ability and nursing performance confidence. J Korea Acad Ind Coop Soc. (2021) 22:116–26. doi: 10.5762/KAIS.2021.22.9.116

  • CrossRef
  • Google Scholar

• 21.

  Bae YS Shin KM . Effects of virtual reality simulation of core fundamental nursing skills for intravenous fluid infusion on nursing students. Korean J Care Manag. (2023) 46:95–119. doi: 10.22589/kaocm.2023.46.95

  • CrossRef
  • Google Scholar

• 22.

  Song YM . Online and blended learning application in psychiatric and mental health nursing practice program among nursing students. J Learn Cent Curric Instr. (2021) 21:289–303. doi: 10.22251/jlcci.2021.21.11.289

  • CrossRef
  • Google Scholar

• 23.

  Raman S Labrague LJ Arulappan J Natarajan J Amirtharaj A Jacob D . Traditional clinical training combined with high fidelity simulation based activities improves clinical competency and knowledge among nursing students on a maternity nursing course. Nurs Forum. (2019) 54:434–40. doi: 10.1111/nuf.12351

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 24.

  Cho IY Yun JY Moon SH . Development and effectiveness of a metaverse reality-based family-centered handoff education program in nursing students. J Pediatr Nurs. (2024) 76:176–91. doi: 10.1016/j.pedn.2024.02.005

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 25.

  Lee E Baek G . Development and effects of a virtual reality simulation nursing education program combined with clinical practice based on an information processing model. Comput Inform Nurs. (2023) 41:1016–25. doi: 10.1097/CIN.0000000000001051

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 26.

  Kim J Heo N . Effect of augmented reality smart glasses-based nursing skills training for nursing students’ medication administration safety competency: a quasi-experimental study. J Korean Acad Fundam Nurs. (2023) 30:449–58. doi: 10.7739/jkafn.2023.30.4.449

  • CrossRef
  • Google Scholar

• 27.

  Park S Yoon HG . Effect of virtual-reality simulation of indwelling catheterization on nursing students’ skills, confidence, and satisfaction. Clin Simul Nurs. (2023) 80:46–54. doi: 10.1016/j.ecns.2023.05.001

  • CrossRef
  • Google Scholar

• 28.

  Karaduman GS Basak T . Is virtual patient simulation superior to human patient simulation: a randomized controlled study. CIN Comput Inform Nu. (2023) 41:467–76. doi: 10.1097/CIN.0000000000000957

  • CrossRef
  • Google Scholar

• 29.

  Moon SH . Metaverse based emergency nursing educational program using V-story. Crisis. (2023) 19:79–89.

  • Google Scholar

• 30.

  Lee JJ . The effect of virtual reality simulation training on critical thinking disposition, clinical competency, and self-efficacy of nursing students. J Korea Acad Ind Coop Soc. (2023) 24:390–7. doi: 10.5762/KAIS.2023.24.12.390

  • CrossRef
  • Google Scholar

• 31.

  Ahn MK . Development and effects of metaverse-based CPR training. J Digit Contents Soc. (2023) 24:1347–52. doi: 10.9728/dcs.2023.24.6.1347

  • CrossRef
  • Google Scholar

• 32.

  Tufanaru C Munn Z Aromataris E Campbell J Hopp L . Chapter 3. Systematic reviews of effectiveness In: AromatarisEMunnZ, editors. JBI manual for evidence synthesis (2020). JBI; 2024.

  • Google Scholar

• 33.

  Borenstein M Hedges LV Higgins JPT Rothstein HR . Introduction to Meta-analysis. West Sussex, UK: John Wiley & Sons (2009).

  • Google Scholar

• 34.

  Higgins JPT Thompson SG . Quantifying heterogeneity in a meta-analysis. Statist Med. (2002) 21:1539–58. doi: 10.1002/sim.1186

  • CrossRef
  • Google Scholar

• 35.

  Higgins JPT , Green SE (2011). Available at:http://www.cochrane-handbook.org (Accessed September 13, 2023).

  • Google Scholar

• 36.

  Mavridis D Salanti G . How to assess publication bias: funnel plot, trim-and-fill method and selection models. Evid Based Ment Health. (2014) 17:30. doi: 10.1136/eb-2013-101699

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 37.

  Brydges CR . Effect size guidelines, sample size calculations, and statistical power in gerontology. Innov. Aging. (2019) 3:igz036. doi: 10.1093/geroni/igz036

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 38.

  Bown MJ Sutton AJ . Quality control in systematic reviews and meta-analyses. Eur J Vasc Endovasc Surg. (2010) 40:669–77. doi: 10.1016/j.ejvs.2010.07.011

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 39.

  Kim MG Kim HW . The effects of classes using virtual reality simulations of the hospital environment on knowledge of the hospital environment, academic self-efficacy, learning flow, educational satisfaction and academic achievement in nursing students. J Korean Acad Fundam Nurs. (2021) 28:520–9. doi: 10.7739/jkafn.2021.28.4.520

  • CrossRef
  • Google Scholar

• 40.

  Lim JH . The effect of virtual reality simulation education on nursing process competency. J Digit Converg. (2021) 19:401–9. doi: 10.14400/JDC.2021.19.9.401

  • CrossRef
  • Google Scholar

• 41.

  Kim S Kim MJ . Effect of learner-centered virtual reality simulation education. J Digit Converg. (2022) 20:705–13. doi: 10.14400/JDC.2022.20.4.705

  • CrossRef
  • Google Scholar

• 42.

  Yang SJ Chae MJ . Effect of nursing students’ practical training on nursing simulation for blood transfusion recipients using online virtual reality. J Digit Contents Soc. (2024) 25:143–51. doi: 10.9728/dcs.2024.25.1.143

  • CrossRef
  • Google Scholar

• 43.

  Fredericks S ElSayed M Hammad M Abumiddian O Istwani L Rabeea A et al . Anxiety is associated with extraneous cognitive load during teaching using high-fidelity clinical simulation. Medical education. Online. (2021) 26:1994691. doi: 10.1080/10872981.2021.1994691

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 44.

  Brennan BA . The impact of self-efficacy based prebriefing on nursing student clinical competency and self-efficacy in simulation: an experimental study. Nurse Educ Today. (2022) 109:105260. doi: 10.1016/j.nedt.2021.105260

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 45.

  Dodson TM Reed JM . Enhancing simulation preparation: Presimulation role rubrics and expert Modeling videos. Clin Simul Nurs. (2024) 87:101498. doi: 10.1016/j.ecns.2023.101498

  • CrossRef
  • Google Scholar

• 46.

  Lim S Yeom YR . The effect of education integrating virtual reality simulation training and outside school clinical practice for nursing students. J Converg Inf Technol. (2020) 10:100–8.

  • Google Scholar

• 47.

  Loomis A Dreifuerst KT Bradley CS . Acquire, apply, and retain knowledge through debriefing for meaningful learning. Clin Simul Nurs. (2022) 68:28–33. doi: 10.1016/j.ecns.2022.04.002

  • CrossRef
  • Google Scholar

• 48.

  Chen FQ Leng YF Ge JF Wang DW Li C Chen B et al . Effectiveness of virtual reality in nursing education: a meta-analysis. J Med Internet Res. (2020) 22:e18290. doi: 10.2196/18290

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 49.

  Padilha JM Machado PP Ribeiro A Ramos J Costa P . Clinical virtual simulation in nursing education: randomized controlled trial. J Med Internet Res. (2019) 21:e11529. doi: 10.2196/11529

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 50.

  Kim HW Suh EY . Nursing students’ immersion experience in a comprehensive simulation scenario using high-fidelity human patient simulator among nursing students: a phenomenological study. J Mil Nurs Res. (2012) 30:89–99.

  • Google Scholar

• 51.

  Shorey S Ng ED . Use of virtual reality simulation among nursing students and registered nurses: a systematic review. Nurse Educ Today. (2021) 98:104662. doi: 10.1016/j.nedt.2020.104662

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 52.

  Hwang YJ Jeong JY Jeong YM . A study on the feasibility of introducing XR in nursing education Core fundamental nursing skills. J Digit Contents Soc. (2023) 24:775–83. doi: 10.9728/dcs.2023.24.4.775

  • CrossRef
  • Google Scholar