Enhancing nursing competency through virtual reality simulation among nursing students: a systematic review and meta-analysis

Aim Studies on the effectiveness of virtual reality (VR) in nursing education have explored its impact on learning outcomes, emotional immersion and engagement, learner self-confidence, and satisfaction, generally showing positive aspects. However, there is a need for a systematic review to examine the specific influence of VR-based education on nursing students’ practical competency. Method According to the PRISMA 2020 guidelines, 22 studies were selected based on inclusion criteria from 579 articles, published from January 1, 2018, to March 31, 2024, across nine major databases including PubMed and EMbase. The target population comprised nursing students, and the intervention focused on VR-based simulations aimed at enhancing competency, compared to control groups receiving either no intervention or conventional non-virtual simulation. The primary outcome, nursing competency, was analyzed using MIX 2.0 Pro (Ver. 2.0.1.6, BiostatXL, 2017) to calculate pooled effect sizes. Result The pooled effect size for nursing competency was determined to be large, with Hedge’s g = 0.88 (95% CI, 0.47 to 1.29). Meta-regression analysis identified several factors associated with an increase in nursing competency. These included studies published after 2022, approval of an IRB, absence of funding, randomized controlled trials (RCTs), interventions reported as shorter than 4 weeks or not reported, sessions fewer than 4 or not reported, session duration under 1 h or not reported, and observational measurement methods. Additional factors enhancing nursing competency were the inclusion of a pre-briefing before simulations, the absence of a debriefing afterward, and the exclusion of other activities during the simulation. Conclusion By combining the results of the included studies, the systematic review and meta-analysis accounted for variations in sample size, study methodology, and independent intervention effects, providing an overall evaluation of the effectiveness of simulation-based education in improving nursing students’ competency. Limitation The selection criteria for the studies analyzed, which included only those published in English or Korean and reported precise means, standard deviations, and sample sizes, could lead to selection bias and limit the generalization of our study results. Systematic review registration PROSPERO International Prospective Register of Systematic Reviews: http://www.crd.york.ac.uk/PROSPERO/, identifier CRD42023446348.


Introduction
Nursing education is an applied discipline in which theory and practical education are combined; prospective nurses prepare to become competent by applying the knowledge learned in theoretical education to the practical education process.The need for nursing education to train professionals who provide nursing and medical services to humans utilizing digital-based, non-face-to-face media such as artificial intelligence (AI) and big data has recently become more urgent (1).In nursing education, there has been an increasing interest in virtual-reality simulation (VRS) education as an alternative and complementary method to traditional simulation education, providing students with new learning experiences in a reproduced clinical environment and enhancing clinical adaptability (2).Virtual reality (VR) is defined as "the use of partial immersion through a digital learning environment (computer, tablet, phone, screen, etc.) to foster a perceived lived experience for an intended outcome (e.g., learning and entertainment)" (3).This study defines VRS to include VR and its derivatives, augmented reality (AR), and mixed reality (MR), using the terminology consistently.In VRS, learners can collaborate with other healthcare professionals to provide interventions, such as solving patients' problems or practicing simple skills (4,5).Improved clinical performance skills, knowledge, and metacognition, as well as enhanced learning satisfaction, communication, self-efficacy, confidence, and teamwork have been reported as effects of these VR programs (4,6).In addition, studies on the effectiveness of nursing education using VR have been conducted on learning effectiveness, emotional engagement and immersion, learner confidence, and satisfaction (7,8).Reportedly, VRS programs for nursing skills are effective in improving skills (9) and have the advantage of enabling safe and repetitive training without time and space constraints (10).Thus, learning through VRS has demonstrated improvement in various factors related to clinical nursing competency, albeit often assessed in a fragmented manner.As various forms of VRS are being applied in nursing education, and diverse elements contributing to nursing competency are considered, there is a need to comprehend the holistic outcomes of these studies.Consequently, this study aims to comprehensively review the results, considering nursing competency in a broader sense that encompasses collaboration, interpersonal relationships, communication, professional development, and the nursing process, skills, and education (11).
Moreover, a systematic review and analysis of nursing students' outcomes are essential for determining specific factors that are deemed effective.Systematic reviews and meta-analyses can amalgamate the results of included studies, accounting for differences in sample size, variations in research approaches, and intervention effects among independent studies.We believe that the systematic review and meta-analysis in this study will enable an assessment of the overall effect of VRS-based education on nursing students' nursing competency.Consequently, this study aims to provide foundational data on VRS by conducting a systematic literature review and metaanalysis, investigating the improvement effect of VRS on nursing students' nursing competency as a primary outcome, and examining knowledge, self-efficacy, problem-solving skills, confidence, and satisfaction as secondary outcomes.
This study aims to acquire and analyze evidence regarding the enhancement of nursing students' nursing competency through VRS.The primary outcome focuses on nursing students' self-reported feelings and reactions, while the secondary outcome assesses nursing students' nursing competency following exposure to VRS.

Search strategy and data sources
The search was jointly conducted by two researchers, Cho, M.-K.and Kim, M.Y., across nine electronic databases or e-journals: PubMed, Cochrane, EMBASE-OVID, CINAHL, World of Science, SCOPUS, PQDT, APA PsycArticles, and Research Information Sharing Service.The primary search, conducted from July 18, 2023, to August 20, 2023, targeted articles published in English and Korean from January 1, 2003, to April 30, 2023.A secondary search was carried out from April 6, 2024, to April 9, 2024, focusing on articles published from May 1, 2023, to March 31, 2024, also in English and Korean.The search strategy and formula, following the PICO-SD framework (population, intervention, comparison, outcome, study design), are detailed in Table 1.The keywords employed in search terms across the nine databases included combinations and variations of "nursing students, " "virtual reality, " "augmented reality, " "extended reality, " "metaverse, " "competency-based education, " "clinical competence, " "competency, " and "controlled clinical trial." These keywords were chosen to comprehensively capture studies relevant to the impact of virtual reality simulation on nursing competency.

Inclusion and exclusion criteria
The reporting of the results adhered to the PRISMA 2020 checklist.Inclusion criteria comprised nursing students aged 19 years or older (Population), interventions involving VRS (Intervention), with conventional learning methods or no intervention as the control (comparison).The primary outcome was nursing competency, and secondary outcomes included knowledge, self-efficacy,

Data extraction
Two researchers, Cho, M.-K.and Kim, M.Y., independently conducted searches and selected studies for analysis based on the predefined inclusion and exclusion criteria.The selected studies were extracted, incorporating information such as author, year of publication, country, publication language, number of schools, institutional review board (IRB) approval, funding details, number of participants, study design, intervention characteristics (type, facilitator, duration, session, time/session, pre-briefing, debriefing, other activities, outcome measurement time, and measurement method), quality assessment score, and dependent variables.This information was meticulously recorded in a coding book created using the Microsoft Excel spreadsheet software.Any disparities in coding were addressed by revisiting the original text to ascertain and input the final coding values (Table 2).

Quality assessment
The quality assessment of selected articles was independently performed by Cho, M.-K.and Kim, M.Y. using the Joanna Briggs Institute (JBI) Checklist for RCTs and the Checklist for Quasi-Experimental Studies.Five RCTs were assessed using the 13-question JBI Checklist; the average score was 8.40, and all five studies lacked clear reporting on "Q2.Was allocation to treatment groups concealed?"and "Q4.Were participants blind to treatment assignment?" Quasi-experimental studies comprised eight articles, and on evaluation using the 9-item JBI Checklist for Quasi-Experimental Studies (32), the average score was 8.50, with generally well-reported items (Table 3).

Statistical analyses
MIX 2.0 Pro (Ver.2.0.1.6,BiostatXL, 2017) was used to calculate and merge effect sizes for both the primary outcome of nursing competency and secondary outcomes.The overall effect was determined using a random-effects model, considering betweensubject variability and heterogeneity between studies.Hedge's g was employed for effect-size calculation, and significance was assessed using 95% confidence intervals (CIs), Z tests, and p-values.The weight of each effect size was determined using the inverse of variance (33)  Heterogeneity was evaluated using Higgin's I 2 (34), with an I 2 of >50% indicating heterogeneity (35).Subgroup analysis, meta-regression, and exclusion-sensitivity analysis were conducted for nursing competency to identify factors contributing to heterogeneity.Publication bias was examined using funnel plots, trim-and-fill plots, Begg's test, Egger's regression, and the trim-and-fill method to correct for the overall effect (36).

Characteristics of the included studies
A total of 579 articles were initially identified from 9 databases following the search strategy.After excluding duplicates, 373 articles were extracted.Following the application of the inclusion and exclusion criteria, 21 research articles were ultimately selected.The research by Sahin Karaduman and Basak (28) was designed using two experimental groups and was analyzed as two separate studies, resulting in 22 studies being analyzed (Figure 1).Of these, six studies were published before 2022; three were conducted in the United States (USA), twelve studies were published in English; nineteen were conducted at a single university; nineteen and nine studies had IRB approval and funding, respectively.The study designs included seven RCTs, fifteen quasi-experimental studies, and eight studies with fewer than 60 participants.Interventions comprised 18 VR/AR simulations and four metaverse.Eight studies had a professor as a facilitator, four had an intervention duration of more than 4 weeks, two had four or more intervention sessions, eight had an intervention time of more than 1 h per session, 19 had a pre-briefing, and nine had a debriefing.Dependent-variable measurements were taken immediately after the intervention in 20 studies, the measurement method was observational measurement in 12 studies, 19 studies had no additional activities, such as reflection, besides the simulation, and 14 studies had an above-average quality assessment score (Table 2).

Effect of VRS-based intervention on nursing competency
The overall effect of nursing competency, as the primary outcome for the 22 VRSs, was found to be Hedge's g = 0.88 (95% CI: 0.47 to 1.29).This was interpreted as a large effect based on the criteria provided by Brydges (37) for interpreting effect sizes (Figure 2).The high degree of heterogeneity among the studies, indicated by Higgins's I 2 of 91.8% in the heterogeneity test, prompted subgroup and meta-regression analyses to explore factors contributing to this heterogeneity.
The exclusion-sensitivity test (38), excluding one study at a time, showed Hedge's g ranging from 0.67 to 0.94, indicating a moderate to large effect size.The 95% CI (0.36 ~ 0.53, 0.98 ~ 1.36) did not include 0, signifying statistical significance.The effect sizes from the exclusionsensitivity test were not significantly different from Hedge's g = 0.88, which included all 22 studies (Table 6).

Publication bias
To evaluate publication bias, funnel-plot and trim-and-fill plot analyses were conducted.Represented by the black circle, the individual effect sizes of the 22 studies included in the study were asymmetrical-slightly skewed to the right-indicating some potential publication bias (Figure 3).The trim-and-fill plot suggested the addition of one study, represented by the white circle, skewing to the left (Figure 4).Further analysis, The coefficient of the bias was 8.58, indicating the initial value (intercept) and the p-value was 0.001.Thus, the null hypothesis was rejected, and the existence of a publication bias could be confirmed.Unlike Egger's regression test result, Begg's test for rank correlation (Tau b = 0.27, ties = 0; Z = 1.75, p = 0.080) confirmed the absence of publication bias.Moreover, the trim-and-fill method suggested adding one article; the effect size of the 23 corrected articles was 0.60 (95% CI: 0.49, 0.72).Although the effect size of nursing competency was somewhat smaller after correction than before, it remained statistically significant.In conclusion, this study was deemed free of publication bias (Table 8).

Discussion
In this study, the impact of simulation-based programs on nursing competency demonstrated a significant effect size of 0.88.It's notable that this simulation-based program yielded encouraging results by positively enhancing nursing competency.This is consistent with similar improvements observed in self-efficacy, a factor linked to nursing competency (19) In the meta-regression analysis evaluating nursing competency, several factors emerged as influential.First, in cases where the publication year was 2022 or later, nursing competency was found to be significantly improved compared to studies that received IRB approval, compared to studies published before then.In the evolving landscape of clinical practice, recent emphasis on patient safety and rights has shifted the focus toward observing nursing behavior rather than direct patient care (42).This shift underscores the active implementation of simulation-based education, suggesting a more systematic adaptation of teaching methods and educational systems to enhance nursing competency compared to previous approaches.Moreover, studies with an intervention duration not reported or one of less than 4 weeks demonstrated a significant effect on nursing competency compared to those lasting more than 4 weeks.In cases of intervention with fewer than four sessions, competency was significantly improved compared to intervention sessions with four or more sessions.Similarly, interventions with time per session not reported or those lasting less than 1 h were associated with a significant improvement in nursing competency compared to those lasting more than 1 h.These findings suggest that shorter, more intensive The effect of virtual reality simulation-based intervention on nursing competency.ES, effect size; CI, confidence interval.An article by Sahin Karaduman and Basak (29), designed using two experimental groups, was divided into two studies (18 a and 18 b ).*Removal of the number of duplicate subjects in the 18th study.interventions may be more effective in enhancing nursing competency through VRS.Establishing short-term intensive courses could thus be a meaningful approach.Even in the case of pre-briefings, which are recognized for their utility, the introduction and assignment of roles and expectations during pre-briefings may not be optimal.This is because simulation anxiety is linked to higher levels of extraneous cognitive load (43).Further investigation into the timing and temporal aspects of these activities is warranted to optimize their effectiveness.Therefore, further research specifically focusing on the temporal aspect is deemed necessary to comprehensively understand its implications.Furthermore, pre-briefing before simulation emerged as a significant factor contributing to the improvement of nursing competency compared to that in the control group.This is consistent with the recognized importance of pre-briefing in face-to-face simulations, in which it influences simulation readiness (44).Given that most included studies conducted virtual pre-briefing activities individually, such as pre-briefing lessons and quizzes, the findings imply that virtual pre-briefing can be actively utilized with comparable effectiveness in face-to-face simulations.Various pre-briefing methods, including role rubrics, are currently under development (45).Further research will be necessary to ascertain the effectiveness of these diverse pre-briefing approaches.
Moreover, this study identified that post-simulation debriefing had a more significant effect of improving nursing competency compared to non-simulation debriefing.This could be attributed to the characteristic of VRS that enables repeated and reflective learning through debriefing with immediate feedback, thus providing learnercustomized learning (46).The ability to facilitate individual improvement in nursing competency through immediate feedback is consistent with previous studies emphasizing the effectiveness and importance of debriefing in simulation (47).While debriefing in a virtual setting may differ from team interaction, reflection, and discussion in a face-to-face simulation, the results underscore the crucial role of debriefing in VRS situations.
Competency improved significantly when observation was measured rather than self-report.Role assignment in nursing simulation often elicits significant anxiety stemming from uncertainty, performing in front of faculty and peers, and social evaluation (45).Moreover, many individuals perceive themselves as lacking proficiency, particularly in terms of nursing competency.Consequently, self-reported improvements in nursing competency may underestimate actual progress observed through objective evaluation.Hence, effective communication and encouragement regarding the significance of simulation are vital when implementing simulation programs.
Nursing competency was statistically significantly improved when compared to those who did not engage in any other activities other than simulation.Other activities take as much time, which suggests that core simulation activities are important for improving nursing competency.Non-simulation activities, denoting the absence of activities other than simulation, exhibited a significant effect on nursing competency.While non-simulation activities may improve competencies such as team cooperation, communication, or empathy, they were not associated with improvements in nursing competency.This suggests that clear simulation content, along with pre-briefing and debriefing activities tailored to enhance nursing competency, directly influence this competency.Meanwhile, several variables did not demonstrate a statistically significant effect of improving nursing competency.The country, number of centers, funding status, research design, and all the variables related to the operation of the intervention program (participants, intervention type, facilitator, intervention session, and outcome-measurement time), as well as the quality score, did not show significant differences in improving nursing competency.The inconsistency in trends observed across these variables can be attributed to the diverse definitions and measurements of nursing competency utilized in the included studies.This variability in research outcomes underscores the absence of a standardized measurement tool for nursing competency, potentially leading to increased heterogeneity in results.
Furthermore, the secondary outcomes measured alongside nursing competency in this study included knowledge, self-efficacy, problem-solving, confidence, and satisfaction.Among these, knowledge and confidence demonstrated statistically significant improvement.These variables, particularly knowledge and confidence, are closely related to nursing competency and can concurrently improve with it in VRS.Conversely, self-efficacy, problem-solving, and satisfaction did not show significant improvement.This is consistent with previous research indicating that VR nursing education improves knowledge (48) and increases learning satisfaction (49) but does not enhance technical skills (48) or significantly impact self-efficacy (49).This suggests that while VRS is effective in improving knowledgerelated competencies, consistent improvements in self-efficacy, problem-solving, and satisfaction may depend on its design and utilization.
Given that learning immersion through simulation has been demonstrated to impact the development of clinical-nursing competence (50), and VR-based programs have been effective in improving cognitive performance, such as theoretical knowledge, through realism (51), VRS holds promise in nursing education.However, the results of this study underscore the need to carefully consider elements that are more challenging to implement in virtual situations than in face-to-face scenarios.Therefore, further research, Funnel plot of virtual reality simulation-based intervention on nursing competency.Precision = 1/standard error; 0.05; limit line = 95% confidence limit. 10.3389/fmed.2024.1351300 Frontiers in Medicine 17 frontiersin.orgsuch as systematic reviews and meta-analyses exploring other variables in VRS, is recommended for a more comprehensive understanding of its impact on nursing education.VR-based nursing education represents an innovative field that has not been previously explored.These simulators offer a range of environments that transcend physical constraints, enabling participants to immerse themselves within the virtual space (52).It's crucial for educators responsible for program development to grasp the distinctions between virtual reality and reality to facilitate effective education.
This study underscores the significance of pre-briefing and debriefing elements in VR-based simulation, highlighting the importance of their organization.Rather than focusing solely on operational time or the duration of the simulation itself, the key lies in how these elements are implemented for optimal educational outcomes.Additionally, when assessing effectiveness, we advocate for a combined approach utilizing both self-reported evaluations and objective evaluations through observation or assessment.

Limitations of the study
This study acknowledges several limitations.First, there is variability in reporting randomization methods among the included studies, with some providing comprehensive discussions on the topic while others lack detailed information on the methods employed.Second, the diverse interpretations and definitions of nursing competency across the included studies may introduce variability in the study outcomes.Third, the absence of a standardized measurement tool nursing competency could contribute to increased heterogeneity.Fourth, the selection criteria for the studies analyzed, which included only those published in English or Korean and reported precise means, standard deviations, and sample sizes, could lead to selection bias and limit the generalization of our study results.Additionally, the studies encompass sample sizes from different countries, further contributing to overall heterogeneity.To enhance the robustness of future research and validate the effectiveness of interventions for nursing students, larger sample sizes and higher-quality studies are recommended.

Conclusion
The meta-analysis of nursing competency in VRS revealed the latter's effectiveness in enhancing nursing competency.Notably, the  Trim and fill plot of virtual reality simulation-based intervention on nursing competency.Precision = 1/standard error; 0.05; limit line = 95% confidence limit.incorporation of key elements from face-to-face simulation, such as pre-briefing and debriefing, significantly improved nursing competency compared to scenarios in which these elements were absent.This study suggests the importance of reflecting core simulation elements in virtual simulations and underscores the need to enhance the quality of pre-briefing and debriefing in virtual contexts.Moreover, the findings suggest intensively operating VRS over a short period could be more effective in improving nursing competency.This implies the significance of considering the effectiveness of short-term intensive courses for nursing-competency improvement within virtual spaces.The study findings provide valuable insights for the design of VRS aimed at enhancing nursing competency.

TABLE 1
Search strategy according to PICO.

TABLE 3
Quality assessment of the included studies.
, enhanced knowledge, educational (28)RE 1PRISMA flow diagram.An article by Sahin Karaduman and Basak(28), designed using two experimental groups, was divided into two studies.

TABLE 4
Subgroup analysis of nursing competency according to study characteristics.

TABLE 5
Meta-regression analysis to evaluate competency.

TABLE 6
Exclusion-sensitivity test of the virtual-reality simulation-based intervention.
K, number of analysis sets; ES, effect size; CI, confidence interval.

TABLE 7
Effects of virtual reality simulation-based intervention on other variables.

TABLE 8
Publication bias test of virtual reality simulation-based intervention on competency.'s test for rank correlation; Egger's regression test for zero intercepts; SE, standard error; CI, confidence interval; K, number of analysis sets; ES, effect size. Begg