REVIEW article

Front. Psychiatry, 12 May 2025

Sec. Neurostimulation

Volume 16 - 2025 | https://doi.org/10.3389/fpsyt.2025.1432314

This article is part of the Research TopicAdvancing Education in Neurostimulation: Nurturing Competent and Collaborative Clinicians and ResearchersView all 5 articles

Advancing education in interventional psychiatry: scoping review of simulation training and the future of virtual reality-based learning

Peter Giacobbe,,*Peter Giacobbe1,2,3*Alice Tu,Alice Tu1,4Lilia Kaustov,Lilia Kaustov4,5Stephen Choi,,Stephen Choi4,5,6Julian Wiegelmann,Julian Wiegelmann4,6Fahad Alam,Fahad Alam4,6
  • 1Department of Psychiatry, Sunnybrook Health Sciences Centre, Toronto, ON, Canada
  • 2Harquail Centre for Neuromodulation, Sunnybrook Health Sciences Centre, Toronto, ON, Canada
  • 3Department of Psychiatry, University of Toronto, Toronto, ON, Canada
  • 4Department of Anesthesia, Sunnybrook Health Sciences Centre, Toronto, ON, Canada
  • 5Perioperative Brain Health Centre, Sunnybrook Research Institute, Toronto, ON, Canada
  • 6Department of Anesthesiology and Pain Medicine, University of Toronto, Toronto, ON, Canada

Objectives: Interventional psychiatric procedures such as electroconvulsive therapy (ECT) and repetitive transcranial magnetic stimulation (rTMS) have become increasingly important therapeutic options for managing severe or treatment-resistant mental illnesses. However, research suggests that gaps in training students in these techniques represent a rate-limiting step for their further dissemination and accessibility for the public. Studies have shown that the majority of psychiatry residents lack necessary competency and self-confidence in performing these treatments. Simulation based training has served as a gold standard for training procedural skills in medicine. Simulation-based training environments, particularly immersive reality technology (e.g., virtual reality [VR]), represent a promising novel avenue for trainees to develop the necessary skills for delivering these treatments. This scoping review discusses the current training in interventional psychiatry and how simulation-based training, specifically VR, can improve pedagogy in this area.

Methods: In this scoping review, a literature search was conducted on the PubMed database using specific search terms such as “simulat*”, “training”, “ECT”, “TMS”, “neuromodulation”, and “interventional psychiatry”. The search was limited to studies with language in English from 1980 to 2023.

Results: The initial search yielded 2094 articles, of which 4 evaluated the effectiveness of simulation approaches for ECT and were included in this review. No published studies were identified regarding VR-based education in ECT or rTMS.

Conclusions: This scoping review provides an overview of the current landscape of pedagogical methods in interventional psychiatry and highlights the identified gaps in both the existing literature and the potential application of simulation-based environments, including VR, within this field. Considering the ongoing shift in medical education towards competency-based training, this review discusses the needs and benefits of VR-based simulators as an avenue to enhance competency in interventional psychiatry. Leveraging existing experience in the use of VR-based simulators in procedural skill acquisition in surgery and anesthesia, as well as recommendations on how to translate this approach to clinical training in psychiatry, are also discussed.

1 Introduction

1.1 The origins and scope of interventional psychiatry

With the continuing paradigmatic shift in the classification of mental disorders away from purely descriptive clinical phenomenology and towards emphasizing transdiagnostic biological systems in the brain underlying psychopathology (1), recent cohorts of psychiatry residents have expressed their interest in revamping clinical training to include wider integration of neuroscience-based education (2). The concept of interventional psychiatry was first described in 2014 to designate treatments for mental illness that are more procedural than general medical care within psychiatry and require specialized training in their delivery (3). Treatments that have traditionally been considered to fall under the rubric of interventional psychiatry have included brain stimulation treatments, such as electroconvulsive therapy (ECT) and repetitive transcranial magnetic stimulation (rTMS). ECT involves the application of an electrical current to the scalp to generate a therapeutic seizure under general anesthesia, whereas rTMS is a non-convulsive treatment that through the principle of electromagnetic induction, produces transient electrical currents in underlying cortical tissue, to either stimulate or inhibit brain activity, depending on the stimulation intensity and frequency (4). Both have proven antidepressant effects compared to sham (5), have first-line evidence as treatments for Major Depressive Disorder (MDD) (6), and have comparable positive effects on self-reported functioning in patients with treatment-resistant forms of depression (7).

The origins of interventional psychiatry have analogues in interventional cardiology and radiology, which both evolved over time through the consistent application of innovative technology to enhance patient outcomes (8, 9). Consistent with this, there has been interest in expanding interventional psychiatry beyond ECT and rTMS treatments, to incorporate newer procedures and modalities which do not directly apply energy to the brain, such as intravenous (IV) ketamine and intranasal esketamine. In the context of establishing the scope and breadth of a training program in interventional psychiatry at the University of Toronto, a survey was distributed to academic psychiatrists, asking which treatments they felt should be taught under the auspices of a competency diploma in interventional psychiatry (10). Participants included those that provide ECT and/or rTMS (deemed “interventionalists”, n=27) and those who do not deliver procedural or device-based treatments in their practice (i.e., “non-interventionalists”, n=37). There was 100% agreement that a competency diploma in interventional psychiatry should provide training in ECT and rTMS, but the majority also felt that IV ketamine (85.9%) and intranasal esketamine (71.9%) should be included, with equivalent rates of agreement amongst interventionalists and non-interventionalists alike. Approximately 1 in 5 respondents (19%) believed that other approaches, most commonly psychiatric neurosurgeries such as deep brain stimulation (DBS), MRI-guided focused ultrasound (FUS), and vagus nerve stimulation (VNS), should also be included.

This survey suggests that amongst academic psychiatrists, including non-interventionalists, there is wide acceptance of the value of training in interventional psychiatry. It sets a training mandate to extend the scope of this field to include ketamine treatments that have similar peri-procedural monitoring requirements as ECT (11). This is especially important in light of recent evidence of the comparable acute antidepressant effects of ECT and IV ketamine (12).

Based on the results of the survey described above (10) indicating that the majority of respondents advocate that training in Interventional Psychiatry should encompass ECT, rTMS, IV ketamine and intranasal esketamine, the remainder of the paper will explore the role of training in these modalities as barriers to their utilization. The majority of the discussion will focus on ECT, which has nearly a century of use as an established treatment and published literature. The exploration of barriers to the use ECT may provide insights into modifiable factors that would allow improved training and greater accessibility and utilization of other newer and emerging interventional treatment approaches in Psychiatry.

1.2 Barriers to greater utilization of interventional psychiatry treatments: lessons learned from ECT

Despite the fact that ECT arguably still represents the one of the most effective antidepressant treatments available and that it is considered a bellwether measure of a nation’s degree of financial investment in mental healthcare (13), evidence suggests that it remains underutilized and its use is stagnant or declining globally. ECT is the most geographically accessible of the interventional psychiatry modalities in Canada, with 84% of the population being able to access it within a one-hour drive (14). However, its rates of use in clinical settings remain low. Kaster analyzed healthcare registry data from 2009 to 2017 for all psychiatric admissions in Ontario with a diagnosis of a major depressive episode and found that only 1 in 11 (9%) involved the use of ECT (15), despite depressive disorders being the dominant clinical indication for ECT (6). During the period of 1992 to 2004, Rapoport et al. reported that approximately 12 per 100,000 Ontarians received ECT per year, without any significant change in utilization over this period (16). Administrative data from Quebec reported a decline in ECT utilization between 1996 and 2013 (17). Similar trends have been reported in the United States (18). To the best of our knowledge, no data from public registries regarding rTMS, IV ketamine, and IV esketamine treatment utilization has been published to date.

A number of factors have been identified as limiting greater ECT utilization. These include social and economic variables related to the cost of its delivery and misconceptions about the procedure among both the public and medical community (10). However, it is the pedagogy in these procedures during psychiatric training that has consistently emerged as the most proximal modifiable factor contributing to the low rates of ECT utilization (10). In a nationwide survey of ECT practitioners in the United States, the lack of trained ECT practitioners was identified as the top barrier to the implementation of an ECT program (19). Education about ECT during clinical training reduces the stigma among psychiatrists and increases the likelihood of patients being referred for this procedure in the future, even if the practitioners do not deliver ECT themselves (20). Therefore, education in ECT, and by extension other interventional approaches, during the critical developmental window of residency training may serve the dual purposes of improving accessibility to these treatments through both enhanced clinician awareness of these options in refractory patients and increasing the number of psychiatrists able to deliver it.

1.3 Trends in ECT and rTMS training

Both the expressed wish by psychiatric residents for more prominent inclusion of training in interventional psychiatry in residency programs and the data reviewed above suggest that educational experiences during this period of residency training are essential in increasing the accessibility of these treatments for the public. However, the existing literature on the rates of clinical competency in the delivery of ECT and rTMS reported by psychiatry trainees suggests that current educational paradigms have been inadequate.

With regards to ECT, training programs in the United States and Canada have advocated for psychiatry residents to obtain knowledge on its indications and uses, but without any formalized requirement for procedural exposure or competency (2123). In Canada, the majority of ECT programs do not have a formal teaching program and in the programs that did provide teaching, methods varied widely with the most common methods of pedagogy being didactic lecture and indirect observation of the procedure without experiential learning and supervision (23).

In a survey of the emerging psychiatric workforce in Canada, of 162 residents from across the country who attended a national review course in preparation for licensure, 91% agreed that familiarity with both the theory and administration of ECT should be required to become a psychiatrist. However, only 24% reported achieving this benchmark at the end of their training (24). Rates of self-reported competency in the administration of ECT have been largely unchanged in Canada over the last three decades, with a rate of 25% (25) reported in 1991 and 18% in 2002 (26).

The stagnant rates of ECT competency amongst trainees in Canada since the 1990s appear to be secondary to a lack of opportunities to perform the procedure during training, rather than a dearth of didactic lectures. In the survey described above, 86% reported adequate teaching in the theory of ECT during their curricula, whereas 69% reported adequate supervision in the procedural aspects of ECT (24). However, rates of self-reported competency were significantly increased in respondents who reported having the opportunity to deliver ECT on more than ten occasions during their training, providing empirical validation for the minimal training requirements that have been advocated in the US (21).

Beyond ECT, there are identified gaps in training for other interventional procedures in psychiatry. For rTMS, 57% of residents indicated that competency in the delivery of this procedure should be required for licensure as a psychiatrist, but only 3% reported achieving this by the end of their residency training (24). To our knowledge, rates of competency to deliver IV ketamine and intranasal esketamine have not yet been reported in the literature. There has been a proposal for a curriculum to provide multidisciplinary training in both non-invasive neurotechnologies and surgical approaches to neuropsychiatric illness such as VNS and DBS (27), although the efficacy of this pedagogical model remains to be evaluated.

In summary, current models of pedagogy in interventional psychiatry which have emphasized learning through didactic lectures and indirect observation of treatment delivery, have proven to be insufficient and are unlikely to be effective in further translating emerging research findings in this area to improve rates of utilization and accessibility for those that can benefit from these procedures. A promising avenue to increase the rates of skilled practitioners may be through evolving post-graduate curricula to introduce educational opportunities that change the focus to the development of procedural competency.

1.4 Pedagogy in interventional psychiatry: future opportunities

With the recognition of the important clinical role for interventional treatments in refractory psychiatric illness, a number of international groups are now advocating for formalized training in this subspecialty within psychiatry (28, 29). The International Federation of Clinical Neurophysiology envisions training programs matched to three distinct classes of trainees: 1) the technician who delivers the treatment to the patient, 2) the clinician who establishes the indication for the treatment and prescribes the appropriate protocol, and 3) the clinician-scientist who designs and provides oversight for a research study involving an interventional approach (28). In this model, while each class of trainee has its own set of core training competencies in line with the role they play in the treatment’s delivery, an individual practitioner may have more than one role. For example, a physician may be expected to be a principal investigator for a study protocol involving TMS or ECT, assess patients for the presence of clinical indications/contraindications for these treatments, and then directly deliver the treatments. As a result, there is a growing need for comprehensive and broad training in a wide variety of interventional treatments to ensure their safe and effective delivery (28).

Another promising change to potentially increase rates of competency in interventional psychiatry is the shift in medical education towards competency-based medical education (CBME) (30). The main tenet of CBME is its greater emphasis on the implementation of outcome-driven education and assessment (i.e., outputs) through a focus on demonstrable abilities and performance needed for clinical practice rather than the traditional time-based focus on training inputs (31). The CBME approach allows for the skills expected of a psychiatrist to be deconstructed into their smaller summative tasks or competencies, which trainees can master through continuous practice. Indeed, competencies to achieve core and advanced proficiency in the delivery of ECT and rTMS have been identified (28, 32, 33). For rTMS, these include basic operation of the TMS device (e.g., turning the machine on/off, attaching coils, setting stimulation parameters, proper coil placement to establish the resting motor threshold and localization of the therapeutic target for stimulation [e.g., the dorsolateral prefrontal cortex] using scalp markers or MRI-based neuronavigation) (28). ECT procedural competencies include selecting the appropriate electrical stimulus parameters, titrating the dose according to types of ECT, interpreting the adequacy of a seizure with clinical and EEG measures, and managing missed seizures and other complications (32). In summary, interventional psychiatry, by virtue of its treatments being composed of multiple sequential physical actions and tasks which can be easily demonstrated, observed and measured, naturally lends itself to a CBME approach for determination of competency.

Despite the enthusiasm regarding CBME in general, several questions remain regarding the implementation of an interventional psychiatry curriculum in this framework. The optimal timing of interventional training is unclear, although there are significant advantages in including it in the core training required by all residents (22). However, due to a scarcity of human and technological resources, many trainees in psychiatry programs may not have local access to the clinical supervision and procedural exposure which are required to develop expertise in these treatments. The majority of ECT programs in Canada do not have a formal teaching program for trainees and there is a lack of uniformity in pedagogical approaches in those that exist (23). In the US, a limited subset of residency programs (i.e., 12 out of over 200 existing programs) train a disproportionate number of ECT practitioners across the country (19). As a result, the use of alternative forms of learning need to be explored to enhance competency in interventional psychiatry procedures.

2 Simulation in interventional psychiatry pedagogy: a scoping review of the literature

Simulation in medical education is a well-established pedagogical practice that offers an alternative to practicing directly with real patients. Through simulation, medical trainees have the opportunity to hone their skills until they reach a defined level of competency.

Given the current challenges of existing medical education models in providing sufficient training in interventional psychiatry and the promise of increased utilization of simulation-based learning to address this gap, it is imperative to review the existing evidence in this area. As a critical part of CBME and a gold standard for training procedural skills, simulation-based education in interventional psychiatry allows trainees to improve fluency in delivering treatments.

2.1 Methods

To gain a comprehensive understanding of the current state of simulation-based training in interventional psychiatry, this scoping review implemented a search strategy using the PubMed database, including human studies conducted from 1980 to 2023. To ensure inclusivity, a search string with wide criteria was employed: (simulat*[Title/Abstract] OR training[Title/Abstract] OR education*[Title/Abstract]) AND (electroconvulsive[Title/Abstract] OR ECT[Title/Abstract] OR “transcranial magnetic stimulation”[Title/Abstract] OR TMS[Title/Abstract] OR “interventional psychiatry” [Title/Abstract] or neuromodulation[Title/Abstract]).

Articles that met the following criteria were considered eligible sources: 1) evaluated any types of simulation-based learning in the field of interventional psychiatry, 2) targeted healthcare providers/trainees as learners, 3) only publications in English were considered, and 4) reviews were excluded. References were identified and screened by team member (AT) by title and abstract. Studies that met the inclusion criteria were further extracted and full-texts were reviewed.

2.2 Results

The initial search yielded 2094 articles. After the pre-screening process based on titles and abstracts, 11 were sought for retrieval for full-text review, and 4 out of 11, which evaluated the effectiveness of simulation approaches for ECT, were included in this review. No published study was identified regarding simulation training for other interventional psychiatric treatments (e.g., rTMS) or VR-based education (Figure 1).

Figure 1
www.frontiersin.org

Figure 1. Flow diagram for literature reviews and retrieval.

2.3 Review of the current use of simulation in ECT training

Based on our literature search, only 2 randomized controlled trials (RCT) (34, 35), 1 study reported in a letter to the editor (36), and 1 published protocol (37) were found regarding the evaluation of simulation training effectiveness for ECT.

The RCTs by Rabheru’s and Raysin’s groups utilized high-fidelity simulation (i.e., mannequin) as their simulation approach, which was compared to traditional didactics training. These two studies used ECT checklists to examine procedural skill competency, including electrode hook-up and placement, dosage strategies, procedure monitoring, etc.

In 2013, Rabheru et al. (34) were the first to investigate the effectiveness of simulation training on ECT clinical skills. Their study included 19 medical residents and assessed their performance on ECT skills and equipment use after receiving a high-fidelity patient simulator (n = 10) compared to didactic seminar training (traditional training; n = 9). In addition to procedural skills, they used standardized assessment scales such as Objective Structured Assessment of Technical Skills (ECT-OSATS) (38) to evaluate the residents’ knowledge and attitudes toward ECT. Their results revealed that although both groups showed similar increases in ECT knowledge and confidence, the residents who received simulator training demonstrated significantly better technical and overall skills compared to those who received didactics only (pass/fail ratio: 10/0 vs 1/8, χ21 = 15.53, p < 0.001).

In 2018, Raysin et al. (35) conducted an RCT comparing a control group of 17 residents who received standard didactic ECT curriculum with a simulation group of 18 residents who received augmented training with high-fidelity simulation in addition to didactics. In their study, a self-created experiential survey-cum-written test was administered to assess the residents’ overall experience with ECT (e.g., procedural knowledge, comfort level with the procedure) in addition to the skills evaluation. The study showed that the simulation group exhibited greater improvements in hands-on proficiency (pre-test to post-test mean scores: 22% to 51%, p < 0.001) compared to the control group (pre-test to post-test mean scores: 22% to 37%, p = 0.001). Differing from Rabheru’s study, their simulation group also demonstrated a significant increase in knowledge gain (pre- to post-test mean scores: 51% to 69%, p = 0.02), while the control group did not (p = 0.2). Higher comfort levels and confidence improvements in delivering ECT were also observed in Raysin’s study in only residents who had received simulation training.

In addition, Daaboul et al. (36) reported their findings from a high-fidelity simulation program in a letter to the editor. The study team used the Attitudes component of the Questionnaire on Attitudes and Knowledge of ECT (39) to assess a total of 31 psychiatry residents’ attitudes toward ECT before and after the program, with higher score indicating more negative attitudes. They found that after the high-fidelity simulation program, there was a significant decrease of 4.1 points (p < 0.001, t = 5.84) in the Attitudes score, suggesting the residents’ improvement in positive representations and attitude toward ECT.

In 2020, Aakhus et al. presented a research protocol (SAFE-ECT) for an ECT simulation program using a VR platform (37). VR-based simulation for surgical skills has become popular in recent years, and Aakhus’s team developed a checklist-based simulation training program for ECT using VR. They designed a prototype using the Unity3D graphic motor with the VR Goggles Oculus Rift, enabling users to repeatedly practice ECT skills step-by-step based on a standardized checklist. Their future plans included adding in different clinical scenarios and assessing the feasibility and applicability of the program in practice settings. While the efficacy of this particular VR-based simulation program for procedural skill acquisition have not yet been reported, if validated, it represents a promising direction for ECT training.

Reflecting on the studies reviewed above, there is limited but encouraging evidence supporting the effectiveness of simulation-based training in enhancing competency in interventional psychiatry, especially for ECT procedural skills. Adopting a simulation-based approach could be a viable solution to address the issue of stagnancy in the development of competency among psychiatry residents to perform these procedures. Furthermore, exploring novel simulation modalities, such as VR-based technology, could provide trainees with unlimited access to practice the logical steps of the treatments before real-life applications, thereby improving trainees’ confidence and enhancing patient safety. Offering this cost-effective and accessible alternative has the potential to expand the capabilities of simulation training in interventional psychiatry irrespective of geographical distance and other restrictions (e.g., remote areas, pandemic) where traditional simulation methods may pose challenges.

3 Immersive reality in medical education: lessons learned from other medical specialties

The current common approach in medical education is mannequin-based simulation requiring highly qualified personnel, equipment, and dedicated space. These factors contribute to high costs associated with high-fidelity medical simulation and also limit access to learners who are able to access these facilities in person for training and continuing education (40).

Accessibility to training is an even greater challenge for trainees in low and middle income countries (LMIC). In a survey conducted on psychiatric training among members of associations of the World Psychiatric Association, wide disparities were seen in the quantity and quality of psychiatric training worldwide, with educational limitations faced by LMIC countries identified as a primary factor (41). Trainees in LMIC countries had significantly less time dedicated to their psychiatric training (a mean of 39.3 months vs. a mean of 56.3 months in high-income countries) (41) and faced greater resource constraints in terms of accessibility to clinical supervisors, as well as ongoing teaching and training (42). With increased internet access in LMICs, many respondents expressed a preference for online training and remote learning to augment the development of skills and knowledge, including simulation learning through technologies such as IR (41, 42). As the cost of technology continues to drop and accessibility grows with each generation of commercially available devices, the role of IR simulation as a pedagogical technique to address gaps in training in psychiatry becomes more prominent. IR simulation allows trainees to have unlimited and on-demand training experiences. Thus, it may especially have a critical role in skill acquisition for interventional psychiatry procedures in locations where access to skilled supervisors is limited. Additionally, pursuing training in a home environment may be especially important in situations, where the ability to learn procedures through iterative face-to-face contact with patients is restricted.

The concept of IR in the healthcare realm is not new. IR is an umbrella term that includes VR, any type of interactive, computer-generated immersive learning element, and augmented reality, which involves overlaying computer-generated elements over a real-life environment (43). IR simulation can be an effective and efficient teaching method and in some cases, more effective than, traditional methods in medical education (44). IR offers several potential advantages, including increased accessibility (learners can use the equipment on-site or access IR educational applications remotely) and easy-to-set-up equipment in small spaces, requiring minimal time. This potentially allows for just in time curriculum delivery in multiple clinical settings. While there are initial costs associated with developing IR applications, the end products can be designed as modular systems with long term cost savings (45). Unlike mannequin-based curriculums, costs can be minimized for IR curriculum delivery and can be scaled for additional learners. These advantages offer the feasibility of a wide range of procedures, virtual patient interaction, and the potential for reduced costs in IR-based simulation training, although the validation of these benefits would still require further well-designed cost-benefit studies.

Educators in acute procedural medical specialties are faced with the challenge of balancing trainee autonomy with reduced clinical exposure. Learning technical skills through passive means, such as reading written text or watching it be done, has not been viewed as the ideal learning pedagogical approach in this area (46, 47). IR technology moves pedagogy by away from traditional methods of learning through two-dimensional videos and images to interactive simulated environments (48). IR allows learners to not only visualize procedural skills from a first-person perspective but also actively practice the required steps needed for completion in an active manner, with or without haptic capabilities. Even though haptic glove technology is still evolving and has not made it to the mainstream commercial market, the sense of touch can be simulated using commercial videogame joysticks, and common sensors attached to common medical tools (e.g., syringes, laryngoscope) (49, 50). These features may be especially important in medical education in interventional psychiatry, where the required knowledge acquisition and consolidation of procedural skills demand experiential, self-directed, and hands-on learning.

IR in procedural skill acquisition offers trainees a safe, realistic, and engaging learning environment. It can provide customized interactive case scenarios, repetitive training, emotional engagement, critical thinking, and immediate feedback to enhance learning outcomes. These interactive virtual curricula can be integrated into educational learning programs to train students, nurses, and doctors. Indeed, a recent scoping review by Mergen et al. reported the various uses and supported future efforts in VR development and its integration into medical education (51).

In the context of surgical subspecialties, IR simulators have been particularly beneficial for training surgical skills in laparoscopic, neuro, endovascular, and orthopedic surgeries (5255). Anesthesia-related applications also benefit significantly from VR simulations. VR simulators can replicate a wide range of scenarios, teaching regional anesthesia techniques (5659), bronchoscopy/intubation (6064) and central vein insertion (65, 66). The development of IR training programs in these specialties have demonstrated positive learning outcomes, allowing trainees to navigate 3D anatomical structures and develop and refine procedural skills in a risk-free environment.

4 Discussion

In summary, there has been tremendous growth in the data supporting the use of interventional psychiatry techniques, especially in the last two decades. Both current trainees in psychiatry and established practitioners recognize the need for training in this area. However, the established models of medical pedagogy in psychiatry emphasizing knowledge acquisition through lecture and observation have been proven to be insufficient, and the majority of psychiatrists graduate without achieving competency in delivering ECT, the most established of the interventional psychiatry procedures. Education in interventional psychiatry during the critical developmental window of residency training may serve the dual purposes of enhancing clinicians’ awareness of these treatment options for refractory patients and increasing the number of psychiatrists able to deliver it. There is a need to shift the model of education in this area to reflect tenets of CBME, to focus on the outputs of training that are demonstrable abilities and performance of skills, rather than inputs or the traditional time-based focus in residency.

Considering the limited clinical exposure opportunities that currently exist for novice professionals and students, there is a need to explore alternative educational approaches that can provide enhanced training experiences in interventional psychiatry. There is currently a limited data on the use of simulation-based learning for interventional psychiatry treatments, but as demonstrated in other medical specialties, VR simulation training offers a safe, realistic, and interactive environment for students to practice complex procedures, develop critical decision-making skills, and improve patient safety. The use of VR-based simulator training by offering immersive, realistic, and on-demand training environments that can be accessed remotely, may play a role in addressing the low rates of competency to deliver interventional treatments by augmenting traditional models of pedagogy.

VR-based simulation training may also contribute to skill acquisition for students in locations where access to skilled supervisors is limited. As technology continues to advance and become more available worldwide, barriers to the deployment of VR simulation in interventional psychiatry will continue to decrease. Future research efforts will need to focus on investigating the effectiveness and practicality of these novel educational strategies.

Author contributions

PG: Writing – original draft, Writing – review & editing, Conceptualization. AT: Writing – original draft, Writing – review & editing, Data curation, Methodology. LK: Conceptualization, Writing – original draft, Writing – review & editing. SC: Writing – review & editing. JW: Writing – review & editing. FA: Conceptualization, Writing – original draft, Writing – review & editing.

Funding

The author(s) declare that no financial support was received for the research and/or publication of this article.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Insel T, Cuthbert B, Garvey M, Heinssen R, Pine DS, Quinn K, et al. Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. Am J Psychiatry. (2010) 167:748–51. doi: 10.1176/appi.ajp.2010.09091379

PubMed Abstract | Crossref Full Text | Google Scholar

2. Hassan T, Prasad B, Meek BP, Modirrousta M. Attitudes of psychiatry residents in Canadian universities toward neuroscience and its implication in psychiatric practice. Can J Psychiatry. (2020) 65:174–83. doi: 10.1177/0706743719881539

PubMed Abstract | Crossref Full Text | Google Scholar

3. Williams NR, Taylor JJ, Snipes JM, Short EB, Kantor EM, George MS. Interventional psychiatry: how should psychiatric educators incorporate neuromodulation into training? Acad Psychiatry. (2014) 38:168–76. doi: 10.4088/JCP.13108745

Crossref Full Text | Google Scholar

4. Lipsman N, Sankar T, Downar J, Kennedy SH, Lozano AM, Giacobbe P. Neuromodulation for treatment-refractory major depressive disorder. CMAJ. (2014) 186:33–9. doi: 10.1503/cmaj.121317

PubMed Abstract | Crossref Full Text | Google Scholar

5. Mutz J, Vipulananthan V, Carter B, Hurlemann R, Fu CHY, Young AH. Comparative efficacy and acceptability of non-surgical brain stimulation for the acute treatment of major depressive episodes in adults: systematic review and network meta-analysis. BMJ. (2019) 364:l1079. doi: 10.1136/bmj.l1079

PubMed Abstract | Crossref Full Text | Google Scholar

6. Milev RV, Giacobbe P, Kennedy SH, Blumberger DM, Daskalakis ZJ, Downar J, et al. Canadian network for mood and anxiety treatments (CANMAT) 2016 clinical guidelines for the management of adults with major depressive disorder: section 4. Neurostimulation treatments. Can J Psychiatry. (2016) 61:561–75. doi: 10.1177/0706743716660033

PubMed Abstract | Crossref Full Text | Google Scholar

7. Giacobbe P, Mithani K, Meng Y, Vila-Rodriguez F, Daskalakis ZJ, Downar J, et al. Evaluation of the effects of rTMS on self-reported quality of life and disability in treatment-resistant depression: A THREE-D study. J Affect Disord. (2020) 268:127–33. doi: 10.1016/j.jad.2020.03.002

PubMed Abstract | Crossref Full Text | Google Scholar

8. Holmes DR, Alkhouli M. Past, present, and future of interventional cardiology. J Am Coll Cardiol. (2020) 75:2738–43. doi: 10.1016/j.jacc.2020.03.066

PubMed Abstract | Crossref Full Text | Google Scholar

9. Murphy TP, Soares GM. The evolution of interventional radiology. Semin Intervent Radiol. (2005) 22:6–9. doi: 10.1055/s-2005-869570

PubMed Abstract | Crossref Full Text | Google Scholar

10. Giacobbe P, Burhan AM, Waxman R, Ng E. Interventional psychiatry and neurotechnologies: education and ethics training. Can J Neurol Sci. (2023) 50:s10–6. doi: 10.1017/cjn.2023.27

PubMed Abstract | Crossref Full Text | Google Scholar

11. McIntyre RS, Rosenblat JD, Nemeroff CB, Sanacora G, Murrough JW, Berk M, et al. Synthesizing the evidence for ketamine and esketamine in treatment-resistant depression: an international expert opinion on the available evidence and implementation. AJP. (2021) 178:383–99. doi: 10.1176/appi.ajp.2020.20081251

PubMed Abstract | Crossref Full Text | Google Scholar

12. Anand A, Mathew SJ, Sanacora G, Murrough JW, Goes FS, Altinay M, et al. Ketamine versus ECT for nonpsychotic treatment-resistant major depression. N Engl J Med. (2023) 388:2315–25. doi: 10.1056/NEJMoa2302399

PubMed Abstract | Crossref Full Text | Google Scholar

13. Rakita U, Bingham K, Fung K, Giacobbe P. Factors associated with global variability in electroconvulsive therapy utilization. J ECT. (2017) 33:253–9. doi: 10.1097/YCT.0000000000000405

PubMed Abstract | Crossref Full Text | Google Scholar

14. Delva NJ, Graf P, Patry S, Gosselin C, Milev R, Gilron I, et al. Access to electroconvulsive therapy services in Canada. J ECT. (2011) 27:300–9. doi: 10.1097/YCT.0b013e318222b1b8

PubMed Abstract | Crossref Full Text | Google Scholar

15. Kaster TS, Blumberger DM, Gomes T, Sutradhar R, Dasklakis ZJ, Wijeysundera DN, et al. Patient-level Characteristics and Inequitable Access to Inpatient Electroconvulsive Therapy for Depression: A Population-based Cross-sectional Study: Caractéristiques au niveau du patient et accès inéquitable à la thérapie électroconvulsive pour patients hospitalisés. Can J Psychiatry. (2021) 66:147–58. doi: 10.1177/0706743720935647

PubMed Abstract | Crossref Full Text | Google Scholar

16. Rapoport MJ, Mamdani M, Herrmann N. Electroconvulsive therapy in older adults: 13-year trends. Can J Psychiatry. (2006) 51:616–9. doi: 10.1177/070674370605100910

PubMed Abstract | Crossref Full Text | Google Scholar

17. Lemasson M, Haesebaert J, Rochette L, Pelletier E, Lesage A, Patry S. Electroconvulsive therapy practice in the province of Quebec: linked health administrative data study from 1996 to 2013. Can J Psychiatry. (2018) 63:465–73. doi: 10.1177/0706743717738492

PubMed Abstract | Crossref Full Text | Google Scholar

18. Case BG, Bertollo DN, Laska EM, Price LH, Siegel CE, Olfson M, et al. Declining use of electroconvulsive therapy in United States general hospitals. Biol Psychiatry. (2013) 73:119–26. doi: 10.1016/j.biopsych.2012.09.005

PubMed Abstract | Crossref Full Text | Google Scholar

19. Wilkinson ST, Kitay BM, Harper A, Rhee TG, Sint K, Ghosh A, et al. Barriers to the implementation of electroconvulsive therapy (ECT): results from a nationwide survey of ECT practitioners. Psychiatr Serv. (2021) 72:752–7. doi: 10.1176/appi.ps.202000387

PubMed Abstract | Crossref Full Text | Google Scholar

20. Dauenhauer LE, Chauhan P, Cohen BJ. Factors that influence electroconvulsive therapy referrals: a statewide survey of psychiatrists. J ECT. (2011) 27:232–5. doi: 10.1097/YCT.0b013e3181f9789c

PubMed Abstract | Crossref Full Text | Google Scholar

21. Dolenc TJ, Philbrick KL. Achieving competency in electroconvulsive therapy: a model curriculum. Acad Psychiatry. (2007) 31:65–7. doi: 10.1176/appi.ap.31.1.65

PubMed Abstract | Crossref Full Text | Google Scholar

22. Trapp NT, Williams NR. The future of training and practice in neuromodulation: an interventional psychiatry perspective. Front Psychiatry. (2021) 12:734487. doi: 10.3389/fpsyt.2021.734487

PubMed Abstract | Crossref Full Text | Google Scholar

23. Patry S, Graf P, Delva NJ, Chan P, Enns M, Gilron I, et al. Electroconvulsive therapy teaching in Canada: cause for concern. J ECT. (2013) 29:109–12. doi: 10.1097/YCT.0b013e31827989b9

PubMed Abstract | Crossref Full Text | Google Scholar

24. Giacobbe P, Ng E, Blumberger DM, Daskalakis ZJ, Downar J, Garcia C, et al. Interventional psychiatry: an idea whose time has come? Can J Psychiatry. (2021) 66:316–8. doi: 10.1177/0706743720963887

PubMed Abstract | Crossref Full Text | Google Scholar

25. Goldbloom DS, Kussin DJ. Electroconvulsive therapy training in Canada: A survey of senior residents in psychiatry. Can J Psychiatry. (1991) 36:126–8. doi: 10.1177/070674379103600211

PubMed Abstract | Crossref Full Text | Google Scholar

26. Yuzda E, Parker K, Parker V, Geagea J, Goldbloom D. Electroconvulsive therapy training in Canada: a call for greater regulation. Can J Psychiatry. (2002) 47:938–44. doi: 10.1177/070674370204701005

PubMed Abstract | Crossref Full Text | Google Scholar

27. Trapp NT, Barbour T, Kritzer MD, Pottanat R, Carreon D, Chen L, et al. Defining a neurostimulation-focused subspecialty: perspectives inspired by a debate at the 2023 clinical TMS society annual meeting. Acad Psychiatry. (2024) 48:463–70. doi: 10.1007/s40596-024-02025-2

PubMed Abstract | Crossref Full Text | Google Scholar

28. Fried PJ, Santarnecchi E, Antal A, Bartres-Faz D, Bestmann S, Carpenter LL, et al. Training in the practice of noninvasive brain stimulation: Recommendations from an IFCN committee. Clin Neurophysiol. (2021) 132:819–37. doi: 10.1016/j.clinph.2020.11.018

PubMed Abstract | Crossref Full Text | Google Scholar

29. Pallanti S, Marras A, Dickson SL, Adan RA, Vieta E, Dell Osso B, et al. Manifesto for an ECNP Neuromodulation Thematic Working Group (TWG): Non-invasive brain stimulation as a new Super-subspecialty. Eur Neuropsychopharmacol. (2021) 52:72–83. doi: 10.1016/j.euroneuro.2021.07.002

PubMed Abstract | Crossref Full Text | Google Scholar

30. Danilewitz M, Ainsworth NJ, Liu C, Vila-Rodriguez F. Towards competency-based medical education in neurostimulation. Acad Psychiatry. (2020) 44:775–8. doi: 10.1007/s40596-020-01195-z

PubMed Abstract | Crossref Full Text | Google Scholar

31. Macaluso M, Kinzie M, Cowley D, Houston LJ, DeJong S, Janssen F, et al. The psychiatry milestones 2.0: how did we get from 1.0 to 2.0 and what can users expect? Acad Psychiatry. (2020) 44:779–84. doi: 10.1007/s40596-020-01275-0

PubMed Abstract | Crossref Full Text | Google Scholar

32. Al-Qadhi SA, Chawla T, Seabrook JA, Campbell C, Burhan AM. Competency by design for electroconvulsive therapy in psychiatry postgraduate training: face and content validation study. J ECT. (2020) 36:18–24. doi: 10.1097/YCT.0000000000000646

PubMed Abstract | Crossref Full Text | Google Scholar

33. Lai KSP, Waxman R, Blumberger DM, Giacobbe P, Hasey G, McMurray L, et al. Competencies for repetitive transcranial magnetic stimulation in postgraduate medical education: expert consensus using a modified Delphi process. Can J Psychiatry. (2023) 68:916–24. doi: 10.1177/07067437231164571

PubMed Abstract | Crossref Full Text | Google Scholar

34. Rabheru K, Wiens A, Ramprasad B, Bourgon L, Antochi R, Hamstra SJ. Comparison of traditional didactic seminar to high-fidelity simulation for teaching electroconvulsive therapy technique to psychiatry trainees. J ECT. (2013) 29:291–6. doi: 10.1097/YCT.0b013e318290f9fb

PubMed Abstract | Crossref Full Text | Google Scholar

35. Raysin A, Gillett B, Carmody J, Goel N, McAfee S, Jacob T. From information to simulation: improving competency in ECT training using high-fidelity simulation. Acad Psychiatry. (2018) 42:653–8. doi: 10.1007/s40596-017-0859-1

PubMed Abstract | Crossref Full Text | Google Scholar

36. Daaboul J, Fovet T, Nunes F, Lebuffe G, Vaiva G, Amad A. High-fidelity simulation: A promising tool to tackle negative representations of electroconvulsive therapy in psychiatry residents. J ECT. (2021) 37:e32–3. doi: 10.1097/YCT.0000000000000774

PubMed Abstract | Crossref Full Text | Google Scholar

37. Aakhus E, Utheim E, Vandli R, Sandaker J, Juell S, Opsahl E. Safety and fidelity in electroconvulsive therapy (SAFE ECT): A novel virtual reality–based training program in electroconvulsive therapy (Phase 1). J ECT. (2020) 36:158–60. doi: 10.1097/YCT.0000000000000653

PubMed Abstract | Crossref Full Text | Google Scholar

38. Faulkner H, Regehr G, Martin J, Reznick R. Validation of an objective structured assessment of technical skill for surgical residents. Acad Med. (1996) 71:1363–5. doi: 10.1097/00001888-199612000-00023

PubMed Abstract | Crossref Full Text | Google Scholar

39. Lutchman RD, Stevens T, Bashir A, Orrell M. Mental health professionals’ attitudes towards and knowledge of electroconvulsive therapy. J Ment Health. (2001) 10:141–50. doi: 10.1080/09638230124779

Crossref Full Text | Google Scholar

40. Isaranuwatchai W, Alam F, Hoch J, Boet S. A cost-effectiveness analysis of self-debriefing versus instructor debriefing for simulated crises in perioperative medicine in Canada. J Educ Eval Health Prof. (2017) 13:44. doi: 10.3352/jeehp.2016.13.44

PubMed Abstract | Crossref Full Text | Google Scholar

41. Ng RMK, Hermans MHM, Belfort E, Bhugra D. A worldwide survey on training provisions for psychiatric trainees in WPA member associations. Int Rev Psychiatry. (2020) 32:98–113. doi: 10.1080/09540261.2019.1648241

PubMed Abstract | Crossref Full Text | Google Scholar

42. Jayaram M, Shields G, Buisman-Pijlman F. Novel methods of teaching psychiatry to medical and postgraduate students. Curr Opin Psychiatry. (2021) 34:491–6. doi: 10.1097/YCO.0000000000000725

PubMed Abstract | Crossref Full Text | Google Scholar

43. Alam F, Matava C. A new virtual world? The future of immersive environments in anesthesiology. Anesth Analgesia. (2022) 135:230–8. doi: 10.1213/ANE.0000000000006118

PubMed Abstract | Crossref Full Text | Google Scholar

44. Curran VR, Xu X, Aydin MY, Meruvia-Pastor O. Use of extended reality in medical education: an integrative review. Med Sci Educ. (2024) 33:275–86. doi: 10.1007/s40670-022-01698-4

PubMed Abstract | Crossref Full Text | Google Scholar

45. Bumbach MD, Culross BA, Datta SK. Assessing the financial sustainability of high-fidelity and virtual reality simulation for nursing education: a retrospective case analysis. Comput Inform Nurs. (2022) 40:615–23. doi: 10.1097/CIN.0000000000000916

PubMed Abstract | Crossref Full Text | Google Scholar

46. Conrad M, Kablitz D, Schumann S. Learning effectiveness of immersive virtual reality in education and training: A systematic review of findings. In: Computers & Education: X Reality. (2024) ScienceDirect.com by Elsevier, 4:100053.

Google Scholar

47. Lewis KO, Popov V, Fatima SS. From static web to metaverse: reinventing medical education in the post-pandemic era. Ann Med (Helsinki). (2024) 56:2305694. doi: 10.1080/07853890.2024.2305694

PubMed Abstract | Crossref Full Text | Google Scholar

48. Li L, Yu F, Shi D, Shi J, Tian Z, Yang J, et al. Application of virtual reality technology in clinical medicine. Am J Transl Res. (2017) 9:3867–80.

Google Scholar

49. Moo-Young J, Weber TM, Kapralos B, Quevedo A, Alam F. Development of unity simulator for epidural insertion training for replacing current lumbar puncture simulators. Cureus. (2021) 13:e13409. doi: 10.7759/cureus.13409

PubMed Abstract | Crossref Full Text | Google Scholar

50. Bejani M, Taghizadieh A, Samad-Soltani T, Asadzadeh A, Rezaei-Hachesu P. The effects of virtual reality-based bronchoscopy simulator on learning outcomes of medical trainees: A systematic review. Health Sci Rep. (2023) 6:e1398. doi: 10.1002/hsr2.1398

PubMed Abstract | Crossref Full Text | Google Scholar

51. Mergen M, Meyerheim M, Graf N. Reviewing the current state of virtual reality integration in medical education – a scoping review protocol. Syst Rev. (2023) 12:97. doi: 10.1186/s13643-023-02266-6

PubMed Abstract | Crossref Full Text | Google Scholar

52. Brunozzi D, Shakur SF, Kwasnicki A, Ismail R, Charbel FT, Alaraj A. Role of immersive touch simulation in neurosurgical training. In: Alaraj A, editor. Comprehensive Healthcare Simulation: Neurosurgery. Springer International Publishing, Cham (2018). p. 185–98. doi: 10.1007/978-3-319-75583-0_14

Crossref Full Text | Google Scholar

53. Rudarakanchana N, Van Herzeele I, Desender L, Cheshire NJW. Virtual reality simulation for the optimization of endovascular procedures: current perspectives. Vasc Health Risk Manage. (2015) 11:195–202. doi: 10.2147/VHRM.S46194

PubMed Abstract | Crossref Full Text | Google Scholar

54. Sugand K, Akhtar K, Khatri C, Cobb J, Gupte C. Training effect of a virtual reality haptics-enabled dynamic hip screw simulator. Acta Orthop. (2015) 86:695–701. doi: 10.3109/17453674.2015.1071111

PubMed Abstract | Crossref Full Text | Google Scholar

55. Aïm F, Lonjon G, Hannouche D, Nizard R. Effectiveness of virtual reality training in orthopaedic surgery. Arthroscopy. (2016) 32:224–32. doi: 10.1016/j.arthro.2015.07.023

PubMed Abstract | Crossref Full Text | Google Scholar

56. O’Sullivan O, Iohom G, O’Donnell BD, Shorten GD. The effect of simulation-based training on initial performance of ultrasound-guided axillary brachial plexus blockade in a clinical setting - a pilot study. BMC Anesthesiol. (2014) 14:110. doi: 10.1186/1471-2253-14-110

PubMed Abstract | Crossref Full Text | Google Scholar

57. Ameri G, Rankin A, Baxter JSH, Moore J, Ganapathy S, Peters TM, et al. Development and evaluation of an augmented reality ultrasound guidance system for spinal anesthesia: preliminary results. Ultrasound Med Biol. (2019) 45:2736–46. doi: 10.1016/j.ultrasmedbio.2019.04.026

PubMed Abstract | Crossref Full Text | Google Scholar

58. Kim JY, Lee JS, Lee JH, Park YS, Cho J, Koh JC. Virtual reality simulator’s effectiveness on the spine procedure education for trainee: a randomized controlled trial. Korean J Anesthesiol. (2023) 76:213–26. doi: 10.4097/kja.22491

PubMed Abstract | Crossref Full Text | Google Scholar

59. Chuan A, Qian J, Bogdanovych A, Kumar A, McKendrick M, McLeod G. Design and validation of a virtual reality trainer for ultrasound-guided regional anaesthesia. Anaesthesia. (2023) 78:739–46. doi: 10.1111/anae.16015

PubMed Abstract | Crossref Full Text | Google Scholar

60. Jiang B, Ju H, Zhao Y, Yao L, Feng Y. Comparison of the efficacy and efficiency of the use of virtual reality simulation with high-fidelity mannequins for simulation-based training of fiberoptic bronchoscope manipulation. Simul Healthc. (2018) 13:83–7. doi: 10.1097/SIH.0000000000000299

PubMed Abstract | Crossref Full Text | Google Scholar

61. Latif R, Bautista A, Duan X, Neamtu A, Wu D, Wadhwa A, et al. Teaching basic fiberoptic intubation skills in a simulator: initial learning and skills decay. J Anesth. (2016) 30:12–9. doi: 10.1007/s00540-015-2091-z

PubMed Abstract | Crossref Full Text | Google Scholar

62. Wong DT, Mehta A, Singh KP, Leong SM, Ooi A, Niazi A, et al. The effect of virtual reality bronchoscopy simulator training on performance of bronchoscopic-guided intubation in patients: A randomised controlled trial. Eur J Anaesthesiol. (2019) 36:227–33. doi: 10.1097/EJA.0000000000000890

PubMed Abstract | Crossref Full Text | Google Scholar

63. De Oliveira GS, Glassenberg R, Chang R, Fitzgerald P, McCarthy RJ. Virtual airway simulation to improve dexterity among novices performing fibreoptic intubation. Anaesthesia. (2013) 68:1053–8. doi: 10.1111/anae.2013.68.issue-10

PubMed Abstract | Crossref Full Text | Google Scholar

64. Cailleau L, Geeraerts T, Minville V, Fourcade O, Fernandez T, Bazin JE, et al. Is there a benefit for anesthesiologists of adding difficult airway scenarios for learning fiberoptic intubation skills using virtual reality training? A randomized controlled study. PloS One. (2023) 18:e0281016. doi: 10.1371/journal.pone.0281016

PubMed Abstract | Crossref Full Text | Google Scholar

65. Sappenfield J, Grek S, Cooper LA, Lizdas DE, Lampotang S. Reduced complications of supraclavicular approach in simulated central venous access: applicability to military medicine. Mil Med. (2019) 184:329–34. doi: 10.1093/milmed/usy381

PubMed Abstract | Crossref Full Text | Google Scholar

66. Rochlen LR, Levine R, Tait AR. First-person point-of-view-augmented reality for central line insertion training: A usability and feasibility study. Simul Healthc. (2017) 12:57–62. doi: 10.1097/SIH.0000000000000185

PubMed Abstract | Crossref Full Text | Google Scholar

Keywords: simulation-based learning, virtual reality, interventional psychiatry, neuromodulation, ECT, rTMS

Citation: Giacobbe P, Tu A, Kaustov L, Choi S, Wiegelmann J and Alam F (2025) Advancing education in interventional psychiatry: scoping review of simulation training and the future of virtual reality-based learning. Front. Psychiatry 16:1432314. doi: 10.3389/fpsyt.2025.1432314

Received: 13 May 2024; Accepted: 18 March 2025;
Published: 12 May 2025.

Edited by:

Albino J. Oliveira-Maia, Champalimaud Foundation, Portugal

Reviewed by:

J. Bernardo Barahona-Correa, Champalimaud Foundation, Portugal
Gábor Gazdag, Jahn Ferenc Dél-Pesti Kórház és Rendelőintézet, Hungary

Copyright © 2025 Giacobbe, Tu, Kaustov, Choi, Wiegelmann and Alam. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Peter Giacobbe, cGV0ZXIuZ2lhY29iYmVAc3Vubnlicm9vay5jYQ==

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.