Patrik Goncalves Rodrigues1
Danieli Mayumi Kimura Leandro1,2Silvia Schoenau de Azevedo1
Marcelo Jenné Mimica1,3
Rafaela Fabri Rodrigues1,2Mauricio Magalhães1,2,3Bruno Fernandes dos Anjos1
Gabriel Fernando Todeschi Variane1,2*
- 1Protecting Brains & Saving Futures, Clinical Research Department, São Paulo, Brazil
- 2Division of Neonatology, Department of Pediatrics, Irmandade da Santa Casa de Misericórdia de São Paulo, São Paulo, Brazil
- 3Santa Casa de São Paulo School of Medicine, São Paulo, Brazil
Mixed reality (MR) and augmented reality (AR) technologies bridge elements of the real and virtual worlds, emerging as tools that allow users to engage with digital cues to aid with tasks encountered in the physical environment. Thus, these holographic-based innovations are potential tools to support real-time patient care. The applications of MR and AR in neonatal care remain significantly underexplored. In the present article, we highlight the applications of MR and AR across medical procedures, physical examinations, medical diagnoses, and telemedicine, further underscoring their transformative potential within neonatal care. The use of MR and AR can be relevant across diverse economic and clinical landscapes, and in-depth research is required to evaluate the advantages of these tools in caring for neonates.
Introduction
Immersive technology approaches are becoming more prominent in the healthcare field. Virtual Reality (VR), a widely known innovation, allows individuals to immerse themselves in a completely simulated environment (1). These computer-generated settings enable unique educational opportunities that simulate scenarios in surgical and clinical care (2, 3). However, VR completely excludes the real-world environment, making its application challenging in tangible, live medical care. For that reason, in this article we focus on the emerging approaches of Augmented Reality (AR) and Mixed Reality (MR).
In contrast to VR, AR allows the projection of digital content in real environments, expanding the visual reality with virtual cues (4). AR has since been further developed into a mixed reality (MR) technology, enabling virtual content to interact with objects in the physical world (5, 6). The holographic-based technologies of AR and MR allow users to interact with the virtual elements to assist with tasks encountered in the physical environment. Thus, the overlap of AR and MR (AR/MR) enables their application to live, real-world patient care.
Despite being widely explored, especially among simulated surgical procedures, the implications of applying AR/MR approaches to the care of neonatal patients necessitate more research (7, 8). The field of neonatology faces many challenges, such as heterogeneity of care and a lack of skilled interventions (9–12). AR/MR have the potential to minimize these disparities by positively contributing to medical procedures, physical examinations, clinical diagnoses, and telemedicine. For instance, instant access to holographically presented protocol instructions, such as neurological exams, may further globalize medical expertise. Therefore, further attention is warranted to assess the advantages of AR/MR technology in neonatal care.
This position paper aims to explore the applications of AR/MR in the clinical care of patients, with a focus on highlighting the prospects of integrating such initiatives into neonatal care. Additionally, we share our experience applying a trial version of an MR-based solution, as a proof-of-concept experience, in the neonatal intensive care unit (NICU) of a tertiary hospital in Brazil.
AR/MR for medical procedures
AR/MR devices have been broadly explored as a tool in surgical specialties, with orthopedic surgery and neurosurgery techniques being the most commonly explored with immersive technology (7). The holographic-based imaging of AR/MR paves the way for various applications within surgery and surgical training. For instance, AR/MR devices may augment traditional image-guided interventions such as laparoscopy (13).
Unlike VR systems, AR/MR could offer the ability to provide see-through visualization. AR/MR may superimpose a clinical image, such as previously obtained CT or MR imaging data, directly on the patient (14–16). Gibby and colleagues explored the application of such a tool for interventional spine procedures across 10 patients, with promising results of AR navigation within such a context (16). The see-through visualization ability of AR/MR could provide access to underlying anatomy without physical incisions and may guide surgeons during pre-operative planning and intra-operative navigation for minimally invasive procedures (17–20). For instance, Wang and colleagues have depicted the clinical potential of AR surgical visualization within a knee arthroplasty context in an experimental environment (19). Under a similar laboratory setting with 80 lumbar spinal vertebra models, Dennler et al. found that utilizing AR may improve the precision of surgical pedicle screw insertion (20).
Moreover, the ability to render a 3D medical image into an AR/MR device could provide useful depth perception and spatial awareness of vital anatomical structures that may not be obtained through a conventional 2D image (21–24). Even during open surgical procedures, AR/MR could aid with actions such as blood vessel searches (25). While still a novel concept and mainly explored among phantom and simulated scenarios, surgical approaches guided by AR/MR carry the potential to reduce incision sizes, diminish potential errors, and reduce operative duration.
Beyond invasive interventions, AR/MR can also contribute to non-surgical clinical procedures. During ultrasound-guided examinations, for instance, an AR/MR device can contribute to the spatial orientation during the procedure as it may display the ultrasound image at the exact site or proximal to the anatomic location of interest (26). Additionally, AR/MR may enhance other invasive procedures, such as intubation or catheter placement (27, 28). Beyond its see-through visualization ability, AR/MR could also provide out-loud or holographic-based instructions for the accurate completion of procedures, a feature particularly relevant for healthcare trainees (29, 30).
AR/MR for physical examination and clinical diagnosis
AR/MR can also be utilized as a tool for clinical diagnosis and examinations at the patient's bedside. Utilizing an AR/MR device for a 3D visualization of a clinical image instead of a conventional 2D view is an emerging reality, as discussed in the previous section (21–23). Immersive technology-based 3D imaging may be beneficial for diagnoses at the patient's bedside. For instance, Venson and colleagues have previously highlighted the value of immersive technology in identifying fractures through a 3D volumetric image visualization (31). Additionally, AR/MR can offer the ability to interact with clinical images hands-free, allowing for a simultaneous examination of the patient and a more detailed image inspection (e.g., zoom-in) (22). As investigated by Butaslac et al. and Klinker et al., AR/MR tools may also provide insights into the physical exam that include the ability to trace angular trajectories of joints or measure the size of wounds (32, 33).
Further, AR/MR technology could contribute to how healthcare providers visualize, monitor, and interact with a patient's medical data. Tanbeer & Sykes have presented a holographic-based system that can allow the observation of trends in vital signs, access to multiple vital modalities simultaneously, digital access to predictive risk scores such as the Hamilton Early Warning Score (HEWS), and instant availability of blood work results (34). The ability to holographically visualize medical data upon visiting a patient paves the way for an uninterrupted overview of vital parameters and information while freeing the hands for needed interventions (35–37).
Holographic-based projections from an AR/MR device have also been applied for educational and guidance purposes during physical exams. Concurrent with a physical examination, the clinician may have access to holographically presented reference videos, written instructions, and pathological images as an aid during care (38, 39). This could prove extremely relevant as the literature emphasizes that multimodal learning, such as an AR/MR approach, is more effective than conventional didactic methods (40, 41).
AR/MR and telemedicine
Immersive technology has also been explored as a telemedicine tool (42, 43). Current AR/MR devices present video-conference capabilities, allowing a remote individual to communicate and access a real-time transmission of the AR/MR user's viewpoint (44). This communication medium enhances the remote individual's sense of presence, allowing the AR/MR device user to relay information without looking away from a context or maneuvering a device (45).
Teleconsulting via AR/MR has previously been applied during medical training and in the intensive care unit (ICU) environment, revolutionizing how medical experts provide consultations and assist trainees (44, 46, 47). This proves even more applicable to rural or disadvantaged settings with limited medical expertise. As a proof-of-concept study, Sirilak & Muneesawang developed a system based on holographic and AR systems to provide consults to a typical ICU. The findings from surveys across 17 medical consults demonstrated the reliability and efficiency of AR/MR systems for remote consultations in such a context (46). Meanwhile, Mistuno et al. assessed remote surgical mentoring utilizing MR through four simulated procedures performed on a craniofacial model, with qualitative feedback indicating various advantages and improvable aspects. Notably, the capability of teleguidance by projecting lines or arrows into the operator's field of view was a key factor for effective surgical telementoring (44). Aiding medical settings via AR/MR has further been proposed across other scenarios, such as telementoring for central venous catheter placement or remote emergency assistance (48).
By observing the first-person viewpoint of the AR/MR user, a remote medical expert may also have more practical and instant access to a specific patient's vitals and medical records (46). The application of AR/MR devices as a telemedicine tool could reduce variability in medical practice and improve patient outcomes, especially in healthcare settings with a shortage of specialized care (49). Furthermore, such an approach could also potentially contribute to cost-effectiveness (50), along with limiting the exposure of vulnerable patients to additional staff that can otherwise accompany cases remotely.
Prospects of applying AR/MR tools to neonatal care
AR/MR technologies present various clinical applications, as depicted in Table 1. Yet, such applications to the care of neonates remain largely unexplored (7). AR/MR presents the potential to tackle the multifactorial challenges that impact neonatal care. Research estimates that simple and effective interventions could prevent most of the approximately 2.5 million neonatal deaths yearly, and many neonates who survive are at risk of developing neurological impairments due to the lack of well-known interventions (51). While low and middle-income countries present the highest rates of neonatal mortality and abnormal neurological outcomes, evidence also sheds light on the high variability of neonatal care across hospitals in high-income countries (9–11, 52). Overall, there is an urgency to improve and standardize neonatal care globally.
Table 1. Augmented and mixed reality studies referenced.
AR/MR's ability to holographically provide instructions, reference media, and digital access to diagnostic assessments while the user remains hands-free for clinical examinations may further improve and homogenize neonatal care. The benefits could be even more significant for managing neonatal emergencies in settings that lack the required expertise for the appropriate response. The application of digital clinical support tools has already shown the potential to improve patient care in different medical scenarios (30, 53, 54). Particularly in neonatology, digital tools have been applied across simulated resuscitation and intubation training, with findings highlighting improvement in guideline adherence (36, 55, 56). Thus, the application of AR/MR should be further explored and invested across various protocols in tangible neonatal care with existing patients.
Beyond the holographic projection of protocol information, the possibility of an AR/MR-based video communication also proves relevant. Access to a real-time transmission of the AR/MR user's viewpoint could redesign how medical expertise reaches underprivileged medical settings globally. Even for on-site consults and case discussions, an AR/MR videoconference could limit the exposure of vulnerable infants to infections and provide a better viewpoint to trainees who would otherwise have limited visualization due to a crowded room. Conversely, trainees could perform examinations being remotely monitored by experts.
Furthermore, utilizing AR/MR-based guidance during medical procedures performed in neonatal patients is another area that yields further exploration. The smaller anatomy, fragile tissues, and low blood volume that neonates present require further specificity upon performing medical procedures. The possibility of AR/MR-based software to superimpose a clinical image onto the patient and allow a see-through visualization of underlying anatomy without incisions could prove particularly relevant when dealing with the immature anatomy and physiology of newborns. Additionally, rendering a 3D medical image into an AR/MR device could provide a level of comprehension and visualization previously unreached, contributing to preoperative planning (23). The added challenge of managing pediatric anatomy underscores the need for additional initiatives that explore AR/MR-guided medical procedures (57, 58). Overall, AR/MR technology may address the multifactorial challenges impacting neonatal care, as illustrated in Figure 1.
Figure 1. Potential applications of augmented and mixed reality (AR/MR) in neonatal care.
A practical example of implementing an MR-based solution in the NICU
To improve and further homogenize the care of high-risk neonates, our group developed an MR-based software entitled NeonLens, an immersive technology solution to assist healthcare professionals in the NICU. NeonLens (Protecting Brains & Saving Futures, São Paulo, Brazil) is compatible with MR devices and provides specialists with assisted guided support through clinical protocols, along with enabling remote hands-free video communication with specialists to improve neonatal care. As part of a preliminary proof-of-concept experience, we applied the NeonLens across the following scenarios in a Brazilian NICU:
○ Medical Sign-out Rounds and Clinical Case Discussions:Medical sign-out rounds at our institution can be highly crowded with physicians, neonatal fellows, pediatric residents, interns, nurses, and other healthcare professionals. Due to this reality, trainees and healthcare providers may encounter an affected view of the patient being examined in the NICU, directly impacting case discussions during rounds. Thus, NeonLens has been proposed as a solution within this context. A neonatal fellow at the bedside, wearing Microsoft HoloLens 2 with NeonLens, conducted sign-out rounds and clinical case discussions while being assisted in real-time by a senior physician located remotely. The NeonLens user had a holographic video and audio of the remote medical expert while examining the patient and the surrounding monitors. Meanwhile, the remote senior physician had access to the user's viewpoint and audio through a video call on a computer. Other trainees and physicians were also able to accompany the discussion both locally or remotely.
○ Clinical Decision Support Tool for the Modified Sarnat Exam:The Modified Sarnat exam is a common assessment tool used to evaluate the severity of hypoxic-ischemic encephalopathy in neonates, and previous literature has reported high heterogeneity among professionals when performing the exam (59). Our group envisioned using NeonLens as an opportunity to reduce this variability and enhance the accuracy of the neurological exam. The physicians experienced using Microsoft HoloLens 2 with NeonLens to access detailed holographic, multimedia-guided instructions for performing the Modified Sarnat exam. The device provided the diagnosis of the Modified Sarnat exam based on the user's response to each exam item. Simultaneously, the NeonLens user at the infant's bedside had remote expert guidance via holographic video and audio to guide the user through each step of the clinical exam.
During this proof-of-concept experience, nine neonatology fellows received training and utilized the NeonLens solution across 45 medical sign-out rounds and 12 Modified Sarnat exams, totaling 34 h of use. As this was a preliminary proof-of-concept implementation, user experience was recorded qualitatively. Overall, the positive aspects were the reassurance of having a remote specialist and the intuitive, user-friendly interface that provided instant access to information that guided the physical exam. The challenges included difficulty hearing other healthcare professionals while using the tool and uncertainty about the exact remote expert view of the newborn. Repeated use led to greater confidence and ease.
Discussion
AR/MR technology is a tool with the potential to revolutionize medicine. As explored in this manuscript, such immersive technology approaches present several applications within a medical context, from the 3D rendering of clinical images to holographically provided protocol information (21, 35). The widespread applications of AR/MR can significantly contribute to clinical training and current advancement warrant its application within tangible, real-world patient care scenarios (56). Holographic access to protocol information, reference media, and video communication with a remote expert—all while remaining hands-free to perform a clinical exam or procedure—could transform and homogenize medical care.
AR/MR should be explored as a strategy to tackle global challenges in neonatal care. While underdeveloped countries present higher rates of newborn mortality that may be due to the lack of basic resources, high-income nations also present challenges, such as the high variability of care among settings (9–12, 52). Therefore, as previously explored, the applicability of AR/MR as a tool to improve care can be envisioned across diverse contexts. For instance, AR/MR technology may expand the mentorship of more experienced physicians to less specialized care settings. Further, presenting protocol instructions through an AR/MR device could ensure that expert physicians across hospitals adhere rigorously to guidelines. Ultimately, the transformative potential of AR/MR may contribute to the reduction of neonatal mortality and morbidity. Future studies should investigate the impact of implementing AR/MR strategies to improve neonatal outcomes.
Understanding the current challenges of AR/MR initiatives in medicine is essential for developing a plan and implementing its use. Current AR/MR software devices present a battery life and field of view that may not be optimal for specific clinical scenarios, especially for more extended examinations and procedures (7). While such limitations are currently being addressed, healthcare professionals also report concerns about the cost associated with immersive technology (8, 60). Thus, the cost-effectiveness of AR/MR also needs to be further evaluated. For example, AR/MR's potential to homogenize medical care may potentially reduce the length of hospitalization and the number of medical tests required, warranting economic analyses of cost-minimization and cost-consequence of its application. Bergin & Craven have recently published the first review addressing the cost-effectiveness of extended reality technologies (an umbrella term including virtual, augmented, and mixed reality), which addressed that such innovations are promising in a value-based healthcare context (61). These findings are encouraging, and future investigations should specifically review the value of AR/MR within various contexts.
Another barrier to AR/MR technology is the limitation in rendering highly detailed clinical imaging data and superimposing such images for pre-operative planning and procedural guidance. Present literature cautions against using this tool for procedures requiring sub-millimeter accuracy as it may be necessary when caring for neonates (7). However, its accuracy still proves relevant for less critical scenarios such as anatomy and surgical training (62, 63). Additional investigations also highlight issues with depth perception upon a 3D visualization and note that cybersickness may be linked to prolonged use (7, 8, 64, 65). While these limitations indicate areas that future work should explore, our preliminary implementation of MR in the NICU did not present significant obstacles.
Nevertheless, the mentioned challenges do not diminish the unparalleled potential of AR/MR technologies. While the surgical applications of AR/MR may necessitate further research, other aspects of the technology can be considered ready for deployment, especially in settings with an urgent need for enhanced medical care. The holographic projection of protocol information and reference media, along with the one-of-a-kind video-communication tool, as we explored in our preliminary implementation of the NeonLens, are currently highly applicable to tangible neonatal care. Improving newborn care can have profound individual and systemic benefits, and key aspects of AR/MR technology are ready to contribute to this mission.
To conclude, AR/MR technology presents the potential to positively impact several facets of medical care, including medical procedures, physical examination and diagnosis, and telemedicine. Each component warrants individual evaluation—while some features may be considered ready for integration, others yield further research. The applications of AR/MR in neonatal care should be regarded as relevant across diverse economic and clinical landscapes. Future investigations should assess each component of this technology in various medical settings, along with integrating AR/MR with other rising tools like artificial intelligence. Furthermore, emerging projects should explore the cost-effectiveness and impact on short- and long-term clinical outcomes.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Author contributions
PR: Writing – original draft, Writing – review & editing. DL: Writing – original draft, Writing – review & editing. Sd: Writing – original draft, Writing – review & editing. MM: Writing – original draft, Writing – review & editing. RR: Writing – original draft, Writing – review & editing. MM: Writing – original draft, Writing – review & editing. Bd: Software, Writing – original draft, Writing – review & editing. GV: Writing – original draft, Writing – review & editing.
Funding
The author(s) declare that financial support was received for the research and/or publication of this article. This study was conducted without external funding.
Acknowledgments
We sincerely thank the Division of Neonatology of the Irmandade da Santa Casa de Misericórdia de São Paulo and the dedicated research staff who made this work possible. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Irmandade da Santa Casa de Misericórdia or the Santa Casa de São Paulo School of Medicine.
Conflict of interest
GV is the founder of Protecting Brains & Saving Futures Organization. DL, MM, PR, BA, MM, RR, and SA are employed by Protecting Brains & Saving Futures, a social enterprise that provides remote neuromonitoring to high-risk newborns and developed the NeonLens system.
Generative AI statement
The author(s) declare that no Generative AI was used in the creation of this manuscript.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1. LaValle SM. Virtual Reality. Cambridge, United Kingdom New York, NY Port Melbourne New Delhi Singapore: Cambridge University Press (2023). p. 373.
2. Curran V, Xu X, Simmons K, Fleet L, Coombs H, Porter R, et al. A phenomenological study of the use of 360° virtual reality (VR) video in pediatric and neonatal resuscitation training. Health Technol. (2022) 12:151–9. doi: 10.1007/s12553-021-00615-w
3. Mao RQ, Lan L, Kay J, Lohre R, Ayeni OR, Goel DP, et al. Immersive virtual reality for surgical training: a systematic review. J Surg Res. (2021) 268:40–58. doi: 10.1016/j.jss.2021.06.045
4. Arena F, Collotta M, Pau G, Termine F. An overview of augmented reality. Computers. (2022) 11:28. doi: 10.3390/computers11020028
5. Hu H, Feng X, Shao Z, Xie M, Xu S, Wu X, et al. Application and prospect of mixed reality technology in medical field. Curr Med Sci. (2019) 39:1–6. doi: 10.1007/s11596-019-1992-8
6. Šlosar L, Voelcker-Rehage C, Paravlić AH, Abazovic E, De Bruin ED, Marusic U. Combining physical and virtual worlds for motor-cognitive training interventions: position paper with guidelines on technology classification in movement-related research. Front Psychol. (2022) 13:1009052. doi: 10.3389/fpsyg.2022.1009052
7. Gsaxner C, Li J, Pepe A, Jin Y, Kleesiek J, Schmalstieg D, et al. The HoloLens in medicine: a systematic review and taxonomy. Med Image Anal. (2023) 85:102757. doi: 10.1016/j.media.2023.102757
8. Palumbo A. Microsoft HoloLens 2 in medical and healthcare context: state of the art and future prospects. Sensors. (2022) 22:7709. doi: 10.3390/s22207709
9. Adams M, Bassler D, Bucher HU, Roth-Kleiner M, Berger TM, Braun J, et al. Variability of very low birth weight infant outcome and practice in Swiss and US neonatal units. Pediatrics. (2018) 141:e20173436. doi: 10.1542/peds.2017-3436
10. Horbar JD, Edwards EM, Greenberg LT, Morrow KA, Soll RF, Buus-Frank ME, et al. Variation in performance of neonatal intensive care units in the United States. JAMA Pediatr. (2017) 171:e164396. doi: 10.1001/jamapediatrics.2016.4396
11. Rysavy MA, Li L, Bell EF, Das A, Hintz SR, Stoll BJ, et al. Between-hospital variation in treatment and outcomes in extremely preterm infants. N Engl J Med. (2015) 372:1801–11. doi: 10.1056/NEJMoa1410689
12. Ehret DY, Patterson JK, Bose CL. Improving neonatal care. Clin Perinatol. (2017) 44:567–82. doi: 10.1016/j.clp.2017.05.002
13. Kitagawa M, Sugimoto M, Haruta H, Umezawa A, Kurokawa Y. Intraoperative holography navigation using a mixed-reality wearable computer during laparoscopic cholecystectomy. Surgery. (2022) 171:1006–13. doi: 10.1016/j.surg.2021.10.004
14. Aaskov J, Kawchuk GN, Hamaluik KD, Boulanger P, Hartvigsen J. X-ray vision: the accuracy and repeatability of a technology that allows clinicians to see spinal x-rays superimposed on a person’s back. PeerJ. (2019) 7:e6333. doi: 10.7717/peerj.6333
15. Yoon JW, Chen RE, Kim EJ, Akinduro OO, Kerezoudis P, Han PK, et al. Augmented reality for the surgeon: systematic review. Robot Comp Surg. (2018) 14:e1914. doi: 10.1002/rcs.1914
16. Gibby J, Cvetko S, Javan R, Parr R, Gibby W. Use of augmented reality for image-guided spine procedures. Eur Spine J. (2020) 29:1823–32. doi: 10.1007/s00586-020-06495-4
17. Agten CA, Dennler C, Rosskopf AB, Jaberg L, Pfirrmann CWA, Farshad M. Augmented reality–guided lumbar facet joint injections. Invest Radiol. (2018) 53:495–8. doi: 10.1097/RLI.0000000000000478
18. Buch VP, Mensah-Brown KG, Germi JW, Park BJ, Madsen PJ, Borja AJ, et al. Development of an intraoperative pipeline for holographic mixed reality visualization during spinal fusion surgery. Surg Innov. (2021) 28:427–37. doi: 10.1177/1553350620984339
19. Wang L, Sun Z, Zhang X, Sun Z, Wang J. A HoloLens based augmented reality navigation system for minimally invasive total knee arthroplasty. In: Yu H, Liu J, Liu L, Ju Z, Liu Y, Zhou D, editors. Intelligent Robotics and Applications. Lecture Notes in Computer Science. Cham: Springer International Publishing (2019). p. 519–30. doi: 10.1007/978-3-030-27529-7_44
20. Dennler C, Jaberg L, Spirig J, Agten C, Götschi T, Fürnstahl P, et al. Augmented reality-based navigation increases precision of pedicle screw insertion. J Orthop Surg Res. (2020) 15:174. doi: 10.1186/s13018-020-01690-x
21. Lau I, Gupta A, Ihdayhid A, Sun Z. Clinical applications of mixed reality and 3D printing in congenital heart disease. Biomolecules. (2022) 12:1548. doi: 10.3390/biom12111548
22. Brun H, Bugge RAB, Suther LKR, Birkeland S, Kumar R, Pelanis E, et al. Mixed reality holograms for heart surgery planning: first user experience in congenital heart disease. Eur Heart J Cardiovasc Imaging. (2019) 20:883–8. doi: 10.1093/ehjci/jey184
23. Gehrsitz P, Rompel O, Schöber M, Cesnjevar R, Purbojo A, Uder M, et al. Cinematic rendering in mixed-reality holograms: a new 3D preoperative planning tool in pediatric heart surgery. Front Cardiovasc Med. (2021) 8:633611. doi: 10.3389/fcvm.2021.633611
24. Morales Mojica CM, Navkar NV, Tsekos NV, Tsagkaris D, Webb A, Birbilis T, et al. Holographic interface for three-dimensional visualization of MRI on HoloLens: a prototype platform for MRI guided neurosurgeries. 2017 IEEE 17th International Conference on Bioinformatics and Bioengineering (BIBE). Washington, DC: IEEE (2017). p. 21–7. doi: 10.1109/BIBE.2017.00-84
25. Katayama M, Ueda K, Mitsuno D, Kino H. Intraoperative 3-dimensional projection of blood vessels on body surface using an augmented reality system. Plast Reconstr Surg Glob Open. (2020) 8:e3028. doi: 10.1097/GOX.0000000000003028
26. Nguyen T, Plishker W, Matisoff A, Sharma K, Shekhar R. HoloUS: augmented reality visualization of live ultrasound images using HoloLens for ultrasound-guided procedures. Int J CARS. (2022) 17:385–91. doi: 10.1007/s11548-021-02526-7
27. Azimi E, Niu Z, Stiber M, Greene N, Liu R, Molina C, et al. An interactive mixed reality platform for bedside surgical procedures. In: Martel AL, Abolmaesumi P, Stoyanov D, Mateus D, Zuluaga MA, Zhou SK, Racoceanu D, Joskowicz L, editors. Medical Image Computing and Computer Assisted Intervention—mICCAI 2020. Lecture Notes in Computer Science. Cham: Springer International Publishing (2020). p. 65–75. doi: 10.1007/978-3-030-59716-0_7
28. Fei G, Guo A, Booth T, Kiss E, Hallac R, Anderson A, et al. Development of an augmented reality system for tracheal intubation guidance of airway management. In: Wong BJF, Ilgner JF, editors. Imaging, Therapeutics, and Advanced Technology in Head and Neck Surgery and Otolaryngology 2024. San Francisco, United States: SPIE (2024). p. 9. doi: 10.1117/12.3007671
29. Huang CY, Thomas JB, Alismail A, Cohen A, Almutairi W, Daher NS, et al. The use of augmented reality glasses in central line simulation: “see one, simulate many, do one competently, and teach everyone.”. AMEP. (2018) 9:357–63. doi: 10.2147/AMEP.S160704
30. Alismail A, Thomas J, Daher NS, Cohen A, Almutairi W, Terry MH, et al. Augmented reality glasses improve adherence to evidence-based intubation practice. AMEP. (2019) 10:279–86. doi: 10.2147/AMEP.S201640
31. Venson JE, Berni J, Maia CS, Da Silva AM, d’Ornelas M, Maciel A. Medical imaging VR: can immersive 3D aid in diagnosis? Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology. Munich Germany: ACM (2016). p. 349–50. doi: 10.1145/2993369.2996333
32. Butaslac II, Luchetti A, Parolin E, Fujimoto Y, Kanbara M, De Cecco M, et al. The feasibility of augmented reality as a support tool for motor rehabilitation. In: Paolis LT D, Bourdot P, editors. Augmented Reality, Virtual Reality, and Computer Graphics. Lecture Notes in Computer Science. Cham: Springer International Publishing (2020). p. 165–73. doi: 10.1007/978-3-030-58468-9_13
33. Klinker K, Wiesche M, Krcmar H. Digital transformation in health care: augmented reality for hands-free service innovation. Inf Syst Front. (2020) 22:1419–31. doi: 10.1007/s10796-019-09937-7
34. Tanbeer SK, Sykes ER. Mivitals– xed reality interface for monitoring: a HoloLens based prototype for healthcare practices. Comput Struct Biotechnol J. (2024) 24:160–75. doi: 10.1016/j.csbj.2024.02.024
35. Shah SHH, Longva B, Hameed IA, Solberg M, Susanne T, Karlsen A. Health data management for nursing practice: an intelligent, holographic mixed-reality system. In: Stephanidis C, Antona M, Ntoa S, editors. HCI International 2021—posters. Communications in Computer and Information Science. Cham: Springer International Publishing (2021). p. 329–36. doi: 10.1007/978-3-030-78642-7_45
36. Tsang KD, Ottow MK, Van Heijst AFJ, Antonius TAJ. Electronic decision support in the delivery room using augmented reality to improve newborn life support guideline adherence: a randomized controlled pilot study. Simul Healthcare. (2022) 17:293–8. doi: 10.1097/SIH.0000000000000631
37. Velazco-Garcia JD, Shah DJ, Leiss EL, Tsekos NV. A modular and scalable computational framework for interactive immersion into imaging data with a holographic augmented reality interface. Comput Methods Programs Biomed. (2021) 198:105779. doi: 10.1016/j.cmpb.2020.105779
38. Sánchez-Margallo JA, Plaza De Miguel C, Fernández Anzules RA, Sánchez-Margallo FM. Application of mixed reality in medical training and surgical planning focused on minimally invasive surgery. Front Virtual Real. (2021) 2:692641. doi: 10.3389/frvir.2021.692641
39. Schoeb DS, Schwarz J, Hein S, Schlager D, Pohlmann PF, Frankenschmidt A, et al. Mixed reality for teaching catheter placement to medical students: a randomized single-blinded, prospective trial. BMC Med Educ. (2020) 20:510. doi: 10.1186/s12909-020-02450-5
40. Caruso TJ, Qian J, Lawrence K, Armstrong-Carter E, Domingue BW. From socrates to virtual reality: a historical review of learning theories and their influence on the training of anesthesiologists. JEPM. (2020) 22:E638. doi: 10.46374/volxxii-issue2-Caruso
41. Turan Z, Karabey SC. The use of immersive technologies in distance education: a systematic review. Educ Inf Technol. (2023) 28:16041–64. doi: 10.1007/s10639-023-11849-8
42. Kenoui M. Telemedicine meets augmented reality: healthcare services delivery and distance training. 2020 4th International Symposium on Informatics and its Applications (ISIA). M’sila, Algeria: IEEE (2020). p. 1–5. doi: 10.1109/ISIA51297.2020.9416532
43. Dinh A, Yin AL, Estrin D, Greenwald P, Fortenko A. Augmented reality in real-time telemedicine and telementoring: scoping review. JMIR Mhealth Uhealth. (2023) 11:e45464. doi: 10.2196/45464
44. Mitsuno D, Hirota Y, Akamatsu J, Kino H, Okamoto T, Ueda K. Telementoring demonstration in craniofacial surgery with HoloLens, skype, and three-layer facial models. J Craniofac Surg. (2019) 30:28–32. doi: 10.1097/SCS.0000000000004899
45. Wang S, Parsons M, Stone-McLean J, Rogers P, Boyd S, Hoover K, et al. Augmented reality as a telemedicine platform for remote procedural training. Sensors. (2017) 17:2294. doi: 10.3390/s17102294
46. Sirilak S, Muneesawang P. A new procedure for advancing telemedicine using the HoloLens. IEEE Access. (2018) 6:60224–33. doi: 10.1109/ACCESS.2018.2875558
47. Orchard L, Van M, Abbas J, Malik R, Stevenson J, Tolley N. Mixed-reality technology for clinical communication: objective assessment of the HoloLens 2 as a clinical communication device in a simulated on-call scenario. J Laryngol Otol. (2023) 137:1165–9. doi: 10.1017/S0022215123000531
48. Rebol M, Pietroszek K, Ranniger C, Hood C, Rutenberg A, Sikka N, et al. Mixed reality communication for medical procedures: teaching the placement of a central venous catheter. 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). Singapore, Singapore: IEEE (2022). p. 346–54. doi: 10.1109/ISMAR55827.2022.00050
49. Leandro DMK, Variane GFT, Dahlen A, Pietrobom RFR, De Castro JARR, Rodrigues DP, et al. Therapeutic hypothermia for neonatal hypoxic–ischemic encephalopathy: reducing variability in practice through a collaborative telemedicine initiative. Am J Perinatol. (2024) 41(16):2263–70. doi: 10.1055/s-0044-1786720
50. Palozzi G, Schettini I, Chirico A. Enhancing the sustainable goal of access to healthcare: findings from a literature review on telemedicine employment in rural areas. Sustainability. (2020) 12:3318. doi: 10.3390/su12083318
51. Rosa-Mangeret F, Benski A-C, Golaz A, Zala PZ, Kyokan M, Wagner N, et al. 2.5 Million annual deaths—are neonates in low- and middle-income countries too small to be seen? A bottom-up overview on neonatal Morbi-mortality. Tropical Med. (2022) 7:64. doi: 10.3390/tropicalmed7050064
52. Kaplan HC, Edwards EM, Soll RF, Morrow KA, Meyers J, Timpson W, et al. Variability in the systems of care supporting critical neonatal intensive care unit transitions. J Perinatol. (2020) 40:1546–53. doi: 10.1038/s41372-020-0720-3
53. Lee W, Shickh S, Assamad D, Luca S, Clausen M, Somerville C, et al. Patient-facing digital tools for delivering genetic services: a systematic review. J Med Genet. (2023) 60:1–10. doi: 10.1136/jmg-2022-108653
54. Green JA, Ephraim PL, Hill-Briggs F, Browne T, Strigo TS, Hauer CL, et al. Integrated digital health system tools to support decision making and treatment preparation in CKD: the PREPARE NOW study. Kidney Med. (2021) 3:565–575.e1. doi: 10.1016/j.xkme.2021.03.009
55. Fuerch JH, Yamada NK, Coelho PR, Lee HC, Halamek LP. Impact of a novel decision support tool on adherence to neonatal resuscitation program algorithm. Resuscitation. (2015) 88:52–6. doi: 10.1016/j.resuscitation.2014.12.016
56. McAdams RM, Trinh G. Using virtual reality–based simulation in neonatal resuscitation program training. Neoreviews. (2024) 25:e567–77. doi: 10.1542/neo.25-9-e567
57. Trudeau MO, Carrillo B, Nasr A, Gerstle JT, Azzie G. Comparison of adult and pediatric surgeons: insight into simulation-based tools that may improve expertise among experts. J Laparoendosc Adv Surg Tech A. (2018) 28:599–605. doi: 10.1089/lap.2017.0214
58. Ferrari LR. The pediatric airway: anatomy, challenges, and solutions. In: Mason MK, editor. Pediatric Sedation Outside of the Operating Room. Cham: Springer International Publishing (2021). p. 125–39. doi: 10.1007/978-3-030-58406-1_8
59. Proietti J, Boylan GB, Walsh BH. Regional variability in therapeutic hypothermia eligibility criteria for neonatal hypoxic-ischemic encephalopathy. Pediatr Res. (2024) 96:1153–61. doi: 10.1038/s41390-024-03184-6
60. Murali S, Paul KD, McGwin G, Ponce BA. Updates to the current landscape of augmented reality in medicine. Cureus. (2021) 13(5):e15054. doi: 10.7759/cureus.15054
61. Gómez Bergin AD, Craven MP. Virtual, augmented, mixed, and extended reality interventions in healthcare: a systematic review of health economic evaluations and cost-effectiveness. BMC Digit Health. (2023) 1:53. doi: 10.1186/s44247-023-00054-9
62. Chen L, Day TW, Tang W, John NW. Recent developments and future challenges in medical mixed reality. 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). Nantes: IEEE (2017). p. 123–35. doi: 10.1109/ISMAR.2017.29
63. Coelho G, Figueiredo EG, Rabelo NN, Rodrigues De Souza M, Fagundes CF, Teixeira MJ, et al. Development and evaluation of pediatric mixed-reality model for neuroendoscopic surgical training. World Neurosurg. (2020) 139:e189–202. doi: 10.1016/j.wneu.2020.03.170
64. Hughes CL, Fidopiastis C, Stanney KM, Bailey PS, Ruiz E. The psychometrics of cybersickness in augmented reality. Front Virtual Real. (2020) 1:602954. doi: 10.3389/frvir.2020.602954
Keywords: mixed reality (MR), augmented reality (AR), immersive technologies, neonatology, neonatal care, newborns and infants, sarnat score
Citation: Rodrigues PG, Leandro DMK, de Azevedo SS, Mimica MJ, Rodrigues RF, Magalhães M, dos Anjos BF and Variane GFT (2025) Transforming neonatal care: a position paper on the potential of augmented and mixed reality. Front. Digit. Health 7:1571521. doi: 10.3389/fdgth.2025.1571521
Received: 5 February 2025; Accepted: 25 March 2025;
Published: 11 April 2025.
Edited by:
Heather Benz, Johnson & Johnson Medtech (US), United StatesReviewed by:
Ramkrishna Mondal, All India Institute of Medical Sciences (Patna), IndiaSun-Yi Yang, Konyang University, Republic of Korea
Copyright: © 2025 Rodrigues, Leandro, de Azevedo, Mimica, Rodrigues, Magalhães, dos Anjos and Variane. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Gabriel Fernando Todeschi Variane, Z2FicmllbC52YXJpYW5lQHBic2YuY29tLmJy