Helge Rhodin1
Anna Lisa Vollmer
Ingo Todt- 1Faculty of Technology, AG Visual AI for Extended Reality, Bielefeld University, Bielefeld, Germany
- 2Department of Otolaryngology, Medical Faculty OWL, Bielefeld University, Campus Klinikum Bielefeld Mitte, Bielefeld, Germany
- 3AG Interactive Robotics, Medical Faculty OWL, Bielefeld University, Bielefeld, Germany
Background: Augmented reality (AR) glasses can be utilized for various medical applications. Primarily, a visual overlay on the optic screen offers additional operational information. A transfer of acoustic information via speech-to-text transcript using AR glasses presents a new non-surgical option to support patients with different forms of hearing loss. This study aimed to evaluate different AR glasses for speech-to-text transcription.
Methods: We compared four different AR glasses systems (Even Realities, G1; Meizu, MYVU IMIKI; XREAL, AIR, and Epson, Moverio 40) in terms of speech-to-text transcription, design, software, microphone and connection in this laboratory based study. Speech-capturing ability was tested using free-field numbers, monosyllables, and OLSA in quiet and in noise.
Results: The AR systems achieved Freiburger monosyllabic speech recognition rates between 20 and 45% at 65 dB. OLSA in quiet results vary between 77 and 100%, with increases of +1.7 dB and +3.5 dB in noise. AR systems differ substantially in terms of design, software, microphone position, and connection. Proposed indication groups are given.
Conclusion: AR glasses provide a potential supportive tool for patients with specific indications suffering from hearing loss. The systems show limitations in challenging hearing situations.
Introduction
The gaming industry, as a key driver for the introduction of extended reality (XR), played a central role in the distribution of augmented reality (AR) and virtual reality (VR) glasses. While VR sets the user in a complete digital visual environment, AR provides a visual overlay on top of real-world perception. Visual capture and overlay by AR and VR are expected to revolutionize how we interact with both the digital and real worlds (1).
Although they have been used so far primarily in non-regular daily settings, various applications, even in otolaryngology, have been tested and described. In a cadaveric study, Chan et al. utilized AR with TORS (trans oral robotic surgery) to enhance the identification of critical structures. Other groups have attempted to utilize AR to enhance parotid surgeries relevant to structure identification (2). Further, it was used for free flap harvesting and reconstructive planning (3).
Otology can be assumed to be a primary driver in digital visual applications. By definition, the ROBOSCOPE system (BHS, Innsbruck, Austria) is a VR system that enables high-precision surgery, such as cochlear implantation, free flap anastomosis, and neurosurgical procedures (4, 5). Even here, augmenting electrophysiological information is helpful (6) and can be implemented into the surgical procedure.
AR-guided surgery has been used to implant bone-conductive devices, improving accuracy (7). An AR system guiding through the Da Vinci Si System was used to perform cadaveric mastoidectomies, posterior tympanotomies, and cochleostomies (8).
However, AR has much more to offer than just an overlay of additional, primarily anatomic visual information. Cross-modal sensory interaction is well-established and applied in various clinical fields, such as tinnitus and vestibular rehabilitation. The principle of the Lenire system involves a tongue stimulator designed to alleviate the burden of tinnitus (9). This concept is based on the approach described by Danilov et al. (10), where a device called the “BrainPort” stimulates the tongue to enhance balance control. In this context, the principle is applied when head movements deviate from an acceptable range, helping to improve balance.
A mixed-case example is the successful use of the Wii board for vestibular rehabilitation, which used proprioceptive training and visual input (11).
VR is used to rehabilitate children with cochlear implants to improve their spatial hearing skills by incorporating visual information into gamified content (12). Here, visual and auditory information is shared as an example of successful cross-modal sensory interaction.
The use of transcripts has been widely recognized for decades as a support for individuals who are hard of hearing when watching television. Newly software-based automated transcripts have changed the working field of simultaneous translators and led to the development of software and microphone systems, allowing for the differentiation of different speakers (e.g., SPEAKSEER).
The recently developed AR glasses enable the integration of transcription software and its visual presentation during a conversation. Related to the design of the glasses, they have an integrative character and allow specific patients with hearing loss or deafness to see, for the first time, a communicative integration.
This study aims to compare different AR systems as a communicative device in terms of their speech to text capturing abilities, identify their advantages and disadvantages, and discuss potential application groups.
Materials and methods
We compared four different systems of AR glasses with the ability for an automated transcript (G1, Even Realities, Shenzen, China; Myvu, Meizu, Guangdong, China; Air, XReal, Haidan, China; Moverio BT 40, Epson, Suwa, Japan) in terms of design, connectivity, software, security design and microphone design. The glasses themselves are shown in Figures 1A–D. The system’s automated transcription capabilities were evaluated in a sound booth using the Freiburger monosyllabic test (at 65 dB and 80 dB), the numbers test (at 65 dB and 80 dB), and the Oldenburger sentence test (OLSA) in an open-set format, both in quiet and in noise conditions. Signal and noise came frontal. The volunteer was normal hearing with regular reading abilities.
Figure 1. (A) Moverio BT 40. (B) AIR. (C) MYVU. (D) G1.
Besides pure communicative support, additional aspects are essential for evaluating the integrative value of the various systems. Regarding the glass design, the AIR and Moverio are based on a screen principle distinct from the G1 and MYVU glasses. The first design is based on prism glasses, which enable the projection of full-color images across entire browser screens. The software used can be switched from a black letter on a white screen or a white letter to a black screen. In contrast, the latter photodiode projection displays information in letters and numbers in a single color (green) (Figures 2A,B). The G1 system allows for the adaptation of the glasses to the individual visual deficit by adjusting the lens. The AIR system resembles sunglasses, covering the prism glass. Limited system control directly on the glasses is possible for the AIR, G1, and MYVU, either by touching the frame’s sides using the accelerometer through head movements.
Figure 2. (A) View the transcript through AIR glass. (B) View the transcript through the MYVU glass.
Further steering of the systems can be performed by a controller (connected via USB-C) for the AIR and Moverio systems or by Bluetooth and a mobile phone for the G1 and MYVU systems. A direct microphone is integrated into the glasses for the G1, AIR, and MYVU systems. The Moverio has its microphone on the controller. Offline functionality is system-dependent: AIR and Moverio support offline communication by projecting content directly from smartphone or controller screens, while the G1 and MYVU lack this capability entirely.
Implementing Laviere microphones to improve individualization and directionality is possible for the AIR and Moverio systems.
While the G1 and MYVU system-based software allows, besides the speech-to-text transcript, translation into and from different languages, the package contains an AI communicator (ChatGPT, Perplexity for G1; unclear for MYVU), a prompter tool, and a navigation tool. The AIR and Moverio systems have access to the Google apps ecosystem. The Google automated transcript software is regularly used by hearing impaired. The G1 system can be utilized by apps based on AugmentOS (Table 1).
Table 1. Comparison of different AR systems.
Results
As a communicative benchmark used in the regular clinical setting, we performed monosyllabic word tests, understanding of numbers, and OLSA testing. This testing allows a comparison with other supportive systems for patients with hearing loss. A primary difference is that speech capturing is stable when using glasses, whereas speech understanding in patients is affected by various variables that influence the auditory pathway.
Patients’ speech understanding of the different systems is shown in Table 2.
Table 2. Best speech understanding of different AR systems.
The tested systems showed good speech-capturing abilities in quiet and for numbers. OLSA’s quiet and tested numbers performance was up to 100%. Speech capturing in noise (OLSA in noise) or in difficult situations (such as monosyllabic speech) was performed for all systems, highlighting the current limitations of the systems.
Discussion
AR and AI bear enormous potential in many medical fields. In the field of otolaryngology, AI-based transcription software has been used to replace sign language interpreters.
AR glass systems in the gaming industry or industrial use are almost unrelated to communicative content. The combination of transcription software and AR glasses has enormous potential as an additional tool for rehabilitating individuals who are deaf. It allows specific groups to integrate into speech-to-text-based communication for the first time. This means, for example, that people who are dependent on pure sign language-based communication could have access to speech-based communication. A key advantage is the glass’s non-surgical character of informational transmission in cases of severe hearing loss and deafness. Surgery is a limiting point in patients with severe disabilities and comorbidities.
A detailed comparison of the technical abilities of the different systems reveals their deficits, pros, and cons, as well as the specific areas where a substantial improvement could significantly enhance communication or where the difference between the systems is a matter of personal preference. A frontal microphone appears to capture speech more effectively (G1). However, this does not affect hearing in noise in this study.
AIR and Moverio are, in terms of design, less inclusive than MYVU and G1. On- or offline communication is of significant importance in terms of communicative security.
Additionally, it is worth noting that a cross-modal sensory solution, such as AR glasses, is not comparable to a sensory support solution (hearing aids, cochlear implants). We see AR glasses as a solution for patient groups without any other options for inclusion into speech-text-based communication as a supportive tool to improve speech-text-based communication during rehabilitation or as an addition. Clinical cases were a surgery for cochlea implantation even in local anesthesia is not possible are well known (e.g., white cochlea, NF II cases, bilateral traumatic cases). Even in cases were hearing aid use is problematic, AR glasses can be a solution. As support before cochlear implantation or during the rehabilitative process, AR glasses may serve as an informational tool to bridge the gap between acoustic fragments and audiological information. Future clinical experience may reveal additional use cases. Related to the visual integration in communicative rehabilitation, possible influential factors that are currently unclear will become apparent.
In contrast to a sensory support system like hearing aids and cochlear implants, which offer passive support to the patient, the AR system requires active cognitive engagement from hearing-impaired or deaf individuals. This performance is influenced by the ability to read (e.g., alphabetic, age dependence, communicative level, attention, fatigue, …).
The outcome of the speech understanding by the systems depends mainly on the microphone’s ability and the software’s speech recognition quality. It can be assumed that further AI integration of speech-to-noise separation will significantly improve speech capture, similar to what is currently observable in the hearing aid field through AI integration. The potential connection of a Laviere microphone may be crucial in further enhancing transcriptive quality in specific interindividual situations in noisy environments.
The glass design plays a significant role in including deaf persons. Here, regular looks like standard glasses; weight and adjustability are essential for addressing visual impairment. On the other hand, connecting to a controller/ mobile phone limits the mobility of wearing the glasses.
The underlying software, particularly in terms of upgradeability, integration with further applications, and ease of adjustment, is another critical factor. The open-source nature of the software, including Android XR and the Augment OS, presents a potential path for future developments in this direction. This would follow the understanding of seeing AR glasses as a platform for further applications. The limits of the transcripts persist in their inability to convey accentuations that deliver emotions, irony, or cynicism.
A significant drawback of AR systems is the dependence on a second device, such as a mobile phone or a controller, for the transcript. A second device can be lost, and battery lifetime can be a relevant limitation for communication. Another essential aspect is third-party communication, as a mobile online connection or Wi-Fi is dependent on two of the four glass systems. This fact raises concerns about the security of interpersonal communication. Communicative security is essential for widespread use, as the nature of interpersonal communication may not be compromised. We see currently the medical indication for an AR device in two fields: (A) Direct sensory support, if a hearing aid, cochlea implant is not possible. This are, e.g., prelingual deafness and reading ability (sign-language dependence), bilateral nerve deficiency, hearing loss, and chronic otitis externa without the possibility of performing an active middle ear implant surgery. (B) Additionally even for the support for the rehabilitation with hearing aids and CI (e.g., low CI outcome, low HA outcome, temporary use during CI rehabilitation.
The limitations of the systems are evident in their inability to capture directionality. Here, further developments in software and microphone techniques are needed to integrate these crucial points for improved speech capturing. It’s important to underline that is not a clinical study. It compares technically the different devices.
Conclusion
AR glasses offer a new assistive communication support for select patients with hearing loss in specific indication groups. Current systems have their particular design advantages and disadvantages and should be chosen on an individual base. The systems show limitations in challenging hearing situations.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent was not required to participate in this study in accordance with the national legislation and the institutional requirements.
Author contributions
HR: Formal analysis, Supervision, Methodology, Writing – review & editing. IE: Investigation, Writing – review & editing. SA: Investigation, Writing – review & editing. CP: Investigation, Writing – review & editing, Visualization. A-LV: Writing – review & editing, Validation, Formal analysis. IT: Supervision, Data curation, Conceptualization, Writing – original draft, Methodology, Resources, Writing – review & editing, Project administration.
Funding
The author(s) declare that no financial support was received for the research and/or publication of this article.
Acknowledgments
We would like to thank the staff of Westkamp School, Bielefeld-Sennestadt, for their invaluable support.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The author(s) declared that they were an editorial board member of Frontiers, at the time of submission. This had no impact on the peer review process and the final decision.
Generative AI statement
The author(s) declare that no Gen AI was used in the creation of this manuscript.
Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1. Xiong, J, Hsiang, EL, He, Z, Zhan, T, and Wu, ST. Augmented reality and virtual reality displays: emerging technologies and future perspectives. Light Sci Appl. (2021) 10:216. doi: 10.1038/s41377-021-00658-8
2. Scherl, C, Ludwig, S, Hesser, J, MonjiAzad, S, Stallkamp, J, Jungbauer, F, et al. Augmented Reality in der Kopf-Hals-Chirurgie [Augmented Reality in head and neck surgery]. Laryngorhinootologie. German. (2025) 104:487–497. doi: 10.1055/a-2463-4599
3. Necker, FN, Chang, M, Leuze, C, Topf, MC, Daniel, BL, Baik, FM, et al. Virtual resection specimen integration using augmented reality holograms to guide margin communication and flap sizing. Otolaryngol Head Neck Surg. (2023) 169:1083–5. doi: 10.1002/ohn.325
4. Holland-Elliott, T, Marineni, S, Patel, N, Ameerally, P, and Mair, M. Novel use of a robot for microvascular anastomosis in head and neck surgery. Br J Oral Maxillofac Surg. (2025), 63:154–155. doi: 10.1016/j.bjoms.2024
5. Piloni, M, Bailo, M, Gagliardi, F, and Mortini, P. Resection of intracranial tumors with a robotic-assisted digital microscope: a preliminary experience with the robotic scope. World Neurosurg. (2021) 152:e205–11. doi: 10.1016/j.wneu.2021.05.075
6. Eichler, T, Lakomek, A, Waschkies, L, Meyer, M, Sadok, N, Lang, S, et al. Two different methods to digitally visualize continuous electrocochleography potentials during cochlear implantation: a first description of feasibility. Eur Arch Otorrinolaringol. (2024) 281:2913–20. doi: 10.1007/s00405-023-08400-3
7. Lui, JT, Dahm, V, Chen, JM, Lin, VY, Irish, JC, le, TN, et al. Using augmented reality to guide bone conduction device implantation. Sci Rep. (2023) 13:7182. doi: 10.1038/s41598-023-33523-2
8. Liu, WP, Azizian, M, Sorger, J, Taylor, RH, Reilly, BK, Cleary, K, et al. Cadaveric feasibility study of da Vinci Si-assisted cochlear implant with augmented visual navigation for otologic surgery. JAMA Otolaryngol Head Neck Surg. (2014) 140:208–14. doi: 10.1001/jamaoto.2013.6443
9. Boedts, M, Buechner, A, Khoo, SG, Gjaltema, W, Moreels, F, Lesinski-Schiedat, A, et al. Combining sound with tongue stimulation for the treatment of tinnitus: a multi-site single-arm controlled pivotal trial. Nat Commun. (2024) 15:6806. doi: 10.1038/s41467-024-50473-z
10. Danilov, YP, Tyler, ME, Skinner, KL, Hogle, RA, and Bach-y-Rita, P. Efficacy of electro tactile, vestibular substitution in patients with peripheral and central vestibular loss. J Vestib Res. (2007) 17:119–30. doi: 10.3233/VES-2007-172-307
11. Lange, B, Flynn, S, Proffitt, R, Chang, CY, and Rizzo, AS. Development of an interactive game-based rehabilitation tool for dynamic balance training. Top Stroke Rehabil. (2010) 17:345–52. doi: 10.1310/tsr1705-345
12. Parmar, BJ, Salorio-Corbetto, M, Picinali, L, Mahon, M, Nightingale, R, Somerset, S, et al. Virtual reality games for spatial hearing training in children and young people with bilateral cochlear implants: the “both ears (BEARS)” approach. Front Neurosci. (2024) 18:1491954. doi: 10.3389/fnins.2024.1491954
Keywords: augmented reality, artificial intelligence, hearing loss, indications of sign language, speech to text transcripts
Citation: Rhodin H, Ersoy I, Aygun S, Pfeiffer CJ, Vollmer AL and Todt I (2025) Comparison of augmented reality glasses for the assistive communication support of hearing loss. Front. Neurol. 16:1635699. doi: 10.3389/fneur.2025.1635699
Edited by:
Isabelle Mosnier, Hôpitaux Universitaires Pitié Salpêtrière, FranceReviewed by:
Hans Thomeer, Utrecht University, NetherlandsMarco Boldreghini, University of Turin, Italy
Copyright © 2025 Rhodin, Ersoy, Aygun, Pfeiffer, Vollmer and Todt. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Ingo Todt, dG9kdEBnbXgubmV0