Extended Reality (XR) is an umbrella term that embraces the concepts of virtual/augmented/mixed reality. Today XR technologies are being increasingly adopted in various fields, such as entertainment, health, social network, education/training, marketing and tourism. The common aim of XR applications is to provide the technology user with an immersive and interactive experience, i.e., sense of being in a virtual or augmented environment, interacting with virtual beings or objects. To allow for a high level of multi-sensory immersive experience, it is crucial for XR technologies to produce auditory cues necessary for accurate sound localisation and the perception of various spatial attributes, as well as realistic visual and haptic cues. Furthermore, sensorimotor contingencies afforded by XR technologies (e.g. head and motion tracking-based controls) allow for a realistic interaction with the virtual environment and beings, which would lead to a more immersive experience.
The overall aim of this article collection is to explore the recent developments in immersive and interactive audio technologies for XR applications, and to provide new insights into future research to advance this field. Although the last decade saw a great advance in three-dimensional (3D) audio technologies for multichannel and binaural recording, reproduction and transmission, there still exist various challenges specific to XR contexts that need to be overcome through further research, e.g. natural and efficient rendering of dynamic cues for 6-degrees-of-freedom (6DoF) experience, individualisation and optimisation of head-related transfer functions, sound field capture and post-processing techniques, efficient virtual acoustic modeling, etc. Furthermore, rapid advances in artificial intelligence, human-computer interaction, internet of things, and cloud and edge processing technologies open up new opportunities for the development of new immersive and interactive audio technologies. In all of these, trade-off between perceived accuracy/quality and computational efficiency need to be considered, and therefore more research into perceptual-model-based approaches might be required.
The scope of this Research Topic covers all audio and acoustic signal processing topics aimed for XR applications. For example:
• Ambisonics
• Auditory-visual interaction
• Audio-haptic feedback
• Binaural synthesis techniques
• HRTF individualisation and optimisation
• Holographic auditory display
• Immersive audio using Internet-of-Things
• Machine learning-based approaches
• Microphone array techniques
• Objective metrics for auditory immersiveness measurements
• Six-degrees-of-freedom audio capture and processing
• Sound field synthesis methods
• Virtual acoustic simulation techniques
Contributors are encouraged to clearly indicate the relevance of their research to an XR context in the abstract. Research conducted in a multi-sensory experimental setting (e.g. audio-visual VR and AR) is especially welcome. But audio-only research is also accepted as long as its practical use case in XR is clearly defined (e.g. 3D music, audio AR, navigation, etc.)
Extended Reality (XR) is an umbrella term that embraces the concepts of virtual/augmented/mixed reality. Today XR technologies are being increasingly adopted in various fields, such as entertainment, health, social network, education/training, marketing and tourism. The common aim of XR applications is to provide the technology user with an immersive and interactive experience, i.e., sense of being in a virtual or augmented environment, interacting with virtual beings or objects. To allow for a high level of multi-sensory immersive experience, it is crucial for XR technologies to produce auditory cues necessary for accurate sound localisation and the perception of various spatial attributes, as well as realistic visual and haptic cues. Furthermore, sensorimotor contingencies afforded by XR technologies (e.g. head and motion tracking-based controls) allow for a realistic interaction with the virtual environment and beings, which would lead to a more immersive experience.
The overall aim of this article collection is to explore the recent developments in immersive and interactive audio technologies for XR applications, and to provide new insights into future research to advance this field. Although the last decade saw a great advance in three-dimensional (3D) audio technologies for multichannel and binaural recording, reproduction and transmission, there still exist various challenges specific to XR contexts that need to be overcome through further research, e.g. natural and efficient rendering of dynamic cues for 6-degrees-of-freedom (6DoF) experience, individualisation and optimisation of head-related transfer functions, sound field capture and post-processing techniques, efficient virtual acoustic modeling, etc. Furthermore, rapid advances in artificial intelligence, human-computer interaction, internet of things, and cloud and edge processing technologies open up new opportunities for the development of new immersive and interactive audio technologies. In all of these, trade-off between perceived accuracy/quality and computational efficiency need to be considered, and therefore more research into perceptual-model-based approaches might be required.
The scope of this Research Topic covers all audio and acoustic signal processing topics aimed for XR applications. For example:
• Ambisonics
• Auditory-visual interaction
• Audio-haptic feedback
• Binaural synthesis techniques
• HRTF individualisation and optimisation
• Holographic auditory display
• Immersive audio using Internet-of-Things
• Machine learning-based approaches
• Microphone array techniques
• Objective metrics for auditory immersiveness measurements
• Six-degrees-of-freedom audio capture and processing
• Sound field synthesis methods
• Virtual acoustic simulation techniques
Contributors are encouraged to clearly indicate the relevance of their research to an XR context in the abstract. Research conducted in a multi-sensory experimental setting (e.g. audio-visual VR and AR) is especially welcome. But audio-only research is also accepted as long as its practical use case in XR is clearly defined (e.g. 3D music, audio AR, navigation, etc.)
