Editorial: Cutting-edge technologies for multi-sensory research at the frontier between disciplines

Editorial on the Research Topic
Cutting-edge technologies for multi-sensory research at the frontier between disciplines

As we continue to push the boundaries of human interaction with the world, it's increasingly clear that the key to unlocking more authentic and engaging experiences lies in the realm of multisensory integration. The convergence of AI and multisensory research transforms our understanding of perception, enabling us to decipher complex patterns and predictive models within expansive sensory data (Liang, 2024).

The potential applications of this research are vast, from healthcare and education to entertainment and beyond. By developing sophisticated sensors and algorithms, we can create immersive experiences that simulate the richness of real-world interactions. Sensory substitution technologies, meanwhile, hold promises to compensate for sensory deficits, improving the lives of millions (Cornelio).

As we move forward, it's crucial that we prioritize interdisciplinary collaboration, bringing together experts from computational science, art, and clinical applications to tackle the challenges of multisensory research. By doing so, we can unlock new frontiers in human perception and create technologies that truly enhance our experiences (Cornelio et al., 2021).

This Frontiers Research Topic consists of five articles covering various topics:

Bartoletti et al. propose an integrative cognitive model to understand how personalized sensory environments impact cognitive performance, behavior and emotions. The model suggests optimal arousal improves focus, while metacognitive misjudgments can lead to suboptimal environment choices. AI-powered personalization can enhance experiences, particularly for older adults, through tailored approaches. Benefits include enhanced attention, therapeutic applications and user experiences in areas like VR and music therapy. However, risks include privacy erosion, metacognitive biases and polarized preferences. The model aims to create meaningful multisensory environments applicable to fields like research, art and marketing.

Moving from this broad cognitive model, Guberman et al. explored how specific visual cues—color and shape—influence taste perception of healthy apple snacks in a mixed reality context. Researchers found that while color or shape alone had no significant effect, the combination of congruent colors and shapes did impact taste perception. Participants associated angular shapes and the green color with sourness. The study highlights the role of visual cues in shaping taste experiences.

Whereas, the previous study altered passive perception, Parra et al. explored how a conversational agent, MyndFood, can actively promote mindful cooking and eating. In a study with 40 participants, those who interacted with the mindful version of the agent reported greater enjoyment and food awareness while preparing sushi compared to those who interacted with a non-mindful version. The findings suggest that conversational agents can enhance the cooking and eating experience by promoting mindfulness.

Addressing the different challenge of accessibility, Welewatta et al. developed SEMA, a multi-sensory system that uses tactile, auditory, and somatosensory cues to help visually impaired students experience and interpret paintings. Tested with 22 visually impaired participants, the system showed strong potential, with 92.6% of users preferring it, and could enhance art experiences for the visually impaired community.

Finally, turning to human-machine interaction in critical settings, Evangelou et al. studied how mid-air haptic technology affects user agency in gesture-based interactions, particularly in automotive settings. They found that mid-air haptics increased users' sense of agency compared to visual feedback, regardless of the meaning behind the feedback. The study involved 36 participants who completed a time perception task using gesture poses with different sensory feedback types. The results showed a positive association between mid-air haptics and user agency, trust, and usability, highlighting its potential benefits in automotive contexts.

Taken together, these five articles illustrate the breadth and potential of multisensory research at the frontier of disciplines.

They demonstrate how technology can both model and personalize our environments to improve cognitive performance (Bartoletti et al.), while also revealing how specific visual cues can alter our core perception of taste (Guberman et al.). These studies pave the way for concrete applications, whether by enriching daily experiences like cooking (Parra et al.) or by compensating for sensory deficits to make art accessible to the visually impaired (Welewatta et al.).

Finally, they highlight the importance of sensory feedback in enhancing user agency and trust in complex interactions, such as in-car gesture controls (Evangelou et al.). These contributions, spanning cognition, perception, accessibility, and human-computer interaction, confirm that multisensory integration is a critical field for shaping future human experiences.

Author contributions

SF: Writing – review & editing, Project administration, Writing – original draft. JT: Project administration, Writing – review & editing, Writing – original draft. J-CS: Writing – review & editing, Project administration, Writing – original draft. BM: Project administration, Writing – review & editing, Writing – original draft.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declare that Gen AI was used in the creation of this manuscript. Gen AI was used to improve the English.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Cornelio, P., Velasco, C., and Obrist, M. (2021). Multisensory integration as per technological advances: a review. Front. Neurosci. 15:652611. doi: 10.3389/fnins.2021.652611

PubMed Abstract | Crossref Full Text | Google Scholar

Liang, P. P. (2024). Foundations of multisensory artificial intelligence. arXiv. doi: 10.48550/arXiv.2404.18976

Crossref Full Text | Google Scholar