AUTHOR=Daeglau Mareike , Otten Jürgen , Grimm Giso , Mirkovic Bojana , Hohmann Volker , Debener Stefan 

TITLE=Neural speech tracking in a virtual acoustic environment: audio-visual benefit for unscripted continuous speech

JOURNAL=Frontiers in Human Neuroscience

VOLUME=Volume 19 - 2025

YEAR=2025

URL=https://www.frontiersin.org/journals/human-neuroscience/articles/10.3389/fnhum.2025.1560558

DOI=10.3389/fnhum.2025.1560558

ISSN=1662-5161

ABSTRACT=The audio-visual benefit in speech perception—where congruent visual input enhances auditory processing—is well-documented across age groups, particularly in challenging listening conditions and among individuals with varying hearing abilities. However, most studies rely on highly controlled laboratory environments with scripted stimuli. Here, we examine the audio-visual benefit using unscripted, natural speech from untrained speakers within a virtual acoustic environment. Using electroencephalography (EEG) and cortical speech tracking, we assessed neural responses across audio-visual, audio-only, visual-only, and masked-lip conditions to isolate the role of lip movements. Additionally, we analysed individual differences in acoustic and visual features of the speakers, including pitch, jitter, and lip-openness, to explore their influence on the audio-visual speech tracking benefit. Results showed a significant audio-visual enhancement in speech tracking with background noise, with the masked-lip condition performing similarly to the audio-only condition, emphasizing the importance of lip movements in adverse listening situations. Our findings reveal the feasibility of cortical speech tracking with naturalistic stimuli and underscore the impact of individual speaker characteristics on audio-visual integration in real-world listening contexts.