AUTHOR=Yilmaz Funda , van Leeuwen Tessa M. , Güçlü Umut , Güçlütürk Yağmur , van Lier Rob 

TITLE=An fMRI study of crossmodal emotional congruency and the role of semantic content in the aesthetic appreciation of naturalistic art

JOURNAL=Frontiers in Neuroscience

VOLUME=Volume 19 - 2025

YEAR=2025

URL=https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2025.1516070

DOI=10.3389/fnins.2025.1516070

ISSN=1662-453X

ABSTRACT=Numerous studies have explored crossmodal correspondences, yet have so far lacked insight into how crossmodal correspondences influence audiovisual emotional integration and aesthetic beauty. Our study investigated the behavioral and neural underpinnings of audiovisual emotional congruency in art perception. Participants viewed ‘happy’ or ‘sad’ paintings in an unimodal (visual) condition or paired with congruent or incongruent music (crossmodal condition). In the crossmodal condition, the music could be emotionally congruent (e.g., happy painting, happy music) or incongruent with the painting (e.g., happy painting, sad music). We also created Fourier Scrambled versions of each painting to test for the influence of semantics. We tested 21 participants with fMRI while they rated the presentations. Beauty ratings did not differ for unimodal and crossmodal presentations (when aggregating across incongruent and congruent crossmodal presentations). We found that crossmodal conditions activated sensory and emotion-processing areas. When zooming in on the crossmodal conditions, the results revealed that emotional congruency between the visual and auditory information resulted in higher beauty ratings than incongruent pairs. Furthermore, semantic information enhanced beauty ratings in congruent trials, which elicited distinct activations in related sensory areas, emotion-processing areas, and frontal areas for cognitive processing. The significant interaction effect for Congruency × Semantics, controlling for low-level features like color and brightness, observed in the behavioral results was further revealed in the fMRI findings, which showed heightened activation in the ventral stream and emotion-related areas for the congruent conditions. This demonstrates that emotional congruency not only increased beauty ratings but also increased the in-depth processing of the paintings. For incongruent versus congruent comparisons, the results suggest that a frontoparietal network and caudate may be involved in emotional incongruency. Our study reveals specific neural mechanisms, like ventral stream activation, that connect emotional congruency with aesthetic judgments in crossmodal experiences. This study contributes to the fields of art perception, neuroaesthetics, and audiovisual affective integration by using naturalistic art stimuli in combination with behavioral and fMRI analyses.