AUTHOR=Vaessen Maarten , Van der Heijden Kiki , de Gelder Beatrice 

TITLE=Modality-specific brain representations during automatic processing of face, voice and body expressions

JOURNAL=Frontiers in Neuroscience

VOLUME=Volume 17 - 2023

YEAR=2023

URL=https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2023.1132088

DOI=10.3389/fnins.2023.1132088

ISSN=1662-453X

ABSTRACT=A central question in affective science and its clinical applications is how emotions are experienced and represented in the brain. This question is complicated by the fact that information about emotions is conveyed by a range of different stimulus categories and different sensory modalities. In the traditional view recognition of emotions is based on the use  of emotion concepts that are also the ones typically used in descriptions of mental states, irrespective of the sensory modality. This perspective previously motivated the search for abstract representations of emotions in the brain, functioning across variations in stimulus type (face, body, voice) and sensory origin (visual, auditory) and is now challenged from different sides. Given that emotion signals serve to trigger rapid automatic behavioral responses under circumstances where the body or the voice signal may trigger more rapid adaptation than the facial expression, the case can be made for modality and stimulus modality specific brain representations. To test this hypothesis, we presented participants with naturalistic dynamic emotion expressions of the face, the whole body, or the voice in a functional magnetic resonance (fMRI) study. To focus on automatic emotion processing and sidestep explicit emotion recognition and associated conceptual processes, participants performed an unrelated target detection task presented in a different sensory modality than the stimulus. By using multivariate analyses to assess neural activity patterns in response to the different stimuli types, we show a stimulus category and modality specific brain organization of affective signals. Our findings are consistent with the notion that under ecological conditions the various sensory emotion expressions of the face, body and voice may have different functional roles, even when from an abstract conceptual vantage point, they may all exemplify the same emotion.