Pupillary Responses to Masked and Gaze-Averted Faces

Cansu Malak^1,2Funda Yildirim^2,3*

• ¹Department of Brain and Cognitive Engineering, College of Engineering, Korea University, Seoul, Seoul, Republic of Korea
• ²Department of Computer Engineering, Yeditepe University, Istanbul, Türkiye
• ³Cognition, Data and Education Section, Faculty of Behavioural, Management and Social Sciences, University of Twente, Enschede, Netherlands

Face masks, a common practice during COVID-19, remain important in various cultural and medical contexts. Studies have shown how face masks affect our ability to recognize emotions, highlighting the role of facial features. Gaze direction plays a key role in modulating the identification of emotions, particularly in the presence of masks. So far, little is known about how gaze and masks influence emotion processing via physiological measures like pupil size. Here, we used pupillometry with 40 participants to investigate how emotion recognition (anger, fear, neutral) is affected by both gaze direction (direct, averted) and face mask conditions (mask, no mask). Behaviorally, our findings align with previous research, showing that the eye region plays a key role in identifying anger and neutral expressions more effectively than fear. Similarly, direct gaze improves accuracy for anger and neutral, while averted gaze enhances fear recognition. Pupillometry results revealed condition-specific changes in pupil size that partially mirrored the behavioral patterns, although no strong correlation with accuracy was found. However, pupil size was even more strongly modulated by recognition errors, with significantly greater dilation during incorrect trials across all emotions, especially for masked fearful faces, suggesting increased cognitive effort and ambiguity. The data also indicate compensatory processing mechanisms, when masks obscured parts of the face, participants appeared to rely more heavily on gaze direction and visible emotional cues. We propose that pupil dilation may reflect the cognitive load of emotion identification, providing important input for adaptive support applications in HCI and VR to improve user experiences.