Seeing Emotions: An Eye-Tracking Study of Emotion Recognition in Deaf Individuals amid Facial Occlusions

Yu Chen^1,2Shengqin Cao^2,3Zhiquan Zhou²Kaiwen Cheng^3*

• ¹School of foreign languages, Southeast University, Nanjing, Jiangsu Province, China
• ²Technical College for the Deaf, Tianjin University of Technology, Tianjin, China
• ³College of Language Intelligence, Sichuan International Studies University, Chongqing, China

Extensive research has demonstrated that facial occlusion significantly affects individuals' emotion recognition abilities. However, whether facial occlusion exacerbate the difficulty in emotion recognition for deaf individuals remains elusive. This study employed eye-tracking technology to investigate the mechanisms underlying emotion perception in deaf individuals under different facial occlusion conditions. We compared the percentage of eye and mouth gaze fixation in deaf and hearing participants as they judged different emotions (positive, neutral, negative) under three occlusion conditions (no occlusion, sunglasses, mask). The behavioral and eye-tracking results reveal that, first, facial occlusion by sunglasses and mask significantly impairs emotion perception and social communication for deaf individuals. Second, the eye area is more crucial for recognizing negative emotions, while the mouth area is critical for recognizing positive emotions. Third, deaf individuals exhibit a "happiness superiority effect", responding more favorably to positive emotions and showing an avoidance bias towards negative emotions. Besides, visual attention allocation strategies of deaf individuals tend to be relatively fixed and less adaptable to task demands. Overall, these findings support the integrative hypothesis of visual function in deaf individuals and provide insights for enhancing facial emotion recognition and optimizing social interaction strategies for the deaf community.