Evaluating Multimodal Physiological Signals for Fear Detection: Relative Utility of Pupillometry, Heart Rate, and EEG

Yuki Ebato¹Tomita Saki¹Isshu Wakita¹Ayumu Ueno¹Tomoaki Ishibashi²Tetsuya Takahashi^2,3,4Sou Nobukawa^1,5*

• ¹Chiba Institute of Technology, Narashino, Japan
• ²University of Fukui, Fukui, Fukui, Japan
• ³Kanazawa University, Kanazawa, Ishikawa, Japan
• ⁴Uozu Shinkei Sanatorium, Uozu, Japan
• ⁵National Center of Neurology and Psychiatry (Japan), Kodaira, Tokyo, Japan

Fear is a fundamental emotion essential for survival; however, excessive fear can lead to anxiety disorders and other adverse consequences. Monitoring fear states is crucial for timely intervention and improved mental well-being. Although functional magnetic resonance imaging (fMRI) has provided valuable insights into the neural networks associated with fear, its high cost and environmental constraints limit its practical application in daily life. Electroencephalography (EEG) offers a more accessible alternative but struggles to capture deep brain activity.Physiological measures such as pupil dynamics and heart rate can provide indirect insights into these deeper processes, yet they are often studied in isolation. In this context, we aimed to evaluate the practical effectiveness and limitations of a multimodal approach that combines pupil dynamics and heart rate̶ indirect indicators of deep brain activity̶ with EEG, a temporally precise but spatially limited measure of cortical responses. We simultaneously recorded EEG, pupillometry, and heart rate in 40 healthy male participants exposed to fear-inducing and neutral visual stimuli, while also assessing their psychological states. Fear-inducing stimuli elicited distinct physiological responses, including increased occipital theta power, pupil dilation, and decreased heart rate. Notably, pupil size was the most sensitive discriminator of emotional state, though the integration of modalities yielded only limited improvement in classification accuracy. These findings provide empirical support for the feasibility of multimodal physiological monitoring of fear and underscore the need for further refinement for real-world applications.