AUTHOR=Saffaryazdi Nastaran , Wasim Syed Talal , Dileep Kuldeep , Nia Alireza Farrokhi , Nanayakkara Suranga , Broadbent Elizabeth , Billinghurst Mark 

TITLE=Using Facial Micro-Expressions in Combination With EEG and Physiological Signals for Emotion Recognition

JOURNAL=Frontiers in Psychology

VOLUME=Volume 13 - 2022

YEAR=2022

URL=https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2022.864047

DOI=10.3389/fpsyg.2022.864047

ISSN=1664-1078

ABSTRACT=Emotions are multimodal processes that play a crucial role in our everyday lives. A wide range of applications requires the ability to recognize emotions, including healthcare, education, human-computer interaction, virtual reality, intelligent agents, and entertainment. The most common modality for communicating emotional states is facial expression. Facial expressions, however, can be controlled and thus may not create reliable recognition. In contrast, facial micro-expressions are involuntary and subtle facial movements in response to external stimuli. Because facial micro-expressions cannot be controlled, we propose combining them with brain and physiological signals in order to detect emotions reliably.
 Using facial microexpressions, EEG, and physiological signals, we describe how to detect arousal and valence levels. A subject-independent strategy is used to evaluate our strategy on the DEAP dataset as well as our dataset. As a conclusion, we show our results compared with state-of-the-art and discuss the limitations and future efforts of using facial micro-expressions and physiological signals for emotion recognition.