AUTHOR=Saffaryazdi Nastaran , Wasim Syed Talal , Dileep Kuldeep , Nia Alireza Farrokhi , Nanayakkara Suranga , Broadbent Elizabeth , Billinghurst Mark TITLE=Using Facial Micro-Expressions in Combination With EEG and Physiological Signals for Emotion Recognition JOURNAL=Frontiers in Psychology VOLUME=Volume 13 - 2022 YEAR=2022 URL=https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2022.864047 DOI=10.3389/fpsyg.2022.864047 ISSN=1664-1078 ABSTRACT=Emotions are multimodal processes that play a crucial role in our everyday lives. A wide range of applications requires the ability to recognize emotions, including healthcare, education, human-computer interaction, virtual reality, intelligent agents, and entertainment. The most common modality for communicating emotional states is facial expression. Facial expressions, however, can be controlled and thus may not create reliable recognition. In contrast, facial micro-expressions are involuntary and subtle facial movements in response to external stimuli. Because facial micro-expressions cannot be controlled, we propose combining them with brain and physiological signals in order to detect emotions reliably. Using facial microexpressions, EEG, and physiological signals, we describe how to detect arousal and valence levels. A subject-independent strategy is used to evaluate our strategy on the DEAP dataset as well as our dataset. As a conclusion, we show our results compared with state-of-the-art and discuss the limitations and future efforts of using facial micro-expressions and physiological signals for emotion recognition.