AUTHOR=Pan Hang , Yang Hongling , Xie Lun , Wang Zhiliang 

TITLE=Multi-scale fusion visual attention network for facial micro-expression recognition

JOURNAL=Frontiers in Neuroscience

VOLUME=Volume 17 - 2023

YEAR=2023

URL=https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2023.1216181

DOI=10.3389/fnins.2023.1216181

ISSN=1662-453X

ABSTRACT=Micro-expressions are facial muscle movements that hide genuine emotions. In response to the challenge of micro-expression low-intensity, recent studies have attempted to locate localized areas of facial muscle movement. However, this ignores the feature redundancy caused by the inaccurate locating of the regions of interest. This paper proposes a novel multi-scale fusion visual attention network (MFVAN), which learns multi-scale local attention weights to mask regions of redundancy features. Specifically, this model extracts the multi-scale features of the apex frame in the microexpression video clips by convolutional neural networks. The attention mechanism focuses on the weights of local region features in the multi-scale feature maps. Then, we mask operate redundancy regions in multi-scale features and fuse local features with high attention weights for microexpression recognition. The self-supervision and transfer learning reduce the influence of individual identity attributes and increase the robustness of multi-scale feature maps. Finally, the multi-scale classification loss, mask loss, and removing individual identity attributes loss joint to optimize the model. The proposed MFVAN method is evaluated on SMIC, CASME II, SAMM, and 3DB-Combined datasets that achieve state-of-the-art performance. The experimental results show that focusing on local at the multi-scale contributes to micro-expression recognition.