AUTHOR=Niinuma Koichiro , Onal Ertugrul Itir , Cohn Jeffrey F. , Jeni László A. 

TITLE=Systematic Evaluation of Design Choices for Deep Facial Action Coding Across Pose

JOURNAL=Frontiers in Computer Science

VOLUME=Volume 3 - 2021

YEAR=2021

URL=https://www.frontiersin.org/journals/computer-science/articles/10.3389/fcomp.2021.636094

DOI=10.3389/fcomp.2021.636094

ISSN=2624-9898

ABSTRACT=The performance of automated facial expression coding has improved steadily. Advances in deep learning techniques have been key to this success. While the advantage of modern deep learning techniques is clear, the contribution of critical design choices remains largely unknown, especially for AU occurrence and intensity across pose. Using the FERA 2017 database that provides a common protocol to evaluate the robustness to pose variation, we systematically evaluated design choices in pre-training, feature alignment, model size selection, and optimizer details. Informed by the findings, we developed an architecture that exceeds state-of-the-art on FERA 2017. The architecture achieved a 3.5% increase in F1 score for occurrence detection and a 5.8% increase in ICC for intensity estimation. To evaluate the generalizability of the architecture to unseen poses and new domains, we performed experiments across pose in FERA 2017 and across domains in DISFA and the UNBC Pain Archive.