AUTHOR=Guo Yanan , Cao Lin , Du Kangning 

TITLE=Cross Task Modality Alignment Network for Sketch Face Recognition

JOURNAL=Frontiers in Neurorobotics

VOLUME=Volume 16 - 2022

YEAR=2022

URL=https://www.frontiersin.org/journals/neurorobotics/articles/10.3389/fnbot.2022.823484

DOI=10.3389/fnbot.2022.823484

ISSN=1662-5218

ABSTRACT=The task of sketch face recognition refers to matching cross-modality facial image from sketch to photo, which has widely application in criminal investigation area. It remains a challenging cross-modality image retrieval problem due to small-scale sketch face samples and the high intraclass variations caused by cross-modality discrepancy. Existing works aim to bridge the cross-modality gap by inter-modality feature alignment approaches, however, the small sample problem has received much less attention, resulting in limited performance. In this paper, an effective Cross Task Modality Alignment Network (CTMAN) is proposed for sketch face recognition. To address the small sample problem, a meta learning training episode strategy is firstly introduced to mimic few-shot tasks. Based on the episode strategy, a two-stream network termed modality alignment embedding learning is used to capture more modality-specific and sharable features, meanwhile, two cross task memory mechanisms are proposed to collect more sufficient negative features to further improve the feature learning. Finally, a cross task modality alignment loss is proposed to captures modality related information of cross task feature for more effective training. Extensive experiments are conducted to validate the superiority of the CTMAN, which significantly outperforms state-of-the-arts methods on the UoM-SGFSv2 set A, set B and CUFSF dataset respectively.