A Retrieval-Augmented Prompting Network for Hateful Meme Detection

Qiuhua Kuang¹Yihao Lin²Junxi Liu²Xiazhi Lai¹Runlu Zhong^3*

• ¹Guangdong University of Education, Guangzhou, Guangdong, China
• ²South China Normal University, Guangzhou, Guangdong, China
• ³Guangzhou Panyu Polytechnic, Guangzhou, China

The rise of user-generated content on social media is making memes a prevalent medium for expression. However, some memes convey offensive information toward individuals or groups on distinct aspects. Detecting such harmful content is essential to mitigate potential conflicts and harm. This paper proposes a Retrieval-Augmented Prompting Network (RAPN) for hateful meme detection. The proposed model utilizes a retrieval-augmented selector to identify semantically relevant prompting examples from diverse sources, enhancing the selection to better match the inference instances. Based on the prompting framework, the attention networks are employed to extract critical features from input instance and examples. By applying contrastive learning to label and feature spaces, the model is capable of learning more discriminative information for classification. Comprehensive evaluations on benchmark datasets demonstrate validate our model outperforms the baseline methods. Thereby, the proposed model sets strong evidence of high accuracy on the task of hateful meme classification. data-saturated social network, manually reviewing and preventing all forms of hate speech seems an impractical endeavor. Thereby, the requirement for automatic detection of hateful memes is firmly emphasized.