A Hybrid Approach for Multi-Class Multi-Label Emotion Recognition in Persian: Stacking Pre-trained LLMs with Small Transformer Models

Toktam Khatibi^1*Elham Farahani²

• ¹Tarbiat Modares University, Tehran, Iran
• ²Islamic Azad University of Arak, Arak, Markazi, Iran

Emotion recognition from text is a crucial task in natural language processing, offering diverse applications across various domains. While substantial progress has been made in English, there remains a pressing need for robust and precise models tailored to emotion analysis in Persian, a language spoken by millions globally. This study introduces a hybrid model, termed SE_LLM_ST, which leverages recent advancements in large language models (LLMs) and transformer-based architectures. SE_LLM_ST functions as a stacking ensemble, integrating pre-trained LLMs with smaller transformer models to effectively capture both contextual and sequential information vital for Persian emotion recognition.To address the imbalanced class distributions inherent in our dataset, we propose a novel training strategy that combines binary cross-entropy and Tversky loss functions. Our analysis utilizes the publicly available 'EmoPars' dataset, specifically designed for multi-class multi-label tasks, where each text can express one or more of six distinct emotions: anger, fear, happiness, hatred, sadness, and wonder. The multi-class multilabel nature of this task presents unique challenges that we aim to overcome through our proposed approach. The SE_LLM_ST model demonstrates an impressive average F1 score of 76.4% across all classes, accurately classifying instances with multiple emotions. These results underscore the efficacy of integrating pre-trained LLMs with smaller transformer models within a stacking ensemble framework for emotion analysis in Persian texts