MetaEmo: a meta-learning and culturally adaptive framework for dynamic emotional recognition in cross-cultural classrooms

Abstract

Introduction:

Cross-cultural classroom emotional recognition is challenged by significant cultural disparities in emotional expression and insufficient labeled data across diverse cultural groups, leading to poor generalization in traditional models.

Methods:

This study proposes MetaEmo, a dynamic recognition framework with three core modules. The Transformer-CME module uses cross-modal Transformer architecture to fuse speech prosody, facial micro-expressions, and body language, constructing culture-agnostic emotional representations by filtering cultural noise. The Cultural Adaptive Augmentation Engine (CAAE) employs conditional generative adversarial networks to generate synthetic samples for underrepresented cultures, while a discriminator enhances sensitivity to cultural expression differences. The MAML module optimizes model initialization via meta-training, enabling rapid adaptation to new cultural scenarios with minimal target data.

Results:

Experiments on CMU-MOSEI, MELD, and CREMA-D show MetaEmo achieves F1-scores of 87.9%, 86.3%, and 84.7%, outperforming the best baseline by 2.9%–3.4%. It reduces cross-racial accuracy variance to 8.4% and cuts training time by 60% compared to traditional models, verifying efficiency and cultural robustness.

Discussion:

MetaEmo effectively addresses cultural differences and data scarcity but has limitations in minority culture coverage. Future work will expand dataset diversity, develop dynamic cultural weighting, and deploy the framework in real-time educational monitoring systems.

1 Introduction

Driven by the wave of globalization, cross-cultural educational settings have spread at an unprecedented pace. The widespread implementation of international academic exchange programs and the normalization of transnational joint education have led to frequent interactions between students and teachers from different cultural backgrounds in classrooms (Ghafoor et al., 2025). This multicultural teaching environment not only enriches the content of education but also raises higher demands for the meticulous management of classroom teaching. Classroom emotion recognition, as a key research direction in the field of educational affective computing, utilizes advanced

artificial intelligence technologies to help teachers promptly capture students' emotional states (Ali et al., 2020; Barnes and Hutson, 2024). For example, when recognizing a student's confusion, the teacher can adjust the teaching approach accordingly. If students show signs of fatigue, the teacher can introduce interactive segments to enliven the classroom atmosphere, thus optimizing teaching strategies and improving learning outcomes (Tiantian, 2024). However, emotion recognition in cross-cultural contexts faces unique challenges, necessitating new technological solutions to break existing barriers.

Early research on classroom emotion recognition mainly relied on single-modal information, such as facial expression recognition based on convolutional neural networks (CNNs). CNNs, with their unique convolutional and pooling layer structures, can automatically extract both local and global features from images (Ghotbi, 2023; Du et al., 2025; Zhao, 2023). By training the model on a large number of labeled facial expression images, the model can learn the feature patterns corresponding to different expressions, achieving certain results in static image emotion recognition tasks. As research progressed, it became evident that single-modality models have limitations in terms of information, and multimodal fusion techniques gradually became mainstream. Researchers began combining multimodal data, including speech, facial expressions, and body movements, employing early fusion (direct concatenation of raw features) or late fusion (independently processing and then integrating classification results) (Li, 2025; Xie et al., 2023; Xiang et al., 2024). Early fusion retains the original correlations between multimodal data, while late fusion fully exploits the unique advantages of each modality. Both strategies aim to enhance emotion recognition accuracy by utilizing complementary information across modalities. For instance, variations in speech tone reflect emotional intensity, while the amplitude of body movements can help determine the type of emotion (Praveen and Alam, 2024; Wang et al., 2026; Kim and Hong, 2024). These technological innovations have significantly advanced the application of classroom emotion recognition in single-cultural settings and have provided strong support for personalized teaching.

However, when applying these mature methods to cross-cultural classrooms, researchers have observed fluctuations in model performance due to significant differences in emotional expression, social norms, and values across cultures. Cultural differences impact emotion recognition in several ways. Regarding communication styles, some cultures prefer indirect and subtle expressions, while others lean toward direct and explicit communication (Qadir et al., 2025; Baradaran et al., 2024). Furthermore, the extent to which emotions are expressed varies widely across cultures. For example, Eastern cultures, influenced by collectivism, tend to express emotions more subtly, often conveying feelings through subtle changes in tone to avoid disturbing group harmony, while Western cultures are more direct and open, with emotions typically displayed through overt facial expressions and body movements, emphasizing individual emotional expression (Geethanjali and Valarmathi, 2024; Shi et al., 2024). Additionally, constructing cross-cultural multimodal datasets presents numerous challenges, such as inconsistent data labeling standards and cultural sensitivities, which hinder data collection (Hosseini et al., 2024; Sharma et al., 2024). These issues make it difficult for existing methods to learn emotional expression patterns across different cultures, failing to meet the accuracy and generalization requirements for emotion recognition in cross-cultural educational settings.

To address the key issues in cross-cultural classroom emotion recognition, this paper proposes a cross-cultural emotion dynamic recognition framework based on meta-learning and cultural adaptation (MetaEmo). This framework consists of three synergistic stages: first, a Transformer-based cross-modal encoder is used to extract emotion representations that are culture-independent; second, a Conditional Adversarial Autoencoder (CAAE) explicitly models cultural differences, enriching the data with cultural diversity; finally, a Model-Agnostic Meta-Learning (MAML) paradigm is employed to train recurrent neural networks, enabling the model to quickly adapt to new cultural environments.

The main contributions of this paper are as follows:

• Innovative framework design: we propose the MetaEmo framework, which combines Transformer, conditional adversarial autoencoders, and meta-learning to decouple emotional content from cultural styles, providing a novel technological pathway for cross-cultural classroom emotion recognition.

• Data augmentation breakthrough: by utilizing a conditional adversarial autoencoder as a “cultural translator,” we generate emotion features in different cultural styles, effectively addressing the issue of limited labeled data and expanding the cultural diversity of the training data.

• Rapid adaptation capability: based on the MAML paradigm, the model is trained to adapt quickly to new cultural environments with only a few samples, significantly improving its generalization ability in cross-cultural scenarios.

The structure of this paper is as follows: Chapter 2 provides a review of related research in multimodal affective computing, meta-learning, and cultural adaptation; Chapter 3 elaborates on the specific architecture and working principles of MetaEmo; Chapter 4 presents experimental verification of the framework's effectiveness and a comparison with existing methods; Chapter 5 summarizes the research findings, discusses limitations, and outlines future work.

2 Review of literature

2.1 Progress in multimodal affective computing

Multimodal affective computing integrates multi-source information such as speech, facial expressions, body movements, and text to provide a more comprehensive perspective for emotion recognition, making it a research hotspot in recent years (Hu et al., 2024; Sangeetha et al., 2024). Early work, limited by technology and theory, often used shallow fusion strategies, such as constructing multi-stream neural networks where each branch network handles a different modality. For example, the speech branch extracts Mel-frequency cepstral coefficients (MFCC), and the facial expression branch focuses on key point coordinates (Kapase and Uke, 2025; Ning et al., 2026). These outputs are then weighted and summed with fixed weights to achieve initial multimodal fusion on small-scale, monocultural datasets.

With the development of deep learning, hybrid fusion strategies emerged. Some proposals suggest concatenating speech and facial expression features at the input layer to retain their original associations, then merging them at the decision layer through a gating mechanism and separately processed body movement features, which effectively improves emotion recognition accuracy (Ramaswamy and Palaniswamy, 2024; Miranda Calero et al., 2024). In the exploration based on attention mechanisms, some studies have constructed multimodal attention networks based on Transformer, utilizing multi-head attention mechanisms to dynamically allocate different modal weights according to the needs of emotion recognition tasks (Pei et al., 2024). In classroom speaking scenarios, this mechanism automatically enhances the weights of key features such as speech tone variation and facial micro-expressions, accurately capturing students' emotional states (Wang et al., 2025a). Additionally, new fusion architectures have been explored, such as hierarchical fusion models that first perform independent feature learning for each modality and then gradually integrate the information through multi-level fusion modules, uncovering complex interactions between modalities from local to global (Zhang and Leong, 2025; Al-Saadawi et al., 2024). There have also been efforts to introduce Graph Neural Networks (GNN), which represent multimodal data as graph structures and achieve collaborative learning and fusion of multimodal features through the transmission and updating of information between nodes and edges, opening up new avenues for multimodal affective computing (Pillalamarri and Shanmugam, 2025; Yang et al., 2024; Jiang et al., 2024).

However, current research in multimodal affective computing still has limitations. In terms of feature extraction, most studies rely on general feature engineering methods and lack specific emotional features for classroom teaching scenarios. Additionally, the issue of temporal alignment across different modalities remains a major challenge in multimodal data processing.

2.2 Applications of meta-learning in affective computing

Meta-learning aims to enable models to learn “how to learn,” offering great potential across various domains. In robotics, some studies have combined meta-learning with reinforcement learning to help robots quickly learn walking strategies in different terrain environments (Zhang et al., 2025; Ion et al., 2024). In the education domain, meta-learning algorithms have been used to optimize intelligent tutoring systems, tailoring personalized learning paths based on students' past learning data.

In the field of affective computing, several studies have explored the use of meta-learning. For example, Model-Agnostic Meta-Learning (MAML) has been applied to basic emotion recognition tasks, optimizing the model's initialization parameters, enabling it to achieve good performance with minimal fine-tuning on new tasks (Tian et al., 2024; Prashanth et al., 2024). However, in cross-cultural contexts, significant differences in emotional expression patterns across cultures pose challenges, such as how “happiness” is expressed through subtle smiles and slight nods in Eastern cultures, while in Western cultures, it is often displayed through broad laughter and enthusiastic body movements (Kusal et al., 2024; Chen et al., 2025; Xiao et al., 2024). Current meta-learning techniques struggle to effectively handle these cultural differences.

Additionally, other meta-learning variants have been explored in affective computing. For instance, Prototypical Networks, which have shown excellent performance in small-sample image classification tasks, have been transferred to emotion recognition tasks, classifying emotions by calculating the distance between samples and class prototypes (Wang J. et al., 2024; Tang Y. et al., 2025; Wang L. et al., 2024). Matching Networks have also been applied to affective computing, using attention mechanisms for effective matching of different emotional samples (Shen et al., 2025). However, these methods still face challenges in meeting high-accuracy emotion recognition demands in complex cross-cultural scenarios.

2.3 Development of cultural adaptation technologies

Cultural adaptation technologies have made significant progress in the fields of computer vision and natural language processing. In computer vision, cultural style transfer techniques based on Generative Adversarial Networks (GANs) have garnered attention (Jiao, 2025; Mattioli and Cabitza, 2024). These techniques train a generator and a discriminator through adversarial games to achieve style transfer of visual elements between different cultures, such as transferring Eastern ink painting styles to Western paintings (Kalateh et al., 2024; Katirai, 2024). In natural language processing, some studies have constructed large-scale cultural context knowledge bases and combined them with knowledge graph completion techniques to improve machine translation models' handling of culturally sensitive vocabulary.

In multimodal affective computing, research on cultural adaptation technologies is gradually advancing. Some studies have attempted to adjust the cultural style of facial expression images from student classroom speeches, converting exaggerated Western facial expressions to more subtle Eastern cultural expressions (Khan et al., 2024). To overcome the limitations of single-modality processing, some studies have begun to explore multimodal collaborative cultural adaptation methods. For example, a multimodal cultural style transfer framework has been proposed, which performs joint style transfer for speech, facial expressions, and body movements, achieving style consistency across modalities by introducing a shared cultural encoding space (Thirunagalingam and Whig, 2025; Joo et al., 2024; Li et al., 2024). Additionally, Conditional Generative Adversarial Networks (cGANs) have been used to generate multimodal emotional data with specific cultural styles based on cultural labels, enriching the cultural diversity of training data (Gkintoni et al., 2025; Wang R. et al., 2024; Tang X. et al., 2025). However, there is still room for improvement in maintaining emotional-semantic consistency in these methods.

In conclusion, while multimodal affective computing, meta-learning, and cultural adaptation technologies have made significant progress in their respective fields, there is no effective framework yet that integrates multimodal information processing, deeply models cultural differences, and achieves rapid adaptive learning for cross-cultural classroom emotion recognition. The MetaEmo framework proposed in this paper innovatively combines Transformer, conditional adversarial autoencoders, and meta-learning techniques to decouple emotional content from cultural styles, providing a new technological path and solution for addressing the challenges of cross-cultural classroom emotion recognition.

3 Method

3.1 Overall architecture and collaborative mechanism of the MetaEmo framework

The MetaEmo framework is designed for cross-cultural classroom emotion dynamic recognition, consisting of a three-tier technical system, which includes multimodal feature extraction, cultural difference modeling and data augmentation, and rapid adaptive recognition training. The architecture is shown in Figure 1.

Figure 1

At the initial stage of data processing, the Transformer-based cross-modal encoder receives multimodal data such as speech, facial expressions, and body movements. This encoder processes each modality in parallel through a multi-head attention mechanism, dynamically capturing key emotional features such as speech tone variation and facial micro-expressions. These features are then encoded into culture-independent common representations, effectively stripping away the cultural-specific interference in emotional expression. The emotion common representations, after cross-modal encoding, are fed into the Conditional Adversarial Autoencoder (CAAE). The generator of the CAAE uses cultural labels as conditions to perform style transfer on the input representations, generating emotion feature data that covers diverse cultural styles. The discriminator, through adversarial training, enhances its ability to distinguish between real and generated data, thereby expanding the cultural diversity of the training data and improving the model's sensitivity to cultural differences. Finally, the feature data processed by CAAE is input into a recurrent neural network (RNN) trained using the Model-Agnostic Meta-Learning (MAML) paradigm. MAML optimizes the model's initialization parameters, granting the network the ability to adapt quickly. When faced with a new cultural environment, the RNN requires only a few samples to fine-tune its parameters and achieve high-precision emotion recognition tasks.

The core modules interact through data and parameter exchange, forming a closed-loop system: the common representations output by the cross-modal encoder provide the foundation for processing in CAAE; the augmented data generated by CAAE supports the training optimization of MAML; and the parameter optimization results from MAML feed back into the previous modules, enhancing the overall framework's effectiveness in handling cross-cultural emotion recognition tasks.

3.2 Transformer-based cross-modal feature extraction module

The Transformer-based cross-modal feature extraction module serves as the primary component of the MetaEmo model for processing multimodal data. Its core objective is to accurately extract emotion common representations free from cultural interference from diverse sources of information, such as speech, facial expressions, and body movements, laying a solid foundation for subsequent emotion recognition and analysis. The module receives multimodal data X = [X_voice, X_face, X_body], where X_voice carries emotional information like speech intonation and speed variations, X_face contains details of facial micro-expressions such as eyebrow furrowing and mouth curvature, and X_body records body language features including limb postures and hand gestures.

In the data preprocessing stage, to transform various forms of multimodal data into a unified format suitable for the Transformer architecture, learnable weight matrices W^Q, W^K, and W^V are introduced. Considering the different contributions of speech, facial expressions, and body movements to emotion conveyance in various classroom scenarios and emotional expression processes, initial importance weights λ_voice, λ_face, and λ_body are further incorporated. The mathematical formulation of this module is presented in Equations 1–8. The query matrix Q, key matrix K, and value matrix V are calculated through weighted summation:

For example, in a classroom interaction scenario, when a student is speaking, the speech modality may be crucial for expressing confidence or nervousness. Assigning a larger weight to λ_voice enables the model to focus on speech feature extraction in subsequent calculations. In a group discussion scenario, facial expressions and body movements can more intuitively reflect students' participation and emotional states, so λ_face and λ_body are increased accordingly. This differential weight setting not only achieves the transformation from raw data to feature representation but also provides more targeted and personalized inputs for subsequent attention calculations, enhancing the model's adaptability to complex classroom environments.

When entering the multi-head attention mechanism calculation stage, based on the traditional scaled dot-product attention calculation method and combined with the research results of cross-cultural emotion expression, a cultural bias vector B_c is introduced. There are significant differences in emotion expression methods and emphases across different cultural backgrounds. For instance, in Eastern cultures, emotion expression is often subtle and reserved, with small body movements, and emotions are mainly conveyed through subtle changes in speech intonation and eye contact. In contrast, Western cultures tend to have more direct and expressive ways of expression, often revealing emotions through rich facial expressions and large body movements. The attention calculation formula considering the cultural bias vector is:

This design allows the model to capture the key emotional features of each modal data while fully considering the impact of cultural factors on attention allocation, avoiding feature extraction biases caused by cultural differences and enhancing the model's adaptability and accuracy in cross-cultural scenarios.

To comprehensively capture emotional features from multiple dimensions, the model adopts a multi-head attention architecture. Meanwhile, to optimize the fusion effect of the outputs of each attention head, an adaptive scaling factor γ_i is introduced. This factor can dynamically adjust the contribution of the output of each attention head according to the data feedback and task requirements during the training process. The calculation of the multi-head attention output is as follows:

where h is the number of attention heads. This design enables the model to flexibly combine the features extracted by different attention heads when processing complex emotional features, improving its representation ability for diverse emotion expression patterns. For example, some attention heads are good at capturing subtle changes in facial expressions, while others are more sensitive to the emotional rhythm in speech. Through the dynamic adjustment of the adaptive scaling factor, the model can reasonably balance the roles of each attention head to achieve more accurate emotional feature extraction.

Considering the sequential information in multimodal data and the interaction relationships between modalities, a position encoding PE and a modality interaction matrix M_int are introduced. The position encoding PE provides the model with the sequential information of the data, helping the model understand the temporal or spatial arrangement of the data. The modality interaction matrix M_int is used to model the interactions between different modalities. The input of the feed-forward neural network (FFN) is obtained by adding the output of the multi-head attention and the position encoding adjusted by modality interaction:

Subsequently, this input is fed into the feed-forward neural network (FFN) for in-depth processing. The FFN further extracts and enhances features through three layers of non-linear transformations and the ReLU activation function:

where W₁, W₂, and W₃ are the weight matrices of the feed-forward neural network, and b₁, b₂, and b₃ are the bias terms. These three layers of non-linear transformations explore the deep relationships between features, transforming low-level feature representations into more abstract and representative high-level features, enhancing the model's ability to understand and express complex emotional information.

Finally, to stabilize the model training process and improve the model's generalization ability, the output of the FFN is subjected to a dual normalization process. First, layer normalization (LN) is performed, which normalizes all features of each sample, alleviating the problems of gradient vanishing and explosion and accelerating model convergence. Then, instance normalization (IN) is combined, which normalizes a single feature of a single sample, reducing stylistic effects and enhancing the model's robustness in handling data with different styles and distributions. After the dual normalization process, the culture-independent emotion common representation is obtained:

Through the above series of computational steps, the Transformer-based cross-modal feature extraction module effectively eliminates the interference of cultural specificity on emotion expression, extracts emotion common features with high representativeness and discrimination, provides a solid feature foundation for subsequent cultural difference modeling and emotion recognition tasks, and supports the excellent performance of the MetaEmo model in cross-cultural classroom emotion dynamic recognition.

3.3 CAAE-based cultural difference modeling module

The Conditional Adversarial Autoencoder (CAAE)-based cultural difference modeling module is a key component of the MetaEmo model for achieving cross-cultural emotion recognition. This module takes the culture-independent emotional common representation output by the cross-modal feature extraction module as the foundation, and constructs emotional expression models under different cultural styles through the adversarial learning mechanism between the generator and discriminator, thereby enhancing the cultural diversity of training data and enabling the model to accurately perceive cultural differences.

At the initial stage of module operation, it receives the emotional common representation Z output by the cross-modal feature extraction module, while introducing cultural labels C. The cultural labels C represent different cultural categories in the form of One-Hot Encoding. The core task of the generator G is to generate emotional features with specific cultural styles based on the input emotional common representation Z and cultural labels C. The mathematical formulation of this module is presented in Equations 9–13. Its mapping relationship can be expressed as:

This formula indicates that the generator converts culture-independent common representations into emotional features conforming to specific cultural styles by learning the characteristics of emotional expression in different cultures. For example, when the cultural label C represents Eastern culture, the generator will make the output emotional features reflect implicit and restrained expression ways; if C represents Western culture, will present enthusiastic and explicit styles.

A critical design objective of the CAAE module is to guarantee emotional-semantic consistency during cross-cultural style transfer. The module takes the culture-independent emotional representation Z as the core input, which inherently retains pure emotional semantics and eliminates cultural expression interference, serving as a fundamental constraint to preserve the core emotional category throughout the generation process. Furthermore, the module adopts dual conditional guidance (emotion label E + cultural label C) for generation, where the emotion label E locks the target emotional category to avoid semantic drift. To strengthen this constraint, an additional emotional classification loss L_cls is integrated into the training framework, which enforces the generated samples to be accurately classified into the original emotional category. This design ensures that when converting explicit Western emotional expressions to subtle Eastern styles, the core emotional semantics remain intact and undiluted.

The role of the discriminator D is to distinguish between real multimodal emotional data X and simulated data generated by the generator. Beyond distinguishing real and generated data, the discriminator is optimized to jointly verify two critical dimensions: whether the sample conforms to the target cultural style, and whether the sample matches the predefined emotional semantics. To enable the discriminator to have strong discrimination ability and guide the generator to generate more realistic data, an adversarial training objective function is introduced. The optimization goal of the discriminator is to maximize the judgment probability of real data and the misjudgment probability of generated data, while the generator tries to minimize the probability that the discriminator correctly identifies the generated data. The adversarial game process between them can be described by the following formula:

In the formula, E_x~pdata(x)[logD(x)] represents the expected judgment probability of the discriminator on samples x in the real data distribution p_data(x), aiming to enable the discriminator to identify real data as accurately as possible; E_{z~pz(z), c~pc(c)}[log(1−D(G(z, c)))] is the expected judgment probability of the data generated by the generator at the discriminator. By minimizing this term, the generator is promoted to generate samples closer to the real data distribution.

In the actual training process, to further stabilize the adversarial training process and avoid problems such as gradient vanishing or mode collapse, a gradient penalty term GP is introduced. The calculation of the gradient penalty term is based on the gradient of the discriminator output with respect to the input data, ensuring that the decision boundary of the discriminator is smooth and making the training process more stable. The calculation formula of the gradient penalty term is:

Where λ is the gradient penalty coefficient used to adjust the penalty intensity; represents the expectation over interpolated data , which is obtained by linear interpolation between real data x and generated data ; is the gradient of the discriminator output with respect to , and ·₂ denotes the 2-norm. By adding the gradient penalty term to the adversarial training objective function, the improved objective function is obtained:

During the iteration of adversarial training, the generator continuously optimizes its own parameters, making the generated emotional features with different cultural styles more and more close to the real data distribution. To quantitatively evaluate the difference between the generated data and the real data distribution, the Wasserstein distance W(p_data, p_gen) is introduced, and its calculation formula is:

Where Π(p_data, p_gen) represents the set of all possible joint distributions between the real data distribution p_data and the generated data distribution p_gen, γ is a sample in the joint distribution, and denotes the distance between real data x and generated data . The Wasserstein distance is employed to evaluate two critical metrics simultaneously: the distribution similarity of cross-cultural styles and the consistency of emotional feature distributions between generated and real samples. The smaller the Wasserstein distance, the closer the generated data distribution is to the real data distribution, that is, the better the generation effect of the generator.

After multiple rounds of adversarial training, when the data generated by the generator reaches a high similarity with the real data in distribution and the discriminator is difficult to accurately distinguish between them, the module outputs emotional feature data containing rich cultural styles. These data not only expand the cultural diversity of training data but also enable the model to accurately perceive the differences in emotional expression under different cultural backgrounds, while strictly preserving the integrity of emotional semantics, providing high-quality data support for the subsequent meta-learning-based fast adaptive recognition training, thereby improving the performance of the entire MetaEmo model in cross-cultural classroom emotion recognition tasks.

3.4 MAML-based fast adaptive recognition module

The Model-Agnostic Meta-Learning (MAML)-based fast adaptive recognition module is a core component of the MetaEmo model for achieving rapid adaptation in cross-cultural scenarios. Its primary objective is to optimize the model's initial parameters so that when facing new cultural environments, the model can complete parameter fine-tuning with only a small number of labeled samples, achieving high-precision emotion recognition. This module receives emotional feature data processed by the cultural difference modeling module and completes model training and fast adaptation through the meta-learning framework, which is particularly suitable for classroom scenarios with diverse cultural backgrounds and limited labeled data.

In the meta-learning training phase, a series of cross-cultural emotion recognition tasks are first defined to form a task set T = {T₁, T₂, ⋯ , T_N}. Each task T_i corresponds to an emotion recognition scenario in a specific cultural background, containing a support set (for parameter fine-tuning) and a query set (for evaluating generalization performance). The initial parameters of the model are θ. For each task T_i, the gradient of the loss function is calculated using the support set data, and task-specific parameters are obtained through gradient descent update. The mathematical formulation of this module is presented in Equations 14–19. The update process can be expressed as:

where α is the inner-loop learning rate, controlling the magnitude of a single parameter update; L_Ti is the loss function of task T_i (cross-entropy loss is adopted to adapt to classification tasks); f_θ represents a recurrent neural network model with θ as parameters, used to capture the temporal dependencies of emotional features.

The core of meta-learning lies in optimizing the initial parameters θ so that the updated task-specific parameters perform optimally on the query set of the corresponding task. Therefore, the outer-loop optimization objective of MAML is to minimize the total loss of all tasks on their query sets:

Through iterative optimization of this objective function using stochastic gradient descent, the initial parameters θ of the model are endowed with good generalization ability—they not only retain the common emotional expression rules across different cultures but also can quickly adapt to the unique patterns of specific cultures, laying a foundation for the rapid adaptation of subsequent new tasks.

When the model faces a new cultural environment (i.e., a new task T_new, such as a cultural background not involved in training), a small number of support samples in the new task (usually only 5–10 labeled samples) are used to fine-tune the initial parameters θ, obtaining parameters adapted to the new task:

The fine-tuning process only requires 1–3 gradient updates to enable the model to quickly adapt to new cultural expression patterns, such as adjusting the sensitivity to specific facial expressions or speech intonations.

To further improve the recognition accuracy of the model on new tasks, a dynamic learning rate adjustment mechanism is introduced to adaptively adjust the learning rate α_new according to the distribution difference between the new task and the training task set:

where β is a adjustment coefficient (set to 0.5 in experiments), and uses KL divergence to measure the difference between the new task T_new and the average distribution of the training task set. The greater the distribution difference, the smaller the learning rate α_new, avoiding overfitting caused by excessive parameter update amplitude; otherwise, a larger learning rate is maintained to accelerate convergence.

After completing parameter fine-tuning, the model performs emotion recognition on the query set of the new task and outputs the prediction result ŷ_new:

where X_new is the input emotional feature in the new cultural environment (such as a sequence of facial expressions of a student). To obtain interpretable probability outputs, the prediction results are converted into an emotion category probability distribution through the softmax function:

where C is the total number of emotion categories (such as six categories including happiness, confusion, and boredom), and represents the output score of the model for the k-th emotion category.

Through the above process, the model can quickly converge to the optimal state in new cultural environments and maintain high recognition accuracy even with scarce samples, effectively solving the problems of large data distribution differences and insufficient labeled samples in cross-cultural scenarios, and providing key support for the stable application of the MetaEmo model in diverse classroom environments.

4 Experiment

4.1 Dataset

4.1.1 Dataset source and overview

In this study, we use three publicly available datasets: CMU-MOSEI, MELD, and CREMA-D, providing data support for the training and evaluation of a cross-cultural classroom multimodal emotion recognition model from multiple dimensions.

CMU-Multimodal Opinion Sentiment and Emotion Intensity (MOSEI) is a classic dataset in the field of multimodal affective computing. The original videos were collected from a wide variety of YouTube comment videos. These videos feature creators from around the world, covering different age groups, genders, and professions, which endows the dataset with significant cultural diversity. The dataset contains approximately 16,000 video clips, with a total duration exceeding 1,000 h. The large data scale provides ample samples for model training, sufficiently meeting the data volume requirements of complex models.

Multimodal EmotionLines Dataset (MELD) is derived from the iconic TV show Friends, focusing on dialogue scenes. As a globally popular TV show, Friends simulates real-life interactions and conversations, resembling classroom environments where teachers and students, or students with each other, engage in communication. The dataset covers 1,433 dialogues, spanning 10 rich emotional categories, including basic emotions like happiness, sadness, anger, fear, as well as emotions like surprise, disgust, and confusion, which are commonly seen in classroom interactions. It provides high-quality and practical data for studying emotion recognition in dialogue contexts.

Crowd-Sourced Emotional Multimodal Actors Dataset (CREMA-D) involves actors from six countries– the United States, India, the United Kingdom, Canada, Italy, and Greece. These countries belong to different cultural circles, and significant differences exist in language habits and emotional expression styles, providing the dataset with a strong cross-cultural attribute. The data primarily consists of speech and video modalities, with professional actors performing six basic emotions (anger, disgust, fear, joy, sadness, and surprise), resulting in a total of 912 samples. While the sample size is relatively small, CREMA-D offers strong cultural coverage, and its standardized labeling allows for effective validation of model adaptability in handling emotional expression differences across cultures, especially in small-sample learning scenarios.

4.1.2 Data modalities and emotion annotations

In terms of data modalities, both CMU-MOSEI and MELD contain text, speech, and video modalities. The text modality consists of transcribed video content or dialogue lines, providing rich semantic cues crucial for emotion recognition. The speech modality captures the speaker's audio information, including prosodic features like tone, speech rate, and volume, which often provide a direct reflection of the speaker's emotional state. The video modality retains facial expressions and body movements, with facial micro-expressions and body posture being important non-verbal means of emotional expression. The CREMA-D dataset primarily contains speech and video modalities, presenting different emotions through the dynamic changes in speech tone and facial expressions performed by the actors.

Regarding emotion annotation, CMU-MOSEI not only labels emotional polarity (positive, negative, neutral) but also provides emotion intensity scores ranging from −3 to 3. This fine-grained annotation helps the model learn the degree of emotional expression. MELD labels each dialogue turn with 10 emotion categories and considers the dynamic changes in emotions across multiple dialogue rounds, providing the potential to study emotion evolution in dialogue scenarios. CREMA-D uses a simple and universal annotation system with six basic emotions, which aids in validating the model's ability to recognize basic emotion categories, while reflecting cultural differences in emotional expression through actors from various cultural backgrounds. The three datasets provide rich, multi-dimensional emotion label information for the model from different perspectives.

4.1.3 Data preprocessing

In the data preprocessing phase, each modality is processed separately to transform raw data into formats suitable for model input.

Audio data preprocessing: professional audio processing algorithms are used to reduce noise, eliminating environmental noise and electrical interference during recording, significantly improving audio quality. Then, the audio is divided into frames of 30 milliseconds, as speech signals tend to have stable characteristics over short periods. This framing allows the model to effectively capture short-term features of the speech signal. Mel-frequency cepstral coefficients (MFCCs) are extracted, with 13 MFCC features per audio segment, and the first and second order differences are computed, resulting in 39-dimensional feature vectors that comprehensively describe the spectral characteristics and dynamic changes of the speech signal. Finally, the feature vectors are normalized to zero mean and unit variance, eliminating scale differences between different audio samples, thereby improving training efficiency and stability.

Video data preprocessing: key frames are extracted from the video at a rate of 10 frames per second. This method preserves the main visual information of the video while reducing data redundancy and processing load. An advanced face detection algorithm is used to locate the facial region in the video and crop it precisely. The cropped facial images are resized to a uniform size, ensuring consistency across images. To further extract image features, normalization is applied to adjust pixel value distributions, making them fit the appropriate range for subsequent processing. For videos with varying lengths of frame sequences, padding and truncation operations are used to standardize all sequences to a fixed length, meeting model input requirements.

Text data preprocessing: text data is first processed with detailed tokenization, and stop words (e.g., common function words and pronouns) that contribute little to emotion expression are removed, leaving key semantic information. The words are then converted into numerical vectors for model input. Text sequences of varying lengths are padded and truncated to ensure they all have a fixed length. Finally, text features are normalized by mapping their values into a suitable range, balancing the feature scales of different text samples, making it easier for the model to learn the semantic and emotional information from the text.

After these preprocessing steps, the different modalities of data across the three datasets are unified in format and dimension, laying a solid foundation for the subsequent training and testing of the model.

4.2 Experimental setup

4.2.1 Model training parameter configuration

To ensure the effectiveness of the experiment, a rigorous and optimized parameter configuration was adopted during the model training process. All models used the Adam optimizer for parameter updates, with an initial learning rate set to 0.001. A learning rate decay strategy was implemented: the learning rate was multiplied by 0.9 every 10 training epochs. This balance ensured rapid convergence in the early stages of training while enabling fine-tuning in the later stages to prevent overfitting.

The batch size was set to 32, which achieved a good balance between utilizing memory resources and maintaining training stability. This configuration allowed the model to process a sufficient number of data samples during training while avoiding memory overflow or training instability caused by excessively large batches.

For the meta-learning-based fast adaptive recognition module, a hierarchical learning rate mechanism was adopted: the inner-loop learning rate was set to 0.01 to enable the model to quickly adapt to data features during fine-tuning for a single task. The outer-loop learning rate was set to 0.001 to optimize the model's initial parameters across multiple tasks, realizing effective updates of global parameters and ensuring the model exhibited good generalization ability across tasks with different cultural backgrounds.

In the adversarial training of the cultural difference modeling module, the training iteration ratio of the generator to the discriminator was set to 1:1 to ensure synchronous optimization of both components. Meanwhile, the gradient penalty coefficient was set to 10. By constraining the gradient norm of the discriminator, the stability of adversarial training was maintained, and the generator was prompted to generate more realistic emotional feature data that conformed to different cultural styles.

4.2.2 Experimental hardware and software environment configuration

This experiment was conducted based on a high-performance hardware and software environment to ensure the efficiency and accuracy of model training and testing.

In terms of hardware, an NVIDIA RTX 3090 GPU was used as the core computing device. Equipped with 24 GB of large memory, it supported parallel computing and storage of large-scale multimodal data, significantly accelerating the model training speed. An Intel Xeon Gold 6248R CPU (2.40 GHz) was paired with the GPU, featuring powerful multi-core processing capabilities to quickly complete computational tasks such as data preprocessing and feature extraction. Additionally, 128 GB of memory was installed to ensure smooth data reading and transmission, preventing training interruptions caused by insufficient memory.

For the software environment, the model was built based on the PyTorch 1.12 (Meta Platforms, Inc., Menlo Park, CA, USA) deep learning framework. This framework provided flexible tensor computing and automatic differentiation functions, offering great convenience for model construction, training, and debugging. The Ubuntu 20.04 LTS operating system was selected for its stable kernel and rich open-source ecosystem, providing a reliable runtime environment for deep learning experiments.

Furthermore, the OpenCV 4.5.5 library was used for preprocessing operations such as key frame extraction and face detection for video data. The Librosa 0.9.1 library was utilized to perform noise reduction, framing, and MFCC feature extraction for audio data. The NLTK 3.6.2 library was employed to implement text preprocessing tasks such as word segmentation and stopword removal. These software tools worked collaboratively to build a complete and efficient system for multimodal data processing and model training.

4.2.3 Model training process

Based on the three public datasets (CMU-MOSEI, MELD, and CREMA-D), this study optimized the model using a “phased training + dynamic adjustment” strategy. The training process was divided into three main stages: the basic capability pre-training stage, the cross-cultural adaptive training stage, and the joint fine-tuning stage.

• Basic capability pre-training stage

  In this stage, the CMU-MOSEI and MELD datasets were used to initialize and train the feature extractors for three unimodal data types (speech, video, and text). CMU-MOSEI had a large data scale and detailed emotional annotations, while MELD focused on conversational scenarios, which were highly relevant to classroom interactions. The combination of these two datasets enabled the model to learn rich emotional expression patterns. During training, the bottom-layer parameters of each unimodal feature extractor (e.g., the speech Mel spectrum extraction layer and the first five layers of the video CNN backbone network) were fixed, and only the top-layer fully connected layers were fine-tuned. The training objective was a “general emotional classification task” (with classification labels including happiness, anger, sadness, fear, and neutrality). This stage involved 100 training epochs and stopped when the validation set accuracy showed no improvement for 10 consecutive epochs. The goal was to enable each modal feature extractor to acquire basic emotional feature capture capabilities, avoiding convergence difficulties caused by random parameter initialization in subsequent cross-cultural training.

• Cross-cultural adaptive training stage

  After pre-training, the CREMA-D dataset was introduced to enter the cross-cultural adaptive training stage, with a focus on activating the model's cultural difference modeling module. First, a “cross-cultural task pool” was constructed, which included data from tasks corresponding to the three datasets with different cultural backgrounds. For each task, 64 samples were randomly sampled from the corresponding dataset (ensuring a balanced number of samples for each emotional category). The meta-learning logic of “task sampling → inner-loop fine-tuning → outer-loop update” was adopted. In each training iteration, two tasks with different cultural backgrounds were randomly selected from the task pool (e.g., Task 1 for emotional classification using MELD data and Task 2 for emotional classification using CREMA-D data from American actors). In the inner loop, the pre-trained model was used as the initial parameter, and the training set of Task 1 was used to fine-tune the cultural adaptive module (with a learning rate of 0.01). The loss was calculated on the validation set of Task 1. In the outer loop, based on the loss from the inner loop, gradient descent was used to update the model's global parameters (excluding the bottom-layer parameters of the unimodal feature extractors). At the same time, adversarial training was initiated: the generator generated “culture-adapted features” based on the cultural labels of the current task, and the discriminator distinguished between “real cultural features” and “generated features.” Both components were optimized iteratively. This stage involved 200 training epochs, with the “average validation F1-score of the cross-cultural task pool” used as the monitoring metric. Training stopped when the improvement of this metric was less than 0.005 for 15 consecutive epochs, ensuring the model could effectively adapt to differences in emotional expressions across different cultural scenarios.

• Joint fine-tuning stage

  Finally, the joint fine-tuning stage was entered, where all modules were integrated for end-to-end training. The remaining data from the three datasets were mixed, and multimodal features (speech, video, and text) were input allocated weights to each modal through the attention fusion layer, adjusted the feature distribution using the “cultural offset coefficient” output by the cultural adaptive module, and finally output the emotional category through the classification head. In this stage, the fixed restriction on the bottom-layer parameters of the unimodal feature extractors was lifted, but their learning rate was set to 0.0001 to avoid drastic parameter changes. The learning rates of other modules remained the same as in previous stages. The training objective was “cross-cultural classroom emotional classification,” with a total of 150 training epochs. An “early stopping strategy” (with a patience value of 12) was adopted during training, and model checkpoints were saved every 20 epochs. The final model was selected as the checkpoint with the highest “average test F1-score of the cross-cultural task pool,” ensuring the model exhibited good emotional recognition accuracy and generalization ability across different cultural scenarios.

4.3 Evaluation Metrics

This study selected three evaluation metrics: Basic classification performance was evaluated using accuracy, precision, recall, and F1 score, using macro- and micro-averaging to comprehensively assess the model's recognition capabilities across emotion categories and across the entire sample. Cross-cultural adaptability was measured using cross-cultural generalization error (the standard deviation of performance across different cultural datasets) and adaptation speed (the rate of increase in performance after small-sample fine-tuning), reflecting the model's ability to cope with cultural differences. Model efficiency was evaluated by recording training time and inference time to assess the model's training complexity and real-time performance in practical applications.

5 Experimental Results

In the experiments, we selected a variety of representative models as baselines to comprehensively evaluate the performance of the proposed MetaEmo framework. These comparative models include: the traditional multi-modal fusion models by (Maji et al. 2023) and (Liu et al. 2024), which adopt basic feature concatenation to handle cross-modal information; HuBERT-CLAP (Nguyen et al., 2024) based on pre-trained speech models, which leverages large-scale speech data pre-training to enhance representation capabilities; CLAMP (He, 2025) using contrastive learning strategies, aiming to strengthen feature discriminability; as well as recently proposed models such as MGCMA (Xie et al., 2025), DCLF (Wang et al., 2025b), CAMEL (Zhang et al., 2024), and DualGATs (Zhang et al., 2023), which optimize multi-modal interaction through different attention mechanisms or network architectures. Comparing with these models enables a multi-dimensional validation of MetaEmo's advantages in cross-cultural emotion recognition tasks.

5.1 Performance comparison of optimization methods

As shown in the Table 1, the MetaEmo model demonstrates significant advantages on the three cross-cultural emotion recognition datasets: CMU-MOSEI, MELD, and CREMA-D. On the CMU-MOSEI dataset, MetaEmo achieves an accuracy of 89.2% and an F1 score of 87.9%, outperforming other models by 3.1 and 2.9 percentage points, respectively, compared to the next best model, MGCMA (86.1% accuracy, 85.0% F1 score). This improvement is attributed to the Transformer-CME module, which enables deep fusion of multimodal features such as speech and facial expressions, effectively eliminating cultural-specific interference. On the MELD dataset (cross-cultural dialogue scenario), MetaEmo achieves an accuracy of 87.5% and an F1 score of 86.3%, surpassing MGCMA (84.2%, 83.4%) by 3.3 and 2.9 percentage points. This highlights the advantage of the CAAE module in handling cultural differences—by generating data with diverse cultural styles, it enhances the model's adaptability to different cultural expression habits. On the CREMA-D dataset, which primarily consists of multi-ethnic data, MetaEmo's accuracy and F1 score are 85.9 and 84.7%, respectively, outperforming MGCMA (82.5%, 81.7%) by 3.4 and 3.0 percentage points. This further validates the value of the MAML module: its rapid adaptation ability allows the model to achieve precise recognition with minimal data fine-tuning when faced with new ethnic cultural samples. In contrast, models like HuBERT-CLAP, which lack cultural adaptation mechanisms, perform significantly worse on CREMA-D, achieving only 75.5% accuracy, far below MetaEmo. Furthermore, MetaEmo demonstrates balanced performance in both precision and recall across all datasets (e.g., on CMU-MOSEI, precision of 83.5% and recall of 92.8%), indicating not only high recognition accuracy but also strong capability in capturing minority class emotions. This is closely linked to its multi-module collaborative design (dynamic attention, cultural enhancement, and rapid adaptation), providing a reliable solution for dynamic emotion recognition in cross-cultural classroom environments.

Table 2 presents a comparison of efficiency-related metrics for different models across multiple datasets, clearly demonstrating MetaEmo's significant efficiency advantages in terms of training time, single-sample inference time, and GPU memory and CPU memory usage. On the CMU-MOSEI dataset, MetaEmo has a training time of just 15.8 min, which is far lower than HuBERT-CLAP's 81.8 min. The single-sample inference time for MetaEmo is 15.7 ms, significantly lower than HuBERT-CLAP's 98.2 ms. Additionally, MetaEmo's GPU memory usage during training is only 4.9GB, with CPU memory usage for inference at 262MB. In comparison, other models, such as CLAMP (training GPU memory usage of 11.2GB, inference memory usage of 532MB), consume significantly more resources. On the IEMOCAP dataset, MetaEmo's training time is 19.3 min, with an inference time of 18.2 milliseconds, both of which are notably superior to other models. For instance, HuBERT-CLAP takes 92.5 min for training and 105.6 milliseconds for inference. MetaEmo also uses only 5.6GB of GPU memory during training and 298MB of CPU memory during inference, showing a clear advantage in resource utilization. On the RAVDESS dataset, MetaEmo has a training time of 17.5 min and an inference time of 16.9 ms, with GPU memory usage during training at 5.2GB and inference memory usage at 281MB. In comparison, Maji et al. requires 31.6 min for training, 30.4 ms for inference, with GPU memory usage of 9.1GB and CPU memory usage of 447MB. This highlights MetaEmo's significant improvements in efficiency. The outstanding efficiency of MetaEmo is attributed to the targeted elimination of redundant computations in its architecture: firstly, the Transformer-CME cross-modal module adopts dynamic modal adaptive weighting, which selectively activates key feature channels according to classroom scenarios and discards invalid modal feature calculations; secondly, the CAAE module uses a 1:1 generator-discriminator iteration ratio with gradient penalty, reducing repeated updates caused by mode collapse; thirdly, the MAML meta-learning module only optimizes initial parameters and performs few-step fine-tuning for new cultural tasks, avoiding full-parameter retraining; finally, the dual normalization operation unifies feature distribution in one step, removing redundant post-processing computations. MetaEmo excels in efficiency due to its lightweight model architecture and efficient multimodal feature fusion strategy, which reduces redundant computations. As a result, it achieves a substantial reduction in training and inference time while minimizing hardware resource consumption. This makes MetaEmo well-suited for real-world applications, such as efficient emotion recognition in cross-cultural classroom scenarios.

5.2 CMU-MOSEI dataset: fine-grained emotion recognition performance and analysis

As shown in the Table 3, the MetaEmo model demonstrates significant advantages across various emotional dimensions and the average F1 score in the multi-dimensional emotion recognition task on the CMU-MOSEI dataset.In the Valence dimension, MetaEmo achieves an accuracy of 90.5% and an F1 score of 89.1%, outperforming the next best model, MGCMA (87.0%, 85.6%), by 3.5 percentage points in both accuracy and F1 score. This indicates MetaEmo's superior ability to judge the emotional valence, both positive and negative. In the Arousal dimension, MetaEmo's F1 score is 86.3%, leading MGCMA (83.9%) by 2.4 percentage points, showing its high sensitivity to emotional activation intensity (such as calm to excited states). For the Dominance dimension, MetaEmo achieves an F1 score of 87.5%, surpassing all comparison models, demonstrating its effectiveness in recognizing the sense of control and agency in emotions (such as confidence vs. submission). In discrete emotional categories, MetaEmo excels in Happiness (88.8% F1 score) and Anger (89.7% F1 score), significantly outperforming models like HuBERT-CLAP (76.5 and 79.0%), indicating its stronger ability to capture high-arousal emotions. The Sadness dimension, with an F1 score of 86.9%, further confirms its stability in recognizing low-arousal negative emotions. Compared to other models, MGCMA shows relatively balanced performance across dimensions but still lags behind MetaEmo, especially in the Anger dimension, where its F1 score of 86.2% is 3.5 percentage points lower than MetaEmo's. CLAMP performs well in the Valence and Anger dimensions but falls short in Arousal and Dominance, with F1 scores of 83.4 and 84.0%, respectively, both lower than MetaEmo. HuBERT-CLAP, lacking cultural adaptation mechanisms, generally shows F1 scores below 80% across dimensions, with only 75.1% in the Sadness dimension, demonstrating its limited ability to recognize subtle emotional variations in cross-cultural contexts.MetaEmo, through the Transformer-CME module's multimodal feature fusion, CAAE's cultural difference enhancement, and MAML's rapid adaptation, achieves precise recognition across different emotional dimensions and categories, particularly excelling in dynamic emotion capture in cross-cultural scenarios, significantly outperforming traditional models.

Table 4 shows the performance comparison of various models in different intensity ranges of three emotional dimensions–Valence, Arousal, and Dominance–on the CMU-MOSEI dataset. In the Valence dimension, MetaEmo achieves accuracy rates of 87.9%, 90.3%, and 89.7% in the low, medium, and high-intensity ranges, respectively, outperforming the next best model, MGCMA (86.1%, 87.2%, 86.9%) by 1.8, 3.1, and 2.8 percentage points. This advantage is particularly significant in the medium intensity range (35.2% of the samples), indicating that MetaEmo excels at recognizing emotions of moderate intensity in everyday conversations.In the Arousal dimension, MetaEmo shows an accuracy of 85.6% in the low-intensity range, leading HuBERT-CLAP (75.3%) by 10.3 percentage points, reflecting its stronger ability to capture calm emotions (such as boredom or indifference). In the medium and high-intensity ranges (scores 4–10), MetaEmo achieves an accuracy of over 86%, improving by 2.1 percentage points over MGCMA in the high-intensity range (84.7%), demonstrating more stable recognition of high-arousal emotions like excitement and anger. In the Dominance dimension, MetaEmo achieves an accuracy of 84.5% in the low-intensity range (0–3, such as submission and hesitation), outperforming Maji et al. (80.1%) by 4.4 percentage points. In the medium and high-intensity ranges (4–10, such as confidence and control), its accuracy exceeds 85%, demonstrating high sensitivity to the differences in emotional agency.Compared to other models, MGCMA shows relatively balanced performance across intensity ranges but still falls behind MetaEmo, especially in the Valence dimension, where its accuracy in the medium-intensity range (87.2%) is 3.1 percentage points lower than MetaEmo's. CLAMP performs better in the high-intensity ranges of Valence and Arousal, but in the Dominance high-intensity range, its accuracy of 83.7% is still lower than MetaEmo's 85.9%. HuBERT-CLAP, lacking multimodal fusion and cultural adaptation mechanisms, shows accuracy below 80% in most intensity ranges, with just 75.3% accuracy in the low-intensity Arousal range, highlighting its insufficient ability to recognize subtle changes in emotional intensity in cross-cultural contexts. MetaEmo, through the dynamic weighting of multimodal features in the Transformer-CME module, cultural intensity expression modeling in the CAAE module, and rapid adaptation to new intensity samples in the MAML module, achieves precise recognition across different intensity ranges of emotional dimensions. Its advantage is particularly evident in the recognition of medium and high-intensity emotions (which are more influenced by cultural expression styles), providing an effective solution for real-time monitoring of dynamic emotional intensity in cross-cultural classrooms.

5.3 MELD dataset: dynamic emotion tracking in conversational scenarios

Table 5 shows the performance comparison of various models in the dynamic emotion tracking task in dialogue scenarios on the MELD dataset. The results clearly demonstrate the significant advantages of the MetaEmo model compared to single-modality baselines, traditional multimodal fusion models, and dialogue temporal models.From the single-modality baselines, Text-CNN, which relies on the text modality, achieves an accuracy of 62.3% and a weighted F1 score of 61.8%, while Audio-LSTM, using the speech modality, reaches an accuracy of 58.7% and a weighted F1 score of 57.9%. These results suggest that the text modality provides more informative advantages in dialogue emotion recognition than the speech modality. However, both single-modality models suffer from the limitation of relying on a single feature set for emotion recognition.In traditional multimodal fusion models, EarlyFusion-LSTM improves accuracy to 65.1% by concatenating early features from text and speech, while LateFusion-Ensemble further boosts accuracy to 66.8% through late-stage decision fusion, demonstrating the complementary value of multimodal information. However, traditional fusion methods are limited to simple feature concatenation or model ensemble, failing to fully exploit temporal dependencies and modality interactions in dialogues. Dialogue temporal models, such as DialogRNN and DSTC7-ModalNet, incorporate temporal modeling capabilities, improving accuracy to 69.2 and 70.5%, respectively. Notably, DSTC7-ModalNet, which adds the visual modality, shows significant performance improvement, highlighting the importance of visual cues like facial expressions for emotion tracking in dialogue scenarios.The MetaEmo model proposed in this paper, which fuses text, speech, and visual modalities, achieves a substantial improvement with an accuracy of 74.3% and a weighted F1 score of 73.6%, significantly outperforming all comparison models. Compared to DSTC7-ModalNet, which only includes text and speech modalities, MetaEmo improves accuracy by 3.8 percentage points, validating the effectiveness of its multimodal fusion mechanism. Beyond the increase in modality numbers, MetaEmo's advantage lies in its dynamic attention mechanism, which captures emotional temporal changes in dialogue (such as emotional transitions between turns) and cross-cultural expression differences. This makes MetaEmo a superior solution for dynamic emotion tracking in dialogue scenarios.

The Figure 2 presents the confusion matrix of the MetaEmo model on the MELD dataset, providing an intuitive view of its performance across different emotion categories. Looking at the diagonal data, the model achieves high accuracy in recognizing Joy, Sadness, Anger, Fear, and Surprise, with accuracy rates of 85.9%, 84.3%, 86.6%, 86.8%, and 85.7%, respectively. This indicates that, with the help of multimodal fusion and cross-cultural adaptation mechanisms, the model effectively captures the features of these typical emotions across text, speech, and visual modalities, showcasing strong classification capabilities. For the Disgust emotion, the recognition accuracy is 83.2%, slightly lower than the aforementioned emotions. However, considering that Disgust may have a relatively low sample proportion in dialogue scenarios, and its modal feature distinction is not as high, the accuracy is still at a good level. For Neutral emotions, the recognition accuracy is 78.8%, which is relatively lower compared to other emotions. Examining the off-diagonal data, some Neutral samples are misclassified as Joy (4.1%) or Surprise (5.0%), possibly because neutral emotions lack distinctive emotional features, making them easily confused with other low-intensity emotions. Additionally, the diversity of neutral expressions in dialogue scenarios (e.g., flat statements or objective descriptions) further complicates the recognition.Overall, the MetaEmo model performs excellently in the multi-emotion classification task on the MELD dataset. The false positive and false negative rates for each emotion category are generally low, with room for improvement in recognizing neutral emotions. Future work could focus on further optimizing the model by addressing the modality feature differences of neutral emotions and their expression characteristics in dialogue scenarios.

Figure 2

5.4 CREMA-D dataset: emotion recognition accuracy and cultural subset performance in cross-cultural scenarios

The Table 6 presents the overall accuracy and performance across different racial subsets for each model on the CREMA-D dataset, clearly showing the significant advantages and stability of the MetaEmo model in cross-racial emotion recognition. MetaEmo achieves an overall accuracy of 85.9%, leading all models, with a 3.4 percentage point improvement over the next best model, MGCMA (82.5%), and a 10.4 percentage point improvement over the traditional model HuBERT-CLAP (75.5%), validating the adaptability of its multi-module collaborative design to multi-racial emotion data. In the performance of racial subsets, MetaEmo achieves the highest accuracy in each group: Caucasian (89.6%), African American (84.7%), Asian (81.2%), Hispanic (83.3%), and Unspecified (83.9%). The Caucasian subset, which has a higher sample proportion and relatively more explicit emotional expression styles, shows generally high accuracy across all models. However, MetaEmo still leads with 89.6%, surpassing MGCMA's 86.2%. In the Asian subset, where emotional expression tends to be more restrained, MetaEmo dynamically strengthens micro-expression feature extraction through the racial adaptation module, achieving an accuracy of 81.2%, which is a 5.4 percentage point improvement over MGCMA (75.8%). This demonstrates MetaEmo's advantage in recognizing emotions in more reserved cultural expressions. The cross-racial maximum error metric further highlights MetaEmo's robustness. Its error is only 8.4% (89.6%–81.2%), while other models have errors exceeding 10% (e.g., HuBERT-CLAP with 10.8%). This shows that MetaEmo, through cultural difference modeling in the CAAE module and rapid adaptation training with MAML, effectively mitigates “recognition bias caused by racial feature differences,” maintaining balanced performance across different racial groups. In contrast, other models like CLAMP have an accuracy of 85.7% in the Caucasian subset, but only 75.2% in the Asian subset, with a cross-racial fluctuation of 10.5%, reflecting the insufficient adaptability of traditional models to multi-racial emotional expression.This result directly validates MetaEmo's ability to precisely recognize emotions across different racial groups in cross-cultural classroom scenarios, providing an effective solution to eliminate emotion recognition bias between races.

The radar chart displayed in the Figure 3 illustrates the emotion recognition performance of different models across the Caucasian, African American, Asian, Hispanic, and Unspecified racial subsets. As shown, MetaEmo (represented by the dark blue line) consistently covers a significantly larger area in the radar charts for all racial subsets compared to other models such as Maji et al. (orange line) and Liu et al. (light blue line). This indicates that MetaEmo outperforms other models in recognizing a wide range of emotions (e.g., Fear, Disgust, Anger, Neutral, Sad, Happy) across different racial groups in terms of accuracy and other performance metrics. Taking the Caucasian racial subset as an example, MetaEmo shows the broadest coverage in the Anger and Happy emotion dimensions, reflecting its precise ability to capture emotional features specific to this group. In the Asian racial subset, despite the typically more reserved emotional expressions of this group, MetaEmo still excels across all emotion dimensions, far outperforming other models in terms of coverage in this subset. This demonstrates its strong adaptability to more subtle emotional expressions.Overall, MetaEmo's multi-module collaborative design allows it to effectively and efficiently perform multi-emotion recognition, whether for racial groups with more explicit emotional expressions or those with more reserved emotional expressions. Its cross-racial emotion recognition performance is both balanced and superior, making it an outstanding solution for diverse cultural contexts.

Figure 3

5.5 Ablation experiment results

The Table 7 presents the ablation experiment results for the MetaEmo model across datasets, systematically verifying the necessity of each core module for cross-cultural emotion recognition. The complete model achieves F1 scores of 87.9%, 86.3%, and 84.7% on the datasets. However, when each module is removed one by one, the performance shows significant and targeted declines, revealing the collaborative mechanism and irreplaceability of the modules. From the overall performance fluctuations, removing the Transformer-CME module causes an average drop of 6.1 percentage points in the F1 score across datasets (e.g., CMU-MOSEI drops from 87.9 to 81.7%). This is because the module performs cross-modal feature fusion and cultural-independent representation extraction through the Transformer architecture, combining features from speech, facial expressions, and body movements. Without this module, the model can only rely on single-modality features (e.g., using only speech tone to determine emotion), losing the multimodal interaction information (such as the synergistic features of “high pitch - smiling” for Joy), which directly results in a decrease in the completeness and robustness of emotion feature extraction. Removing the CAAE module causes an average drop of 3.2 percentage points in the F1 score (e.g., MELD drops from 86.3 to 82.1%), confirming the core value of this module in modeling cultural differences. CAAE expands minority cultural samples (e.g., African and Latin American cultural emotion data) through conditional adversarial generation and strengthens the perception of cultural differences, such as “Western exaggerated expressions” vs. “Eastern subtle expressions,” via the discriminator. Without this module, the model's resistance to cultural-specific interference is weakened. Particularly in the culturally diverse CREMA-D dataset, the F1 score drops from 84.7 to 80.5%, highlighting CAAE's key role in mitigating the problem of “imbalanced cultural samples.” Removing the MAML module leads to an average drop of 4.3 percentage points in the F1 score (e.g., CREMA-D drops from 84.7 to 79.2%), emphasizing the necessity of rapid adaptive training for cross-cultural scenarios. MAML optimizes the model's initialization parameters, enabling the network to adapt quickly with minimal samples when encountering new cultural environments (e.g., adding East Asian student classroom data). Without this module, the model must rely on retraining with a large amount of data, resulting in a decrease in recall rate from 92.8 to 88.2% on the CMU-MOSEI dataset. This shows a significant drop in the ability to capture rare emotion categories and indicates that the model is unable to meet the recognition needs for “dynamically added samples” in classroom scenarios. The ablation experiments form a rigorous validation logic: Transformer-CME addresses the “information foundation” problem of multimodal feature fusion, CAAE tackles the “adaptation accuracy” issue of cross-cultural differences, and MAML solves the “rapid generalization” problem for model deployment. Together, these three modules interact through data and parameters, collectively building the performance advantages of MetaEmo. The absence of any module breaks this collaborative mechanism, leading to single or multiple shortcomings in feature extraction, cultural adaptation, or generalization ability. This ultimately verifies the scientific validity and necessity of MetaEmo's three-tier technical system.

6 Conclusion and future directions

6.1 Research summary

This study proposes the MetaEmo framework for cross-cultural classroom emotion dynamic recognition, based on meta-learning and cultural adaptation, aiming to address the core issues of emotional expression differences and scarce annotated data in cross-cultural scenarios. The framework integrates cross-modal features from speech, facial expressions, and body movements through the Transformer-CME module, generating culture-independent universal emotion representations. The CAAE module generates diverse cultural emotion samples conditioned on cultural labels, enhancing the model's ability to perceive different cultural expression styles through adversarial training. The MAML module optimizes the model's initialization parameters, enabling rapid adaptive fine-tuning for new cultural environments. Experiments show that MetaEmo achieves F1 scores of 87.9%, 86.3%, and 84.7% on the CMU-MOSEI, MELD, and CREMA-D datasets, respectively, improving by 2.9%–3.4% compared to baseline models. The maximum cross-racial error is only 8.4%, and training time and inference delay are significantly optimized compared to traditional models. These results validate its innovation and effectiveness in multimodal feature fusion, cross-cultural difference modeling, and rapid generalization capability.

6.2 Research limitations

This study still has several limitations: Regarding datasets, the current samples mainly cover Western, African, and some Asian cultural groups, with insufficient coverage of Middle Eastern, Southeast Asian, and other niche cultural scenarios. This may lead to fluctuations in the model's adaptability in extreme cultural differences. In complex cultural interaction scenarios, such as multicultural group discussions or classroom dialogues containing cross-cultural metaphors, the model's accuracy in capturing emotional features from intertwined cultural cues may decline. Furthermore, the model still has room for improvement in recognizing neutral emotions, micro-expressions, and paralinguistic features, especially lacking deep modeling of the relationship between context and emotion, leading to an F1 score of about 80% for neutral emotion recognition, lower than other emotion categories. Specifically, neutral emotional states present ambiguous feature distributions and heavily overlap with low-intensity emotions such as mild joy and surprise in feature space, which is the primary cause of relatively low classification accuracy for this category.

6.3 Future research directions

Future research will focus on the following directions: first, building multimodal emotion databases covering niche cultures such as Middle Eastern and Southeast Asian, incorporating special scenarios like religious ceremonies and dialect expressions to enhance the model's cultural robustness. Second, exploring dynamic cultural weighting mechanisms, enabling the CAAE module to automatically adjust the perception of cultural differences based on real-time input data, and deeply integrating this with MAML to achieve dynamic adaptation between “cultural scenarios” and “model parameters.” Third, targeting the poor performance on neutral emotion recognition: we will introduce context-aware encoders to model long-range semantic dependencies within continuous dialogues, construct task-oriented loss functions to narrow feature distribution overlap between neutral emotions and low-intensity emotions, and enrich sample diversity of neutral emotional expressions to boost classification capability. Additionally, expanding the framework to diverse educational scenarios such as online education and cross-national remote teaching, combining educational psychology theory to construct models that link emotions to learning outcomes, providing data support for personalized teaching interventions. Lastly, optimizing the lightweight design further through model distillation and parameter pruning techniques, advancing the deployment of the framework on edge devices like smart whiteboards and wearables, enabling real-time classroom emotion monitoring and feedback. Considering that the multimodal data adopted in this field contains real human facial micro-expressions and speech signals, future practical promotion and application will also attach great importance to privacy protection and ethical norms. We will study corresponding data management strategies and usage constraints to avoid improper use of sensitive personal information and guarantee the rational application of the emotion recognition system in real classroom environments.

References

• 1

  AliG.AliA.AliF.DrazU.MajeedF.YasinS.et al. (2020). Artificial neural network based ensemble approach for multicultural facial expressions analysis. IEEE Access8, 134950–134963. doi: 10.1109/ACCESS.2020.3009908

  • CrossRef
  • Google Scholar

• 2

  Al-SaadawiH. F. T.DasB.DasR. (2024). A systematic review of trimodal affective computing approaches: text, audio, and visual integration in emotion recognition and sentiment analysis. Expert Syst. Appl. 255:124852. doi: 10.1016/j.eswa.2024.124852

  • CrossRef
  • Google Scholar

• 3

  BaradaranM.ZohariP.MahyarA.MotamedniaH.RahmatiD.GorginS. (2024). “Cultural-aware AI model for emotion recognition,” in 2024 13th Iranian/3rd international machine vision and image processing conference (MVIP) (Tehran: IEEE), 1–6. doi: 10.1109/MVIP62238.2024.10491176

  • CrossRef
  • Google Scholar

• 4

  BarnesE.HutsonJ. (2024). Developing empathetic AI: exploring the potential of artificial intelligence to understand and simulate family dynamics and cultural identity. DS J. Artif. Intell. Robot. 2, 1–24. doi: 10.59232/AIR-V2I3P101

  • CrossRef
  • Google Scholar

• 5

  ChenC.FangH.YangY.ZhouY. (2025). Model-agnostic meta-learning for EEG-based inter-subject emotion recognition. J. Neural Eng. 22:016008. doi: 10.1088/1741-2552/ad9956

  • CrossRef
  • Google Scholar

• 6

  DuX.TongW.JiangL.YuD.WuZ.DuanQ.et al. (2025). SNN-IoT: efficient partitioning and enabling of deep spiking neural networks in IoT services. IEEE Trans. Serv. Comput. 18, 3191–3205. doi: 10.1109/TSC.2025.3592380

  • CrossRef
  • Google Scholar

• 7

  GeethanjaliR.ValarmathiA. (2024). A novel hybrid deep learning IChOA-CNN-LSTM model for modality-enriched and multilingual emotion recognition in social media. Sci. Rep. 14:22270. doi: 10.1038/s41598-024-73452-2

  • CrossRef
  • Google Scholar

• 8

  GhafoorA.NorrenS.FatimaA.MahmoudH. E. A. H. (2025). Cross-cultural emotion recognition in AI: enhancing multimodal NLP for empathetic interaction. Soc. Sci. Spectrum4, 575–588. doi: 10.71085/sss.04.02.295

  • CrossRef
  • Google Scholar

• 9

  GhotbiN. (2023). The ethics of emotional artificial intelligence: a mixed method analysis. Asian Bioeth. Rev. 15, 417–430. doi: 10.1007/s41649-022-00237-y

  • CrossRef
  • Google Scholar

• 10

  GkintoniE.AroutzidisA.AntonopoulouH.HalkiopoulosC. (2025). From neural networks to emotional networks: a systematic review of EEG-based emotion recognition in cognitive neuroscience and real-world applications. Brain Sci. 15:220. doi: 10.3390/brainsci15030220

  • CrossRef
  • Google Scholar

• 11

  HeX. (2025). CLAMP: contrastive learning with adaptive multi-loss and progressive fusion for multimodal aspect-based sentiment analysis. arXiv. [preprint]. arXiv:2507.16854. doi: 10.48550/arXiv.2507.16854

  • CrossRef
  • Google Scholar

• 12

  HosseiniS.YamaghaniM.Poorzaker ArabaniS. (2024). A review of the methods of recognition multimodal emotions in sound, image and text. Int. J. Appl. Operat. Res. 12, 29–41. doi: 10.22105/jarie.2024.413341.1560

  • CrossRef
  • Google Scholar

• 13

  HuG.XinY.LyuW.HuangH.SunC.ZhuZ.et al. (2024). Recent trends of multimodal affective computing: A survey from nlp perspective. arXiv. [preprint]. arXiv:2409.07388. doi: 10.48550/arXiv.2409.07388

  • CrossRef
  • Google Scholar

• 14

  IonD.-G. Smădu, R.-A.CercelD.-C.PopF.CercelM.-C. (2024). “A cross-lingual meta-learning method based on domain adaptation for speech emotion recognition,” in International conference on web information systems engineering (Cham: Springer), 114–129. doi: 10.1007/978-981-96-0579-8_9

  • CrossRef
  • Google Scholar

• 15

  JiangW.-B.LiuX.-H.ZhengW.-L.LuB.-L. (2024). “SEED-VII: a multimodal dataset of six basic emotions with continuous labels for emotion recognition,” in IEEE transactions on affective computing (Piscataway, NJ: IEEE). doi: 10.1109/TAFFC.2024.3485057

  • CrossRef
  • Google Scholar

• 16

  JiaoY. (2025). “Artificial intelligence-assisted emotion recognition and adaptive teaching strategies in cross-cultural language teaching,” in 2025 7th international congress on human-computer interaction, optimization and robotic applications (ICHORA) (Ankara: IEEE), 1–5. doi: 10.1109/ICHORA65333.2025.11017254

  • CrossRef
  • Google Scholar

• 17

  JooK.KimJ.HwangJ. (2024). Effects of foodservice consumers' perceptions of face recognition payment on attitude, desire, and behavioral intentions: a cross-cultural study. J. Travel Tour. Market. 41, 359–376. doi: 10.1080/10548408.2024.2318429

  • CrossRef
  • Google Scholar

• 18

  KalatehS.Estrada-JimenezL. A.Nikghadam-HojjatiS.BarataJ. (2024). A systematic review on multimodal emotion recognition: building blocks, current state, applications, and challenges. IEEE Access12, 103976–104019. doi: 10.1109/ACCESS.2024.3430850

  • CrossRef
  • Google Scholar

• 19

  KapaseA. B.UkeN. (2025). A comprehensive review in affective computing: an exploration of artificial intelligence in unimodal and multimodal emotion recognition systems. Int. J. Speech Technol. 28, 541–563. doi: 10.1007/s10772-025-10202-3

  • CrossRef
  • Google Scholar

• 20

  KatiraiA. (2024). Ethical considerations in emotion recognition technologies: a review of the literature. AI Ethics4, 927–948. doi: 10.1007/s43681-023-00307-3

  • CrossRef
  • Google Scholar

• 21

  KhanU. A.XuQ.LiuY.LagstedtA. Alamäki, A.KauttonenJ. (2024). Exploring contactless techniques in multimodal emotion recognition: insights into diverse applications, challenges, solutions, and prospects. Multimed. Syst. 30:115. doi: 10.1007/s00530-024-01302-2

  • CrossRef
  • Google Scholar

• 22

  KimH.HongT. (2024). Enhancing emotion recognition using multimodal fusion of physiological, environmental, personal data. Expert Syst. Appl. 249:123723. doi: 10.1016/j.eswa.2024.123723

  • CrossRef
  • Google Scholar

• 23

  KusalS.PirbhaiA.PatilS. (2024). “Meta-learning in textual sentiment and emotion analysis: a comprehensive review,” in 2024 MIT art, design and technology school of computing international conference (MITADTSoCiCon) (Pune: IEEE), 1–8. doi: 10.1109/MITADTSoCiCon60330.2024.10574942

  • CrossRef
  • Google Scholar

• 24

  LiH. (2025). Multicultural data assistance mining analysis for ideological and political education in smart education platforms using artificial intelligence. Wirel. Netw. 31, 567–581. doi: 10.1007/s11276-024-03772-8

  • CrossRef
  • Google Scholar

• 25

  LiX.ChenC. P.ChenB.ZhangT. (2024). “Gusa: Graph-based unsupervised subdomain adaptation for cross-subject eeg emotion recognition,” in IEEE transactions on affective computing (Piscataway, NJ: IEEE), 15, 1451–1462. doi: 10.1109/TAFFC.2024.3349770

  • CrossRef
  • Google Scholar

• 26

  LiuR.ZuoH.LianZ.SchullerB. W.LiH. (2024). Contrastive learning based modality-invariant feature acquisition for robust multimodal emotion recognition with missing modalities. IEEE transactions on affective computing (Piscataway, NJ: IEEE), 15, 1856–1873. doi: 10.1109/TAFFC.2024.3378570

  • CrossRef
  • Google Scholar

• 27

  MajiB.SwainM.GuhaR.RoutrayA. (2023). “Multimodal emotion recognition based on deep temporal features using cross-modal transformer and self-attention,” in ICASSP 2023-2023 IEEE international conference on acoustics, speech and signal processing (ICASSP) (Rhodes Island: IEEE), 1–5. doi: 10.1109/ICASSP49357.2023.10096937

  • CrossRef
  • Google Scholar

• 28

  MattioliM.CabitzaF. (2024). Not in my face: challenges and ethical considerations in automatic face emotion recognition technology. Mach. Learn. Knowl. Extr. 6, 2201–2231. doi: 10.3390/make6040109

  • CrossRef
  • Google Scholar

• 29

  Miranda CaleroJ. A.Gutiérrez-MartínL.Rituerto-GonzálezE.Romero-PeralesE.Lanza-GutiérrezJ. M.Peláez-MorenoC.et al. (2024). WEMAC: women and emotion multi-modal affective computing dataset. Sci. Data11:1182. doi: 10.1038/s41597-024-04002-8

  • CrossRef
  • Google Scholar

• 30

  NguyenL. H.PhamN. T.KhanM.OthmaniA.EI SaddikA. (2024). “HuBERT-CLAP: contrastive learning-based multimodal emotion recognition using self-alignment approach,” in Proceedings of the 6th ACM international conference on multimedia in Asia (New York, NY), 1–6. doi: 10.1145/3696409.3700183

  • CrossRef
  • Google Scholar

• 31

  NingX.LiQ.HuangX.ChenQ.HeF.LiW.et al. (2026). “Neural network optimization reimagined: Decoupled techniques for scratch and fine-tuning,” in IEEE transactions on pattern analysis and machine intelligence (Piscataway, NJ: IEEE). doi: 10.1109/TPAMI.2026.3683792

  • CrossRef
  • Google Scholar

• 32

  PeiG.LiH.LuY.WangY.HuaS.LiT. (2024). Affective computing: recent advances, challenges, and future trends. Intell. Comput. 3:0076. doi: 10.34133/icomputing.0076

  • CrossRef
  • Google Scholar

• 33

  PillalamarriR.ShanmugamU. (2025). A review on EEG-based multimodal learning for emotion recognition. Artif. Intell. Rev. 58:131. doi: 10.1007/s10462-025-11126-9

  • CrossRef
  • Google Scholar

• 34

  PrashanthV.GopinathS.UdithS.et al. (2024). “Emotion recognition using meta-learning based on facial expressions,” in 2024 5th IEEE global conference for advancement in technology (GCAT) (Bangalore: IEEE), 1–7. doi: 10.1109/GCAT62922.2024.10923890

  • CrossRef
  • Google Scholar

• 35

  PraveenR. G.AlamJ. (2024). “Recursive joint cross-modal attention for multimodal fusion in dimensional emotion recognition,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (Piscataway, NJ), 4803–4813. doi: 10.1109/CVPRW63382.2024.00483

  • CrossRef
  • Google Scholar

• 36

  QadirI.IqbalM. A.AshrafS.AkramS. (2025). A fusion of CNN and SIFT for multicultural facial expression recognition. Multimed. Tools Appl. 84, 33505–33523. doi: 10.1007/s11042-024-20589-x

  • CrossRef
  • Google Scholar

• 37

  RamaswamyM. P. A.PalaniswamyS. (2024). Multimodal emotion recognition: a comprehensive review, trends, and challenges. WIREs Data Min Knowl Discov. 14:e1563. doi: 10.1002/widm.1563

  • CrossRef
  • Google Scholar

• 38

  SangeethaS.ImmanuelR. R.MathivananS. K.ChoJ.EaswaramoorthyS. V. (2024). “An empirical analysis of multimodal affective computing approaches for advancing emotional intelligence in artificial intelligence for healthcare,” in IEEE access (Piscataway, NJ: IEEE). doi: 10.1109/ACCESS.2024.3444494

  • CrossRef
  • Google Scholar

• 39

  SharmaS.RamaneswaranS.AkhtarM. S.ChakrabortyT. (2024). “Emotion-aware multimodal fusion for meme emotion detection.,” in IEEE transactions on affective computing (Piscataway, NJ: IEEE), 15, 1800–1811. doi: 10.1109/TAFFC.2024.3378698

  • CrossRef
  • Google Scholar

• 40

  ShenL.-Y.FangS.-X.LinY.-C.ChouH.-C.LeeH.-y. (2025). Meta-perSER: Few-shot listener personalized speech emotion recognition via meta-learning. arXiv. [preprint] arXiv:2505.16220. doi: 10.48550/arXiv.2505.16220

  • CrossRef
  • Google Scholar

• 41

  ShiC.ZhangY.LiuB. (2024). A multimodal fusion-based deep learning framework combined with local-global contextual tcns for continuous emotion recognition from videos. Appl. Intell. 54, 3040–3057. doi: 10.1007/s10489-024-05329-w

  • CrossRef
  • Google Scholar

• 42

  TangX.HuangJ.LinY.ZhangS.WangL.LiuY.et al. (2025). Speech emotion recognition via CNN-transformer and multidimensional attention mechanism. Speech Commun. 171:103242. doi: 10.1016/j.specom.2025.103242

  • CrossRef
  • Google Scholar

• 43

  TangY.WuY.HeY.LiuJ.ZhangW. (2025). Dual-task contrastive meta-learning for few-shot cross-domain emotion recognition. Comput. Mater. Continua82, 2331–2352. doi: 10.32604/cmc.2024.059115

  • CrossRef
  • Google Scholar

• 44

  ThirunagalingamA.WhigP. (2025). “Emotional AI integrating human feelings in machine learning,” in Humanizing Technology with Emotional Intelligence (Hershey, PA: IGI Global Scientific Publishing), 19–32. doi: 10.4018/979-8-3693-7011-7.ch002

  • CrossRef
  • Google Scholar

• 45

  TianF.ZhangL.ZhuL.ZhaoM.LiuJ.DongQ.et al. (2024). “Advancements in affective disorder detection: using multimodal physiological signals and neuromorphic computing based on SNNs,” in IEEE transactions on computational social systems (Piscataway, NJ: IEEE). doi: 10.1109/TCSS.2024.3420445

  • CrossRef
  • Google Scholar

• 46

  TiantianZ. (2024). Exploration of the theory and application of artificial intelligence in emotion recognition. J. Artif. Intell. Pract. 7, 129–140. doi: 10.23977/jaip.2024.070217

  • CrossRef
  • Google Scholar

• 47

  WangJ.QiangW.ZhengC.SunF. (2024). Spatio-temporal fuzzy-oriented multi-modal meta-learning for fine-grained emotion recognition. arXiv. [preprint]. arXiv:2412.13541. doi: 10.48550/arXiv.2412.13541

  • CrossRef
  • Google Scholar

• 48

  WangL.ZhuJ.DuL.JinB.WeiX. (2024). “FreAML: a frequency-domain adaptive meta-learning framework for eeg-based emotion recognition,” in 2024 IEEE international conference on bioinformatics and biomedicine (BIBM) (Lisbon: IEEE), 1191–1198. doi: 10.1109/BIBM62325.2024.10822213

  • CrossRef
  • Google Scholar

• 49

  WangR.ZhuJ.WangS.WangT.HuangJ.ZhuX. (2024). Multi-modal emotion recognition using tensor decomposition fusion and self-supervised multi-tasking. Int. J. Multimed. Inf. Retriev. 13:39. doi: 10.1007/s13735-024-00347-3

  • CrossRef
  • Google Scholar

• 50

  WangX.LiC.-Z.SunZ.XuY. (2025a). “Design and analysis of a closed-loop emotion regulation system based on multimodal affective computing and emotional Markov chain,” in IEEE transactions on systems, man, and cybernetics: systems (Piscataway, NJ: IEEE), doi: 10.1109/TSMC.2024.3523342

  • CrossRef
  • Google Scholar

• 51

  WangX.ZhaoS.SunH.WangH.ZhouJ.QinY. (2025b). “Enhancing multimodal emotion recognition through multi-granularity cross-modal alignment,” in ICASSP 2025-2025 IEEE international conference on acoustics, speech and signal processing (ICASSP) (Hyderabad: IEEE), 1–5. IEEE. doi: 10.1109/ICASSP49660.2025.10889156

  • CrossRef
  • Google Scholar

• 52

  WangZ.ShangZ.ZhangL.GhayvatH.HeS.NingX. (2026). Fusing physical priors and visual mamba: an snr-aware framework for low-light enhancement in HVI space. Inf. Fusion134:104350. doi: 10.1016/j.inffus.2026.104350

  • CrossRef
  • Google Scholar

• 53

  XiangA.QiZ.WangH.YangQ.MaD. (2024). “A multimodal fusion network for student emotion recognition based on transformer and tensor product,” in 2024 IEEE 2nd international conference on sensors, electronics and computer engineering (ICSECE) (Jinzhou: IEEE), 1–4. doi: 10.1109/ICSECE61636.2024.10729485

  • CrossRef
  • Google Scholar

• 54

  XiaoJ.ZhaoS.DengK. (2024). “Emotion analysis of low resource domain based on transfer learning and meta-learning methods,” in International conference on frontier computing (Cham: Springer), 131–140. doi: 10.1007/978-981-96-2794-3_15

  • CrossRef
  • Google Scholar

• 55

  XieJ.WangJ.WangQ.YangD.GuJ.TangY.et al. (2023). A multimodal fusion emotion recognition method based on multitask learning and attention mechanism. Neurocomputing556:126649. doi: 10.1016/j.neucom.2023.126649

  • CrossRef
  • Google Scholar

• 56

  XieY.SunC.-J.CaoZ.LiuB.JiZ.LiuY.et al. (2025). “A dual contrastive learning framework for enhanced multimodal conversational emotion recognition,” in Proceedings of the 31st international conference on computational linguistics (New York, NY), 4055–4065.

  • Google Scholar

• 57

  YangP.LiuN.LiuX.ShuY.JiW.RenZ.et al. (2024). A multimodal dataset for mixed emotion recognition. Sci. Data11:847. doi: 10.1038/s41597-024-03676-4

  • CrossRef
  • Google Scholar

• 58

  ZhangD.ChenF.ChenX. (2023). “Dualgats: dual graph attention networks for emotion recognition in conversations,” in Proceedings of the 61st annual meeting of the association for computational linguistics Vol. 1. (Long Papers, Toronto, New York, NY: Association for Computational Linguistics), 7395–7408. doi: 10.18653/v1/2023.acl-long.408

  • CrossRef
  • Google Scholar

• 59

  ZhangD. J.LiD.LeH.ShouM. Z.XiongC.SahooD. (2025). Moonshot: towards controllable video generation and editing with motion-aware multimodal conditions. Int. J. Comput. Vis. 133, 3629–3644. doi: 10.1007/s11263-025-02346-1

  • CrossRef
  • Google Scholar

• 60

  ZhangL.JinL.XuG.LiX.XuC.WeiK.et al. (2024). “CAMEL: capturing metaphorical alignment with context disentangling for multimodal emotion recognition,” in Proceedings of the AAAI conference on artificial intelligence (Vol. 38), 9341–9349. doi: 10.1609/aaai.v38i8.28787

  • CrossRef
  • Google Scholar

• 61

  ZhangN.LeongW. Y. (2025). Intelligent emotional computing with deep convolutional neural networks: Multimodal feature analysis and application in smart learning environments. Eurasia J. Math. Sci. Technol. Educ. 21:em2680. doi: 10.29333/ejmste/16661

  • CrossRef
  • Google Scholar

• 62

  ZhaoM. (2023). The emotion recognition in psychology of human-robot interaction. Psychomachina1, 1–11. doi: 10.59388/pm00331

  • CrossRef
  • Google Scholar