Exploiting Facial Emotion Recognition System for Ambient Assisted Living Technologies Triggered by Interpreting the User's Emotional State

Samuele Russo¹Imad Eddine TIBERMACINE^2*Cristian Randieri³Abdelaziz Rabehi⁴Amal H. Alharbi⁵El-Sayed M. El-kenawy^6,7Christian Napoli^10,8,9

• ¹Department of Psychology, Sapienza University of Rome, Italy, Rome, Italy
• ²Sapienza University of Rome, Rome, Italy
• ³eCampus University, Novedrate, Italy
• ⁴University of Djelfa, Algeria, Djelfa, Algeria
• ⁵Department of Computer Sciences, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, Riyadh, Saudi Arabia
• ⁶School of ICT, Faculty of Engineering, Design and Information & Communications Technology (EDICT), Bahrain Polytechnic, Isa Town, Bahrain
• ⁷Applied Science Research Center. Applied Science Private University, Amman, Jordan, Amman, Jordan
• ⁸Department of Computer, Automation and Management Engineering, Sapienza University of Rome, Italy, Rome, Italy
• ⁹Institute for Systems Analysis and Computer Science, Italian National Research Council, Rome, Italy
• ¹⁰Department of Artificial Intelligence, Czestochowa University of Technology, Czestochowa, Poland

Introduction: Facial Emotion Recognition (FER) enables smart environments and robots to adapt their behaviour to a user's affective state. Translating those recognized emotions into ambient cues, such as colored lighting, can improve comfort and engagement in Ambient Assisted Living (AAL) settings.We design a FER pipeline that combines a Spatial Transformer Network for poseinvariant region focusing with a novel Multiple Self-Attention (MSA) block comprising parallel attention heads and learned fusion weights. The MSA-enhanced block is inserted into a compact VGG-style backbone trained on the FER+ dataset using weighted sampling to counteract class 1 Russo et al.imbalance. The resulting soft-max probabilities are linearly blended with prototype hues derived from a simplified Plutchik wheel to drive RGB lighting in real time.The proposed VGGFac-STN-MSA model achieves 82.54% test accuracy on FER+, outperforming a CNN baseline and the reproduced Deep-Emotion architecture. Ablation shows that MSA contributes +1% accuracy. Continuous colour blending yields smooth, intensity-aware lighting transitions in a proof-of-concept demo. Discussion: Our attention scheme is architecture-agnostic, adds minimal computational overhead, and markedly boosts FER accuracy on low-resolution faces. Coupling the probability distribution directly to the RGB gamut provides a fine-grained, perceptually meaningful channel for affect-adaptive AAL systems.