Multimodal human action recognition in real or virtual environments

Human action recognition (HAR) is a rapidly evolving field that bridges artificial intelligence, robotics, and immersive technologies. Multimodal approaches leverage diverse data sources such as visual, skeletal, audio, haptic, and physiological signals, to enhance the accuracy and robustness of action recognition in both real and virtual environments. These approaches are crucial for applications in human-robot interaction, virtual reality (VR), augmented reality (AR), healthcare, sports analytics, surveillance, and gaming.

In real-world settings, multimodal HAR improves the interpretation of complex human behaviors by combining data from RGB or thermal cameras, depth sensors, inertial measurement units (IMUs), and physiological signals such as electromyography (EMG) or electroencephalography (EEG). These integrations enable robots and AI systems to recognize subtle human movements, predict intentions, and respond adaptively.

In virtual environments, the use of avatars and motion-tracking technologies facilitates immersive and interactive experiences. Multimodal inputs, such as gaze tracking, speech, and body posture, enhance the realism and responsiveness of VR and AR applications. These technologies are pivotal for training simulations, remote collaboration, and neurorehabilitation.

Despite significant progress, challenges remain, including data fusion complexities, real-time processing demands, domain adaptation between real and virtual settings, and generalization across diverse users and environments. Advances in computational methods, deep learning, transformer models, and self-supervised learning are opening new pathways for overcoming these obstacles.

This article collection explores cutting-edge research on multimodal human action recognition, focusing on novel methodologies, applications, and future directions in both real and virtual worlds. It aims to foster interdisciplinary discussions on integrating AI, robotics, and immersive technologies for enhanced action understanding and interaction.

This Research Topic focuses on novel ideas, models, and methods in artificial neural networks for multimodal human action recognition. Authors are invited to submit papers reporting novel methods with applications in human-computer interaction, virtual reality, gaming, robotics, healthcare, and artificial intelligence to this Article Collection.

Topics of interest include, but are not limited to, the following:

• Novel architectures for multimodal fusion in HAR
• AI-driven applications in surveillance, healthcare, and sports analytics
• Body movement, voice, and facial expressions
• Multimodal data fusion
• Applications in robotics and human-robot interaction
• Human motion for robotics and intelligent systems
• Gaming
• 3D avatar-based recognition
• Applications of human action recognition in virtual and augmented, and mixed reality
• Cognitive and neurological approaches for action recognition
• Multimodal action recognition in healthcare and rehabilitation
• Challenges in dataset creation, benchmarking, and real-time processing

Article types and fees

This Research Topic accepts the following article types, unless otherwise specified in the Research Topic description:

• Brief Research Report
• Data Report
• Editorial
• FAIR² Data
• FAIR² DATA Direct Submission
• General Commentary
• Hypothesis and Theory
• Methods
• Mini Review
• ... View all formats

Articles that are accepted for publication by our external editors following rigorous peer review incur a publishing fee charged to Authors, institutions, or funders.

Article types

This Research Topic accepts the following article types, unless otherwise specified in the Research Topic description:

• Brief Research Report
• Data Report
• Editorial
• FAIR² Data
• FAIR² DATA Direct Submission
• General Commentary
• Hypothesis and Theory
• Methods
• Mini Review
• Opinion
• Original Research
• Perspective
• Review
• Systematic Review
• Technology and Code

Keywords: human action recognition, multimodal recognition, data fusion, human-robot interaction, immersive technologies

Important note: All contributions to this Research Topic must be within the scope of the section and journal to which they are submitted, as defined in their mission statements. Frontiers reserves the right to guide an out-of-scope manuscript to a more suitable section or journal at any stage of peer review.