Computational Multimodal Sensing and Perception for Robotic Systems

As robots increasingly participate in smart homes, agriculture, and industries, advanced sensing and perception have become crucial. Notably, multimodal sensing has drawn growing attention for its potential to enhance perceptual quality and accuracy. For instance, visuo-tactile (ViTac) data - two sensory modalities with distinct properties - has been extensively studied in robot manipulation. Beyond ViTac, emerging sensing modalities, such as radar, neuromorphic, or even superhuman sensing, are expanding robotic perceptual capabilities. However, integrating diverse sensory data usually requires task-specific and modality-specific design, which can introduce biases and limit generalizability. Recent research has thus shifted toward computational methods for sensor fusion, learning architectures, and perception frameworks to enhance system adaptability in complex real-world scenarios. This Research Topic seeks to advance the field of multimodal sensing and perception in robotics, emphasizing novel sensing systems and algorithms. We aim to drive interdisciplinary research to find new solutions to the increasingly complex perceptual challenges faced by robotic systems.

Human sensing and perception systems excel at processing multimodal sensory data streams to support daily tasks. For instance, when blindly searching for keys in a backpack, limited visual cues are often compensated by haptic and auditory feedback, effectively guiding us to locate the keys. In contrast, such multimodal sensory integration is a significant challenge for current robotic systems. This raises important questions at the intersection of robotics, cognitive science, and artificial intelligence regarding how robots can leverage multimodal sensory information to interact with their environment and perform complex tasks:
- Can we develop a generalized approach to integrating multimodal sensory information that transcends specific task constraints, sensor configurations, and intrinsic characteristics of sensory data?
- How can multimodal sensory information empower robots to learn, coordinate, and adapt effectively across centralized and decentralized architectures?

Addressing these questions, multimodal sensing in robots holds great potential to improve the performance of diverse tasks. However, current multisensory pipelines remain significantly constrained by sensor reliability, crossmodal feature representation, multi-level sensor fusion, and multimodal learning architectures. This Research Topic aims to advance robotic systems by drawing inspiration from human sensing mechanisms, while also pushing beyond the principles of human perception to explore new paradigms in robotic sensing and perception.

In this Research Topic, we are interested in all aspects of multimodal sensing and perception with a focus on robotic applications, including but not limited to:
1. Multimodal Sensor Design and Dataset Development
(a) Novel sensor architectures
(b) Multimodal sensory dataset collection and benchmarking
2. Multimodal Sensory Signal Processing
(a) Multimodal signal alignment, fusion, and integration
(b) Multimodal feature representation, translation, and cross-modal mapping
3. Multimodal Learning Paradigms for Robotic Tasks
(a) Learning approaches including supervised, unsupervised, transfer, and co-learning
(b) Adaptive learning frameworks for complex sensory information
4. Multimodal Distributed and Coordinated Robotic Systems
(a) Multi-robot and multi-sensor integration
(b) Agile-coordinated multi-robot and multi-sensor architectures
(c) Distributed sensor networks for robotic feedback and performance
5. Their applications on robotic system development across robot locomotion, manipulation, navigation, and more