ORIGINAL RESEARCH article
Front. Robot. AI
Sec. Human-Robot Interaction
A Framework for Semantics-based Situational Awareness during Mobile Robot Deployments
Provisionally accepted
Tianshu Ruan1
Aniketh Ramesh1Hao Wang1
Alix Johnstone-Morfoisse1Gokcenur Altindal1
Paul Norman1
GRIGORIOS NIKOLAOU2
Rustam Stolkin1
Manolis Chiou3*- 1University of Birmingham, Birmingham, United Kingdom
- 2University of West Attica, Athens, Greece
- 3Queen Mary University of London, London, United Kingdom
Select one of your emails
You have multiple emails registered with Frontiers:
Notify me on publication
Please enter your email address:
If you already have an account, please login
You don't have a Frontiers account ? You can register here
Deployment of robots into hazardous environments typically involves a "Human-Robot Teaming" (HRT) paradigm, in which a human supervisor interacts with a remotely operating robot inside the hazardous zone. Situational Awareness (SA) is vital for enabling HRT, to support navigation, planning, and decision-making. This paper explores issues of higher-level "semantic" information and understanding in SA. In semi-autonomous, or variable-autonomy paradigms, different types of semantic information may be important, in different ways, for both the human operator and an autonomous agent controlling the robot. We propose a generalizable framework for acquiring and combining multiple modalities of semantic-level SA during remote deployments of mobile robots. We demonstrate the framework with an example application of Search and Rescue (SAR) in disaster response robotics. We propose a set of "environment semantic indicators" that can reflect a variety of different types of semantic information, e.g. indicators of risk, or Signs of Human Activity (SHA), as the robot encounters different scenes. Based on these indicators, we propose a metric to describe the overall situation of the environment called "Situational Semantic Richness (SSR)". This metric combines multiple semantic indicators to summarise the overall situation. The SSR indicates if an information-rich and complex situation has been encountered, which may require advanced reasoning for robots and humans and hence the attention of the expert human operator. The framework is tested on a Jackal robot in a mock-up disaster response environment. Experimental results demonstrate that the proposed semantic indicators are sensitive to changes in different modalities of semantic information in different scenes, and the SSR metric reflects overall semantic changes in the situations encountered.
Keywords: Situational Awareness, semantics, semantic understanding, Human-robot teaming, Disaster response robotics, search andrescue robotics
Received: 28 Aug 2025; Accepted: 27 Oct 2025.
Copyright: © 2025 Ruan, Ramesh, Wang, Johnstone-Morfoisse, Altindal, Norman, NIKOLAOU, Stolkin and Chiou. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
* Correspondence: Manolis Chiou, m.chiou@qmul.ac.uk
Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.