Impact Factor 2.606
2017 JCR, Clarivate Analytics 2018

The world's most-cited Neurosciences journals

This article is part of the Research Topic

Shared Autonomy

Editorial ARTICLE Provisionally accepted The full-text will be published soon. Notify me

Front. Neurorobot. | doi: 10.3389/fnbot.2019.00016

Shared Autonomy – Learning of Joint Action and Human-Robot Collaboration

  • 1Bielefeld University, Germany
  • 2Institute of Computer Science, Albert Ludwigs University Freiburg, Germany
  • 3Carnegie Institute of Technology, Carnegie Mellon University, United States
  • 4Cluster of Excellence Cognitive Interaction Technology (CITEC), Bielefeld University, Germany

ABSTRACT
With the continuous advancement in the areas of robotics and machine learning, more and more autonomous robot solutions find their way into our daily lives. These technologies not only have the potential to take care of monotonous tasks or tasks too dangerous to be executed by a human, but can also help humans to regain or enhance their autonomy. It is desirable that those robots (or agents) possess at least some degree of autonomy in order to free us from controlling all the details of operation. Shared Autonomy as a concept describes how all the agents can remain autonomous, following overall their own intentions and goals, but at the same time deal with possible conflicts. This special issue, on the one hand, showcases the diversity of forms of flexible collaboration between humans and (multiple) agents. On the other hand, it introduces learning and robotic architectures demonstrating such cooperation as well as perspectives on theoretical dimensions of Shared Autonomy.
MAIN TEXT (Starts with no subheading)
Advancing the autonomy of artificial agents, such as robots, comes with an important challenge: how to shape the autonomy of an individual agent in such a manner that bringing several such agents together leads to a suitable fusion or sharing of (parts of) their individual autonomy spaces. This challenge has opened up important new perspectives on how to make robots more skillful, versatile and easier to deploy for tasks and scenarios where robots can no longer act solitarily.
The special issue aims at, first, providing a theoretical perspective to describe autonomy in collaborative settings, and, secondly, presenting novel results that highlight common traits and underlying dimensions and can guide further developments on adaptive interaction in teams of humans and robots.
Dimensions of Shared Autonomy
Accordingly, Shared Autonomy has to address the question on how to mediate autonomy between participating actors. This question can be approached through a distinction of different levels of autonomy and by defining characteristic dimensions describing collaborative scenarios [Schilling et al., 2016]. In their review, Alonso and colleagues [2018] identify transparency as one such characteristic which is understood as observability and predictability. They show that this translates well to interactive multi-agent scenarios and provides useful dimensions to account for interaction patterns. As predictability decreased in more complex tasks, systems were found to act more autonomously. They reasoned that this is due to limited observability which can be counteracted by subsuming low-level details in higher-level representations as are goals. Here, Gildert and colleagues [2018] continue with their review on joint action which highlights a shared context as a requirement for prediction. Furthermore, they point out the role of attention and how implicit and explicit communication can guide attention between agents in order to mediate interaction patterns. Both articles connect the issue of shared information in a further step towards the notion of trust in interactive settings.
*** FIGURE 1 ABOUT HERE: ***
CAPTION: Distinguishing multiple levels of autonomy (shown on the left). Conceptually, these different levels characterize the freedom of decision making arising at different levels of an abstraction hierarchy. The notion of Shared Autonomy allows to analyze and design types of interaction patterns between human users and other agents (shown on the right). For details see [Schilling et al., 2016].
***
Shared Autonomy for Human-Robot Collaboration
The special issue also provides examples that highlight current advances in human-robot collaboration and show that different levels of a representational hierarchy may call for different mechanisms and strategies: lower levels deal with the optimal selection of means to achieve immediate goals; autonomy on an intermediate level determines strategies that fulfill higher level purposes which are given on the highest level (as intentions). Importantly, studies on coordination with robots on a higher-level further stress that this can’t be addressed solely on an abstract level, but always requires grounding in lower action levels. This further emphasizes the importance of a differentiated perspective on Shared Autonomy.
For coordination on the lower level of immediate action, Trinh et al. [2018] provide a solution to navigation for multiple agents as a prototypical example for Shared Autonomy scenarios. Each agent preplans its own route to a target location which is represented as a static flow field. To avoid collisions, this representation is overlaid with a constantly updated dipole flow field of other agents’ locations. The agents’ behavior is mostly driven by the preplanned flow field, but the dipole flow field shapes the routes of the agents autonomously. Ewerton et al. [2018] directly address this change of control and autonomy. Their framework deals with visual and haptic feedback to users while learning motor skills. In their case users are intended to learn drawing Japanese characters. Probabilistic models of skills are learned following a reinforcement learning approach and the skill likelihood is used to regulate the amount of feedback. The system autonomously steps in when needed and starts to provide guidance through haptic feedback.
The focus of Akkaladevi et al. [2018] is on how cooperation can be organized between multiple agents in a collaborative assembly process. On the lowest level, this entails learning basic actions. On a higher level, the system aims at putting these into a sequence. In addition, the system attempts to propose suitable actions that are chosen depending on background knowledge about users’ capabilities, current context and current goals. Knowledge was learned using incremental reinforcement learning. The Interactive Shared Solution Shaping paradigm by Reardon and colleagues [2018] focusses further on an intermediate level of representation for negotiating a shared plan: while the autonomous agent has its own planning process, a human user provides feedback based on his expert domain knowledge. This was realized for planning the route of a surveillance robot. While the user doesn’t need to know all the details about, e.g., collision avoidance, mediation of the planning process requires transparency between the robot system and the user. A form of interaction is needed that details current routes to the user and allows to influence the (re)planning process. Establishing such a form of fast and reliable communication is crucial for systems that should work autonomously, but at the same time contribute towards a user’s intention and provide valuable information. In Shukla et al. [2018] such multimodal coordination is further analyzed and applied. They introduce the probabilistic Proactive Incremental Learning framework that learns to associate hand gestures with manipulation actions in a collaborative assembly task. There, communication is required to establish common ground and align mutual beliefs between robot and user which exploits information about gaze. The system anticipated users’ behaviors and goals in order to proactively assist the user. In a study with non-roboticist users they found that their proactive system reduced the interaction effort and was mostly preferred, but further cautioned that this requires trust.
In a further user study, Schulz et al. [2018] turn towards the question of how preferred interaction with robots is task dependent. They categorize forms of collaboration along two dimensions: first, distinguishing independent actions and joint interaction; secondly, distinguishing sequences where order is crucial versus not. This was assessed in a series of table-top scenarios in which a human collaborated with a robot to build a given design in a blocks world. They found that mostly autonomous actions of the robot are more efficient and preferred.
Conclusion
The special issue brings together current work and trends in Shared Autonomy. Collaboration in teams of humans and robots shifts more and more towards higher level tasks that not only include low level coordination of immediate action. This requires coordination of complex plans between multiple agents that can be adapted at runtime. Therefore, each agent is tasked with autonomous control on different levels, but also has to respect the autonomy of other agents and therefore has to adjust its degree of autonomy. Shared Autonomy provides a useful perspective on the underlying interaction patterns, how these are adapted and what requirements this poses to systems, like establishing common ground, transparency and shared beliefs as well as goals, trust, and efficient forms of communication.

MAIN TEXT (Starts with no subheading)
Advancing the autonomy of artificial agents, such as robots, comes with an important challenge: how to shape the autonomy of an individual agent in such a manner that bringing several such agents together leads to a suitable fusion or sharing of (parts of) their individual autonomy spaces. This challenge has opened up important new perspectives on how to make robots more skillful, versatile and easier to deploy for tasks and scenarios where robots can no longer act solitarily.
The special issue aims at, first, providing a theoretical perspective to describe autonomy in collaborative settings, and, secondly, presenting novel results that highlight common traits and underlying dimensions and can guide further developments on adaptive interaction in teams of humans and robots.
Dimensions of Shared Autonomy
Accordingly, Shared Autonomy has to address the question on how to mediate autonomy between participating actors. This question can be approached through a distinction of different levels of autonomy and by defining characteristic dimensions describing collaborative scenarios [Schilling et al., 2016]. In their review, Alonso and colleagues [2018] identify transparency as one such characteristic which is understood as observability and predictability. They show that this translates well to interactive multi-agent scenarios and provides useful dimensions to account for interaction patterns. As predictability decreased in more complex tasks, systems were found to act more autonomously. They reasoned that this is due to limited observability which can be counteracted by subsuming low-level details in higher-level representations as are goals. Here, Gildert and colleagues [2018] continue with their review on joint action which highlights a shared context as a requirement for prediction. Furthermore, they point out the role of attention and how implicit and explicit communication can guide attention between agents in order to mediate interaction patterns. Both articles connect the issue of shared information in a further step towards the notion of trust in interactive settings.

Shared Autonomy for Human-Robot Collaboration
The special issue also provides examples that highlight current advances in human-robot collaboration and show that different levels of a representational hierarchy may call for different mechanisms and strategies: lower levels deal with the optimal selection of means to achieve immediate goals; autonomy on an intermediate level determines strategies that fulfill higher level purposes which are given on the highest level (as intentions). Importantly, studies on coordination with robots on a higher-level further stress that this can’t be addressed solely on an abstract level, but always requires grounding in lower action levels. This further emphasizes the importance of a differentiated perspective on Shared Autonomy.
For coordination on the lower level of immediate action, Trinh et al. [2018] provide a solution to navigation for multiple agents as a prototypical example for Shared Autonomy scenarios. Each agent preplans its own route to a target location which is represented as a static flow field. To avoid collisions, this representation is overlaid with a constantly updated dipole flow field of other agents’ locations. The agents’ behavior is mostly driven by the preplanned flow field, but the dipole flow field shapes the routes of the agents autonomously. Ewerton et al. [2018] directly address this change of control and autonomy. Their framework deals with visual and haptic feedback to users while learning motor skills. In their case users are intended to learn drawing Japanese characters. Probabilistic models of skills are learned following a reinforcement learning approach and the skill likelihood is used to regulate the amount of feedback. The system autonomously steps in when needed and starts to provide guidance through haptic feedback.
The focus of Akkaladevi et al. [2018] is on how cooperation can be organized between multiple agents in a collaborative assembly process. On the lowest level, this entails learning basic actions. On a higher level, the system aims at putting these into a sequence. In addition, the system attempts to propose suitable actions that are chosen depending on background knowledge about users’ capabilities, current context and current goals. Knowledge was learned using incremental reinforcement learning. The Interactive Shared Solution Shaping paradigm by Reardon and colleagues [2018] focusses further on an intermediate level of representation for negotiating a shared plan: while the autonomous agent has its own planning process, a human user provides feedback based on his expert domain knowledge. This was realized for planning the route of a surveillance robot. While the user doesn’t need to know all the details about, e.g., collision avoidance, mediation of the planning process requires transparency between the robot system and the user. A form of interaction is needed that details current routes to the user and allows to influence the (re)planning process. Establishing such a form of fast and reliable communication is crucial for systems that should work autonomously, but at the same time contribute towards a user’s intention and provide valuable information. In Shukla et al. [2018] such multimodal coordination is further analyzed and applied. They introduce the probabilistic Proactive Incremental Learning framework that learns to associate hand gestures with manipulation actions in a collaborative assembly task. There, communication is required to establish common ground and align mutual beliefs between robot and user which exploits information about gaze. The system anticipated users’ behaviors and goals in order to proactively assist the user. In a study with non-roboticist users they found that their proactive system reduced the interaction effort and was mostly preferred, but further cautioned that this requires trust.
In a further user study, Schulz et al. [2018] turn towards the question of how preferred interaction with robots is task dependent. They categorize forms of collaboration along two dimensions: first, distinguishing independent actions and joint interaction; secondly, distinguishing sequences where order is crucial versus not. This was assessed in a series of table-top scenarios in which a human collaborated with a robot to build a given design in a blocks world. They found that mostly autonomous actions of the robot are more efficient and preferred.
Conclusion
The special issue brings together current work and trends in Shared Autonomy. Collaboration in teams of humans and robots shifts more and more towards higher level tasks that not only include low level coordination of immediate action. This requires coordination of complex plans between multiple agents that can be adapted at runtime. Therefore, each agent is tasked with autonomous control on different levels, but also has to respect the autonomy of other agents and therefore has to adjust its degree of autonomy. Shared Autonomy provides a useful perspective on the underlying interaction patterns, how these are adapted and what requirements this poses to systems, like establishing common ground, transparency and shared beliefs as well as goals, trust, and efficient forms of communication.

Keywords: Shared autonomy, human-robot interaction, Learning, collaboration, interactive robots

Received: 01 Feb 2019; Accepted: 09 Apr 2019.

Edited by:

Florian Röhrbein, Technische Universität München, Germany

Reviewed by:

Tadej Petric, Jožef Stefan Institute (IJS), Slovenia  

Copyright: © 2019 Schilling, Burgard, Muelling, Wrede and Ritter. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

* Correspondence: Dr. Malte Schilling, Bielefeld University, Bielefeld, 94707, CA, Germany, malteschilling@googlemail.com