Impact Factor 3.000 | CiteScore 3.51
More on impact ›

Methods ARTICLE Provisionally accepted The full-text will be published soon. Notify me

Front. Neurorobot. | doi: 10.3389/fnbot.2019.00076

Hybrid Brain-Computer-Interfacing for Human-Compliant Robots: Inferring Continuous Subjective Ratings with Deep Regression

 Lukas D. Fiederer1, 2*, Martin Völker1, 2, 3, Robin T. Schirrmeister1, 2,  Wolfram Burgard2, 4,  Joschka Boedecker2, 5 and  Tonio Ball1, 2
  • 1Translational Neurotechnologie Labor, Albert-Ludwigs-Universität Freiburg, Germany
  • 2BrainLinks BrainTools, Exzellenzcluster, Albert Ludwigs Universität Freiburg, Germany
  • 3Graduate School of Robotics, Institute for Computer Science, Albert Ludwigs University of Freiburg, Germany
  • 4Autonomous Intelligent Systems, Department of Computer Science, University of Freiburg, Germany
  • 5Institute of Computer Science, Albert Ludwigs University Freiburg, Germany

Appropriate robot behavior during human-robot interaction is a key part in the development of human-compliant assistive robotic systems. This study poses the question of how to continuously evaluate the quality of robotic behavior in a hybrid brain-computer interfacing (BCI) task, combining brain and non-brain signals, and how to use the collected information to adapt the robot's behavior accordingly.
To this aim, we developed a rating system compatible with EEG recordings, requiring the users to execute only small movements with their thumb on a wireless controller to rate the robot's behavior on a continuous scale. The ratings were recorded together with dry EEG, respiration, ECG, and robotic joint angles in ROS. Pilot experiments were conducted with 3 users that had different levels of previous experience with robots.
The results demonstrate the feasibility to obtain continuous rating data that give insight into the subjective user perception during direct human-robot interaction. The rating data suggests differences in subjective perception for users with no, moderate, or substantial previous robot experience. Furthermore, a variety of regression techniques, including deep CNNs, allowed us to predict the subjective ratings. Performance was better when using the position of the robotic hand than when using EEG, ECG or respiration. A consistent advantage of features expected to be related to a motor bias could not be found. Across-user predictions showed that the models most likely learned a combination of general and individual features across-users. A transfer of pre-trained regressor to a new user was especially accurate in users with more experience. For future research, studies with more participants will be needed to evaluate the methodology for its use in practice. Data and code to reproduce this study are available at

Keywords: Autonomous Robots, BCI (Brain Computer Interface), EEG, Convolutional Neural Networks - CNN, Deep learning (DL) approaches, Deep regression networks, Robot behavior, Robot Operating System (ROS), Regression -, Random Forest., Suppor Vector Machines

Received: 30 Nov 2018; Accepted: 27 Aug 2019.

Copyright: © 2019 Fiederer, Völker, Schirrmeister, Burgard, Boedecker and Ball. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

* Correspondence: Dr. Lukas D. Fiederer, Translational Neurotechnologie Labor, Albert-Ludwigs-Universität Freiburg, Freiburg, Germany,