AUTHOR=Amaya Camilo , von Arnim Axel 

TITLE=Neurorobotic reinforcement learning for domains with parametrical uncertainty

JOURNAL=Frontiers in Neurorobotics

VOLUME=Volume 17 - 2023

YEAR=2023

URL=https://www.frontiersin.org/journals/neurorobotics/articles/10.3389/fnbot.2023.1239581

DOI=10.3389/fnbot.2023.1239581

ISSN=1662-5218

ABSTRACT=Neuromorphic hardware paired with braininspired learning strategies have enormous potential for robot control. Explicitly, these advantages include low energy consumption, low latency, and adaptability. Therefore, developing and improving learning strategies, algorithms, and neuromorphic hardware integration in simulation is a key to moving the state-of-the-art forward. In this work we use the Neurorobotics Platform (NRP) simulation framework to implement spiking reinforcement learning control for a robotic arm. We implement a force-torque feedback based classic object insertion task ("peg-inhole") and control the robot for the first time with neuromorphic hardware in the loop. We therefore provide a solution for training the system in uncertain environmental domains by using randomized simulation parameters. This leads to policies that are robust to realworld parameter variations in the target domain, filling the sim-to-real gap. To our knowledge it is the first neuromorphic implementation of the peg-in-hole task in simulation with the neuromorphic Loihi chip in the loop, and with scripted accelerated interactive training in the Neurorobotics Platform, including randomized domains.