AUTHOR=Duczek Nicolas , Kerzel Matthias , Allgeuer  Philipp , Wermter  Stefan 

TITLE=Self-organized Learning from Synthetic and Real-World Data for a Humanoid Exercise Robot

JOURNAL=Frontiers in Robotics and AI

VOLUME=Volume 9 - 2022

YEAR=2022

URL=https://www.frontiersin.org/journals/robotics-and-ai/articles/10.3389/frobt.2022.669719

DOI=10.3389/frobt.2022.669719

ISSN=2296-9144

ABSTRACT=In order to detect and correct physical exercises, a Grow-When-Required (GWR) network with recurrent connections, episodic memory and a novel subnode mechanism is developed in order to 
learn spatiotemporal relationships of body movements and poses. Once an exercise is performed,
the information of pose and movement per frame is stored in the so-called Subnode-GWR network. For every frame, the current pose and motion pair is compared against a predicted output of the GWR, allowing for feedback not only on the pose but also on the velocity of the motion. In the application scenario, a physical exercise is performed in the presence of an expert like a physiotherapist, and then used as a reference for a humanoid robot like Pepper to give feedback on further executions of the same exercise. Since the humanoid robot is mobile, it can be employed in different environments. This approach, however, comes with two challenges. First, the positioning of the user in the humanoid robot’s field of view is variable and has to be considered by the GWR as well, requiring robustness against the user’s positioning in the field of view of the humanoid robot. Second, since both the pose and motion are dependent on the body morphology of a user, the exercise demonstration by one individual cannot easily be used as a reference for further users. This paper tackles the first challenge by designing an architecture that allows for tolerances in translation and rotations regarding the center of the field of view. For the second challenge, we allow the GWR to grow online with each further demonstration. This requires careful implementation to ensure that the GWR does not forget previously learnt exercise demonstrations. For evaluation, we developed a new synthetic exercise dataset with virtual avatars called the Virtual-Squat dataset. We also test our method on real data recorded in an office scenario. Overall, we claim that our novel GWR-based architecture can use a learned exercise reference for different body variations through incremental online learning while preventing catastrophic forgetting, enabling an engaging long-term human-robot experience with a humanoid robot.