AUTHOR=Marques Monteiro Richard , Shi Jialei , Wurdemann Helge , Iida Fumiya , George Thuruthel Thomas 

TITLE=Visuo-dynamic self-modelling of soft robotic systems

JOURNAL=Frontiers in Robotics and AI

VOLUME=Volume 11 - 2024

YEAR=2024

URL=https://www.frontiersin.org/journals/robotics-and-ai/articles/10.3389/frobt.2024.1403733

DOI=10.3389/frobt.2024.1403733

ISSN=2296-9144

ABSTRACT=Soft robots exhibit complex nonlinear dynamics with large degrees of freedom, making their modeling and control challenging. Typically, reducedorder models in time or space are used due to these challenges. However, such simplified models limit control accuracy and restrict the motion repertoire of the soft robots. In this work, we introduce an end-to-end learning-based approach for fully dynamic modeling of any general robotic system that do not rely on predefined structures. The key idea is to learn dynamic models of the robot directly in the visual space. The generated models possess identical dimensionality to the observation space, resulting in models whose complexity is determined by the sensory system without explicitly decomposing the problem. To validate the effectiveness of our proposed method, we apply it to a fully soft robotic manipulator. We demonstrate its applicability in controller development through an open-loop optimization-based controller. Our work thus far provides the most comprehensive strategy for controlling a general soft robotic system, without constraints on the shape, properties, and dimensionality of the system. This enables us to derive a wide range of dynamic control tasks including shape control, trajectory tracking and obstacle avoidance using a model derived from just 90 minutes of real-world data.