Visuo-dynamic self-modelling of soft robotic systems Provisionally Accepted

Richard M. Monteiro¹ Jialei Shi² Helge A. Wurdemann² Fumiya Iida¹ Thomas George Thuruthel^2*

• ¹University of Cambridge, United Kingdom
• ²University College London, United Kingdom

Soft robots exhibit complex nonlinear dynamics with large degrees of freedom, making their modeling and control challenging. Typically, reducedorder models in time or space are used due to these challenges. However, such simplified models limit control accuracy and restrict the motion repertoire of the soft robots. In this work, we introduce an end-to-end learning-based approach for fully dynamic modeling of any general robotic system that do not rely on predefined structures. The key idea is to learn dynamic models of the robot directly in the visual space. The generated models possess identical dimensionality to the observation space, resulting in models whose complexity is determined by the sensory system without explicitly decomposing the problem. To validate the effectiveness of our proposed method, we apply it to a fully soft robotic manipulator. We demonstrate its applicability in controller development through an open-loop optimization-based controller. Our work thus far provides the most comprehensive strategy for controlling a general soft robotic system, without constraints on the shape, properties, and dimensionality of the system. This enables us to derive a wide range of dynamic control tasks including shape control, trajectory tracking and obstacle avoidance using a model derived from just 90 minutes of real-world data.