Generalization of Finger-Joint Kinematics for Cleaning Tasks

Clara Pham^1*Jan-Philipp Tauscher¹Colin Groth²Jochen J. Steil¹

• ¹Technical University of Braunschweig, Braunschweig, Germany
• ²New York University, New York, United States

Achieving robust, dexterous manipulation in unstructured environments remains a central challenge in robotics, particularly for continuous, contact-rich tasks like cleaning. While learning from demonstration using motion primitives has shown success, high dimensionality and the complex coupling of multi-finger kinematics often limit their effectiveness beyond simple grasping. This paper addresses this gap by adopting a data-driven framework for representing and reproducing dexterous manipulation trajectories, using cleaning motions as a test bed. While motion primitives can also be learned directly in full joint space, a compact, synergy-based representation provides a shared latent coordinate system that simplifies interpretation, modulation, and cross-task composition. We adopt a data-driven framework for representing and reproducing dexterous manipulation trajectories, using cleaning motions as a test bed. To model these movements, we combine Principal Component Analysis (PCA) with Probabilistic Movement Primitives (ProMPs), leveraging hand synergies. While the PCA and ProMP combination itself is established, our focus in this study, is on the cleaning use case and on the compositional generalization across tasks. PCA, applied in joint space, provides a compact, low-dimensional synergy space for coordinated finger movements, while the ProMPs encode the time-varying structure and variability of trajectories within this space. We first recorded a kinematic dataset of human cleaning motions with 20 degrees of freedom (DOF) haptic exoskeleton gloves across thirteen tasks and learn one ProMP per five selected training tasks in the PCA space. This dataset is then used as a basis to learn cleaning motions using the PCA+ProMPs. We demonstrate the ability of the learned primitives to reconstruct and reproduce kinematic patterns in simulation (Shadow Hand) and successfully deploy them on a physical robotic hand (Aeon Robotics). These results indicate that motion primitives, when grounded in synergy-informed coordinates, can generalize beyond grasping to encode and modulate contact-rich dexterous manipulation skills. Moreover, a library of the five task-specific ProMPs compositionally approximates trajectories from eight unseen cleaning tasks, with nearest-expert selection outperforming convex blends and Product-of-Experts combinations.