Addressing Data Imbalance in Sim2Real: ImbalSim2Real scheme and its application in Finger Joint Stiffness Self-sensing for Soft-Robot-Assisted Rehabilitation Provisionally Accepted

Zhou Zhongchao¹ Yuxi Lu² Pablo E. Tortós² Ruian Qin² Shota Kokubu² fuko matsunaga² Qiaolian Xie³ Wenwei Yu^4*

• ¹Department of Medical System Engineering, Chiba University, Japan
• ²Department of Medical System Engineering, Faculty of Engineering, Chiba University, Japan
• ³Institute of rehabilitation engineering and technology, University of Shanghai for Science and Technology, China
• ⁴Center for Frontier Medical Engineering, Chiba University, Japan

The simulation-to-reality (sim2real) problem is a common issue when deploying simulation-trained models to real-world scenarios, especially given the extremely high imbalance between simulation and real-world data (scarce real-world data). Although the Cycle-Consistent Generative Adversarial Networks (CycleGAN) has demonstrated promise in addressing some sim2real issues, it encounters limitations in situations of data imbalance due to lower capacity of the discriminator and the indeterminacy of learned sim2real mapping. To overcome such problems, we proposed the Imbalanced Sim2Real scheme (ImbalSim2Real). Differing from the CycleGAN, the ImbalSim2Real scheme segments the dataset into paired and unpaired data for two-fold training. The unpaired data incorporated Discriminator-enhanced Samples to further squash the solution space of the discriminator, for enhancing the discriminator's ability. For paired data, a term Targeted Regression Loss was integrated to ensure specific and quantitative mapping and further minimize the solution space of the generator. The ImbalSim2Real scheme was validated through numerical experiments, demonstrating its superiority over conventional sim2real methods. In addition, as an application of the proposed ImbalSim2Real scheme, we designed a finger joint stiffness self-sensing framework, where the validation loss for estimating real-world finger joint stiffness was reduced by roughly 41% compared to the supervised learning method that was trained with scarce real-world data and by 56% relative to the CycleGAN trained with the imbalanced dataset. Our proposed scheme and framework have potential applicability to bio-signal estimation when facing an imbalanced sim2real problem.