AUTHOR=Zhou Junjie , Chen Jiahao , Deng Hu , Qiao Hong 

TITLE=From Rough to Precise: Human-Inspired Phased Target Learning Framework for Redundant Musculoskeletal Systems

JOURNAL=Frontiers in Neurorobotics

VOLUME=Volume 13 - 2019

YEAR=2019

URL=https://www.frontiersin.org/journals/neurorobotics/articles/10.3389/fnbot.2019.00061

DOI=10.3389/fnbot.2019.00061

ISSN=1662-5218

ABSTRACT=Redundant muscles in human-like musculoskeletal robotics bring extra dimensions to solution space, which makes the computation of muscle excitation become an open question. Conventional methods like dynamic optimization and reinforcement learning usually have a high computational cost or an unstable learning process when they are applied into a complex musculoskeletal system. In this paper, inspired by the learning process of human, we build a phased target learning framework which provides different targets to learners from different levels to guide their training process and avoid local optima. By introducing an extra layer of neurons with preference, we improved the Q-network method to generate continuous excitation. In addition, based on the process of information transmission in human nervous system, two kinds of biological noises are designed in our algorithm to enhance exploration ability of our method in solution space. Tracking experiments based on a simplified musculoskeletal arm model indicate that under the guiding of phased targets, our method avoids divergence of excitation and obtains a stable training process. Besides, with its enhanced ability of exploration, our method shows smaller motion errors during the motion. Importantly, the phased target learning framework can be expanded as a general reinforcement learning framework, and it is a preliminary interpretation for modeling human motion learning manners.