AUTHOR=Sakai Kurima , Minato Takashi , Ishi Carlos T. , Ishiguro Hiroshi 

TITLE=Novel Speech Motion Generation by Modeling Dynamics of Human Speech Production

JOURNAL=Frontiers in Robotics and AI

VOLUME=Volume 4 - 2017

YEAR=2017

URL=https://www.frontiersin.org/journals/robotics-and-ai/articles/10.3389/frobt.2017.00049

DOI=10.3389/frobt.2017.00049

ISSN=2296-9144

ABSTRACT=We have developed a method to automatically generate humanlike trunk motions based on speech (i.e., the neck and waist motions involved in speech) for a conversational android from its speech in real time.
To generate humanlike movements, a mechanical limitation of the android (i.e., limited number of joint) needs to be compensated in order to express emotional states through motion.
By enforcing the synchronization of speech and motion in the android, the method enables us to compensate for its mechanical limitations.
%By expressly presenting the synchronization of speech and motion in the android, the method enables us to compensate for its mechanical limitations.
Moreover, the motion can be modulated for expressing emotions by tuning the parameters in the dynamical model.
This method is based on a spring-damper dynamical model driven by voice features to simulate human trunk movement involved in speech.
In contrast to the existing methods based on machine learning, our system can easily modulate the motions generated due to speech patterns because the model's parameters correspond to muscle stiffness.
The experimental results show that the android motions generated by our model can be perceived as more natural and thus motivate users to talk longer with the android compared to a system that simply copies human motions.
In addition, it is possible to make the model generate emotional speech motions by tuning its parameters.