Event Abstract

Back to Event

Unsupervised learning of simultaneous motor primitives through imitation

Olivier Mangin1* and Pierre-Yves Oudeyer1
  • 1 Institut National de Recherche en Informatique et Automatique, Bordeaux Sud Ouest, France

I. INTRODUCTION

We propose to build a system able to learn motor primitives from simultaneous demonstrations of several such primitives. Our approach is based on compact local descriptors of the motor trajectory similar to those used to learn acoustic words amongst sentences or objects inside visual scenes.
Learning by demonstration aims at making robots able to learn from human demonstrations of motor skills. Learning complex skills in a life-long perspective requires to recognize and re-use common parts between those skills to avoid learning them over and over, but also to allow the complexity of learned skills to increase.
A challenge of learning by demonstration is to make it accessible to non-expert public which would not adapt to complex requirement such as the decomposition of each demonstration in low level building blocks.

A. Combinations of motor primitives

Motor primitives have been introduced as a form of such preliminary knowledge, and may be used as elementary building blocks for more complex motor control and skills. They can both be found in biological and robotic systems, and can be either innate or aquired [1].
The main function of an aquired motor primitives is to make motor knowledge and skills re-usable, in a modular way, and to provide high level motor command instead of a trajectory level of control.
The notion of combination of motor primitives can take different forms depending both on the kind of combination one wants to achieve and on the definition and representation one has in mind for the concept of motor or sensori-motor primitives. For example, it is possible to combine alternative primitives in a context dependent manner, to compose them as functions taking inputs and producing outputs, to stack them in successive time sequences, or to treat them as constraints that can be satisfied in a parallel, competing or subordinate manner.

B. Related work achieving combination of motor primitives

Primitives combined in an alternative way, often called ex- perts, are present in Gaussian mixture encoding of movement [2], and more recently on a higher level of complexity, in work from Grollman et al. [3]. Calinon and Billard have shown [4] that it is possible to produce action from simultaneously active motor primitives, represented in Gaussian mixture framework, both for the competing and subordinate combination.
Time sequences of motor primitives are studied in the following, very different approaches: as time signals by Li et al. [5], using dictionary learning techniques, by using the dependency relationships between successive motor primitives [6] , and as an extension of the reinforcement learning frame- work by Sutton et al. in the option framework [7] . Again, those works does not handle the kind of simultaneous primitive we are interested in.

II. EXPERIMENTAL SETUP

As stated in previous section, very few work tries to learn the motor primitives when they are simultaneously demon- strated. A first step to adress this issue is to learn movements that are only active on certain degrees of freedom of the system, for example independent movements on different limbs. We thus propose an approache to tackle this issue.

A. Problem settlement

We are interested in learning motor primitives that are hap- pening simultaneously in demonstrations and that we can, in first approximation consider as independent. Such a situation is present in dancing movements: choregraphies are composed of elementary postures and transitions that happen independently on different limbs, in the same way that different instruments simultaneously take part in an orchestra.
We consider a two arm robot and two sets of movements, associated respectively to left and right arm. We provide the system with demonstrations each composed of one left arm movement and one right arm movement, simultaneously executed. Movement pairs are chosen randomly.
Our objective is to make the system able to learn mo- tor primitives, and use them to represent demonstrations as pairs of learned primitives, instead of learning each particular demonstration in a flat manner. For a sufficient number or primitives, the former achieves better compression than the latter.

B. Movement representation and features

Movements are represented as time sequences of positions. However, by introducing some time related features such as velocities along with the position vector, we obtain a sequence of vectors from a new feature space, whose samples may
be approximated as independent. In such an approximation movement may be represented by the probability distribution on those samples.
However trying to represent this distribution over the whole feature space lead to high dimensional descriptors, heavy to store and compare. We thus make a further approximation by considering separate distributions on each degree of freedom, and we represent those distribution by simple histograms.
This approximation is not meant to provide a complete representation of a motor skill but to allow efficient and cheap discriminative representation of motor skills. Indeed it is easier to efficiently represent a motor skill when relevant degrees of freedom to this motor skill are known. However one needs an efficient method to simultaneously discover both the primitives and their relevant degrees of freedom.
In the following we build feature vectors by concatenating position histograms on the different joints or degrees of freedom of our system.

C. Unsupervised learning of motor primitives with non- negative matrix factorization

Histograms introduced in previous section, and their combi- nations in the case of simultaneous primitives fit well the ad- ditive properties required by non-negative matrix factorization (NMF) [8], which is an efficient technique to discover non- negative composants of a signal in an unsupervised scenario.
NMF takes as input a data matrix X of dimension n × p where n is the number of demonstrations, and p the dimension of our fearures space. Those data are assumed to be non- negative, which is true for the histogram features. Here p is the sum of resolution of histograms, wich typicaly is DOF × resolution.
Given a parameter k, NMF then provides two non-negative matrices W and H, of dimension respectively n × k and k × p, such that X ≃ W · H. Lines of H provide a basis of prototypical elements of the data, that is to say some kind of motor primitives. The coefficients of W are then interpreted as the degree of activity of those primitives in the demonstrations.

D. Preliminary results

We use a database of movements captured on two six degrees of freedom robotic arms. 5 different movements have been associated to each arm, with 20 demonstration of pairs of thos movements. Data is acquired through joint position sensors integrated to motors, at a frequency of 50Hz.
The obtained database of one hundred pairs of examples is then presented to the system, which has to discover elementary histograms corresponding to motor primitves, without being given the labels.
In a first experiment we use the known number of primitives as the parameter of the NMF algorithm, in order to compare discovered primitives and real primitives.

Figure 1
Image

Acknowledgements

This research project was partially funded by ERC Starting Grant EXPLORERS 240007.

References

[1] J.Konczak,“Onthenotionofmotorprimitivesinhumansandrobots,”in
Fifth International Workshop on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems, vol. 123. Lund University Cognitive Studies, 2005, pp. 47–53.
[2] S. Calinon, F. Guenter, and A. Billard, “On learning, representing, and generalizing a task in a humanoid robot.” IEEE transactions on systems, man, and cybernetics. Part B, Cybernetics : a publication of the IEEE Systems, Man, and Cybernetics Society, vol. 37, no. 2, pp. 286–98, Apr. 2007.
[3] D. H. Grollman and O. C. Jenkins, “Incremental Learning of Subtasks from Unsegmented Demonstration,” in IROS, Taipei, Taiwan, 2010. [4] S. Calinon and A. G. Billard, “Statistical Learning by Imitation of Com-
peting Constraints in Joint Space and Task Space,” Advanced Robotics,
vol. 23, pp. 2059–2076, 2009. [5] Y. Li, C. Fermuller, Y. Aloimonos, and H. Ji, “Learning shift-invariant
sparse representation of actions,” in 2010 IEEE Computer Society Con- ference on Computer Vision and Pattern Recognition. San-Francisco: IEEE, Jun. 2010, pp. 2630–2637.
[6] M. Pardowitz, S. Knoop, R. Dillmann, and R. D. Zo ̈llner, “Incremental learning of tasks from user demonstrations, past experiences, and vocal comments.” IEEE transactions on systems, man, and cybernetics. Part B, Cybernetics : a publication of the IEEE Systems, Man, and Cybernetics Society, vol. 37, no. 2, pp. 322–32, Apr. 2007.
[7] R. S. Sutton, D. Precup, and S. Singh, “Between MDPs and semi- MDPs: A framework for temporal abstraction in reinforcement learning,” Artificial intelligence, vol. 112, no. 1, pp. 181–211, 1999.
[8] D.D.LeeandH.S.Seung,“Learningthepartsofobjectsbynon-negative matrix factorization.” Nature, vol. 401, no. 6755, pp. 788–91, Oct. 1999

Keywords: Developmental learning, Learning by demonstration, non-negative matrix factorization, Social learning, Task learning

Conference: IEEE ICDL-EPIROB 2011, Frankfurt, Germany, 24 Aug - 27 Aug, 2011.

Presentation Type: Poster Presentation

Topic: Social development

Citation: Mangin O and Oudeyer P (2011). Unsupervised learning of simultaneous motor primitives through imitation. Front. Comput. Neurosci. Conference Abstract: IEEE ICDL-EPIROB 2011. doi: 10.3389/conf.fncom.2011.52.00004

Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.

The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.

Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.

For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.

Received: 20 Jun 2011; Published Online: 12 Jul 2011.

* Correspondence: Mr. Olivier Mangin, Institut National de Recherche en Informatique et Automatique, Bordeaux Sud Ouest, Talence, Cedex, 33405, France, olivier.mangin@yale.edu