Event Abstract

Back to Event

Experience-based Learning Mechanism with a Concept of Vigilance

John Nassour1, 2*, Patrick Henaff3, Fethi Benouezdou1 and Gordon Cheng2
  • 1 Versailles Univercity, France
  • 2 TU Munich, Germany
  • 3 Cergy-Pointoise Univercity, France

Cognitive Neuroscience studies have identified an early warning system in the human brain that can avoid to make past mistakes again. They have shown how the brain remembers details about past dangers [1]. An activity was found in the Anterior Cingulate Cortex (ACC) after making mistakes [2]. This cortex area works as an early warning system that adjusts its behaviour to avoid dangerous situations. It responds not only to the sources of errors (external error feedback), but also to the earliest sources of error information available (internal error detection) [3]. It becomes active in proportion to the occurrence likelihood of an error [4]. Therefore, it can learn to identify situations where humans may make mistakes, and then help avoid such situations to occur [2]. It learns to predict error likelihood even for situations where no error occurs previously. Through the observation of particular areas located in cerebral cortex that has been shown to be responsible for cognitive control. Neuropsychological studies demonstrated a switching in human learning strategies around the age of twelve years. This switching, goes from learning with positive feedback to learning with negative feedback -- probably comes from the combination of brain maturing and experience [5]. we have produced an early warning mechanism that can help avoid repeating past errors in the generation of bipedal motion patterns for a humanoid robot to achieve robust walking. The objectives of this learning mechanism is to adapt parameters of a low-level controller. In detecting its domain of viability, which increases adaptation to external perturbations [6][7].
We specified by the state space “V” of those intrinsic parameters. The mechanism must be able to learn from negative feedback (failure) and positive feedback (success). Therefore, it must have experience with success and other with failure within the state space “V”. As each vector “v” from “V” leads to either success or failure, the mechanism will evaluate whether this vector belongs to the success domain or to the failure domain. The decision mechanism (“go”, “nogo”) works as an early warning system similar to that in ACC [2].
Psychological studies suggest that some people are more tolerant to risk than others who are more cautious, [8]. The vigilance is related to human learning approaches and decision making. In the standard psychological assessment of risk taking, people are classed as risk seeking or risk averse [9].
In our study the vigilance is represented by a threshold that is used to adjust the early warning signal in the decision mechanism. This threshold describes the tolerance of risk. According to vigilance threshold, we can distinguish between two different behaviors for the system, risk taking and risk averse. Thanks to the two behaviors the system can gain experience in walking, and in case of risky behavior the system learns better with more failed trials. Changing vigilance in learning phases between trials will change the behavior of the system to risks.

Figure 1
Image

References

[1] T. Singer, B. Seymour, J. O’Doherty, H. Kaube, R. J. Dolan, and C. D. Frith. Empathy for pain involves the affective but not sensory components of pain. Science, 303(5661): 1157– 1162, 2004.
[2] J. W. Brown and T. S. Braver. A computational model of risk, conflict, and individual difference effects in the anterior cingulate cortex. Brain Research, 1202(5661): 99– 108, 2008.
[3] R. B. Mars, M. G. Coles, M. J. Grol, C. B. Holroyd, S. Nieuwenhuis, W. Hulstijn, and I. Toni. Neural dynamics of error processing in medial frontal cortex. NeuroImage, 28(4): 1007– 1013, 2005.
[4] H. Gemba, K. Sasaki, and V. B. Brooks. Error potentials in limbic cortex ( anterior cingulate area 24) of monkeys during motor learning. Neuroscience letters, 70(2): 223– 227, 1986.
[5] L. Van Leijenhorst, P. M. Westenberg, and E. A. Crone. A developmental study of risky decisions on the cake gambling task: Age and gender analyses of probability estimation and reward evaluation. Developmental Neuropsychology, 33(2): 179– 196, 2008.
[6] Nassour, J., Hénaff, P., Ouezdou, F. B., and Cheng, G. The ieee/rsj international conference on intelligent robots and systems. st.Louis, MO, USA. In Experience- based learning mechanism for neural controller adaptation: Application to walking biped robots, pages 2616– 2621, 2009.
[7] Nassour, J., Hénaff, P., Ouezdou, F. B., and Cheng, G. A study of adaptive locomotive behaviors of a biped robot: Patterns generation and classification. In From Animals to Animats 11, volume 6226 of Lecture Notes in Computer Science, pages 313– 324. Springer Berlin / Heidelberg. 2010.
[8] J. L. van Gelder, R. E. de Vries, and J. van der Plight. Evaluating a dual- process model of risk: affect and cognition as determinants of risky choice. Behavioral Decision Making, 22(1): 45– 61, 2008.
[9] D. J. Kruger, X. T. Wang, and A. Wilke. Towards the development of an evolutionarily valid domain- specific risk- taking scale. Evolutionary Psychology, 5(3): 555– 568, 2007.

Keywords: anterior cingulate cortex, biped locomotion, Learning

Conference: BC11 : Computational Neuroscience & Neurotechnology Bernstein Conference & Neurex Annual Meeting 2011, Freiburg, Germany, 4 Oct - 6 Oct, 2011.

Presentation Type: Abstract

Topic: learning and plasticity (please use "learning and plasticity" as keyword)

Citation: Nassour J, Henaff P, Benouezdou F and Cheng G (2011). Experience-based Learning Mechanism with a Concept of Vigilance. Front. Comput. Neurosci. Conference Abstract: BC11 : Computational Neuroscience & Neurotechnology Bernstein Conference & Neurex Annual Meeting 2011. doi: 10.3389/conf.fncom.2011.53.00099

Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.

The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.

Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.

For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.

Received: 23 Aug 2011; Published Online: 04 Oct 2011.

* Correspondence: Mr. John Nassour, Versailles Univercity, velizy, France, john.nassour@informatik.tu-chemnitz.de