Event Abstract

Modelling of the McGurk effect

  • 1 University of Surrey, Department of Computing, United Kingdom

The McGurk effect [1] is a perceptual illusion that occurs when a visual input of lip movements, such as /ga/, is combined with a different audio input, such as /ba/. For the incongruent input, people perceive a different sound from both visual and audio input, such as /da/.
Studies on speech perception show that lip reading activates the primary cortex [2], suggesting the importance of visual speech articulation in speech perception. Other studies illustrate that the auditory-visual speech perception is influenced by the listener’s phonological repertoire [3].
The current study investigates the McGurk effect by modelling with a neural network. A previous model of the McGurk effect used a symmetric feed-forward neural network [4], with two identical sub-systems corresponding to the auditory and visual processing systems, that receive the same information, the end-product of phonological interpretation.
In our approach, the simulation takes into account the influence of noise, one channel speech perception and language on auditory-visual speech perception. Several neural network structures have been used. The input and output patterns for the neural network replicate the phoneme and the viseme, the basic unit of speech in the visual domain. The results of the simulation show that noise does influence the appearance of the McGurk effect.

References

1. McGurk et al. 1976. Nature 264,746-748.

2. Pekkola et al. 2005. NeuroReport 16,125-128.

3. Kaoru Sekiyama et al. 1991. The Journal of the Acoustical Society of America 90,1797 -1805.

4. Skouras 2006. Proceeding of The 7th International Conference on Cognitive Modeling, 286-291.

Conference: 41st European Brain and Behaviour Society Meeting, Rhodes Island, Greece, 13 Sep - 18 Sep, 2009.

Presentation Type: Poster Presentation

Topic: Poster presentations

Citation: Sporea I and Gruning A (2009). Modelling of the McGurk effect. Front. Behav. Neurosci. Conference Abstract: 41st European Brain and Behaviour Society Meeting. doi: 10.3389/conf.neuro.08.2009.09.307

Received: 15 Jun 2009; Published Online: 15 Jun 2009.

* Correspondence: Ioana Sporea, University of Surrey, Department of Computing, Guildford, United Kingdom, i.nica@surrey.ac.uk

© 2007 - 2018 Frontiers Media S.A. All Rights Reserved