Impact Factor 3.648
2018 JCR, Web of Science Group 2019

Frontiers journals are at the top of citation and impact metrics


Front. Neurosci., 01 August 2008 |

Merging of the senses

  • Department of Neurobiology, Duke University, USA; Department of Biomedical Engineering, USA; Center for Neuroengineering, Duke University, USA

When we are constantly and simultaneously bombarded with various types of sensory inputs, how do we make decisions or motor outputs to deal with the world in a meaningful manner? This is a central issue in multisensory integration. One example of such type of processing is eating, as placing food in one’s mouth provides taste, olfactory and somatosensory (texture, temperature) information (Verhagen and Engelen, 2006 , Simon et al., 2006 ). Indeed, virtually all combinations of the senses are multimodal (Verhagen and Engelen, 2006 ). Whatever the combination of modalities, information from widely separated areas is rapidly integrated to give a motor response. In the case of taste the response to swallow or reject is yielded. One possibility that our brains have evolved to deal with multisensory stimuli could be due to the combined response reveals more about the nature of the external event. Our grand challenge is to know how information from different modalities that originally seem to be processed as unimodal events is combined during development to produce a coherent percept. This is known as the “binding problem” (Singer, 2001 ) and is important to all multisensory processing.

Many issues regarding multisensory processing are found in a very recent and excellent review by Stein and Stanford (Stein and Stranford, 2008 ), and given the space limitations, I will not repeat them but rather give my opinion as to what is missing and what experiments would lead to a better understanding.

It has been proposed that the entire neocortex is multisensory (Ghazanfar and Schroeder, 2006 ) and, although there has been some controversy about this proposal, it would be useful to have an atlas of the multisensory specificity of all subcortical and cortical areas.

In electrophysiological studies of multisensory integration most recordings have been from single units, and confined to a single area. No doubt these recordings have lead to many advances and the formulation of rules that govern multisensory processing (e.g. superadditivity (Stein and Stranford, 2008 ), but it would be more informative if ensembles of neurons could be recorded in different areas while animals are being tested. This could be done either with implanted arrays of electrodes or with optical imaging (Nicolelis, 2007 ; Gradinaru, 2007 ). In this manner we will learn how different neurons in the same and different areas interact (synchronously or asynchronously) at different time scales. Such details can be use to determine which area signified the event (say congruent or non-congruent), even on a single trial (Nicolelis, 2007 ). We will also understand how the temporal structure of the neuronal dynamics provides information as to the behavioal task. The new technique involving the reversible use of light-induced activation or inactivation (Gradinaru, 2007 ) of neuronal ensembles in a given area will enable researchers to directly determine how the input of one modality affects the other modalitities and to determine whether multisensory information is feedforward and/or arises from top-down influences. This method may also permit researchers to begin to understand how animals determine saliency of cues to the extent as to which sensory modality yields most information. With these advances in animal research together with fMRI, EEG and evoked potential experiments in humans much progress will be made towards achieving our grand challenge – namely, to understand how the many facets of the sensory world around us becomes one coherent percept in our brain.


Verhagen, J. V., Engelen, L. (2006). The neurocognitive bases of human multimodal food perception: Sensory integration. Neuroscience and Biobehavioral Reviews 30: 613–650

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Simon, S. A., de Araujo, I. E., Gutierrez, R., Nicolelis, M. A. (2006). The neural mechanisms of gustation: a distributed processing code. Nature Reviews Neurosci. 7, 890–901.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Stein, B. E., Stranford, T. R. (2008). Multisensory integration: current issues from the perspective of the single neuron. Nature Reviews Neuroscience 9, 255–266.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Singer, W. (2001). Consciousness and the binding problem. Ann. N.Y. Acad. Sci. 929, 123–146.

Pubmed Abstract | Pubmed Full Text

Ghazanfar, A. A. and Schroeder, C. E. (2006). Is neocortex essentially multisensory? Trends Cogn. Sci. 10, 278–285.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Nicolelis, M. A. L. (2007). Methods for Neural Ensemble Recordings. (Frontiers in Neuroscience, Series) CRC Press Inc; 2Rev Ed edition, 296 pages.

Gradinaru, V., Thompson, K. R., Zhang, F., Mogri, M., Kay, K., Schneider, M. B., Deisseroth, K. (2007). Targeting and readout strategies for fast optical neural control in vitro and in vivo. J. Neuroscience. 27, 14231–14238.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Citation: Simon SA (2008) Merging of the senses. Front. Neurosci. 2:19. 13–14. doi:10.3389/neuro.01.019.2008

Copyright: © 2008 Simon. This is an openaccess publication subject to an exclusive license agreement between the authors and the Frontiers Research Foundation, which permits unrestricted use, distribution, and reproduction in any medium, provided the original authors and source are credited.

Correspondence: Sidney A. Simon, Department of Neurobiology, Duke University, USA; Department of Biomedical Engineering, USA; Center for Neuroengineering, Duke University, USA.