- 1 Department of Psychology, University of Milano–Bicocca, Milan, Italy
- 2 Neuropsychological Laboratory, Istituto di Recovero e Cura a Carattere Scientifico Istituto Auxologico Italiano, Milan, Italy
Most of current knowledge about the mechanisms of multisensory integration of environmental stimuli by the human brain derives from neuroimaging experiments. However, neuroimaging studies do not always provide conclusive evidence about the causal role of a given area for multisensory interactions, since these techniques can mainly derive correlations between brain activations and behavior. Conversely, techniques of non-invasive brain stimulation (NIBS) represent a unique and powerful approach to inform models of causal relations between specific brain regions and individual cognitive and perceptual functions. Although NIBS has been widely used in cognitive neuroscience, its use in the study of multisensory processing in the human brain appears a quite novel field of research. In this paper, we review and discuss recent studies that have used two techniques of NIBS, namely transcranial magnetic stimulation and transcranial direct current stimulation, for investigating the causal involvement of unisensory and heteromodal cortical areas in multisensory processing, the effects of multisensory cues on cortical excitability in unisensory areas, and the putative functional connections among different cortical areas subserving multisensory interactions. The emerging view is that NIBS is an essential tool available to neuroscientists seeking for causal relationships between a given area or network and multisensory processes. With its already large and fast increasing usage, future work using NIBS in isolation, as well as in conjunction with different neuroimaging techniques, could substantially improve our understanding of multisensory processing in the human brain.
Introduction
Although multisensory integration has been widely investigated in animals (Stein and Stanford, 2008) and humans (Calvert, 2001; Driver and Noesselt, 2008; Macaluso and Maravita, 2010), only recent research work has started to gain deeper knowledge into the causal involvement of different brain regions, thanks to non-invasive brain stimulation (NIBS) techniques.
In the present paper we will first give an overview on the available techniques of NIBS as valid means for modulating brain activity. Then, we will review research that implied the use of transcranial magnetic stimulation (TMS) to clarify basic aspects of multisensory integration. We will show how single-pulse TMS (sTMS) critically reveals multisensory influence on visual cortical excitability. Second, we will discuss instances in which repetitive TMS (rTMS) was used to induce a temporary interference with brain activity, with the aim of exploring the causal role of heteromodal or unisensory regions in multisensory integration. Finally, we will address the importance of transcranial direct current stimulation (tDCS) for improving multisensory processing.
Non-Invasive Brain Stimulation: Basic Aspects
There are two main techniques of NIBS, namely TMS and tDCS. Both TMS and tDCS appear to be attractive tools for the study of multisensory interactions in the human brain, given that they are non-invasive and safe methods to effectively modulate sensory processing in the cortex. The present paper briefly illustrates the basic technical features of these two techniques (for comprehensive descriptions of NIBS see Pascual-Leone et al., 2000; Priori, 2003; Nitsche et al., 2008; Wassermann et al., 2008; Zaghi et al., 2009; Bolognini and Ro, 2010; Sandrini et al., 2010).
Transcranial magnetic stimulation relies upon the properties of electromagnetic induction; a rapidly changing magnetic field is generated when a high voltage current is passed through a coil. When this coil is held in close proximity to any electrically conducting medium, such as the brain, this time-varying magnetic field induces electrical current that interferes with normal neuronal activity, hence temporarily altering the function of underlying brain areas (see Wassermann et al., 2008, for a comprehensive overview of the physiological mechanisms of TMS). In cognitive studies, TMS has been generally applied with the aim of causing a transient and reversible disruption of cortical activity, and the TMS data have traditionally been interpreted in the “virtual lesion” framework (Pascual-Leone et al., 2000; Walsh and Pascual-Leone, 2003). This terminology was proposed by analogy with neuropsychological and animal lesion studies (Miniussi et al., 2009) and, consistent with this interpretation, TMS has been used in order to define the putative causal role of different cortical areas during the execution of given cognitive and perceptual tasks. However, the interpretation of TMS effects and the establishment of causal links between activity in the targeted brain area and a given behavioral effect is much more complex than suggested by the virtual lesion hypothesis (Miniussi et al., 2009). In fact, the functional effects of TMS can be interpreted only in the context of complex interactions between the characteristics of the stimulation, the anatomical/functional properties of the neural system, and its state of activation. Different mechanisms of action have been proposed by which TMS might interfere with ongoing brain activity and ultimately with behavior: e.g., the suppression of the relevant signal (Harris et al., 2008) or the addition of random neural noise to the ongoing processing (Silvanto and Muggleton, 2008; Ruzzoli et al., 2010; for other relevant studies that have challenged the virtual lesion view, see Siebner et al., 2009; Ziemann, 2010). Adopting a psychophysical approach appears essential to dissociate these two mechanisms and, in turn, to link in a specific way the effects of TMS on behavioral performance and its putative mechanisms of action at the neural level (Miniussi et al., 2009). Another confounding factor in interpreting TMS data is that the relative predominance of either behavioral facilitation or suppression is dependent on the initial activation state of cortical neurons (state-dependency). This state-dependency is critical since the neural impact of any external stimulus represents an interaction with the ongoing brain activity at the time of stimulation (Silvanto and Pascual-Leone, 2008; Silvanto et al., 2008).
The spatial resolution of TMS is highly dependent upon the shape of the stimulating coil, but it can be in the order of a few millimeters (e.g., when using figure-of-eight coil, with circular components of 45 mm), and focal enough to stimulate regions as small as individual fingers representations in the primary motor cortex (Ro et al., 1999).
The coil can be positioned over the brain either functionally, e.g., for motor cortex, searching for the area where the activation of the contralateral hand muscles is induced (Rossini et al., 1994), or by choosing known anatomical landmarks, e.g., the primary somatosensory area of the hand is located at about 3 cm posterior to the motor hotspot for the contralateral hand (Bolognini and Maravita, 2007). With frameless stereotaxic systems, the TMS coil can be navigated more precisely to target specific anatomical sites based on individual subjects’ structural (Paus, 1999; Sparing et al., 2010) and functional (Sack et al., 2009) brain images.
The temporal resolution of TMS depends upon the employed stimulation parameters. When sTMS is used, the temporal resolution can be very high and can provide information about brain function in the order of milliseconds, thus allowing the assessment of chronometry of cortical processing (Pascual-Leone et al., 2000; Bolognini and Ro, 2010). If one does not have a temporal hypothesis about when to deliver a single pulse, a different approach would be to use rTMS, which consists of the application of rhythmic trains of multiple TMS pulses. When using rTMS, stimulation frequency seems to be the key parameter determining the direction of the effects. From a physiological point of view, low-frequency rTMS (≤1 Hz) usually results in decreased cortical excitability, whereas at higher frequencies (>1 Hz) cortical excitability is usually increased (Wassermann et al., 2008; Ziemann et al., 2008). Typically, high-frequency rTMS protocols are applied either as a single short train of pulses or several trains with different inter-train intervals, while low-frequency rTMS is typically given as a prolonged continuous stimulation (Bolognini and Ro, 2010; Sandrini et al., 2010). Noteworthy, as mentioned earlier, the effects of rTMS (i.e., facilitation versus inhibition) are not uniquely related to pulse frequency, but they also depend on the initial state of the stimulated brain region (Silvanto and Pascual-Leone, 2008).
There are two main protocols for delivering rTMS. During the “on-line” approach, subjects perform the task and, at stimulus presentation, or with a specific interval preceding or following it, a train of TMS pulses is given to a particular area of the brain. Another popular approach is to stimulate at the site of interest for some minutes before starting a cognitive task (the so-called “off-line” rTMS protocol). Indeed, a crucial feature of rTMS is that it seems capable of changing the activity in a brain area even beyond the duration of the rTMS application itself (Bolognini and Ro, 2010; Sandrini et al., 2010).
A more recent, alternative protocol is “theta burst TMS,” in which short bursts of 50-Hz rTMS are repeated at a rate in the theta range (5 Hz). Inhibitory or excitatory effects of this type of stimulation can be obtained using both continuous or intermittent delivery of theta bursts, according to the stimulation parameters (Huang et al., 2005).
Finally, a new emerging paradigm is the use of rhythmic TMS at frequencies mimicking brain rhythms recorded through electroencephalography (EEG) for the study of brain oscillations (Sauseng et al., 2009; Thut and Miniussi, 2009; Miniussi and Thut, 2010; Romei et al., 2010). There is much correlational evidence showing that activity in specific frequency bands is linked to specific cognitive functions. Hence, rhythmic TMS provides the opportunity to entrain brain oscillations. Then, if behavioral modulation is concurrently observed, one can draw causal links between synchronization in functional networks and specific aspects of a task (Miniussi and Thut, 2010). In this context, EEG can be used to study how TMS interacts with rhythmic brain activity, and/or how rhythmic brain stimulation can be used to modify brain functions.
Through TMS is possible not only to modulate the neural activity at the site of stimulation, but also reveal the functional connectivity between different cortical areas by means of paired pulse TMS (Koch and Rothwell, 2009). In this paradigm, two TMS pulses are delivered by two separate coils. A conditioning stimulus is delivered to a brain site, followed by a test stimulus on a different site, on the same or opposite hemisphere. What is measured is the effect of the conditioning stimulus on the response to the test stimulus and, depending on the intensity of the conditioning stimulus and the interstimulus interval, both facilitation and inhibition can be obtained (Ziemann et al., 1996). The effect of the conditioning pulse can change during the execution of a given task, thus providing clues on the causal role of putative intracortical connections over time for that specific task.
The other main method of NIBS, tDCS, is a form of brain polarization that uses a prolonged low-intensity electric current (1–2 mA), delivered to the scalp through two large electrodes (usually 5 cm × 7 cm; Nitsche et al., 2008). tDCS has been used in humans since the distant past, but a reappraisal of this technique with claims of behavioral effects and clinical benefit took place at the turn of this century (for an historical overview, see Priori, 2003). tDCS is now becoming an attractive tool for cognitive neuroscientists, especially in the context of modulating cortical excitability to facilitate skill acquisition, learning, and neural plasticity in the human brain (Wassermann and Grafman, 2005; Nitsche et al., 2008; Bolognini et al., 2009b).
Transcranial direct current stimulation can up- or down-regulate neural activity in the stimulated regions. Increased excitability of the underlying neurons occurs with anodal stimulation, while decreased excitability is seen after cathodal stimulation. With only 13 min of tDCS stimulation, effects on neural excitability outlast the period of stimulation by up to 90 min (Nitsche and Paulus, 2001). tDCS does not stimulate neurons directly, thus increasing their firing, as TMS does. Rather, it most likely targets neuronal signaling by influencing the permeability of ion channels or shifting electrical gradients, therefore modulating the resting membrane threshold (Ardolino et al., 2005). Chemical neurotransmission, either pre- or post-synaptic, may also play a role in tDCS effects (Liebetanz et al., 2002). tDCS has some advantages over TMS, such as its safer and easier application, and the possibility of applying a sham stimulation in a truly double-blind fashion. On the other hand, tDCS has low spatial resolution as compared to TMS, especially when using focal figure-of-eight coil. Yet, computer-based modeling studies of tDCS indicate that maximum current density magnitudes are located beneath the electrodes, at the cortical level (Wagner et al., 2007). Accordingly, despite the very weak current used, DC polarization delivered to specific cortical areas can alter physiological, perceptual, and higher-order cognitive processes in a pretty selective way (Priori, 2003; Wassermann and Grafman, 2005; Nitsche et al., 2008; Bolognini et al., 2009b; Zaghi et al., 2009).
Overall, TMS and tDCS are rapidly becoming essential tools available to neuroscientists for assessing brain functions. As we will illustrate in this review, TMS can be used to determine whether a brain area is causing some multisensory effects or to measure crossmodal changes in cortical excitability, hence nicely complementing other neuroimaging techniques, such as event-related potential (ERPs) and functional magnetic resonance imaging (fMRI). On the other hand, tDCS appears a valuable tool to modulate cortical excitability to facilitate crossmodal interactions, with potential long-lasting modulatory effects on multisensory perception.
Multisensory Influences on Visual Cortical Excitability as Revealed by TMS: Evidence for Crossmodal Interactions in Primary Visual Areas
Perception has traditionally been viewed as a modular function with the different sensory modalities operating largely as separate and independent processes. In this view, sensory information is believed to remain isolated by modality within primary sensory areas and the merging of sensory experience is the results of processing occurring within higher-order “heteromodal” associative areas of the brain (Calvert, 2001). However, recent evidence has forced to reconsider this oversimplification. Results from both anatomical and physiological studies suggest that crossmodal interactions occur not only within regions deemed heteromodal, but also within primary sensory areas, i.e., areas traditionally considered to be located very early along the cortical processing stream and receiving direct input from single sensory modalities (Schroeder and Foxe, 2005; Macaluso, 2006; Driver and Noesselt, 2008). This conclusion is strengthened by recent works investigating the crossmodal modulation of visual phosphenes induced by sTMS. The application of sTMS to the occipital visual areas in the human brain can elicit phosphenes, i.e., bright spots of light in specific regions of the visual field (Fernandez et al., 2002; McKeefry et al., 2009). Phosphenes are generated within coextensive regions of the cortex and could be induced by application of TMS to virtually all early visual areas, including the striate cortex (V1), extrastriate areas (V2/V3), and cortico-cortical tracts projecting from V2/V3 back to V1 (Kammer et al., 2005a,b). The features of phosphenes elicited by the stimulation of a given site seem to reflect the perceptual specialization of that area (McKeefry et al., 2009) For instance, while phosphenes elicited by TMS of V1 are stationary, those evoked by TMS of visual motion area V5/MT are often moving (Pascual-Leone et al., 2005).
The TMS output threshold needed to generate phosphenes is believed to provide a direct measure of the excitability of low-level visual cortex (Fernandez et al., 2002; Kammer et al., 2005a). For this reason, one can directly assess crossmodal effects in visual cortex by testing the changes of TMS intensity needed for inducing phosphenes, following a concurrent stimulation in another sensory modality. Using phosphenes, instead of presenting external visual stimuli, may have the advantage of testing crossmodal effects more directly, since phosphenes are generated by direct cortical stimulation, thus bypassing peripheral and subcortical pathways.
Using this approach, it has been shown that a peripheral somatosensory stimulus can modify visual cortex excitability in such a way that phosphenes perception can be now induced using a lower TMS intensity (Ramos-Estebanez et al., 2007). The facilitatory effect of touch over phosphene perception holds a high degree of spatial specificity, since the advantage occurs only for touches delivered exactly at the same spatial location where phosphenes are reported, following the stimulation of the contralateral visual cortex (Bolognini and Maravita, 2007; Ramos-Estebanez et al., 2007). This effect is in line with the crossmodal spatial congruency effects found using fMRI. For instance, Macaluso et al. (2000) demonstrated that a tactile stimulus enhances activity within unimodal visual cortical areas, but only when it is on the same side as the visual target (Macaluso et al., 2000). Moreover, thanks to the optimal temporal resolution of sTMS, the optimal temporal window for the tactile enhancement of phosphenes was found for somatosensory stimuli preceding the occipital sTMS by 60 ms (Ramos-Estebanez et al., 2007), in broadly agreement with ERP evidence for crossmodal tactile modulation of visual responses (Eimer, 2001).
Not only somatosensory, but even auditory inputs can boost visual cortical excitability. Romei et al. (2009) first showed that structured looming sounds, which were of a duration below the psychophysical discrimination threshold, considerably enhance phosphenes perception, as compared to other sound categories (e.g., sounds moving away from the observer or stationary). The onset of the enhancement effect started when TMS followed the auditory stimulus by 80 ms. These findings are of interest since they indicate that cortical excitability in low-level visual areas is rapidly and efficiently boosted by sounds through early, pre-perceptual, and stimulus-selective modulation of neuronal excitability (Romei et al., 2009). Subsequently, it was shown that: (1) the auditory enhancement of phosphenes perception requires a strict spatial correspondence between the two stimuli; (2) the effect of auditory signals occurs only for phosphenes generated in the peripheral visual field (>30°), but not in the central visual field; (3) audio–visual facilitation is present only at subthreshold TMS intensity for phosphenes induction, suggesting that crossmodal interactions depend on the relative physiological salience of the visual information (Bolognini et al., 2010c). Under these conditions, the auditory modulation of phosphenes was maximal when the auditory stimulus preceded the occipital TMS pulse by 40 ms. This timing is in agreement with ERPs recording in humans showing relatively early auditory influences on visual processing (<50 ms from stimulus onset; Giard and Peronnet, 1999; Molholm et al., 2002). Yet, under different experimental conditions, a slightly different latency was found, with an initial auditory modulation of phosphenes starting at 60–80 ms after the sound onset (Romei et al., 2007, 2009). A similar shift in time of audio–visual effects in early visual cortices was also described in a recent ERPs’ study by Cappe et al. (2010), who described multisensory interactions occurring at 60–95 ms after audio–visual stimuli. Multisensory effects were shown to be simultaneously originating within a network including primary auditory cortices, primary visual cortices, and posterior superior temporal regions (Cappe et al., 2010).
Overall, the available evidence indicates that auditory inputs can enhance visual cortical excitability over a longer time window, depending on the auditory stimulus selectivity, the perceptual gain, the temporal and spatial features of the combined stimuli. Noteworthy, Romei et al. (2007) also found that sTMS over the occipital pole produced opposing behavioral effects during a simple reaction time task to visual and auditory stimuli, with TMS slowing down reaction times to visual stimuli, but facilitating reaction times to auditory stimuli (Romei et al., 2009). This evidence indicates the existence of bidirectional influences between the stimulus-evoked auditory and TMS-induced visual cortex activities, suggesting state-dependent effects of TMS in the context of multisensory interactions.
Collectively, the above evidence of crossmodal modulation of phosphenes suggests the presence of early multisensory effects, i.e., somatosensory-driven or auditory-driven sensitivity changes in low-level visual areas. This is compatible with the existence of specific pathways linking specialized areas across sensory modalities. These sensory interactions can be revealed under subthreshold conditions and follow the principles of spatial and temporal congruency. The auditory or tactile input might be transmitted to unisensory visual areas through one of two possible pathways (Macaluso, 2006; Driver and Noesselt, 2008). The first pathway consists of a direct, feedforward projections from primary or associative auditory/somatosensory cortices to early visual areas (Falchier et al., 2002; Rockland and Ojima, 2003; Cappe and Barone, 2005; Schroeder and Foxe, 2005; Cappe et al., 2009). In this view, early crossmodal interactions could originate at the level of primary or secondary visual regions. Alternatively, an indirect pathway may be implicated in which feedforward auditory inputs reach areas of multisensory convergence (e.g., superior temporal polysensory region or posterior parietal cortex (PPC), respectively for auditory and tactile input) and are then transmitted via feedback connections to earlier unisensory visual areas (Jones and Powell, 1970; Meredith and Stein, 1983; Giard and Peronnet, 1999; Calvert, 2001; Macaluso, 2006; Meienbrock et al., 2007; Driver and Noesselt, 2008).
Guided by the available evidence from neurophysiologic studies in animals and brain-imaging studies in humans, the studies of crossmodal modulation of phosphenes appear a valuable approach to understand how crossmodal interactions alter visual neuronal excitability in the human low-level sensory cortices. On the other hand, phosphenes can be reported by subjects, but cannot be directly quantified by observers (Boroojerdi et al., 2002). Therefore the use of adequate psychophysical approaches seems important (e.g., Bolognini and Maravita, 2007; Romei et al., 2009). Hence, future work on crossmodal phosphenes might benefit from the combined used of neuroimaging techniques in order to measures the functional effects of the interactions between external stimulus-evoked activity and TMS-induced visual cortex activity.
TMS Disruption of Multisensory Processes
Transcranial magnetic stimulation allows the investigation of causality in the brain–behavior relationship, by temporarily altering the activity of neurons in brain areas that are underneath the magnetic field (Bolognini and Ro, 2010). In this context, an advantage of TMS over other neuroimaging methods is that TMS can be used to demonstrate that a brain region is causally essential for performing a given task. In this view, sTMS or rTMS have been used to establish whether a putative heteromodal area of the human brain is essential for multisensory processing.
Audio–Visual Interactions with Respect to Extrapersonal Space Representation and Speech Perception
The representation of the space around us is intrinsically multisensory. For instance, the occurrence of sensory signals from multiple sensory modalities, holding close spatial and temporal proximity, are typically integrated in an optimal way since they likely suggest a common source of stimulation (Stein, 1998). Behavioral studies have demonstrated many audio–visual effects in the extrapersonal space: multisensory cues can often either enhance our ability to detect or localize upcoming events (i.e., multisensory enhancement; Gielen et al., 1983; Hughes et al., 1998; Spence et al., 1998; Stein, 1998; Bolognini et al., 2005) or bias the localization of unisensory events (i.e., ventriloquism; Howard and Templeton, 1966; Hairston et al., 2003; Alais and Burr, 2004). As discussed above, these multisensory spatial effects may be subserved by converging information from sensory-specific cortices into multisensory areas, which in turn would affect early unisensory regions via feedback projections (Macaluso, 2006).
Starting from these considerations, TMS was used to explore the role of the PPC in audio–visual spatial interactions. The PPC is a heteromodal region of sensory convergence that contains both neurons responding to isolated visual and auditory stimuli (Bushara et al., 1999; Bremmer et al., 2001) and multisensory neurons (Andersen, 1997). These latter cells may be the ideal locus for multisensory integration, thus contributing to supramodal cognitive functions, such as spatial orienting and spatial awareness (Andersen, 1997). By applying an off-line rTMS paradigm (i.e., 20 min of 1-Hz rTMS, delivered before the task), we assessed whether the rTMS interference over the right inferior parietal lobule (IPL) could cause a disruption of modality-specific spatial orienting and/or a disturbance of the typical response speed advantage observed with crossmodal targets, as measured by the redundant target effect (Maravita et al., 2008). The results showed that low-frequency rTMS over IPL increased reaction times to spatially lateralized modality-specific visual and auditory stimuli, without affecting the response to bimodal audio–visual stimuli. Crucially, a residual advantage for multisensory stimuli, supported by a neural co-activation mechanism, was retained in spite of the parietal interference (Bolognini et al., 2009a). Control rTMS over V1 impaired only contralateral visual responses, without affecting the response to auditory or audio–visual targets. The modality-specific auditory and visual spatial deficits observed after IPL-rTMS could be due to the selective disruption of neighboring unisensory spatial representations for each of the two modalities within the IPL. Indeed, neuroimaging evidence shows modality-specific activations within multiple subregions of the PPC (Andersen, 1997; Bushara et al., 1999; Downar et al., 2000; Bremmer et al., 2001). On the other hand, the persistence of audio–visual speed advantage after disruption suggests that other structures beyond IPL may have compensated any local interference induced by rTMS (Maravita et al., 2008), supporting a still efficient integration of audio–visual inputs.
Bertini et al. (2010) investigated the neural substrate of audio–visual interactions subserving localization abilities by applying continuous theta burst TMS (cTBS, 40 s of cTBS consisting of bursts of three TMS pulses delivered at 50 Hz, with each train burst repeated every 200 ms) before an auditory localization task with unisensory auditory, and spatially congruent and incongruent audio–visual stimuli. The authors found that the visual enhancement of auditory localization induced by congruent audio–visual stimuli was disrupted by cTBS of the temporo-parietal cortex (TPC), whereas the ventriloquism effect (i.e., the perceptual translocation of the sound toward the visual stimulus during incongruent audio–visual stimulation) was reduced by cTBS of the occipital cortex (OC). Even in this task, cTBS of IPL did not affect audio–visual localization (Bertini et al., 2010). Such findings are in broad agreement with both neuroimaging (Calvert, 2001; Bonath et al., 2007) and neuropsychological evidence (Frassinetti et al., 2005; Leo et al., 2008). Unfortunately, the authors did not control for visual suppression effects during occipital cTBS, thus it remains to be clarified whether the reduction of the ventriloquism effect could be ascribed, partially or totally, to a mere reduction of visual sensitivity.
With respect to multisensory integration of speech-related stimuli, a compelling example of multisensory integration is the McGurk effect, in which an auditory syllable is perceived very differently depending on whether it is accompanied by a visual movie of a speaker pronouncing the same syllable or a different, incongruent syllable (McGurk and MacDonald, 1976). Anatomical and physiological studies in human and non-human primates have suggested that the superior temporal sulcus (STS) is involved in audio–visual integration for both speech and non-speech stimuli (Calvert, 2001; Beauchamp et al., 2004). Across three experiments, the causal role of STS in the occurrence of the McGurk effect was tested by Beauchamp et al. (2010) using fMRI-guided sTMS. The chronometry of STS involvement was also assessed by delivering sTMS at different time intervals, i.e., from 298 ms before to 362 ms after the auditory stimulus onset, with steps of 66 ms. The results showed that disruption of the left STS, but not of a control site located dorsally and posteriorly to the STS, significantly reduced the likelihood of the McGurk effect. Such an effect could not be ascribed to a more general impairment of speech perception, as assessed by using control stimuli. Moreover, the McGurk effect was reduced only when sTMS was delivered within a 200-ms time window, spanning 100 ms before to 100 ms after auditory stimulus presentation (Beauchamp et al., 2010). This time–effect is in line with previous electrophysiological evidence demonstrating strong responses in STS beginning around 100 ms after stimulus presentation (Schroeder and Foxe, 2002; Barraclough et al., 2005; Canolty et al., 2007; Puce et al., 2007).
Hence, studies of audio–visual interactions, despite the methodological differences among them, highlight a causal role of heteromodal regions around the temporal cortex, namely TPC for spatial audio–visual interactions, and STS for audio–visual speech-related processing. Some degree of crossmodal spatial interactions also emerged within primary visual areas, although this would benefit from further investigation (Falchier et al., 2002; Ghazanfar and Schroeder, 2006; Driver and Noesselt, 2008). Instead, IPL appears to process both visual and auditory spatial signals, but their integration seems to take place outside this cortical region, at least for relatively low-level orienting response. In the next future, the multisensory functions of other subregions of the PPC (e.g., superior parietal lobule, SPL) should be tested.
Visuotactile Interactions with Respect to Body and Peripersonal Space Representation
Our continuous interactions with the external world are subtended by specific brain mechanisms controlling the integration between bodily perceptions and spatially localized visual stimuli. In this respect, near peripersonal, as opposed to far extrapersonal space, is defined as the space near the body where action typically occurs and a close control of bodily stimuli (e.g., a touch) and visual stimuli (e.g., an object contacting our body) must be constantly maintained. Specific neural structures contain bimodal neurons responding to visual stimuli near the body. These bimodal neurons are considered critical to visuotactile integration in the peripersonal space in animals (Rizzolatti et al., 1997; Graziano, 2001) and in humans (Maravita et al., 2003; Ladavas and Farnè, 2004). The critical aspect of this kind of multisensory integration is that it is maintained across bodily movements. The hand, for example, keeps interacting with visual stimuli in its surroundings, even if it continuously moves across different spatial positions, as shown by electrophysiological research (Graziano et al., 1997) and research on neurologically healthy subjects and brain-damaged patients (Maravita et al., 2003). Bolognini and Maravita (2007) showed the causal involvement of the PPC in maintaining the constant mapping of visual and tactile reference frames across hand movements. These authors showed that the enhancement of TMS-induced phosphenes by spatially congruent touches (see previous paragraph for details about this experiment) was abolished if off-line 1-Hz rTMS was applied to the right PPC, in a position putatively targeting the ventral intraparietal cortex (area VIP). This area was shown to be specifically activated during the monitoring of tactile input across hand postures in a previous fMRI study (Lloyd et al., 2003). In particular, before rTMS, a touch on the index finger increased the rate of reported phosphenes both when the hands were uncrossed (e.g., detection of phosphenes in the left hemifield following right occipital TMS was increased by left-hand touches) and crossed (i.e., left-hemifield phosphenes now increased following right-hand touches), showing a spatial realignment of visual and somatosensory frames of reference following hand crossing. Critically, following rTMS to VIP, the rate of phosphenes in a given hemifield (e.g., the left) was enhanced by tactile stimuli delivered to the hand (the left, in this example) controlled by the same brain hemisphere (the right one), regardless of the hand posture. In other words, no remapping of visual and somatosensory representations occurred following changes of hand posture, but now crossmodal facilitation was mediated by an “intrahemispheric” facilitation mechanism (left/right hand always increasing the report of phosphenes in the homologous left/right hemifield).
Evidence for the causal involvement of intraparietal cortex in the proprioceptive remapping of touch into external space was recently further confirmed by Azanon et al. (2010) using sTMS.
As outlined in the previous chapter in relation to audio–visual integration, even for visuotactile interactions an important issue is which mechanism subserves the integration of stimuli belonging to different sensory modalities (Macaluso and Maravita, 2010). A recent study by Chambers et al. (2007) has clarified the causal role of the inferior parietal cortex in the reflexive orienting toward visual or somatosensory events, induced by non-informative visual or tactile cues. On-line rTMS was delivered synchronously with cue onset for 100 ms (pulses at 0, 50, and 100 ms post-cue onset). A disruption of the angular gyrus caused a deficit in the exogenous attention orienting induced by a tactile cue on the detection of both tactile and visual targets, in a task requiring an elevation judgment of visual and tactile events delivered close to the hands. This area showed, therefore, a selective role in biasing attention within and across sensory modalities, following a somatosensory, but not a visual cue. By contrast, rTMS to the supramarginal gyrus reduced reflexive orienting only across, but not within, sensory modalities, showing selective crossmodal effects (Chambers et al., 2007).
The above cited studies underline the importance of parietal areas for multisensory integration. On one side, the intraparietal sulcus seems critical to maintain a constant update of limb posture, thus updating the relative position of extrapersonal visual and somatosensory bodily stimuli for visuotactile effects, which are so typical of the peripersonal space (Maravita et al., 2003). Furthermore, attentional orienting toward unisensory and bimodal visual and tactile events are subtended by discrete subregions of the parietal lobe.
Uncovering Crossmodal Effects in Unisensory Visual or Somatosensory Areas
Another emerging line of investigation aims at uncovering whether a modality-specific cortical region, commonly associated with the processing of unisensory information from a given sense, can be recruited in a crossmodal manner to process purely unisensory stimuli from a different sensory modality. This line of investigation differs from those described above (see paragraph 3), which assessed the occurrence of multisensory interactions among bimodal stimuli, in unisensory and heteromodal cortices. In humans, sensory deprivation has been widely used as a model to explore the role of experience-dependent crossmodal recruitment of modality-specific areas, as revised elsewhere (e.g., Neville and Bavelier, 2002; Merabet and Pascual-Leone, 2010). Here, we will take into consideration only recent studies showing crossmodal processing in traditionally viewed modality-specific areas of the healthy, not sensory-deprived, human brain. TMS evidence of the contribution of visual imagery in the crossmodal recruitment of visual cortex in tactile processing are not considered here, but are reviewed elsewhere (e.g., Sathian and Zangaladze, 2002).
In this context, Mancini et al. (2010) took advantage of the Müller–Lyer illusion (i.e., arrowheads at the ends of a line may affect its estimated length) to address the causal role of the regions activated by the visual illusion in the crossmodal generation of the haptic, and crossmodal visuo-haptic illusory effects (Mancini et al., 2010). Off-line 1-Hz rTMS was administered to the lateral occipital cortex (LOC), and to the right SPL of both hemispheres. SPL is known to be involved in visual (Corbetta et al., 1995) and crossmodal spatial attention and localization (Bushara et al., 1999), whereas LOC is considered a modality-specific visual area, although LOC might be also implicated in crossmodal object recognition, responding to both familiar and unfamiliar visual and tactile shapes (Amedi et al., 2002; Deshpande et al., 2010). rTMS to left and right LOC impaired both unisensory (visual, haptic) and crossmodal processing of the Müller–Lyer illusion in a similar fashion. Conversely, rTMS to left and right SPC did not affect the illusion in any modality. These results showed that the visual area LOC plays a crossmodal role in tactile perception of both illusory and non-illusory shapes; although being traditional identified as a visual area, LOC is involved in the processing of purely somatosensory stimuli, as well as visuo-haptic stimuli.
A further experiment tested whether disruption of somatosensory cortices could impair the processing of visual stimuli depicting a touch. This situation is being recently put forward by different researchers, as a possible somatosensory analog of the mirror system for movement. While the motor mirror system show activation of motor areas by the vision of moving bodily segments (Rizzolatti and Craighero, 2004), its analog in the somatosensory system would show activation of primary (S1) and/or secondary (S2) somatosensory cortices by the vision of a tactile event (Keysers et al., 2010). This system may contribute to the understanding of sensory events occurring in other people (Gallese, 2007), in a broader circuit of embodied simulation, whereby visual events may be mapped onto our own body. This mechanism could involve the activation of multisensory brain areas and be altered if the PPC are disrupted by TMS (Pasalar et al., 2010). Bolognini et al. (2011b) showed that on-line 13-Hz rTMS delivered over S1 disrupted the performance in a visual discrimination task requiring to judge whether a moving finger was or not touching a hand. rTMS to S1 selectively reduced performance for contralateral stimuli. The effect was specific for the perception of touch, since the discrimination of a finger moving, without touch, was unaffected by S1 stimulation (Bolognini et al., 2011b).
The traditional definition of “purely” unisensory areas has been effectively changed by the above described TMS studies, showing that typical unisensory visual areas may exert somatosensory or multisensory functions (Mancini et al., 2010), whereas early somatosensory areas can process purely visual stimuli depicting touches (Bolognini et al., 2011b). These results add novel insights to the neuroimaging literature, showing the causal involvement of unisensory cortices in crossmodal perception.
Facilitating Multisensory Interactions by tDCS
Ongoing studies in our laboratory are focusing on the use of tDCS to modulate multisensory processing. For instance, the polarization of the right PPC may enhance spatial orienting across different sensory modalities. Specifically, 15 min of anodal tDCS (2 mA, delivered before the task) of the right PPC can decrease response latency to unimodal visual and auditory stimuli, as well as to bimodal audio–visual stimuli, during a reaction time task; the modulation was specific for stimuli contralateral to the tDCS side, with control stimulation of the OC affecting only contralateral visual processing (Bolognini et al., 2010b). As discussed above, low-frequency rTMS over the right PPC induced opposite effects, with increased latency of responses to unisensory visual and auditory stimuli (Bolognini et al., 2009a). Collectively the two studies, apart from the different NIBS techniques used, converge in showing the causal involvement of the right PPC in supramodal spatial orienting across different sensory modalities. Moreover, a dissociation of the tDCS effects was found with respect to the crossmodal processing of blue audio–visual stimuli, which are likely not detected by the human collicular pathway (Leh et al., 2010), versus red audio–visual stimuli, which are detected by the superior colliculus and the PPC. The tDCS-induced facilitation was indeed stronger for blue audio–visual stimuli, mostly integrated at a cortical level, whereas responses to red audio–visual stimuli, which likely involve a subcortical level of processing (Maravita et al., 2008), appeared less susceptible to the DC polarization of the PPC.
Based on these findings, and given the putative facilitatory effects of anodal tDCS on learning (Reis et al., 2008), we then explored the effect of coupling a multisensory visual field exploration training with anodal tDCS (2 mA for 30 min, delivered during the practice). We found that anodal tDCS delivered to the right, but not left, PPC is effective in facilitating performance during the audio–visual exploration training, inducing a speeding up of responses to audio–visual stimuli in the early phase of practice. Instead, without tDCS, performance improves more gradually and later during training, only at the end of practice. Additionally, the multisensory training combined with anodal tDCS translated in a greater gain in a variety of tasks assessing visuo-spatial attention and search, as compared to the effect induced by the training given in isolation, not coupled with tDCS (Bolognini et al., 2010a).
Crossmodal illusions are the flip side of sensory coherence, and they illustrate some of the consequences of disrupting the normal relationships among different sensory cues (Stein, 1998). One of the most powerful example of multisensory perception is the “sound-induced flash illusion” (Shams et al., 2000). When a single flash is presented along with two or more beeps, observers often report seeing two or more flashes, the so-called “fission” illusion. A corresponding “fusion” illusion has also been described, where a single beep causes the fusion of a double flash stimulus (Andersen et al., 2004). These multisensory phenomena highlight how sensory-specific perceptual judgments concerning one sense (i.e., vision) can be dramatically affected by their interaction with other senses (i.e., audition). Using this audio–visual illusion, we explored the possibility of modulating multisensory perception by polarizing with tDCS putatively relevant cortical regions, which are likely involved in the generation of the illusion (Watkins et al., 2006, 2007). We found that up- or down-regulating cortical excitability by tDCS can facilitate or reduce audio–visual interactions, depending on the current polarity, the targeted area, and the illusory percept. Specifically, the perceptual “fission” of a single flash, due to multiple beeps, was increased after anodal tDCS of the temporal cortex (i.e., active electrode placed at the level of the superior temporal gyrus), and decreased after anodal tDCS of the OC (i.e., active electrode placed at the level of V1). A reversal of such effects was induced by cathodal tDCS. Conversely, the perceptual “fusion” of multiple flashes due to a single beep was unaffected by tDCS and polarizing the parietal cortex was overall ineffective (Bolognini et al., 2011a). This study shows that tDCS can modulate audio–visual interactions, by non-invasively shifting cortical excitability in occipital or in temporal cortices, and it elucidates the causal association between neural activity in the occipital and temporal areas, and the conscious visual experience brought about by a multisensory stimulation.
The available data concerning multisensory processes shows that tDCS may facilitate multisensory interactions in a variety of experimental conditions. The observed facilitation of behavior depends on different factors, including the site of the stimulated area, the tDCS parameters (duration, intensity), the type of task, and also the stage of learning, when a training paradigm is used.
Conclusion and Future Directions
Current evidence suggests that multisensory processing occurs in higher-order heteromodal areas, as well as the earliest stages of sensory processing, up to primary sensory cortices in the human brain. Exactly how our brain integrates inputs from the different senses remains the subject of intensive investigation. It is becoming increasingly clear that many multisensory phenomena may reflect the causal interplay between interconnected regions of the brain, rather than just the function of an isolated area (Macaluso, 2006; Driver and Noesselt, 2008). The chance to directly inquire whether a given area or network has a causal or merely subsidiary role in shaping multisensory interactions can be assuaged by techniques of NIBS, which appear essential tools to substantially improve our understanding of multisensory processes, extending the knowledge derived from neuroimaging studies. Except for the study of crossmodal plasticity after deafferentation, the study of multisensory processing in the healthy human brain by using TMS or tDCS has began quite recently. TMS approaches allow to transiently interfere with the activity in a given area, in order to assess the relationship between focal cortical activity and multisensory-related behavior and to trace the timing at which activity in a particular cortical region contributes to multisensory interactions. On the other hand, tDCS consists of a subthreshold stimulation that modulates cortical excitability, consequently modulating the amount of multisensory effects. Another interesting approach for future studies is to probe the functional connectivity of different cortical areas with the use of paired pulse TMS. Double-pulse paradigms could be successfully used to study the time course of intra- and inter-hemispheric cortico-cortical pathways subserving multisensory interactions. Moreover, TMS can be combined with other techniques for measuring brain function, such as EEG, PET, fMRI, in order to explore the functional interplay between heteromodal and modality-specific brain areas. There is a clear theoretical advantage in combining different approaches. Combining two different methods, such as TMS and neuroimaging, has the advantage of overcoming the intrinsic limitations of either techniques used in isolation. In particular, with this approach it is possible to supplement the information provided by correlational analysis of neuroimaging, with a technique that can establish a causal link between brain function and behavior (Sack and Linden, 2003; Miniussi and Thut, 2010). On the other hand, a great advantage of functional neuroimaging is the ability to acquire measurements of activity in the entire brain, thus providing a broader picture of the cortical responses to TMS; hence, combining TMS with fMRI or PET allows to investigate interregional interactions and their possible functional consequences for perception and cognition (Sack and Linden, 2003; Ruff et al., 2006). Furthermore, the TMS–EEG integration can offer real-time information on cortical reactivity and connectivity, and on how functional activity links to behavior through the study of TMS-induced modulations (Miniussi and Thut, 2010).
We also need to go beyond the standard “virtual lesion” approach, used so far in TMS studies of multisensory processing. Based on the state-dependent effects of TMS, new paradigms were proposed which might discriminate between functionally distinct neuronal representations in the stimulated area.
Finally, although TMS cannot directly target subcortical structures, a recent study has shown that activity in the thalamus can be modulated by stimulation of parietal cortex, hence opening up new horizons for the studies of cortical–subcortical interactions in multisensory processing (Blankenburg et al., 2008).
Whichever approach is adopted, the use of NIBS in the study of multisensory interactions requires carefully designed psychophysical tasks and the testing of control sites, in order to ensure the selectivity of the neuronal disruption of a given area on unisensory versus multisensory processing. In the next future, the knowledge derived from NIBS studies can undoubtedly extend our comprehension of the multisensory mechanisms operating in the human brain.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgment
The preparation of the present paper was supported by Funding (FAR 2010) from the University of Milano–Bicocca to Angelo Maravita and Nadia Bolognini.
References
Alais, D., and Burr, D. (2004). The ventriloquist effect results from near-optimal bimodal integration. Curr. Biol. 14, 257–262.
Amedi, A., Jacobson, G., Hendler, T., Malach, R., and Zohary, E. (2002). Convergence of visual and tactile shape processing in the human lateral occipital complex. Cereb. Cortex 12, 1202–1212.
Andersen, R. A. (1997). Multimodal integration for the representation of space in the posterior parietal cortex. Philos. Trans. R. Soc. Lond., B, Biol. Sci. 352, 1421–1428.
Andersen, T. S., Tiippana, K., and Sams, M. (2004). Factors influencing audiovisual fission and fusion illusions. Brain Res. Cogn. Brain Res. 21, 301–308.
Ardolino, G., Bossi, B., Barbieri, S., and Priori, A. (2005). Non-synaptic mechanisms underlie the after-effects of cathodal transcutaneous direct current stimulation of the human brain. J. Physiol. 568, 653–663.
Azanon, E., Longo, M. R., Soto-Faraco, S., and Haggard, P. (2010). The posterior parietal cortex remaps touch into external space. Curr. Biol. 20, 1304–1309.
Barraclough, N. E., Xiao, D., Baker, C. I., Oram, M. W., and Perrett, D. I. (2005). Integration of visual and auditory information by superior temporal sulcus neurons responsive to the sight of actions. J. Cogn. Neurosci. 17, 377–391.
Beauchamp, M. S., Argall, B. D., Bodurka, J., Duyn, J. H., and Martin, A. (2004). Unraveling multisensory integration: patchy organization within human STS multisensory cortex. Nat. Neurosci. 7, 1190–1192.
Beauchamp, M. S., Nath, A. R., and Pasalar, S. (2010). fMRI-guided transcranial magnetic stimulation reveals that the superior temporal sulcus is a cortical locus of the McGurk effect. J. Neurosci. 30, 2414–2417.
Bertini, C., Leo, F., Avenanti, A., and Ladavas, E. (2010). Independent mechanisms for ventriloquism and multisensory integration as revealed by theta-burst stimulation. Eur. J. Neurosci. 31, 1791–1799.
Blankenburg, F., Ruff, C. C., Bestmann, S., Bjoertomt, O., Eshel, N., Josephs, O., Weiskopf, N., and Driver, J. (2008). Interhemispheric effect of parietal TMS on somatosensory response confirmed directly with concurrent TMS-fMRI. J. Neurosci. 28, 13202–13208.
Bolognini, N., Frassinetti, F., Serino, A., and Ladavas, E. (2005). “Acoustical vision” of below threshold stimuli: interaction among spatially converging audiovisual inputs. Exp. Brain Res. 160, 273–282.
Bolognini, N., Fregni, F., Casati, C., Olgiati, E., and Vallar, G. (2010a). Brain polarization of parietal cortex augments training-induced improvement of visual exploratory and attentional skills. Brain Res. 1349, 76–89.
Bolognini, N., Olgiati, E., Rossetti, A., and Maravita, A. (2010b). Enhancing multisensory spatial orienting by brain polarization of the parietal cortex. Eur. J. Neurosci. 31, 1800–1806.
Bolognini, N., Senna, I., Maravita, A., Pascual-Leone, A., and Merabet, L. B. (2010c). Auditory enhancement of visual phosphene perception: the effect of temporal and spatial factors and of stimulus intensity. Neurosci. Lett. 477, 109–114.
Bolognini, N., and Maravita, A. (2007). Proprioceptive alignment of visual and somatosensory maps in the posterior parietal cortex. Curr. Biol. 17, 1890–1895.
Bolognini, N., Miniussi, C., Savazzi, S., Bricolo, E., and Maravita, A. (2009a). TMS modulation of visual and auditory processing in the posterior parietal cortex. Exp. Brain Res. 195, 509–517.
Bolognini, N., Pascual-Leone, A., and Fregni, F. (2009b). Using non-invasive brain stimulation to augment motor training-induced plasticity. J. Neuroeng. Rehabil. 6, 8.
Bolognini, N., and Ro, T. (2010). Transcranial magnetic stimulation: disrupting neural activity to alter and assess brain function. J. Neurosci. 30, 9647–9650.
Bolognini, N., Rossetti, A., Casati, C., Mancini, F., and Vallar, G. (2011a). Neuromodulation of multisensory perception: a tDCS study of the sound-induced flash illusion. Neuropsychologia 49, 231–237.
Bolognini, N., Rossetti, R., Maravita, A., and Miniussi, C. (2011b). Seeing touches in the somatosensory cortex: a TMS study of the visual perception of touch. Hum. Brain Mapp. doi: 10.1002/hbm.21172. [Epub ahead of print].
Bonath, B., Noesselt, T., Martinez, A., Mishra, J., Schwiecker, K., Heinze, H. J., and Hillyard, S. A. (2007). Neural basis of the ventriloquist illusion. Curr. Biol. 17, 1697–1703.
Boroojerdi, B., Meister, I. G., Foltys, H., Sparing, R., Cohen, L. G., and Topper, R. (2002). Visual and motor cortex excitability: a transcranial magnetic stimulation study. Clin. Neurophysiol. 113, 1501–1504.
Bremmer, F., Schlack, A., Shah, N. J., Zafiris, O., Kubischik, M., Hoffmann, K., Zilles, K., and Fink, G. R. (2001). Polymodal motion processing in posterior parietal and premotor cortex: a human fMRI study strongly implies equivalencies between humans and monkeys. Neuron 29, 287–296.
Bushara, K. O., Weeks, R. A., Ishii, K., Catalan, M. J., Tian, B., Rauschecker, J. P., and Hallett, M. (1999). Modality-specific frontal and parietal areas for auditory and visual spatial localization in humans. Nat. Neurosci. 2, 759–766.
Calvert, G. A. (2001). Crossmodal processing in the human brain: insights from functional neuroimaging studies. Cereb. Cortex 11, 1110–1123.
Canolty, R. T., Soltani, M., Dalal, S. S., Edwards, E., Dronkers, N. F., Nagarajan, S. S., Kirsch, H. E., Barbaro, N. M., and Knight, R. T. (2007). Spatiotemporal dynamics of word processing in the human brain. Front. Neurosci. 1:1. doi: 10.3389/neuro.01/1.1.014.2007
Cappe, C., and Barone, P. (2005). Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey. Eur. J. Neurosci. 22, 2886–2902.
Cappe, C., Morel, A., Barone, P., and Rouiller, E. M. (2009). The thalamocortical projection systems in primate: an anatomical support for multisensory and sensorimotor interplay. Cereb. Cortex 19, 2025–2037.
Cappe, C., Thut, G., Romei, V., and Murray, M. M. (2010). Auditory-visual multisensory interactions in humans: timing, topography, directionality, and sources. J. Neurosci. 30, 12572–12580.
Chambers, C. D., Payne, J. M., and Mattingley, J. B. (2007). Parietal disruption impairs reflexive spatial attention within and between sensory modalities. Neuropsychologia 45, 1715–1724.
Corbetta, M., Shulman, G. L., Miezin, F. M., and Petersen, S. E. (1995). Superior parietal cortex activation during spatial attention shifts and visual feature conjunction. Science 270, 802–805.
Deshpande, G., Hu, X., Lacey, S., Stilla, R., and Sathian, K. (2010). Object familiarity modulates effective connectivity during haptic shape perception. Neuroimage 49, 1991–2000.
Downar, J., Crawley, A. P., Mikulis, D. J., and Davis, K. D. (2000). A multimodal cortical network for the detection of changes in the sensory environment. Nat. Neurosci. 3, 277–283.
Driver, J., and Noesselt, T. (2008). Multisensory interplay reveals crossmodal influences on ‘sensory-specific’ brain regions, neural responses, and judgments. Neuron 57, 11–23.
Eimer, M. (2001). Crossmodal links in spatial attention between vision, audition, and touch: evidence from event-related brain potentials. Neuropsychologia 39, 1292–1303.
Falchier, A., Clavagnier, S., Barone, P., and Kennedy, H. (2002). Anatomical evidence of multimodal integration in primate striate cortex. J. Neurosci. 22, 5749–5759.
Fernandez, E., Alfaro, A., Tormos, J. M., Climent, R., Martinez, M., Vilanova, H., Walsh, V., and Pascual-Leone, A. (2002). Mapping of the human visual cortex using image-guided transcranial magnetic stimulation. Brain Res. Brain Res. Protoc. 10, 115–124.
Frassinetti, F., Bolognini, N., Bottari, D., Bonora, A., and Ladavas, E. (2005). Audiovisual integration in patients with visual deficit. J. Cogn. Neurosci. 17, 1442–1452.
Gallese, V. (2007). Embodied simulation: from mirror neuron systems to interpersonal relations. Novartis Found. Symp. 278, 3–12; discussion 12–19, 89–96, 216–221.
Ghazanfar, A. A., and Schroeder, C. E. (2006). Is neocortex essentially multisensory? Trends Cogn. Sci. 10, 278–285.
Giard, M. H., and Peronnet, F. (1999). Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J. Cogn. Neurosci. 11, 473–490.
Gielen, S. C., Schmidt, R. A., and Van den Heuvel, P. J. (1983). On the nature of intersensory facilitation of reaction time. Percept. Psychophys. 34, 161–168.
Graziano, M. S., Hu, X. T., and Gross, C. G. (1997). Coding the locations of objects in the dark. Science 277, 239–241.
Hairston, W. D., Wallace, M. T., Vaughan, J. W., Stein, B. E., Norris, J. L., and Schirillo, J. A. (2003). Visual localization ability influences cross-modal bias. J. Cogn. Neurosci. 15, 20–29.
Harris, I. M., Benito, C. T., Ruzzoli, M., and Miniussi, C. (2008). Effects of right parietal transcranial magnetic stimulation on object identification and orientation judgments. J. Cogn. Neurosci. 20, 916–926.
Huang, Y. Z., Edwards, M. J., Rounis, E., Bhatia, K. P., and Rothwell, J. C. (2005). Theta burst stimulation of the human motor cortex. Neuron 45, 201–206.
Hughes, H. C., Nelson, M. D., and Aronchick, D. M. (1998). Spatial characteristics of visual-auditory summation in human saccades. Vision Res. 38, 3955–3963.
Jones, E. G., and Powell, T. P. (1970). An anatomical study of converging sensory pathways within the cerebral cortex of the monkey. Brain 93, 793–820.
Kammer, T., Puls, K., Erb, M., and Grodd, W. (2005a). Transcranial magnetic stimulation in the visual system. II. Characterization of induced phosphenes and scotomas. Exp. Brain Res. 160, 129–140.
Kammer, T., Puls, K., Strasburger, H., Hill, N. J., and Wichmann, F. A. (2005b). Transcranial magnetic stimulation in the visual system. I. The psychophysics of visual suppression. Exp. Brain Res. 160, 118–128.
Keysers, C., Kaas, J. H., and Gazzola, V. (2010). Somatosensation in social perception. Nat. Rev. Neurosci. 11, 417–428.
Koch, G., and Rothwell, J. C. (2009). TMS investigations into the task-dependent functional interplay between human posterior parietal and motor cortex. Behav. Brain Res. 202, 147–152.
Ladavas, E., and Farnè, A. (2004). Visuo-tactile representation of near-the-body space. J. Physiol. Paris 98, 161–170.
Leh, S. E., Ptito, A., Schonwiesner, M., Chakravarty, M. M., and Mullen, K. T. (2010). Blindsight mediated by an S-cone-independent collicular pathway: an fMRI study in hemispherectomized subjects. J. Cogn. Neurosci. 22, 670–682.
Leo, F., Bolognini, N., Passamonti, C., Stein, B. E., and Ladavas, E. (2008). Cross-modal localization in hemianopia: new insights on multisensory integration. Brain 131, 855–865.
Liebetanz, D., Nitsche, M. A., Tergau, F., and Paulus, W. (2002). Pharmacological approach to the mechanisms of transcranial DC-stimulation-induced after-effects of human motor cortex excitability. Brain 125, 2238–2247.
Lloyd, D. M., Shore, D. I., Spence, C., and Calvert, G. A. (2003). Multisensory representation of limb position in human premotor cortex. Nat. Neurosci. 6, 17–18.
Macaluso, E. (2006). Multisensory processing in sensory-specific cortical areas. Neuroscientist 12, 327–338.
Macaluso, E., Frith, C. D., and Driver, J. (2000). Modulation of human visual cortex by crossmodal spatial attention. Science 289, 1206–1208.
Macaluso, E., and Maravita, A. (2010). The representation of space near the body through touch and vision. Neuropsychologia 48, 782–795.
Mancini, F., Bolognini, N., Bricolo, E., and Vallar, G. (2010). Cross-modal processing in the occipito-temporal cortex: a TMS study of the Muller–Lyer illusion. J. Cogn. Neurosci. doi:10.1162/jocn.2010.21561. [Epub ahead of print].
Maravita, A., Bolognini, N., Bricolo, E., Marzi, C. A., and Savazzi, S. (2008). Is audiovisual integration subserved by the superior colliculus in humans? Neuroreport 19, 271–275.
Maravita, A., Spence, C., and Driver, J. (2003). Multisensory integration and the body schema: close to hand and within reach. Curr. Biol. 13, R531–R539.
McKeefry, D. J., Gouws, A., Burton, M. P., and Morland, A. B. (2009). The noninvasive dissection of the human visual cortex: using FMRI and TMS to study the organization of the visual brain. Neuroscientist 15, 489–506.
Meienbrock, A., Naumer, M. J., Doehrmann, O., Singer, W., and Muckli, L. (2007). Retinotopic effects during spatial audio-visual integration. Neuropsychologia 45, 531–539.
Merabet, L. B., and Pascual-Leone, A. (2010). Neural reorganization following sensory loss: the opportunity of change. Nat. Rev. Neurosci. 11, 44–52.
Meredith, M. A., and Stein, B. E. (1983). Interactions among converging sensory inputs in the superior colliculus. Science 221, 389–391.
Miniussi, C., Ruzzoli, M., and Walsh, V. (2009). The mechanism of transcranial magnetic stimulation in cognition. Cortex 46, 128–130.
Miniussi, C., and Thut, G. (2010). Combining TMS and EEG offers new prospects in cognitive neuroscience. Brain Topogr. 22, 249–256.
Molholm, S., Ritter, W., Murray, M. M., Javitt, D. C., Schroeder, C. E., and Foxe, J. J. (2002). Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. Brain Res. Cogn. Brain Res. 14, 115–128.
Neville, H., and Bavelier, D. (2002). Human brain plasticity: evidence from sensory deprivation and altered language experience. Prog. Brain Res. 138, 177–188.
Nitsche, M. A., Cohen, L. G., Wassermann, E. M., Priori, A., Lang, N., Antal, A., Paulus, W., Hummel, F., Boggio, P. S., Fregni, F., and Pascual-Leone, A. (2008). Transcranial direct current stimulation: state of the art 2008. Brain Stimul. 1, 206–223.
Nitsche, M. A., and Paulus, W. (2001). Sustained excitability elevations induced by transcranial DC motor cortex stimulation in humans. Neurology 57, 1899–1901.
Pasalar, S., Ro, T., and Beauchamp, M. S. (2010). TMS of posterior parietal cortex disrupts visual tactile multisensory integration. Eur. J. Neurosci. 31, 1783–1790.
Pascual-Leone, A., Amedi, A., Fregni, F., and Merabet, L. B. (2005). The plastic human brain cortex. Annu. Rev. Neurosci. 28, 377–401.
Pascual-Leone, A., Walsh, V., and Rothwell, J. (2000). Transcranial magnetic stimulation in cognitive neuroscience – virtual lesion, chronometry, and functional connectivity. Curr. Opin. Neurobiol. 10, 232–237.
Paus, T. (1999). Imaging the brain before, during, and after transcranial magnetic stimulation. Neuropsychologia 37, 219–224.
Priori, A. (2003). Brain polarization in humans: a reappraisal of an old tool for prolonged non-invasive modulation of brain excitability. Clin. Neurophysiol. 114, 589–595.
Puce, A., Epling, J. A., Thompson, J. C., and Carrick, O. K. (2007). Neural responses elicited to face motion and vocalization pairings. Neuropsychologia 45, 93–106.
Ramos-Estebanez, C., Merabet, L. B., Machii, K., Fregni, F., Thut, G., Wagner, T. A., Romei, V., Amedi, A., and Pascual-Leone, A. (2007). Visual phosphene perception modulated by subthreshold crossmodal sensory stimulation. J. Neurosci. 27, 4178–4181.
Reis, J., Robertson, E., Krakauer, J. W., Rothwell, J., Marshall, L., Gerloff, C., Wassermann, E., Pascual-Leone, A., Hummel, F., Celnik, P. A., Classen, J., Floel, A., Ziemann, U., Paulus, W., Siebner, H. R., Born, J., and Cohen, L. G. (2008). Consensus: “can tDCS and TMS enhance motor learning and memory formation?”. Brain Stimul. 1, 363–369.
Rizzolatti, G., and Craighero, L. (2004). The mirror-neuron system. Annu. Rev. Neurosci. 27, 169–192.
Rizzolatti, G., Fadiga, L., Fogassi, L., and Gallese, V. (1997). The space around us. Science 277, 190–191.
Ro, T., Cheifet, S., Ingle, H., Shoup, R., and Rafal, R. (1999). Localization of the human frontal eye fields and motor hand area with transcranial magnetic stimulation and magnetic resonance imaging. Neuropsychologia 37, 225–231.
Rockland, K. S., and Ojima, H. (2003). Multisensory convergence in calcarine visual areas in macaque monkey. Int. J. Psychophysiol. 50, 19–26.
Romei, V., Gross, J., and Thut, G. (2010). On the role of prestimulus alpha rhythms over occipito-parietal areas in visual input regulation: correlation or causation? J. Neurosci. 30, 8692–8697.
Romei, V., Murray, M. M., Cappe, C., and Thut, G. (2009). Preperceptual and stimulus-selective enhancement of low-level human visual cortex excitability by sounds. Curr. Biol. 19, 1799–1805.
Romei, V., Murray, M. M., Merabet, L. B., and Thut, G. (2007). Occipital transcranial magnetic stimulation has opposing effects on visual and auditory stimulus detection: implications for multisensory interactions. J. Neurosci. 27, 11465–11472.
Rossini, P. M., Barker, A. T., Berardelli, A., Caramia, M. D., Caruso, G., Cracco, R. Q., Dimitrijević, M. R., Hallett, M., Katayama, Y., Lücking, C. H., de Noordhout, M., Marsden, C. D., Murray, N. M. F., Rothwell, J. C., Swash, M., and Tomberg, C. (1994). Non-invasive electrical and magnetic stimulation of the brain, spinal cord and roots: basic principles and procedures for routine clinical application. Report of an IFCN committee. Electroencephalogr. Clin. Neurophysiol. 91, 79–92.
Ruff, C. C., Blankenburg, F., Bjoertomt, O., Bestmann, S., Freeman, E., Haynes, J. D., Rees, G., Josephs, O., Deichmann, R., and Driver, J. (2006). Concurrent TMS-fMRI and psychophysics reveal frontal influences on human retinotopic visual cortex. Curr. Biol. 16, 1479–1488.
Ruzzoli, M., Marzi, C. A., and Miniussi, C. (2010). The neural mechanisms of the effects of transcranial magnetic stimulation on perception. J. Neurophysiol. 103, 2982–2989.
Sack, A. T., Cohen Kadosh, R., Schuhmann, T., Moerel, M., Walsh, V., and Goebel, R. (2009). Optimizing functional accuracy of TMS in cognitive studies: a comparison of methods. J. Cogn. Neurosci. 21, 207–221.
Sack, A. T., and Linden, D. E. (2003). Combining transcranial magnetic stimulation and functional imaging in cognitive brain research: possibilities and limitations. Brain Res. Brain Res. Rev. 43, 41–56.
Sandrini, M., Umilta, C., and Rusconi, E. (2010). The use of transcranial magnetic stimulation in cognitive neuroscience: a new synthesis of methodological issues. Neurosci. Biobehav. Rev. 35, 516–536.
Sathian, K., and Zangaladze, A. (2002). Feeling with the mind’s eye: contribution of visual cortex to tactile perception. Behav. Brain Res. 135, 127–132.
Sauseng, P., Klimesch, W., Heise, K. F., Gruber, W. R., Holz, E., Karim, A. A., Glennon, M., Gerloff, C., Birbaumer, N., and Hummel, F. C. (2009). Brain oscillatory substrates of visual short-term memory capacity. Curr. Biol. 19, 1846–1852.
Schroeder, C. E., and Foxe, J. (2005). Multisensory contributions to low-level, ‘unisensory’ processing. Curr. Opin. Neurobiol. 15, 454–458.
Schroeder, C. E., and Foxe, J. J. (2002). The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Brain Res. Cogn. Brain Res. 14, 187–198.
Shams, L., Kamitani, Y., and Shimojo, S. (2000). Illusions. What you see is what you hear. Nature 408, 788.
Siebner, H. R., Hartwigsen, G., Kassuba, T., and Rothwell, J. C. (2009). How does transcranial magnetic stimulation modify neuronal activity in the brain? Implications for studies of cognition. Cortex 45, 1035–1042.
Silvanto, J., Muggleton, N., and Walsh, V. (2008). State-dependency in brain stimulation studies of perception and cognition. Trends Cogn. Sci. 12, 447–454.
Silvanto, J., and Muggleton, N. G. (2008). New light through old windows: moving beyond the “virtual lesion” approach to transcranial magnetic stimulation. Neuroimage 39, 549–552.
Silvanto, J., and Pascual-Leone, A. (2008). State-dependency of transcranial magnetic stimulation. Brain Topogr. 21, 1–10.
Sparing, R., Hesse, M. D., and Fink, G. R. (2010). Neuronavigation for transcranial magnetic stimulation (TMS): where we are and where we are going. Cortex 46, 118–120.
Spence, C., Nicholls, M. E., Gillespie, N., and Driver, J. (1998). Cross-modal links in exogenous covert spatial orienting between touch, audition, and vision. Percept. Psychophys. 60, 544–557.
Stein, B. E. (1998). Neural mechanisms for synthesizing sensory information and producing adaptive behaviors. Exp. Brain Res. 123, 124–135.
Stein, B. E., and Stanford, T. R. (2008). Multisensory integration: current issues from the perspective of the single neuron. Nat. Rev. Neurosci. 9, 255–266.
Thut, G., and Miniussi, C. (2009). New insights into rhythmic brain activity from TMS-EEG studies. Trends Cogn. Sci. 13, 182–189.
Wagner, T., Fregni, F., Fecteau, S., Grodzinsky, A., Zahn, M., and Pascual-Leone, A. (2007). Transcranial direct current stimulation: a computer-based human model study. Neuroimage 35, 1113–1124.
Walsh, V., and Pascual-Leone, A. (eds). (2003) Transcranial Magnetic Stimulation: A Neurochronometrics of Mind. Cambridge, MA: The Mitt Press.
Wassermann, E. M., Epstein, C. M., Ziemann, U., Walsh, V., Paus, T., and Lisanby, S. H. (2008). The Oxford Handbook of Transcranial Stimulation. (New York: Oxford University Press).
Wassermann, E. M., and Grafman, J. (2005). Recharging cognition with DC brain polarization. Trends Cogn. Sci. 9, 503–505.
Watkins, S., Shams, L., Josephs, O., and Rees, G. (2007). Activity in human V1 follows multisensory perception. Neuroimage 37, 572–578.
Watkins, S., Shams, L., Tanaka, S., Haynes, J. D., and Rees, G. (2006). Sound alters activity in human V1 in association with illusory visual perception. Neuroimage 31, 1247–1256.
Zaghi, S., Acar, M., Hultgren, B., Boggio, P. S., and Fregni, F. (2009). Noninvasive brain stimulation with low-intensity electrical currents: putative mechanisms of action for direct and alternating current stimulation. Neuroscientist. 16, 285–307.
Ziemann, U., Paulus, W., Nitsche, M. A., Pascual-Leone, A., Byblow, W. D., Berardelli, A., Siebner, H. R., Classen, J., Cohen, L. G., and Rothwell, J. C. (2008). Consensus: motor cortex plasticity protocols. Brain Stimul. 1, 164–182.
Keywords: multisensory processing, brain, TMS, tDCS, vision, touch, audition
Citation: Bolognini N and Maravita A (2011) Uncovering multisensory processing through non-invasive brain stimulation. Front. Psychology 2:46. doi: 10.3389/fpsyg.2011.00046
Received: 30 November 2010;
Accepted: 04 March 2011;
Published online: 21 March 2011.
Edited by:
Gregor Thut, University of Glasgow, UKReviewed by:
Juha Silvanto, Helsinki University of Technology, FinlandVincenzo Romei, University of Glasgow, UK
Copyright: © 2011 Bolognini and Maravita. This is an open-access article subject to an exclusive license agreement between the authors and Frontiers Media SA, which permits unrestricted use, distribution, and reproduction in any medium, provided the original authors and source are credited.
*Correspondence: Nadia Bolognini, Department of Psychology, University of Milano–Bicocca, Building U6, Piazza dell’Ateneo Nuovo 1, 20126 Milan, Italy. e-mail: nadia.bolognini@unimib.it