Original Research ARTICLE
Early modality-specific somatosensory cortical regions are modulated by attended visual stimuli: interaction of vision, touch, and behavioral intent
- Department of Kinesiology, University of Waterloo, Waterloo, ON, Canada
Bimodal interactions between relevant visual and tactile inputs can facilitate attentional modulation at early stages in somatosensory cortices to achieve goal-oriented behaviors. However, the specific contribution of each sensory system during attentional processing and, importantly, how these interact with the required behavioral motor goals remains unclear. Here we used electroencephalography and event-related potentials (ERPs) to test the hypothesis that activity from modality-specific somatosensory cortical regions would be enhanced with task-relevant bimodal (visual-tactile) stimuli and that the degree of modulation would depend on the difficulty of the associated sensory-motor task demands. Tactile stimuli were discrete vibrations to the index finger and visual stimuli were horizontal bars on a computer screen, both with random amplitudes. Streams of unimodal (tactile) and crossmodal (visual and tactile) stimuli were randomly presented and participants were instructed to attend to one type of stimulus (unimodal or crossmodal) and responses involved either an indication of the presence of an attended stimulus (detect), or the integration and summation of two stimulus amplitudes using a pressure-sensitive ball (grade). Force-amplitude associations were learned in a training session, and no feedback was provided during the task. ERPs were time-locked to tactile stimuli and extracted for early modality-specific components (P50, P100, N140). The P50 was enhanced with bimodal (visual-tactile) stimuli that were attended to. This was maximal when the motor requirements involved integration of the two stimuli in the grade task and when the visual stimulus occurred before (100 ms) the tactile stimulus. These results suggest that visual information relevant for movement modulates somatosensory processing as early as the primary somatosensory cortex (S1) and that the motor behavioral context influences this likely through interaction of top-down attentional and motor preparatory systems with more bottom-up crossmodal influences.
It is well-known that attention can modulate neurophysiological responses in modality-specific cortices including: visual (Motter, 1993; Gazzaley et al., 2007; Andersen et al., 2008), auditory (Woldorff et al., 1993; Jäncke et al., 1999; Petkov et al., 2004), and somatosensory cortices (Josiassen et al., 1990; Hsiao et al., 1993; Johansen-Berg et al., 2000; Staines et al., 2002). However, recent investigations have begun to examine whether attention influences neural responses across sensory modalities when sensory input from more than one modality is present. Behavioral studies have shown that crossmodal input can also improve performance as indexed by faster reaction times (Hershenson, 1962; Gielen et al., 1983), improved detection of weak stimuli (Frens and Van Opstal, 1995; Driver and Spence, 1998; McDonald et al., 2001), and improved sensory perception of illusory effects such as the ventriloquist or McGurk illusions (Howard and Templeton, 1966; McGurk and MacDonald, 1976). Human and animal studies have shown that the mere presence of additional sensory input even when it is irrelevant for performance of a task can enhance neural excitability in the attended sensory modality (Calvert et al., 1997; Macaluso et al., 2000, 2002; Calvert, 2001; Foxe et al., 2002; Kayser et al., 2005, 2007; Lehmann et al., 2006; Pekkola et al., 2006; Lakatos et al., 2007; Meehan and Staines, 2009), suggesting that interactions between modality-specific cortical representations exist. By contrast, other studies have shown crossmodal enhancement in modality-specific sensory cortex only occurs when both stimulus events are relevant for behavior (Dionne et al., 2010, 2013). These findings suggest that crossmodal processing is likely governed by both bottom-up sensory–sensory interactions and top-down attentional mechanisms in order to allow for the selection, amplification, and integration of sensory input relevant for initiating goal-oriented responses. Bottom-up interactions can occur when salient stimuli from an unattended sensory modality influence neural excitability in the attended modality, while top-down processing occurs when attention is voluntarily directed towards relevant stimuli in the presence of environmental distracters. However, while both these attentional mechanisms can modulate neural responses in modality-specific sensory cortex, it is unclear how these attentional mechanisms interact during sensory processing of crossmodal stimuli.
Excitability of somatosensory cortex is modulated by the relevance of stimuli to behavior, with the goal of facilitating the extraction of relevant sensory information for further cortical processing. The modulation of somatosensory information during movement provides evidence that the primary somatosensory cortex (S1) is sensitive to the relevance of somatosensory stimuli to behavior. Inhibition of afferent information ascending to the cortex is seen when somatosensory evoked potentials (SEPs) to passive somatosensory stimuli are attenuated during movement, a phenomenon often called movement-related gating (Jones et al., 1989; Cheron and Borenstein, 1991; Chapman, 1994; Brooke, 2004). Interestingly, this gating effect is lifted, at least partially, when the stimuli are made relevant to the performance of a task (Staines et al., 1997, 2000).
In a non-gating context, recent findings using event-related potentials (ERPs) provide evidence that task-relevance of stimuli facilitates crossmodal modulation of early and mid-latency somatosensory-specific ERP components, namely the P50, a positive potential peaking at approximately 50 ms after presentation and generated in the primary somatosensory cortex (Hämäläinen et al., 1990), as well as the P100 and N140 (Dionne et al., 2013; Popovich and Staines, 2014) generated in secondary somatosensory cortex (Mima et al., 1998; Frot and Mauguiere, 1999; Gu, 2002) and influenced by attention. These results are novel as crossmodal effects on the somatosensory P50 have not been previously reported, and even attentional modulation of this potential is not consistently observed (Desmedt and Robertson, 1977; Michie et al., 1987; Eimer and Forster, 2003; Zopf et al., 2004; Schubert et al., 2008). Part of the difficulty in reconciling the inconsistent reports of P50 modulation can be attributed to a lack of clarity in what it represents. Although the P50 is typically thought to reflect S1 excitability, early reports have suggested that the latency of this potential makes it unlikely to reflect processing of the evoking stimulus, but instead is more likely to represent the application of cognitive strategies to stimulus processing (Desmedt and Tomberg, 1989). If this is the case, then modulation of the P50 could be highly dependent on elements of the task that contribute to the strategy used by the subject, which could account for the lack of consensus on modulation of this potential across different tasks and paradigms. Early interactions have also been shown between auditory and somatosensory cortices in tasks with simultaneous stimulus presentation (Foxe et al., 2000). Further, Foxe and Simpson (2002) showed that early modality-specific visual cortex is active as early as 56 ms after stimulus onset with evidence of dorsolateral frontal cortex by 80 ms.
Neuroimaging studies in humans complement the sensory-to-sensory interactions reported above by showing that the presence of crossmodal input can modulate neural excitability in modality-specific sensory cortices. Several functional magnetic resonance imaging studies have reported increased blood oxygenation level dependent (BOLD) responses in modality-specific cortices due to the mere presence of stimuli from another modality. These interactions have been found between: visual and auditory cortices (Calvert et al., 1997; Calvert, 2001; Lehmann et al., 2006; Pekkola et al., 2006), auditory and somatosensory cortices (Foxe et al., 2002; Schürmann et al., 2006), as well as visual and somatosensory cortices (Macaluso et al., 2000, 2002). In addition, Dionne et al. (2010) investigated crossmodal effects on BOLD responses generated in SI when both stimuli were relevant for guiding a motor response. Here, relevant unimodal (visual or tactile) and crossmodal stimuli (simultaneous visual + tactile) were presented and participants squeezed a pressure-sensitive bulb with a force that was dependent on the summation of both stimuli. Results showed that the greatest BOLD responses were elicited in S1 during crossmodal versus unimodal interactions suggesting that combining visual-tactile (VT) information relevant for behavior enhances modality-specific excitability in S1 (Dionne et al., 2010).
The objective of the current study was to investigate the role of specific task requirements in mediating the previously observed crossmodal modulation of early modality-specific somatosensory cortical responses, represented by the P50. Importantly, this crossmodal modulation occurred when both the visual and tactile target stimuli were attended to and necessary for an impending motor task. The current study investigates the role of the motor requirements of the impending task. Two experiments were performed and in each it was predicted that P50 modulation would be dependent on task demands, specifically that modulation would scale with the degree of stimulus feature extraction required to execute the motor response. To manipulate this relationship between stimulus and response in Experiment 1, three tasks were chosen to represent a gradient of reliance on the attributes of the stimulus to inform the motor response: a detection task, a discrimination task, and a graded motor task. The behavioral task requirements were manipulated to emphasize the target stimulus relevance during motor preparation. Based on the evidence discussed above we hypothesized that the amplitude of the P50, likely generated from S1, would be most sensitive to crossmodal modulation in the graded motor task. Based on previous findings that the N140 at frontal electrode sites is sensitive to the task-relevance of stimuli, it was also predicted that the frontal N140 would scale with the increased demand to extract sensory information and incorporate it into a motor response.
Materials and Methods
Electroencephalography (EEG) was recorded from 20 right-handed (by self-report) healthy participants, 10 each participated in 1 of 2 experiments (Experiment 1: mean age = 24, 3 males, 7 females; Experiment 2: mean age = 24, 5 males, 5 females). Experimental procedures were approved by the University of Waterloo Office of Research Ethics. All subjects provided informed written consent.
The behavioral task required participants to attend to discrete unimodal vibrotactile stimuli or crossmodal vibrotactile/visual stimuli and make a motor response to the attended stimulus by squeezing a pressure-sensitive bulb with their right hand. Stimulus amplitudes were randomly varied throughout the blocks of trials but the average amplitude across each block was the same. Stimuli were always presented in pairs, either unimodally where two vibrotactile stimuli were presented to the left index finger sequentially (Figure 1, TT), or bimodally (vibrotactile and visual) that were presented simultaneously [Figure 1, visual target was presented at the same time as the tactile (SIM), Experiment 1 and 2] or with a 100 ms delay between the onset of the visual and tactile [Figure 1, visual target was presented prior to the tactile (VTd), Experiment 2]. As depicted by the schematic diagrams in Figure 1, for each block of trials, participants attended and responded to either the unimodal (Figures 1C,D) or crossmodal (Figures 1A,B) stimuli in specific blocks of trials. When attention was directed to one stimulus type in a block, those of the other type were task irrelevant and were not responded to. The required responses were manipulated as described below.
FIGURE 1. Experimental task: (A) and (B) illustrate the task blocks in which participants attended and responded to crossmodal visual-tactile stimuli. Responses were either scaled to the summed amplitudes of the two stimuli (Grade) or simply indicated the presence of the attended stimuli (Detect). (C) and (D) illustrate the presentation of the same stimuli; however, in these blocks participants attended and responded to the unimodal tactile stimuli. In Experiment 1 stimuli included the TT and simultaneously presented VT pairs (SIM). Participants performed three motor tasks where the integrated the amplitudes between the two stimuli (Grade), indicated the presence of the attended stimulus type (Detect) or judged which of the two in the stimulus pair would require a larger response (Judge – similar response to Detect; see text for details). In Experiment 2 there were also stimulus pairs where the onset of the visual stimulus preceded the tactile stimulus by 100 ms. Each block of trials lasted approximately 6 min.
Prior to the EEG collection in both experiments participants underwent a 5 min training session with visual feedback in a sound attenuated booth to learn the relationship between the amplitudes of the stimuli and the corresponding force required to apply to the bulb. During training, a horizontal target bar appeared on the visual display and subjects were instructed to squeeze the pressure-sensitive bulb with enough force to raise another visual horizontal bar to the same level as the target bar. At the same time, as subjects applied force to the bulb with their right hand the vibrotactile device vibrated against the volar surface of their left index finger with corresponding changes in amplitude, i.e., as they squeezed harder on the bulb the amplitude of the vibration increased proportionately. Subjects were instructed to pay attention to these changes in amplitude as they related to the force they were applying to the bulb, and in this way subjects became familiar with the relationship between the vibrotactile stimulus amplitude and the corresponding force applied to the bulb. To control for force-related trial-to-trial differences, stimulus amplitudes were scaled such that no single stimulus required a squeeze of more than 25% of an individual’s maximum force, thus the response for adding two stimuli was never more than 50% of an individual’s maximum force.
In Experiment 1, three versions of the behavioral task were performed, although the stimuli were identical throughout and consisted of a stream of pseudorandomly intermixed unimodal (tactile paired with tactile, TT) stimuli and crossmodal (tactile paired simultaneously with visual, VT/SIM) stimuli. In the “detect” task participants were instructed to press a button with their right index finger when they detected the target stimulus (detect TT or detect VT). In this task participants discriminated between the presence of unimodal and crossmodal stimuli. In the “judge” task, participants were required to discriminate between the two target stimuli (judge TT or judge VT) and press pre-assigned buttons to indicate which one they perceived as larger in amplitude. The “grade” task required participants to judge the amplitude of the target stimuli and make a graded motor response to represent the sum of these amplitudes by squeezing a pressure-sensitive bulb with their right hand (grade TT or grade VT).
Paired tactile stimuli were presented for 500 ms each, separated by a 30 ms inter-stimulus interval. Paired visual and tactile stimuli were presented for 1 s with simultaneous onset or where the visual stimulus onset preceded the tactile stimulus by 100 ms (VTd; Experiment 2). An individual trial consisted of one set of paired stimuli followed by 2.5 s for the participant to make a response prior to the onset of the next pair of stimuli. Each individual condition was presented in two blocks of 70 trials [total of 12 blocks × 70 trials (35 TT, 35 VT) per block: 840 individual trials separated into the 6 conditions (Detect/Judge/Grade for unimodal and crossmodal)]. The order of the 6 task conditions was counter-balanced across subjects.
In Experiment 2, participants were presented with similar streams of pseudorandomly intermixed unimodal (tactile paired with tactile, TT) stimuli and crossmodal (tactile paired with visual) stimuli that were either presented simultaneously (VT/SIM), or with the visual stimulus preceding the tactile stimulus by 100 ms (VTd, 500 ms each, visual presented first). Participants were instructed to respond to only the unimodal or crossmodal stimuli in separate blocks (Figure 1) and performed only the “detect” and the “grade” tasks as performed in Experiment 1. Crossmodal grade and detect conditions were presented in 5 blocks of 90 stimulus events each (30 each of TT, VT/SIM and VTd), totaling 10 blocks of 900 stimulus events with each block lasting approximately 6 min. Unimodal grade and detect conditions occurred in two blocks of 150 stimulus events each for a total of four blocks of 600 stimulus events. Force-amplitude associations were learned in a training session as described above and no performance feedback was provided.
Visual stimuli consisted of a centrally presented horizontal bar (6 cm wide) which raised to varying heights on a computer monitor positioned 50 cm in front of the participant and represented different visual amplitudes. Vibrotactile stimuli consisted of discrete vibrations delivered by a custom-made vibrotactile device applied to the volar surface of the left index finger. Vibrotactile stimulation was controlled by converting digitally generated waveforms to an analog signal (DAQCard 6024E, National Instruments, Austin, TX, USA) and then amplifying the signal (Bryston 2B-LP, Peterborough, ON, Canada) using a custom program written in LabVIEW (version 8.5, National Instruments, Austin, TX, USA). Varying the amplitude of the driving voltage to the vibrotactile device produced proportional changes in vibration of the device on the finger. The amplitude of each discrete vibration was constant within a trial and varied randomly between trials. The average stimulus amplitude across all trials of a block did not differ between the experimental conditions. The frequency of the vibration was held constant at 25 Hz. Participants received 70 dB whitenoise (Stim2, Neuroscan, Compumedics, Charlotte, NC, USA) throughout the training session and the experiment to prevent auditory perception of the vibrotactile stimulus.
Data Acquisition and Recording Parameters
EEG data was recorded from 32 electrode sites (32 channel Quik-Cap, Neuroscan, Compumedics, NC, USA) with an extension of the international 10–20 system for electrode placement, and referenced to the linked mastoids offline (impedance < 5 kohms). EEG data were amplified (20 000x), filtered (DC-200 Hz) and digitized at 500 Hz (Neuroscan 4.3, Compumedics, NC, USA) before being saved on computer for subsequent analysis. Individual traces were high-pass filtered (2 Hz) and visually inspected for artifacts (i.e., blinks, eye movements, or muscle contractions). Any contaminated epochs were eliminated before averaging. A minimum of at least 80 artifact-free trials were required to generate the ERP for each condition in each participant (range 80–140 trials/average).
Event-related potentials were averaged to the onset of each stimulus relative to a 100 ms pre-stimulus baseline. Somatosensory ERPs were derived from individual participant averages for each task condition. Amplitudes and latencies were computed for each subject within specified time windows centered around the post-stimulus latencies of early somatosensory ERP components: P50 (40–70 ms), P100 (80–140 ms), and N140 (150–200 ms). Figures 2 and 6 illustrate the distribution of these potentials. Amplitude and latency of the P50 component was measured from electrode site CP4 located over right sensory-motor cortex, contralateral to the vibrotactile stimulus. The P100 is typically observed bilaterally at parietal electrode sites thus amplitude and latency of this component was measured from P3, PZ, and P4. The N140 has multiple generators and broad distributions and can be recorded at frontal and parietal electrode sites; therefore for this component amplitude and latency were measured from midline electrode sites FCZ and PZ. All amplitudes were measured as raw voltage relative to the pre-stimulus baseline.
FIGURE 2. Grand average waveforms in Experiment 1 (n = 10) for the visual + tactile (VT) attended condition with the three response criteria: detect the presence of the crossmodal stimuli (black trace), judge which stimulus in the pair would require a larger response (grey trace) and grade the amplitude of the response dependent on the summed amplitudes required for the visual and tactile target stimuli (blue trace). ERPs are shown at FCZ (top), CP4 (middle), and PZ (bottom) electrode sites. ERP components of interest are labeled on the trace for electrode site CP4, indicating the latency and topographical distribution of each component. Voltage topographs over the 32 channels are shown for a 30 ms window centered around the peak for the P50, P100, and N140 components for the VT stimuli. The larger black dots represent FCZ (top), CP4 (right), and PZ (bottom). The dotted vertical line represents the time of stimulus delivery.
ERP data analysis
In Experiment 1, to test the hypothesis that crossmodal effects will be mediated by task demands, a two-way repeated measures ANOVA was carried out on the amplitude of the P50 component with motor task (three levels: detect, judge, grade) and attended modality (two levels: unimodal, crossmodal) as within subject factors. Two-way ANOVAs were also carried out on both the amplitude and latency of the P100 at PZ, and the N140 at FCZ and PZ to test for main effects and interactions on each potential. Tukey’s post hoc tests were carried out following a main effect of motor task and significant interactions were investigated by testing the effect of the attended modality across each of the task conditions using one-way ANOVAs. In Experiment 2, a repeated measures ANOVA was carried out on P50 amplitudes with the following within subject factors: motor task (two levels: detect, grade), attended modality (two levels: unimodal, crossmodal) and stimulus type (three levels: TT, SIM, VTd). Tukey’s post hoc tests were carried out following a main effect of stimulus type. Our statistical approach was restricted to the P50 component in Experiment 2 since the constant 100 ms temporal delay between the visual and tactile stimuli in the VTd condition produced an interaction with the visual ERPs over the time windows used to quantify the P100 and N140 components. Previous work using the same temporal delay between the visual and tactile stimuli as used here showed that the distribution of visual-locked ERPs spread to the central-parietal electrodes that overlap the time of the P100 but not the P50 (Popovich and Staines, 2014). Grand average traces illustrating the electrode sites and ERP components of interest are shown in Figures 2 and 6 for Experiments 1 and 2, respectively.
Behavioral data analysis
In both experiments, behavioral data were analyzed for each task. For the detection task in Experiment 1, the number of hits, misses, and false alarms were counted and for the judge task the number of correct and incorrect responses were counted and percent accuracy was calculated for both tasks. For the graded task in both Experiments, behavioral data were analyzed by summing the amplitudes of the two target stimuli and comparing this to the amplitude of the response, i.e., the force applied to the pressure-sensitive bulb. The percent difference between the summed target stimulus amplitude and the actual response amplitude was calculated. Paired t-tests were conducted to assess statistical differences between unimodal and crossmodal stimuli for each task.
Early ERP components (P50, P100, N140)
All subjects demonstrated a clear P50 component (mean +/- SE latency 56 +/- 7 ms) in response to vibrotactile stimuli and this potential was maximal at electrode site CP4, overlying contralateral somatosensory cortex (Figure 2). Main effects of attended modality (F1,9 = 5.7, p = 0.04) and motor task (F2,18 = 14.1, p = 0.001) were observed, as well as a significant interaction of attended modality × motor task (F2,18 = 4.3, p = 0.03). The VT condition was associated with greater P50 amplitude than the TT condition for the graded task (p = 0.01), and there was no difference between these conditions for detect or judge tasks (p > 0.05 for both). These results are illustrated in Figure 3A.
FIGURE 3. (A) Group means (n = 10) in Experiment 1 for P50 amplitude measured at CP4. White ars represent group data for the crossmodal visual + tactile task condition (VT), black bars represent group data for the unimodal tactile condition (TT). * Denotes significance (p < 0.05) between the specific conditions marked by the bracket. (B) Group means (n = 10) for P100 amplitude measured at PZ. Bars represent group data for the three task conditions collapsed across modality Error bars show SEM. * Denotes significance of the main effect of task (p < 0.05) encompassed by the bracket.
The P100 component was present in all task conditions (mean latency 120 +/- 7 ms) with a bilateral distribution at central and parietal sites and maximal amplitude at electrode site PZ. Figure 3B illustrates the effect of motor task observed at PZ (F2,18 = 6.1, p = 0.009), collapsed across TT and VT tasks as the attended modality was not a significant factor and no interaction was observed. Post hoc tests show that the amplitude of the P100 was larger in the detect task compared to the grade task (p = 0.02).
The N140 component was observed in all task conditions (mean latency at FCZ: 164 +/- 6 ms, at PZ: 170 +/-8 ms). A main effect of motor task was observed at PZ (F2,18 = 5.7, p = 0.01) and is illustrated in Figure 4A, collapsed across attended modality as the effect of attended modality and the interaction with motor task were not significant. Post hoc tests revealed that the detect task was associated with greater negativity than the judge (p = 0.02) and grade (p = 0.04) tasks. At FCZ, main effects of motor task (F2,18 = 12.1, p = 0.001) and attended modality (F1,9 = 7.1, p = 0.026) were significant, but no interaction was observed. As illustrated in Figure 4B and revealed by post hoc tests, for the TT condition the grade task was associated with greater negativity than the detect task (p = 0.01). This was also true of the VT condition (p = 0.02), as well as a significant difference between the grade task and the judge task (p = 0.04).
FIGURE 4. Group means (n = 10) for N140 amplitude measured at PZ (A) and FCZ (B) in Experiment 1. White bars represent group data for the crossmodal visual + tactile TT task condition (VT), black bars represent group data for the unimodal tactile condition (TT). Error bars show SE. * Denotes significance (p < 0.05) between the specific conditions marked by the brackets (details in the text).
Paired t-tests were carried out on the behavioral data for each task to test whether the unimodal task differed from the crossmodal task. No significant differences were found for any of the tasks (p > 0.05; Figure 5A).
FIGURE 5. Group means (n = 10) for behavioural performance in Experiments 1 (A) and 2 (B). Performance is shown for the crossmodal visual + tactile task condition where the stimuli were presented simultaneously (VT, white bars) or with a 100 ms delay (VTd, gray bars). Black bars represent group data for the unimodal tactile condition (TT). Error bars show SEM, * denotes significance p < 0.05.
Early ERP components (P50)
ERP analyses in Experiment 2 were focused on the P50 because the constant 100 ms temporal delay between the visual and tactile stimuli in the VTd condition produced an interaction with the visual ERPs over the time window (90–125 ms) chosen for the P100 peak amplitude. All subjects demonstrated a clear P50 component in response to vibrotactile stimuli presented to the left index finger. Figure 6 shows the grand averaged waveforms for the VTd stimuli in all conditions at electrode site CP4 approximately overlying the contralateral somatosensory cortex. Ignore VTd represents ERPs to VTd stimuli (timelocked to the tactile stimulus) when participants were responding to unimodal (TT) stimuli. Scalp topography maps representing group averaged data were created by averaging neural responses generated over the 30 ms time window (40–70 ms) centered around the P50 peak to observe task-specific differences in cortical modulation (Figure 6B). As illustrated in Figure 6, all conditions including vibrotactile stimuli elicited robust neural activity over somatosensory regions contralateral to stimulation. Notably, the VTd conditions also elicited robust activation over modality-specific visual cortex. Statistical results showed a main effect of the stimulus type (F2,99 = 11.1, p < 0.0001) and the post hoc Tukey’s test revealed that the P50 amplitude was largest in the VTd conditions where the visual stimulus preceded the tactile stimulus by 100 ms compared to both the simultaneous (VT/SIM) or unimodal tactile (TT) conditions at electrode CP4 (p < 0.05). Figure 7 shows P50 amplitudes for the Grade and Detect tasks for both VTd and TT stimulation. P50 amplitude was maximal in the condition where participants graded their force to the crossmodal stimuli and the visual stimulus preceded the stimulus by 100 ms.
FIGURE 6. (A) Grand average waveforms for the crossmodal VTd conditions in Experiment 2 at CP4. The P50 ERP component is labeled on the trace for electrode site CP4. Blue, red, and black traces show VTd stimuli in the detect, grade or when attention was directed to the unimodal stimuli (Ignore VTd), respectively. The gray trace shows the ERP time-locked to the TT stimuli when participants were responding to the crossmodal stimuli in the Grade response condition (Ignore TT-Grade). (B) Scalp Topography Maps of the P50 Component: Inset shows modulation of the P50 ERP waveforms in response to conditions in A. Images show group averaged data of peak areas of cortical activity generated over a 30 ms time window (40–70 ms) centered around the P50 ERP peak. All values are in microvolts (μV).
FIGURE 7. (A) Group means (n = 10) in Experiment 2 for P50 amplitudes measured at CP4 for the Grade and Detect tasks for both VTd and TT stimulation. P50 amplitude was maximal in the condition where participants graded their force to the crossmodal stimuli and the visual stimulus preceded the tactile TT stimulus by 100 ms (Attend VTd-G; purple bar). Purple and green bars represent group data for the crossmodal visual + tactile task conditions (VTd) in which subjects graded their force for the response (G) or indicated detection of the target stimulus (D). The gray bar represents the same VTd condition during blocks where attention was directed to the unimodal (TT) stimuli (blue/red bars). Error bars show SEM.
Paired t-tests were carried out on the behavioral data for each task to test whether the unimodal task differed from the crossmodal task. Performance in the graded task was significantly more accurate when the target stimuli were crossmodal and the VTd compared to the unimodal task (p = 0.02). No other significant differences were found for any of the other task conditions (Figure 5B).
This study set out to probe the role of behavioral task requirements in mediating crossmodal modulation of early modality-specific somatosensory cortical processing represented by early ERP components. The greater purpose was to provide insight into what the somatosensory P50 may represent by testing the hypothesis that it would be sensitive to changes in task set despite identical stimulus parameters. It was predicted that P50 modulation would be sensitive to task demands, specifically that crossmodal modulation would be maximal in the grade task, when the relationship between stimulus attributes and motor response was greatest. More importantly this condition required cortical networks involved in motor preparation and selective attention to be both active. In addition, it was hypothesized that having the visual target information onset slightly earlier, allowing sufficient processing time for the potential interaction between the sensory modalities, would enhance such crossmodal modulation. Overall, the hypotheses were supported by the data. Responses from modality-specific somatosensory cortex (S1) were greatest when the required motor task was dependent on extracting stimulus details in order to accurately carry out the execution of the specific movement. Such a task involves attention directed to the sensory target modalities as well as preparation in the cortical motor areas. This enhancement of the modality-specific cortical response was further enhanced when the temporal arrangement of the crossmodal stimuli allowed for interaction between the visual and somatosensory modalities.
Early Modality Specific Effects
In the present study, crossmodal modulation of the P50 was replicated as in previous studies when the behavioral task required a graded motor response linked to the amplitude of the stimuli. Yet this modulation was not observed when the task requirements were changed despite the stimuli being identical. This finding provides fairly compelling evidence that crossmodal effects on the P50 are mediated by the demands of the task. The detection, discrimination, and graded tasks represent varying requirements to extract sensory information in order to make the appropriate motor response, with the detection task being the least demanding and the graded task the most demanding. In support of this argument, the behavioral data show that the detection task was performed with near perfect accuracy, whereas the graded task was associated with less accurate performance, which would suggest it was the most difficult to perform. However, in Experiment 1 there were no differences in the accuracy of each task between the VT and TT conditions, so although the crossmodal stimuli are associated with increased P50 amplitude in the most demanding task in a way that unimodal tactile stimuli are not, this difference is not associated with a behavioral benefit in terms of performance accuracy. When the conditions were altered in Experiment 2 such that the visual stimulus was available prior to the tactile target there was an association between behavioral performance and the excitability of early somatosensory responses. As shown in Figures 5B and 6, performance in the VTd grade task was significantly more accurate than in the unimodal task (Figure 5B) while the P50 amplitude was significantly enhanced relative to the others (Figure 6).
Considering all these findings together, some interesting possibilities arise about what the somatosensory P50 might represent. The lack of crossmodal effects in the detection and discrimination tasks argues strongly against the idea of a global crossmodal facilitation in response to visual and tactile stimuli. The finding that crossmodal effects only emerge under specific task requirements suggests that this potential is likely to reflect cognitive strategies being applied to the processing of the sensory inputs at an early stage, as was suggested by Desmedt and Tomberg (1989) in their discussion of what they termed the cognitive P40, which showed a similar topographical distribution to the P50 recorded in this experiment. A study by Schubert et al. (2008) provides support for this idea and suggests that early sensory-specific modulations are associated with more demanding tasks. Such enhancements are presumably mediated via reciprocal thalamo-cortical networks that act to bias processing towards selected inputs (Yingling and Skinner, 1976; Brunia, 1993).
It is important to note that in all cases the somatosensory ERPs are time-locked to the onset of the tactile stimuli. As such the amplitudes of the early components reported here will not be affected by the small difference in stimulus durations for the tactile stimuli presented in the uni- and bimodal conditions of Experiment 1. This is supported by the similarities of the waveforms, especially the consistent timing of the P50, P100, and N140 components in the unimodal and bimodal conditions reported in previous studies using these same parameters (Dionne et al., 2013; Popovich and Staines, 2014). In addition, in the case when the visual stimulus precedes the tactile stimulus by 100 ms (VT Grade) shown in Figure 6 (red trace) there is some indication of alpha-like activity. The absence of this in the ERPs time-locked to the unimodal, tactile stimulus in the same block suggests that this was not inherent to the task.
An entirely different pattern of results was observed for the P100 component in Experiment 1, which did not demonstrate sensitivity to crossmodal effects but did show an effect of task. The detection task was associated with larger P100 amplitude than the graded task, regardless of whether the stimuli were crossmodal or unimodal. The P100 is bilaterally distributed over parietal electrode sites and is thought to be generated in bilateral secondary somatosensory cortex (S2; Mima et al., 1998; Gu, 2002). Several studies report sensitivity of this potential to effects of attention (Josiassen et al., 1982; Desmedt et al., 1983; Michie et al., 1987; Eimer and Forster, 2003; Kida et al., 2004; Schubert et al., 2006), and functional imaging studies show modulation of S2 when attention is directed towards TT stimuli (Meyer et al., 1991; Johansen-Berg et al., 2000; Staines et al., 2002; Nelson et al., 2004). While the effects of attention on the P100 appear fairly consistent, studies investigating crossmodal influences on mid-latency components are mixed. Studies employing an oddball detection task that required subjects to monitor both hands for deviant stimuli in one modality while ignoring the other and reporting detections with a verbal response also failed to find crossmodal effects on this potential (Eimer and Driver, 2000; Eimer, 2001). Dionne et al. (2013) reported that the P100 was sensitive to the presence of crossmodal stimuli but not to the task-relevance of those stimuli during the performance of a graded motor task, yet that same effect was not observed in the current study. However, there are differences in the design of these two experiments that may account for this discrepancy. In the current study the stimuli received by participants were intermixed pairs of either tactile (TT) or visual and tactile (VT) stimuli, and they were required to attend to either the unimodal (TT) or crossmodal (VT) events, essentially a crossmodal oddball task with varying motor requirements. In the previous experiment, pairs of crossmodal or unimodal stimuli were presented in different blocks while participants performed the same task. Thus it seems that the requirement to selectively attend to crossmodal stimuli intermixed with unimodal stimuli does not modulate the P100, but sustaining attention to crossmodal stimuli enhances this component compared to sustaining attention to unimodal stimuli.
Longer Latency Effects
The N140 component was sensitive to task requirements at both the parietal and frontal sites, but while the N140 was more sensitive to crossmodal stimuli than unimodal stimuli at the frontal site, at the parietal site it was not. Also of interest, the two sites showed different patterns of task-related N140 modulation. At the parietal site, greater negativity was associated with the detection task than the discrimination and graded tasks, whereas at the frontal site the graded task was associated with largest negativity.
The somatosensory N140 is evoked by tactile and electrical stimuli in association with cognitive tasks and typically occurs at latencies between 100 and 180 ms (Desmedt and Robertson, 1977; Josiassen et al., 1990; Allison et al., 1992). The topography of this large negative deflection most often begins over the contralateral frontal electrodes and extends towards the midline and ipsilateral sides. The prevailing thought on the generators of the N140 is that it reflects activation of prefrontal cortex, an area that is known to play a role in retrieving and maintaining representations of stimulus relevancy in relation to behavioral goals (Desmedt and Tomberg, 1989; Allison et al., 1991; Pardo et al., 1991; Knight et al., 1995), however, the bilateral distribution of the N140 at parietal sites has also prompted some researchers to suggest a generator in S2 (Frot and Mauguiere, 1999), which as mentioned previously has been implicated in attention to tactile stimuli. Similar to the current study, the results of Dionne et al. (2013) also showed different patterns of N140 modulation at frontal and parietal sites and these results provide further support that this component might be associated with multiple generators such that the frontal and parietal generators are sensitive to different factors. Given the results of these two studies, it seems that the parietal component of the N140 shows specificity for task demands but not crossmodal stimuli, and the frontal component is sensitive to both crossmodal stimuli in addition to task requirements. The finding that different task demands produce different patterns of modulation at frontal and parietal sites likely speaks to the functions associated with the underlying cortical areas. While both frontal and parietal regions are part of a fronto-parietal network attention network, the nodes in this network subserve different aspects of attending to and responding to stimuli. Areas of the parietal cortex, particularly the temporoparietal junction are known to play a role in stimulus-driven shifts of attention, regardless of stimulus modality (Downar et al., 2001; Corbetta and Shulman, 2002; Kincade et al., 2005), thus parietal contributions to the N140 may be most responsive to the detection task that relies more on this form of stimulus-driven attention. In contrast, at frontal sites the N140 is most responsive to the graded task, which requires greater extraction of stimulus features to execute an accurate motor response. This finding fits with a role for prefrontal cortex in transforming sensory information into behavior via gating mechanisms that bias activity in sensory-specific cortex in response to behavioral demands (Gazzaley et al., 2005, 2007).
Crossmodal effects on early somatosensory cortical processing, represented by the P50 component, depends on the engagement of the attentional system and on the specific requirements of the behavioral task, suggesting this component may reflect the application of cognitive strategies to sensory processing and extraction of relevant features. There may be functional interaction of systems responsible for attention, multimodal sensory integration and motor preparation that contribute to modulation of modality-specific somatosensory cortex. Task demands also modulate mid-latency components, with frontal and parietal sites showing sensitivity to different task requirements.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
This work was supported by funding to WRS from the Natural Sciences and Engineering Research Council of Canada (NSERC) and the Canada Research Chairs program. Christina Popovich was supported by graduate scholarship funds from the Ontario Graduate Scholarship (OGS) program.
Allison, T., McCarthy, G., and Wood, C. C. (1992). The relationship between human long-latency somatosensory evoked potentials recorded from the cortical surface and from the scalp. Electroencephalogr. Clin. Neurophysiol. 84, 301–314. doi: 10.1016/0168-5597(92)90082-M
Allison, T., McCarthy, G., Wood, C. C., and Jones, S. J. (1991). Potentials evoked in human and monkey cerebral cortex by stimulation of the median nerve. A review of scalp and intracranial recordings. Brain 114, 2465–2503. doi: 10.1093/brain/114.6.2465
Andersen, S. K., Hillyard, S. A., and Muller, M. M. (2008). Attention facilitates multiple stimulus features in parallel in human visual cortex. Curr. Biol. 18, 1006–1009. doi: 10.1016/j.cub.2008.06.030
Calvert, G. A., Bullmore, E. T., Brammer, M. J., Campbell, R., Williams, S. C., McGuire, P. K., et al. (1997). Activation of auditory cortex during silent lipreading. Science 276, 593–596. doi: 10.1126/science.276.5312.593
Chapman, C. E. (1994). Active versus passive touch: factors influencing the transmission of somatosensory signals to primary somatosensory cortex. Can. J. Physiol. Pharmacol. 72, 558–570. doi: 10.1139/y94-080
Cheron, G., and Borenstein, S. (1991). Gating of the early components of the frontal and parietal somatosensory evoked potentials in different sensory-motor interference modalities. Electroencephalogr. Clin. Neurophysiol. 80, 522–530. doi: 10.1016/0168-5597(91)90134-J
Desmedt, J. E., Huy, N. T., and Bourguet, M. (1983). The cognitive P40, N60 and P100 components of somatosensory evoked potentials and the earliest electrical signs of sensory processing in man. Electroencephalogr. Clin. Neurophysiol. 56, 272–282. doi: 10.1016/0013-4694(83)90252-3
Desmedt, J. E., and Robertson, D. (1977). Differential enhancement of early and late components of the cerebral somatosensory evoked potentials during forced-paced cognitive tasks in man. J. Physiol. 271, 761–782.
Desmedt, J. E., and Tomberg, C. (1989). Mapping early somatosensory evoked potentials in selective attention: critical evaluation of control conditions used for titrating by difference the cognitive P30, P40, P100 and N140. Electroencephalogr. Clin. Neurophysiol. 74, 321–346. doi: 10.1016/0168-5597(89)90001-4
Dionne, J. K., Legon, W., and Staines, W. R. (2013). Crossmodal influences on early somatosensory processing: interaction of vision, touch, and task-relevance. Exp. Brain Res. 226, 503–512. doi: 10.1007/s00221-013-3462-z
Downar, J., Crawley, A. P., Mikulis, D. J., and Davis, K. D. (2001). The effect of task relevance on the cortical response to changes in visual and auditory stimuli: an event-related fMRI study. Neuroimage 14, 1256–1267. doi: 10.1006/nimg.2001.0946
Eimer, M. (2001). Crossmodal links in spatial attention between vision, audition, and touch: evidence from event-related brain potentials. Neuropsychologia 39, 1292–1303. doi: 10.1016/S0028-3932(01)00118-X
Eimer, M., and Driver, J. (2000). An event-related brain potential study of cross-modal links in spatial attention between vision and touch. Psychophysiology 37, 697–705. doi: 10.1111/1469-8986.3750697
Foxe, J. J., and Simpson, G. V. (2002). Flow of activation from V1 to frontal cortex in humans. A framework for defining “early” visual processing. Exp. Brain Res. 142, 139–150. doi: 10.1007/s00221-001-0906-7
Foxe, J. J., Morocz, I. A., Murray, M. M., Higgins, B. A., Javitt, D. C., and Schroeder, C. E. (2000). Multisensory auditory-somatosensory interactions in early cortical processing revealed by high-density electrical mapping. Brain Res. 10, 77–83. doi: 10.1016/S0926-6410(00)00024-0
Foxe, J. J., Wylie, G. R., Martinez, A., Schroeder, C. E., Javitt, D. C., Guilfoyle, D., et al. (2002). Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study. J. Neurophysiol. 88, 540–543.
Frot, M., and Mauguiere, F. (1999). Timing and spatial distribution of somatosensory responses recorded in the upper bank of the sylvian fissure (SII area) in humans. Cereb. Cortex 9, 854–863. doi: 10.1093/cercor/9.8.854
Gazzaley, A., Cooney, J. W., McEvoy, K., Knight, R. T., and D’Esposito, M. (2005). Top-down enhancement and suppression of the magnitude and speed of neural activity. J. Cogn. Neurosci. 17, 507–517. doi: 10.1162/0898929053279522
Gazzaley, A., Rissman, J., Cooney, J., Rutman, A., Seibert, T., Clapp, W., et al. (2007). Functional interactions between prefrontal and visual association cortex contribute to top-down modulation of visual processing. Cereb. Cortex 17:(Suppl. 1), i125–il35. doi: 10.1093/cercor/bhm113
Hämäläinen, H., Kekoni, J., Sams, M., Reinikainen, K., and Naatanen, R. (1990). Human somatosensory evoked potentials to mechanical pulses and vibration: contributions of SI and SII somatosensory cortices to P50 and P100 components. Electroencephalogr. Clin. Neurophysiol. 75, 13–21. doi: 10.1016/0013-4694(90)90148-D
Hsiao, S. S., O’Shaughnessy, D. M., and Johnson, K. O. (1993). Effects of selective attention on spatial form processing in monkey primary and secondary somatosensory cortex. J. Neurophysiol. 70, 444–447.
Jäncke, L., Mirzazade, S., and Shah, N. J. (1999). Attention modulates activity in the primary and the secondary auditory cortex: a functional magnetic resonance imaging study in human subjects. Neurosci. Lett. 266, 125–128. doi: 10.1016/S0304-3940(99)00288-8
Jones, S. J., Halonen, J. P., and Shawkat, F. (1989). Centrifugal and centripetal mechanisms involved in the ‘gating’ of cortical SEPs during movement. Electroencephalogr. Clin. Neurophysiol. 74, 36–45. doi: 10.1016/0168-5597(89)90049-X
Josiassen, R. C., Shagass, C., Roemer, R. A., Ercegovac, D. V., and Straumanis, J. J. (1982). Somatosensory evoked potential changes with a selective attention task. Psychophysiology 19, 146–159. doi: 10.1111/j.1469-8986.1982.tb02536.x
Josiassen, R. C., Shagass, C., Roemer, R. A., Slepner, S., and Czartorysky, B. (1990). Early cognitive components of somatosensory event-related potentials. Int. J. Psychophysiol. 9, 139–149. doi: 10.1016/0167-8760(90)90068-O
Kayser, C., Petkov, C. I., Augath, M., and Logothetis, N. K. (2007). Functional imaging reveals visual modulation of specific fields in auditory cortex. J. Neurosci. 27, 1824–1835. doi: 10.1523/JNEUROSCI.4737-06.2007
Kida, T., Nishihira, Y., Wasaka, T., Nakata, H., and Sakamoto, M. (2004). Passive enhancement of the somatosensory P100 and N140 in an active attention task using deviant alone condition. Clin. Neurophysiol. 115, 871–879. doi: 10.1016/j.clinph.2003.11.037
Kincade, J. M., Abrams, R. A., Astafiev, S. V., Shulman, G. L., and Corbetta, M. (2005). An event-related functional magnetic resonance imaging study of voluntary and stimulus-driven orienting of attention. J. Neurosci. 25, 4593–4604. doi: 10.1523/JNEUROSCI.0236-05.2005
Lakatos, P., Chen, C. M., O’Connell, M. N., Mills, A., and Schroeder, C. E. (2007). Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron 53, 279–292. doi: 10.1016/j.neuron.2006.12.011
Lehmann, C., Herdener, M., Esposito, F., Hubl, D., Di Salle, F., Scheffler, K., et al. (2006). Differential patterns of multisensory interactions in core and belt areas of human auditory cortex. Neuroimage 31, 294–300. doi: 10.1016/j.neuroimage.2005.12.038
Macaluso, E., Frith, C. D., and Driver, J. (2002). Crossmodal spatial influences of touch on extrastriate visual areas take current gaze direction into account. Neuron 34, 647–658. doi: 10.1016/S0896-6273(02)00678-5
Meehan, S. K., and Staines, W. R. (2009). Task-relevance and temporal synchrony between tactile and visual stimuli modulates cortical activity and motor performance during sensory-guided movement. Hum. Brain Mapp. 30, 484–496. doi: 10.1002/hbm.20520
Meyer, E., Ferguson, S. S., Zatorre, R. J., Alivisatos, B., Marrett, S., Evans, A. C., et al. (1991). Attention modulates somatosensory cerebral blood flow response to vibrotactile stimulation as measured by positron emission tomography. Ann. Neurol. 29, 440–443. doi: 10.1002/ana.410290418
Michie, P. T., Bearpark, H. M., Crawford, J. M., and Glue, L. C. (1987). The effects of spatial selective attention on the somatosensory event-related potential. Psychophysiology 24, 449–463. doi: 10.1111/j.1469-8986.1987.tb00316.x
Mima, T., Nagamine, T., Nakamura, K., and Shibasaki, H. (1998). Attention modulates both primary and second somatosensory cortical activities in humans: a magnetoencephalographic study. J. Neurophysiol. 80, 2215–2221.
Nelson, A. J., Staines, W. R., Graham, S. J., and Mcllroy, W. E. (2004). Activation in SI and SII: the influence of vibrotactile amplitude during passive and task-relevant stimulation. Cogn. Brain Res. 19, 174–184. doi: 10.1016/j.cogbrainres.2003.11.013
Pekkola, J., Ojanen, V., Autti, T., Jääskeläinen, I. P., Möttönen, R., and Sams, M. (2006). Attention to visual speech gestures enhances hemodynamic activity in the left planum temporale. Hum. Brain Mapp. 27, 471–477. doi: 10.1002/hbm.20190
Popovich, C., and Staines, W. R. (2014). The attentional-relevance and temporal dynamics of visual-tactile crossmodal interactions differentially influence early stages of somatosensory processing. Brain Behav. 4, 247–260. doi: 10.1002/brb3.210
Schubert, R., Blankenburg, F., Lemm, S., Villringer, A., and Curio, G. (2006). Now you feel it–now you don’t: ERP correlates of somatosensory awareness. Psychophysiology 43, 31–40. doi: 10.1111/j.1469-8986.2006.00379.x
Schubert, R., Ritter, P., Wustenberg, T., Preuschhof, C., Curio, G., Sommer, W., et al. (2008). Spatial attention related SEP amplitude modulations covary with BOLD signal in S1–a simultaneous EEG–fMRI study. Cereb. Cortex 18, 2686–2700. doi: 10.1093/cercor/bhn029
Staines, W. R., Brooke, J. D., Cheng, J., Misiaszek, J. E., and MacKay, W. A. (1997). Movement-induced gain modulation of somatosensory potentials and soleus H-reflexes evoked from the leg. I. Kinaesthetic task demands. Exp. Brain Res. 115, 147–155. doi: 10.1007/PL00005674
Staines, W. R., Graham, S. J., Black, S. E., and McIlroy, W. E. (2002). Task-relevant modulation of contralateral and ipsilateral primary somatosensory cortex and the role of a prefrontal-cortical sensory gating system. Neuroimage 15, 190–199. doi: 10.1006/nimg.2001.0953
Woldorff, M. G., Gallen, C. C., Hampson, S. A., Hillyard, S. A., Pantev, C., Sobel, D., et al. (1993). Modulation of early sensory processing in human auditory cortex during auditory selective attention. Proc. Natl. Acad. Sci. U.S.A. 90, 8722–8726. doi: 10.1073/pnas.90.18.8722
Yingling, C. D., and Skinner, J. E. (1976). Selective regulation of thalamic sensory relay nuclei by nucleus reticularis thalami. Electroencephalogr. Clin. Neurophysiol. 41, 476–482. doi: 10.1016/0013-4694(76)90059-6
Zopf, R., Giabbiconi, C. M., Gruber, T., and Muller, M. M. (2004). Attentional modulation of the human somatosensory evoked potential in a trial-by-trial spatial cueing and sustained spatial attention task measured with high density 128 channels EEG. Cogn. Brain Res. 20, 491–509. doi: 10.1016/j.cogbrainres.2004.02.014
Keywords: event-related potentials (ERPs), attention, crossmodal, visual, tactile, somatosensory cortex, sensorimotor integration
Citation: Staines WR, Popovich C, Legon JK and Adams MS (2014) Early modality-specific somatosensory cortical regions are modulated by attended visual stimuli: interaction of vision, touch, and behavioral intent. Front. Psychol. 5:351. doi: 10.3389/fpsyg.2014.00351
Received: 26 November 2013; Accepted: 03 April 2014;
Published online: 24 April 2014.
Edited by:Alexander Jones, Middlesex University London, UK
Reviewed by:Harold H. Greene, University of Detroit Mercy, USA
Tobias Heed, Universität Hamburg, Germany
Copyright © 2014 Staines, Popovich, Legon and Adams. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: W. Richard Staines, Department of Kinesiology, University of Waterloo, BMH 3031, 200 University Avenue West, Waterloo, ON N2L 3G1, Canada e-mail: firstname.lastname@example.org