Convergent approaches toward the study of multisensory perception

Introduction

Because we live in a world composed of a complex amalgam of sensory information, it is only through the ability to combine the various forms of this information that a meaningful behavioral and perceptual gestalt (an organized “whole” greater than the sum of its parts) can be formed. Furthermore, the adaptive advantages that multisensory integration confers are critical to survival, and often allow appropriate behavioral responses to be generated under circumstances in which information from one sense is inadequate. Enhanced orientation (Stein et al., 1988, 1989), improved target detection (Frassinetti et al., 2002; Lovelace et al., 2003), and faster responses (Hershenson, 1962; Hughes et al., 1994; Frens et al., 1995; Harrington and Peck, 1998; Murray et al., 2001; Corneil et al., 2002; Forster et al., 2002; Molholm et al., 2002; Amlot et al., 2003; Diederich et al., 2003) are among the multitude of behavioral benefits seen when information is combined from two or more sensory modalities.

These behavioral and perceptual changes invoked under multisensory conditions reflect a series of neural computations involving the convergence and integration of inputs from the different sensory modalities. The presence of such convergence and integration can be inferred from the fact that behavioral responses are often faster than those predicted by a simple probability summation of the responses to the sensory cues presented individually (Hughes et al., 1994, 1998; Corneil and Munoz, 1996; Harrington and Peck, 1998). Further evidence for links between neural activity and its behavioral correlates have been seen when comparing and relating the activity of multisensory neurons to behavioral responses. For example, stimulus combinations that enhance the activity of multisensory neurons in the superior colliculus (SC) also enhance an animal's orientation abilities (Stein et al., 1988, 1989). Spatially and temporally coincident audiovisual stimulus combinations improve the animal's ability to detect and approach the correct location, whereas spatially disparate stimuli reduce the percentage of correct responses (Stein et al., 1988, 1989). Furthermore, when stimulus intensity is manipulated, the least effective stimuli (e.g., a dim LED that yields a low neuronal response) produce the greatest behavioral gains (Stein et al., 1989).

Multisensory Processing: Classical Neurophysiological Analyses

Although there is a high degree of multisensory convergence at many sites throughout the central nervous system, the foundation for examining the physiological underpinnings of multisensory integration has focused on a midbrain structure, the SC (e.g., Stein and Meredith, 1993). The reasons for the choice and preeminence of this model are manifold, but include its high incidence of multisensory neurons, well-established topographic organization, and well-characterized role in mediating orientation movements of the eyes and head. Using the SC as a model, Stein and Meredith conducted seminal studies that characterized the basic principles by which multisensory neurons synthesize their inputs from multiple modalities (Meredith and Stein, 1983, 1985, 1986a,b, 1996; Meredith et al., 1987; Stein, 1988; Stein et al., 1988). Their work showed that multisensory (e.g., visual-auditory) stimulus pairs presented in close spatial and temporal proximity typically resulted in large response enhancements, a gain that makes intuitive sense given that stimuli resulting from a singular event are bound by common spatial and temporal properties (Meredith and Stein, 1983, 1986a; Meredith et al., 1987; Wallace et al., 1997, 1998; Jiang et al., 2002; Burnett et al., 2004). Furthermore, they showed that the largest response enhancements were seen with the pairing of weakly effective stimuli, whereas combining increasingly effective stimuli yielded less gain (Perrault et al., 2005; Stanford et al., 2005). This principle, known as inverse effectiveness, also makes intuitive and ethological sense given that response amplification from the additional sensory signal becomes unnecessary when one signal alone is highly salient, and therefore sufficient. Whereas such midbrain studies targeted the deep layers of the SC—where inputs from auditory, somatosensory, and visual modalities converge—recent studies have assessed more perceptual aspects of multisensory processing by targeting cortical areas such as the anterior ectosylvian sulcus (AES) of cats and ferrets, or the posterior parietal cortex, particularly the superior temporal sulcus (STS) of primates (for review, see Stein and Stanford, 2008). In contrast to mediation of head and eye movements directly involved with the SC, cortical multisensory areas are guided by similar substrates (changes in activity related to spatial and temporal congruence as well as stimulus efficacy) to mediate perceptual binding, including cross-modal cues involving congruent motion and audiovisual vocal communication (Barraclough et al., 2005; Royal et al., 2009).

The focus of these early neurophysiological response analyses of multisensory neurons [defined as neurons that respond to, or are influenced by, stimuli from more than one sensory modality (Stein and Stanford, 2008)] was on the mean number of spikes evoked per stimulus presentation (analyzed as single unit activity, or SUA). Using spike count metrics, this work characterized the responses to both single modality (i.e., visual alone, auditory alone) and combined modality (i.e., visual-auditory) stimulation, and used this to gauge whether multisensory stimuli resulted in response enhancement (a significant increase in the mean number of spikes when compared with the most effective unisensory stimulus), response depression (a significant decrease relative to the best modality response), or no interaction between the sensory modalities (Meredith and Stein, 1983, 1986b; Meredith et al., 1987; Wallace et al., 1996, 1998; Jiang et al., 2002; Burnett et al., 2004). To quantify the magnitude of these effects, the interactive index (% interaction, or ii) (Meredith and Stein, 1983) is calculated as:

$[(CM-S{M}_{\text{max}})/S{M}_{\text{max}}]\times 100=\%\text{\hspace{0.17em}interaction}$

where CM is the mean number of spikes per trial evoked by combined-modality stimulation and SM_max is the mean number of spikes evoked by the most effective single-modality stimulus. The power of the interactive index is that it shows the gain (or loss) of response attributable to the presence of a stimulus in a second sensory modality—thereby demonstrating the presence or absence of a multisensory interaction—a measure with undeniable ethological validity.

Multisensory Processing: Additional Analytical Approaches

One shortfall of the interactive index measure is its use of the strongest unisensory response alone as the comparator, which fails to incorporate the influence of the second sensory modality. To circumvent this concern, analyses structured around an additive model began to be commonly used. This method creates a predicted multisensory response based on the addition of the two unisensory responses, which can then be contrasted against the actual observed response using the mean statistical contrast (or multisensory contrast, msc) measure:

$\sum \text{[}(SA-S{M}_1)-(S{M}_2-CM)]/n=\text{mean statistical contrast}$

where SA is the spontaneous activity, SM₁ is the first single-modality response (e.g., auditory), SM₂ is the second single-modality response (e.g., visual), CM is the combined-modality response, and n is the number of trials. In each case the response is defined as the mean number of spikes per trial evoked for the duration specific to each response (SM₁, SM₂, and CM, generally using 10 ms bins) while SA is the average spikes per trial during 500 ms interval prior to stimulus onset. Using this equation, only responses that exceed the level of spontaneous activity affect the mean statistical contrast. This model assumes independence between inputs from each sensory modality and utilizes additive factors logic to distinguish between superadditive (contrast > 0) and subadditive (contrast < 0) responses (Perrault et al., 2003, 2005; Stanford et al., 2005). Thus, msc characterizes the type of integration present, beyond simply determining enhancement vs. depression of the response, by incorporating both component unisensory responses—rather than only the strongest—as a metric for classifying integration effects.

Although mean statistical contrast is a powerful tool to measure multisensory integration (and can be complemented with use of the interactive index), it must be recognized that these metrics still rely on changes in the mean firing profile of the neurons under study. Studies within sensory systems have illustrated that information can be encoded in forms that differ from these traditional spike-based measures—a series of findings that are beginning to be extended into multisensory systems. Alternative measures such as mean response duration, response latency (measured either as mean response latency or first spike latency), and peak firing rate (measured from the time bin in which the maximum number of spikes occurred post-stimulus) are also used to quantify multisensory integration. These measures provide valuable insights into temporal response dynamics and their effect on the integrative capacity of multisensory neurons (Meredith et al., 1987; Royal et al., 2009; Ghose et al., 2012). Together with measures of response variability (further described below), such metrics help to reveal encoding strategies that may not be evident in studying firing rate changes alone.

Changes in Neuronal Response Variability: Fano Factor Analysis

An integral concept when studying sensory (and multisensory) systems is that of reliability. In psychophysical studies, reliability is generally framed from the perspective of cue weighting, with the relative cue weights being a function of the reliability of the various sensory inputs (Ernst and Banks, 2002; Shams et al., 2005; Burr and Alais, 2006; Burge et al., 2010). Observers of sensory stimuli tend to employ an optimal strategy that weights each cue in proportion to its reliability, a behavioral finding supported by predictive neural population responses as well (Fetsch et al., 2012). Cue reliability also has strong relevance for multisensory encoding. In a simple multisensory context, one can envision a situation in which one of the sensory inputs (e.g., vision) is providing much more reliable information than the other modality (e.g., audition), and hence should be weighted more in an evaluation of the sensory evidence, such as during the localization of an object in space. The concept of cue reliability can be readily extended into the neural domain, in which the metric of interest is the variability of the neuronal response (in essence the opposite of reliability, since the variability of the response would directly impact how reliably a stimulus is encoded) upon repeated stimulus presentations. Response variability of spike counts is captured in the Fano factor (FF) calculation:

$FF={\sigma }^2/\mu$

in which the ratio of variance (σ²) to the mean (μ) of spike counts is computed across trials and averaged over a specific time window of single unit neuronal response (Fano, 1947). A FF-value of one indicates neuronal responses that are as reliable as would be found from a Poisson process (Fano, 1947; Softky and Koch, 1993; De Ruyter Van Steveninck et al., 1997; Kara et al., 2000; Eden and Kramer, 2010). It is important to note that the level of neuronal activity (magnitude of firing rate) plays an important role in the determination of FF, since mean firing rate is incorporated into the FF calculation. Thus, at high firing rates, responses are typically less variable (Tolhurst et al., 1983; Softky and Koch, 1993; Holt et al., 1996; Kara et al., 2000; Carandini, 2004; Gur and Snodderly, 2006). One reason for this decline in variability at very high firing rates is the refractory period, which constrains the temporal profile of a response (Berry and Meister, 1998; Kara et al., 2000). Recent studies have also shown that FF-values can vary depending on the brain region and the effectiveness of the sensory stimulus (Kara et al., 2000; Gur and Snodderly, 2006; Kayser et al., 2010; Mochol et al., 2010). For instance, in the cat SC, FF has been used to distinguish between parallel processing channels (W and Y, with superficial layer neurons receiving inputs from each channel) such that slow-moving visual stimuli showed increased FF-values with increased firing rate, whereas fast-moving stimuli resulted in FF-values that correlated negatively with firing rate (Mochol et al., 2010).

Changes in response reliability are potentially very meaningful from an information encoding perspective, as they could be used as weighting factors in neural processes responsible for cue combination (Fetsch et al., 2012). One tangible example of the use of FF as a tool has come from studies that have tied response variability to functional relevance and behavioral outcomes. In the prefrontal cortex of macaques, FF-values were shown to change during the components of a motion discrimination task (Figure 1). Thus, a stimulus-induced drop in FF was present when compared to fixation, delay, and post-test periods, along with a preparatory drop in FF preceding the test component in neurons that were able to discriminate between sample and test stimuli (random-dot stimuli, with the “sample” phase intended for identification and memorization of stimulus direction and the “test” phase intended for comparison in order to assess accuracy of direction discrimination by varying the difference between directions in each). FF-values also varied with behavioral performance such that higher variability was observed during passive fixation conditions compared to visual discrimination tasks, an effect that is thought to reflect the level of engagement in the task (Hussar and Pasternak, 2010).

FIGURE 1

Figure 1. Average Fano factor values change with the components of a motion discrimination task (A) in neuronal recordings from monkey prefrontal cortex. A stimulus-induced drop in FF was present compared to fixation, delay, and post-test periods (B), along with a preparatory drop in FF preceding the test component in neurons that were able to discriminate between sample and test stimuli (C). Fano factor values also varied with behavioral performance such that higher variability was observed during passive fixation conditions than during the visual discrimination task, an effect that was thought to reflect the level of engagement in the task (D). “Excitatory” and “suppressive” (B) refers to 2 classes of neurons: excitatory broad-spiking putative pyramidal cells with spike durations of >200 μs and narrow-spiking putative inhibitory interneurons with spike durations of <200 μs, respectively [reprinted with permission from Hussar and Pasternak (2010)].

Recent studies from our laboratory have begun to highlight the utility of FF analysis as a tool for elucidating information content in multisensory systems. In recordings from the SC of both awake and anesthetized cats, different modes of multisensory interactions (i.e., enhancement vs. depression) were discovered to be associated with distinctly different changes in FF. Thus, whereas response enhancements are accompanied by an increase in response variability under multisensory conditions, response depressions are characterized by decreased variability (Figure 2) (Sarko et al., 2012). We assess this by calculating the change in FF-values (ΔFF) between the maximum unisensory response (U_ff, unisensory FF-value) and the multisensory response (M_ff, multisensory FF-value):

$\Delta FF=U_{ff}-M_{ff}$

whereas a positive ΔFF-value indicates unisensory responses that are more variable (less reliable) than multisensory responses, a negative ΔFF-value indicates the reverse. On average, neuronal responses that demonstrated response enhancements under multisensory conditions were less reliable (with higher FF-values) compared to unisensory responses, resulting in negative ΔFF-values (Figures 2A,C). In contrast, response depressions were associated with positive ΔFF-values indicative of greater response reliability and lower FF-values under multisensory conditions (Figures 2B,C). This suggests that response variability offers an additional neural encoding mechanism beyond firing rate alone, and runs somewhat counter to what would be predicted based on changes in firing rate alone—that responses should be more reliable under multisensory enhancement conditions. It may be that high firing rates are sufficient to bind multisensory stimuli under conditions of response enhancement, whereas more reliably encoded multisensory stimuli are necessary in conditions that produce weak multisensory responses. It has further been shown that encoding variability may differ according to behavioral states of discrimination vs. detection, with detection possible despite high response variability and discrimination more reliant on spike timing precision (Reich et al., 1997). Thus, by extension, because multisensory response enhancements are associated with greater neuronal response variability, they may be more behaviorally relevant for accurate detection of multisensory stimuli. Furthermore, response depressions (associated with decreased variability, or greater reliability) may be more relevant in discrimination of multisensory stimuli (e.g., discriminating the spatially disparate stimuli that are known to elicit response depressions). This could be behaviorally tested by combining neural recordings in the AES with a saccade task in which the animal is presented with audiovisual stimuli that are moving either congruently (in the same direction) or incongruently under varying degrees of motion coherence, thereby titrating cue reliability. One prediction from such experiments would be that the activity of multisensory neurons would reflect behavioral outcomes (faster reaction times and enhanced accuracy of saccade direction matched to the most reliable cue condition). Neurons tuned to a particular direction—for instance, left—would exhibit greater firing rates when the more reliable cue was moving left, and greater multisensory enhancement when both cues were moving congruently. In contrast the introduction of less motion coherence would likely result in greater neuronal response variability, slower reaction times, and impaired accuracy in saccade direction.

FIGURE 2

Figure 2. Response variability in representative multisensory neurons from electrophysiological recordings in the superior colliculus of an awake cat. Note the significant enhancement of response under multisensory conditions (visual-auditory) compared to the best unisensory condition (92% enhancement of firing rate denoted by the Interactive Index, II; (A) Preliminary evidence from our laboratory demonstrates an increase in response variability (Fano factor, or FF) in multisensory conditions exhibiting response enhancements, whereas a decrease in variability (increased reliability) is observed for response depressions (B,C). A, auditory, V, visual, and V0A50 denotes audiovisual stimuli presented with a temporal offset of 50 ms.

Recent studies by Kayser et al. have further illustrated the importance of response variability as a possible information source under multisensory circumstances. In recordings from primate auditory cortex, they showed that naturalistic audiovisual stimuli and their degree of congruence play an important role in response variability and information gain (Kayser et al., 2010). Epochs of weak auditory response became more variable with the addition of visual input, whereas epochs of strong auditory response became less variable and more reliable under multisensory conditions. This in turn had a direct effect on the information encoded, with information gain directly related to increased response reliability. This information gain decreased when auditory and visual cues were mismatched, reflecting the dependence on feature matching between modalities rather than the addition of a visual stimulus alone, and linking analyses of response variability and information gain to perceptual meaning.

Changes in Synaptic Processes: Local Field Potentials

Although the emphasis of multisensory work has been on neurons that are overtly responsive to stimuli from two or more sensory modalities, recent studies have highlighted that multisensory interactions can manifest in neurons that are only overtly responsive to one sensory modality, with the second modulating responses of the driving modality (Schroeder et al., 2001, 2003; Schroeder and Foxe, 2002; Foxe and Schroeder, 2005; Ghazanfar et al., 2005; Carriere et al., 2007, 2008; Allman et al., 2008; Krueger et al., 2009; Meredith and Allman, 2009). Indeed, a recent paradigm shift in the multisensory field has stemmed from the suggestion that such modulatory influences can impact sensory processes even in very early sensory cortical domains typically characterized as unisensory (Morrell, 1972; Giard and Peronnet, 1999; Foxe et al., 2000; Schroeder et al., 2001; Falchier et al., 2002; Molholm et al., 2002; Schroeder and Foxe, 2002; Fu et al., 2003; Rockland and Ojima, 2003; Besle et al., 2004; Brosch et al., 2005; Ghazanfar et al., 2005; Ghazanfar and Schroeder, 2006; Kayser and Logothetis, 2007).

Analysis of local field potentials (LFPs) elucidates subthreshold influences through sampling pooled voltage changes (Berens et al., 2008a,b, 2010). The low frequency component of the LFP signal (<200 Hz) has been the focus of the majority of LFP experiments and is believed to reflect excitatory and inhibitory postsynaptic potential changes, in addition to subthreshold membrane oscillations and after-potentials of somatodendritic action potentials, in the vicinity of the electrode tip (Mitzdorf, 1985, 1987; Kamondi et al., 1998; Buzsaki et al., 2002; Logothetis, 2003, 2008; Hasenstaub et al., 2005; Berens et al., 2008b; Trevelyan, 2009) (but see also Kajikawa and Schroeder, 2011, regarding the sampling area of LFPs). Standard extracellular recording methods can be used to detect both spiking activity and the LFP—depending on the filtering parameters applied to the signal—with each carrying distinct functional implications (Figure 3). In addition to its amplitude and latency, the raw LFP signal can be decomposed by Fourier analysis into its component frequency bands (delta, 1–4 Hz; theta, 4–8 Hz; alpha, 8–12 Hz; beta, 12–30 Hz; and gamma, >30 Hz) similar to those characteristic of electroencephalogram (EEG) studies. Spectral analysis of the LFP signal further assesses changes in the power spectrum of particular frequency bands that coincide with distinct stimulus conditions (Henrie and Shapley, 2005).

FIGURE 3

Figure 3. (A) Placing an extracellular electrode in the brain measures the mean extracellular field potential originating from the activity of a population of neurons surrounding the electrode tip. The signal is high-pass filtered and the resulting multiunit action potentials are detected. The local field potential (LFP) is the low frequency component (up to 200 Hz) of the signal with a frequency composition that varies over time. Here, prominent gamma band oscillations (between 30 and 90 Hz) are visible in the later part of the LFP trace. (B) In this example from the primary visual cortex of an awake monkey, the raw trace (black) has been filtered to isolate the gamma LFP (gray) prominent during visual stimulation. (C) Depiction of a pyramidal cell showing the current sink at the dendritic tree and the current source at the soma. (D) Alignment of pyramidal cells creating an open field arrangement such that synchronized synaptic input can produce strong dipoles without current flowing from individual cells canceling each other [figure reprinted with permission from Berens et al. (2010)].

Local Field Potentials and Current Source Density Examination of Unisensory Processing

Much that is known about the role of LFP modulations in sensory processing comes from work restricted to the individual sensory modalities. The onset of visual stimulation causes a shift from low-frequency to fast gamma LFP oscillations in the primary visual cortex (V1) of awake macaques (Berens et al., 2008a,b). In primate V1, gamma band activity shows the highest stimulus selectivity associated with coding stimulus features such as orientation preference and ocular dominance (Berens et al., 2008a,b). Moreover, gamma power has been reported to increase in different visual areas during perceptual (Gail et al., 2004; Wilke et al., 2006), memory-associated (Pesaran et al., 2002), and attentional (Fries et al., 2001; Taylor et al., 2005; Fries et al., 2008) processes. Feature selectivity (for both stimulus direction and speed) (Liu and Newsome, 2006), attentional allocation (Fries et al., 2001; Taylor et al., 2005), and object category selectivity (Kreiman et al., 2006) have also been related to changes in LFPs in visual cortex. Furthermore, attention to a visual stimulus results in enhanced oscillatory coupling at gamma frequencies in the frontal eye field (FEF) and area V4, which in turn is thought to optimize the postsynaptic impact of spikes from one area to the other and improve communication between the areas during attention (Gregoriou et al., 2009). In the auditory system, primary auditory cortex (A1) of awake rhesus monkeys demonstrates frequency tuning profiles that are matched across high frequency LFP and single or multi-unit activity domains (Kayser et al., 2007). In the somatosensory system, high gamma LFP oscillations are closely synchronized with the occurrence of action potentials in SII of awake monkeys, suggesting that high gamma power in LFPs may be an index of population firing rate (Ray et al., 2008a,b). Because LFPs are essentially an index of local synaptic processing, they provide information about local inputs to a given brain area (Pesaran, 2009), thereby creating an essential bridge between analyzing inputs to and outputs from a region of interest by linking LFP and spiking activity. Such studies conducted in the principal sensory modalities have established important relationships between firing rate and LFP encoding of stimulus properties, as well as perceptual and attentional correlates of LFP activity that can be extended to multisensory applications.

Although LFP fluctuations provide an important window into synaptic function that complements spiking information, there are conceptual caveats that must be considered when interpreting LFP signals. One of the most important of these is the high degree of lateral (~200–400 μm) and vertical (several mm) spread of the LFP (Kajikawa and Schroeder, 2011), which is substantially greater than had been originally estimated (Katzner et al., 2009; Xing et al., 2009), and which is undoubtedly due to volume conduction (Mitzdorf, 1985; Nunez et al., 1991; Schroeder et al., 1995; Kocsis et al., 1999). This complicates the spatial interpretation of LFP recordings in attempting to localize activity to specific regional confines as an index of underlying synaptic processes, since the observed LFP activity reflects a mixture of both local and relatively distant electrophysiological events. A recent study utilizing a detailed biophysical modeling approach has investigated the spatial extent of LFP signal spread and suggests that it depends on a variety of factors including neuronal morphology, synapse distribution, and synaptic activity correlation (e.g., uncorrelated synaptic activity produces less spatial spread than correlated activity) (Linden et al., 2011). Multicontact electrodes can be used to circumvent this issue by measuring LFPs at a variety of depths for a single penetration, spanning the cortical thickness of a given region and allowing a laminar analysis of the LFP. In particular, the spatial derivative of these LFP signals can be used to create a current source density (CSD) profile, revealing current “sources” and “sinks” (for calculations, see Nicholson and Freeman, 1975; Tenke et al., 1993). The CSD also reflects subthreshold synaptic currents but avoids the spatial confounds associated with the LFP, and also has been shown to have stimulus selectivity comparable to that of multiunits [e.g., narrow response tuning to best frequency—comparable to that of multiunit activity—during recordings from A1 of macaques, in contrast to LFP signals that demonstrated wider tuning (Kajikawa and Schroeder, 2011)]. Perhaps most importantly, CSD analysis allows a view the laminar flow of information within a specific cortical circuit by attenuating far-field contamination (i.e., volume conduction confounds).

Local Field Potentials: Implications for Multisensory Processing

Though LFP analysis has been widely used within individual sensory systems, its application to the multisensory realm remains limited. Recent studies have begun to examine changes in the amplitude and frequency of LFP oscillations in different brain areas in response to multisensory stimuli. For instance, Kayser et al. (2008) reported visual modulation of activity in the auditory cortex of rhesus monkeys, demonstrated by changes in the amplitude of the LFP signal under cross-modal conditions. The authors demonstrate that visual stimuli modulate auditory processing in both primary and secondary auditory fields as reflected by amplitude modulations in LFP recordings and changes in firing rate (Figure 4). Audiovisual interactions were detectable in both LFPs and spiking activity, and depended on stimulus efficacy as well as relative timing (Kayser et al., 2008). The addition of visual stimuli resulted in response modulations that ranged from enhancement (Figure 4, top LFP panel), to depression (Figure 4, LFP middle panel, all spiking examples), or no change (Figure 4, lower LFP panel). In a similar fashion, Ghazanfar and colleagues demonstrated multisensory integration of faces and voices using LFP and SUA analyses in the auditory cortex of rhesus monkeys (Figure 5) (Ghazanfar et al., 2005). Recent studies have begun to demonstrate that such subthreshold multisensory influences are more ubiquitous than previously realized and not restricted to cortical areas alone. For instance, traditionally the superficial layers of the SC were thought to be purely visual in nature and to play a role in visual form discrimination (Casagrande et al., 1972) but recent findings through LFP and multiunit activity (MUA) recordings demonstrate that visual activity is modulated by simultaneous presentation of an auditory stimulus (Ghose et al., 2012), which may in turn facilitate visual form discrimination under multisensory conditions. These studies illustrate the utility of LFP signal analysis in elucidating how modulatory influences from a second modality cue contribute to multisensory processing and how such modulatory influences might ultimately shape behaviors that rely on multisensory integration, such as form perception, vocalization, and communication.

FIGURE 4

Figure 4. LFPs (left column) and spiking activity (right column) recorded from auditory cortex in rhesus monkeys presented with visual (blue), auditory (orange), and audiovisual (black) stimuli. LFP curves demonstrate the mean values for each of the 3 stimulus conditions, with the horizontal gray line along the x axis representing the stimulus interval. LFP values are shown in units of standard deviation (SD) from baseline (z-score). MUA, multi-unit activity; Spk, spikes; SUA, single-unit activity [reprinted with permission from Kayser et al. (2008)].

FIGURE 5

Figure 5. Responses of auditory cortex (core region) to multimodal vocalizations. Rectified local field potentials compare responses to face + voice (F + V), voice alone (V), and face alone (F) components of coos and grunts (non-human primate vocalizations commonly emitted during affiliative social interactions) in order to examine multisensory integration of social signals through communication. LFPs were recorded in auditory cortex of awake behaving rhesus monkeys while they viewed vocalizing conspecifics, and integration of faces and voices was observed through changes in LFP activity. (A) The solid vertical line indicates the onset of the face stimulus whereas the dotted vertical lines indicate the onset and offset of the voice stimulus (mean across 10 trials with baseline activity subtracted). Bar graphs show mean and SEM of the maximum response (F + V or V alone using a 20 ms window) between voice onset and offset compared to other stimulus conditions. (B) Multisensory integration index (MSI) was computed for each example and demonstrates a response enhancement (top) vs. suppression (bottom) [reprinted with permission from Ghazanfar et al. (2005), Figure 2, p 5007].

Beyond simple changes in LFP amplitude induced under multisensory circumstances, recent work has highlighted the utility of LFP and CSD analysis to yield insight into mechanistic questions within multisensory systems. For instance, Lakatos and colleagues used laminar CSD and multiunit activity analyses to demonstrate that in A1 of macaque monkeys the phase of ongoing oscillations is reset by somatosensory inputs (Lakatos et al., 2007). Such phase resetting of subsequent auditory inputs can be either enhanced or suppressed depending on the timing of the auditory and somatosensory stimuli relative to the oscillatory cycle. Each oscillation cycle of field potentials has periods of high and low excitability (Figures 6A,B, red and blue shaded areas, respectively) driving neurons toward or away from their firing threshold. During transient windows of opportunity, the phase of oscillation in the local neuronal ensemble can lock to relevant stimulus inputs (Figure 6C). Such phase-locking (i.e., synchronization) can serve to amplify neuronal representations, facilitate sensory discrimination, and increase response speed and accuracy (Lakatos et al., 2008; Schroeder and Lakatos, 2009b). Phase resetting to a high excitability state produces facilitation of responses to coincident sensory input, whereas phase resetting to a low excitability state yields suppression (Lakatos et al., 2007), allowing sharpened tuning of neuronal responses (O'Connell et al., 2011). Synchronization of cross-modal inputs likely underlie the enhanced discrimination, detection, and orientation behaviors observed behaviorally when multimodal stimuli are paired. This mechanism of crosstalk between sensory inputs is now providing an essential causal link between neuronal networks activity and behavioral gains. For instance, behaviorally, in the classic flash-beep test in which a visual (flash) and auditory (beep) are separated by an increasing degree of temporal offset (the stimulus onset asynchrony, or SOA), beyond a certain window of this offset (the “temporal binding window,” ~300 ms in normal human subjects), the stimuli are perceived separate rather than synchronous events (Shams et al., 2000; Powers et al., 2009; Foss-Feig et al., 2010). It seems likely that LFP recordings in the posterior superior temporal sulcus (pSTS) of non-human primates would reflect decreased phase coupling related to temporally offset audiovisual inputs (>300 ms), which would further manifest behaviorally as slower reaction times. However, synchronous flash-beeps would likely produce increased phase coupling, facilitation of responses to the coincident sensory inputs, and speeded reaction times demonstrating multisensory gains.

FIGURE 6

Figure 6. Cross-frequency coupling between delta, theta, and gamma bands. (A) Recorded oscillations (top, green) reflect complex (Comp.) combinations of components at different frequencies. Blue traces illustrate component delta (1.5 Hz), theta (7 Hz), and gamma (40 Hz) band oscillations that together make up the complex, combined signal (green illustration) recorded prior to filtering the signal into component frequency bands. Coupling between frequencies is hierarchical in organization such that gamma oscillatory amplitude varies with the phase of underlying theta oscillation, and theta in turn varies with the phase of underlying delta oscillation. (B) Action potential firing rate and local field potentials illustrate high and low excitability phases of ongoing neuronal oscillations. (C) Simulated single-trial responses demonstrate the effects of visual inputs on A1 activity. Oscillations within a given frequency are highly phase-variable across trials (black drop line on left) until a modulatory event (i.e., one that alters activity to a sub-threshold degree but does not reach significant enhancement or depression of the response; arrow) occurs. This event can cause a phase-reset of ongoing oscillations and can produce strong phase coherence. During such coherence, peaks, and troughs of ongoing oscillations align across trials (red solid and dotted lines, respectively) with both optimal and non-optimal phases that occur in temporally predictable patterns (red and blue lines illustrating low vs. high phase variability across trials, respectively). Sensory inputs (in this case, visual inputs to A1) can be timed such that they arrive either: (1) in random phase with the ongoing oscillation (black drop line in C) and generate highly variable response amplitudes (bottom; observed as random phase effects); (2) during the optimal phase (red drop line in C), amplifying the resulting signal (bottom, observed as optimal phase effects); or (3) during the non-optimal phase (blue line in C), suppressing the resulting signal (observed as non-optimal phase effects of the visual input across trials). Over time, the coherence observed across trials dissipates, and the system returns to resting (random phase) state [reprinted with permission from Schroeder and Lakatos (2009a), Figure 1. With kind permission from Springer Science + Business Media B.V.].

Oscillations Recorded Through Electroencephalography (EEG): Population Measures of Fluctuations in Neural Activity

Beyond field potentials generated by local neuronal ensembles, larger neuronal populations demonstrate rhythmic shifts (oscillations) between states of high and low excitability on a more global scale (Buzsaki and Draguhn, 2004; Fries, 2005; Lakatos et al., 2005, 2008). These can be detected through EEG signals recorded on the scalp surface, or through the closely related blood oxygen level-dependent (BOLD) signal that forms the basis for functional magnetic resonance imaging (fMRI) (Logothetis et al., 2001; Viswanathan and Freeman, 2007; Rauch et al., 2008; Magri et al., 2012). Event-related potentials (ERPs) can be used to analyze EEG activity by averaging responses that are time-locked to stimulus presentation. In EEG recordings, higher frequency oscillations (i.e., gamma) are believed to be derived from coordinated activity in local neural assemblies whereas larger-scale networks are believed to be indexed through lower frequency oscillations (Von Stein and Sarnthein, 2000; Steriade, 2001; Csicsvari et al., 2003). These patterns create a dynamic hierarchy of neuronal oscillations with the flexibility to modulate both local and distributed network activity. A number of studies have associated the different frequency bands with distinct functional roles. Although by no means exhaustive, this includes linking delta (1–4 Hz) to motivational processes, reward, and deep sleep (Basar et al., 2000; Knyazev, 2007), theta (4–8 Hz) to working memory, emotional arousal, and fear conditioning (Jensen and Lisman, 2005; Knyazev, 2007), alpha (8–12 Hz) to working memory and awake resting state in the absence of sensory inputs (Palva and Palva, 2007), beta (12–30 Hz) to sensorimotor processing (Brovelli et al., 2004), and gamma (>30 Hz) to a variety of cortical functions including visual feature integration (Gray et al., 1989; Engel et al., 1991; Tallon-Baudry et al., 1996), attention (Muller et al., 2000; Fries et al., 2001), memory formation (Tallon-Baudry et al., 1998; Osipova et al., 2006), auditory processing (Kaiser et al., 2002; Debener et al., 2003), somatosensory processing (Bauer et al., 2006), olfactory processing (Wehr and Laurent, 1996), sensorimotor integration (Roelfsema et al., 1997), and movement preparation (Sanes and Donoghue, 1993). More recently, beyond simply looking at functional significance limited to certain oscillatory bands, studies have shown that phase synchrony and modulation across different frequency bands may be involved in complex tasks such as speech processing and memory encoding (Jensen and Lisman, 2005; Palva and Palva, 2007; Schroeder et al., 2008). Oscillatory activity also appears to be organized hierarchically, allowing sensory cortex to optimize the temporal structure of its activity pattern in order to best drive baseline excitability, and ultimately stimulus-driven responses (Schroeder et al., 1998; Lakatos et al., 2005; Schroeder and Lakatos, 2009a,b). In this way, fluctuations in the power and phase of oscillatory activity optimize the processing of rhythmic inputs and, through selective enhancement of neuronal response when sensory inputs arrive at an optimal phase of excitability for the neuronal ensemble, drive enhancement of perceptual and behavioral outcomes (Schroeder and Lakatos, 2009b).

Oscillatory Coherence as a Tool in Characterizing Multisensory Interactions

Although relatively understudied to date, oscillations have significant implications for multisensory processing, perception, and behavior. For instance, strength of synchronization was found to predict perception of ambiguous audiovisual stimuli as well as the integration of audiovisual information, particularly with respect to beta and gamma oscillations (Hipp et al., 2011). This finding implicates frequency-specific synchronization in widely distributed cortical networks as driving the formation of cross-modal associations. Other studies by Romei et al. demonstrated cross-modal phase locking of visual cortex activity (alpha oscillations) to the introduction of a sound. Phase-locking of the cross-modal stimuli introduced a periodicity that affected the pattern of phosphene perception, thus directly linking oscillatory phase-locking to behavioral outcomes (Romei et al., 2012). Thus, coherence of phase coupling between distinct brain areas may serve as a neural substrate influencing single-cell firing properties that ultimately bind anatomically segregated functional networks (Fries, 2005; Canolty et al., 2010). Specifically, phase coupling might subserve “integration through coherence,” thus bridging across different sensory modalities and enabling flexible, context-dependent binding that selectively strengthens those connections that are optimally adaptive for behavior (Engel et al., 1992, 2001; Singer, 1993; Gray, 1994; Singer and Gray, 1995; Salinas and Sejnowski, 2001; Fries, 2005; Womelsdorf et al., 2007; Senkowski et al., 2008; Benchenane et al., 2010). Given that phase-locked discharges of distributed neuronal assemblies are thought to be involved in binding stimulus features into a coherent percept (Gray et al., 1989; Engel et al., 2001), an essential aspect of multisensory processing, future experiments should target the effects of multisensory stimuli on phase coupling and coherence, particularly of beta and gamma frequency oscillations. For instance, targeting these frequency bands, changes in synchrony across subdivisions of the AES (the auditory subdivision, FAES; visual subdivision, AEV; and somatosensory subdivision, SIV) could be evaluated according to the degree of oscillatory congruence following presentation of auditory and visual motion cues. Using a saccade task to investigate accurate perception of motion direction and speed of reaction time, the prediction would be that synchrony would increase under audiovisual conditions in which auditory and visual motion occurred in the same direction. Increased synchrony would serve as a neural substrate for behavioral gains in directional assessment of congruent multisensory stimuli and would result in speeded, more accurate responses assessing the direction of stimulus movement. Furthermore, since the behavioral benefits of multisensory integration include enhanced detection and discrimination as well as speeded reaction times, it is noteworthy that stimulus discriminability is affected by whether task-relevant stimuli match an anticipated low-frequency rhythm of oscillatory activity (Jones et al., 2002; Praamstra et al., 2006; Lakatos et al., 2007, 2008; Schroeder et al., 2008; Schroeder and Lakatos, 2009a). Thus, the timing of sensory inputs from one modality relative to the phase of ongoing oscillations related to a second sensory modality is likely to be a key element in the multisensory enhancement or suppression of a response. Ultimately this would operate as an instrument of sensory selection in determining whether cross-modal stimuli are bound as a unified percept.

Spike Synchrony

It has been suggested that the activity of a group of neurons producing coincident spiking patterns forms an integral part of the neural code guiding behavior and perception (Shadlen and Newsome, 1994; Eggermont, 2001, 2006; Casagrande et al., 2002). These population-based approaches toward questions of neural encoding are becoming more commonly employed at all levels of the nervous system through analysis of correlated firing patterns, but their application toward understanding multisensory systems continues to lag behind studies within single sensory modalities.

Population encoding can be assessed in numerous ways. As highlighted in the earlier sections of this review, some of these indices include measures that are based largely on synaptic function, such as LFPs and oscillations. Others rely on spiking activity and include analyses of spike synchrony, the temporal correlation of spikes belonging to a group of neurons that are simultaneously recorded from either a local circuit or from distant areas (Singer, 1993; Singer and Gray, 1995; Usrey and Reid, 1999; Engel and Singer, 2001; Engel et al., 2001; Jermakowicz and Casagrande, 2007). Correlation of neural activity can refer to detection of temporal coincidences in the firing of two neighboring neurons, detection of co-variation in the firing rates of those neurons, or even co-variation in the postsynaptic activity generated by a cell's many inputs (Eggermont, 2007). This correlated neural activity can be measured using cross-correlograms (CCGs), an analysis tool that correlates activity between a pair of neurons and depict changes in the probability of a target neuron discharge relative to the discharge timing of a reference neuron. Time-locked discharges of a pair of neurons, known as coincident events, appear as peaks or valleys in the CCG and indicate excitatory or inhibitory interactions, respectively (Perkel et al., 1967; Gochin et al., 1989). A preferred method for such correlational analyses uses Joint Post-Stimulus Time Histograms (JPSTHs) due to their enhanced temporal resolution and facilitated observation of spike coincidence over time after a stimulus or behavioral event (Gerstein et al., 1989).

Functional Relevance of Spike Synchrony in Sensory Processing

Synchrony between single-unit pairs has been widely studied in visual cortical areas, and has revealed important features of correlated activity (for reviews see Singer, 1993; Engel et al., 1999; Usrey and Reid, 1999; Jermakowicz and Casagrande, 2007). Neurons with similar receptive field properties have synchronous discharges in the striate cortex of squirrel monkeys (Livingstone, 1996). Similarly, in extrastriate cortical areas such as the caudal STS of macaques, neurons exhibited synchronized activity that was dependent on specific stimulus properties (Kreiter and Singer, 1992; Gray and Viana Di Prisco, 1997). Spike synchrony in auditory cortex has been only minimally studied to date, though synchrony between auditory cortical neurons has been implicated in processing sound movement and localization (Ahissar et al., 1992). As described in visual areas, the correlation strength of auditory cortical neurons appears to depend on the receptive field properties of paired neurons and has been specifically associated with spectro-temporal receptive fields, binaural interactions, and temporal response properties (e.g., response onset/offset as well as the temporal pattern of discharge) (Brosch and Schreiner, 1999; Eggermont, 2006).

Neural synchrony has also been demonstrated in somatosensory processing such that the degree of correlated activity parallels the extent of receptive field overlap for neurons in primary and secondary somatosensory areas (Dinse et al., 1993; Roy and Alloway, 1999; Alloway et al., 2002). Neural synchronization may also encode more complex stimulus features such as movement (Roy and Alloway, 1999), discrimination, and localization (Figure 7) (Reed et al., 2008) as well as surface texture (Wolfe et al., 2008; Jadhav et al., 2009). Spike synchrony may also be attentionally modulated (Roy et al., 2000; Steinmetz et al., 2000), serving to increase the signal-to-noise ratio for the stimulus-driven response (Crick and Koch, 1990; De Oliveira et al., 1997; Salinas and Romo, 2000; Fries et al., 2001) with behavioral outcomes such as improving performance during a vibrotactile discrimination task (Romo et al., 2003). However, it is also important to examine the noise correlation of simultaneously recorded neuronal pairs, which can affect the information in population coding of responses and ultimately influence computational strategies in neuronal networks (Averbeck et al., 2006). Through such noise correlations, small changes in individual neuron activity can have a large impact at the population scale, altering both the encoding and decoding of a signal. Ultimately, analyses of spike synchrony may provide insights into not only the temporal aspects of neural coding but also their coordinating role in sensory perception, potentiating the activity of targeted neural networks and bridging between sensory modalities to generate a unified percept.

FIGURE 7

Figure 7. Correlated spike activity recorded from adjacent electrodes in a 10 × 10 multielectrode array recording from the thenar pad representation of area 3b in an owl monkey. Spike synchrony between two neurons is shown in a JPSTH matrix, with a psth for each neuron shown to the left and below to illustrate responses to skin indentations of the thenar pad and digital (P1) pad for 100 trials. The colored pixels in the JPSTH matrix represent the magnitude of the normalized correlation at different lag times over a poststimulus time of 700 ms. The magnitude of the normalized correlation shown in the JPSTH and in the cross-correlation histogram reveal a strong spike synchrony that occurred at 0 ms lag time. The cross-correlation histogram (black) revealed a peak correlation of 0.16 that exceeded the mean correlation from the shuffled trials (red). The time-averaged cross-correlogram was computed by summing the JPSTH bins parallel to the main diagonal, measuring the average positive or negative correlation across the entire interval of analysis in 1 ms bins [reprinted with permission from Reed et al. (2008)].

Spike Synchrony: Applications for Understanding Multisensory Interactions

Although spike synchrony has been widely studied within individual sensory systems, surprisingly few studies have examined coincident firing patterns and their role in multisensory encoding (e.g., Ghoshal et al., 2011). Future studies of particular interest would utilize awake animal recordings to target multisensory neurons in cortical areas such as the AES, the ventral intraparietal area (VIP), and the STS in order to examine how spike synchrony contributes to behavioral gains and perceptual benefits conferred by cross-modal stimuli. It can be postulated that spike synchrony might be linked to integrative processing and its behavioral manifestations through stimulus feature encoding, attentional modulation, and increasing the signal-to-noise ratio of correlated activity in multisensory neurons. This could be examined through an experiment in which animals are trained to perform a visual detection task while a multielectrode array records neuronal activity spanning distinct regions of visual cortex. Behavioral performance and neuronal activity would then be compared under visual vs. audiovisual conditions. One underlying hypothesis of these experiments would be that multisensory integration might not manifest itself through changes in firing rate, but rather through increased spike synchrony between neuronal pairs (either within the same visual area, between visual areas, or a combination of both). Furthermore, increased spike synchrony might be correlated with behavioral manifestations such as enhanced stimulus detection (e.g., greater response accuracy, speeded reaction time, enhanced detection or discrimination) under multisensory conditions.

Signal Detection Theory: Receiver Operating Characteristic (ROC) Analysis and Implications for Multisensory Processing

The nervous system is faced with the incredible challenge of successfully extracting valuable information from a highly noisy environment. The sources of noise can be manifold and broadly belong to two categories: (1) external noise, such as that generated within the environment, and (2) internal noise, such as that generated within the nervous system. Signal detection theory (see Green and Swets, 1966; Heeger, 1997, 2003; Macmillan and Creelman, 2004, for review) is a conceptual framework that has great utility in extracting meaningful signals in the presence of noise, and in relating neural activity to behavioral outcomes. Responses in a signal detection framework are generally divided into a 2 × 2 array, comprised of: (1) a hit (successful detection of signal), (2) a false alarm (detection of a signal when there is none), (3) a miss (failure to detect a signal when it is present), or (4) a correct rejection (detection of no signal when there is none). When quantifying responses, the criterion threshold level (also called response bias, or decision bias, and expressed as β) greatly affects the outcome of a signal detection task.

Discrimination of a signal from noise depends primarily on the separation of the noise from the combined signal and noise distributions, as well as the spread of the two distributions or the amount of overlap. Higher noise levels involve more overlap and hence greater spread whereas lower noise levels involving reduced overlap and spread, enabling easier signal detection. This can be expressed by the discriminability—or sensitivity—index (d′):

${\text{d}}^{\prime }=\text{separation}/\text{spread}$

where separation is the difference between the means of the two distributions (noise and signal), spread is the standard deviation of the distributions, and d' is a true measure of the internal response free from subjective bias. All possible outcomes of a signal detection task can be expressed in a single curve—the Receiver Operating Characteristic (ROC) curve—that is dependent on the criterion chosen. ROC curves are generally expressed with false alarm rate on the x-axis and hit rate on the y-axis. When a signal is detected reliably, the area under the ROC curve is higher, but higher false alarm rates (unreliably detected signals) result in decreased area under the curve.

Numerous studies performed in the visual system have used analyses derived from signal detection theory to test the ability of neuronal responses to predict stimulus characteristics and/or behavior (Tolhurst et al., 1983; Bradley et al., 1987; Britten et al., 1992; Guido et al., 1995; Thompson et al., 1996). However, because single neuron responses in isolation provide little information about stimulus characteristics, the responses of multiple neurons tuned to different but overlapping ranges of stimulus dimensions must be considered when predicting psychophysical discrimination thresholds. Otherwise, it is possible for some neuronal responses to reflect smaller differences in stimulus features (such as the orientation of a bar of light) than those observed through behavioral measures (e.g., discrimination thresholds). Using the slope of ROC curves, Bradley et al. showed that single neurons in the visual cortex can reliably signal stimulus orientation and spatial frequency differences that were considerably smaller than their tuning widths (Bradley et al., 1987). Similarly, studies recording from the FEF have demonstrated that at the single neuron level there exists an explicit dissociation between perceptual processing and response generation (Thompson et al., 1996). By plotting the area under the ROC curves as a function of time for a simple popout visual search discrimination task, it was found that the activity of FEF neurons could reliably discriminate targets from distractors after 130 ms of search stimulus array presentation. In addition, a separate study showed that presaccadic movement neurons in FEF are activated ~100 ms before saccades and that only when their activity reaches a certain threshold does a saccade occur (Hanes et al., 1995; Hanes and Schall, 1996). These studies illustrate the utility of ROC analyses in differentiating between two stages of processing in the visual system—target discrimination and response generation. Furthermore, studies in the LGN have elucidated how distinct neuronal response modes (burst vs. tonic) relate to signal detection ability in visual processing (with higher signal detection characterizing the burst mode; Figure 8) (Guido et al., 1995). Thus, ROC analyses can be used to reveal the dynamic relationship between various aspects of stimulus processing, neuronal activity, and ultimately behavior.

FIGURE 8

Figure 8. ROC curves for tonic (A) and burst (B) mode of response for a neuron recorded from the lateral geniculate nucleus of a cat. The ROC curves plot the probability of correctly detecting visually-driven activity from spontaneous activity. The probability of detecting the signal, P(hit), is plotted on the y-axis against the probability of mistaking spontaneous activity for visually driven activity, P(false alarm), at all possible criterion levels. The area under each curve is shaded, and the dashed line divides the total area of the curve in half. A value of 1 indicates perfect detection of signal from noise whereas a value of 0.5 indicates that signal and noise are indistinguishable. Here, the area under the curve is higher for burst mode compared to tonic mode, indicating enhanced signal detection conferred by the burst mode of response [reprinted with the permission of Cambridge University Press and adapted from Guido et al. (1995)].

Neuronal recordings in awake animals performing a multisensory task, in conjunction with ROC analyses, would provide the necessary link between neuronal responses and behavioral outcomes that has been critically absent in the multisensory field. The behavioral benefits revealed by psychometric functions (such as greater response accuracy and speeded reaction times) seen for cross-modal trials would be predicted to coincide with increased ROC areas computed from neuronal recordings as the animal performed a task (e.g., a saccade in the direction of perceived audiovisual motion). Additionally, the time course of the neuronal response plotted as a function of ROC area would help to distinguish the contribution of neuronal activity to different stages of the task being performed, thereby directly relating multisensory neural activity to behaviorally and perceptually relevant outcomes. Recent studies have begun to study such neurometric/psychometric relationships using ROC analysis in a multisensory context. For instance, simultaneous recording of neuronal activity in the dorsal medial superior temporal (MSTd) area of monkeys was performed during a multisensory discrimination task using visual and vestibular cues to discriminate heading (Gu et al., 2008). Results of this study revealed that MSTd neuronal activity closely paralleled behavioral choice with respect to improvements in both perceptual sensitivity and cue weighting, demonstrating direct neural links to multisensory enhancement of behavior.

Summary: Convergent Approaches Toward Understanding Sensory Integration

An increasing number of studies have begun to address interactions across multiple sensory modalities, improving our understanding of the neurophysiological mechanisms that underlie behavioral and perceptual outcomes of cross-modal processing. Going forward, methodologies that have advanced our understanding of individual sensory systems in isolation can be applied toward bridging the gap of how these senses interact to form a unified percept of our surroundings. Analyses that go beyond classical firing rate measures to assess multisensory gain within the realms of response variability, LFPs, CSD, oscillatory coherence, spike synchrony, and ROC are promising tools for understanding the neural encoding of multisensory stimulus features. Analytical tools including temporal coding, response variability (FF), and measures of signal detection outcomes (ROC analyses) are integral in relating the reliability and efficacy of neural processing to behavioral gains such as improved target detection under multisensory conditions. Sensory integration, in part, also requires binding of unisensory representations through interactions between sensory cortices. This in turn requires widely distributed functional coupling, poising slow frequency bands of LFP or CSD signals as a promising experimental focus, given that these bands have been particularly implicated in long-range interactions. Future studies should address direct links between oscillatory activity, perception and behavior as it relates to cross-modal processing. Furthermore, using multisite and multiarea recordings in behaving animals to examine cross-talk between neurons of different sensory areas through increased or decreased spike synchrony, independent of changes in firing rate, might elucidate how cue weighting of significant sensory events works to perceptually bind stimuli as a unified percept or dissociate them (Fetsch et al., 2013). Of course the current review is by no means comprehensive, and other methods constitute critical tools for encapsulating multisensory processing, including mutual information theory, discriminant analysis, Bayesian modeling, and population vectors. Ultimately, mechanisms such as phase coupling and oscillatory synchronization, modulations in LFPs or CSD, and changes in response variability, signal detection, and spike synchrony are likely to work in concert to varying degrees in order to effectively integrate sensory cues across modalities, optimize behavior, and ultimately derive perceptual meaning from our sensory surroundings as a synthesized whole.

Funding

This work was supported by National Institute of Mental Health Grant MH-63861 and Vanderbilt Kennedy Center Development Funds to Mark T. Wallace.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Ahissar, M., Ahissar, E., Bergman, H., and Vaadia, E. (1992). Encoding of sound-source location and movement: activity of single neurons and interactions between adjacent neurons in the monkey auditory cortex. J. Neurophysiol. 67, 203–215.

Pubmed Abstract | Pubmed Full Text

Allman, B. L., Keniston, L. P., and Meredith, M. A. (2008). Subthreshold auditory inputs to extrastriate visual neurons are responsive to parametric changes in stimulus quality: sensory-specific versus non-specific coding. Brain Res. 1242, 95–101. doi: 10.1016/j.brainres.2008.03.086

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Alloway, K. D., Zhang, M., Dick, S. H., and Roy, S. A. (2002). Pervasive synchronization of local neural networks in the secondary somatosensory cortex of cats during focal cutaneous stimulation. Exp. Brain Res. 147, 227–242. doi: 10.1007/s00221-002-1233-3

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Amlot, R., Walker, R., Driver, J., and Spence, C. (2003). Multimodal visual-somatosensory integration in saccade generation. Neuropsychologia 41, 1–15. doi: 10.1016/S0028-3932(02)00139-2

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Averbeck, B. B., Latham, P. E., and Pouget, A. (2006). Neural correlations, population coding and computation. Nat. Rev. Neurosci. 7, 358–366. doi: 10.1038/nrn1888

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Barraclough, N. E., Xiao, D., Baker, C. I., Oram, M. W., and Perrett, D. I. (2005). Integration of visual and auditory information by superior temporal sulcus neurons responsive to the sight of actions. J. Cogn. Neurosci. 17, 377–391. doi: 10.1162/0898929053279586

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Basar, E., Basar-Eroglu, C., Karakas, S., and Schurmann, M. (2000). Brain oscillations in perception and memory. Int. J. Psychophysiol. 35, 95–124. doi: 10.1016/S0167-8760(99)00047-1

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Bauer, M., Oostenveld, R., Peeters, M., and Fries, P. (2006). Tactile spatial attention enhances gamma-band activity in somatosensory cortex and reduces low-frequency activity in parieto-occipital areas. J. Neurosci. 26, 490–501. doi: 10.1523/JNEUROSCI.5228-04.2006

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Benchenane, K., Peyrache, A., Khamassi, M., Tierney, P. L., Gioanni, Y., Battaglia, F. P., et al. (2010). Coherent theta oscillations and reorganization of spike timing in the hippocampal- prefrontal network upon learning. Neuron 66, 921–936. doi: 10.1016/j.neuron.2010.05.013

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Berens, P., Keliris, G. A., Ecker, A. S., Logothetis, N. K., and Tolias, A. S. (2008a). Comparing the feature selectivity of the gamma-band of the local field potential and the underlying spiking activity in primate visual cortex. Front. Syst. Neurosci. 2:2. doi: 10.3389/neuro.06.002.2008

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Berens, P., Keliris, G. A., Ecker, A. S., Logothetis, N. K., and Tolias, A. S. (2008b). Feature selectivity of the gamma-band of the local field potential in primate primary visual cortex. Front. Neurosci. 2, 199–207. doi: 10.3389/neuro.01.037.2008

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Berens, P., Logothetis, N. K., and Tolias, A. S. (2010). Local field potentials, BOLD and spiking activity – relationships and physiological mechanisms. Nat. Prec.

Berry, M. J. 2nd., and Meister, M. (1998). Refractoriness and neural precision. J. Neurosci. 18, 2200–2211.

Pubmed Abstract | Pubmed Full Text

Besle, J., Fort, A., Delpuech, C., and Giard, M. H. (2004). Bimodal speech: early suppressive visual effects in human auditory cortex. Eur. J. Neurosci. 20, 2225–2234. doi: 10.1111/j.1460-9568.2004.03670.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Bradley, A., Skottun, B. C., Ohzawa, I., Sclar, G., and Freeman, R. D. (1987). Visual orientation and spatial frequency discrimination: a comparison of single neurons and behavior. J. Neurophysiol. 57, 755–772.

Pubmed Abstract | Pubmed Full Text

Britten, K. H., Shadlen, M. N., Newsome, W. T., and Movshon, J. A. (1992). The analysis of visual motion: a comparison of neuronal and psychophysical performance. J. Neurosci. 12, 4745–4765.

Pubmed Abstract | Pubmed Full Text

Brosch, M., and Schreiner, C. E. (1999). Correlations between neural discharges are related to receptive field properties in cat primary auditory cortex. Eur. J. Neurosci. 11, 3517–3530. doi: 10.1046/j.1460-9568.1999.00770.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Brosch, M., Selezneva, E., and Scheich, H. (2005). Nonauditory events of a behavioral procedure activate auditory cortex of highly trained monkeys. J. Neurosci. 25, 6797–6806. doi: 10.1523/JNEUROSCI.1571-05.2005

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Brovelli, A., Ding, M., Ledberg, A., Chen, Y., Nakamura, R., and Bressler, S. L. (2004). Beta oscillations in a large-scale sensorimotor cortical network: directional influences revealed by Granger causality. Proc. Natl. Acad. Sci. U.S.A. 101, 9849–9854. doi: 10.1073/pnas.0308538101

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Burge, J., Girshick, A. R., and Banks, M. S. (2010). Visual-haptic adaptation is determined by relative reliability. J. Neurosci. 30, 7714–7721. doi: 10.1523/JNEUROSCI.6427-09.2010

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Burnett, L. R., Stein, B. E., Chaponis, D., and Wallace, M. T. (2004). Superior colliculus lesions preferentially disrupt multisensory orientation. Neuroscience 124, 535–547. doi: 10.1016/j.neuroscience.2003.12.026

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Burr, D., and Alais, D. (2006). Combining visual and auditory information. Prog. Brain Res. 155, 243–258. doi: 10.1016/S0079-6123(06)55014-9

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Buzsaki, G., Csicsvari, J., Dragoi, G., Harris, K., Henze, D., and Hirase, H. (2002). Homeostatic maintenance of neuronal excitability by burst discharges in vivo. Cereb. Cortex 12, 893–899. doi: 10.1093/cercor/12.9.893

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Buzsaki, G., and Draguhn, A. (2004). Neuronal oscillations in cortical networks. Science 304, 1926–1929. doi: 10.1126/science.1099745

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Canolty, R. T., Ganguly, K., Kennerley, S. W., Cadieu, C. F., Koepsell, K., Wallis, J. D., et al. (2010). Oscillatory phase coupling coordinates anatomically dispersed functional cell assemblies. Proc. Natl. Acad. Sci. U.S.A. 107, 17356–17361. doi: 10.1073/pnas.1008306107

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Carandini, M. (2004). Amplification of trial-to-trial response variability by neurons in visual cortex. PLoS Biol. 2:E264. doi: 10.1371/journal.pbio.0020264

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Carriere, B. N., Royal, D. W., Perrault, T. J., Morrison, S. P., Vaughan, J. W., Stein, B. E., et al. (2007). Visual deprivation alters the development of cortical multisensory integration. J. Neurophysiol. 98, 2858–2867. doi: 10.1152/jn.00587.2007

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Carriere, B. N., Royal, D. W., and Wallace, M. T. (2008). Spatial heterogeneity of cortical receptive fields and its impact on multisensory interactions. J. Neurophysiol. 99, 2357–2368. doi: 10.1152/jn.01386.2007

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Casagrande, V. A., Harting, J. K., Hall, W. C., Diamond, I. T., and Martin, G. F. (1972). Superior colliculus of the tree shrew: a structural and functional subdivision into superficial and deep layers. Science 177, 444–447. doi: 10.1126/science.177.4047.444

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Casagrande, V. A., Xu, X., and Sary, G. (2002). Static and dynamic views of visual cortical organization. Prog. Brain Res. 136, 389–408. doi: 10.1016/S0079-6123(02)36032-1

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Corneil, B. D., and Munoz, D. P. (1996). The influence of auditory and visual distractors on human orienting gaze shifts. J. Neurosci. 16, 8193–8207.

Pubmed Abstract | Pubmed Full Text

Corneil, B. D., Van Wanrooij, M., Munoz, D. P., and Van Opstal, A. J. (2002). Auditory-visual interactions subserving goal-directed saccades in a complex scene. J. Neurophysiol. 88, 438–454.

Pubmed Abstract | Pubmed Full Text

Crick, F., and Koch, C. (1990). Some reflections on visual awareness. Cold Spring Harb. Symp. Quant. Biol. 55, 953–962. doi: 10.1101/SQB.1990.055.01.089

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Csicsvari, J., Jamieson, B., Wise, K. D., and Buzsaki, G. (2003). Mechanisms of gamma oscillations in the hippocampus of the behaving rat. Neuron 37, 311–322. doi: 10.1016/S0896-6273(02)01169-8

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

De Oliveira, S. C., Thiele, A., and Hoffmann, K. P. (1997). Synchronization of neuronal activity during stimulus expectation in a direction discrimination task. J. Neurosci. 17, 9248–9260.

Pubmed Abstract | Pubmed Full Text

De Ruyter Van Steveninck, R. R., Lewen, G. D., Strong, S. P., Koberle, R., and Bialek, W. (1997). Reproducibility and variability in neural spike trains. Science 275, 1805–1808. doi: 10.1126/science.275.5307.1805

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Debener, S., Herrmann, C. S., Kranczioch, C., Gembris, D., and Engel, A. K. (2003). Top-down attentional processing enhances auditory evoked gamma band activity. Neuroreport 14, 683–686. doi: 10.1097/00001756-200304150-00005

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Diederich, A., Colonius, H., Bockhorst, D., and Tabeling, S. (2003). Visual-tactile spatial interaction in saccade generation. Exp. Brain Res. 148, 328–337. doi: 10.1007/s00221-002-1302-7

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Dinse, H. R., Recanzone, G. H., and Merzenich, M. M. (1993). Alterations in correlated activity parallel ICMS-induced representational plasticity. Neuroreport 5, 173–176. doi: 10.1097/00001756-199311180-00020

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Eden, U. T., and Kramer, M. A. (2010). Drawing inferences from Fano factor calculations. J. Neurosci. Methods 190, 149–152. doi: 10.1016/j.jneumeth.2010.04.012

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Eggermont, J. J. (2001). Between sound and perception: reviewing the search for a neural code. Hear. Res. 157, 1–42. doi: 10.1016/S0378-5955(01)00259-3

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Eggermont, J. J. (2006). Properties of correlated neural activity clusters in cat auditory cortex resemble those of neural assemblies. J. Neurophysiol. 96, 746–764. doi: 10.1152/jn.00059.2006

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Eggermont, J. J. (2007). Correlated neural activity as the driving force for functional changes in auditory cortex. Hear. Res. 229, 69–80. doi: 10.1016/j.heares.2007.01.008

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Engel, A. K., Fries, P., Konig, P., Brecht, M., and Singer, W. (1999). Temporal binding, binocular rivalry, and consciousness. Conscious. Cogn. 8, 128–151. doi: 10.1006/ccog.1999.0389

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Engel, A. K., Fries, P., and Singer, W. (2001). Dynamic predictions: oscillations and synchrony in top-down processing. Nat. Rev. Neurosci. 2, 704–716. doi: 10.1038/35094565

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Engel, A. K., Konig, P., Kreiter, A. K., Schillen, T. B., and Singer, W. (1992). Temporal coding in the visual cortex: new vistas on integration in the nervous system. Trends Neurosci. 15, 218–226. doi: 10.1016/0166-2236(92)90039-B

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Engel, A. K., Konig, P., and Singer, W. (1991). Direct physiological evidence for scene segmentation by temporal coding. Proc. Natl. Acad. Sci. U.S.A. 88, 9136–9140. doi: 10.1073/pnas.88.20.9136

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Engel, A. K., and Singer, W. (2001). Temporal binding and the neural correlates of sensory awareness. Trends Cogn. Sci. 5, 16–25. doi: 10.1016/S1364-6613(00)01568-0

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ernst, M. O., and Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 429–433. doi: 10.1038/415429a

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Falchier, A., Clavagnier, S., Barone, P., and Kennedy, H. (2002). Anatomical evidence of multimodal integration in primate striate cortex. J. Neurosci. 22, 5749–5759.

Pubmed Abstract | Pubmed Full Text

Fano, U. (1947). Ionization yield of rations. II. The fluctuations of the number of ions. Phys. Rev. 72, 26–29. doi: 10.1103/PhysRev.72.26

CrossRef Full Text

Fetsch, C. R., Deangelis, G. C., and Angelaki, D. E. (2013). Bridging the gap between theories of sensory cue integration and the physiology of multisensory neurons. Nat. Rev. Neurosci. 14, 429–442. doi: 10.1038/nrn3503

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Fetsch, C. R., Pouget, A., Deangelis, G. C., and Angelaki, D. E. (2012). Neural correlates of reliability-based cue weighting during multisensory integration. Nat. Neurosci. 15, 146–154. doi: 10.1038/nn.2983

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Forster, B., Cavina-Pratesi, C., Aglioti, S. M., and Berlucchi, G. (2002). Redundant target effect and intersensory facilitation from visual-tactile interactions in simple reaction time. Exp. Brain Res. 143, 480–487. doi: 10.1007/s00221-002-1017-9

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Foss-Feig, J. H., Kwakye, L. D., Cascio, C. J., Burnette, C. P., Kadivar, H., Stone, W. L., et al. (2010). An extended multisensory temporal binding window in autism spectrum disorders. Exp. Brain Res. 203, 381–389. doi: 10.1007/s00221-010-2240-4

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Foxe, J. J., Morocz, I. A., Murray, M. M., Higgins, B. A., Javitt, D. C., and Schroeder, C. E. (2000). Multisensory auditory-somatosensory interactions in early cortical processing revealed by high-density electrical mapping. Brain Res. Cogn. Brain Res. 10, 77–83. doi: 10.1016/S0926-6410(00)00024-0

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Foxe, J. J., and Schroeder, C. E. (2005). The case for feedforward multisensory convergence during early cortical processing. Neuroreport 16, 419–423. doi: 10.1097/00001756-200504040-00001

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Frassinetti, F., Bolognini, N., and Ladavas, E. (2002). Enhancement of visual perception by crossmodal visuo-auditory interaction. Exp. Brain Res. 147, 332–343. doi: 10.1007/s00221-002-1262-y

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Frens, M. A., Van Opstal, A. J., and Van Der Willigen, R. F. (1995). Spatial and temporal factors determine auditory-visual interactions in human saccadic eye movements. Percept. Psychophys. 57, 802–816. doi: 10.3758/BF03206796

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Fries, P. (2005). A mechanism for cognitive dynamics: neuronal communication through neuronal coherence. Trends Cogn. Sci. 9, 474–480. doi: 10.1016/j.tics.2005.08.011

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Fries, P., Reynolds, J. H., Rorie, A. E., and Desimone, R. (2001). Modulation of oscillatory neuronal synchronization by selective visual attention. Science 291, 1560–1563. doi: 10.1126/science.1055465

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Fries, P., Womelsdorf, T., Oostenveld, R., and Desimone, R. (2008). The effects of visual stimulation and selective visual attention on rhythmic neuronal synchronization in macaque area V4. J. Neurosci. 28, 4823–4835. doi: 10.1523/JNEUROSCI.4499-07.2008

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Fu, K. M., Johnston, T. A., Shah, A. S., Arnold, L., Smiley, J., Hackett, T. A., et al. (2003). Auditory cortical neurons respond to somatosensory stimulation. J. Neurosci. 23, 7510–7515.

Pubmed Abstract | Pubmed Full Text

Gail, A., Brinksmeyer, H. J., and Eckhorn, R. (2004). Perception-related modulations of local field potential power and coherence in primary visual cortex of awake monkey during binocular rivalry. Cereb. Cortex 14, 300–313. doi: 10.1093/cercor/bhg129

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gerstein, G. L., Bedenbaugh, P., and Aertsen, M. H. (1989). Neuronal assemblies. IEEE Trans. Biomed. Eng. 36, 4–14. doi: 10.1109/10.16444

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ghazanfar, A. A., Maier, J. X., Hoffman, K. L., and Logothetis, N. K. (2005). Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex. J. Neurosci. 25, 5004–5012. doi: 10.1523/JNEUROSCI.0799-05.2005

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ghazanfar, A. A., and Schroeder, C. E. (2006). Is neocortex essentially multisensory? Trends Cogn. Sci. 10, 278–285. doi: 10.1016/j.tics.2006.04.008

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ghose, D., Barnett, Z. P., and Wallace, M. T. (2012). Superficial layers of the superior colliculus: purely visual or multisensory? Soc. Neurosci. Abstr.

Ghoshal, A., Tomarken, A., and Ebner, F. (2011). Cross-sensory modulation of primary sensory cortex is developmentally regulated by early sensory experience. J. Neurosci. 31, 2526–2536. doi: 10.1523/JNEUROSCI.5547-10.2011

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Giard, M. H., and Peronnet, F. (1999). Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J. Cogn. Neurosci. 11, 473–490. doi: 10.1162/089892999563544

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gochin, P. M., Kaltenbach, J. A., and Gerstein, G. L. (1989). Coordinated activity of neuron pairs in anesthetized rat dorsal cochlear nucleus. Brain Res. 497, 1–11. doi: 10.1016/0006-8993(89)90963-3

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gray, C. M. (1994). Synchronous oscillations in neuronal systems: mechanisms and functions. J. Comput. Neurosci. 1, 11–38. doi: 10.1007/BF00962716

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gray, C. M., Konig, P., Engel, A. K., and Singer, W. (1989). Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature 338, 334–337. doi: 10.1038/338334a0

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gray, C. M., and Viana Di Prisco, G. (1997). Stimulus-dependent neuronal oscillations and local synchronization in striate cortex of the alert cat. J. Neurosci. 17, 3239–3253.

Pubmed Abstract | Pubmed Full Text

Green, D. M., and Swets, J. A. (1966). Signal Detection Theory and Psychophysics. New York, NY: John Wiley and Sons, Inc.

Gregoriou, G. G., Gotts, S. J., Zhou, H., and Desimone, R. (2009). High-frequency, long-range coupling between prefrontal and visual cortex during attention. Science 324, 1207–1210. doi: 10.1126/science.1171402

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gu, Y., Angelaki, D. E., and Deangelis, G. C. (2008). Neural correlates of multisensory cue integration in macaque MSTd. Nat. Neurosci. 11, 1201–1210. doi: 10.1038/nn.2191

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Guido, W., Lu, S. M., Vaughan, J. W., Godwin, D. W., and Sherman, S. M. (1995). Receiver operating characteristic (ROC) analysis of neurons in the cat's lateral geniculate nucleus during tonic and burst response mode. Vis. Neurosci. 12, 723–741. doi: 10.1017/S0952523800008993

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gur, M., and Snodderly, D. M. (2006). High response reliability of neurons in primary visual cortex (V1) of alert, trained monkeys. Cereb. Cortex 16, 888–895. doi: 10.1093/cercor/bhj032

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hanes, D. P., and Schall, J. D. (1996). Neural control of voluntary movement initiation. Science 274, 427–430. doi: 10.1126/science.274.5286.427

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hanes, D. P., Thompson, K. G., and Schall, J. D. (1995). Relationship of presaccadic activity in frontal eye field and supplementary eye field to saccade initiation in macaque: poisson spike train analysis. Exp. Brain Res. 103, 85–96. doi: 10.1007/BF00241967

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Harrington, L. K., and Peck, C. K. (1998). Spatial disparity affects visual-auditory interactions in human sensorimotor processing. Exp. Brain Res. 122, 247–252. doi: 10.1007/s002210050512

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hasenstaub, A., Shu, Y., Haider, B., Kraushaar, U., Duque, A., and McCormick, D. A. (2005). Inhibitory postsynaptic potentials carry synchronized frequency information in active cortical networks. Neuron 47, 423–435. doi: 10.1016/j.neuron.2005.06.016

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Heeger, D. J. (1997). Signal Detection Theory [Online]. [Accessed].

Heeger, D. J. (2003). Signal Detection Theory [Online]. Available online at: http://teachline.ls.huji.ac.il/72633/SDTintro.pdf. [Accessed].

Henrie, J. A., and Shapley, R. (2005). LFP power spectra in V1 cortex: the graded effect of stimulus contrast. J. Neurophysiol. 94, 479–490. doi: 10.1152/jn.00919.2004

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hershenson, M. (1962). Reaction time as a measure of intersensory facilitation. J. Exp. Psychol. 63, 289–293. doi: 10.1037/h0039516

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hipp, J. F., Engel, A. K., and Siegel, M. (2011). Oscillatory synchronization in large-scale cortical networks predicts perception. Neuron 69, 387–396. doi: 10.1016/j.neuron.2010.12.027

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Holt, G. R., Softky, W. R., Koch, C., and Douglas, R. J. (1996). Comparison of discharge variability in vitro and in vivo in cat visual cortex neurons. J. Neurophysiol. 75, 1806–1814.

Pubmed Abstract | Pubmed Full Text

Hughes, H. C., Nelson, M. D., and Aronchick, D. M. (1998). Spatial characteristics of visual-auditory summation in human saccades. Vision Res. 38, 3955–3963. doi: 10.1016/S0042-6989(98)00036-4

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hughes, H. C., Reuter-Lorenz, P. A., Nozawa, G., and Fendrich, R. (1994). Visual-auditory interactions in sensorimotor processing: saccades versus manual responses. J. Exp. Psychol. Hum. Percept. Perform. 20, 131–153. doi: 10.1037/0096-1523.20.1.131

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hussar, C., and Pasternak, T. (2010). Trial-to-trial variability of the prefrontal neurons reveals the nature of their engagement in a motion discrimination task. Proc. Natl. Acad. Sci. U.S.A. 107, 21842–21847. doi: 10.1073/pnas.1009956107

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Jadhav, S. P., Wolfe, J., and Feldman, D. E. (2009). Sparse temporal coding of elementary tactile features during active whisker sensation. Nat. Neurosci. 12, 792–800. doi: 10.1038/nn.2328

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Jensen, O., and Lisman, J. E. (2005). Hippocampal sequence-encoding driven by a cortical multi-item working memory buffer. Trends Neurosci. 28, 67–72. doi: 10.1016/j.tins.2004.12.001

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Jermakowicz, W. J., and Casagrande, V. A. (2007). Neural networks a century after Cajal. Brain Res. Rev. 55, 264–284. doi: 10.1016/j.brainresrev.2007.06.003

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Jiang, W., Jiang, H., and Stein, B. E. (2002). Two corticotectal areas facilitate multisensory orientation behavior. J. Cogn. Neurosci. 14, 1240–1255. doi: 10.1162/089892902760807230

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Jones, M. R., Moynihan, H., Mackenzie, N., and Puente, J. (2002). Temporal aspects of stimulus-driven attending in dynamic arrays. Psychol. Sci. 13, 313–319. doi: 10.1111/1467-9280.00458

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kaiser, J., Lutzenberger, W., Ackermann, H., and Birbaumer, N. (2002). Dynamics of gamma-band activity induced by auditory pattern changes in humans. Cereb. Cortex 12, 212–221. doi: 10.1093/cercor/12.2.212

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kajikawa, Y., and Schroeder, C. E. (2011). How local is the local field potential? Neuron 72, 847–858. doi: 10.1016/j.neuron.2011.09.029

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kamondi, A., Acsady, L., Wang, X. J., and Buzsaki, G. (1998). Theta oscillations in somata and dendrites of hippocampal pyramidal cells in vivo: activity-dependent phase-precession of action potentials. Hippocampus 8, 244–261.

Pubmed Abstract | Pubmed Full Text

Kara, P., Reinagel, P., and Reid, R. C. (2000). Low response variability in simultaneously recorded retinal, thalamic, and cortical neurons. Neuron 27, 635–646. doi: 10.1016/S0896-6273(00)00072-6

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Katzner, S., Nauhaus, I., Benucci, A., Bonin, V., Ringach, D. L., and Carandini, M. (2009). Local origin of field potentials in visual cortex. Neuron 61, 35–41. doi: 10.1016/j.neuron.2008.11.016

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kayser, C., and Logothetis, N. K. (2007). Do early sensory cortices integrate cross-modal information? Brain Struct. Funct. 212, 121–132. doi: 10.1007/s00429-007-0154-0

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kayser, C., Logothetis, N. K., and Panzeri, S. (2010). Visual enhancement of the information representation in auditory cortex. Curr. Biol. 20, 19–24. doi: 10.1016/j.cub.2009.10.068

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kayser, C., Petkov, C. I., and Logothetis, N. K. (2007). Tuning to sound frequency in auditory field potentials. J. Neurophysiol. 98, 1806–1809. doi: 10.1152/jn.00358.2007

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kayser, C., Petkov, C. I., and Logothetis, N. K. (2008). Visual modulation of neurons in auditory cortex. Cereb. Cortex 18, 1560–1574. doi: 10.1093/cercor/bhm187

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Knyazev, G. G. (2007). Motivation, emotion, and their inhibitory control mirrored in brain oscillations. Neurosci. Biobehav. Rev. 31, 377–395. doi: 10.1016/j.neubiorev.2006.10.004

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kocsis, B., Bragin, A., and Buzsaki, G. (1999). Interdependence of multiple theta generators in the hippocampus: a partial coherence analysis. J. Neurosci. 19, 6200–6212.

Pubmed Abstract | Pubmed Full Text

Kreiman, G., Hung, C. P., Kraskov, A., Quiroga, R. Q., Poggio, T., and Dicarlo, J. J. (2006). Object selectivity of local field potentials and spikes in the macaque inferior temporal cortex. Neuron 49, 433–445. doi: 10.1016/j.neuron.2005.12.019

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kreiter, A. K., and Singer, W. (1992). Oscillatory neuronal responses in the visual cortex of the awake macaque monkey. Eur. J. Neurosci. 4, 369–375. doi: 10.1111/j.1460-9568.1992.tb00884.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Krueger, J., Royal, D. W., Fister, M. C., and Wallace, M. T. (2009). Spatial receptive field organization of multisensory neurons and its impact on multisensory interactions. Hear. Res. 258, 47–54. doi: 10.1016/j.heares.2009.08.003

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Lakatos, P., Chen, C. M., O'Connell, M. N., Mills, A., and Schroeder, C. E. (2007). Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron 53, 279–292. doi: 10.1016/j.neuron.2006.12.011

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Lakatos, P., Karmos, G., Mehta, A. D., Ulbert, I., and Schroeder, C. E. (2008). Entrainment of neuronal oscillations as a mechanism of attentional selection. Science 320, 110–113. doi: 10.1126/science.1154735

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Lakatos, P., Shah, A. S., Knuth, K. H., Ulbert, I., Karmos, G., and Schroeder, C. E. (2005). An oscillatory hierarchy controlling neuronal excitability and stimulus processing in the auditory cortex. J. Neurophysiol. 94, 1904–1911. doi: 10.1152/jn.00263.2005

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Linden, H., Tetzlaff, T., Potjans, T. C., Pettersen, K. H., Grun, S., Diesmann, M., et al. (2011). Modeling the spatial reach of the LFP. Neuron 72, 859–872. doi: 10.1016/j.neuron.2011.11.006

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Liu, J., and Newsome, W. T. (2006). Local field potential in cortical area MT: stimulus tuning and behavioral correlations. J. Neurosci. 26, 7779–7790. doi: 10.1523/JNEUROSCI.5052-05.2006

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Livingstone, M. S. (1996). Oscillatory firing and interneuronal correlations in squirrel monkey striate cortex. J. Neurophysiol. 75, 2467–2485.

Pubmed Abstract | Pubmed Full Text

Logothetis, N. K. (2003). The underpinnings of the BOLD functional magnetic resonance imaging signal. J Neurosci 23, 3963–3971.

Pubmed Abstract | Pubmed Full Text

Logothetis, N. K. (2008). What we can do and what we cannot do with fMRI. Nature 453, 869–878. doi: 10.1038/nature06976

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Logothetis, N. K., Pauls, J., Augath, M., Trinath, T., and Oeltermann, A. (2001). Neurophysiological investigation of the basis of the fMRI signal. Nature 412, 150–157. doi: 10.1038/35084005

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Lovelace, C. T., Stein, B. E., and Wallace, M. T. (2003). An irrelevant light enhances auditory detection in humans: a psychophysical analysis of multisensory integration in stimulus detection. Brain Res Cogn Brain Res 17, 447–453. doi: 10.1016/S0926-6410(03)00160-5

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Macmillan, N. A., and Creelman, C. D. (2004). Detection Theory: A User's Guide. Mahwah, NJ: Lawrence Erlbaum Associates.

Magri, C., Schridde, U., Murayama, Y., Panzeri, S., and Logothetis, N. K. (2012). The amplitude and timing of the BOLD signal reflects the relationship between local field potential power at different frequencies. J. Neurosci. 32, 1395–1407. doi: 10.1523/JNEUROSCI.3985-11.2012

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Meredith, M. A., and Allman, B. L. (2009). Subthreshold multisensory processing in cat auditory cortex. Neuroreport 20, 126–131. doi: 10.1097/WNR.0b013e32831d7bb6

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Meredith, M. A., Nemitz, J. W., and Stein, B. E. (1987). Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. J. Neurosci. 7, 3215–3229.

Pubmed Abstract | Pubmed Full Text

Meredith, M. A., and Stein, B. E. (1983). Interactions among converging sensory inputs in the superior colliculus. Science 221, 389–391. doi: 10.1126/science.6867718

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Meredith, M. A., and Stein, B. E. (1985). Descending efferents from the superior colliculus relay integrated multisensory information. Science 227, 657–659. doi: 10.1126/science.3969558

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Meredith, M. A., and Stein, B. E. (1986a). Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain Res. 365, 350–354. doi: 10.1016/0006-8993(86)91648-3

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Meredith, M. A., and Stein, B. E. (1986b). Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. J. Neurophysiol. 56, 640–662.

Pubmed Abstract | Pubmed Full Text

Meredith, M. A., and Stein, B. E. (1996). Spatial determinants of multisensory integration in cat superior colliculus neurons. J. Neurophysiol. 75, 1843–1857.

Pubmed Abstract | Pubmed Full Text

Mitzdorf, U. (1985). Current source-density method and application in cat cerebral cortex: investigation of evoked potentials and EEG phenomena. Physiol. Rev. 65, 37–100.

Pubmed Abstract | Pubmed Full Text

Mitzdorf, U. (1987). Properties of the evoked potential generators: current source-density analysis of visually evoked potentials in the cat cortex. Int. J. Neurosci. 33, 33–59. doi: 10.3109/00207458708985928

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Mochol, G., Wojcik, D. K., Wypych, M., Wrobel, A., and Waleszczyk, W. J. (2010). Variability of visual responses of superior colliculus neurons depends on stimulus velocity. J. Neurosci. 30, 3199–3209. doi: 10.1523/JNEUROSCI.3250-09.2010

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Molholm, S., Ritter, W., Murray, M. M., Javitt, D. C., Schroeder, C. E., and Foxe, J. J. (2002). Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. Brain Res. Cogn. Brain Res. 14, 115–128. doi: 10.1016/S0926-6410(02)00066-6

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Morrell, F. (1972). Visual system's view of acoustic space. Nature 238, 44–46. doi: 10.1038/238044a0

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Muller, M. M., Gruber, T., and Keil, A. (2000). Modulation of induced gamma band activity in the human EEG by attention and visual information processing. Int. J. Psychophysiol. 38, 283–299. doi: 10.1016/S0167-8760(00)00171-9

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Murray, M. M., Foxe, J. J., Higgins, B. A., Javitt, D. C., and Schroeder, C. E. (2001). Visuo-spatial neural response interactions in early cortical processing during a simple reaction time task: a high-density electrical mapping study. Neuropsychologia 39, 828–844. doi: 10.1016/S0028-3932(01)00004-5

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Nicholson, C., and Freeman, J. A. (1975). Theory of current source-density analysis and determination of conductivity tensor for anuran cerebellum. J. Neurophysiol. 38, 356–368.

Pubmed Abstract | Pubmed Full Text

Nunez, P. L., Pilgreen, K. L., Westdorp, A. F., Law, S. K., and Nelson, A. V. (1991). A visual study of surface potentials and Laplacians due to distributed neocortical sources: computer simulations and evoked potentials. Brain Topogr. 4, 151–168. doi: 10.1007/BF01132772

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

O'Connell, M. N., Falchier, A., McGinnis, T., Schroeder, C. E., and Lakatos, P. (2011). Dual mechanism of neuronal ensemble inhibition in primary auditory cortex. Neuron 69, 805–817. doi: 10.1016/j.neuron.2011.01.012

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Osipova, D., Takashima, A., Oostenveld, R., Fernandez, G., Maris, E., and Jensen, O. (2006). Theta and gamma oscillations predict encoding and retrieval of declarative memory. J. Neurosci. 26, 7523–7531. doi: 10.1523/JNEUROSCI.1948-06.2006

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Palva, S., and Palva, J. M. (2007). New vistas for alpha-frequency band oscillations. Trends Neurosci. 30, 150–158. doi: 10.1016/j.tins.2007.02.001

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Perkel, D. H., Gerstein, G. L., and Moore, G. P. (1967). Neuronal spike trains and stochastic point processes. II. Simultaneous spike trains. Biophys. J. 7, 419–440. doi: 10.1016/S0006-3495(67)86597-4

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Perrault, T. J. Jr., Vaughan, J. W., Stein, B. E., and Wallace, M. T. (2003). Neuron-specific response characteristics predict the magnitude of multisensory integration. J. Neurophysiol. 90, 4022–4026. doi: 10.1152/jn.00494.2003

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Perrault, T. J. Jr., Vaughan, J. W., Stein, B. E., and Wallace, M. T. (2005). Superior colliculus neurons use distinct operational modes in the integration of multisensory stimuli. J. Neurophysiol. 93, 2575–2586. doi: 10.1152/jn.00926.2004

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Pesaran, B. (2009). Uncovering the mysterious origins of local field potentials. Neuron 61, 1–2. doi: 10.1016/j.neuron.2008.12.019

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Pesaran, B., Pezaris, J. S., Sahani, M., Mitra, P. P., and Andersen, R. A. (2002). Temporal structure in neuronal activity during working memory in macaque parietal cortex. Nat. Neurosci. 5, 805–811. doi: 10.1038/nn890

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Powers, A. R. 3rd., Hillock, A. R., and Wallace, M. T. (2009). Perceptual training narrows the temporal window of multisensory binding. J. Neurosci. 29, 12265–12274. doi: 10.1523/JNEUROSCI.3501-09.2009

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Praamstra, P., Kourtis, D., Kwok, H. F., and Oostenveld, R. (2006). Neurophysiology of implicit timing in serial choice reaction-time performance. J. Neurosci. 26, 5448–5455. doi: 10.1523/JNEUROSCI.0440-06.2006

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Rauch, A., Rainer, G., and Logothetis, N. K. (2008). The effect of a serotonin-induced dissociation between spiking and perisynaptic activity on BOLD functional MRI. Proc. Natl. Acad. Sci. U.S.A. 105, 6759–6764. doi: 10.1073/pnas.0800312105

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ray, S., Crone, N. E., Niebur, E., Franaszczuk, P. J., and Hsiao, S. S. (2008a). Neural correlates of high-gamma oscillations (60-200 Hz) in macaque local field potentials and their potential implications in electrocorticography. J. Neurosci. 28, 11526–11536. doi: 10.1523/JNEUROSCI.2848-08.2008

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ray, S., Hsiao, S. S., Crone, N. E., Franaszczuk, P. J., and Niebur, E. (2008b). Effect of stimulus intensity on the spike-local field potential relationship in the secondary somatosensory cortex. J. Neurosci. 28, 7334–7343. doi: 10.1523/JNEUROSCI.1588-08.2008

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Reed, J. L., Pouget, P., Qi, H. X., Zhou, Z., Bernard, M. R., Burish, M. J., et al. (2008). Widespread spatial integration in primary somatosensory cortex. Proc. Natl. Acad. Sci. U.S.A. 105, 10233–10237. doi: 10.1073/pnas.0803800105

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Reich, D. S., Victor, J. D., Knight, B. W., Ozaki, T., and Kaplan, E. (1997). Response variability and timing precision of neuronal spike trains in vivo. J. Neurophysiol. 77, 2836–2841.

Pubmed Abstract | Pubmed Full Text

Rockland, K. S., and Ojima, H. (2003). Multisensory convergence in calcarine visual areas in macaque monkey. Int. J. Psychophysiol. 50, 19–26. doi: 10.1016/S0167-8760(03)00121-1

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Roelfsema, P. R., Engel, A. K., Konig, P., and Singer, W. (1997). Visuomotor integration is associated with zero time-lag synchronization among cortical areas. Nature 385, 157–161. doi: 10.1038/385157a0

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Romei, V., Gross, J., and Thut, G. (2012). Sounds reset rhythms of visual cortex and corresponding human visual perception. Curr. Biol. 22, 807–813. doi: 10.1016/j.cub.2012.03.025

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Romo, R., Hernandez, A., Zainos, A., and Salinas, E. (2003). Correlated neuronal discharges that increase coding efficiency during perceptual discrimination. Neuron 38, 649–657. doi: 10.1016/S0896-6273(03)00287-3

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Roy, S., and Alloway, K. D. (1999). Synchronization of local neural networks in the somatosensory cortex: a comparison of stationary and moving stimuli. J. Neurophysiol. 81, 999–1013.

Pubmed Abstract | Pubmed Full Text

Roy, A., Steinmetz, P. N., and Niebur, E. (2000). Rate limitations of unitary event analysis. Neural Comput. 12, 2063–2082. doi: 10.1162/089976600300015060

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Royal, D. W., Carriere, B. N., and Wallace, M. T. (2009). Spatiotemporal architecture of cortical receptive fields and its impact on multisensory interactions. Exp. Brain Res. 198, 127–136. doi: 10.1007/s00221-009-1772-y

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Salinas, E., and Romo, R. (2000). Neurobiology. A chorus line. Nature 404, 131–133. doi: 10.1038/35004686

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Salinas, E., and Sejnowski, T. J. (2001). Correlated neuronal activity and the flow of neural information. Nat. Rev. Neurosci. 2, 539–550. doi: 10.1038/35086012

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Sanes, J. N., and Donoghue, J. P. (1993). Oscillations in local field potentials of the primate motor cortex during voluntary movement. Proc. Natl. Acad. Sci. U.S.A. 90, 4470–4474. doi: 10.1073/pnas.90.10.4470

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Sarko, D. K., Nidiffer, A. R., and Wallace, M. T. (2012). Multisensory integration is related to changes in neuronal response reliability in awake animals. Soc. Neurosci. Abstr.

Schroeder, C. E., and Foxe, J. J. (2002). The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Brain Res. Cogn. Brain Res. 14, 187–198. doi: 10.1016/S0926-6410(02)00073-3

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Schroeder, C. E., and Lakatos, P. (2009a). The gamma oscillation: master or slave? Brain Topogr. 22, 24–26. doi: 10.1007/s10548-009-0080-y

CrossRef Full Text

Schroeder, C. E., and Lakatos, P. (2009b). Low-frequency neuronal oscillations as instruments of sensory selection. Trends Neurosci. 32, 9–18. doi: 10.1016/j.tins.2008.09.012

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Schroeder, C. E., Lakatos, P., Kajikawa, Y., Partan, S., and Puce, A. (2008). Neuronal oscillations and visual amplification of speech. Trends Cogn. Sci. 12, 106–113. doi: 10.1016/j.tics.2008.01.002

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Schroeder, C. E., Lindsley, R. W., Specht, C., Marcovici, A., Smiley, J. F., and Javitt, D. C. (2001). Somatosensory input to auditory association cortex in the macaque monkey. J. Neurophysiol. 85, 1322–1327.

Pubmed Abstract | Pubmed Full Text

Schroeder, C. E., Mehta, A. D., and Givre, S. J. (1998). A spatiotemporal profile of visual system activation revealed by current source density analysis in the awake macaque. Cereb. Cortex 8, 575–592. doi: 10.1093/cercor/8.7.575

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Schroeder, C. E., Smiley, J., Fu, K. G., McGinnis, T., O'Connell, M. N., and Hackett, T. A. (2003). Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing. Int. J. Psychophysiol. 50, 5–17. doi: 10.1016/S0167-8760(03)00120-X

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Schroeder, C. E., Steinschneider, M., Javitt, D. C., Tenke, C. E., Givre, S. J., Mehta, A. D. et al., Jr. (1995). Localization of ERP generators and identification of underlying neural processes. Electroencephalogr. Clin. Neurophysiol. Suppl. 44, 55–75.

Pubmed Abstract | Pubmed Full Text

Senkowski, D., Schneider, T. R., Foxe, J. J., and Engel, A. K. (2008). Crossmodal binding through neural coherence: implications for multisensory processing. Trends Neurosci. 31, 401–409. doi: 10.1016/j.tins.2008.05.002

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Shadlen, M. N., and Newsome, W. T. (1994). Noise, neural codes and cortical organization. Curr. Opin. Neurobiol. 4, 569–579. doi: 10.1016/0959-4388(94)90059-0

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Shams, L., Kamitani, Y., and Shimojo, S. (2000). Illusions. What you see is what you hear. Nature 408, 788. doi: 10.1038/35048669

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Shams, L., Ma, W. J., and Beierholm, U. (2005). Sound-induced flash illusion as an optimal percept. Neuroreport 16, 1923–1927. doi: 10.1097/01.wnr.0000187634.68504.bb

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Singer, W. (1993). Synchronization of cortical activity and its putative role in information processing and learning. Annu. Rev. Physiol. 55, 349–374. doi: 10.1146/annurev.ph.55.030193.002025

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Singer, W., and Gray, C. M. (1995). Visual feature integration and the temporal correlation hypothesis. Annu. Rev. Neurosci. 18, 555–586. doi: 10.1146/annurev.ne.18.030195.003011

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Softky, W. R., and Koch, C. (1993). The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs. J. Neurosci. 13, 334–350.

Pubmed Abstract | Pubmed Full Text

Stanford, T. R., Quessy, S., and Stein, B. E. (2005). Evaluating the operations underlying multisensory integration in the cat superior colliculus. J. Neurosci. 25, 6499–6508. doi: 10.1523/JNEUROSCI.5095-04.2005

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Stein, B. E. (1988). Superior colliculus-mediated visual behaviors in cat and the concept of two corticotectal systems. Prog. Brain Res. 75, 37–53. doi: 10.1016/S0079-6123(08)60464-1

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Stein, B. E., Huneycutt, W. S., and Meredith, M. A. (1988). Neurons and behavior: the same rules of multisensory integration apply. Brain Res. 448, 355–358. doi: 10.1016/0006-8993(88)91276-0

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Stein, B. E., and Meredith, M. A. (1993). The Merging of the Senses. Cambridge, MA: MIT Press.

Stein, B. E., Meredith, M. A., Huneycutt, W. S., and McDade, L. (1989). Behavioral indices of multisensory integration: orientation to visual cues is affected by auditory stimuli. J. Cogn. Neurosci. 1, 12–24. doi: 10.1162/jocn.1989.1.1.12

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Stein, B. E., and Stanford, T. R. (2008). Multisensory integration: current issues from the perspective of the single neuron. Nat. Rev. Neurosci. 9, 255–266. doi: 10.1038/nrn2331

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Steinmetz, P. N., Roy, A., Fitzgerald, P. J., Hsiao, S. S., Johnson, K. O., and Niebur, E. (2000). Attention modulates synchronized neuronal firing in primate somatosensory cortex. Nature 404, 187–190. doi: 10.1038/35004588

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Steriade, M. (2001). Impact of network activities on neuronal properties in corticothalamic systems. J. Neurophysiol. 86, 1–39.

Pubmed Abstract | Pubmed Full Text

Tallon-Baudry, C., Bertrand, O., Delpuech, C., and Pernier, J. (1996). Stimulus specificity of phase-locked and non-phase-locked 40 Hz visual responses in human. J. Neurosci. 16, 4240–4249.

Pubmed Abstract | Pubmed Full Text

Tallon-Baudry, C., Bertrand, O., Peronnet, F., and Pernier, J. (1998). Induced gamma-band activity during the delay of a visual short-term memory task in humans. J. Neurosci. 18, 4244–4254.

Pubmed Abstract | Pubmed Full Text

Taylor, K., Mandon, S., Freiwald, W. A., and Kreiter, A. K. (2005). Coherent oscillatory activity in monkey area v4 predicts successful allocation of attention. Cereb. Cortex 15, 1424–1437. doi: 10.1093/cercor/bhi023

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Tenke, C. E., Schroeder, C. E., Arezzo, J. C., and Vaughan, H. G. Jr. (1993). Interpretation of high-resolution current source density profiles: a simulation of sublaminar contributions to the visual evoked potential. Exp. Brain Res. 94, 183–192. doi: 10.1007/BF00230286

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Thompson, K. G., Hanes, D. P., Bichot, N. P., and Schall, J. D. (1996). Perceptual and motor processing stages identified in the activity of macaque frontal eye field neurons during visual search. J. Neurophysiol. 76, 4040–4055.

Pubmed Abstract | Pubmed Full Text

Tolhurst, D. J., Movshon, J. A., and Dean, A. F. (1983). The statistical reliability of signals in single neurons in cat and monkey visual cortex. Vision Res. 23, 775–785. doi: 10.1016/0042-6989(83)90200-6

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Trevelyan, A. J. (2009). The direct relationship between inhibitory currents and local field potentials. J. Neurosci. 29, 15299–15307. doi: 10.1523/JNEUROSCI.2019-09.2009

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Usrey, W. M., and Reid, R. C. (1999). Synchronous activity in the visual system. Annu. Rev. Physiol. 61, 435–456. doi: 10.1146/annurev.physiol.61.1.435

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Viswanathan, A., and Freeman, R. D. (2007). Neurometabolic coupling in cerebral cortex reflects synaptic more than spiking activity. Nat. Neurosci. 10, 1308–1312. doi: 10.1038/nn1977

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Von Stein, A., and Sarnthein, J. (2000). Different frequencies for different scales of cortical integration: from local gamma to long range alpha/theta synchronization. Int. J. Psychophysiol. 38, 301–313. doi: 10.1016/S0167-8760(00)00172-0

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Wallace, M. T., McHaffie, J. G., and Stein, B. E. (1997). Visual response properties and visuotopic representation in the newborn monkey superior colliculus. J. Neurophysiol. 78, 2732–2741.

Pubmed Abstract | Pubmed Full Text

Wallace, M. T., Meredith, M. A., and Stein, B. E. (1998). Multisensory integration in the superior colliculus of the alert cat. J. Neurophysiol. 80, 1006–1010.

Pubmed Abstract | Pubmed Full Text

Wallace, M. T., Wilkinson, L. K., and Stein, B. E. (1996). Representation and integration of multiple sensory inputs in primate superior colliculus. J. Neurophysiol. 76, 1246–1266.

Pubmed Abstract | Pubmed Full Text

Wehr, M., and Laurent, G. (1996). Odour encoding by temporal sequences of firing in oscillating neural assemblies. Nature 384, 162–166. doi: 10.1038/384162a0

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Wilke, M., Logothetis, N. K., and Leopold, D. A. (2006). Local field potential reflects perceptual suppression in monkey visual cortex. Proc. Natl. Acad. Sci. U.S.A. 103, 17507–17512. doi: 10.1073/pnas.0604673103

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Wolfe, J., Hill, D. N., Pahlavan, S., Drew, P. J., Kleinfeld, D., and Feldman, D. E. (2008). Texture coding in the rat whisker system: slip-stick versus differential resonance. PLoS Biol. 6:e215. doi: 10.1371/journal.pbio.0060215

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Womelsdorf, T., Schoffelen, J. M., Oostenveld, R., Singer, W., Desimone, R., Engel, A. K., et al. (2007). Modulation of neuronal interactions through neuronal synchronization. Science 316, 1609–1612. doi: 10.1126/science.1139597

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Xing, D., Yeh, C. I., and Shapley, R. M. (2009). Spatial spread of the local field potential and its laminar variation in visual cortex. J. Neurosci. 29, 11540–11549. doi: 10.1523/JNEUROSCI.2573-09.2009

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text