Abstract
The ability of animals to detect motion is critical for survival, and errors or even delays in motion perception may prove costly. In the natural world, moving objects in the visual field often produce concurrent sounds. Thus, it can highly advantageous to detect motion elicited from sensory signals of either modality, and to integrate them to produce more reliable motion perception. A great deal of progress has been made in understanding how visual motion perception is governed by the activity of single neurons in the primate cerebral cortex, but far less progress has been made in understanding both auditory motion and audiovisual motion integration. Here we, review the key cortical regions for motion processing, focussing on translational motion. We compare the representations of space and motion in the visual and auditory systems, and examine how single neurons in these two sensory systems encode the direction of motion. We also discuss the way in which humans integrate of audio and visual motion cues, and the regions of the cortex that may mediate this process.
The natural world abounds with motion, making this a highly salient cue to guide animals in interacting with the environment. It is therefore not surprising that most, if not all brains have dedicated neural circuits for the perception of motion. In primates, the cerebral cortex contains a network of regions that are specialized for motion processing, but the systems for processing the motion of visual features and sounds are mediated by different brain regions, and underpinned by different physiological mechanisms. In this mini-review article, we will discuss the encoding of direction of motion in the visual and auditory systems, with emphasis on the cortical systems that are involved in translational motion, especially in azimuth (leftwards and rightwards motion), as this is the most common type of motion used in audiovisual integration studies.
Encoding of Direction of Motion in the Activity of Cortical Neurons
Spatial features are represented in fundamentally different ways in the visual and auditory systems. In the visual system, most neurons have spatially defined receptive fields, which are ultimately defined by inputs from specific regions of the retina. Therefore, the responses of neurons in the visual system are inherently capable of coding the spatial location of visual stimuli, and in theory, could encode direction of motion by the sequential activation of populations of neurons with different receptive field locations. However, the visual system goes one step further, with direction of motion being explicitly represented at the level of the single cell. Specifically, the spiking (action potential) responses of neurons are tuned to the direction of moving stimuli, meaning that they are more active in response to a specific direction of motion compared to other directions (Dubner and Zeki, 1971; Baker et al., 1981; Maunsell and Van Essen, 1983a; Albright, 1984; Desimone and Ungerleider, 1986; Saito et al., 1986; Tanaka and Saito, 1989; Chaplin et al., 2017). Thus, direction selective neurons in the visual system can encode the direction of motion within their receptive fields. For example, Figure 1A shows the response of a direction tuned neuron: the neuron shows strong responses to motion towards the upper left quadrant, and progressively weaker responses for directions further away.
Figure 1
Encoding of direction of motion in the visual and auditory systems. (A) A typical visual direction tuning curve from a neuron in the marmoset visual cortex (area MT) in response to a moving dot stimulus (data from Chaplin et al., 2017). The vertical line indicates the preferred direction of motion, and the inset shows the mean spiking responses (with the spontaneous rate subtracted) in polar plot form, showing clear direction selectivity. (B) The temporal spiking response of a neuron in the macaque auditory cortex (A1) in response to a moving auditory stimulus. Here, the difference in firing rate between two directions of motion is quite modest, and is most obvious in the later part of the response. Redrawn with permission from the authors of Ahissar et al. (1992). (C) Inflated model of the macaque cerebral cortex showing some of the motion processing areas in the primate cerebral cortex (Van Essen, 2002; Van Essen and Dierker, 2007). Light blue areas: visual areas where a subpopulation of neurons shows direction selectivity, dark blue areas: visual motion processing areas MT, MSTd and MSTl, orange: A1, red: areas of the caudal auditory belt (CM, CL) which have been implicated in auditory motion processing, purple: areas that show auditory and visual motion responses and may be involved in integrating the two modalities.
In contrast, most neurons in the auditory system respond to specific ranges of acoustic frequencies, since they ultimately receive inputs from defined regions of the cochlea. Thus, the auditory system needs to exploit other auditory cues to extract spatial information from the stimulus. The principal cues for locating sounds in the azimuth are binaural—interaural time differences (ITDs) and interaural level differences (ILDs; Middlebrooks and Green, 1991). Several brain regions are involved in the perception of sound location, and neurons in these regions can be tuned for ITDs or ILDs (Masterton et al., 1967; Rajan et al., 1990a,b; Semple and Kitzes, 1993a,b; Irvine et al., 1996; Tian et al., 2001; Woods et al., 2006; Miller and Recanzone, 2009; Grothe et al., 2010; Slee and Young, 2010; Kusmierek and Rauschecker, 2014; Keating and King, 2015; Lui et al., 2015; Mokri et al., 2015).
The encoding of the direction of auditory motion by the activity of single cortical neurons has not been studied extensively in primates—to our knowledge, there is only published study (Ahissar et al., 1992), in which they recorded spiking activity in the primary auditory cortex (A1) of monkeys. They found that while many cells (62%) in A1 code for the spatial location of stationary sounds, some cells (32%) also showed a preference for leftwards or rightwards direction of motion. However, the differences in responses were far less marked than those observed in direction selective cells in the visual system. There were only modest differences in firing rates, which were evident in the late part of the responses (Figure 1B). These results suggest that the encoding of the direction of motion of auditory stimuli is likely to be a much more distributed representation across a neuronal populations, compared to direction of motion encoding in the visual system (Cohen and Newsome, 2009), or that explicit encoding of auditory motion relies on other areas beyond A1.
Visual Motion Processing Areas
The neural circuits for visual motion processing are among the best understood aspects of the structure and function of the primate cerebral cortex (Figure 1C, blue areas). The primary visual cortex (V1) is the first stage of visual processing in the cerebral cortex in which direction selectivity first appears, but only a small proportion of V1 neurons are direction selective (~15%, Yu et al., 2010; Yu and Rosa, 2014; Davies et al., 2016). Direction selective neurons have been observed in several other visual areas (Orban et al., 1986; Desimone and Schein, 1987; Felleman and Van Essen, 1987; Lui et al., 2005, 2006; Orban, 2008; Fattori et al., 2009; Li et al., 2013), but it is the middle temporal (MT) and medial superior temporal (MST) areas that appear to be most specialized for motion processing. The vast majority of cells in these regions are direction selective (MT ~85%: Allman and Kaas, 1971; Dubner and Zeki, 1971; Maunsell and Van Essen, 1983b; Albright, 1984; MST ~90%: Desimone and Ungerleider, 1986; Saito et al., 1986; Tanaka and Saito, 1989; Celebrini and Newsome, 1994; Elston and Rosa, 1997). Furthermore, it is known that damage to MT and MST results in motion perception impairments (Newsome and Paré, 1988; Pasternak and Merigan, 1994; Orban et al., 1995; Schenk and Zihl, 1997; Rudolph and Pasternak, 1999), and electrical stimulation of these regions can influence the perception of motion (Celebrini and Newsome, 1994, 1995; Salzman and Newsome, 1994; Britten and Van Wezel, 2002; Nichols and Newsome, 2002; Fetsch et al., 2014). Thus, a causal relationship has been established between neural activity in MT and MST and the perception of visual motion.
MST can be divided to two subregions: a lateral part (MSTl) involved in the perception of moving objects and smooth pursuit eye movements (Komatsu and Wurtz, 1988a,b; Eifuku and Wurtz, 1998), and dorsal part (MSTd), which is associated with the perception of complex motion patterns (Graziano et al., 1994; Mineault et al., 2012), especially self-motion (Saito et al., 1986; Komatsu and Wurtz, 1988a; Duffy and Wurtz, 1991; Duffy, 1998), and has a well described role in the integration of visual and vestibular motion cues (Gu et al., 2007, 2008). Differences between MT and MST have been well studied in monkeys, but in human studies these areas are typically grouped into a single region called the human MT complex (hMT+, Zeki et al., 1991; Huk et al., 2002), due to the spatial resolution limits of fMRI.
Auditory Motion Processing Areas
In comparison to the visual system, the regions and circuitry of the cortex involved in auditory motion processing are not as well characterized (Figure 1C). While there is some evidence for motion sensitivity and direction selectivity in the A1 (Ahissar et al., 1992; Griffiths et al., 2000; Lewis et al., 2000), many human imaging studies have identified the planum temporale, a region of auditory cortex caudal to primary cortex, as being the key site for auditory motion processing (Baumgart et al., 1999; Pavani et al., 2002; Warren et al., 2002; Alink et al., 2012b). In agreement with these findings, a recent imaging study in macaques also found that the caudal regions of auditory cortex are differentially activated by auditory motion compared to stationary stimuli (Poirier et al., 2017). Furthermore, studies of humans with lesions to caudal auditory cortex have found deficits in auditory motion processing (Ducommun et al., 2004; Lewald et al., 2009; Thaler et al., 2016).
It remains controversial whether auditory motion perception relies on specialized motion detectors, similar to direction selective cells in the visual cortex (Perrott and Musicant, 1977), or utilizes “snapshots” of the current sound source location (Ahissar et al., 1992; Poirier et al., 2017), as several human imaging studies have reported there is no difference in cortical activation between stationary and moving stimuli (Smith et al., 2004, 2007; Krumbholz et al., 2005, 2007). Since neurons in the auditory system show sensitivity to localization cues (e.g., ITDs and ILDs), the perception of motion could be mediated by the sequential activation of neurons that code for adjacent spatial locations (Ahissar et al., 1992). In general, in the auditory system the integration of binaural cues for sound localization occurs at early subcortical stages of processing, such as the superior olivary complex, the nuclei of the lateral lemniscus and the inferior colliculus (Moore, 1991). In monkeys, the caudal part of auditory cortex encompasses the caudomedial (CM) and caudolateral (CL) areas of the auditory belt (Hackett et al., 1998; Kaas et al., 1999), and these are known to play a role in the localization of auditory stimuli (Recanzone et al., 2000; Tian et al., 2001; Woods et al., 2006; Miller and Recanzone, 2009; Kusmierek and Rauschecker, 2014). Therefore, the sensitivity of neurons in these areas to the location of static stimuli is a potential confound in auditory motion studies, as it can be difficult to distinguish true motion sensitivity from sensitivity to spatial location. For example, it has been suggested that apparent sensitivities to motion in the inferior colliculus could be explained by adaptation to stationary stimuli, which would result in reduced spiking activity for stationary stimuli compared to moving stimuli (Ahissar et al., 1992; Wilson and O’Neill, 1998; McAlpine et al., 2000; Ingham et al., 2001; Poirier et al., 2017). However, the recent imaging study by Poirier et al. (2017) did take steps to control for this effect in their choice of stimuli and regressions analyses, and still found that the caudal auditory cortex was differentially activated by auditory motion compared to static motion. Further electrophysiological studies in monkeys will be required to address the question of how auditory motion is encoded by the spiking activity of neurons in these regions.
The neural representation of auditory motion does not necessarily have to be located in purely auditory regions. Direct reciprocal connections between MT/MST and the auditory cortex have been identified in primates (Palmer and Rosa, 2006), and two recent electrophysiological studies (Chaplin et al., 2018; Kafaligonul et al., 2018) have reported evoked potentials in areas MT/MST in response to stationary auditory clicks. Two human imaging studies have reported that the hMT+ complex responds to auditory motion (Poirier et al., 2005; Strnad et al., 2013), but it has also been argued that observed auditory responses in hMT+ could be explained by localization errors (Jiang et al., 2014), and no study has found any evidence for spiking activity in response to auditory stimuli (moving or stationary) in the monkey MT complex. Furthermore, a case study of involving lesions of hMT+ did not find any impairment in the perception of auditory motion (Zihl et al., 1983). Thus, current evidence suggests that MT and MST are not involved in auditory motion processing.
Integration of Auditory and Visual Motion Cues
Given the differences in the neural representation of motion in the auditory and visual systems, it is interesting to consider how the information from the two modalities could be combined to improve motion perception. Psychophysical studies have investigated audiovisual motion integration in humans using motion detection tasks, and have provided valuable insights into how auditory and visual motion can be integrated in the brain. Some of these studies have reported that humans perform better in audiovisual motion tasks compared to unimodal tasks, but there is disagreement as to whether this increase in performance is “statistically optimal” or the result of “probability summation.” When probability summation occurs, observers perform better on bimodal trials because they essentially have two chances to answer correctly—using either the visual or the auditory cue (Wuerger et al., 2003; Alais and Burr, 2004). When statistically optimal integration occurs, observers combine the information obtained by the different senses by weighting according to their reliability, to make optimal use of the information available (Meyer and Wuerger, 2001). Therefore, statistically optimal integration exceeds the performance of probability summation. Multisensory integration has shown be statistically optimal in other contexts (Ernst and Banks, 2002; Angelaki et al., 2009; Fetsch et al., 2009; Drugowitsch et al., 2014; Rohde et al., 2016).
It has been argued that statistically optimal integration of multisensory cues relies on neural computations occurring in early sensory cortex (e.g., MT/MST), rather than in higher-level areas (Ma et al., 2006; Beck et al., 2008; Bizley et al., 2016). In contrast, when multisensory integration is the result of probability summation, it may rely on higher-order areas (e.g., prefrontal or posterior parietal cortex, Alais and Burr, 2004; Bizley et al., 2016).
Audiovisual Motion Integration in the Primate Cerebral Cortex
Human imaging studies and monkey electrophysiological/anatomical studies have suggested several candidate cortical regions for the integration of audiovisual motion. The human superior temporal sulcus is typically activated by moving audiovisual stimuli (Lewis et al., 2000; Baumann and Greenlee, 2007; von Saldern and Noppeney, 2013). This region likely corresponds to the superior temporal polysensory (STP) area of macaques (Bruce et al., 1981), and the presence of multisensory neurons in STP is well known (Bruce et al., 1981; Hikosaka et al., 1988; Watanabe and Iwai, 1991). STP is typically associated with processing more complex visual and auditory signals, such as faces and speech (Beauchamp, 2005) and biological motion (Oram and Perrett, 1994; Barraclough et al., 2005), especially in complex tasks (Meyer et al., 2011; Wuerger et al., 2012), but there is evidence of subregional specializations (Padberg et al., 2003).
The posterior parietal cortex may also be important for audiovisual motion integration, as areas in this complex have found to be active during audiovisual stimulation in humans (Baumann and Greenlee, 2007; Wuerger et al., 2012), and is thought play a key role in coordinating multisensory integration (Brang et al., 2013). Cells in the ventral intraparietal area (VIP) are known to respond to both visual motion (Cook and Maunsell, 2002; Kaminiarz et al., 2014) and auditory stimuli (Bremmer et al., 2001; Schlack et al., 2005). The lateral intraparietal area (LIP) has been demonstrated to be involved in the integration visual motion signals over time to form perceptual decisions (Roitman and Shadlen, 2002), and also responds to auditory stimulation (Grunewald et al., 1999; Linden et al., 1999). Therefore, it is possible that LIP could integration information from both senses, by preforming similar computations.
Integration could also occur at the level of the prefrontal cortex (PFC), as regions in the dorsolateral PFC (areas 8a, 45 and 46) are known to receive inputs from MT and MST (Lewis and Van Essen, 2000; Reser et al., 2013) as well as caudal auditory cortex (Romanski et al., 1999a,b). Furthermore, direction selective responses to visual motion have been demonstrated in this region (Zaksas and Pasternak, 2006), and like LIP, PFC neurons show activity that is consistent with accumulating sensory evidence to form perceptual decisions (Kim and Shadlen, 1999). Cells in the ventrolateral subdivision of the PFC, such as area 12, have been shown to integrate audiovisual cues, but like STP, are generally associated with higher level sensory processing, responding to individual faces and calls (Romanski, 2007, 2012). However, human imaging studies of audiovisual motion have generally not reported comparable activation in the PFC (Lewis et al., 2000; Baumann and Greenlee, 2007; von Saldern and Noppeney, 2013), although audiovisual biological motion can modulate activity in premotor areas (areas 6R and 44) when there is a mismatch between the auditory and visual cues (Wuerger et al., 2012).
A number of imaging studies have also found that audiovisual stimulation produces distinct activation (compared to visual only stimulation) in hMT+ (Alink et al., 2008; Lewis and Noppeney, 2010; Strnad et al., 2013; von Saldern and Noppeney, 2013), suggesting that auditory stimuli can modulate visually evoked responses (although this is not always the case, e.g., Wuerger et al., 2012). These regions receive sparse inputs from auditory cortex (Palmer and Rosa, 2006), and show evoked potentials in response to auditory stimuli (Chaplin et al., 2018; Kafaligonul et al., 2018). Additionally, auditory motion has been shown to affect various aspects of visual perception, such as improving visual motion detection (Kim et al., 2012), improve learning in visual motion tasks (Seitz et al., 2006), and induce visual illusions (Sekuler et al., 1997; Meyer and Wuerger, 2001; Kitagawa and Ichihara, 2002; Beer and Röder, 2004; Soto-Faraco et al., 2005; Freeman and Driver, 2008; Alink et al., 2012a; Kafaligonul and Stoner, 2012; Kafaligonul and Oluk, 2015). Altogether, these studies suggest that auditory stimuli, especially when moving, could modulate responses to visual stimuli in MT/MST.
To specifically test this hypothesis, we have investigated if auditory motion cues are integrated with visual motion cues in MT/MST, by recording spiking activity and characterizing the ability of neurons to encode the direction of motion, using ideal observer analysis (Chaplin et al., 2018). We presented random dot patterns that moved either leftwards or rightwards, and manipulated the strength of the visual motion signal by reducing the coherence of the dots (i.e., making some proportion of the dots move in random directions). Reducing motion coherence reduces the both the psychophysical performance of observers (i.e., makes it more difficult to discriminate the directions of motion) and the neurometric performance of single neurons (i.e., reduces the neuronal information; Newsome et al., 1989). We hypothesized that the addition of an auditory stimulus that moved in the same direction as the visual stimulus would increase the information carried by single neurons and therefore increase neurometric performance, just as it can increase psychophysical performance in humans (Meyer and Wuerger, 2001; Kim et al., 2012). In particular, we predicted that auditory cues would be most likely be integrated at low motion coherence levels, in line with Bayesian models of multisensory integration (Ernst and Banks, 2002; Ma et al., 2006; Gu et al., 2008). However, we found no evidence of spike rate modulations (Figure 2A) or improvements in neurometric performance (Figure 2B) due to the auditory stimulus, in MT or MST. It may be the case that the audiovisual responses observed in hMT+ are the result of task related signals (Alink et al., 2012b; Bizley et al., 2016; Kayser et al., 2017), such as the binding of the two modalities to form a unified percept (Nahorna et al., 2012, 2015; Bizley and Cohen, 2013), attentional effects (Beer and Röder, 2004, 2005; Lakatos et al., 2008), or choice-related signals from the decision making process (Cumming and Nienborg, 2016).
Figure 2
(A) Responses of a marmoset MT neuron to visual, auditory and audiovisual stimuli. The raster plots (black dots) and spike rate functions (colored lines) show a clear response to visual but not auditory stimuli (blue vs. green lines). The combination of auditory and visual stimuli (red line) was not significantly different to the visual only response (blue vs. red lines). (B) Neurometric performance (measured as the area under the receiver operating characteristic (ROC) curve, Britten et al., 1992, which corresponds to the performance of an ideal observer discriminating the direction of motion using the spiking activity of the neuron) of a marmoset MT neuron when discriminating leftwards and rightwards motion under visual (blue) and audiovisual (red) conditions at different levels of motion coherence (strength of motion signal). The addition of the auditory stimulus did not shift the neurometric curve to the left as would be expected if the neuron was integrating the auditory motion cue (adapted from Chaplin et al., 2018).
Only one other study has investigated the effects of auditory stimuli on the responses of MT neurons (Kafaligonul et al., 2018). This study aimed to test if the activity of MT neurons mediated the temporal ventriloquist illusion, in which stationary auditory clicks induce influence the perception of visual speed. The authors hypothesized that the auditory clicks would alter the speed tuning and response duration of MT neurons in response to apparent visual motion. However, the auditory stimuli did not alter speed tuning or response duration in a way that would support the perception of the illusion, even though there was a possible modulation of the temporal spiking response. Therefore, electrophysiological studies in monkeys so far suggest that auditory stimuli do not influence visual motion perception through changes in activity to MT/MST neurons. However, since the projections from auditory to visual cortex are known to arrive at the peripheral representation of the visual field (Palmer and Rosa, 2006; Majka et al., 2018), it possible that their role of auditory inputs to facilitate the detection and localization of visual features, especially for orienting (Perrott et al., 1993; Wang et al., 2008).
Conclusion
In conclusion, the processing of auditory and visual motion in the primate cerebral cortex utilizes different brain areas and physiological mechanisms. While good progress has been made in identifying the cortical regions involved in processing auditory and audiovisual motion, the mechanisms of audiovisual integration remain unclear. The current evidence from single neuron studies suggests that the integration of auditory and visual motion cues is not mediated by the early visual areas MT and MST, and therefore such integration likely occurs in higher level cortical areas. Another possibility is that the integration of audiovisual motion signals is not mediated by a single brain region, but instead by synchronized network activity (Lewis and Noppeney, 2010).
Statements
Author contributions
TC wrote the first draft of the manuscript. MR and LL wrote sections of the manuscript. All authors contributed to manuscript revision, read and approved the submitted version.
Funding
This project was funded by the Australian Research Council (DE130100493 to LL; CE140100007 to MR) and by the National Health and Medical Research Council of Australia (APP1066232 to LL, APP1083152 to MR and APP1159764 to TC). TC was funded by an Australian Postgraduate Award and a Monash University Faculty of Medicine Bridging Postdoctoral Fellowship.
Acknowledgments
We thank Ramesh Rajan for advice on the manuscript and Merav Ahissar for giving permission for Figure 1B.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
1
AhissarM.AhissarE.BergmanH.VaadiaE. (1992). Encoding of sound-source location and movement: activity of single neurons and interactions between adjacent neurons in the monkey auditory cortex. J. Neurophysiol.67, 203–215. 10.1152/jn.1992.67.1.203
2
AlaisD.BurrD. C. (2004). No direction-specific bimodal facilitation for audiovisual motion detection. Cogn. Brain Res.19, 185–194. 10.1016/j.cogbrainres.2003.11.011
3
AlbrightT. D. (1984). Direction and orientation selectivity of neurons in visual area MT of the macaque. J. Neurophysiol.52, 1106–1130. 10.1152/jn.1984.52.6.1106
4
AlinkA.EulerF.GaleanoE.KrugliakA.SingerW.KohlerA. (2012a). Auditory motion capturing ambiguous visual motion. Front. Psychol.3, 1–8. 10.3389/fpsyg.2011.00391
5
AlinkA.EulerF.KriegeskorteN.SingerW.KohlerA. (2012b). Auditory motion direction encoding in auditory cortex and high-level visual cortex. Hum. Brain Mapp.33, 969–978. 10.1002/hbm.21263
6
AlinkA.SingerW.MuckliL. (2008). Capture of auditory motion by vision is represented by an activation shift from auditory to visual motion cortex. J. Neurosci.28, 2690–2697. 10.1523/jneurosci.2980-07.2008
7
AllmanJ. M.KaasJ. H. (1971). A representation of the visual field in the caudal third of the middle temporal gyrus of the owl monkey (Aotus trivirgatus). Brain Res.31, 85–105. 10.1016/0006-8993(71)90635-4
8
AngelakiD. E.GuY.DeAngelisG. C. (2009). Multisensory integration: psychophysics, neurophysiology and computation. Curr. Opin. Neurobiol.19, 452–458. 10.1016/j.conb.2009.06.008
9
BakerJ. F.PetersenS. E.NewsomeW. T.AllmanJ. M. (1981). Visual response properties of neurons in four extrastriate visual areas of the owl monkey (Aotus trivirgatus): a quantitative comparison of medial, dorsomedial, dorsolateral and middle temporal areas. J. Neurophysiol.45, 397–416. 10.1152/jn.1981.45.3.397
10
BarracloughN. E.XiaoD.BakerC. I.OramM. W.PerrettD. I. (2005). Integration of visual and auditory information by superior temporal sulcus neurons responsive to the sight of actions. J. Cogn. Neurosci.17, 377–391. 10.1162/0898929053279586
11
BaumannO.GreenleeM. W. (2007). Neural correlates of coherent audiovisual motion perception. Cereb. Cortex17, 1433–1443. 10.1093/cercor/bhl055
12
BaumgartF.Gaschler-MarkefskiB.WoldorffM. G.HeinzeH. J.ScheichH. (1999). A movement-sensitive area in auditory cortex. Nature400, 724–726. 10.1038/23385
13
BeauchampM. S. (2005). See me, hear me, touch me: multisensory integration in lateral occipital-temporal cortex. Curr. Opin. Neurobiol.15, 145–153. 10.1016/j.conb.2005.03.011
14
BeckJ. M.MaW. J.KianiR.HanksT.ChurchlandA. K.RoitmanJ. D.et al. (2008). Probabilistic population codes for Bayesian decision making. Neuron60, 1142–1152. 10.1016/j.neuron.2008.09.021
15
BeerA. L.RöderB. (2004). Unimodal and crossmodal effects of endogenous attention to visual and auditory motion. Cogn. Affect. Behav. Neurosci.4, 230–240. 10.3758/cabn.4.2.230
16
BeerA. L.RöderB. (2005). Attending to visual or auditory motion affects perception within and across modalities: an event-related potential study. Eur. J. Neurosci.21, 1116–1130. 10.1111/j.1460-9568.2005.03927.x
17
BizleyJ. K.CohenY. E. (2013). The what, where and how of auditory-object perception. Nat. Rev. Neurosci.14, 693–707. 10.1038/nrn3565
18
BizleyJ. K.JonesG. P.TownS. M. (2016). Where are multisensory signals combined for perceptual decision-making?Curr. Opin. Neurobiol.40, 31–37. 10.1016/j.conb.2016.06.003
19
BrangD.TaichZ. J.HillyardS. A.GraboweckyM.RamachandranV. S. (2013). Parietal connectivity mediates multisensory facilitation. Neuroimage78, 396–401. 10.1016/j.neuroimage.2013.04.047
20
BremmerF.SchlackA.ShahN. J.ZafirisO.KubischikM.HoffmannK.et al. (2001). Polymodal motion processing in posterior parietal and premotor cortex: a human fMRI study strongly implies equivalencies between humans and monkeys. Neuron29, 287–296. 10.1016/s0896-6273(01)00198-2
21
BrittenK. H.ShadlenM. N.NewsomeW. T.MovshonJ. A. (1992). The analysis of visual motion: a comparison of neuronal and psychophysical performance. J. Neurosci.12, 4745–4765. 10.1523/jneurosci.12-12-04745.1992
22
BrittenK. H.Van WezelR. J. A. (2002). Area MST and heading perception in macaque monkeys. Cereb. Cortex12, 692–701. 10.1093/cercor/12.7.692
23
BruceC.DesimoneR.GrossC. G. (1981). Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. J. Neurophysiol.46, 369–384. 10.1152/jn.1981.46.2.369
24
CelebriniS.NewsomeW. T. (1994). Neuronal and psychophysical sensitivity to motion signals in extrastriate area MST of the macaque monkey. J. Neurosci.14, 4109–4124. 10.1523/jneurosci.14-07-04109.1994
25
CelebriniS.NewsomeW. T. (1995). Microstimulation of extrastriate area MST influences performance on a direction discrimination task. J. Neurophysiol.73, 437–448. 10.1152/jn.1995.73.2.437
26
ChaplinT. A.AllittB. J.HaganM. A.PriceN. S. C.RajanR.RosaM. G. P.et al. (2017). Sensitivity of neurons in the middle temporal area of marmoset monkeys to random dot motion. J. Neurophysiol.118, 1567–1580. 10.1101/104117
27
ChaplinT. A.AllittB. J.HaganM. A.RosaM. G. P.RajanR.LuiL. L. (2018). Auditory motion does not modulate spiking activity in the middle temporal and medial superior temporal visual areas. Eur. J. Neurosci.48, 2013–2029. 10.1111/ejn.14071
28
CohenM. R.NewsomeW. T. (2009). Estimates of the contribution of single neurons to perception depend on timescale and noise correlation. J. Neurosci.29, 6635–6648. 10.1523/jneurosci.5179-08.2009
29
CookE. P.MaunsellJ. H. R. (2002). Dynamics of neuronal responses in macaque MT and VIP during motion detection. Nat. Neurosci.5, 985–994. 10.1038/nn924
30
CummingB. G.NienborgH. (2016). Feedforward and feedback sources of choice probability in neural population responses. Curr. Opin. Neurobiol.37, 126–132. 10.1016/j.conb.2016.01.009
31
DaviesA. J.ChaplinT. A.RosaM. G. P.YuH.-H. (2016). Natural motion trajectory enhances the coding of speed in primate extrastriate cortex. Sci. Rep.6:19739. 10.1038/srep19739
32
DesimoneR.ScheinS. J. (1987). Visual properties of neurons in area V4 of the macaque: sensitivity to stimulus form. J. Neurophysiol.57, 835–868. 10.1152/jn.1987.57.3.835
33
DesimoneR.UngerleiderL. G. (1986). Multiple visual areas in the caudal superior temporal sulcus of the macaque. J. Comp. Neurol.248, 164–189. 10.1002/cne.902480203
34
DrugowitschJ.DeAngelisG. C.KlierE. M.AngelakiD. E.PougetA. (2014). Optimal multisensory decision-making in a reaction-time task. Elife3:e03005. 10.7554/eLife.03005
35
DubnerR.ZekiS. M. (1971). Response properties and receptive fields of cells in an anatomically defined region of the superior temporal sulcus in the monkey. Brain Res.35, 528–532. 10.1016/0006-8993(71)90494-x
36
DucommunC. Y.MichelC. M.ClarkeS. (2004). Cortical motion deafness. Neuron43, 765–777. 10.1016/j.neuron.2004.08.020
37
DuffyC. J. (1998). MST neurons respond to optic flow and translational movement. J. Neurophysiol.80, 1816–1827. 10.1152/jn.1998.80.4.1816
38
DuffyC. J.WurtzR. H. (1991). Sensitivity of MST neurons to optic flow stimuli. II. Mechanisms of response selectivity revealed by small-field stimuli. J. Neurophysiol.65, 1346–1359. 10.1152/jn.1991.65.6.1346
39
EifukuS.WurtzR. H. (1998). Response to motion in extrastriate area MSTl: center-surround interactions. J. Neurophysiol.80, 282–296. 10.1152/jn.1998.80.1.282
40
ElstonG. N.RosaM. G. (1997). The occipitoparietal pathway of the macaque monkey: comparison of pyramidal cell morphology in layer III of functionally related cortical visual areas. Cereb. Cortex7, 432–452. 10.1093/cercor/7.5.432
41
ErnstM. O.BanksM. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature415, 429–433. 10.1038/415429a
42
FattoriP.PitzalisS.GallettiC. (2009). The cortical visual area V6 in macaque and human brains. J. Physiol. Paris103, 88–97. 10.1016/j.jphysparis.2009.05.012
43
FellemanD. J.Van EssenD. C. (1987). Receptive field properties of neurons in area V3 of macaque monkey extrastriate cortex. J. Neurophysiol.57, 889–920. 10.1152/jn.1987.57.4.889
44
FetschC. R.KianiR.NewsomeW. T.ShadlenM. N. (2014). Effects of cortical microstimulation on confidence in a perceptual decision. Neuron83, 797–804. 10.1016/j.neuron.2014.07.011
45
FetschC. R.TurnerA. H.DeAngelisG. C.AngelakiD. E. (2009). Dynamic reweighting of visual and vestibular cues during self-motion perception. J. Neurosci.29, 15601–15612. 10.1523/JNEUROSCI.2574-09.2009
46
FreemanE.DriverJ. (2008). Direction of visual apparent motion driven solely by timing of a static sound. Curr. Biol.18, 1262–1266. 10.1016/j.cub.2008.07.066
47
GrazianoM. S.AndersenR. A.SnowdenR. J. (1994). Tuning of MST neurons to spiral motions. J. Neurosci.14, 54–67. 10.1523/JNEUROSCI.14-01-00054.1994
48
GriffithsT. D.GreenG. G.ReesA.ReesG. (2000). Human brain areas involved in the analysis of auditory movement. Hum. Brain Mapp.9, 72–80. 10.1002/(sici)1097-0193(200002)9:2<72::aid-hbm2>3.0.co;2-9
49
GrotheB.PeckaM.McAlpineD. (2010). Mechanisms of sound localization in mammals. Physiol. Rev.90, 983–1012. 10.1152/physrev.00026.2009
50
GrunewaldA.LindenJ. F.AndersenR. A. (1999). Responses to auditory stimuli in macaque lateral intraparietal area I. Effects of training. J. Neurophysiol.82, 330–342. 10.1152/jn.1999.82.1.330
51
GuY.DeAngelisG. C.AngelakiD. E. (2007). A functional link between area MSTd and heading perception based on vestibular signals. Nat. Neurosci.10, 1038–1047. 10.1038/nn1935
52
GuY.AngelakiD. E.DeAngelisG. C. (2008). Neural correlates of multisensory cue integration in macaque MSTd. Nat. Neurosci.11, 1201–1210. 10.1038/nn.2191
53
HackettT. A.StepniewskaI.KaasJ. H. (1998). Subdivisions of auditory cortex and ipsilateral cortical connections of the parabelt auditory cortex in macaque monkeys. J. Comp. Neurol.394, 475–495. 10.1002/(sici)1096-9861(19980518)394:4<475::aid-cne6>3.0.co;2-z
54
HikosakaK.IwaiE.SaitoH.TanakaK. (1988). Polysensory properties of neurons in the anterior bank of the caudal superior temporal sulcus of the macaque monkey. J. Neurophysiol.60, 1615–1637. 10.1152/jn.1988.60.5.1615
55
HukA. C.DoughertyR. F.HeegerD. J. (2002). Retinotopy and functional subdivision of human areas MT and MST. J. Neurosci.22, 7195–7205. 10.1523/jneurosci.22-16-07195.2002
56
InghamN. J.HartH. C.McAlpineD. (2001). Spatial receptive fields of inferior colliculus neurons to auditory apparent motion in free field. J. Neurophysiol.85, 23–33. 10.1152/jn.2001.85.1.23
57
IrvineD. R.RajanR.AitkinL. M. (1996). Sensitivity to interaural intensity differences of neurons in primary auditory cortex of the cat. I. types of sensitivity and effects of variations in sound pressure level. J. Neurophysiol.75, 75–96. 10.1152/jn.1996.75.1.75
58
JiangF.SteckerG. C.FineI. (2014). Auditory motion processing after early blindness. J. Vis.14:4. 10.1167/14.13.4
59
KaasJ. H.HackettT. A.TramoM. J. (1999). Auditory processing in primate cerebral cortex. Curr. Opin. Neurobiol.9, 164–170. 10.1016/S0959-4388(99)80022-1
60
KafaligonulH.AlbrightT. D.StonerG. R. (2018). Auditory modulation of spiking activity and local field potentials in area MT does not appear to underlie an audiovisual temporal illusion. J. Neurophysiol.120, 1340–1355. 10.1152/jn.00835.2017
61
KafaligonulH.OlukC. (2015). Audiovisual associations alter the perception of low-level visual motion. Front. Integr. Neurosci.9:26. 10.3389/fnint.2015.00026
62
KafaligonulH.StonerG. R. (2012). Static sound timing alters sensitivity to low-level visual motion. J. Vis.12:2. 10.1167/12.11.2
63
KaminiarzA.SchlackA.HoffmannK.-P.LappeM.BremmerF. (2014). Visual selectivity for heading in the macaque ventral intraparietal area. J. Neurophysiol.112, 2470–2480. 10.1152/jn.00410.2014
64
KayserS. J.PhiliastidesM. G.KayserC. (2017). Sounds facilitate visual motion discrimination via the enhancement of late occipital visual representations. Neuroimage148, 31–41. 10.1016/j.neuroimage.2017.01.010
65
KeatingP.KingA. J. (2015). Sound localization in a changing world. Curr. Opin. Neurobiol.35, 35–43. 10.1016/j.conb.2015.06.005
66
KimR.PetersM. A.ShamsL. (2012). 0 + 1 > 1: how adding noninformative sound improves performance on a visual task. Psychol. Sci.23, 6–12. 10.1177/0956797611420662
67
KimJ. N.ShadlenM. N. (1999). Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque. Nat. Neurosci.2, 176–185. 10.1038/5739
68
KitagawaN.IchiharaS. (2002). Hearing visual motion in depth. Nature416, 172–174. 10.1038/416172a
69
KomatsuH.WurtzR. H. (1988a). Relation of cortical areas MT and MST to pursuit eye movements. I. Localization and visual properties of neurons. J. Neurophysiol.60, 580–603. 10.1152/jn.1988.60.2.580
70
KomatsuH.WurtzR. H. (1988b). Relation of cortical areas MT and MST to pursuit eye movements. III. Interaction with full-field visual stimulation. J. Neurophysiol.60, 621–644. 10.1152/jn.1988.60.2.621
71
KrumbholzK.Hewson-StoateN.SchönwiesnerM. (2007). Cortical response to auditory motion suggests an asymmetry in the reliance on inter-hemispheric connections between the left and right auditory cortices. J. Neurophysiol.97, 1649–1655. 10.1152/jn.00560.2006
72
KrumbholzK.SchönwiesnerM.RübsamenR.ZillesK.FinkG. R.Von CramonD. Y. (2005). Hierarchical processing of sound location and motion in the human brainstem and planum temporale. Eur. J. Neurosci.21, 230–238. 10.1111/j.1460-9568.2004.03836.x
73
KusmierekP.RauscheckerJ. P. (2014). Selectivity for space and time in early areas of the auditory dorsal stream in the rhesus monkey. J. Neurophysiol.111, 1671–1685. 10.1152/jn.00436.2013
74
LakatosP.KarmosG.MehtaA. D.UlbertI.SchroederC. E. (2008). Entrainment of neuronal oscillations as a mechanism of attentional selection. Science320, 110–113. 10.1126/science.1154735
75
LewaldJ.PetersS.CorballisM. C.HausmannM. (2009). Perception of stationary and moving sound following unilateral cortectomy. Neuropsychologia47, 962–971. 10.1016/j.neuropsychologia.2008.10.016
76
LewisJ. W.BeauchampM. S.DeYoeE. A. (2000). A comparison of visual and auditory motion processing in human cerebral cortex. Cereb. Cortex10, 873–888. 10.1093/cercor/10.9.873
77
LewisR.NoppeneyU. (2010). Audiovisual synchrony improves motion discrimination via enhanced connectivity between early visual and auditory areas. J. Neurosci.30, 12329–12339. 10.1523/JNEUROSCI.5745-09.2010
78
LewisJ. W.Van EssenD. C. (2000). Corticocortical connections of visual, sensorimotor and multimodal processing areas in the parietal lobe of the macaque monkey. J. Comp. Neurol.428, 112–137. 10.1002/1096-9861(20001204)428:1<112::aid-cne8>3.0.co;2-9
79
LiP.ZhuS.ChenM.HanC.XuH.HuJ.et al. (2013). A motion direction preference map in monkey V4. Neuron78, 376–388. 10.1016/j.neuron.2013.02.024
80
LindenJ. F.GrunewaldA.AndersenR. A.ModulationI. I. B. (1999). Responses to auditory stimuli in macaque lateral intraparietal area II. Behavioral modulation. J. Neurophysiol.82, 343–358. 10.1152/jn.1999.82.1.343
81
LuiL. L.BourneJ. A.RosaM. G. P. (2005). Single-unit responses to kinetic stimuli in New World monkey area V2: physiological characteristics of cue-invariant neurones. Exp. Brain Res.162, 100–108. 10.1007/s00221-004-2113-9
82
LuiL. L.BourneJ. A.RosaM. G. P. (2006). Functional response properties of neurons in the dorsomedial visual area of New World monkeys (Callithrix jacchus). Cereb. Cortex16, 162–177. 10.1093/cercor/bhi094
83
LuiL. L.MokriY.ReserD. H.RosaM. G. P.RajanR. (2015). Responses of neurons in the marmoset primary auditory cortex to interaural level differences: comparison of pure tones and vocalizations. Front. Neurosci.9:132. 10.3389/fnins.2015.00132
84
MaW. J.BeckJ. M.LathamP. E.PougetA. (2006). Bayesian inference with probabilistic population codes. Nat. Neurosci.9, 1432–1438. 10.1038/nn1790
85
MajkaP.RosaM. G. P.BaiS.ChanJ. M.HuoB.-X.JermakowN.et al. (2018). Unidirectional monosynaptic connections from auditory areas to the primary visual cortex in the marmoset monkey. Brain Struct. Funct. [Epub ahead of print]. 10.1007/s00429-018-1764-4
86
MastertonB.DiamondI. T.HarrisonJ. M.BeecherM. D. (1967). Medial superior olive and sound localization. Science155, 1696–1697. 10.1126/science.155.3770.1696-a
87
MaunsellJ. H. R.Van EssenD. C. (1983a). Functional properties of neurons in middle temporal visual area of the macaque monkey. I. Selectivity for stimulus direction, speed, and orientation. J. Neurophysiol.49, 1127–1147. 10.1152/jn.1983.49.5.1127
88
MaunsellJ. H. R.Van EssenD. C. (1983b). Functional properties of neurons in middle temporal visual area of the macaque monkey. II. Binocular interactions and sensitivity to binocular disparity. J. Neurophysiol.49, 1148–1167. 10.1152/jn.1983.49.5.1148
89
McAlpineD.JiangD.ShackletonT. M.PalmerA. R. (2000). Responses of neurons in the inferior colliculus to dynamic interaural phase cues: evidence for a mechanism of binaural adaptation. J. Neurophysiol.83, 1356–1365. 10.1152/jn.2000.83.3.1356
90
MeyerG. F.GreenleeM.WuergerS. M. (2011). Interactions between auditory and visual semantic stimulus classes: evidence for common processing networks for speech and body actions. J. Cogn. Neurosci.23, 2291–2308. 10.1162/jocn.2010.21593
91
MeyerG. F.WuergerS. M. (2001). Cross-modal integration of auditory and visual motion signals. Neuroreport12, 2557–2560. 10.1097/00001756-200108080-00053
92
MiddlebrooksJ.GreenD. (1991). Sound localization by human listeners. Annu. Rev. Psychol.42, 135–159. 10.1146/annurev.psych.42.1.135
93
MillerL. M.RecanzoneG. H. (2009). Populations of auditory cortical neurons can accurately encode acoustic space across stimulus intensity. Proc. Natl. Acad. Sci. U S A106, 5931–5935. 10.1073/pnas.0901023106
94
MineaultP. J.KhawajaF. A.ButtsD. A.PackC. C. (2012). Hierarchical processing of complex motion along the primate dorsal visual pathway. Proc. Natl. Acad. Sci. U S A109, E972–E980. 10.1073/pnas.1115685109
95
MokriY.WorlandK.FordM.RajanR. (2015). Effect of background noise on neuronal coding of interaural level difference cues in rat inferior colliculus. Eur. J. Neurosci.42, 1685–1704. 10.1111/ejn.12914
96
MooreD. R. (1991). Anatomy and physiology of binaural hearing. Audiology30, 125–134. 10.3109/00206099109072878
97
NahornaO.BerthommierF.SchwartzJ.-L. (2012). Binding and unbinding the auditory and visual streams in the McGurk effect. J. Acoust. Soc. Am.132, 1061–1077. 10.1121/1.4728187
98
NahornaO.BerthommierF.SchwartzJ.-L. (2015). Audio-visual speech scene analysis: characterization of the dynamics of unbinding and rebinding the McGurk effect. J. Acoust. Soc. Am.137, 362–377. 10.1121/1.4904536
99
NewsomeW. T.BrittenK. H.MovshonJ. A. (1989). Neuronal correlates of a perceptual decision. Nature341, 52–54. 10.1038/341052a0
100
NewsomeW. T.ParéE. B. (1988). A selective impairment of motion perception following lesions of the middle temporal visual area (MT). J. Neurosci.8, 2201–2211. 10.1523/jneurosci.08-06-02201.1988
101
NicholsM. J.NewsomeW. T. (2002). Middle temporal visual area microstimulation influences veridical judgments of motion direction. J. Neurosci.22, 9530–9540. 10.1523/jneurosci.22-21-09530.2002
102
OramM. W.PerrettD. I. (1994). Responses of anterior superior temporal polysensory (STPa) neurons to “biological motion” stimuli. J. Cogn. Neurosci.6, 99–116. 10.1162/jocn.1994.6.2.99
103
OrbanG. A. (2008). Higher order visual processing in macaque extrastriate cortex. Physiol. Rev.88, 59–89. 10.1152/physrev.00008.2007
104
OrbanG. A.KennedyH.BullierJ. (1986). Velocity sensitivity and direction selectivity of neurons in areas V1 and V2 of the monkey: influence of eccentricity. J. Neurophysiol.56, 462–480. 10.1152/jn.1986.56.2.462
105
OrbanG. A.SaundersR. C.VandenbusscheE. (1995). Lesions of the superior temporal cortical motion areas impair speed discrimination in the macaque monkey. Eur. J. Neurosci.7, 2261–2276. 10.1111/j.1460-9568.1995.tb00647.x
106
PadbergJ.SeltzerB.CusickC. G. (2003). Architectonics and cortical connections of the upper bank of the superior temporal sulcus in the rhesus monkey: an analysis in the tangential plane. J. Comp. Neurol.467, 418–434. 10.1002/cne.10932
107
PalmerS. M.RosaM. G. P. (2006). A distinct anatomical network of cortical areas for analysis of motion in far peripheral vision. Eur. J. Neurosci.24, 2389–2405. 10.1111/j.1460-9568.2006.05113.x
108
PasternakT.MeriganW. H. (1994). Motion perception following lesions of the superior temporal sulcus in the monkey. Cereb. Cortex4, 247–259. 10.1093/cercor/4.3.247
109
PavaniF.MacalusoE.WarrenJ.DriverJ.GriffithsT. (2002). A common cortical substrate activated by horizontal and vertical sound movement in the human brain. Curr. Biol.12, 1584–1590. 10.1016/s0960-9822(02)01143-0
110
PerrottD. R.CostantinoB.CisnerosJ. (1993). Auditory and visual localization performance in a sequential task. J. Acoust. Soc. Am.93, 2134–2138. 10.1121/1.406675
111
PerrottD. R.MusicantA. D. (1977). Minimum auditory movement angle: binaural localization of moving sound sources. J. Acoust. Soc. Am.62, 1463–1466. 10.1121/1.381675
112
PoirierC.BaumannS.DheerendraP.JolyO.HunterD.BalezeauF.et al. (2017). Auditory motion-specific mechanisms in the primate brain. PLoS Biol.15:e2001379. 10.1371/journal.pbio.2001379
113
PoirierC.CollignonO.DeVolderA. G.RenierL.VanlierdeA.TranduyD.et al. (2005). Specific activation of the V5 brain area by auditory motion processing: an fMRI study. Cogn. Brain Res.25, 650–658. 10.1016/j.cogbrainres.2005.08.015
114
RajanR.AitkinL. M.IrvineD. R. (1990a). Azimuthal sensitivity of neurons in primary auditory cortex of cats. II. Organization along frequency-band strips. J. Neurophysiol.64, 888–902. 10.1152/jn.1990.64.3.888
115
RajanR.AitkinL. M.IrvineD. R.McKayJ. (1990b). Azimuthal sensitivity of neurons in primary auditory cortex of cats. I. Types of sensitivity and the effects of variations in stimulus parameters. J. Neurophysiol.64, 872–887. 10.1152/jn.1990.64.3.872
116
RecanzoneG. H.GuardD. C.PhanM. L.SuT. K. (2000). Correlation between the activity of single auditory cortical neurons and sound-localization behavior in the macaque monkey. J. Neurophysiol.83, 2723–2739. 10.1152/jn.2000.83.5.2723
117
ReserD. H.BurmanK. J.YuH.-H.ChaplinT. A.RichardsonK. E.WorthyK. H.et al. (2013). Contrasting patterns of cortical input to architectural subdivisions of the area 8 complex: a retrograde tracing study in marmoset monkeys. Cereb. Cortex23, 1901–1922. 10.1093/cercor/bhs177
118
RohdeM.van DamL. C. J.ErnstM. O. (2016). Statistically optimal multisensory cue integration: a practical tutorial. Multisens. Res.29, 279–317. 10.1163/22134808-00002510
119
RoitmanJ. D.ShadlenM. N. (2002). Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. J. Neurosci.22, 9475–9489. 10.1523/jneurosci.22-21-09475.2002
120
RomanskiL. M. (2007). Representation and integration of auditory and visual stimuli in the primate ventral lateral prefrontal cortex. Cereb. Cortex17, i61–i69. 10.1093/cercor/bhm099
121
RomanskiL. M. (2012). Integration of faces and vocalizations in ventral prefrontal cortex: implications for the evolution of audiovisual speech. Proc. Natl. Acad. Sci. U S A109, 10717–10724. 10.1073/pnas.1204335109
122
RomanskiL. M.BatesJ. F.Goldman-RakicP. S. (1999a). Auditory belt and parabelt projections to the prefrontal cortex in the rhesus monkey. J. Comp. Neurol.403, 141–157. 10.1002/(sici)1096-9861(19990111)403:2<141::aid-cne1>3.0.co;2-v
123
RomanskiL. M.TianB.FritzJ.MishkinM.Goldman-RakicP. S.RauscheckerJ. P. (1999b). Dual streams of auditory afferents target multiple domains in the primate prefrontal cortex. Nat. Neurosci.2, 1131–1136. 10.1038/16056
124
RudolphK.PasternakT. (1999). Transient and permanent deficits in motion perception after lesions of cortical areas MT and MST in the macaque monkey. Cereb. Cortex9, 90–100. 10.1093/cercor/9.1.90
125
SaitoH. A.YukieM.TanakaK.HikosakaK.FukadaY.IwaiE. (1986). Integration of direction signals of image motion in the superior temporal sulcus of the macaque monkey. J. Neurosci.6, 145–157. 10.1523/jneurosci.06-01-00145.1986
126
SalzmanC. D.NewsomeW. T. (1994). Neural mechanisms for forming a perceptual decision. Science264, 231–237. 10.1126/science.8146653
127
SchenkT.ZihlJ. (1997). Visual motion perception after brain damage: I. Deficits in global motion perception. Neuropsychologia35, 1289–1297. 10.1016/s0028-3932(97)00004-3
128
SchlackA.Sterbing-D’AngeloS. J.HartungK.HoffmannK.-P.BremmerF. (2005). Multisensory space representations in the macaque ventral intraparietal area. J. Neurosci.25, 4616–4625. 10.1523/JNEUROSCI.0455-05.2005
129
SeitzA. R.KimR.ShamsL. (2006). Sound facilitates visual learning. Curr. Biol.16, 1422–1427. 10.1016/j.cub.2006.05.048
130
SekulerR.SekulerA. B.LauR. (1997). Sound alters visual motion perception. Nature385:308. 10.1038/385308a0
131
SempleM. N.KitzesL. M. (1993a). Binaural processing of sound pressure level in cat primary auditory cortex: evidence for a representation based on absolute levels rather than interaural level differences. J. Neurophysiol.69, 449–461. 10.1152/jn.1993.69.2.449
132
SempleM. N.KitzesL. M. (1993b). Focal selectivity for binaural sound pressure level in cat primary auditory cortex: two-way intensity network tuning. J. Neurophysiol.69, 462–473. 10.1152/jn.1993.69.2.462
133
SleeS. J.YoungE. D. (2010). Sound localization cues in the marmoset monkey. Hear. Res.260, 96–108. 10.1016/j.heares.2009.12.001
134
SmithK. R.OkadaK.SaberiK.HickokG. (2004). Human cortical auditory motion areas are not motion selective. Neuroreport15, 1523–1526. 10.1097/01.wnr.0000130233.43788.4b
135
SmithK. R.SaberiK.HickokG. (2007). An event-related fMRI study of auditory motion perception: no evidence for a specialized cortical system. Brain Res.1150, 94–99. 10.1016/j.brainres.2007.03.003
136
Soto-FaracoS.SpenceC.KingstoneA. (2005). Assessing automaticity in the audiovisual integration of motion. Acta Psychol.118, 71–92. 10.1016/j.actpsy.2004.10.008
137
StrnadL.PeelenM. V.BednyM.CaramazzaA. (2013). Multivoxel pattern analysis reveals auditory motion information in MT+ of both congenitally blind and sighted individuals. PLoS One8:e63198. 10.1371/journal.pone.0063198
138
TanakaK.SaitoH. A. (1989). Analysis of motion of the visual field by direction, expansion/contraction, and rotation cells clustered in the dorsal part of the medial superior temporal area of the macaque monkey. J. Neurophysiol.62, 626–641. 10.1152/jn.1989.62.3.626
139
ThalerL.PacioccoJ.DaleyM.LesniakG. D.PurcellD. W.FraserJ. A.et al. (2016). A selective impairment of perception of sound motion direction in peripheral space: a case study. Neuropsychologia80, 79–89. 10.1016/j.neuropsychologia.2015.11.008
140
TianB.ReserD.DurhamA.KustovA.RauscheckerJ. P. (2001). Functional specialization in rhesus monkey auditory cortex. Science292, 290–293. 10.1126/science.1058911
141
Van EssenD. C. (2002). Surface-based atlases of cerebellar cortex in the human, macaque, and mouse. Ann. N Y Acad. Sci.978, 468–479. 10.1111/j.1749-6632.2002.tb07588.x
142
Van EssenD. C.DierkerD. (2007). Surface-based and probabilistic atlases of primate cerebral cortex. Neuron56, 209–225. 10.1016/j.neuron.2007.10.015
143
von SaldernS.NoppeneyU. (2013). Sensory and striatal areas integrate auditory and visual signals into behavioral benefits during motion discrimination. J. Neurosci.33, 8841–8849. 10.1523/jneurosci.3020-12.2013
144
WangY.CelebriniS.TrotterY.BaroneP. (2008). Visuo-auditory interactions in the primary visual cortex of the behaving monkey: electrophysiological evidence. BMC Neurosci.9:79. 10.1186/1471-2202-9-79
145
WarrenJ.ZielinskiB.GreenG. (2002). Perception of sound-source motion by the human brain. Neuron34, 139–148. 10.1016/s0896-6273(02)00637-2
146
WatanabeJ.IwaiE. (1991). Neuronal activity in visual, auditory and polysensory areas in the monkey temporal cortex during visual fixation task. Brain Res. Bull.26, 583–592. 10.1016/0361-9230(91)90099-6
147
WilsonW. W.O’NeillW. E. (1998). Auditory motion induces directionally dependent receptive field shifts in inferior colliculus neurons. J. Neurophysiol.79, 2040–2062. 10.1152/jn.1998.79.4.2040
148
WoodsT. M.LopezS. E.LongJ. H.RahmanJ. E.RecanzoneG. H. (2006). Effects of stimulus azimuth and intensity on the single-neuron activity in the auditory cortex of the alert macaque monkey. J. Neurophysiol.96, 3323–3337. 10.1152/jn.00392.2006
149
WuergerS. M.HofbauerM.MeyerG. F. (2003). The integration of auditory and visual motion signals at threshold. Percept. Psychophys.65, 1188–1196. 10.3758/bf03194844
150
WuergerS. M.ParkesL.LewisP. A.Crocker-BuqueA.RutschmannR.MeyerG. F. (2012). Premotor cortex is sensitive to auditory-visual congruence for biological motion. J. Cogn. Neurosci.24, 575–587. 10.1162/jocn_a_00173
151
YuH.-H.RosaM. G. P. (2014). Uniformity and diversity of response properties of neurons in the primary visual cortex: selectivity for orientation, direction of motion and stimulus size from center to far periphery. Vis. Neurosci.31, 85–98. 10.1017/s0952523813000448
152
YuH.-H.VermaR.YangY.TibballsH. A.LuiL. L.ReserD. H.et al. (2010). Spatial and temporal frequency tuning in striate cortex: functional uniformity and specializations related to receptive field eccentricity. Eur. J. Neurosci.31, 1043–1062. 10.1111/j.1460-9568.2010.07118.x
153
ZaksasD.PasternakT. (2006). Directional signals in the prefrontal cortex and in area MT during a working memory for visual motion task. J. Neurosci.26, 11726–11742. 10.1523/jneurosci.3420-06.2006
154
ZekiS. M.WatsonJ. D.LueckC. J.FristonK. J.KennardC.FrackowiakR. S. (1991). A direct demonstration of functional specialization in human visual cortex. J. Neurosci.11, 641–649. 10.1523/jneurosci.11-03-00641.1991
155
ZihlJ.von CramonD.MaiN. (1983). Selective disturbance of movement vision after bilateral brain damage. Brain106, 313–340. 10.1093/brain/106.2.313
Summary
Keywords
visual motion, auditory motion, audiovisual integration, primates, cerebral cortex
Citation
Chaplin TA, Rosa MGP and Lui LL (2018) Auditory and Visual Motion Processing and Integration in the Primate Cerebral Cortex. Front. Neural Circuits 12:93. doi: 10.3389/fncir.2018.00093
Received
03 August 2018
Accepted
08 October 2018
Published
26 October 2018
Volume
12 - 2018
Edited by
Greg Stuart, Australian National University, Australia
Reviewed by
Sophie Wuerger, University of Liverpool, United Kingdom; Hulusi Kafaligonul, Bilkent University, Turkey
Updates
Copyright
© 2018 Chaplin, Rosa and Lui.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Tristan A. Chaplin tristan.chaplin@monash.edu Leo L. Lui leo.lui@monash.edu
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.