Influence of task-specific instructions on cross-modal sensory interactions
-
1
University of Toronto, Department of Computer Science, Canada
-
2
University of Edinburgh, United Kingdom
-
3
Army Research Lab, United States
We analyze and model subjects’ behavior when localizing an auditory cue in the presence of a visual distractor. In this paradigm, the ventriloquist illusion has been reliably demonstrated: subjects are biased towards the visual stimulus. Strong biases are typically accompanied by a perception of unity wherein the stimuli appear to originate from a common source. We probe whether and how different task instructions can affect perceptual biases and uncertainty. Subjects are modeled as observers performing inference on a generative model of the stimuli. The visual and auditory cues result under the model either from a cause at a single location or from two independent locations. The Bayes-optimal inference strategy combines predictions from the two discrete causal hypotheses weighted by the belief in each of them. An alternative strategy first selects the most probable hypothesis and bases subsequent inferences on that hypothesis alone. These two strategies make qualitatively different predictions (Natarajan et al., 2009). Recent studies that have utilized the above model to explain perceptual biases (Koerding and Tenenbaum, 2006; Koerding et al., 2007; Stocker and Simoncelli, 2008) have reached different conclusions with regards to the inference strategy that subjects appear to use. We analyze data from 3 new behavioral experiments that have the same ventriloquist paradigm but differ only in task instructions: participants are required to perform auditory localization alone, localization followed by a unity judgment or to report unity perception alone. We perform a comparative analysis of the data from the three experiments, and fit our model of subject behavior to the data. The dominant account of the ventriloquism effect is that target localization is an early sensory process that is largely immune to top-down influences (Vroomen and deGelder, 2004). Our results challenge this notion — we show that subjects’ localization bias depends strongly on whether or not they are instructed to make unity judgments. Our computational model suggests that task-specific instructions change both subjects’ expectations and their inference strategy. Subjects are described well by Bayesian model averaging when instructed only to localize; when reporting unity percepts alongside localization, subjects use only the model that best fits their unity judgments. The only fitted model parameter value that differed significantly across experiments was the prior belief that stimuli will be unified. When just localizing a target, subjects strongly believed stimuli would be unified. This belief diminished when also asked to report on unity, and subjects instructed to make unity judgments without localization had very low unity probability. Our results suggest the existence of top-down influences on sensory processing; the task-specific unity priors suggest that subjects have strong task-related expectations regarding stimuli. Our model’s task-specific inference strategies can be understood as the result of limited computational resources in the brain. Instructing the subjects to make unity judgements forces them to select a model. When computation is limited it seems reasonable to use that model selection for future inferences, maintaining one consistent explanation of the observations. We hope that these results contribute to our understanding of stimulus representations and models of hierarchical sensory processing.
Conference:
Computational and Systems Neuroscience 2010, Salt Lake City, UT, United States, 25 Feb - 2 Mar, 2010.
Presentation Type:
Oral Presentation
Topic:
Oral presentations
Citation:
Natarajan
R,
Zemel
R,
Murray
I and
Hairston
D
(2010). Influence of task-specific instructions on cross-modal sensory interactions.
Front. Neurosci.
Conference Abstract:
Computational and Systems Neuroscience 2010.
doi: 10.3389/conf.fnins.2010.03.00015
Copyright:
The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers.
They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.
The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.
Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.
For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.
Received:
17 Feb 2010;
Published Online:
17 Feb 2010.
*
Correspondence:
Rama Natarajan, University of Toronto, Department of Computer Science, Toronto, Canada, rama@cs.toronto.edu