Attention to the color of a moving stimulus modulates motion-signal processing in macaque area MT: evidence for a unified attentional system

Visual attention is the mechanism that selectively modulates sensory processing according to behavioral relevance. Attention can be directed to a location in space, to a non-spatial stimulus feature, like a specific color or motion direction, or to an entire object.

Numerous single-unit studies in awake behaving primates have documented neural correlates of spatial and feature-based attention in various areas of visual cortex (reviewed in Desimone and Duncan, 1995 ; Treue, 2001 ; Reynolds and Chelazzi, 2004 ; Maunsell and Treue, 2006 ). The neural correlate of spatial attention consists of an increase in firing rates if attention is directed to the stimulus inside the receptive field (RF) of the recorded neuron, as opposed to somewhere else. The neural correlate of feature-based attention consists of an increase in firing rates if the attended feature matches the preferences of the neuron under study, independent of the spatial focus of attention (note that the term ‘feature’ refers to a particular property within a given stimulus dimension, e.g., upwards motion is a feature within the stimulus dimension of motion, and blue is a feature within the stimulus dimension of color). Only few single-unit studies have investigated neural correlates of object-based attention (Roelfsema et al., 1998 ; Fallah et al., 2007 ; Wannig et al., 2007 ).

While it is well established that attention modulates firing rates of individual neurons, there is substantial variability in the magnitude of these modulations. The range of attentional effects varies between cortical areas and between neurons within the same cortical area. The reasons for this variability are not well understood. A potential source of variability could be the extent to which the tuning properties of the recorded neurons can represent the stimulus properties relevant for the behavioral task (Maunsell, 2004 ). Accordingly, neurons with tuning properties optimally suited to solve the task at hand should show the strongest attentional effects.

We investigated the role of the perceptual task on attentional effects by studying extracellular responses of individual neurons in area MT of two macaque monkeys. The monkeys were trained to attend either to the color or to the direction of motion of a random dot pattern (RDP). MT neurons are highly selective for processing of motion direction and speed, rather than color (Ungerleider and Mishkin, 1982 ; Felleman and Van Essen, 1991 ). Therefore, if attentional effects were strongest for optimally suited neurons, we would expect a stronger modulation of firing rates when attention was directed to the motion signal as compared to when it was directed to the color.

We found that the modulation of firing rates by spatial attention and feature-based attention did not depend on whether the monkeys were attending to the motion or the color signal. We conclude that, under our task conditions, the degree to which the properties of the recorded neurons match the perceptual task at hand is not a likely explanation for the variability in the size of attentional effects.

All experimental procedures were approved by the regional animal welfare office and complied with relevant laws and institutional guidelines. Headpost and recording chamber were implanted using standard surgical techniques (Martínez-Trujillo and Treue, 2004 ).

Two monkeys (Macaca mulatta) were trained to attend either to the motion signal (direction task) or to the color (color task) of a moving RDP (Figure 1 A). To start a trial, the animal had to maintain fixation within a window of 1.25° radius, centered on a fixation square (0.2° × 0.2°). 150 ms after touching a lever, a cue appeared for 500 ms, signaling the position of the relevant stimulus and indicating whether a direction or a color task had to be performed. After the cue had disappeared, two moving colored RDPs were presented at equal eccentricity in opposite visual hemifields, one of them inside the classical RF of the neuron under study. To ensure that the monkeys were correctly attending to the cued feature (target feature) of the cued stimulus (target location), one or two of the following events could happen within the next 500–3550 ms, randomized in time and order (Figure 1 B): the target feature event happened at the target location, the target feature event happened at the uncued location (location distractor), the uncued feature event happened at the target location (dimension distractor), or the uncued feature event happened at the uncued location (dimension and location distractor).

In the direction task, the monkeys received a liquid reward for releasing the lever within a response time window of 100–500 ms following the target event, i.e., a brief change in the direction of motion of the cued stimulus. In contrast, in the color task the monkeys were rewarded for responding to a brief change in the color of the cued stimulus. Trials were ended immediately following any response. The first change in the stimulus was a target event with a probability of 0.5. Given the first change was a distractor, the conditional probability for the second change of being a target event was also 0.5. If only distractor events occurred, the monkeys were required to continue depressing the lever and were rewarded at the end of the trial (4050 ms after cue onset). While distractor events at the wrong location allowed us to control that the monkeys attended to the target location, distractor events in the wrong dimension ensured that they selectively attended to the target feature. Trials were terminated without reward if the monkeys responded to any of the distractor types, did not respond to the target feature at the target location, or broke fixation (i.e., moved their gaze outside the fixation window). Color and direction tasks were performed in separate blocks of trials, alternating every 20 correctly completed trials.

Monkeys were seated in a primate chair with their head restrained at a distance of 57 cm from a computer monitor (resolution 40 pixels per degree of visual angle, refresh rate 76 Hz). The eye position was monitored with a high-speed video-based eye tracker at a sampling rate of 230 Hz (ET49, Thomas Recording, Giessen, Germany). The stimuli were RDPs moving within a stationary virtual aperture. A single dot subtended 0.1° of visual angle and the dot density was 8 dots/deg². The size of the RDP, the speed of the dots, and the direction of motion were matched to the properties of the neuron under study. On a given trial, each RDP was independently assigned one of two neuronally isoluminant colors (blue vs. yellow), and one of two motion directions (preferred vs. null direction of the recorded neuron). In case of a direction change, all dots simultaneously changed their direction by 30–60°, depending on eccentricity and speed of the target stimulus. For color changes, 80% of all dots changed their color to either yellow or blue, depending on the original stimulus color. All changes lasted for 132 ms before the original stimulus properties were restored. The cues consisted of small RDPs subtending 0.75° of visual angle, with a dot size of 0.075° and a density of 40 dots/deg². They were always presented at a distance of 2° from fixation, positioned on a virtual line connecting the fixation point to the target location. For the direction cue, dots were achromatic and moved at a speed of 3°/s in the direction which had to be detected. In the color task, the dots of the cue were stationary and were plotted in the to-be-detected color.

Single-unit activity was recorded from area MT using a five-channel multi-electrode recording system (Mini-Matrix, Thomas Recording, and Plexon data acquisition system, Plexon Inc.). For most of the recording sessions, five electrodes were simultaneously advanced to isolate individual MT neurons with overlapping RFs (linear electrode arrangement, 305 �m interelectrode spacing). Cells were characterized as MT neurons based on their directional tuning, RF location, and position in the cortex. The locations and sizes of individual RFs were mapped manually using a moving bar. Direction and speed tuning were determined by presenting a single RDP inside the joint RF, moving in 12 different directions at each of 8 different speeds (0.5–64°/s), while the monkeys were maintaining fixation. The preferred direction was estimated by fitting a Gaussian to the individual neuronal responses, plotted as a function of stimulus direction, separately for each speed. Out of the 58 recorded neurons, 23 were obtained from parallel recordings (either from two, three, or four cells simultaneously). For these groups of cells, the stimulus for the subsequent experiment was optimized for the neuron exhibiting the strongest direction selectivity. Neuronal isoluminance was established by presenting a preferred-direction RDP inside the RF (3.7–5.7 s) that changed its color every 500 ms, while the monkey maintained fixation. Colors were randomly selected from a sample consisting of a single blue of fixed intensity (45 cd/m²) and various intensities of yellow, bracketing the level of intensity for blue (31–63 cd/m²). Selecting the appropriate level of intensity for yellow ensured that both colors provided equally strong inputs for individual MT neurons. We chose to present preferred/non-preferred directions of motion and neuronally isoluminant colors to ensure that individual neurons were well suited to contribute to the direction task but not the color task.

For all cells included in the analysis of neuronal data, responses to the preferred direction were at least three times as large as responses to the null direction. For any given recording session, only those blocks of trials were included for which the analysis of behavior ensured that the monkeys were following the attentional instructions (Figure 2 ). Finally, individual trials were only included if they were correctly completed, and for these trials neuronal data were only analyzed until the first change in either of the two stimuli occurred, be it a target or distractor. The latter restriction ensures that the conditions we compared were identical in terms of sensory stimulation. To obtain reliable estimates of single neuron average firing rates, a sufficient number of correct repetitions were collected for each neuron in all comparisons (minimum = 5; mean = 21.75). All firing rates plotted represent values of the spike density function at steps of 15 ms, which were obtained by convolving spike trains with a Gaussian kernel (σ = 30 ms). For the main comparisons attentional effects were quantified by computing an attentional index, defined as the difference in firing rates between two conditions, divided by their sum, after subtraction of the spontaneous firing rate. For plotting single neuron data and population activity, neuronal responses were normalized by the peak response to a preferred-direction stimulus (averaged across colors) obtained in the direction task with attention inside the RF. To assess statistical significance of the attentional effects one-sample or paired t-test (always two-tailed) were performed on the raw indices described in the corresponding figure captions, irrespective of the binning into histograms. All of the comparisons also reached significance if the corresponding non-parametric Wilcoxon-test was used instead.

In separate blocks of trials, the monkeys were cued to attend either to the motion direction or to the color in one of two moving RDPs (target) to perform a direction or a color task, respectively. In the direction task, the monkeys had to detect a brief change in the direction of motion of the target. In contrast, in the color task, the monkeys were required to detect a brief change in the color (Figure 1 A). To ensure that the monkeys attended to the cued feature of the target, we used different types of distractors, randomized in time and order (Figure 1 B). For instance, in the direction task, the target could briefly change color, or the distractor stimulus could briefly change direction of motion, or color, or both. The monkeys only received a juice reward if they responded to the cued feature of the target, and were not rewarded if they responded to any of the distractor types.

Examination of the behavioral performance revealed that both monkeys reliably followed the attentional instructions given by the cue. The comparison of neuronal activity between the direction and color task critically requires correct attentional deployment. Therefore, we examined the monkeys’ behavioral performance in every single block of trials for each recording session. Specifically, we calculated the cumulative probability F(x) of achieving at least the number of correct trials x simply by chance, given the total number of trials in that block. Under the null hypothesis, the monkeys were guessing as to whether they should be responding to a color or a direction change (binomial test with chance probability p of success = 0.5). Figure 2 A illustrates this approach for one of our recording sessions. For all further analyses, we included only those individual blocks of trials, for which the behavioral performance was reliably different from chance level (1−F(x) < 0.05). For the session shown in Figure 2 A, we excluded blocks 11 and 16 from all further analysis. Averaged across the remaining blocks of this session, the animal detected 99.8 and 91.96% of the direction and the color targets, respectively. Reaction times (RTs) were 334 ms for direction targets, and 342 ms for color targets. The animal successfully ignored 85.43% of the distractors in the direction task, and 82.6% of the distractors in the color task. Across all recording sessions, we excluded 17% (61 out of 354) of all blocks obtained from Monkey 1, and 5% (29 out of 552) of those obtained from Monkey 2 (Figure 2 B). The mean RTs for targets and the mean percentage of detected targets and successfully ignored distractors are shown in Figure 2 C.

Target-related performance was evaluated with 2 × 2 ANOVAs, separately for RTs (Figure 2 C, left panel) and percentage of detected targets (middle panel), involving the between-subjects factor animal (monkey 1 vs. monkey 2) and the within-subjects factor type of task (direction task vs. color task). Overall, monkey 2 responded faster than monkey 1 (351 vs. 392 ms, p < 0.0001), but there was no difference in mean RTs between the tasks (p = 0.84), and no interaction between animal and type of task (p = 0.25). Concerning target detection rates, overall performance was slightly better in the color task (91.06%) than in the direction task (88.65%, p = 0.015), but neither the main effect of animal (p = 0.14), nor the interaction between animal and type of task (p = 0.27) reached statistical significance.

Distractor-related performance was evaluated with a 2 × 2 × 3 ANOVA on percentages of ignored distractors (Figure 2 C, right panel), involving the between-subjects factor animal (monkey 1 vs. monkey 2), and the within-subjects factors type of task (direction task vs. color task), and type of distractor (dimension vs. dimension and location vs. location). When assessing effects of type of distractor, the resulting p-values were adjusted for violations of the sphericity assumption using the Greenhouse-Geyser correction. The ANOVA revealed a significant main effect of animal, indicating that, overall, monkey 2 was better in ignoring distractors than monkey 1 (91.2 vs. 85.07%). There was no main effect of type of task (p = 0.67), and none of the interactions involving the factor type of task reached statistical significance (p > 0.86). However, the main effect of type of distractor was significant (p < 0.0001). Post-hoc analyses confirmed that distractors in the wrong location were easier to ignore than distractors involving changes in the wrong dimension. The effect of type of distractor also depended on the animal (p < 0.0001 for the interaction term), with monkey 1 showing a more pronounced difference in percentages than monkey 2 (95.84% vs. 73.52% for monkey 1, and 97.4% vs. 85.07% for monkey 2).

In summary, monkey 2 showed an overall advantage in mean RT to targets, and was less influenced by the different types of distractors. Yet, these differences did not depend on which task was being performed, and the overall pattern of results was very similar for the two monkeys. They performed both attentional tasks with a high level of accuracy, suggesting that they were selectively attending to the cued feature of the cued stimulus.

Effects of spatial attention on firing rates of individual neurons in area MT do not depend on whether the monkeys perform a direction or a color task (Figure 3 ). We examined the modulation of firing rates caused by directing attention to the stimulus outside vs. inside the RF, separately for the direction (Figure 3 A, dotted traces) and the color task (solid traces). Although MT neurons are highly selective for processing of motion direction and speed, rather than color, we found strong attentional effects in both the direction task (Figure 3 B, white histogram, p < 0.0001, one-sample t-test) and the color task (gray histogram, p < 0.0001, one-sample t-test). In the direction task, attention enhanced processing of the motion signal by 18.8% (95%-confidence interval: 12–25.6%). Remarkably, attending to the color of a moving stimulus also enhanced processing of the irrelevant motion signal by 18.3% (95%-confidence interval: 11.7–24.9%). This effect was statistically reliable for individual monkeys (p = 0.02 with n = 15 cells for monkey 1, and p < 0.0001 with n = 43 cells for monkey 2). Across the population of recorded neurons, there was no difference in the size of the attentional effect between the tasks (p = 0.86, paired t-test), and these effects were positively correlated (Figure 3 C, Pearson’s r = 0.58, p < 0.0001). Furthermore, with attention directed to the stimulus inside the RF, average activity in the direction task (Figure 3 A, dotted red trace) was not different from average activity in the color task (solid red trace) (p = 0.89, paired t-test). Taken together, these results indicate that modulation of firing rates of MT neurons by spatial attention does not require a motion-related task.

Effects of feature-based attention on firing rates of individual neurons in area MT do not depend on whether the monkeys perform a direction or a color task (Figure 4 ). To dissociate the spatial focus of attention from the object driving the neurons under study, we examined conditions in which the monkeys’ attention was always directed to the stimulus outside the RF. Here, we compared neuronal activity between conditions in which the monkeys’ attention was directed either to the motion or the color of a preferred- vs. null-direction stimulus outside the RF, while the stimulus inside always moved in the preferred direction of the neuron. In the direction task, attending to the preferred (Figure 4 A, dotted red trace) vs. null direction (dotted black trace) outside the RF increased responses to the irrelevant stimulus in the RF, replicating the well-known effect of feature-based attention (Treue and Martínez-Trujillo, 1999 ; Martínez-Trujillo and Treue, 2004 ; Maunsell and Treue, 2006 ). Across our population, feature-based attention enhanced activity by 11.8% (95%-confidence interval: 5.7–18.3%) in the direction task (white histogram, p = 0.0002, one-sample t-test). Most remarkably, firing rates were also higher when attention was directed to the color of a preferred- (Figure 4 A, solid red trace) vs. null-direction stimulus (solid black trace) outside the RF. Here, activity was enhanced by 9.4% across our population (95%-confidence interval: 3.8–15.3%, gray histogram, p = 0.0012, one-sample t-test). This effect was present in both monkeys individually (p = 0.039 with n = 15 in monkey 1, and p = 0.0062 with n = 43 in monkey 2). Again, the attentional effects were not different between the color and direction task (p = 0.44, paired t-test), and their size was positively correlated (Figure 4 C, Pearson’s r = 0.48, p = 0.0001). Thus, the results in the color task demonstrate enhanced processing of irrelevant motion signals outside the spatial focus of attention. Since spatial attention remains constant between conditions, and is directed to a location distant from the RF, space-based modulations cannot account for these effects.

Spatial and feature-based mechanisms of attention modulate firing rates of MT neurons independent of the match between the tuning properties of the recorded neurons and the perceptual task at hand. We recorded extracellular activity from individual neurons in the motion-selective area MT of monkeys that were instructed to attend either to the color or the motion direction of a moving stimulus. Despite the fact that MT neurons show poor tuning for color, we found that directing attention to the color of the moving stimulus created an attentional modulation very similar to the one observed when the monkeys’ attention was directed to the motion of the stimulus.

Our design and analysis excludes the possibility that the attentional modulation in the color task reflects a feature-based effect of color. While it is generally assumed that color and direction of motion are processed in anatomically distinct and functionally separate visual processing streams, several neurophysiological studies have shown that information about color is indeed available to the visual motion processing system (Dobkins and Albright, 1994 ; Gegenfurtner et al., 1994 ; Croner and Albright, 1999 ; Seidemann et al., 1999 ; Thiele et al., 2001 ; Nassi et al., 2006 ). Given such evidence, we want to emphasize that our findings do not reflect a feature-based effect of color. First, our measurements were conducted under neuronal isoluminance conditions by presenting pairs of yellow and blue RDPs that provided equally strong input for individual MT cells (see Materials and Methods). Second, even in the absence of isoluminance, any feature-based effect of color would cancel out since we always averaged across the two colors. Thus, feature-based effects of color do not contribute to our results.

A potential concern is the possibility that our results could reflect a strategy of divided attention. More specifically, one could argue that we observed attentional modulations of MT neurons in the color task because the monkeys attended to the direction of motion as well, and only at post-perceptual stages decided whether or not to respond. In such a case, MT activity in the color task would simply reflect well-known direction-specific attentional effects. We consider this an unlikely explanation, for the following two reasons. First, our behavioral results indicate that the monkeys used the information provided by the cue. Both monkeys reliably responded to changes in the target feature and were able to ignore various kinds of distractors. These observations suggest that the monkeys did not attend to different features simultaneously. Second, data collected in one of our stimulus conditions directly argue against a strategy of divided attention: Attending to the color of a null-direction stimulus outside the RF increased responses if the preferred-direction stimulus inside the RF had the same color, compared to when the two colors where different (n = 57, p = 0.0015, one-sample t-test). Importantly, this modulation was specific for the color task: Whether the two stimuli had the same or different colors did not matter in the direction task (n = 57, p = 0.5, one-sample t-test). If the monkeys always divided their attention between color and direction of motion, one would expect this modulation to be the same in the two tasks. Therefore, a strategy of divided attention cannot account for this pattern of results.

Our study allows us to constrain potential explanations for the variability in the size of attentional effects. It is commonly observed that attentional modulations of single neurons strongly vary in magnitude, between different areas but also between neurons within the same area. Most likely, there is no single explanation for this observation, and few studies have so far addressed this issue. Data obtained from cortical areas V4 (Mitchell et al., 2007 ) and V1 (Chen et al., 2008 ) indicate that attentional effects can strongly depend on the type of cell being recorded. It has also been suggested that the variability in the magnitude of attentional effects could to some extent reflect the match between the properties of the recorded neuron and the challenges of the behavioral task (Maunsell, 2004 ). Our data provide a direct test of this hypothesis. They show that attentional modulation of motion-signal processing in area MT is unaffected by directing attention to the color of a moving stimulus. Since MT neurons are poorly tuned for color, the match between the recorded neuron and the challenges of the perceptual task does not seem to play a major role in determining the magnitude of attentional modulation.

Our findings are inconsistent with a simple notion of task-dependent modulation, according to which attentional modulation affects only those cortical areas specialized to process the cued feature (Corbetta et al., 1990 ; Beauchamp et al., 1997 ; Huk and Heeger, 2000 ; Schoenfeld et al., 2007 ). Directly related to the current study are recent single-unit data obtained from area V4 (Mirabella et al., 2007 ). In this experiment, the monkeys were instructed to discriminate either the orientation or the color of a bar presented inside the RF. Mirabella et al. (2007) reported that the activity of a number of cells was modulated by the perceptual task: Responses to a given stimulus depended on whether the monkeys were attending to its orientation or its color. We think that this apparent discrepancy most likely is related to the differences between task designs. In their task, four orientations and four colors were used. Two of each were assigned to one behavioral response, and the remaining orientations and colors to an alternative response. Such a stimulus-response mapping strongly encourages preferential processing of the cued dimension and active suppression of distracting information. Indeed, Mirabella et al. (2007) found the strongest task-dependent modulation if the orientation and the color of the stimulus called for opposite and therefore conflicting responses. We didn’t have such a conflicting situation, and we found no task-dependent modulation. Instead, our single-unit data are in agreement with recent fMRI results (Buracas et al., 2005 ) demonstrating that the influence of perceptual task on neuronal activity in a given cortical area might be less exclusive than previously assumed.

Our findings are in agreement with predictions made by theories of object-based attention. These theories predict that attention directed to a single feature of an object will transfer to other features of the same object, thereby enhancing the processing of even those features that are currently irrelevant (e.g., O’Craven et al., 1999 ; Blaser et al., 2000 ). Data from both our experiments are consistent with these predictions. First, we found that attending to the color of a moving stimulus inside the RF results in the same level of activity as attending to its motion signal (Figure 3 A, red traces). Consistent with theories of object-based attention this could indicate a transfer of attention, from color to the task-irrelevant direction, within the same object. Yet, an alternative explanation based on spatial attention cannot be ruled out for this comparison: the observed modulation could reflect an increase in gain due to the allocation of spatial attention inside the RF, independent of the feature attended within the ‘spotlight of attention’. However, spatial attention cannot account for the results of our second experiment. Here, responses to a motion stimulus are also modulated if attention is directed to the color of a preferred- vs. null-direction stimulus at a remote location (Figure 4 A, solid traces). As spatial attention is constantly directed outside the RF, these results indicate a transfer of attention from relevant to irrelevant features, within an attended object and across the visual field. Indeed, recent behavioral and fMRI studies have demonstrated that attentional modulation of motion signal processing across the visual field does not require a motion-related task (Melcher et al., 2005 ; Sohn et al., 2005 ; Boynton et al., 2006 ). In this sense, our findings extend the results provided by the few single-unit studies investigating object-based attentional effects (Roelfsema et al., 1998 ; Fallah et al., 2007 ; Wannig et al., 2007 ).

The transfer of attention from relevant to irrelevant stimulus features, within an attended object and across the visual field, could contribute to an integrated saliency map, which specifically enhances the representation of the attended object (Treue, 2003 ; Maunsell and Treue, 2006 ; Treue and Katzner, 2009 ). By extending the modulation to features that are not directly attended but simply associated with the attended object a more sophisticated, multi-dimensional modulation profile across visual space can be formed. In this integrated saliency map, the gain change applied to the inputs at a given location would reflect the overlap between all features of an attended object and the features present at the given location. As a consequence, the highest attentional gain would apply to those locations in the integrated saliency map that contain objects most similar to the attended object, without any need to explicitly invoke a sophisticated object-recognition system (Kaping et al., 2007 ).

We conclude that, in a simple detection situation, directing attention to a single feature of a visual object can modulate firing rates in areas that are poorly tuned for the attended feature. Our data are consistent with theories of object-based attention postulating a transfer of attention, from relevant to irrelevant features, within the attended object and across the visual field. Furthermore, while sensory processing in visual cortex is characterized by a diversification into areas with responses that are highly selective for specific stimulus properties, our data provide evidence for an attentional system that is much more unified – as it modulates responses to a given stimulus in all areas, independent of the attended stimulus property. Such a modulation could result in an integrated saliency map specifically enhancing the representation of the attended object.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

We thank Dirk Prüße, Leonore Burchardt, and Ralf Brockhausen for excellent technical support. This work was supported by the German Ministry for Education and Science Grant BMBF 01GQ0433 to the Bernstein Center for Computational Neuroscience, Göttingen.