A flicker change detection task reveals object-in-scene memory across species
- 1 Department of Biology, Centre for Vision Research, York University, Toronto ON
- 2 Department of Psychology, York University, Toronto ON
- 3 Neuroscience Graduate Diploma Program, York University, Toronto ON
- 4 Rotman Research Institute, Baycrest, Toronto, ON, Canada
- 5 Department of Psychology, University of Toronto, Toronto, ON, Canada
Tests of recognition memory in macaques typically assay memory for objects or isolated images, over time spans of seconds to hours from stimulus presentation, and/or require extensive training. Here, we propose a new application of the flicker change detection task that could measure object-in-scene memory days after single-trial exposures. In three experiments, participants searched for a changing object – or “target” – embedded within a scene as their eye movements were tracked. For new targets-in-scenes, the change is difficult to detect and requires extensive search. Once the target is found, however, the change becomes obvious. We reasoned that the decreased times required to find a target in a repeated scene would indicate memory for the target. In humans, targets were found faster when the targets-and-scenes were explicitly remembered than when they were forgotten, or had never been seen before. This led to faster repeated-trial compared to novel-trial search times. Based solely on repeated-trial search times, we were able to select distributions comprised of predominantly remembered or predominantly forgotten trials. Macaques exhibited the same repetition effects as humans, suggesting that remembered trials could be dissociated from novel or forgotten trials using the same procedures we established in humans. Finally, an anterograde amnesic patient with damage that included the medial temporal lobe (MTL) showed no search time differences, suggesting that memory revealed through search times on this task requires MTL integrity. Together, these findings indicate that the time required to locate a changing object reveals object-in-scene memory over long retention intervals in humans and macaques.
Introduction
Memories are organized into several distinct processes. A long-held view is that amnesic patients who show impairments in long-term memory for facts and events that they can consciously recall (explicit/declarative memory) also show intact long-term memory for habits/patterns without conscious awareness (implicit/non-declarative memory) and have intact short-term memory (see Squire et al., 1993 for review). According to this view, the hippocampus, along with surrounding medial temporal lobe (MTL) structures, has been implicated in the formation of explicit/declarative memories, whereas procedural and implicit memories are generally thought to be mediated by non-MTL structures (Zola-Morgan and Squire, 1993). An alternate account posits that the hippocampus supports associations among multiple items, termed relational memory, which may constitute an important prerequisite for episodic/declarative memory but which could also operate without awareness and on short retention intervals (Ryan and Cohen, 2003; Konkel and Cohen, 2009). An additional distinction may be that the hippocampus plays a time-limited role in memory processing (Squire et al., 1984, but see Nadel and Moscovitch, 1997). Cutting across these accounts, the hippocampus is thought to be important for the formation of long-term relational memory, even when learned over a single exposure, or “episode.”
The unique functions of the hippocampus among other MTL structures can be evaluated with the greatest precision using direct measures, such as selective lesions or intracranial electrophysiology involving animal models. One challenge with the use of animals is the selection of appropriate memory tasks. In macaques, the delayed non-matching/matching to sample [D(N)MS] task is the most commonly used measure of object recognition memory to assess MTL function (Gaffan, 1974; Mishkin and Delacour, 1975; Mahut et al., 1982; Bachevalier et al., 1985; Zola-Morgan and Squire, 1985). Subtle changes in delay period activity across experiments can lead to disparate results (Nemanic et al., 2004), and the use of extensive training and motivational incentives can lead to variability in rule learning on this task (see reviews: Winters et al., 2008; Clark and Squire, 2010). The visual-paired comparison (VPC) and visual preferential looking (VPL) tasks are also widely used measures of recognition memory. These tasks take advantage of the natural tendency to look longer at novel images, circumventing issues of training and differential reinforcement (starting in infants: Fagan, 1970; for review Rose et al., 2007; and later in monkeys: Gunderson and Sackett, 1984; Buffalo et al., 1999; Pascalis and Bachevalier, 1999; Zola et al., 2000; Pascalis and de Haan, 2003; Nemanic et al., 2004; Jutras et al., 2009). VPC tasks typically test object recognition (item) memory in the absence of memory for spatial contexts, and may therefore recruit a distinct set of neural structures, with the role of the hippocampus across species still in debate (Winters et al., 2008; Clark and Squire, 2010). Thus, one-trial DNMS and VPC tests may be well-suited for measuring object recognition memory within a session, but they are difficult to adapt to – and may require different neural substrates from – tests that include objects in context, particularly when they involve one-trial memory with spans over days. When spatial or “relational” object arrays have been used with a VPC design, monkeys were tested at delays of only seconds (Bachevalier and Nemanic, 2008). Even when retention intervals were increased to days, there is concern that some “spatial” variants of these tasks may be accomplished based on a (MTL-independent) egocentric reference frame rather than requiring allocentric information of spatial relationships among objects (Banta Lavenex and Lavenex, 2009).
Macaques have successfully learned spatial or object-in-context tasks that use single-trial exposures, but only at short delays (Parkinson et al., 1988; Angeli et al., 1993; Malkova and Mishkin, 2003). They can also perform spatial tasks at longer delays but require repeated learning trials (and explicit reinforcement; Gaffan, 1994; Hampton et al., 2005). Finally, some studies have used spatial/relational tasks that include both repetition and within-day testing (Murray et al., 1998; Hampton et al., 2004; Banta Lavenex et al., 2006). At least in macaques, there are no standard tests of memory across days, and following single exposures.
In humans, relational or object-in-scene memory has been assessed after single exposures using a change detection task, which is thought to require long-term memory (Ryan and Cohen, 2004). In this paradigm, an object embedded in a scene is removed, added, or shifted from its earlier presentation(s), causing increased gaze toward the location of the manipulation on the screen (Ryan et al., 2000; Ryan and Cohen, 2004; Smith et al., 2006; Smith and Squire, 2008). Amnesic patients tested over longer retention intervals with intervening trials during the retention interval (Ryan et al., 2000) or with damage restricted to the hippocampus (Smith et al., 2006) fail to notice the change. If the initial viewing of the scene could be made interesting enough to macaques, their gaze toward a changed object might indicate memory for objects in scene contexts. For this, the flicker change detection task may be advantageous, though to our knowledge it has not yet been used as a test of memory.
In the present study, our aim was to evaluate whether eye movements during a flicker change detection task could be used as a measure of object-in-scene memory in macaques. In flicker change detection, the presentation of an original scene alternates with that of an object-manipulated scene, with a brief blank screen inserted in between that provides a visual disruption (Rensink et al., 1997). Multiple alternations and extensive searching are typically required before the change is detected, due to the so-called “change blindness” effect (Simons and Levin, 1997). Once found, however, the change becomes obvious, attracting attention (and gaze). Memory of the change may prevent the “blindness” characteristic of novel-image viewing, quickly capturing attention (and gaze) at the location of the change. Thus, in this variant of the change detection task, both learning and test stimuli have identical presentation characteristics and all trials share the same behavioral goal: to locate the change. The difference is that a remembered change should lead to rapid detection (i.e., visual fixation).
To test this hypothesis, in the first experiment, human participants viewed novel or repeated change detection trials, and reported whether they remembered the changing object or not. If search times for remembered items are significantly faster than search times for forgotten items, we would conclude that search times can be used as a measure of memory. To determine if we could identify remembered or forgotten trials based solely on search times, we divided the distribution of search times into “fast” and “slow” groups. For each group, we determined the proportion of trials that were reported as remembered or forgotten.
In the second experiment, four rhesus macaques performed the same change detection memory task. If monkeys show the same shift in search times predicted in humans – specifically, decreased search times for repeated trials – we could expect to apply our search time thresholds to reliably differentiate between remembered and forgotten images in macaques.
In a third experiment, we addressed whether object-in-scene memory is dependent upon MTL integrity. Behavioral data from an amnesic patient (D. A.) with bilateral MTL damage were compared to those of age-matched controls.
The present study is a novel test of object-in-scene memory. It can be used to test memory over days, and is of interest when tasks requiring navigation are undesirable, on theoretical grounds (e.g., testing relational memory without navigation), or due to experimental limitations (e.g., during electrophysiological recordings or in populations with limited mobility).
Materials and Methods
Experiment 1: Human Participants
Subjects
Twelve York University students [five males, ages 19–26 years, mean (SD) age 22.42 (1.88) years] volunteered to take part in the study. Subjects had normal or corrected-to-normal vision. Written informed consent was obtained, and the study was conducted in accordance with the ethical guidelines set by the York Human Participants Review Subcommittee.
Stimuli
Images for change blindness trials were selected from a wide variety of natural scenes, including wildlife, city, rural, and indoor scenes. Target objects were modified in location, color, and presence (comprising 18.89, 3.89, and 77.22% of all trials) using Adobe Photoshop. Target location (quadrant on screen), size, and category (animate/inanimate) were balanced across trials in a session, to reduce the chance of biases in the search strategies of participants. Sample stimuli are shown in Figure 1A.
Figure 1. Stimuli and experimental paradigm. (A) Sample stimuli with target objects including a disappearing plant (left image, near top of image), a color change (pink to blue on building), and a shifted object (man near tires in sand). (B) Trial sequence on training and testing sessions. During testing sessions, an additional verbal report screen was shown after each trial.
Behavioral procedure
Three sessions were held on different days to test for memory of the changing objects. Each session was divided into three blocks of 20 images each. During each block, the participant’s head was positioned 51 cm away from a 38-cm by 30-cm screen and was placed on a chinrest to minimize head movements. Eye tracking data were collected using iView X infrared eye tracking system (SensoMotoric Instruments, SMI, Berlin, Germany). Before the change blindness trials began, eye gaze was calibrated using a 13-point system. Briefly, calibration provided the pupil and corneal reflectance values when gaze was directed toward 13 known locations. Through a proprietary algorithm, the iView system then interpolates the observed pupil and corneal reflectance values to produce an estimate of the point of regard (i.e., gaze location) anywhere on the screen. The iView system sent gaze information to our stimulus presentation software, Presentation (NeuroBehavioral Systems, CA, USA). This allowed the online detection of fixation locations and durations at various places on the screen, including fixations on the target object. The selection and timing of visual stimuli were also controlled in Presentation.
Training session
During the initial training session (day 1), only novel stimuli were shown. An example of a changing image was given prior to the start of the experimental trials. Each trial consisted of the original image alternating with its modified image for 500 ms each, with a gray screen intervening for 50 ms between each alternation (Figure 1B). These intervals maximized the time with the scene displayed, while staying within the range of times commonly seen in flicker change detection tasks. Once the participant fixated on the object for 1.2 s or 60 s had passed, the changing object was made visible by alternating the original and modified image with no intervening gray screen.
Test sessions
There were two follow-up testing sessions that occurred 1 day and approximately 1 month later (day 2 and day 30, respectively). Half of the original images were shown on day 2 and the other half were shown on day 30, with each block consisting of half novel and half repeated images presented in random order. The procedure was otherwise similar to the initial training session, with images comprised of half animate and inanimate targets, balanced across the novel and repeated images. After each test session trial, a screen was shown asking participants: “Had you seen this picture before?” and “If so, did you remember which object was changing?” to which verbal yes/no responses were given and recorded. If participants responded yes to both questions, the image was classified as remembered; if they answered yes to the first question (scene was familiar) and no to the second (target item forgotten), it was classified as forgotten (familiar); if they answered no to remembering both the scene and target, it was classified as forgotten (unfamiliar). Participants never answered no to the first question and yes to the second, hence no additional category was required. Some participants distinguished between whether they recalled where the change occurred or what object changed; in response, they were instructed to answer yes to the second question if they remembered either the position or identity of the target. To examine search time differences in remembering the identity or location of the target object, for the last three participants, the second question was replaced with: “If so, did you remember where the object was? Did you remember what the object was?” If participants answered yes to at least one of these questions, the trial was classified as remembered.
Data analysis
Fixation data and search times were analyzed using MATLAB. The search time required to locate the target object was calculated as the time from the first fixation after the trial started to the time when the subject fixated on the target object for 1.2 s. Occasionally, the eye tracking was inaccurate whether due to calibration errors or non-linearities in the signal (e.g., from the curvature of the eye), leading to 1 or 2 discarded trials per 120 trials (1: N = 5 subjects; 2: N = 1 participant). Search times were grouped according to novel or repeated-trial types. Search times from repeated trials were further classified as remembered or forgotten based on subjects’ verbal reports. For novel trials, images were either correctly identified as novel or incorrectly recognized as remembered or forgotten (i.e., false positives). Median search times were compared statistically across categories (novel/remembered/forgotten) using a repeated measures one-way ANOVA. All trials, including trials in which the target was not found, were included. If the ANOVA was significant at an alpha of 0.05, planned comparisons were calculated for repetition effects: novel v. repeated; for implicit memory effects: novel v. forgotten (unfamiliar); familiarity effects: novel v. forgotten (familiar); and for explicit memory effects: remembered v. forgotten (familiar/unfamiliar).
Experiment 2: Macaques
Subjects and stimuli
Four female rhesus macaques were tested (three aged 3 years, weighing 4–5 kg; one aged 8 years, weighing 10 kg), with one macaque completing an additional run using new stimulus sets. Procedures were approved by the York University Animal Care Committee and were performed in accordance with the Canadian Council on Animal Care. Stimuli were made in the same manner as described for human participants. The macaques received juice reward for fixating a large blinking object presented in random locations on a screen that had a white background. As training progressed, smaller objects were used and the length of fixation required to obtain juice was progressively lengthened from 0.5 to 1.2 s. Next, they learned to fixate on large, salient flickering objects within simple natural scenes (for example, a person in the foreground of a sparse mountain scene). All four macaques completed these “easy embedded” trials within the first few exposures, and upon completion of a set of 20 easy trials, they advanced to scenes with increasing complexity, interspersed with previously seen familiar scenes to reinforce appropriate looking behavior. Total embedded-image training time for each of the four macaques spanned 1–2 weeks with a daily session of cooperative behavior involving 3–6 sets of 20 trials per set. Testing began the Thursday after they completed the necessary number of novel trials whose difficulty was similar to those used in this study.
Eye tracking data recordings
Eye tracking data was collected and visual stimuli were presented in the same manner as was done with human participants; however, during the experiment, macaques sat within a chair apparatus in a closed cubicle and were able to freely move their head. During calibration, fixation on each calibration point was rewarded with juice or a preferred food reward (controlled manually) through a spout. The macaques rapidly learned (1–5 sessions, minimum 13 calibration trials) to hold their head in a position that placed their eyes within the tracker camera’s view, as juice was only delivered when the eye could be tracked. As before, viewing was binocular but only the left eye position was tracked.
Behavioral procedure
The experimental task used was identical to the one shown in Figure 2, except that no verbal report screen was shown. Three sessions were held: the training session took place on day 1, and two follow-up testing sessions took place on days 2 and 4 (1 and 3 days after the training session, respectively). The second testing session took place 4 days later, as opposed to 1 month later as in humans to avoid interference from other experiments with visual stimuli in which the macaques were involved during that 1 month period. Two sets of 20 images were used per session. Juice was given as a reward after each trial once the macaque fixated on the target for either 0.75 or 1 s (the required fixation duration was decreased for two macaques who tended to refixate outside the target at 1 s). These threshold values are near the human dwell times toward manipulated objects (Ryan and Cohen, 2004), and lie far outside of these macaques’ dwell times during scanning of these images (99th percentile for each of the 4 macaques, data not shown), making it highly unlikely that the macaques would unintentionally “trigger” the fixation threshold within the target window. Macaques had a maximum time of 60 s to find the changing target, after which it was made visible by presenting the original and modified images without an interleaving gray screen, and no juice was given.
Figure 2. Scanpath examples of (A) aremembered trial from a human participant in Experiment 1, and (B) a short search time trial from a macaque in Experiment 2, based on the individually derived search time threshold. Search times are shown in the lower left corner of each image, with consecutive fixations connected by a pink line; target objects are indicated by yellow boxes, none of which were displayed during a trial.
Eye tracking data and search time analysis
As with the human participants’ data, fixation data and search times were analyzed using MATLAB. The search time required to locate the target object was defined as the time of initial fixation on the screen after the trial started to the time of fixation on the target, lasting for at least 0.75 or 1 s. Whereas humans were instructed to look at the screen and behaved accordingly, macaques occasionally looked away from the screen; thus, a few additional adjustments were made. For trials where macaques were not viewing the screen for at least 50% of the trial, the data were discarded as a misbehave trial and were not included in the calculations. In total, 27 out of 400 trials during the test sessions across four macaque subjects (80/80/80/160 test trials presented to the respective monkeys); individual number of misbehave/total trials were 15, 10, 1, and 1. In addition, for the one macaque subject that completed the additional experimental run, a “lookaway” contingency was added to the stimulus presentation code. Whenever the left eye was determined to be looking either outside of the screen window or if no eye signal was present at all (i.e., if the macaque was misbehaving by choosing not to look at the screen at all), the monitor went blank and the experiment put on pause until the eye signal returned to the screen. Hence, the total looking time could be kept at a constant maximum time of 60 s, and no “misbehave” trials would have to be thrown out. Data analysis compared the median search times of novel versus repeated trials, as well as between human and macaque search times in those respective categories.
Experiment 3: Amnesic Patient D. A. and Controls
Subjects and stimuli
D. A., a 58-year-old, right-handed amnesic person with 17 years of education participated in this study. He suffered extensive bilateral MTL damage that included his hippocampus and perirhinal, entorhinal, and parahippocampal cortices, due to contracting viral encephalitis in 1993. His more extensive right-sided damage extends to his anterior temporal cortex. Volume loss was also observed in other right hemispheric regions, including the posterior temporal, ventral frontal, and occipital regions, as well as the anterior cingulate. A detailed description of this case is provided in Rosenbaum et al. (2008). D. A.’s performance was compared to that of five male, right-handed controls ages 48–56 years with no known history of neurological or psychiatric disorders [average age 51.80 (2.86) years, education level 19.60 (6.65) years mean (SD)]. All participants had normal or corrected-to-normal vision and were paid $10/hour for their time. Written informed consent was obtained, and the study was conducted in accordance with the ethical guidelines set by the York Human Participants Review Subcommittee. The stimuli were identical to those used in Experiment 1, except that only a subset of 40 images of the original 60 images per testing session were used.
Behavioral procedure
As in Experiments 1 and 2, there were three sessions in total: the first follow-up testing session was held after the initial training session following a 5-min break, and the second testing session took place 24 h later. Participants were required to fixate on the target for at least 1 s and had up to 40 s to find the changing object, after which the object was revealed. Because we were concerned about floor effects for memory in our older-adult control participants and in D. A., we reduced the retention delays and trial duration before revealing the change. Based on the search times from Experiment 1, it was clear that 1 s was well beyond the fixation durations of our participants, therefore there was no concern for accidentally triggering a “found” location. It was also clear that 40 s was already at asymptotic levels for finding the target, and reducing trial duration would allow participants to be retested more quickly after learning. Note that in this experiment we are not making comparisons between young and old adults, but between D. A. and his control group, all of whom experienced the same trial delays and contingencies.
As was done for three participants in Experiment 1, the test questions appearing after each trial were: “Had you seen this picture before?” and “If so, did you remember where the object was? Did you remember what the object was?” to clarify differences between memory for location compared to identity of objects. If participants responded yes to knowing either what and/or where the target object was, the image was classified as remembered. On testing sessions, the length of the verbal report screen was not fixed to give participants extra time to report their answers as needed, and the experimenter initiated the next trial by pressing a key when participants were ready. In all aspects other than those just described, the procedure was identical to Experiment 1.
Data analysis
Of the 80 trials completed by the 5 control participants, one trial was removed for three of them and two trials for a fourth, due to tracker error. For D. A., only one trial of 80 was discarded. Search time corrections and data analyses were performed as described in Experiment 1, with the exception that D. A.’s search times were compared with those of the controls using a modified t test procedure that treats an individual as a sample, for comparison with a small normative sample that is treated as statistics rather than as population parameters, using the following formula (Crawford and Howell, 1998):
where X1 is the individual’s score, X2 is the mean score in the control sample, s2 is the SD of scores in the control sample, and N2 is the number of people in the control sample. This method determines if an individual’s score is significantly lower than the control sample, and estimates the percentage of a control population that would score below the individual.
Additional testing session
Due to the fact that D. A. expressed no explicit recollection of any of the images shown in the experiment just described, an additional testing session was conducted 1 month later to see if any recall could be observed with fewer trials and over shorter delays (i.e., within seconds to minutes of initial presentation). Five sets of 12 images were shown. In each set, the first 4 images were novel, and the next 8 images were composed of half novel and half repeated (from the first 4 images) stimuli. Search times of novel and repeated images were pooled together and analyzed as described above.
Predicting Explicit Recall from Search Times
We sought to test whether search times could be used to predict if a target was remembered or forgotten. Given the results from the previous experiments, search times that were shorter than a specified short search time threshold should typically belong to trials with remembered targets; search times that were longer than a specified long search time threshold should typically belong to trials with forgotten targets. Accuracy of the short search time threshold was indicated by the percentage of trials under threshold in which the target was reported as explicitly remembered (remembered trials) compared to the target not being recalled (forgotten trials). Accuracy of the long search time threshold was indicated by the percentage of trials above threshold that were forgotten compared to those that were remembered. The short search time threshold was calculated from the distribution of repeated-trial search times, at the inflection point after the first mode, for all participants in Experiments 1 and 3. To determine the inflection point, smoothing was conducted on the discrete search time data in MATLAB (“ksdensity”) using a Gaussian kernel of optimized bandwidth, similar to smoothing operations used for single unit data (spike-density functions) and imaging data. This operation provides a continuous probability estimate from discrete data, permitting the inflection point calculation. The long search time threshold was set to 30 s for all participants, which was well beyond the first mode for repeated trials and thus unlikely to include remembered trials [note that we found similar predictive ratios when using a range of long search time thresholds, from 20 to 40 s (data not shown)].
Some changing targets could, in principle, be more easily detected than others, leading to rapid novel and repeated search times, irrespective of memory. When a trial’s initial presentation led to very rapid detection (defined as search times under each individual’s repeated median search time [mean (SD) median search time: 6.45 (2.89) s; mean (SD) number discarded: 7(5) trials], the repeated-trial data were discarded from this analysis. This helped to mitigate ceiling effects whereby a repeated target would be located rapidly due to some type of inherent salience and not memory.
Examination of each individual’s responses revealed that only one participant had very few remembered trials, presumably leading to a search time distribution that did not display the characteristic skewed repeated search time mode compared to that of novel search times. Consequently, this participant’s short search time threshold (32.83 s) was 2 SD above the mean of the log-transformed data, and exceeded the long search time threshold. Based on these factors (i.e., insufficient number of remembered trials, overlapping novel and repeated search time distributions, and a prohibitively long search time threshold), this participant was excluded from the accuracy proportions reported.
For macaques in Experiment 2, search time thresholds were ascertained using the inflection point of the kernel smoothed distributions, as described for human participants, and compared to data from humans.
Results
Experiment 1: Human Participants
Eye scan paths
Example scan paths of a remembered trial are displayed in Figure 2A.
Behavioral results
For all participants, during the training and two testing sessions, 83 (3.7), 91 (2.8), and 90 (3.1)% [mean (SD)] of targets were found prior to the 60-s time limit, respectively. Participants showed forgetting between the two testing sessions; the number of remembered items decreased over the two testing sessions [day 2: 16.42 (6.27) trials, day 30: 10.50 (6.97) trials; t(11) = 2.84, p < 0.05], and the number of forgotten (unfamiliar) items increased [day 2: 3.75 (3.72) trials, day 30: 8.58 (6.65) trials; t(11) = −3.38, p < 0.01]. No trend was observed for forgotten (familiar) items [day 2: 8.92 (4.96) trials, day 30: 10.58 (7.35) trials; t(11) = −0.92, p > 0.30]. Falsely reported remembered and forgotten (familiar) trials were rare and in some cases absent [false-positive remembered, day 2: 0.33 (0.65) trials, day 30: 1.00 (0.95) trials; false-positive forgotten (familiar), day 2: 2.00 (2.52) trials, day 30: 2.75 (2.01) trials].
The proportion of found targets was greater for repeated trials than for novel trials [χ2 (1, n = 705) = 101.79, p < 0.00001] regardless of reported memory [remembered: χ2 (1, n = 323) = 73.16, p < 0.00001; forgotten (familiar): χ2 (1, n = 234) = 21.80, p < 0.00001; and forgotten (unfamiliar): χ2 (1, n = 148) = 12.34, p < 0.001]. Thus, even when the scenes and targets were forgotten, successful searches were more common in repeated than novel trials.
Search time differences were also evident across memory category (Figure 3). The median search times of all novel, remembered, and forgotten trials did not differ across the two testing sessions (day 2 and day 30), and therefore were pooled for all analyses. For comparison with macaques, median search times of trials with found targets were analyzed. Half of the repeated trials’ targets were found within 6.26 s, compared to 10.04 s for novel-trial targets (Figure 3A). Faster search times on repeated trials can be attributed to the speed of detecting remembered targets, half of which were found within 3.70 s, compared to 9.89 s for forgotten targets (Figure 3A). Within forgotten-target trials, half of forgotten (familiar) trials were found within 7.08 s, while half of forgotten (unfamiliar) trials were detected within 10.88 s. The two types of forgotten-target trials were not significantly different [t(11) = 0.94, p > 0.35]. Median search times for novel, remembered, and forgotten trials, including all targets (found or missed), are displayed in Figure 3B.
Figure 3. Characterization of search times from healthy humans in Experiment 1. (A) Cumulative frequency of novel and repeated search times, collapsed across all participants. Trials not found in the 60-s time limit were excluded. (B) Median search times by memory category. Each line represents the median time to detect the changing object for an individual participant; bars indicate the group mean. All trials (both found and not found) are included.
Search times differed across novel, remembered, and forgotten trials [F(2,22) = 52.73, p < 10−10]. Targets on repeated trials were found faster than those on novel trials [t(11) = 12.36, p < 10−7], and this effect was dominated by search times for explicitly remembered targets [remembered < forgotten, t(11) = 6.30, p < 10−4; remembered < novel, t(11) = −13.17, p < 10−7]. For trials in which the target was forgotten, there was no difference in search times for forgotten (familiar) and forgotten (unfamiliar) trials [t(11) = 0.94, p > 0.35]. There was, however, a significant repetition effect, even when the target was not explicitly recalled (forgotten < novel, t(11) = −3.16, p < 0.01). Finally, of the low number of false-positive remembered and forgotten (familiar) trials reported, the median search times were 4.42 (2.55) and 29.42 (21.83) s [median (SD)], and of the nine subjects who reported at least one false-positive remembered target, false-positive remembered trials were found significantly faster than novel trials [t(8) = −6.69, p < 0.001].
The time required to locate the target upon initial presentation influenced the chances of later recall. Forgotten targets-and-scenes were initially found faster during the training session [novel search time: 7.04 (3.86) s, median (SD)] compared to the initial search times of remembered targets [novel search time: 22.73 (9.34) s, median (SD); t(11) = 5.59, p < 0.001] or forgotten (familiar) trials [novel search time: 15.84 s, median (SD); t(11) = 3.01, p < 0.05].
The effect of changes in location or presence on search times across memory categories were also examined (due to the low number of color change trials, these trials were not analyzed). For novel trials, no difference in search time was found for changes in location [14.65 (9.42) s, median (SD)] or presence [15.70 (4.64) s, median (SD); t(11) = 0.34, p > 0.70]. Search times on repeated trials for changes in presence [7.34 (3.12) s, median (SD)] were longer than for location [4.35 (3.00) s, median (SD); t(11) = 3.50, p < 0.01], although when broken down into remembered, familiar, and forgotten trials, no differences were seen [ts(4–38) < 1.63, ps > 0.10].
Experiment 2: Macaques
Eye scan paths
An example of the eye scan paths on a repeated-trial whose search time was under the short search time threshold is shown in Figure 2B.
Behavioral results
For macaques, during the training and two testing sessions, 64 (5.0), 83 (10), and 70 (15)% [mean (SD)] of targets were found prior to the 60-s time limit, respectively. The proportion found targets did not differ between novel and repeated trials [χ2 (1, n = 143) = 0.11, p > 0.70]; however, the search times for repeated trials were faster than for those for novel trials [t(3) = 3.48, p < 0.05]. Similar to humans, repeated trials were found approximately twice as fast as compared to novel trials (from 7.56 to 14.53 s, respectively; Figures 4A,B). Here, search times were collapsed across day 2 and day 4 trials (the difference between the two testing sessions was not significant), and search times of the macaque who completed two experimental runs were collapsed.
Figure 4. Macaque search time patterns in Experiment 2. (A) Cumulative frequency of search times for novel and repeated trials across species. Human data from Figure 3A is plotted alongside macaque data (bolded lines) for comparison. (B) Median search times for novel and repeated trials. Population data is shown by bars; individual macaque median search times indicated by each line.
Experiment 3: Amnesic Patient D. A. and Controls
Behavioral results
For all participants (including D. A.), during the training and two testing sessions (5 min later and day 2), 59 (8.6), 65 (11), and 69 (4.1)% [mean (SD)] of targets were found before the 40-s time limit, respectively. Across the two testing sessions on consecutive days for control participants, no difference in the number of remembered and forgotten trials were found, nor in the number of false-positive remembered and forgotten (familiar) trials, although there was a trend toward fewer remembered images on day 2 [t(4) = 2.26, p = 0.087].
In control participants, the proportion of found targets was greater for repeated trials than for novel trials [χ2 (1, n = 196) = 45.35, p < 10−4]. When broken down by repeated-trial type, effects were only found for remembered [χ2 (1, n = 106) = 60.92, p < 10−4] and forgotten (unfamiliar) [χ2 (1, n = 21) = 7.16, p < 0.01] trials, but not for forgotten (familiar) trials [χ2 (1, n = 69) = 0.04, p = 1]. For D. A., the proportion of found targets did not differ between novel and repeated trials [χ2 (1, n = 39) = 1.31, p = 1]. Thus, only controls showed performance improvements in the absence of memory (i.e., implicit effects); D. A.’s performance was unaltered.
D. A. reported no explicit memory for scenes or target objects; all trials were classified as forgotten. Median search times of control participants and D. A. across memory categories is shown in Figure 5A. For D. A., median (SD) search times for novel and repeated trials were 26.47 (15.43) and 24.76 (14.42) s, respectively. D. A.’s repeated search times were longer than those of controls [t(4) = 2.88, p < 0.05], whereas novel search times between D. A. and control subjects did not differ [t(4) = 0.37, p > 0.70] (Figure 5B).
Figure 5. Behavioral performance of amnesic patient D. A. in Experiment 3. (A) Median search times and (B) cumulative frequency of D. A. compared to controls. For (A), conventions as in Figure 3B. D. A.’s performance is indicated by the red diamonds; all repeated trials were forgotten. In (B), the characteristic leftward shift in repeated trials observed in healthy humans and macaques is not apparent in D. A.’s search times. Only trials found within the 40-s time limit are shown here.
In contrast, the median novel and repeated search times in control participants were 30.81 (10.27) and 9.15 (4.95) s [median (SD)], respectively. Within repeated trials, median search times for remembered and forgotten trials were 4.38 (2.34) and 25.32 (8.84) s [median (SD)]. For targets that were forgotten, median search times for forgotten (familiar) and forgotten (unfamiliar) were 29.41 (6.56) and 16.78 (5.59) s. Forgotten (familiar) trials were found faster on day 2 compared to 5 min later [t(4) = 3.37, p < 0.05]. There was also a trend toward remembered trials being found faster on day 2 compared to 5 min later [t(4) = 2.46, p = 0.087]. Of the few false-positive remembered and forgotten (familiar) trials reported, the median search times were 6.12 (3.93) and 22.35 (8.23) s [median (SD)].
Search times varied across novel, remembered, and forgotten trials [F(2,8) = 13.77, p < 0.001]. Similar to the younger adults in Experiment 1, remembered trials were found faster than both forgotten trials and novel trials [v. forgotten: t(4) = 5.59, p < 0.01; v. novel: t(4) = 6.07, p < 0.01]. Note that whereas the older control subjects took longer to find novel and forgotten images than the younger adults [novel: t(15) = 4.47, p < 0.001; forgotten: t(15) = 4.59, p < 0.001], they found remembered targets just as quickly [t(15) = 0.16, p > 0.85]. Overall, repeated trials were still found faster than novel trials [t(4) = 4.88, p < 0.01]. Forgotten (familiar/unfamiliar) and forgotten (familiar) search times were not significantly different from novel search times [t(4) = 0.76, p > 0.45; and t(4) = 0.21, p > 0.8, respectively], but there was a trend for forgotten (unfamiliar) compared to novel trials [t(3) = 2.39, p = 0.097]. Finally, the inter-participant and inter-age-group differences in novel versus forgotten search times led us to compare the relationship for this control group between age and search times. We found no linear relationship [novel-forgotten search time, age: r(5) = 0.71, p > 0.15] though the two youngest participants had the shortest novel search time and the longest forgotten search times.
Additional testing session search times in Experiment 2
A follow-up experimental session was conducted to see if D. A. would report any explicit target memory if the memory demands were made easier. For blocks in which only four training trials occurred immediately before testing began, D. A. still reported all repeated images as forgotten (all unfamiliar). In both novel and repeated trials, he found 75% of the targets before the maximum time. Although median search times for novel and repeated trials were generally fast, they were also quite variable, and did not differ significantly between novel and repeated groups [novel: 15.17 (15.33) s and repeated: 10.01 (15.53) s (median (SD)); t(58) = 0.57, p > 0.55].
Predicting Remembered and Forgotten Trials
The shift in repeated trials toward shorter search times was driven by trials whose target was explicitly remembered (Figure 6A). We asked how predictive are short search times of remembered targets and, conversely, how predictive are long search times of forgotten targets, with the goal of extracting distributions that would be biased toward remembering or forgetting, but without requiring verbal report. On repeated trials, individual subjects’ search time distributions had an initial mode that peaked under 10 s, and a second smaller mode at the maximum trial time of 60 s (see Figure 6A for an example). When repeated search times were further divided into remembered and forgotten images, the number of remembered images at the first mode was greater than the number of forgotten images. This is in contrast to the second mode, where the proportion of forgotten images was greater than the proportion of remembered images. In addition, the distribution of forgotten trials’ search times was similar to the distribution of novel search times, irrespective of scene familiarity.
Figure 6. Classification of remembered trials in humans. (A) Search time probability densities of a representative human participant. Search time thresholds are derived from the inflection point of the repeated-trial distribution’s initial “mode.” For the participant shown here, the threshold was 9.33 s. (B) Overall accuracy of identifying remembered or forgotten targets, based on short search time threshold (individual inflection points) and long search time threshold (30 s).
To establish a search time threshold using these distributions, the inflection point of the first mode was taken to encompass the majority of trials under the first mode. Across human subjects in Experiments 1 and 3, the average search time threshold was 8.80 (5.11) s, under which 74.03 (17.80)% of trials were remembered targets. Within the remaining forgotten trials, 17.50 (17.03)% were from trials with familiar scenes and 8.48 (6.97)% were completely forgotten, median (SD; Figure 6B).
To identify trials that were likely forgotten, a long search time “forgetting” threshold was set at 30 s for all participants. On average, 88.31 (14.96)% [median (SD)] of trials above this threshold were comprised of forgotten targets (36.53% forgotten (unfamiliar), 51.78% forgotten (familiar). This was based on individual repeated search time distributions and could thus be applied to repeated search times in monkeys, to determine whether individual trials in monkeys were likely remembered or forgotten.
In macaques, the inflection point thresholds identified 62.43 (20.03)% of repeated trials [mean (SD)]. If we assume the results seen in humans apply here, 74% of these trials were remembered, or 40/54 trials correctly classified remembered trials. Similarly, using the 30-s “forgetting” threshold, 23.75 (13.77)% [mean (SD)] of repeated trials were selected. If we assume an 88% accuracy rate as seen in humans, this would yield 34/39 trials correctly classified as forgotten. The effect size based on the odds ratio is 21 [(short search time remember/forget ratio)/(long search time remember/forget ratio)]. Determining whether this is sufficient depends on the underlying two distributions.
Differences in Remembering Identity or Location of Objects
In the older adults in Experiment 3 and three younger adults in Experiment 1, participants distinguished between remembering what and where the target object was (data from younger and older participants as well as across sessions were collapsed, as no differences in search times of the different types of remembered trials were found [ts(13–131) < 0.78, ps > 0.40]. Search times differed across trials where participants remembered both identity and location, identity only, and location only [F(2,186) = 14.94, p < 10−6]. Trials where participants remembered both identity and location of the target object [3.39 (6.75) s, median (SD)] had faster search times compared to remembering object identity only [13.31 (17.69) s, median (SD); v. both: t(172) = 8.63, p < 10−14] or object location only [7.46 (10.16) s, median (SD); v. both: t(146) = 2.28, p < 0.05] details. As well, trials in which participants recalled the location of the target object had faster search times than recalling the identity of the target object [t(54) = 2.38, p < 0.05].
Discussion
This study demonstrates the utility of a novel test of object-in-scene memory that includes long-term retention following single exposures. We tested across species – in macaques and humans – and across MTL function – in an amnesic and healthy humans. Using a flicker change detection task, humans found explicitly remembered targets faster than forgotten or novel-trial targets. This positively skewed the distribution of search times on repeated trials compared to that of novel trials (i.e., short search times occurred more frequently on repeated trials). The increased occurrence of very rapid search times could be attributed to explicit memory of the target object. Though not as clear as the relation between search times and explicit effects, we found one measurement that was sensitive to implicit memory: targets were more likely to be found on repeated trials than on novel trials, even when the target object and scene had been forgotten.
Overall, the influence of memory on search times was robust across younger and middle-aged adults. We were able to identify remembered and forgotten trials using only the repeated search time distributions of a given individual; for a threshold delimiting the early skewed peak, 74% of trials were remembered and for a threshold delimiting long search times, 88% of trials were forgotten. Individuals who showed little to no target memory also showed no shift in repeated search times (one healthy, young adult participant and D. A.). In macaques, search times from repeated trials displayed the same characteristic positively skewed peak compared to those of novel trials, equivalent to what was observed in healthy human participants. This suggests that, like humans, faster search times in macaques are associated with memory for the target objects in this task. We used the same threshold methods to derive search time thresholds in macaques. Assuming that the common shift in search times across species reflects a common process, this technique allowed us to identify predominantly remembered/forgotten trials in macaques. Finally, an amnesic patient showed no memory for repeated targets and had unaffected search times with repetition – i.e., he showed neither explicit nor implicit memory on this task. These findings, in conjunction with previous literature measuring amnesic performance on change detection tasks, suggest that the improvements in search times in healthy participants (and the humans’ explicit recall) are likely to depend on MTL function. As such, this task may be a new tool to measure MTL-dependent memory across species.
Amnesic Profile
In the current study, D. A. showed search times on novel images that were indistinguishable from those of healthy controls, indicating that his ability to perform the standard flicker change detection task was unimpaired. But in striking contrast to controls, he reported no memory for any of the scenes or target objects, and he showed no benefit in search times on repeated trials; his repeated and novel search time distributions were indistinguishable. D. A.’s previous performance on tests of anterograde and retrograde episodic memory is generally below average, despite preserved abilities based on standard neuropsychological tests of other cognitive functions (Rosenbaum et al., 2008) and on other memory tasks such as the transverse patterning task (Ryan et al., 2009). Generally, his deficits in this task are consistent with his anterograde amnesia.
In other studies of amnesic patients, relational memory may be impaired not only over the long-term, but during short-term intervals as well (Graf and Schacter, 1985; Gabrieli et al., 1998; Chun and Phelps, 1999; Hannula et al., 2006). In support of this finding, during the follow-up testing session in which repeated trials were viewed within minutes of their initial presentation, D. A. still did not report any recognition and again, no differences in his search times were observed. This is in contrast to evidence of some preservation of memory in amnesics for another change detection task when the change was presented within ∼2 s following initial scene presentation and no stimuli intervened (Ryan and Cohen, 2004). One difference may be the retention intervals involved. Patients with MTL lesions have shown deficits in visual working memory (WM) tasks, provided the retention intervals exceeded 1- to 2-s (Nichols et al., 2006; Olson et al., 2006). Indeed, electrophysiological data suggest different underlying memory processes for 1 and 6 s retention intervals (Bankó and Vidnyánszky, 2010). In addition, our patient was required to retain memory for the object over intervening trials. Finally, other studies have used several gaze measures of change detection that differ from the search times used in the present study. Nevertheless, it would appear that the present findings are relevant to current ideas about declarative/relational memory that can operate at relatively brief delays, and including intervening stimuli.
We are cautious in generalizing data from one patient; variability in memory performance in various tasks has been observed among patients with similar or overlapping lesion profiles (for example, Holdstock et al., 2008). Moreover, numerous studies of amnesics with selective MTL lesions provide evidence of the role of the MTL in memory performance, but it is important to note that D. A. has additional damage outside of his MTL that cannot be discounted, notably in his right anterior temporal cortex. For this task, at least, D. A. showed a profound memory impairment, with no sign of any implicit search time benefit for repeated stimuli. Moreover, one healthy young participant consistently reported forgetting, and indeed, had a repeated search time distribution that resembled that of the novel distribution, further suggesting the link between memory and short search times. More data will be needed to clarify whether and how the hippocampus and surrounding MTL and temporal lobe structures contribute to the explicit memory and search time measures on this task.
Declarative/Relational Memory for Targets-in-Scenes
Although the macaques could not explicitly report which images and/or targets they remembered, the same search time profiles seen across species hint at a common underlying memory process. In change detection experiments using natural scenes, humans rely on contextual cues for object detection (Brockmole et al., 2006; Hollingworth, 2006; Torralba et al., 2006; Becker and Rasmussen, 2008). The relational account of memory proposes that the hippocampus is responsible for encoding relations among multiple items. In the flicker change detection memory paradigm, this would involve recalling the embedded changing target object in relation to surrounding items within the scene. Successful performance may be hippocampally dependent, given the difficulty D. A. showed, and based on other findings of deficits in patients with more restricted hippocampal damage. In healthy humans, explicitly reported memory led to the fastest recall, though this could be the “downstream” effect of stronger relational memory, and not the conscious act of recalling. To determine which structures are critical for relational memory formation, future studies could test macaques or humans with more precise lesions within and around the hippocampus using the task described here.
Eye Movements Reveal Memory
In lieu of verbal report, our task uses the speed of target fixation as an indication of memory. For decades, gaze direction and/or duration have been used as dependent measures in non-declarative settings, such as with infants (Fantz, 1963; Fagan, 1970; Cohen and Strauss, 1979; Richmond et al., 2007) and with macaques (Bagshaw et al., 1970a,b; Wilson and Goldman-Rakic, 1994; Pascalis and Bachevalier, 1999). Gaze can signal item and relational memory, whereby novel stimuli or relations among stimuli are viewed longer than familiar stimuli (see Hannula et al., 2010, for review). In adult humans, changes in eye movement patterns toward scenes have been linked to memory. Some groups suggest this results from explicit or declarative memory (Smith and Squire, 2008), others suggest awareness is not essential (Ryan et al., 2000; Hannula and Ranganath, 2008), and a third possibility is that participants report explicit recall as a consequence of rapid search and not the other way around (though we only saw false positives that would support this assertion in about 2% of novel images).
Other features of eye movements may shed light on memory processing. Conscious memory for familiar scenes leads to fewer sampled regions and fewer fixations across the entire image both at study (Sharot et al., 2008) and at test (Smith and Squire, 2008). On a (non-flicker) change detection task, in which repeated images were altered, Smith and Squire (2008) found that participants made more eye movements to where the change occurred, but only when they explicitly reported knowledge of the change. In contrast, Ryan et al. (2000) demonstrated on a similar change detection task that people look more at the critical changed region even when they are not aware of it. Amnesic patients did not display this behavior for longer delays of minutes, but did show recall when the manipulation followed the original image with no intervening stimuli (Ryan and Cohen, 2004). Furthermore, on a face-scene association task, Hannula and Ranganath (2009) found that implicit relational memory was linked to eye movements, and this behavior was predicted by hippocampal activity independent of explicit recall. Differences in the paradigms may explain the discrepancies between explicit and implicit memory related to eye movements. Further research is required to determine what specific types of memory may be revealed through various aspects of eye movements, and how different experimental settings may preferentially lead to implicit or explicit memory-related eye movements. In our study, we were interested in how quickly a changing target was located. Notably, the “change” was present in novel and repeated trials alike; we expected that if repeated trials were remembered, the change would be obvious and therefore fixated rapidly relative to novel or forgotten trials. For this task, using this measure, there was a strong correspondence between remembered trials and fast search times, which could be used in cross-species studies of memory.
Using the Flicker Change Detection Task to Evaluate Memory Formation Across Species
This task fills a unique niche as a memory task for use in macaques: it is a non-verbal measure of object-in-scene memory that does not require extensive training and can be evaluated and sorted on a single-trial basis. The distribution of search times in a given session can be taken as an indicator of overall retention: poor retention will lead to a repeated search time distribution that resembles that of novel trials, whereas good retention will be accompanied by a prominent skewness in the repeated search time distribution. This measure can also be used to determine what types of activity during encoding predict memory (or forgetting), on the basis of individual-trial outcomes (in humans: Brewer et al., 1998; Wagner et al., 1998; Fell et al., 2001; Paller and Wagner, 2002; Sederberg et al., 2007; and in macaques Jutras et al., 2009). Moreover, memory success or failure may be manipulated by altering the duration of scene viewing. Study duration is known to influence scene memory (Melcher, 2006) and the current findings revealed that forgotten trials were predicted by short encoding durations and remembered trials were predicted by long encoding durations. Both a posteriori trial sorting and a priori duration manipulations are relevant for investigations of the neural bases of memory formation, such as memory trace reactivation (Hoffman and McNaughton, 2002). To date, there is little evidence linking the neural phenomenon of trace reactivation to subsequent memory recall (for review see Hoffman et al., 2007). Using the same rationale applied to predictors during encoding, one could evaluate whether trial-specific neural activity observed after encoding –such as memory trace reactivation – is related to subsequent memory strength. In sum, this task allows object-in-scene memory to be measured on a trial-by-trial basis over long retention intervals, providing a means of determining which neural structures, and what neural mechanisms, predict long-term memory formation.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
The authors thank our reviewers for their insightful comments, and Niki Mofidi and all of our participants for making this study possible. This work was supported by NSERC, Canada Foundation for Innovation, the Alfred P. Sloan Foundation, and an Ontario Ministry of Research and Innovation Early Researcher Award.
References
Angeli, S. J., Murray, E. A., and Mishkin, M. (1993). Hippocampectomized monkeys can remember one place but not 2. Neuropsychologia 31, 1021–1030.
Bachevalier, J., and Nemanic, S. (2008). Memory for spatial location and object-place associations are differently processed by the hippocampal formation, parahippocampal areas TH/TF and perirhinal cortex. Hippocampus 18, 64–80.
Bachevalier, J., Saunders, R. C., and Mishkin, M. (1985). Visual recognition in monkeys – effects of transection of fornix. Exp. Brain Res. 57, 547–553.
Bagshaw, M. H., Mackworth, N. H., and Pribram, K. H. (1970a). The effect of inferotemporal cortex ablations on eye movements of monkeys during discrimination training. Int. J. Neurosci. 1, 153–158.
Bagshaw, M. H., Mackworth, N. H., and Pribram, K. H. (1970b). Method for recording and analyzing visual fixations in the unrestrained monkey. Percept. Mot. Skills 31, 219–222.
Bankó, E. M., and Vidnyánszky, Z. (2010). Retention interval affects visual short-term memory encoding. J. Neurophysiol. 103, 1425–1430.
Banta Lavenex, P., Amaral, D. G., and Lavenex, P. (2006). Hippocampal lesion prevents spatial relational learning in adult macaque monkeys. J. Neurosci. 26, 4546–4558.
Banta Lavenex, P., and Lavenex, P. (2009). Spatial memory and the monkey hippocampus: not all space is created equal. Hippocampus 19, 8–19.
Becker, M. W., and Rasmussen, I. P. (2008). Guidance of attention to objects and locations by long-term memory of natural scenes. J. Exp. Psychol. Learn. Mem. Cogn. 34, 1325–1338.
Brewer, J. B., Zhao, Z., Desmond, J. E., Glover, G. H., and Gabrieli, J. D. E. (1998). Making memories: brain activity that predicts how well visual experience will be remembered. Science 281, 1185–1187.
Brockmole, J. R., Castelhano, M. S., and Henderson, J. M. (2006). Contextual cueing in naturalistic scenes: global and local contexts. J. Exp. Psychol. Learn. Mem. Cogn. 32, 699–706.
Buffalo, E. A., Ramus, S. J., Clark, R. E., Teng, E., Squire, L. R., and Zola, S. M. (1999). Dissociation between the effects of damage to perirhinal cortex and area TE. Learn. Mem. 6, 572–599.
Chun, M. M., and Phelps, E. A. (1999). Memory deficits for implicit contextual information in amnesic subjects with hippocampal damage. Nat. Neurosci. 2, 844–847.
Clark, R. E., and Squire, L. R. (2010). An animal model of recognition memory and medial temporal lobe amnesia: history and current issues. Neuropsychologia 48, 2234–2244.
Cohen, L. B., and Strauss, M. S. (1979). Concept acquisition in the human infant. Child Dev. 50, 419–424.
Crawford, J. R., and Howell, D. C. (1998). Comparing an individual’s test score against norms derived from small samples. Clin. Neuropsychol. 12, 482–486.
Fell, J., Klaver, P., Lehnertz, K., Grunwald, T., Schaller, C., Elger, C. E., and Fernández, G. (2001). Human memory formation is accompanied by rhinal-hippocampal coupling and decoupling. Nat. Neurosci. 4, 1259–1264. [See comments].
Gabrieli, J. D. E., Cohen, N. J., and Corkin, S. (1998). The impaired learning of semantic knowledge following bilateral medial temporal-lobe resection. Brain Cogn. 7, 157–177.
Gaffan, D. (1974). Recognition impaired and association intact in the memory of monkeys after transection of the fornix. J. Comp. Physiol. Psychol. 86, 1100–1109.
Gaffan, D. (1994). Scene-specific memory for objects: a model of episodic memory impairment in monkeys with fornix transection. J. Cogn. Neurosci. 6, 305–320.
Graf, P., and Schacter, D. L. (1985). Implicit and explicit memory for new associations in normal, and amnesic subjects. J. Exp. Psychol. Learn. Mem. Cogn. 13, 45–53.
Gunderson, V. M., and Sackett, G. P. (1984). Development of pattern recognition in infant pigtailed macaques (macaca nemestrina). Dev. Psychol. 20, 418–426.
Hampton, R. R., Hampstead, B. M., and Murray, E. A. (2004). Selective hippocampal damage in rhesus monkeys impairs spatial memory in an open-field test. Hippocampus 14, 808–818.
Hampton, R. R., Hampstead, B. M., and Murray, E. A. (2005). Rhesus monkeys (macaca mulatta) demonstrate robust memory for what and where, but not when, in an open-weld test of memory. Learn. Motiv. 36, 245–259.
Hannula, D. E., Althoff, R. R., Warren, D. E., Riggs, L., Cohen, N. J., and Ryan, J. D. (2010). Worth a glance: using eye movements to investigate the cognitive neuroscience of memory. Front. Hum. Neurosci. 4:166. doi: 10.3389/fnhum.2010.00166
Hannula, D. E., and Ranganath, C. (2008). Medial temporal lobe activity predicts successful relational memory binding. J. Neurosci. 23, 116–124.
Hannula, D. E., and Ranganath, C. (2009). The eyes have it: hippocampal activity predicts expression of memory in eye movements. Neuron 63, 592–599.
Hannula, D. E., Tranel, D., and Cohen, N. J. (2006). The long and the short of it: relational memory impairments in amnesia, even at short lags. J. Neurosci. 26, 8352–8359.
Hoffman, K. L., Battaglia, F. P., Harris, K. D., MacLean, J. N., Marshall, L., and Mehta, M. (2007). The upshot of up states in neocortex: from slow oscillations to memory formation. J. Neurosci. 27, 11838–11841.
Hoffman, K. L., and McNaughton, B. L. (2002). Coordinated reactivation of distributed memory traces in primate neocortex. Science 297, 2070–2073.
Holdstock, J. S., Parslow, D. M., Morris, R. G., Fleminger, S., Abrahams, S., Denby, C., Montaldi, D., and Mayes, A. R. (2008). Two case studies illustrating how relatively selective hippocampal lesions in humans can have quite different effects on memory. Hippocampus 18, 679–691.
Hollingworth, A. (2006). Visual memory for natural scenes: evidence from change detection and visual search. Vis. cogn. 14, 781–807.
Jutras, M. J., Fries, P., and Buffalo, E. A. (2009). Gamma-band synchronization in the macaque hippocampus and memory formation. J. Neurosci. 29, 12521–12531.
Konkel, A., and Cohen, N. J. (2009). Relational memory and the hippocampus: representations and methods. Front. Neurosci. 3:2. doi: 10.3389/neuro.01.023.2009
Mahut, H., Zola-Morgan, S., and Moss, M. (1982). Hippocampal resections impair associative learning and recognition memory in the monkey. J. Neurosci. 2, 1214–1229.
Malkova, L., and Mishkin, M. (2003). One-trial memory for object-place associations after separate lesions of hippocampus and posterior parahippocampal region in the monkey. J. Neurosci. 23, 1956–1965.
Mishkin, M., and Delacour, J. (1975). An analysis of short-term visual memory in the monkey. J. Exp. Psychol. Anim. Behav. Process. 1, 326–334.
Murray, E. A., Baxter, M. G., and Gaffan, D. (1998). Monkeys with rhinal cortex damage or neurotoxic hippocampal lesions are impaired on spatial scene learning and object reversals. Behav. Neurosci. 112, 1291–1303.
Nadel, L., and Moscovitch, M. (1997). Memory consolidation, retrograde amnesia and the hippocampal complex. Curr. Opin. Neurobiol. 7, 217–227.
Nadel, L., Winocur, G., Ryan, L., and Moscovitch, M. (2007). Systems consolidation and hippocampus: two views. Debates Neurosci. 1, 55–66.
Nemanic, S., Alvarado, M. C., and Bachevalier, J. (2004). The Hippocampal/Parahippocampal regions and recognition memory: insights from visual paired comparison versus object-delayed nonmatching in monkeys. J. Neurosci. 24, 2013–2026.
Nichols, E. A., Kao, Y. C., Verfaellie, M., and Gabrieli, J. D. E. (2006). Working memory and long-term memory for faces: evidence from fMRI and global amnesia for involvement of the medial temporal lobes. Hippocampus 16, 604–616.
Olson, I. R., Moore, K. S., Stark, M., and Chatterjee, A. (2006). Visual working memory is impaired when the medial temporal lobe is damaged. J. Cogn. Neurosci. 18, 1087–1097.
Paller, K. A., and Wagner, A. D. (2002). Observing the transformation of experience into memory. Trends Cogn. Sci. (Regul. Ed.) 6, 93–102.
Parkinson, J. K., Murray, E. A., and Mishkin, M. (1988). A selective mnemonic role for the hippocampus in monkeys – memory for the location of objects. J. Neurosci. 8, 4159–4167.
Pascalis, O., and Bachevalier, J. (1999). Neonatal aspiration lesions of the hippocampal formation impair visual recognition memory when assessed by paired-comparison task but not by delayed nonmatching-to-sample task. Hippocampus 9, 609–616.
Pascalis, O., and de Haan, M. (2003). “Recognition memory and novelty preference: what model?” in Progress in Infancy Research, eds H. Hayne and J. Fagen (New Jersey: Lawrence Erlbaum Associates), 95–120.
Rensink, R. A., O’Regan, K. J., and Clark, J. J. (1997). To see or not to see: the need for attention to perceive changes in scenes. Psychol. Sci. 8, 368–373.
Richmond, J., Colombo, M., and Hayne, H. (2007). Interpreting visual preferences in the visual paired-comparison task. J. Exp. Psychol. Learn. Mem. Cogn. 33, 823–831.
Rose, S. A., Feldman, J. F., and Jankowski, J. J. (2007). Developmental Aspects of Visual Recognition Memory in Infancy. Short- and Long-Term Memory in Infancy and Early Childhood. New York, NY: Oxford University Press, 153–178.
Rosenbaum, S. R., Moscovitch, M., Foster, J. K., Schnyer, D. M., Gao, F., Kovacevic, N., Verfaellie, M., Black, S. E., and Levine, B. (2008). Patterns of autobiographical memory loss in medial-temporal lobe amnesic patients. J. Cogn. Neurosci. 20, 1490–1506.
Ryan, J. D., Althoff, R. R., Whitlow, S., and Cohen, N. J. (2000). Amnesia is a deficit in relational memory. Psychol. Sci. 11, 454–461.
Ryan, J. D., and Cohen, N. J. (2003). Evaluating the neuropsychological dissociation evidence for multiple memory systems. Cogn. Affect. Behav. Neurosci. 3, 168–185.
Ryan, J. D., and Cohen, N. J. (2004). The nature of change detection and online representations of scenes. J. Exp. Psychol. Hum. Percept. Perform. 30, 988–1015.
Ryan, J. D., Ostreicher, M., Kamino, D., Moses, S. N., and Rosenbaum, R. S. (2009). “Processing and acquisition of relations in amnesia and healthy aging,” in 39th Annual Society for Neuroscience Conference, Chicago, IL.
Sederberg, P. B., Schulze-Bonhage, A., Madsen, J. R., Bromfield, E. B., McCarthy, D. C., Brandt, A., Tully, M. S., and Kahana, M. J. (2007). Hippocampal and neocortical gamma oscillations predict memory formation in humans. Cereb. Cortex 17, 1190–1196.
Sharot, T., Davidson, M. L., Carson, M. M., and Phelps, E. A. (2008). Eye movements predict recollective experience. PLoS ONE 3, e2884. doi: 10.1371/journal.pone.0002884
Simons, D. J., and Levin, D. T. (1997). Change blindness. Trends Cogn. Sci. (Regul. Ed.) 1, 261–267.
Smith, C. N., Hopkins, R. O., and Squire, L. R. (2006). Experience-dependent eye movements, awareness, and hippocampus-dependent memory. J. Neurosci. 26, 11304–11312.
Smith, C. N., and Squire, L. R. (2008). Experience-dependent eye movements reflect hippocampus-dependent (aware) memory. J. Neurosci. 28, 12825–12833.
Squire, L. R., Cohen, N. J., and Nadel, L. (1984). The medial temporal region and memory consolidation: A new hypothesis. in Memory consolidation eds H. Weingartner, and E. Parker (Hillsdale, NJ: Lawrence Erlbaum Associates), 185–210.
Squire, L. R., Knowlton, B., and Musen, G. (1993). The structure and organization of memory. Annu. Rev. Neurosci. 44, 453–495.
Torralba, A., Oliva, A., Castelhano, M. S., and Henderson, J. M. (2006). Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. Psychol. Rev. 113, 766–786.
Wagner, A. D., Schacter, D. L., Rotte, M., Koutstaal, W., Maril, A., Dale, A. M., Rosen, B. R., and Buckner, R. L. (1998). Building memories: remembering and forgetting of verbal experiences as predicted by brain activity. Science 281, 1188–1191. [See comments].
Wilson, F. A., and Goldman-Rakic, P. S. (1994). Viewing preferences of rhesus monkeys related to memory for complex pictures, colours and faces. Behav. Brain Res. 60, 79–89.
Winters, B. D., Saksida, L. M., and Bussey, T. J. (2008). Object recognition memory: neurobiological mechanisms of encoding, consolidation and retrieval. Neurosci. Biobehav. Rev. 32, 1055–1070.
Zola, S. M., Squire, L. R., Teng, E., Stefanacci, L., Buffalo, E. A., and Clark, R. E. (2000). Impaired recognition memory in monkeys after damage limited to the hippocampal region. J. Neurosci. 20, 451–463.
Zola-Morgan, S., and Squire, L. (1985). Medial temporal lesions in monkeys impair memory in a variety of tasks sensitive to human amnesia. Behav. Neurosci. 100, 165–170.
Keywords: episodic memory, spatial memory, hippocampus, memory consolidation, explicit, monkey, familiarity, relational
Citation: Chau VL, Murphy EF, Rosenbaum RS, Ryan JD and Hoffman KL (2011) A flicker change detection task reveals object-in-scene memory across species. Front. Behav. Neurosci. 5:58. doi: 10.3389/fnbeh.2011.00058
Received: 01 October 2010;
Accepted: 21 August 2011;
Published online: 20 September 2011.
Edited by:
Agnes Gruart, University Pablo de Olavide, SpainReviewed by:
Pierre Lavenex, University of Fribourg, SwitzerlandZoltán Vidnyánszky, Pazmany Peter Catholic University, Hungary
Copyright: © 2011 Chau, Murphy, Rosenbaum, Ryan and Hoffman. This is an open-access article subject to a non-exclusive license between the authors and Frontiers Media SA, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and other Frontiers conditions are complied with.
*Correspondence: Kari L. Hoffman, Centre for Vision Research, Department of Psychology, York University, 4700 Keele Street, Toronto, ON, Canada M3J IP3. e-mail: khoffman@yorku.ca