A Flicker Change Detection Task Reveals Object-in-Scene Memory Across Species

Tests of recognition memory in macaques typically assay memory for objects or isolated images, over time spans of seconds to hours from stimulus presentation, and/or require extensive training. Here, we propose a new application of the flicker change detection task that could measure object-in-scene memory days after single-trial exposures. In three experiments, participants searched for a changing object – or “target” – embedded within a scene as their eye movements were tracked. For new targets-in-scenes, the change is difficult to detect and requires extensive search. Once the target is found, however, the change becomes obvious. We reasoned that the decreased times required to find a target in a repeated scene would indicate memory for the target. In humans, targets were found faster when the targets-and-scenes were explicitly remembered than when they were forgotten, or had never been seen before. This led to faster repeated-trial compared to novel-trial search times. Based solely on repeated-trial search times, we were able to select distributions comprised of predominantly remembered or predominantly forgotten trials. Macaques exhibited the same repetition effects as humans, suggesting that remembered trials could be dissociated from novel or forgotten trials using the same procedures we established in humans. Finally, an anterograde amnesic patient with damage that included the medial temporal lobe (MTL) showed no search time differences, suggesting that memory revealed through search times on this task requires MTL integrity. Together, these findings indicate that the time required to locate a changing object reveals object-in-scene memory over long retention intervals in humans and macaques.


INTRODUCTION
Memories are organized into several distinct processes. A longheld view is that amnesic patients who show impairments in long-term memory for facts and events that they can consciously recall (explicit/declarative memory) also show intact longterm memory for habits/patterns without conscious awareness (implicit/non-declarative memory) and have intact short-term memory (see Squire et al., 1993 for review). According to this view, the hippocampus, along with surrounding medial temporal lobe (MTL) structures, has been implicated in the formation of explicit/declarative memories, whereas procedural and implicit memories are generally thought to be mediated by non-MTL structures (Zola-Morgan and Squire, 1993). An alternate account posits that the hippocampus supports associations among multiple items, termed relational memory, which may constitute an important prerequisite for episodic/declarative memory but which could also operate without awareness and on short retention intervals (Ryan and Cohen, 2003;Konkel and Cohen, 2009). An additional distinction may be that the hippocampus plays a timelimited role in memory processing (Squire et al., 1984, but see Nadel and Moscovitch, 1997). Cutting across these accounts, the hippocampus is thought to be important for the formation of long-term relational memory, even when learned over a single exposure, or "episode." The unique functions of the hippocampus among other MTL structures can be evaluated with the greatest precision using direct measures, such as selective lesions or intracranial electrophysiology involving animal models. One challenge with the use of animals is the selection of appropriate memory tasks. In macaques, the delayed non-matching/matching to sample [D(N)MS] task is the most commonly used measure of object recognition memory to assess MTL function (Gaffan, 1974;Mishkin and Delacour, 1975;Mahut et al., 1982;Bachevalier et al., 1985;Zola-Morgan and Squire, 1985). Subtle changes in delay period activity across experiments can lead to disparate results (Nemanic et al., 2004), and the use of extensive training and motivational incentives can lead to variability in rule learning on this task (see reviews: Winters et al., 2008;Clark and Squire, 2010). The visual-paired comparison (VPC) and visual preferential looking (VPL) tasks are also widely used measures of recognition memory. These tasks take advantage of the natural tendency to look longer at novel images, circumventing issues of training and differential reinforcement (starting in infants: Fagan, 1970; for review Rose et al., 2007;and later in monkeys: Gunderson and Sackett, 1984;Buffalo et al., 1999;Pascalis and Bachevalier, 1999;Zola et al., 2000;Pascalis and de Haan, 2003;Nemanic et al., 2004;Jutras et al., 2009). VPC tasks typically test object recognition (item) memory in the absence of memory for spatial contexts, and may therefore recruit a distinct set of neural structures, with the role of the hippocampus across species still in debate (Winters et al., 2008;Clark and Squire, 2010). Thus, one-trial DNMS and VPC tests may be well-suited for measuring object recognition memory within a session, but they are difficult to adapt to -and may require different neural substrates from -tests that include objects in context, particularly when they involve one-trial memory with spans over days. When spatial or "relational" object arrays have been used with a VPC design, monkeys were tested at delays of only seconds (Bachevalier and Nemanic, 2008). Even when retention intervals were increased to days, there is concern that some "spatial" variants of these tasks may be accomplished based on a (MTL-independent) egocentric reference frame rather than requiring allocentric information of spatial relationships among objects (Banta Lavenex and Lavenex, 2009).
Macaques have successfully learned spatial or object-in-context tasks that use single-trial exposures, but only at short delays (Parkinson et al., 1988;Angeli et al., 1993;Malkova and Mishkin, 2003). They can also perform spatial tasks at longer delays but require repeated learning trials (and explicit reinforcement; Gaffan, 1994;Hampton et al., 2005). Finally, some studies have used spatial/relational tasks that include both repetition and within-day testing (Murray et al., 1998;Hampton et al., 2004;Banta Lavenex et al., 2006). At least in macaques, there are no standard tests of memory across days, and following single exposures.
In humans, relational or object-in-scene memory has been assessed after single exposures using a change detection task, which is thought to require long-term memory (Ryan and Cohen, 2004). In this paradigm, an object embedded in a scene is removed, added, or shifted from its earlier presentation(s), causing increased gaze toward the location of the manipulation on the screen (Ryan et al., 2000;Ryan and Cohen, 2004;Smith et al., 2006;Smith and Squire, 2008). Amnesic patients tested over longer retention intervals with intervening trials during the retention interval (Ryan et al., 2000) or with damage restricted to the hippocampus (Smith et al., 2006) fail to notice the change. If the initial viewing of the scene could be made interesting enough to macaques, their gaze toward a changed object might indicate memory for objects in scene contexts. For this, the flicker change detection task may be advantageous, though to our knowledge it has not yet been used as a test of memory.
In the present study, our aim was to evaluate whether eye movements during a flicker change detection task could be used as a measure of object-in-scene memory in macaques. In flicker change detection, the presentation of an original scene alternates with that of an object-manipulated scene, with a brief blank screen inserted in between that provides a visual disruption (Rensink et al., 1997). Multiple alternations and extensive searching are typically required before the change is detected, due to the socalled "change blindness" effect (Simons and Levin, 1997). Once found, however, the change becomes obvious, attracting attention (and gaze). Memory of the change may prevent the "blindness" characteristic of novel-image viewing, quickly capturing attention (and gaze) at the location of the change. Thus, in this variant of the change detection task, both learning and test stimuli have identical presentation characteristics and all trials share the same behavioral goal: to locate the change. The difference is that a remembered change should lead to rapid detection (i.e., visual fixation).
To test this hypothesis, in the first experiment, human participants viewed novel or repeated change detection trials, and reported whether they remembered the changing object or not. If search times for remembered items are significantly faster than search times for forgotten items, we would conclude that search times can be used as a measure of memory. To determine if we could identify remembered or forgotten trials based solely on search times, we divided the distribution of search times into "fast" and "slow" groups. For each group, we determined the proportion of trials that were reported as remembered or forgotten.
In the second experiment, four rhesus macaques performed the same change detection memory task. If monkeys show the same shift in search times predicted in humans -specifically, decreased search times for repeated trials -we could expect to apply our search time thresholds to reliably differentiate between remembered and forgotten images in macaques.
In a third experiment, we addressed whether object-in-scene memory is dependent upon MTL integrity. Behavioral data from an amnesic patient (D. A.) with bilateral MTL damage were compared to those of age-matched controls.
The present study is a novel test of object-in-scene memory. It can be used to test memory over days, and is of interest when tasks requiring navigation are undesirable, on theoretical grounds (e.g., testing relational memory without navigation), or due to experimental limitations (e.g., during electrophysiological recordings or in populations with limited mobility).

Subjects
Twelve York University students [five males, ages 19-26 years, mean (SD) age 22.42 (1.88) years] volunteered to take part in the study. Subjects had normal or corrected-to-normal vision. Written informed consent was obtained, and the study was conducted in accordance with the ethical guidelines set by the York Human Participants Review Subcommittee.

Stimuli
Images for change blindness trials were selected from a wide variety of natural scenes, including wildlife, city, rural, and indoor scenes. Target objects were modified in location, color, and presence (comprising 18.89, 3.89, and 77.22% of all trials) using Adobe Photoshop. Target location (quadrant on screen), size, and category (animate/inanimate) were balanced across trials in a session, to reduce the chance of biases in the search strategies of participants. Sample stimuli are shown in Figure 1A.

Behavioral procedure
Three sessions were held on different days to test for memory of the changing objects. Each session was divided into three blocks of 20 images each. During each block, the participant's head was positioned 51 cm away from a 38-cm by 30-cm screen and was placed on a chinrest to minimize head movements. Eye tracking Frontiers in Behavioral Neuroscience www.frontiersin.org data were collected using iView X infrared eye tracking system (SensoMotoric Instruments, SMI, Berlin, Germany). Before the change blindness trials began, eye gaze was calibrated using a 13point system. Briefly, calibration provided the pupil and corneal reflectance values when gaze was directed toward 13 known locations. Through a proprietary algorithm, the iView system then interpolates the observed pupil and corneal reflectance values to produce an estimate of the point of regard (i.e., gaze location) anywhere on the screen. The iView system sent gaze information to our stimulus presentation software, Presentation (NeuroBehavioral Systems, CA, USA). This allowed the online detection of fixation locations and durations at various places on the screen, including fixations on the target object. The selection and timing of visual stimuli were also controlled in Presentation.

Training session
During the initial training session (day 1), only novel stimuli were shown. An example of a changing image was given prior to the start of the experimental trials. Each trial consisted of the original image alternating with its modified image for 500 ms each, with a gray screen intervening for 50 ms between each alternation ( Figure 1B). These intervals maximized the time with the scene displayed, while staying within the range of times commonly seen in flicker change detection tasks. Once the participant fixated on the object for 1.2 s or 60 s had passed, the changing object was made visible by alternating the original and modified image with no intervening gray screen.

Test sessions
There were two follow-up testing sessions that occurred 1 day and approximately 1 month later (day 2 and day 30, respectively). Half of the original images were shown on day 2 and the other half were shown on day 30, with each block consisting of half novel and half repeated images presented in random order. The procedure was otherwise similar to the initial training session, with images comprised of half animate and inanimate targets, balanced across the novel and repeated images. After each test session trial, a screen was shown asking participants: "Had you seen this picture before?" and "If so, did you remember which object was changing?" to which verbal yes/no responses were given and recorded. If participants responded yes to both questions, the image was classified as remembered; if they answered yes to the first question (scene was familiar) and no to the second (target item forgotten), it was classified as forgotten (familiar); if they answered no to remembering both the scene and target, it was classified as forgotten (unfamiliar). Participants never answered no to the first question and yes to the second, hence no additional category was required. Some participants distinguished between whether they Frontiers in Behavioral Neuroscience www.frontiersin.org recalled where the change occurred or what object changed; in response, they were instructed to answer yes to the second question if they remembered either the position or identity of the target. To examine search time differences in remembering the identity or location of the target object, for the last three participants, the second question was replaced with:"If so, did you remember where the object was? Did you remember what the object was?" If participants answered yes to at least one of these questions, the trial was classified as remembered.

Data analysis
Fixation data and search times were analyzed using MATLAB. The search time required to locate the target object was calculated as the time from the first fixation after the trial started to the time when the subject fixated on the target object for 1.2 s. Occasionally, the eye tracking was inaccurate whether due to calibration errors or non-linearities in the signal (e.g., from the curvature of the eye), leading to 1 or 2 discarded trials per 120 trials (1: N = 5 subjects; 2: N = 1 participant). Search times were grouped according to novel or repeated-trial types. Search times from repeated trials were further classified as remembered or forgotten based on subjects' verbal reports. For novel trials, images were either correctly identified as novel or incorrectly recognized as remembered or forgotten (i.e., false positives). Median search times were compared statistically across categories (novel/remembered/forgotten) using a repeated measures one-way ANOVA. All trials, including trials in which the target was not found, were included. If the ANOVA was significant at an alpha of 0.05, planned comparisons were calculated for repetition effects: novel v. repeated; for implicit memory effects: novel v. forgotten (unfamiliar); familiarity effects: novel v. forgotten (familiar); and for explicit memory effects: remembered v. forgotten (familiar/unfamiliar).

Subjects and stimuli
Four female rhesus macaques were tested (three aged 3 years, weighing 4-5 kg; one aged 8 years, weighing 10 kg), with one macaque completing an additional run using new stimulus sets. Procedures were approved by the York University Animal Care Committee and were performed in accordance with the Canadian Council on Animal Care. Stimuli were made in the same manner as described for human participants. The macaques received juice reward for fixating a large blinking object presented in random locations on a screen that had a white background. As training progressed, smaller objects were used and the length of fixation required to obtain juice was progressively lengthened from 0.5 to 1.2 s. Next, they learned to fixate on large, salient flickering objects within simple natural scenes (for example, a person in the foreground of a sparse mountain scene). All four macaques completed these "easy embedded" trials within the first few exposures, and upon completion of a set of 20 easy trials, they advanced to scenes with increasing complexity, interspersed with previously seen familiar scenes to reinforce appropriate looking behavior. Total embedded-image training time for each of the four macaques spanned 1-2 weeks with a daily session of cooperative behavior involving 3-6 sets of 20 trials per set. Testing began the Thursday after they completed the necessary number of novel trials whose difficulty was similar to those used in this study.

Eye tracking data recordings
Eye tracking data was collected and visual stimuli were presented in the same manner as was done with human participants; however, during the experiment, macaques sat within a chair apparatus in a closed cubicle and were able to freely move their head. During calibration, fixation on each calibration point was rewarded with juice or a preferred food reward (controlled manually) through a spout. The macaques rapidly learned (1-5 sessions, minimum 13 calibration trials) to hold their head in a position that placed their eyes within the tracker camera's view, as juice was only delivered when the eye could be tracked. As before, viewing was binocular but only the left eye position was tracked.

Behavioral procedure
The experimental task used was identical to the one shown in Figure 2, except that no verbal report screen was shown. Three sessions were held: the training session took place on day 1, and two follow-up testing sessions took place on days 2 and 4 (1 and 3 days after the training session, respectively). The second testing session took place 4 days later, as opposed to 1 month later as in humans to avoid interference from other experiments with visual stimuli in which the macaques were involved during that 1 month period. Two sets of 20 images were used per session. Juice was given as a reward after each trial once the macaque fixated on the target for either 0.75 or 1 s (the required fixation duration was decreased for two macaques who tended to refixate outside the target at 1 s). These threshold values are near the human dwell times toward manipulated objects (Ryan and Cohen, 2004), and lie far outside of these macaques' dwell times during scanning of these images (99th percentile for each of the 4 macaques, data not shown), making it highly unlikely that the macaques would unintentionally "trigger" the fixation threshold within the target window. Macaques had a maximum time of 60 s to find the changing target, after which it was made visible by presenting the original and modified images without an interleaving gray screen, and no juice was given.

Eye tracking data and search time analysis
As with the human participants' data, fixation data and search times were analyzed using MATLAB. The search time required to locate the target object was defined as the time of initial fixation on the screen after the trial started to the time of fixation on the target, lasting for at least 0.75 or 1 s. Whereas humans were instructed to look at the screen and behaved accordingly, macaques occasionally looked away from the screen; thus, a few additional adjustments were made. For trials where macaques were not viewing the screen for at least 50% of the trial, the data were discarded as a misbehave trial and were not included in the calculations. In total, 27 out of 400 trials during the test sessions across four macaque subjects (80/80/80/160 test trials presented to the respective monkeys); individual number of misbehave/total trials were 15, 10, 1, and 1. In addition, for the one macaque subject that completed the additional experimental run, a "lookaway" contingency was added to the stimulus presentation code. Whenever the left eye was determined to be looking either outside of the screen window or if no eye signal was present at all (i.e., if the macaque was misbehaving by choosing not to look at the screen at all), the monitor went blank and the experiment put on pause until the Frontiers in Behavioral Neuroscience www.frontiersin.org eye signal returned to the screen. Hence, the total looking time could be kept at a constant maximum time of 60 s, and no "misbehave" trials would have to be thrown out. Data analysis compared the median search times of novel versus repeated trials, as well as between human and macaque search times in those respective categories.

EXPERIMENT 3: AMNESIC PATIENT D. A. AND CONTROLS
Subjects and stimuli D. A., a 58-year-old, right-handed amnesic person with 17 years of education participated in this study. He suffered extensive bilateral MTL damage that included his hippocampus and perirhinal, entorhinal, and parahippocampal cortices, due to contracting viral encephalitis in 1993. His more extensive right-sided damage extends to his anterior temporal cortex. Volume loss was also observed in other right hemispheric regions, including the posterior temporal, ventral frontal, and occipital regions, as well as the anterior cingulate. A detailed description of this case is provided in Rosenbaum et al. (2008). D. A.'s performance was compared to that of five male, right-handed controls ages 48-56 years with no known history of neurological or psychiatric disorders [average age 51.80 (2.86) years, education level 19.60 (6.65) years mean (SD)]. All participants had normal or correctedto-normal vision and were paid $10/hour for their time. Written informed consent was obtained, and the study was conducted in accordance with the ethical guidelines set by the York Human Participants Review Subcommittee. The stimuli were identical to those used in Experiment 1, except that only a subset of 40 images of the original 60 images per testing session were used.

Behavioral procedure
As in Experiments 1 and 2, there were three sessions in total: the first follow-up testing session was held after the initial training session following a 5-min break, and the second testing session took place 24 h later. Participants were required to fixate on the target for at least 1 s and had up to 40 s to find the changing object, after which the object was revealed. Because we were concerned about floor effects for memory in our older-adult control participants and in D. A., we reduced the retention delays and trial duration before revealing the change. Based on the search times from Experiment 1, it was clear that 1 s was well beyond the fixation durations of our participants, therefore there was no concern for accidentally triggering a "found" location. It was also clear that 40 s was already at asymptotic levels for finding the target, and reducing trial duration would allow participants to be retested more quickly after learning. Note that in this experiment we are not making comparisons between young and old adults, but between D. A. and his control group, all of whom experienced the same trial delays and contingencies. As was done for three participants in Experiment 1, the test questions appearing after each trial were: "Had you seen this picture before?" and "If so, did you remember where the object was? Did you remember what the object was?" to clarify differences between memory for location compared to identity of objects. If participants responded yes to knowing either what and/or where the target object was, the image was classified as remembered. On testing sessions, the length of the verbal report screen was not fixed to give participants extra time to report their answers as needed, and the experimenter initiated the next trial by pressing a key when participants were ready. In all aspects other than those just described, the procedure was identical to Experiment 1.

Data analysis
Of the 80 trials completed by the 5 control participants, one trial was removed for three of them and two trials for a fourth, due to tracker error. For D. A., only one trial of 80 was discarded. Search time corrections and data analyses were performed as described Frontiers in Behavioral Neuroscience www.frontiersin.org in Experiment 1, with the exception that D. A.'s search times were compared with those of the controls using a modified t test procedure that treats an individual as a sample, for comparison with a small normative sample that is treated as statistics rather than as population parameters, using the following formula (Crawford and Howell, 1998): where X 1 is the individual's score, X 2 is the mean score in the control sample, s 2 is the SD of scores in the control sample, and N 2 is the number of people in the control sample. This method determines if an individual's score is significantly lower than the control sample, and estimates the percentage of a control population that would score below the individual.

Additional testing session
Due to the fact that D. A. expressed no explicit recollection of any of the images shown in the experiment just described, an additional testing session was conducted 1 month later to see if any recall could be observed with fewer trials and over shorter delays (i.e., within seconds to minutes of initial presentation). Five sets of 12 images were shown. In each set, the first 4 images were novel, and the next 8 images were composed of half novel and half repeated (from the first 4 images) stimuli. Search times of novel and repeated images were pooled together and analyzed as described above.

PREDICTING EXPLICIT RECALL FROM SEARCH TIMES
We sought to test whether search times could be used to predict if a target was remembered or forgotten. Given the results from the previous experiments, search times that were shorter than a specified short search time threshold should typically belong to trials with remembered targets; search times that were longer than a specified long search time threshold should typically belong to trials with forgotten targets. Accuracy of the short search time threshold was indicated by the percentage of trials under threshold in which the target was reported as explicitly remembered (remembered trials) compared to the target not being recalled (forgotten trials). Accuracy of the long search time threshold was indicated by the percentage of trials above threshold that were forgotten compared to those that were remembered. The short search time threshold was calculated from the distribution of repeatedtrial search times, at the inflection point after the first mode, for all participants in Experiments 1 and 3. To determine the inflection point, smoothing was conducted on the discrete search time data in MATLAB ("ksdensity") using a Gaussian kernel of optimized bandwidth, similar to smoothing operations used for single unit data (spike-density functions) and imaging data. This operation provides a continuous probability estimate from discrete data, permitting the inflection point calculation. The long search time threshold was set to 30 s for all participants, which was well beyond the first mode for repeated trials and thus unlikely to include remembered trials [note that we found similar predictive ratios when using a range of long search time thresholds, from 20 to 40 s (data not shown)].
Some changing targets could, in principle, be more easily detected than others, leading to rapid novel and repeated search times, irrespective of memory. When a trial's initial presentation led to very rapid detection (defined as search times under each individual's repeated median search time [mean (SD) median search time: 6.45 (2.89) s; mean (SD) number discarded: 7(5) trials], the repeated-trial data were discarded from this analysis. This helped to mitigate ceiling effects whereby a repeated target would be located rapidly due to some type of inherent salience and not memory.
Examination of each individual's responses revealed that only one participant had very few remembered trials, presumably leading to a search time distribution that did not display the characteristic skewed repeated search time mode compared to that of novel search times. Consequently, this participant's short search time threshold (32.83 s) was 2 SD above the mean of the logtransformed data, and exceeded the long search time threshold. Based on these factors (i.e., insufficient number of remembered trials, overlapping novel and repeated search time distributions, and a prohibitively long search time threshold), this participant was excluded from the accuracy proportions reported.
For macaques in Experiment 2, search time thresholds were ascertained using the inflection point of the kernel smoothed distributions, as described for human participants, and compared to data from humans.

Eye scan paths
Example scan paths of a remembered trial are displayed in Figure 2A.
Search time differences were also evident across memory category (Figure 3). The median search times of all novel, Frontiers in Behavioral Neuroscience www.frontiersin.org remembered, and forgotten trials did not differ across the two testing sessions (day 2 and day 30), and therefore were pooled for all analyses. For comparison with macaques, median search times of trials with found targets were analyzed. Half of the repeated trials' targets were found within 6.26 s, compared to 10.04 s for novel-trial targets ( Figure 3A). Faster search times on repeated trials can be attributed to the speed of detecting remembered targets, half of which were found within 3.70 s, compared to 9.89 s for forgotten targets ( Figure 3A). Within forgotten-target trials, half of forgotten (familiar) trials were found within 7.08 s, while half of forgotten (unfamiliar) trials were detected within 10.88 s. The two types of forgotten-target trials were not significantly different [t (11) = 0.94, p > 0.35]. Median search times for novel, remembered, and forgotten trials, including all targets (found or missed), are displayed in Figure 3B. Search times differed across novel, remembered, and forgotten trials [F (2,22) = 52.73, p < 10 −10 ]. Targets on repeated trials were found faster than those on novel trials [t (11) = 12.36, p < 10 −7 ], and this effect was dominated by search times for explicitly remembered targets [remembered < forgotten, t (11) = 6.30, p < 10 −4 ; remembered < novel, t (11) = −13.17, p < 10 −7 ]. For trials in which the target was forgotten, there was no difference in search times for forgotten (familiar) and forgotten (unfamiliar) trials [t (11) = 0.94, p > 0.35]. There was, however, a significant repetition effect, even when the target was not explicitly recalled (forgotten < novel, t (11) = −3.16, p < 0.01). Finally, of the low number of false-positive remembered and forgotten (familiar) trials reported, the median search times were 4.42 (2.55) and 29.42 (21.83) s [median (SD)], and of the nine subjects who reported at least one false-positive remembered target, false-positive remembered trials were found significantly faster than novel trials [t (8) = −6.69, p < 0.001].
The time required to locate the target upon initial presentation influenced the chances of later recall. Forgotten targets-and-scenes were initially found faster during the training session [novel search time: 7.04 (3.86) s, median (SD)] compared to the initial search times of remembered targets [novel search time: 22.73 (9.34) s, median (SD); t (11) = 5.59, p < 0.001] or forgotten (familiar) trials [novel search time: 15.84 s, median (SD); t (11) = 3.01, p < 0.05].
The effect of changes in location or presence on search times across memory categories were also examined (due to the low number of color change trials, these trials were not analyzed). For novel trials, no difference in search time was found for changes in location [14.65 (9.42) s, median (SD)] or presence [15.70 (4.64) s, median (SD); t (11) = 0.34, p > 0.70]. Search times on repeated trials for changes in presence [7.34 (3.12) s, median (SD)] were longer than for location [4.35 (3.00) s, median (SD); t (11) = 3.50, p < 0.01], although when broken down into remembered, familiar, and forgotten trials, no differences were seen [t s (4-38) < 1.63, ps > 0.10].

Eye scan paths
An example of the eye scan paths on a repeated-trial whose search time was under the short search time threshold is shown in Figure 2B.

Behavioral results
For macaques, during the training and two testing sessions, 64 (5.0), 83 (10), and 70 (15)% [mean (SD)] of targets were found prior to the 60-s time limit, respectively. The proportion found targets did not differ between novel and repeated trials [χ 2 (1, n = 143) = 0.11, p > 0.70]; however, the search times for repeated trials were faster than for those for novel trials [t (3) = 3.48, p < 0.05]. Similar to humans, repeated trials were found approximately twice as fast as compared to novel trials (from 7.56 to 14.53 s, respectively; Figures 4A,B). Here, search times were collapsed across day 2 and day 4 trials (the difference between the two testing sessions was not significant), and search times of the macaque who completed two experimental runs were collapsed.

Behavioral results
For all participants (including D. A.), during the training and two testing sessions (5 min later and day 2), 59 (8.6), 65 (11), and 69 (4.1)% [mean (SD)] of targets were found before the 40-s time limit, respectively. Across the two testing sessions on consecutive days for control participants, no difference in the number of remembered and forgotten trials were found, nor in the number of false-positive remembered and forgotten (familiar) trials, although there was a trend toward fewer remembered images on day 2 [t (4) = 2.26, p = 0.087].
In control participants, the proportion of found targets was greater for repeated trials than for novel trials [χ 2 (1, Frontiers in Behavioral Neuroscience www.frontiersin.org A. reported no explicit memory for scenes or target objects; all trials were classified as forgotten. Median search times of control participants and D. A. across memory categories is shown in Figure 5A. For D. A., median (SD) search times for novel and repeated trials were 26.47 (15.43) and 24.76 (14.42) s, respectively. D. A.'s repeated search times were longer than those of controls [t (4) = 2.88, p < 0.05], whereas novel search times between D. A. and control subjects did not differ [t (4) = 0.37, p > 0.70] (Figure 5B).
In contrast, the median novel and repeated search times in control participants were 30.81 (10.27) and 9.15 (4.95) s [median (SD)], respectively. Within repeated trials, median search times for remembered and forgotten trials were 4.38 (2.34) and 25.32 (8.84) s [median (SD)]. For targets that were forgotten, median search times for forgotten (familiar) and forgotten (unfamiliar) were 29.41 (6.56) and 16.78 (5.59) s. Forgotten (familiar) trials were found faster on day 2 compared to 5 min later [t (4) = 3.37, p < 0.05]. There was also a trend toward remembered trials being found faster on day 2 compared to 5 min later [t (4) = 2.46, p = 0.087]. Of the few false-positive remembered and forgotten (familiar) trials reported, the median search times were 6.12 (3.93) and 22.35 (8.23) s [median (SD)].

FIGURE 5 | Behavioral performance of amnesic patient D. A. in Experiment 3. (A) Median search times and (B) cumulative frequency of D.
A. compared to controls. For (A), conventions as in Figure 3B. D. A.'s performance is indicated by the red diamonds; all repeated trials were forgotten. In (B), the characteristic leftward shift in repeated trials observed in healthy humans and macaques is not apparent in D. A.'s search times. Only trials found within the 40-s time limit are shown here.
Search times varied across novel, remembered, and forgotten trials [F (2,8) = 13.77, p < 0.001]. Similar to the younger adults in Experiment 1, remembered trials were found faster than both forgotten trials and novel trials [v. forgotten: t (4) = 5.59, p < 0.01; v. novel: t (4) = 6.07, p < 0.01]. Note that whereas the older control subjects took longer to find novel and forgotten images than the younger adults [novel: t (15) = 4.47, p < 0.001; forgotten: t (15) = 4.59, p < 0.001], they found remembered targets just as quickly [t (15) = 0.16, p > 0.85]. Overall, repeated trials were still found faster than novel trials [t (4) = 4.88, p < 0.01]. Forgotten (familiar/unfamiliar) and forgotten (familiar) search times were not significantly different from novel search times [t (4) = 0.76, p > 0.45; and t (4) = 0.21, p > 0.8, respectively], but there was a trend for forgotten (unfamiliar) compared to novel trials [t (3) = 2.39, p = 0.097]. Finally, the inter-participant and inter-age-group differences in novel versus forgotten search times led us to compare the relationship for this control group between age and search times. We found no linear relationship [novelforgotten search time, age: r (5) = 0.71, p > 0.15] though the two youngest participants had the shortest novel search time and the longest forgotten search times.

Additional testing session search times in Experiment 2
A follow-up experimental session was conducted to see if D. A. would report any explicit target memory if the memory demands were made easier. For blocks in which only four training trials Frontiers in Behavioral Neuroscience www.frontiersin.org occurred immediately before testing began, D. A. still reported all repeated images as forgotten (all unfamiliar). In both novel and repeated trials, he found 75% of the targets before the maximum time. Although median search times for novel and repeated trials were generally fast, they were also quite variable, and did not differ significantly between novel and repeated groups [novel: 15.17 (15.33) s and repeated: 10.01 (15.53) s (median (SD)); t (58) = 0.57, p > 0.55].

PREDICTING REMEMBERED AND FORGOTTEN TRIALS
The shift in repeated trials toward shorter search times was driven by trials whose target was explicitly remembered (Figure 6A). We asked how predictive are short search times of remembered targets and, conversely, how predictive are long search times of forgotten targets, with the goal of extracting distributions that would be biased toward remembering or forgetting, but without requiring verbal report. On repeated trials, individual subjects' search time distributions had an initial mode that peaked under 10 s, and a second smaller mode at the maximum trial time of 60 s (see Figure 6A for an example). When repeated search times were further divided into remembered and forgotten images, the number of remembered images at the first mode was greater than the number of forgotten images. This is in contrast to the second mode, where the proportion of forgotten images was greater than the proportion of remembered images. In addition, the distribution of forgotten trials' search times was similar to the distribution of novel search times, irrespective of scene familiarity.
To establish a search time threshold using these distributions, the inflection point of the first mode was taken to encompass the majority of trials under the first mode. Across human subjects in Experiments 1 and 3, the average search time threshold was 8.80 (5.11) s, under which 74.03 (17.80)% of trials were remembered targets. Within the remaining forgotten trials, 17.50 (17.03)% were from trials with familiar scenes and 8.48 (6.97)% were completely forgotten, median (SD; Figure 6B).
To identify trials that were likely forgotten, a long search time "forgetting" threshold was set at 30 s for all participants. On average, 88.31 (14.96)% [median (SD)] of trials above this threshold were comprised of forgotten targets (36.53% forgotten (unfamiliar), 51.78% forgotten (familiar). This was based on individual repeated search time distributions and could thus be applied to repeated search times in monkeys, to determine whether individual trials in monkeys were likely remembered or forgotten.
In macaques, the inflection point thresholds identified 62.43 (20.03)% of repeated trials [mean (SD)]. If we assume the results seen in humans apply here, 74% of these trials were remembered, or 40/54 trials correctly classified remembered trials. Similarly, using the 30-s "forgetting" threshold, 23.75 (13.77)% [mean (SD)] of repeated trials were selected. If we assume an 88% accuracy rate as seen in humans, this would yield 34/39 trials correctly classified as forgotten. The effect size based on the odds ratio is 21 [(short search time remember/forget ratio)/(long search time remember/forget ratio)]. Determining whether this is sufficient depends on the underlying two distributions.

DIFFERENCES IN REMEMBERING IDENTITY OR LOCATION OF OBJECTS
In the older adults in Experiment 3 and three younger adults in Experiment 1, participants distinguished between remembering what and where the target object was (data from younger and older participants as well as across sessions were collapsed, as no differences in search times of the different types of remembered trials were found [t s

DISCUSSION
This study demonstrates the utility of a novel test of object-inscene memory that includes long-term retention following single exposures. We tested across species -in macaques and humansand across MTL function -in an amnesic and healthy humans. Using a flicker change detection task, humans found explicitly remembered targets faster than forgotten or novel-trial targets. This positively skewed the distribution of search times on repeated trials compared to that of novel trials (i.e., short search times occurred more frequently on repeated trials). The increased occurrence of very rapid search times could be attributed to explicit memory of the target object. Though not as clear as the relation between search times and explicit effects, we found one measurement that was sensitive to implicit memory: targets were more Frontiers in Behavioral Neuroscience www.frontiersin.org likely to be found on repeated trials than on novel trials, even when the target object and scene had been forgotten. Overall, the influence of memory on search times was robust across younger and middle-aged adults. We were able to identify remembered and forgotten trials using only the repeated search time distributions of a given individual; for a threshold delimiting the early skewed peak, 74% of trials were remembered and for a threshold delimiting long search times, 88% of trials were forgotten. Individuals who showed little to no target memory also showed no shift in repeated search times (one healthy, young adult participant and D. A.). In macaques, search times from repeated trials displayed the same characteristic positively skewed peak compared to those of novel trials, equivalent to what was observed in healthy human participants. This suggests that, like humans, faster search times in macaques are associated with memory for the target objects in this task. We used the same threshold methods to derive search time thresholds in macaques. Assuming that the common shift in search times across species reflects a common process, this technique allowed us to identify predominantly remembered/forgotten trials in macaques. Finally, an amnesic patient showed no memory for repeated targets and had unaffected search times with repetition -i.e., he showed neither explicit nor implicit memory on this task. These findings, in conjunction with previous literature measuring amnesic performance on change detection tasks, suggest that the improvements in search times in healthy participants (and the humans' explicit recall) are likely to depend on MTL function. As such, this task may be a new tool to measure MTL-dependent memory across species.

AMNESIC PROFILE
In the current study, D. A. showed search times on novel images that were indistinguishable from those of healthy controls, indicating that his ability to perform the standard flicker change detection task was unimpaired. But in striking contrast to controls, he reported no memory for any of the scenes or target objects, and he showed no benefit in search times on repeated trials; his repeated and novel search time distributions were indistinguishable. D. A.'s previous performance on tests of anterograde and retrograde episodic memory is generally below average, despite preserved abilities based on standard neuropsychological tests of other cognitive functions (Rosenbaum et al., 2008) and on other memory tasks such as the transverse patterning task (Ryan et al., 2009). Generally, his deficits in this task are consistent with his anterograde amnesia.
In other studies of amnesic patients, relational memory may be impaired not only over the long-term, but during short-term intervals as well (Graf and Schacter, 1985;Gabrieli et al., 1998;Chun and Phelps, 1999;Hannula et al., 2006). In support of this finding, during the follow-up testing session in which repeated trials were viewed within minutes of their initial presentation, D. A. still did not report any recognition and again, no differences in his search times were observed. This is in contrast to evidence of some preservation of memory in amnesics for another change detection task when the change was presented within ∼2 s following initial scene presentation and no stimuli intervened (Ryan and Cohen, 2004). One difference may be the retention intervals involved.
Patients with MTL lesions have shown deficits in visual working memory (WM) tasks, provided the retention intervals exceeded 1-to 2-s (Nichols et al., 2006;Olson et al., 2006). Indeed, electrophysiological data suggest different underlying memory processes for 1 and 6 s retention intervals (Bankó and Vidnyánszky, 2010). In addition, our patient was required to retain memory for the object over intervening trials. Finally, other studies have used several gaze measures of change detection that differ from the search times used in the present study. Nevertheless, it would appear that the present findings are relevant to current ideas about declarative/relational memory that can operate at relatively brief delays, and including intervening stimuli.
We are cautious in generalizing data from one patient; variability in memory performance in various tasks has been observed among patients with similar or overlapping lesion profiles (for example, Holdstock et al., 2008). Moreover, numerous studies of amnesics with selective MTL lesions provide evidence of the role of the MTL in memory performance, but it is important to note that D. A. has additional damage outside of his MTL that cannot be discounted, notably in his right anterior temporal cortex. For this task, at least, D. A. showed a profound memory impairment, with no sign of any implicit search time benefit for repeated stimuli. Moreover, one healthy young participant consistently reported forgetting, and indeed, had a repeated search time distribution that resembled that of the novel distribution, further suggesting the link between memory and short search times. More data will be needed to clarify whether and how the hippocampus and surrounding MTL and temporal lobe structures contribute to the explicit memory and search time measures on this task.

DECLARATIVE/RELATIONAL MEMORY FOR TARGETS-IN-SCENES
Although the macaques could not explicitly report which images and/or targets they remembered, the same search time profiles seen across species hint at a common underlying memory process. In change detection experiments using natural scenes, humans rely on contextual cues for object detection (Brockmole et al., 2006;Hollingworth, 2006;Torralba et al., 2006;Becker and Rasmussen, 2008). The relational account of memory proposes that the hippocampus is responsible for encoding relations among multiple items. In the flicker change detection memory paradigm, this would involve recalling the embedded changing target object in relation to surrounding items within the scene. Successful performance may be hippocampally dependent, given the difficulty D. A. showed, and based on other findings of deficits in patients with more restricted hippocampal damage. In healthy humans, explicitly reported memory led to the fastest recall, though this could be the "downstream" effect of stronger relational memory, and not the conscious act of recalling. To determine which structures are critical for relational memory formation, future studies could test macaques or humans with more precise lesions within and around the hippocampus using the task described here.

EYE MOVEMENTS REVEAL MEMORY
In lieu of verbal report, our task uses the speed of target fixation as an indication of memory. For decades, gaze direction and/or duration have been used as dependent measures in non-declarative Frontiers in Behavioral Neuroscience www.frontiersin.org settings, such as with infants (Fantz, 1963;Fagan, 1970;Cohen and Strauss, 1979;Richmond et al., 2007) and with macaques (Bagshaw et al., 1970a,b;Wilson and Goldman-Rakic, 1994;Pascalis and Bachevalier, 1999). Gaze can signal item and relational memory, whereby novel stimuli or relations among stimuli are viewed longer than familiar stimuli (see Hannula et al., 2010, for review). In adult humans, changes in eye movement patterns toward scenes have been linked to memory. Some groups suggest this results from explicit or declarative memory (Smith and Squire, 2008), others suggest awareness is not essential (Ryan et al., 2000;Hannula and Ranganath, 2008), and a third possibility is that participants report explicit recall as a consequence of rapid search and not the other way around (though we only saw false positives that would support this assertion in about 2% of novel images).
Other features of eye movements may shed light on memory processing. Conscious memory for familiar scenes leads to fewer sampled regions and fewer fixations across the entire image both at study (Sharot et al., 2008) and at test (Smith and Squire, 2008). On a (non-flicker) change detection task, in which repeated images were altered, Smith and Squire (2008) found that participants made more eye movements to where the change occurred, but only when they explicitly reported knowledge of the change. In contrast, Ryan et al. (2000) demonstrated on a similar change detection task that people look more at the critical changed region even when they are not aware of it. Amnesic patients did not display this behavior for longer delays of minutes, but did show recall when the manipulation followed the original image with no intervening stimuli (Ryan and Cohen, 2004). Furthermore, on a face-scene association task, Hannula and Ranganath (2009) found that implicit relational memory was linked to eye movements, and this behavior was predicted by hippocampal activity independent of explicit recall. Differences in the paradigms may explain the discrepancies between explicit and implicit memory related to eye movements. Further research is required to determine what specific types of memory may be revealed through various aspects of eye movements, and how different experimental settings may preferentially lead to implicit or explicit memory-related eye movements. In our study, we were interested in how quickly a changing target was located. Notably, the "change" was present in novel and repeated trials alike; we expected that if repeated trials were remembered, the change would be obvious and therefore fixated rapidly relative to novel or forgotten trials. For this task, using this measure, there was a strong correspondence between remembered trials and fast search times, which could be used in cross-species studies of memory.

USING THE FLICKER CHANGE DETECTION TASK TO EVALUATE MEMORY FORMATION ACROSS SPECIES
This task fills a unique niche as a memory task for use in macaques: it is a non-verbal measure of object-in-scene memory that does not require extensive training and can be evaluated and sorted on a single-trial basis. The distribution of search times in a given session can be taken as an indicator of overall retention: poor retention will lead to a repeated search time distribution that resembles that of novel trials, whereas good retention will be accompanied by a prominent skewness in the repeated search time distribution. This measure can also be used to determine what types of activity during encoding predict memory (or forgetting), on the basis of individual-trial outcomes (in humans: Brewer et al., 1998;Wagner et al., 1998;Fell et al., 2001;Paller and Wagner, 2002;Sederberg et al., 2007;and in macaques Jutras et al., 2009). Moreover, memory success or failure may be manipulated by altering the duration of scene viewing. Study duration is known to influence scene memory (Melcher, 2006) and the current findings revealed that forgotten trials were predicted by short encoding durations and remembered trials were predicted by long encoding durations. Both a posteriori trial sorting and a priori duration manipulations are relevant for investigations of the neural bases of memory formation, such as memory trace reactivation (Hoffman and McNaughton, 2002). To date, there is little evidence linking the neural phenomenon of trace reactivation to subsequent memory recall (for review see Hoffman et al., 2007). Using the same rationale applied to predictors during encoding, one could evaluate whether trial-specific neural activity observed after encoding -such as memory trace reactivation -is related to subsequent memory strength. In sum, this task allows object-in-scene memory to be measured on a trial-by-trial basis over long retention intervals, providing a means of determining which neural structures, and what neural mechanisms, predict long-term memory formation.