The Frontiers in Neuroscience journal series is the 1st most cited in Neurosciences

Original Research ARTICLE

Front. Syst. Neurosci., 26 May 2008 | https://doi.org/10.3389/neuro.06.001.2008

Recollection- and familiarity-based decisions reflect memory strength

1
Institute of Neuroinformatics, University and ETH Zurich, Zurich, Switzerland
2
Institute of Neuroradiology, University of Zurich, Zurich, Switzerland
We used event-related fMRI to investigate whether recollection- and familiarity-based memory judgments are modulated by the degree of visual similarity between old and new art paintings. Subjects performed a flower detection task, followed by a Remember/Know/New surprise memory test. The old paintings were randomly presented with new paintings, which were either visually similar or visually different. Consistent with our prediction, subjects were significantly faster and more accurate to reject new, visually different paintings than new, visually similar ones. The proportion of false alarms, namely remember and know responses to new paintings, was significantly reduced with decreased visual similarity. The retrieval task evoked activation in multiple visual, parietal and prefrontal regions, within which remember judgments elicited stronger activation than know judgments. New, visually different paintings evoked weaker activation than new, visually similar items in the intraparietal sulcus. Contrasting recollection with familiarity revealed activation predominantly within the precuneus, where the BOLD response elicited by recollection peaked significantly earlier than the BOLD response evoked by familiarity judgments. These findings suggest that successful memory retrieval of pictures is mediated by activation in a distributed cortical network, where memory strength is manifested by differential hemodynamic profiles. Recollection- and familiarity-based memory decisions may therefore reflect strong memories and weak memories, respectively.

Introduction

Functional brain imaging studies have shown that medial temporal, parietal and prefrontal cortices are involved in recognition memory of prior episodes (Rugg and Wilding, 2000 ; Rugg and Yonelinas, 2003 ). The functional role that these regions play in memory retrieval, however, is still debated. Specifically, it is unclear whether recollection, the retrieval of specific information about a past experience, and familiarity, a sense that an event has been previously experienced (Tulving, 1985 ), are mediated by dissociated neural systems or separate strong memories from weak memories. Some studies suggest that separate cortical networks (Yonelinas et al., 2005 ) and differential activation in parietal cortex (Vilberg and Rugg, 2007 ) mediate these two distinct memory processes, whereas other studies suggest that recollection and familiarity reflect differences in the strength of a common memory trace (Donaldson, 1996 ; Dunn, 2004 ; Gonsalves et al., 2005 ; Squire et al., 2007 ; Wixted, 2007 ).
In most episodic retrieval studies written words have been used as stimuli (e.g., Cabeza et al., 2001 ; Henson et al., 1999 ) and the neural correlates of retrieving pictures from memory are not fully understood. We have recently conducted a series of event-related fMRI studies to investigate the neural mechanisms of retrieving pictorial information from long-term memory and the effects of visual similarity between old and new pictures. Subjects memorized paintings (Yago and Ishai, 2006 ) or unfamiliar Asian faces (Ishai and Yago, 2006 ) and 4 days later performed an old-new recognition memory task in the MR scanner. The old pictures were presented with new ones that were visually similar, somewhat similar or visually different from the old paintings. Consistent with our hypothesis, subjects were slower and less accurate to reject new, visually similar paintings. We found activation in a distributed cortical network that included face- and object-selective regions in the visual cortex, as well as prefrontal areas where the old paintings evoked stronger activation than the new ones, regardless of their visual similarity. Moreover, activation elicited by new paintings in parietal cortex was reduced with decreased similarity to the old items, whereas in the hippocampus and precuneus, stronger responses were evoked by the new, visually different paintings. We concluded that recognition memory is mediated by classification of new items as a match or a mismatch, based on their degree of visual similarity to old items (Ishai and Yago, 2006 ; Yago and Ishai, 2006 ). Our old-new task, however, did not address the issue of memory processes and the extent to which the observed behavioral and neural responses were due to recollection- or familiarity-based memory decisions.
To test whether recollection and familiarity judgments were influenced by the degree of visual similarity between old and new pictures, we used event-related fMRI with portraits, landscapes and abstract compositions by painters with a unique style (Figure 1 ). Based on our previous study, we predicted that subjects would correctly reject more new paintings that were visually different from the old ones, and would make more false alarms to new, visually similar paintings. Furthermore, we hypothesized that if recollection and familiarity judgments about complex pictures reflect strong memories and weak memories, respectively, correctly remembered items would be associated with stronger neural activation than known items.
Figure 1. Stimuli and tasks. During study, subjects were presented with portraits, landscapes and abstract paintings (shown from top to bottom are examples from Renoir, Pissaro and Kandinsky) and performed a flower detection task. Ten minutes later, subjects performed a surprise Remember/Know/New memory retrieval test in which the paintings from the study phase (“old”) were randomly presented with new paintings that were either visually similar or visually different.

Materials and Methods

Subjects

Twenty-one normal, right-handed subjects (11 females, mean age 25 years) with normal or corrected to normal vision participated in the study. All subjects gave informed written consent for the procedure in accordance with protocols approved by the University Hospital of Zurich. The subjects, students from the University of Zurich, had no formal art education and reported visiting art museums once a year or less. Post-scan questionnaires revealed that all subjects were unfamiliar with the paintings and had not seen them prior to the experiment.

Stimuli and Tasks

Stimuli were displayed using Presentation (www.neurobs.com, version 9.13) and were projected with a magnetically shielded LCD video projector onto a translucent screen placed at the feet of the subject. During the study phase, subjects performed a flower detection task on 20 portraits by Modigliani and Renoir, 20 landscapes by Pissarro and Van Gogh and 20 abstract compositions by Kandinsky and Miro (Figure 1 ). Each painting was presented for 3 s and subjects had to indicate whether it includes any flowers by pressing one of two buttons. In each of the three time series collected during the study phase, paintings (4 epochs of 15 s each) alternated with 15-s fixation epochs.
We then collected the anatomical images and after this 10-min scan, subjects performed a surprise memory test, in which the 60 old paintings were randomly presented with 60 new paintings. The new paintings were either visually similar or visually different from the old paintings. The degree of visual similarity between the new paintings and the old ones was assessed in a separate behavioral pilot (for details see Yago and Ishai, 2006 ). Each painting was presented for 3 s and subjects had to make a Remember/Know/New decision by pressing one of three buttons. Subjects were instructed to respond “Remember” if they were absolutely convinced that the picture was presented during the flower detection task; “Know” if the picture seemed familiar; and “New” if the picture was not presented in the flower detection task. In each of the three time series collected in the test phase, paintings (8 epochs of 15 s each) alternated with 15-s fixation epochs. In both study and test, each run included one category of paintings (i.e., portraits, landscapes or abstract compositions).

Data Acquisition

Data were collected using a 3T Philips Intera whole body MR scanner (Philips Medical Systems, Best, The Netherlands). Changes in blood-oxygenation level-dependent MRI signal were measured by using sensitivity encoded gradient-echo echoplanar sequence (SENSE, Pruessmann et al., 1999 ) with 35 axial slices, TR = 3 s, TE = 35 ms, flip angle = 82°, field of view = 220 mm, acquisition matrix = 80 × 80, reconstructed voxel size = 1.72 mm × 1.72 mm × 4 mm and SENSE acceleration factor R = 2.
High-resolution spoiled gradient recalled echo structural images were collected in the same session for all the subjects (180 axial slices, TR = 20 ms, TE = 2.3 ms, field of view = 220 mm, acquisition matrix = 224 × 224, reconstructed voxel size = 0.9 mm × 0.9 mm × 0.75 mm). These high-resolution anatomical images were collected after the study phase and provided detailed anatomical information for the region-of-interest (ROI) analysis and for 3D normalization to the Talairach and Tournoux (1998) atlas.

Data Analysis

Accuracies and reaction times were computed for each subject, category of paintings (portraits, landscapes, abstract paintings) and response type (Yes/No during the flower detection task; Remember/Know/New during the memory retrieval test). ANOVA was used to compare the various conditions.
Functional MRI data were analyzed in BrainVoyager QX Version 1.8 (Brain Innovation, Maastricht, The Netherlands). All volumes were realigned to the first volume, corrected for motion artefacts and spatially smoothed using a 5-mm FWHM Gaussian filter. The main effects during the study and test were analyzed using multiple regression (Friston et al., 1995 ). Based on the contrast of paintings vs. fixation, a set of ROIs was anatomically defined for each subject with clusters that showed a significant effect (p < 0.0001, uncorrected). These regions included the inferior occipital gyrus (IOG), fusiform gyrus (FG), dorsal occipital cortex (DOC), intraparietal sulcus (IPS), inferior frontal gyrus (IFG), insula and the anterior cingulate cortex (ACC). The contrasts of Remember vs. Know and Remember vs. New further revealed significant activation in the precuneus and in two medial temporal lobe structures, the parahippocampal cortex (PHC) and the hippocampus. In each subject and each ROI, the mean parameter estimates were calculated separately for each response type (Yes/No during flower detection task; Remember/Know/New during memory test) and were used for between-subjects random-effects analyses.
Finally, we tested whether reaction times and fMRI activation during the study phase could predict subsequent behavioral and neural responses to the old paintings during the test phase. Thus, responses during the flower detection task were sorted based on subsequent Remember and Know judgments subjects made during the retrieval test.

Results

Study Phase

Behavioral data. In this phase subjects were presented with portraits, landscapes and abstract paintings and indicated whether each painting contained any flowers. The behavioral data collected while subjects performed the task in the scanner are shown in Figure S1 . As most paintings did not include flowers, the proportion of No responses was higher than the proportion of Yes responses [t(124) = 10.9, p < 0.00001 for portraits, landscapes and abstract paintings]. Furthermore, subjects made significantly faster Yes responses than No responses [t(120) = 3.3, p < 0.001 for portraits, landscapes and abstract paintings]. Interestingly, subjects reported seeing flowers in more than 20% of the abstract paintings, which do not depict natural objects, but rather use purely visual forms of line, color and shape. Moreover, it took subjects significantly longer to decide whether these abstract compositions contained flowers. Thus, the differences in response latencies between Yes abstract responses and Yes responses to portraits and landscapes were significant [t(36) = 4.2, p < 0.001 and t(36) = 2.9, p < 0.01, respectively], as well as the difference in response latencies between No responses to abstract paintings and No responses to portraits [t(40) = 4.4, p < 0.001].
Imaging data. The main effect, namely responses evoked by all paintings as compared with the fixation baseline, revealed activation within a distributed cortical network that included multiple visual, parietal and prefrontal regions (Figure S2 ). Significant activation was found in the DOC (mean Talairach coordinates: −28, −82, 12; 33, −82, 12), IOG (−24, −80, −3; 21, −80, −5), FG (−28, −53, −12; 28, −53, −10) IPS (−32, −51, 47; 29, −51, 44), IFG (−44, 3, 33; 43, 4, 34) and ACC (8, 7, 53). Within all regions, activation evoked by Yes and No responses during the flower detection task was virtually identical, ruling out differential effects of attention during task performance.
We then tested whether behavioral and neural responses during study could predict subsequent memory performance during test. We found that shorter response latencies for paintings with flowers predicted subsequent Remember judgments during the memory test. In terms of the neural response, we found that Yes responses during the flower detection task resulted in similar activation in the FG during subsequent Remember and Know judgments (Figure S3 ).

Test Phase

Behavioral data. Ten minutes after performing the flower detection task, subjects performed a surprise memory task. In this test phase the paintings from the study phase were randomly presented with new paintings that were either visually similar or visually different from the old ones (see Figure 1 ) and subjects made Remember/Know/New decisions. The behavioral data collected while subjects performed the task in the scanner are shown in Figure 2 . The proportion of remember responses to the old items was significantly higher than both know and new responses [t(61) = 5.8, p < 0.000001 in both comparisons]. Consistent with our prediction, the proportion of false alarms, namely remember and know responses to new items, significantly decreased with decreased visual similarity between the old and the new items [remember new similar vs. remember new different, t(61) = 5.7, p < 0.000001; know new similar vs. know new different, t(61) = 10.5, p < 0.000001]. In terms of response latencies, know decisions took significantly longer than both remember and new responses [know vs. remember, t(187) = 9.0, p < 0.000001; know vs new, t(187) = 8.2, p < 0.000001 for all paintings]. Finally, consistent with our prediction, subjects responded significantly faster to the new, visually different than to the new, visually similar paintings [t(61) = 4.9, p < 0.00001].
Figure 2. Behavioral data collected during the memory retrieval test. Left: Mean Remember/Know/New responses to old and new paintings, averaged across all subjects. Right: Mean response latencies during Remember/Know/New responses to old and new items. In this and subsequent graphs error bars indicate standard error of the mean (SEM).
Imaging data. The main effect, namely responses evoked by all paintings as compared with the fixation baseline, revealed activation within a distributed cortical network that included multiple visual, parietal and prefrontal regions (Figure 3 ). Significant activation was found in the DOC (mean Talairach coordinates: −27, −81, 11; 32, −81, 11), IOG (−29, −77, −1; 28, −77, −6), FG (−28, −55, −15; 33, −55, −14) IPS (−23, −54, 38; 27, −54, 39), IFG (−44, 3, 33; 43, 4, 34), insula (−42, 8, 5; 39, 8, 2) and ACC (2, 15, 40). Within the DOC, IOG, FG, IPS and IFG activation elicited by correctly remembered paintings was significantly higher than activation evoked by correctly know and new judgments (p < 0.000001 in both comparisons).
Figure 3. Activation evoked by portraits, landscapes and abstract paintings during the memory retrieval test. Top: coronal sections, taken from a representative subject, illustrating activation in DOC, IOG, FG, precuneus, IPS, IFG, insula and ACC. Bottom: Mean parameter estimates, averaged across all subjects and both hemispheres, for correct Remember/Know/New judgments.
We then analyzed the activation evoked by correct and incorrect responses in all ROIs (Figure 4 ). Within the IPS, hits, namely correct responses to old items, evoked stronger activation than misses (p < 0.05), correct rejection of new paintings (p < 0.01); and false alarms (p < 0.001). To test our hypothesis about visual similarity, we compared correct responses to old paintings with correct responses to new, visually similar and new, visually different items (Figure 6 ). Consistent with our previous study (Yago and Ishai, 2006 ), we found that within the IPS, new, visually different paintings evoked less activation than new, visually similar ones (p < 0.01). To further understand the effect of visual similarity on memory decisions, we compared remember responses to old items (correct responses) with remember responses to new, visually similar items (false alarms). Within the FG, the difference between these responses was not significant, however in the IPS, remember responses to old items evoked significantly stronger activation than remember responses to new, visually similar items (p < 0.02), consistent with previous reports about the role of the IPS in mediating the old/new effect.
Figure 4. Task performance and visual similarity. Top: A coronal section, taken from a representative subject, illustrates activation in the FG and IPS. Bottom: Left: Mean parameter estimates for correct responses to old items (HT); incorrect responses to old items (MS); Correct responses to new items (CR) and incorrect responses to new items (FA). Right: Mean parameter estimates for correctly recognized old and new items, as a function of their visual similarity. Parameter estimates were averaged across all subjects and both hemispheres.
We then contrasted the memory responses. Interestingly, comparing correct remember judgments with correct know judgments revealed activation predominantly in the precuneus (Figure 5 ). Analysis of the BOLD response within this region indicated not only stronger responses to remembered items, but also a BOLD latency shift, with a significantly earlier peak for recollection (5.6 s) than familiarity [10.3 s, t(40) = 3.1, p < 0.004]. Contrasting correctly remembered pictures with correct rejections of new pictures further revealed a similar pattern of activation, albeit with a lower threshold. Activation in the precuneus was stronger and peaked significantly earlier during remember (5.7 s) than new [10.2 s, t(40) = 3.37, p < 0.002] judgments. Finally, contrasting correct know responses with correct rejection of new items revealed similar hemodynamic response profiles and the peak of the BOLD response (11.1 and 10.2 s, respectively) was not statistically significant [t(40) = 0.54, p = 0.59].
Figure 5. Recollection and familiarity. Group statistical maps illustrating significant activation in the precuneus during the following memory contrasts: correct remember vs. correct know (left); correct remember vs. correct new (middle); and correct know vs. correct new (right). Bottom: Analysis of the BOLD signal indicated that correctly remembered paintings evoked responses with higher amplitude and a significantly earlier peak than both correctly known and new items. Data were averaged across all subjects.
Comparing correct remember judgments with correct know judgments also revealed activation in the medial temporal lobe, namely in the PHC and the hippocampus (Figure 6 ). In both regions, correctly remembered paintings evoked significantly stronger activation than correctly known items [t(40) = 2.75, p < 0.009 in the PHC and t(40) = 2.47, p < 0.018 in the hippocampus]. The difference between remembered and new paintings, however, was not statistically significant [t(40) = 1.55, p = 0.13 in the PHC and t(40) = 1.16, p = 0.25 in the hippocampus].
Figure 6. Activation in medial temporal lobe. Top: Group statistical map illustrating activation in the parahippocampal cortex (PHC) and the hippocampus (HIP). Bottom: Mean parameter estimates for correct Remember/Know/New responses, averaged across all subjects.

Discussion

We investigated the neural correlates that mediate recognition memory of portraits, landscapes and abstract paintings. During the study phase, subjects performed an attention demanding flower detection task. The behavioral data showed that response latencies were significantly shorter for the representational paintings, i.e., portraits and landscapes, in which familiar objects were clearly depicted. In contrast, the visual search for flowers in abstract compositions took much longer, in line with our previous studies of object indeterminacy in art paintings (Ishai et al., 2007 ). The flower detection task evoked activation in a distributed cortical network, in which paintings with and without flowers evoked virtually identical responses, ruling out differential effects of attention during encoding. Interestingly, shorter response latencies for paintings with flowers predicted subsequent Remember judgments during the memory test, suggesting that fast identification and detection of the flower targets facilitated later recollection of information about these paintings. Stronger activation in the FG for paintings with flowers predicted subsequent Remember and Know judgments, whereas reduced activation for paintings without flowers predicted subsequent Know judgments.
The surprise Remember/Know/New memory retrieval test revealed that most of the old paintings were correctly recognized and that responses to the new items depended on their visual similarity to the old ones. Consistent with our hypothesis, the proportion of false alarms, namely remember and know responses to new paintings, was significantly reduced with decreased visual similarity. Consistent with previous findings (e.g., Dewhurst and Conway, 1994 ), correct and incorrect know responses were associated with significantly longer latencies, suggesting that subjects hesitated before deciding that a picture looked familiar but they could not recollect additional information about its prior experience.
The recognition memory task evoked activation within a distributed cortical network that included similar visual, parietal and prefrontal regions to those activated during the study phase. Within DOC, IOG, FG, IPS and IFG, correctly remembered old items elicited stronger activation than both correctly known and new items. In some of these regions, greater activation for remembered than known words has been previously found (Henson et al., 1999 ), suggesting that recollection elicits enhanced activation than familiarity regardless of stimulus format. When remember judgments were directly contrasted with know judgments, a significant cluster of activation was found in the precuneus, a region implicated in many episodic memory retrieval (e.g., Fletcher et al., 1995 ; Shannon and Buckner, 2004 ; Yago and Ishai, 2006 ; Yonelinas et al., 2005 ) and visual imagery (Ishai et al., 2000a , 2002 ; Mechelli et al., 2004 ) studies. Within the precuneus, correctly remembered paintings not only elicited stronger activation than known items, but a latency shift of the BOLD response was observed, with a significantly earlier peak during recollection than familiarity-based judgments. Within the temporal lobe, activation in the hippocampus and PHC was stronger during recollection than during familiarity-based memory decisions. Our findings provide empirical evidence in support of a recent perspective according to which the Remember/Know procedure separates strong memories from weak memories (Squire et al., 2007 ; Wixted, 2007 ). It is of interest that within the PHC and the hippocampus, activation evoked by remembered paintings was not statistically significant from activation evoked by new items. Our fMRI findings are consistent with a previous report, in which activation in the hippocampus was recorded in epileptic patients using depth electrodes. Some hippocampal neurons increased their firing rate in response to old pictures, whereas other neurons signaled novelty by increased firing rate in response to new pictures (Rutishauser et al., 2006 ). Taken collectively, our findings that recollection evoked stronger activation than familiarity within multiple regions suggest that the memory decisions reflect memory strength and not independent memory processes.
Activation in parietal cortex during the memory test revealed stronger responses to the old, correctly remembered paintings than to the new items, consistent with previous ERP and fMRI studies (e.g., Curran and Cleary, 2003 ; Kahn et al., 2004 ; Wilding, 2000 ). Furthermore, activation within the IPS was reduced with decreased similarity between the new paintings and the old ones. The IPS, traditionally considered a region of the dorsal frontoparietal attention network, was implicated in many cognitive studies of attention, particularly in target detection tasks (Corbetta et al., 2000 ; Kincade et al., 2005 ; Shulman et al., 2001 ) and the segmentation of old from new items (Pollmann et al., 2003 ). Numerous recognition memory studies have further shown that posterior parietal cortex does not merely detect old items but, rather, mediates higher order cognitive processes associated with memory retrieval (Konishi et al., 2000 ; Shannon and Buckner, 2004 ; Wheeler and Buckner, 2003 ). Taken collectively, our previous (Ishai and Yago 2006 ; Yago and Ishai, 2006 ) and current findings suggest not only that the parietal cortex mediates the old/new effect, but also processes the degree of visual similarity between old and new items. These findings support the “mnemonic accumulator” hypothesis, according to which recognition memory decisions are based on the integration of sensory signals (Wagner et al., 2005 ).
Models of recognition memory assume that recollection and familiarity are independent processes during retrieval (Yonelinas, 2002 ). Evidence for such neuroanatomical dissociation came from studies in which subjects were instructed to rate their memory confidence (Yonelinas et al., 2005 ) or to indicate the amount of recollected information (Vilberg and Rugg, 2007 ). Future studies will determine the extent to which the dissociation of the neural correlates of recollection and familiarity could be generalized across various experimental paradigms. Although remember and know responses are exclusive, recollected items are also familiar ones. It is therefore highly likely that retrieval of mnemonic information about complex pictures is modulated by activation within a distributed neural system, where memory strength modulates the neural response. The redundant relationship between recollection and familiarity has been corroborated by neuropsychological (Knowlton, 1998 ) and electrophysiological (Yovel and Paller, 2004 ) studies. Our current findings suggest that the same cortical structures are activated during recollection- and familiarity-based judgments and are consistent with models of a continuum of mnemonic information on which the subject establishes a criterion (Donaldson, 1996 ; Dunn, 2004 ; Gonsalves et al., 2005 ). This criterion, as shown in our current and previous studies (Ishai and Yago, 2006 ; Yago and Ishai, 2006 ), depends on the degree of visual similarity between old and new items.
A distributed neural system for recognition memory is consistent with recent neuroanatomical findings in amnesic patients, indicating that the ability to recollect remote memories depends not only on the medial temporal lobe but on widely distributed neocortical areas in the occipital, parietal and prefrontal lobes (Bayley et al., 2005 ; Squire and Bayley, 2007 ). Furthermore, a distributed memory network is not only physiologically and ecologically plausible, but also confirms with previous fMRI studies, showing that the representation of objects and faces in the human brain is not modular, but rather distributed across a wide expanse of cortex (Haxby et al., 2001 ; Ishai et al., 1999 , 2000b ). Within this distributed neural system, the memory strength is manifested by differential BOLD responses during recollection and familiarity judgments. Future studies will determine the extent to which these two memory processes are manifested by differential patterns of effective connectivity among regions.
In summary, our results show that recollection- and familiarity-based memory decisions depend on the degree of visual similarity between old and new items. Furthermore, recognition memory of complex pictures is mediated by activation within a distributed cortical network, where remembered and known items evoke differential BOLD responses that reflect their memory strength.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgements

We thank Edward Wilding and Andrew Yonelinas for reading the manuscript. This study was supported by the Swiss National Science Foundation grant 3200B0-105278 and by the Swiss National Center for Competence in Research: Neural Plasticity and Repair.

Supplementary Material

See Figures S1 –S3 .
Figure S1. Behavioral data during the flower detection task. Mean responses (left) and reaction times (right), averaged across 21 subjects. Error bars indicate standard error of the mean (SEM).
Figure S2. Activation evoked by portraits, landscapes and abstract paintings during the flower detection task. Top: coronal sections, taken from a representative subject, illustrating activation in DOC, IOG, FG, IPS, IFG and ACC. Bottom: mean parameter estimates averaged across all subjects and both hemispheres. Yes-flower and No-flower responses evoked virtually identical activation within all ROIs. Error bars indicate standard error of the mean (SEM).
Figure S3. Study predicts subsequent memory responses. Left: Behavioral data. Paintings to which subjects responded "Yes flower" faster, were subsequently associated with remember memory decisions. The difference between Yes responses for paintings that were later remembered and Yes responses for items that were later known was significant for all paintings (p < 0.001). The differences between Yes responses for paintings that were later remembered and No responses that were later either remembered or known were also signifi cant (p < 0.001 and p < 0.0001, respectively). Longer reaction times to paintings that did not contain flowers resulted later in similar remember and know judgments. Right: Activation in the fusiform gyrus. An event-related analysis revealed that in the FG, "Yes" responses during the flower task subsequently resulted in similar activation during both remember- and know-memory decisions. Stronger activation in the FG elicited by paintings that did not include flowers resulted in subsequent remember judgments, whereas weaker activation elicited by paintings without flowers resulted in subsequent know judgments. The difference between Yes-remember and No-remember, was not statistically significant (p < 0.06).

References

Bayley, P. J., Gold, J. J., Hopkins, R. O., and Squire, L. R. (2005). The neuroanatomy of remote memory. Neuron 46, 799–810.
Cabeza, R., Rao, S. M., Wagner, A. D., Mayer, A. R., and Schacter, D. L. (2001). Can medial temporal lobe regions distinguish true from false? An event-related functional MRI study of veridical and illusory recognition memory. Proc. Natl. Acad. Sci. USA 98, 4805–4810.
Corbetta, M., Kincade, J. M., Ollinger, J. M., McAvoy, M. P., and Shulman, G. L. (2000). Voluntary orienting is dissociated from target detection in human posterior parietal cortex. Nat. Neurosci. 3, 292–297.
Curran, T., and Cleary, A. M. (2003). Using ERPs to dissociate recollection from familiarity in picture recognition. Brain Res. Cogn. Brain Res. 15, 191–205.
Dewhurst, S. A., and Conway, M. A. (1994). Pictures, images, and recollective experience. J. Exp. Psychol. Learn. Mem. Cogn. 20, 1088–1098.
Donaldson, W. (1996). The role of decision processes in remembering and knowing. Mem. Cogn. 14, 523–533.
Dunn, J. C. (2004). Remember-know: a matter of confidence. Psychol. Rev. 111, 524–542.
Fletcher, P. C., Frith, C. D., Baker, S. C., Shallice, T., Frackowiak, R. S., and Dolan, R. J. (1995). The mind’s eye – precuneus activation in memory-related imagery. Neuroimage 2, 195–200.
Friston, K. J., Holmes, A. P., Poline, J. B., Grasby, P. J., Williams, S. C., Frackowiak, R. S., and Turner, R. (1995). Analysis of fMRI time-series revisited. Neuroimage 2, 45–53.
Gonsalves, B. D., Kahn, I., Curran, T., Norman, K. A., and Wagner, A. D. (2005). Memory strength and repetition suppression: multimodal imaging of medial temporal cortical contributions to recognition. Neuron 47, 751–761.
Haxby, J. V., Gobbini, M. I., Furey, M. L., Ishai, A., Schouten, J. L., and Pietrini, P. (2001). Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 293, 2425–2430.
Henson, R. N. A., Rugg, M. D., Shallice, T., Josephs, O., and Dolan, R. J. (1999). Recollection and familiarity in recognition memory: an event-related functional magnetic resonance imaging study. J. Neurosci. 19, 3962–3972.
Ishai, A., Fairhall, S. L., and Pepperell, R. (2007). Perception, memory and aesthetics of indeterminate art. Brain Res. Bull. 73, 319–324
Ishai, A., Haxby, J. V., and Ungerleider, L. G. (2002). Visual imagery of famous faces: effects of memory and attention revealed by fMRI. Neuroimage 17, 1729–1741.
Ishai, A., Ungerleider, L. G., and Haxby, J. V. (2000a). Distributed neural systems for the generation of visual images. Neuron 28, 979–990.
Ishai, A., Ungerleider, L. G., Martin, A., and Haxby, J. V. (2000b). The representation of objects in the human occipital and temporal cortex. J. Cogn. Neurosci. 12, 35–51.
Ishai, A., Ungerleider, L. G., Martin, A., Schouten, J. L., and Haxby, J. V. (1999). Distributed representation of objects in the human ventral visual pathway. Proc. Natl. Acad. Sci. USA 96, 9379–9384.
Ishai, A., and Yago, E. (2006). Recognition memory of newly learned faces. Brain Res. Bull. 71, 167–173.
Kahn, I., Davachi, L., and Wagner, A. D. (2004). Functional-neuroanatomic correlates of recollection: implications for models of recognition memory. J. Neurosci. 24, 4172–4180.
Kincade, J. M., Abrams, R. A., Astafiev, S. V., Shulman, G. L., and Corbetta, M. (2005). An event-related functional magnetic resonance imaging study of voluntary and stimulus-driven orienting of attention. J. Neurosci. 25, 4593–4604.
Knowlton, B. (1998). The relationship between remembering and knowing: a cognitive neuroscience perspective. Acta Psychol. 98, 253–265.
Konishi, S., Wheeler, M. E., Donaldson, D. I., and Buckner, R. L. (2000). Neural correlates of episodic retrieval success. Neuroimage 12, 276–286.
Mechelli, A., Price, C. J., Friston, K. J., and Ishai, A. (2004). Where bottom-up meets top-down: neuronal interactions during perception and imagery. Cereb. Cortex 14, 1256–1265.
Pollmann, S., Weidner, R., Humphreys, G. W., Olivers, C. N., Muller, K., Lohmann, G., Wiggins, C. J., and Watson, D. G. (2003). Separating distractor rejection and target detection in posterior parietal cortex – an event-related fMRI study of visual marking. Neuroimage 18, 310–323.
Pruessmann, K. P., Weiger, M., Scheidegger, M. B., and Boesiger, P. (1999). SENSE: sensitivity encoding for fast MRI. Magn. Reson. Med. 42, 952–962.
Rugg, M. D., and Wilding, E. L. (2000). Retrieval processing and episodic memory. Trends Cogn. Sci. 4, 108–115.
Rugg, M. D., and Yonelinas, A. P. (2003). Human recognition memory: a cognitive neuroscience perspective. Trends Cogn. Sci. 7, 313–319.
Rutishauser, U., Mamelak, A. N., and Schuman, E. M. (2006). Single-trial learning of novel stimuli by individual neurons of the human hippocampus-amygdala complex. Neuron 49, 805–813.
Shannon, B. J., and Buckner, R. L. (2004). Functional-anatomic correlates of memory retrieval that suggest nontraditional processing roles for multiple distinct regions within posterior parietal cortex. J. Neurosci. 24, 10084–10092.
Shulman, G. L., Ollinger, J. M., Linenweber, M., Petersen, S. E., and Corbetta, M. (2001). Multiple neural correlates of detection in the human brain. Proc. Natl. Acad. Sci. USA 98, 313–318.
Squire, L. R., and Bayley, P. J. (2007). The neuroscience of remote memory. Curr. Opin. Neurobiol. 17, 185–196.
Squire, L. R., Wixted, J. T., and Clark, R. E. (2007). Recognition memory and the medial temporal lobe: a new perspective. Nat. Rev. Neurosci. 8, 872–883.
Talairach, J., and Tournoux, P. (1998). Co-Planar Stereotaxic Atlas of the Human Brain. New York, NY, Thieme Medical, pp. 61–80.
Tulving, E. (1985). Memory and consciousness. Can. Psychol. 26, 1–12.
Vilberg, K. L., and Rugg, M. D. (2007). Dissociation of the neural correlates of recognition memory according to familiarity, recollection, and amount of recollected information. Neuropsychologia 45, 2216–2225.
Wagner, A. D., Shannon, B. J., Kahn, I., and Buckner, R. L. (2005). Parietal lobe contributions to episodic memory retrieval. Trends Cogn. Sci. 9, 445–453.
Wheeler, M. E., and Buckner, R. L. (2003). Functional dissociation among components of remembering: control, perceived oldness, and content. J. Neurosci. 23, 3869–3880.
Wilding, E. L. (2000). In what way does the parietal ERP old/new effect index recollection? Int. J. Psychophysiol. 35, 81–87.
Wixted, J. T. (2007). Dual-process theory and signal-detection theory of recognition memory. Psychol. Rev. 114, 152–176.
Yago, E., and Ishai, A. (2006). Recognition memory is modulated by visual similarity. Neuroimage 31, 807–817.
Yonelinas, A. P. (2002). The nature of recollection and familiarity: a review of 30 years of research. J. Mem. Lang. 46, 441–517.
Yonelinas, A. P., Otten, L. J., Shaw, K. N., and Rugg, M. D. (2005). Separating the brain regions involved in recollection and familiarity in recognition memory. J. Neurosci. 25, 3002–3008.
Yovel, G., and Paller K. A. (2004). The neural basis of the butcher-on-the-bus phenomenon: when a face seems familiar but is not remembered. Neuroimage 21, 789–800.
Keywords:
encoding, episodic memory, fMRI, retrieval
Citation:
Wiesmann M and Ishai A (2008). Recollection- and familiarity-based decisions reflect memory strength. Front. Syst. Neurosci. 2:1. doi: 10.3389/neuro.06.001.2008
Received:
24 January 2008;
 Paper pending published:
18 March 2008;
Accepted:
08 May 2008;
 Published online:
26 May 2008.

Edited by:

Wolfram Schultz, University of Cambridge, UK

Reviewed by:

Zoe Kourtzi, School of Psychology, Birmingham University, UK
Larry Squire, University of California, San Diego, USA
Copyright:
© 2008 Wiesmann and Ishai. This is an open-access article subject to an exclusive license agreement between the authors and the Frontiers Research Foundation, which permits unrestricted use, distribution, and reproduction in any medium, provided the original authors and source are credited.
*Correspondence:
Alumit Ishai, Institute of Neuroradiology, University of Zurich, Winterthurerstrasse 190, 8057 Zurich, Switzerland. e-mail: ishai@hifo.uzh.ch