Impact Factor 2.990 | CiteScore 3.5
More on impact ›

OPINION article

Front. Psychol., 07 December 2020 | https://doi.org/10.3389/fpsyg.2020.580667

Real-World Objects Are Represented in Visual Long-Term Memory Both as Unbound Features and as Bound Objects

  • Department of Psychology, University of Regensburg, Regensburg, Germany

Recent research has shown that observers store a vast amount of viewed real-world objects in visual long-term learning with high precision (e.g., Standing, 1973; Vogt and Magnussen, 2007; Brady et al., 2008), even when objects have been processed without any attention and intention of learning (Kuhbandner et al., 2017). However, one open issue that has attracted considerable attention recently is the nature of the stored visual long-term memory representations. Since objects are higher-level constructs that represent patterns of lower-level features, two contrasting views have been put forward: objects may be stored in in the form of sets of independent feature representations or in the form of unitary feature-bound object representations (e.g., Brady et al., 2013; van den Honert et al., 2017).

In two simultaneously published recent papers, contradictory conclusions are drawn. In a paper by Utochkin and Brady (2020), the authors conclude that objects are stored as sets of independent features, based on a series of experiments showing that object features are only weakly bound and can easily be unbound in long-term memory, a conclusion which is also supported by a previous study of Brady et al. suggesting that objects features are forgotten independently of each other (Brady et al., 2013). By contrast, in a paper by Balaban et al. (2019), the authors conclude that objects are stored as unitary feature-bound representations, based on a series of experiments following the protocol of the study by Brady et al. (2013; Experiment 2) but analyzing the data with an alternative analytical method, suggesting that object features are forgotten in a dependent manner.

At first glance, one could be tempted to conclude from such contradictory findings that further research is needed to clarify which view is actually correct. However, there is another possibility that is not considered in either of the two papers: it may be that visual information can be flexibly stored in visual long-term memory both feature-based and object-based, depending on the requirements of the current situation. If so, debates about whether visual objects are stored either feature-based or object-based may be misleading. Instead, the relevant question that should be explored in future research would be which factors determine whether objects are stored in visual long-term memory feature-based or object-based.

Such a theoretical assertion is based on two assumptions. First, it must be the case that real-world objects can be stored in visual long-term memory both feature-based and object-based. Second, it must be the case that the different storage formats have different functionalities and can be flexibly used. Regarding the first assumption, the contradictory findings reported in the papers by Brady et al. (2013), Balaban et al. (2019), and Utochkin and Brady (2020) can be taken as evidence that real-world objects can be stored both feature-based and object-based based. This is also supported by the fact that both in studies examining feature memory (e.g., Magnussen and Dyrnes, 1994; Magnussen et al., 2003) and in studies examining object memory (e.g., Ceraso et al., 1998; Walker and Cuthbert, 1998), the existence of long-lasting memory representations has been proven. The assumption that visual objects can be stored both feature-based and object-based is also found in prominent memory models such as the multiple-entry, modular memory framework (Johnson, 1983), postulating that there are feature-based and object-based memory subsystems. Furthermore, the existence of qualitatively different types of representational formats has also been recognized in theories about the hierarchical structure of visual memory (for a review, see, e.g., Brady et al., 2011). In fact, in research on visual working memory, it has been shown that both feature-based and object-based representations have to be assumed to fully explain the observed performance patterns (e.g., Fougnie et al., 2010, 2013).

To shed light on the different functionalities of feature-based and object-based memory representations, it is helpful to see how real-world objects are initially represented in the cognitive system during perception. Broadly speaking, two qualitatively different processing steps are involved (e.g., Tarr, 1995; Riesenhuber and Poggio, 1999; Serences and Yantis, 2006). First, visual features such as orientation, colors, and so forth, are extracted from the visual input, a process by which a representation of the visual input in terms of a collection of independent features is created. Second, informative features are recoded into bound object representations and uninformative features discounted, leading to the phenomenological experience of perceiving coherent objects. Importantly, object representations are not ad hoc formed independent of previous visual experiences. Rather, the recoding of features is informed by a stored inner model of the structure of the world that that reflects the current visual knowledge about objects derived from previous visual experiences.

As already postulated by Piaget (1970) and recently elaborated in theories about the so-called predictive brain (e.g., Clark, 2013), in order to allow adaptive learning, two opposing requirements have to be met by our visual system. On the one hand, to keep stability, incoming information has to be processed with respect to stored inner model of the world (assimilation). On the other hand, to allow adaptation, the current inner world model has to be continuously updated based on inconsistent incoming information (accommodation). The storing of feature-based vs. object-based memory representations may serve the fulfillment of these opposing requirements. As long as the current inner world model is appropriate, it can be imposed on incoming visual information so that visual experiences can be resource-efficiently stored as coherent objects based on current inner object models. Thus, object-based memory representations may serve the function of assimilation. In situations where the current inner world model does not sufficiently represent the incoming information, the inner model itself has to be updated based on the inconsistent information. In such a case, it would be functional to store visual experiences in the form of feature representations. Thus, feature-based memory representations may serve the function of accommodation (for an illustration, see Figure 1).

FIGURE 1
www.frontiersin.org

Figure 1. Illustration of the two types of visual long-term memory representations. During perception, at an initial step, the visual input is represented in terms of a collection of independent features such colors, orientations, and so forth. At a subsequent step, the features are recoded into bound object representations based on already stored object templates that reflect the currently stored inner model of the structure of the world derived from previous experiences. The outputs of both processing steps can be stored in memory, leading to unbound feature-based memory representations and to coherent object-based memory representations. From a functional perspective, as long as the current inner model of the structure of the world is appropriate, visual experiences can be resource-efficiently stored as coherent objects based on currently stored object-templates (“assimilation”). In situations where the current inner world model does not sufficiently represent the incoming information, feature-based memory representation can be used to refine the current inner model of the structure of the world (“accommodation”).

Therefore, depending on the appropriateness of the current inner model of the world, visual experiences may be stored either as unitary feature-bound objects representations (assimilation) or as independent feature representations (accommodation). A recent study examining the effect of affective state on the storing of real-world objects in visual long-term memory provides both behavioral and neurophysiological evidence that this is indeed the case (Spachtholz and Kuhbandner, 2017). As proposed in prominent theories on affect-cognition interactions (e.g., Clore and Huntsinger, 2007) and demonstrated in numerous studies (e.g., Fiedler et al., 2003; Kuhbandner et al., 2009), affect signals the validity of one's current inner model of the world, with positive affect validating and negative affect invalidating it, with the consequence that positive affect triggers processes of assimilation and negative affect processes of accommodation. Consistent with this, when real-world objects were encoded while experiencing positive affect, objects were more likely stored in the form of feature-bound object representations mediated by attention-related brain activities. By contrast, when real-world objects were encoded while experiencing negative affect, objects were more likely stored as independent feature representations mediated by pre-attentive brain activities.

Evidence for the assumption that the nature of memory representations is not unitary but a mixture of both feature-based and object-based representations is indeed also provided by the results of the studies by Balaban et al. (2019) and Utochkin and Brady (2020). For instance, in the study by Balaban et al. (2019), there is a number of memory reports where observers remember one feature but forget another one, indicating that memory representations are not purely object-based. Similarly, in the study by Utochkin and Brady (2020), interference is observed from irrelevant features when observers are asked to report relevant features, indicating that memory representations are not purely feature-based.

Taken together, the current state of research suggests that visual long-term memory is not a unitary system that stores real-world objects only in one specific representational format. Rather, real-world objects can be flexibly stored both as sets of independent features and as unitary feature-bound objects, depending on the requirements of the current situation. In particular, there may be a number of other factors beyond affect that influence the way real-world objects are stored in long-term memory. For instance, the storage format may vary as a function of the amount of previous experiences with encountered objects, making inner object models more or less appropriate. Furthermore, similar to affective state, current physical, motivational, and cognitive states may play important roles, which have been shown to systematically influence assimilation-accommodation tendencies as well (e.g., Leipold et al., 2014). At the interindividual level, individual habitual assimilation-accommodation tendencies may influence whether real-world objects are stored preferentially feature-based or object-based, and it may even be that there are cultural differences as suggested by studies showing that cultures systematically vary whether local or global visual information is favored during perception (e.g., Lao et al., 2013). Examining such factors may be more fruitful than trying to determine an illusionary unitary representational format of visual long-term memory.

Author Contributions

The author confirms being the sole contributor of this work and has approved it for publication.

Conflict of Interest

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Balaban, H., Assaf, D., Arad Meir, M., and Luria, R. (2019). Different features of real-world objects are represented in a dependent manner in long-term memory. J. Exp. Psychol. Gen. 149, 1275–1293. doi: 10.1037/xge0000716

PubMed Abstract | CrossRef Full Text | Google Scholar

Brady, T. F., Konkle, T., and Alvarez, G. A. (2011). A review of visual memory capacity: beyond individual items and towards structured representations. J. Vis. 11:4. doi: 10.1167/11.5.4

PubMed Abstract | CrossRef Full Text | Google Scholar

Brady, T. F., Konkle, T., Alvarez, G. A., and Oliva, A. (2008). Visual long-term memory has a massive storage capacity for object details. Proc. Natl. Acad. Sci. U.S.A. 105, 14325–14329. doi: 10.1073/pnas.0803390105

PubMed Abstract | CrossRef Full Text | Google Scholar

Brady, T. F., Konkle, T., Alvarez, G. A., and Oliva, A. (2013). Real-world objects are not represented as bound units: independent forgetting of different object details from visual memory. J. Exp. Psychol. Gen. 142, 791–808. doi: 10.1037/a0029649

PubMed Abstract | CrossRef Full Text | Google Scholar

Ceraso, J., Kourtzi, Z., and Ray, S. (1998). The integration of object properties. J. Exp. Psychol. Learn. Mem. Cogn. 24, 1152–1161. doi: 10.1037/0278-7393.24.5.1152

PubMed Abstract | CrossRef Full Text | Google Scholar

Clark, A. (2013). Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behav. Brain Sci. 36, 181–204. doi: 10.1017/S0140525X12000477

PubMed Abstract | CrossRef Full Text | Google Scholar

Clore, G. L., and Huntsinger, J. R. (2007). How emotions inform judgment and regulate thought. Trends Cogn. Sci. 11, 393–399. doi: 10.1016/j.tics.2007.08.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Fiedler, K., Nickel, S., Asbeck, J., and Pagel, U. (2003). Mood and the generation effect. Cogn. Emot. 17, 585–608. doi: 10.1080/02699930302301

PubMed Abstract | CrossRef Full Text | Google Scholar

Fougnie, D., Asplund, C. L., and Marois, R. (2010). What are the units of storage in visual working memory? J. Vis. 10:27. doi: 10.1167/10.12.27

PubMed Abstract | CrossRef Full Text | Google Scholar

Fougnie, D., Cormiea, S. M., and Alvarez, G. A. (2013). Object-based benefits without object-based representations. J. Exp. Psychol. Gen. 142, 621–626. doi: 10.1037/a0030300

PubMed Abstract | CrossRef Full Text | Google Scholar

Johnson, M. K. (1983). “A multiple-entry, modular memory system,” in The Psychology of Learning and Motivation: Advances in Research and Theory, Vol. 17, ed G. H. Bower (New York, NY: Academic Press), 81–123.

Google Scholar

Kuhbandner, C., Hanslmayr, S., Maier, M. A., Pekrun, R., Spitzer, B., Pastotter, B., et al. (2009). Effects of mood on the speed of conscious perception: behavioural and electrophysiological evidence. Soc. Cogn. Affect. Neurosci. 4, 286–293. doi: 10.1093/scan/nsp010

PubMed Abstract | CrossRef Full Text | Google Scholar

Kuhbandner, C., Rosas-Corona, E. A., and Spachtholz, P. (2017). High-fidelity visual long-term memory within an unattended blink of an eye. Front. Psychol. 8:1859. doi: 10.3389/fpsyg.2017.01859

PubMed Abstract | CrossRef Full Text | Google Scholar

Lao, J., Vizioli, L., and Caldara, R. (2013). Culture modulates the temporal dynamics of global/local processing. Cult. Brain 1, 158–174. doi: 10.1007/s40167-013-0012-2

CrossRef Full Text | Google Scholar

Leipold, B., Bermeitinger, C., Greve, W., Meyer, B., Arnold, M., and Pielniok, M. (2014). Short-term induction of assimilation and accommodation. Q. J. Exp. Psychol. 67, 2392–2408. doi: 10.1080/17470218.2014.931443

PubMed Abstract | CrossRef Full Text | Google Scholar

Magnussen, S., and Dyrnes, S. (1994). High-fidelity perceptual long-term memory. Psychol. Sci. 5, 99–102. doi: 10.1111/j.1467-9280.1994.tb00638.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Magnussen, S., Greenlee, M. W., Aslaksen, P. M., and Kildebo, O. O. (2003). High-fidelity perceptual long-term memory revisited - and confirmed. Psychol. Sci. 14, 74–76. doi: 10.1111/1467-9280.01421

PubMed Abstract | CrossRef Full Text | Google Scholar

Piaget, J. (1970). Science of Education and the Psychology of the Child. New York, NY: Orion Press.

Google Scholar

Riesenhuber, M., and Poggio, T. (1999). Hierarchical models of object recognition in cortex. Nat. Neurosci. 2, 1019–1025. doi: 10.1038/14819

PubMed Abstract | CrossRef Full Text | Google Scholar

Serences, J. T., and Yantis, S. (2006). Selective visual attention and perceptual coherence. Trends Cogn. Sci. 10, 38–45. doi: 10.1016/j.tics.2005.11.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Spachtholz, P., and Kuhbandner, C. (2017). Visual long-term memory is not unitary: flexible storage of visual information as features or objects as a function of affect. Cogn. Affect. Behav. Neurosci. 17, 1141–1150. doi: 10.3758/s13415-017-0538-4

CrossRef Full Text | Google Scholar

Standing, L. (1973). Learning 10,000 pictures. Q. J. Exp. Psychol. 25, 207–222. doi: 10.1080/14640747308400340

PubMed Abstract | CrossRef Full Text | Google Scholar

Tarr, M. J. (1995). Rotating objects to recognize them: a case study on the role of viewpoint dependency in the recognition of three-dimensional objects. Psychon. Bull. Rev. 2, 55–82. doi: 10.3758/BF03214412

PubMed Abstract | CrossRef Full Text | Google Scholar

Utochkin, I. S., and Brady, T. F. (2020). Independent storage of different features of real-world objects in long-term memory. J. Exp. Psychol. Gen. 149, 530–549. doi: 10.1037/xge0000664

PubMed Abstract | CrossRef Full Text | Google Scholar

van den Honert, R. N., McCarthy, G., and Johnson, M. K. (2017). Holistic versus feature-based binding in the medial temporal lobe. Cortex 91, 56–66. doi: 10.1016/j.cortex.2017.01.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Vogt, S., and Magnussen, S. (2007). Long-term memory for 400 pictures on a common theme. Exp. Psychol. 54, 298–303. doi: 10.1027/1618-3169.54.4.298

PubMed Abstract | CrossRef Full Text | Google Scholar

Walker, P., and Cuthbert, L. (1998). Remembering visual feature conjunctions: visual memory for shape-colour associations is object-based. Vis. Cogn. 5, 409–455. doi: 10.1080/713756794

CrossRef Full Text | Google Scholar

Keywords: visual memory, feature memory, feature binding, object memory, perceptual long-term memory

Citation: Kuhbandner C (2020) Real-World Objects Are Represented in Visual Long-Term Memory Both as Unbound Features and as Bound Objects. Front. Psychol. 11:580667. doi: 10.3389/fpsyg.2020.580667

Received: 06 July 2020; Accepted: 16 November 2020;
Published: 07 December 2020.

Edited by:

Guomei Zhou, Sun Yat-sen University, China

Reviewed by:

Igor S. Utochkin, National Research University Higher School of Economics, Russia

Copyright © 2020 Kuhbandner. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Christof Kuhbandner, christof.kuhbandner@ur.de