Abstract
Numerous studies examining the responses of individual neurons in the inferior temporal (IT) cortex have revealed their characteristics such as two-dimensional or three-dimensional shape tuning, objects, or category selectivity. While these basic selectivities have been studied assuming that their response to stimuli is relatively stable, physiological experiments have revealed that the responsiveness of IT neurons also depends on visual experience. The activity changes of IT neurons occur over various time ranges; among these, repetition suppression (RS), in particular, is robustly observed in IT neurons without any behavioral or task constraints. I observed a similar phenomenon in the ventral visual neurons in macaque monkeys while they engaged in free viewing and actively fixated on one consistent object multiple times. This observation indicates that the phenomenon also occurs in natural situations during which the subject actively views stimuli without forced fixation, suggesting that this phenomenon is an everyday occurrence and widespread across regions of the visual system, making it a default process for visual neurons. Such short-term activity modulation may be a key to understanding the visual system; however, the circuit mechanism and the biological significance of RS remain unclear. Thus, in this review, I summarize the observed modulation types in IT neurons and the known properties of RS. Subsequently, I discuss adaptation in vision, including concepts such as efficient and predictive coding, as well as the relationship between adaptation and psychophysical aftereffects. Finally, I discuss some conceptual implications of this phenomenon as well as the circuit mechanisms and the models that may explain adaptation as a fundamental aspect of visual processing.
1 Introduction
The inferior temporal (IT) cortex occupies a later stage in the ventral visual pathway and is responsible for object recognition. Compared to the lower visual areas such as the primary or secondary visual areas (V1 and V2) which have small receptive fields and relatively faithful responses to low-level image features or intensity, neurons in the IT cortex are characterized by their large receptive field, sensitivity to complex shapes, object images or categories, and invariance to substantial changes in the pixel values caused by resizing, rotation, or shading (Ito et al., 1995; Logothetis et al., 1995). Output from the IT neurons innervates various areas such as the rhinal cortex, amygdala, striatum, and prefrontal cortex (Kravitz et al., 2013). Thus, throughout the ventral visual pathway, information dealt within the corresponding area seems to drastically change from the one strongly related to the outside environment and input image statistics to the one advantageous for understanding the surrounding environment, memorizing, associating important items with values, or performing tasks that may have considerable consistency with our thoughts and languages.
Numerous studies examining the responses of individual IT neurons have revealed interesting characteristics such as two- or three-dimensional shape tunings (Op de Beeck et al., 2001; Brincat and Connor, 2004; Yamane et al., 2008; Hung et al., 2012), objects or their category selectivity (Hung et al., 2005; Kiani et al., 2007), as well as faces (Gross, 2008) or scenes (Vaziri et al., 2014). While these basic selectivities have been studied assuming that their response to the stimuli is relatively stable (Op de Beeck et al., 2001; Srihasam et al., 2014), physiological experiments have revealed that the responsiveness of IT neurons is also dependent on visual experience (Tovee et al., 1996; Kobatake et al., 1998; Li and DiCarlo, 2010; Meyer and Olson, 2011; Woloszyn and Sheinberg, 2012; Vogels, 2016). The time required to evoke changes in responsiveness ranges from within 1 s such as repeated presentations to months or years of exposure. Some of these changes occur without task demands such as remembering or attending to the image; however, simply viewing the object stimuli in a particular order or frequency is sufficient to evoke changes. There are questions regarding what significance those changes hold for visual processing, how they impact coding by population neurons, and how a dynamic response can be compatible with stable coding. In the following sections, I summarize various modulation types observed in the IT neurons. Among these, I particularly focuse on repetition suppression (RS), where the responses of IT neurons to stimuli that are the same or similar to the preceding stimulus are attenuated (Desimone, 1996). The more widespread word, sensory adaptation, is also used in other visual areas. Although adaptation and RS are often used interchangeably, especially in visual processing in the ventral stream, I will refer to the (mainly suppressive) modulation of IT neurons as RS and modulation of other areas as adaptation here. I discuss the properties of RS and introduce a study on adaptation in the lower visual areas and its implications. Additionally, the discussion extends to aftereffects, context modulation, and efficient coding. I argue that the RS and adaptation are closely related to visual information processing, and finally, discuss some computational models including efficient coding and excitation/inhibition (E/I) balanced networks, that possibly connect hierarchical visual processing as inference and adaptation.
2 Types of changes of the IT neural activity by experience
2.1 Task-driven modulations
Several studies have shown that experience modulates the activity of the IT neurons. Task-dependent and -independent modulations and changes in activity have been reported. In the research on task-dependent modulation, the underlying concept revolves around examining how feature selectivity of the IT neurons is influenced by learning in adults achieving proficiency in tasks such as discrimination and categorization. For instance, Kobatake et al. (1998) subjected monkeys to a shape discrimination task, comparing the responses of the IT neurons to stimuli used in the discrimination task and those not used in the task. Neurons exhibiting strong responses to stimuli used in the discrimination task were more prevalent in the trained monkeys. In this experiment, animals were anesthetized during neural recording, eliminating considerations for factors such as attention or reward influences. Moreover, in experiments involving a categorization task (Sigala and Logothetis, 2002), visual features crucial for categorization were reported to exhibit sharper selectivity than irrelevant features after learning. These studies suggest that the representations of visual features relevant to the task undergo selective changes during month-long training sessions. Additionally, the IT neurons modulate their responses through association learning. The learned association between two stimuli enables the IT neurons to respond to the pair of associated stimuli through collaboration with the memory system, as revealed in a series of studies (Miyashita and Hayashi, 2000; Miyashita, 2004; Hirabayashi and Miyashita, 2014).
2.2 Long-term visual experience causes modulations
In addition to task-driven modulation, changes solely caused by visual experience have been widely reported. For instance, after months of passive exposure to specific stimuli without any task demanding memory or discrimination, IT neuronal responses differed between familiar (exposed) and novel stimuli in animals (Anderson et al., 2008; Woloszyn and Sheinberg., 2012). Woloszyn et al. demonstrated that for familiar stimuli, responses in the majority of neurons decreased; however, within them, the selectivity of putative excitatory neurons responding to familiar stimuli increased. Furthermore, in addition to stimuli familiarity, passive viewing with a consistent stimuli sequence (in which one specific stimulus is always followed by another, whereas the order of other stimuli is randomly shuffled) can implicitly be associated with stimuli (statistical learning), inducing changes in the IT neuronal activity (Meyer and Olson, 2011; Meyer et al., 2014; Ramachandran et al., 2016, 2017; Kaposvari et al., 2018; Esmailpour et al., 2023). These studies revealed that expected stimuli led to a decrease in the firing rate and violating the presentation order of the stimuli led to an increase in the firing rates, indicating compatibility with predictive coding (Rao and Ballard, 1999). The relationship between expectation-related modulation and predictive coding has been the focus of much attention, leading to various experimental investigations across different task paradigms (Feuerriegel et al., 2021). The concept of “predictive coding” is revisited later in this manuscript.
Thus, presenting stimuli with controlled statistical properties for relatively extended periods such as weeks or months can modulate IT responses with the consistent stimuli sequence. Notably, IT neurons do not distinguish triplets (the order of three consecutive stimuli) or higher orders even though they are referred to as “sequences” (Meyer et al., 2014). This emphasizes the importance of temporal proximity in inducing changes in the IT neurons. Indeed, temporal proximity is a characteristic of the usual input to the visual system (i.e., temporal continuity: objects do not abruptly appear or disappear), and it is conceivable that such statistical properties of the external world influence object representations in the IT cortex. Stimuli repeatedly presented with temporal closeness have been demonstrated to produce more similar representations in single neurons (Li and DiCarlo, 2010; Jia et al., 2021).
2.3 Short-term modulation
IT neuronal activity can be modulated without long-term exposure. Attention can increase the activity of single IT neurons (Moran and Desimone, 1985) and change the population representation of objects (Sereno and Lehky, 2018). Sereno et al. examined the discriminability of stimulus shape identity by IT neuronal population when recorded while the subject is attending to shape or to location. They found that the discriminability is higher during attention to shape. The effect of attention is thought to originate from the frontal cortex (Ramezanpour and Fallah, 2022), and the IT cortex is suggested to be involved in an object-based attention network. RS is another modulation type. Responses to stimuli that are the same or similar to the immediately preceding stimulus are attenuated. The RS has been extensively studied since it was reported [Response reduction by stimulus repetition was first reported by Brown et al. (1987) and Baylis and Rolls (1987). The name ‘RS’ was used in a later paper (Desimone, 1996)]. Initially, it was suggested that the IT neurons may be the filter to pass new, unexpected stimuli (Miller et al., 1991), or that RS may be important to discriminate novel objects (Ringo, 1996), or that RS is a neural correlate of the psychological phenomenon of priming in which once an image is presented, psychophysical discriminability is increased (Wiggs and Martin, 1998). The phenomenon where the neural response to the same stimulus changes is generally called sensory adaptation, and it is common across different sensory areas [ex., olfactory bulb (Scott, 1977), barrel cortex (Khatri et al., 2004), and auditory cortex (Ulanovsky et al., 2003)]. Adaptation-like modulation has been observed in the visual neurons of lower and higher orders, including V1 (Müller et al., 1999; Patterson et al., 2013), V4 (Wang et al., 2011), MT (Kohn and Movshon, 2004; Patterson et al., 2014), MST (Price and Born, 2013), and IT (Baylis and Rolls, 1987; Brown et al., 1987). Some psychological phenomena have been associated with visual neuronal adaptations, and various mechanisms have been proposed. Despite the relatively large amount of literature and observation throughout ventral visual areas, its significance in visual processing is yet to be completely elucidated. The following sections will provide various discussions regarding the properties of RS in IT and adaptation in other visual areas and topics related to adaptation in the visual system, including the possible interpretation and mechanism of adaptation.
3 Properties of repetition suppression in IT neurons
3.1 General property
While reduction of response to a similar stimulus is observed throughout the ventral visual pathway, it appears to be most prominent in the IT neurons when compared using the same stimuli across areas (Yamane et al., 2023). An interesting property of this suppression in IT is that the extent of the suppression induced by repeated stimuli depends on the combination of the stimulus used and the recorded neuron. In other words, even if two different stimuli evoke similar response strengths, the magnitude of the RS effect (suppression) induced by the two stimuli differs for each neuron (Sawamura et al., 2006; Liu et al., 2009; De Baene and Vogels, 2010). Additionally, RS can be induced invariantly in position or size (De Baene and Vogels, 2010), demonstrating similarity to general stimulus selectivity in individual IT neurons. The reduction of the response cannot be explained by attention reduction. De Baene and Vogels (2010) compared RS during an attention-demanding task and a passive fixation task to explore the interaction between attention and RS in IT. The results showed that the strength of RS did not differ between the attended and non-attended conditions. This lack of difference suggests that there is likely no strong interaction between attention and RS. This finding contrasts with expectation suppression discussed in the later section, which often requires attention. RS-like suppression occurs even during free viewing. Repeated fixations on the images of the same object evoke suppression of the response of IT neurons (Yamane et al., 2023). Therefore, it is a phenomenon that occurs both under limited (fixation task) conditions and under relatively free conditions that allow free viewing.
RS is closely related to the stimulus selectivity of individual IT neurons and cannot be merely explained by stimulus-nonspecific neuronal fatigue. Following this notion, a comparison of IT excitability before and after direct (without using visual stimuli) photostimulation of IT neurons demonstrated sufficient responses even after photostimulation, contradicting the theory of fatigue in IT neurons themselves (Fabbrini et al., 2019). Thus, changes in input (which may separately vary with each stimulus) such as synaptic depression are suggested to be involved in stimulus-dependent RS (Vogels, 2016). Examining the relationship between neuronal selectivity and suppression in the IT neurons is extremely challenging, primarily because of the multidimensional and complex nature of selectivity in the IT neurons.
3.2 Adaptation of dorsal stream neurons
In the middle temporal (MT) area, adaptation has been studied using moving gratings, and it has been reported that motion direction tuning becomes sharper due to adaptation (Kohn and Movshon, 2004). For such simple stimuli, it has been shown that inherited input from V1 plays a significant role (Kohn and Movshon, 2003). Adaptation to more complex stimuli, such as plaids, can also be explained by pooling the changes in V1 input (Patterson et al., 2014). However, in the case of dot stimuli, there is a report that direction tuning does not change after adaptation, but speed preference does (Yang and Lisberger, 2009). Therefore, the type of stimulus can be an important factor. However, in general, the selectivity of MT cells is easier to parameterize than that of IT cells, making it easier to examine the effects of adaptation to the tuning. In addition, a notable feature of adaptation in MT is that there are a significant number of cells that are not suppressed but enhanced. This enhancement occurs because MT cells are enhanced by motion stimuli in the direction opposite to that of the adaptors’ motion. Such a feature is not observed in RS in IT. This difference is thought to be due to the distinct computations performed in each area—motion direction extraction in MT and complex feature extraction in IT (Kar and Krekelberg, 2016).
4 Studies in the lower visual areas and their implication in the RS of IT neurons
Insights into the nature, mechanism, and implication of the visual computation of adaptation in the lower visual neurons with relatively (though not absolutely) simple stimulus selectivity and known circuits may help understand and allow for specification of the RS mechanism and impact on the population activity of the IT neurons. I hypothesize that the underlying principle of the adaptation in IT is similar to the lower visual area and the empirical difference related to RS roots from the computations performed by the cell populations in both areas (e.g., contrast extraction or complex shapes). In V1, adaptation is discussed as the short-term modulation of neural activity influenced by previous stimuli; this includes suppression as well as enhancement.
Relatively few reports have examined adaptation via electrophysiological experiments in the intermediate areas of the ventral visual pathway such as visual areas V2 and V4 neurons (Tolias et al., 2005; Crowder et al., 2006; Wang et al., 2011). However, a considerable amount of literature is available on the adaptation in the V1 neurons (see review; Kohn, 2007). I focus on some intriguing topics in the experimental and theoretical fields concerning V1 neuronal adaptation and attempt to extend the discussion to RS in the IT neurons. Repetition of the same or similar stimuli can be considered as the repeated presentation of redundant visual input, and reducing this redundancy in the neural code aligns naturally with efficient coding (Attneave, 1954; Barlow, 1961; Wainwright, 1999), where sensory circuits encode maximal information about their inputs, reducing redundancy. Here, I explore the discussion related to information maximization conveyed by neurons through redundancy reduction and consider experimental results that support and argue for this notion. Additionally, the discussion includes the tilt aftereffect, a well-known psychological phenomenon, and its neuroscientific underpinnings. These topics are interrelated.
4.1 Efficient coding and adaptation
The concept of efficient coding has evolved since it was first proposed in the field of neuroscience (Attneave, 1954; Barlow, 1961) and has been used to explain various experimental results (Simoncelli and Olshausen, 2001; Chalk et al., 2018; Price and Gavornik, 2022). In vision specifically, efficient coding often refers to reducing redundant information and maximizing the information content when visual images are coded. For instance, contrast adaptation is known to exist in the retina and V1 neurons, which involves the strength of the response adaptively changes based on the history of past stimulus contrasts (mean and std), thereby maximizing the information conveyed in the current stimulus distribution. Such adaptive changes are observed in the motion-selective neurons of flies (Brenner et al., 2000; Fairhall et al., 2001) and the auditory cortex of songbirds (Nagel and Doupe, 2006), suggesting a mechanism shared among different animals and across different sensory modalities. However, adaptation extends beyond these primary statistical measures even at the early visual stages. In the retinas of salamanders and rabbits, different adaptations occur for stimuli with the same contrast and luminance but with different stimulus patterns (Hosoya et al., 2005). In V1 neurons, receptive fields (RF) have been demonstrated to differently adapt to artificial (noise images) and natural images to maximize the information content that is specific to adapted images (Sharpee et al., 2006). In their study, the visual system adjusted the output of the V1 neurons to efficiently account for the characteristics of natural images, where low-frequency components had a higher probability of occurrence.
The aforementioned examples reveal that given its limited resources, the neural system can adapt to maximize the information relevant to the processing stage. The importance of the lower visual features such as contrast, orientation, and frequency in areas such as the retina and V1 neurons is unquestionably important. However, confirming whether information is processed more efficiently becomes more complex as the visual hierarchy ascends. We must consider the relevance of the information in the processing stage, which is directly related to neural coding. In cases higher than V1, what specific information the neurons are attentive to or extracting is not always clear.
Nonetheless, psychophysical experiments have provided the statistical feature that is important for perceptual judgment and guided the analysis of efficient coding in V2. Hermundstad et al. (2014) carefully identified the statistical properties of natural images and identified those most variable and thus the least predictable and, therefore, the most informative statistical feature. Based on the identified features, Yu et al. (2015) examined the type of efficient coding at V2. In subsequent recordings of V1 and V2 neurons from macaque monkeys, they demonstrated the emergence of the response dependency in V2 neurons on the identified informative feature. This result indicates that the constraint on efficiency in V2 originates from input sampling— in other words, the relevance of information of input to behavior— not output capacity, as the original efficient coding suggested. Thus, the result demonstrates that different types of efficiency are optimized in different areas.
In addition to the difficulty in identifying relevant information, a population’s amount of encodable information can vary depending on the strength of correlations between the cells in the population (Moreno-Bote et al., 2014; Shamir, 2014). Therefore, concluding whether changes due to adaptation in single cell response make population coding more efficient is not straightforward. Simulations of the V1 population indicate that coding accuracy can increase or decrease depending on the adaptation mechanism and stimulus (Cortes et al., 2012). Consequently, whether the information coded by the population is genuinely improved and becomes efficient through adaptation, even in V1, remains unclear.
In addition to the challenge mentioned earlier in evaluating efficiency, several potential conflicting issues exist that may prevent encoding efficiency such as limitation of resources or sampling, or, more importantly, constraints from decordability. Even though highly efficient encoding is possible, it can be challenging to decode in downstream areas and thus can be not helpful for behavior (Tesileanu et al., 2022). Furthermore, the behaviors in which the ventral visual system is engaged may be diverse. Task information reformatting, rather than maximization (Gaspar et al., 2019), is also an essential direction that needs to be considered, especially when considering coding efficiency in the IT cortex.
4.2 Aftereffects and adaptation
The tilt aftereffect is a well-studied intriguing psychophysical adaptation phenomenon (Gibson and Radner, 1937). It refers to a systematic bias in the perception of the orientation after exposure to a grating with a particular orientation. The difference between adaptor and test stimulus orientations determines the perception of tilt direction. Interestingly, aftereffects occur also with more complex stimuli. For example, the difference in the three-dimensional viewpoint of a face image evokes bias in viewpoint (Fang and He, 2005). Some properties of face angle aftereffect are similar to the tilt aftereffect in oriented gratings. For example, the angular tuning function of the face viewpoint aftereffect is similar to the angular tuning function of the tilt aftereffect (Chen et al., 2010), indicating a common process across extensive regions of the visual pathway, including the IT cortex (Leopold et al., 2001).
One interesting hypothesis suggests that the mechanism (or principle) underlying the occurrence of the tilt aftereffect and the tilt illusion may be the same (Schwartz et al., 2007; Clifford, 2014). In tilt illusion, spatially separated adapters and stimuli are presented simultaneously. The oriented grating shown in the center is perceived as tilted in the opposite direction to the orientation of the surround. Tilt aftereffects and tilt illusions share common psychophysical properties such as the relationship between the tilt angles of the stimuli and the evoked biases (Schwartz et al., 2007; Clifford, 2014). Our visual environment shares statistical redundancy as a common property across time and space. Under the efficient coding hypothesis, coding should be adjusted to reduce redundancy. Thus, both phenomena may be illusions related to the same redundancy-reduction process. Mechanisms involving adaptive gain control such as divisive normalization (Heeger, 1992) are speculated to be crucial for the tilt illusion and the tilt aftereffect. A more recent study suggested that the statistical similarity between the center and surrounding stimuli gates the surround suppression in the cortical neurons (Coen-Cagli et al., 2009, 2015). The following study revealed that short-term temporal regularities (e.g., stability of temporal input within one fixation) are learned and may explain the adaptation in the V1 neurons (Snow et al., 2017). However, no canonical model has ever simultaneously explained the multiple psychophysical phenomena, including tilt aftereffect and tilt illusion (Sanchez-Giraldo et al., 2019; Northoff and Mushiake, 2020).
The circuit responsible for surround modulation includes feedforward, feedback across areas, and lateral connections within an area (Angelucci et al., 2017). Feedforward connections contribute to a temporally fast and untuned component of surround modulation near the classical receptive field and emerge first in layer 4 in cats and primates in V1. Feedback connections contribute spatially extensive, and tuned components to surround modulation and are generated outside layer 4 (Angelucci et al., 2017). In V1, the target of feedback connection is both excitatory and inhibitory neurons (Gonchar and Burkhalter, 2003; Anderson and Martin, 2009). Horizontal connections are most prominent in layers 2/3 and contribute to a spatially extensive and tuned component and the modulation includes suppression and facilitation. The target of horizontal connections are excitatory and inhibitory neurons of similar orientation preferences at least in layers 2/3 (McGuire et al., 1991; Ko et al., 2011). To examine the effect of adaptation to the surround modulation, adaptation properties of the V1 neurons to simple artificial stimuli have been investigated. Multiple modulation patterns including enhancement, suppression, and change in tuning by adaptation have been observed (Wissig and Kohn, 2012; Patterson et al., 2013). One explanation for these multiple modulation types is the balance between the feedforward driving input, suppressive surrounding input, and divisive normalization (Dhruv et al., 2011; Solomon and Kohn, 2014). One of the speculated roles of these modulation types is to achieve a higher discriminability via the sharpening of tuning. However, the experimental results varied and were inconclusive, and even if sharpening was observed, whether it leads to an increase in discrimination accuracy by the neural populations remains unclear (Kohn, 2007; Cortes et al., 2012; Solomon and Kohn, 2014).
4.3 Gamma oscillations
Gamma frequency oscillations have been hypothesized to play a role in the relationship between the classical receptive field and the surround (Vinck and Bosman, 2016). In this section, I discuss gamma oscillations and adaptation. Gamma frequency oscillations are hypothesized to play a fundamental role in cortical processes such as attention (Fries et al., 2001; Fries, 2009). Another view is that gamma oscillations are reflections of the underlying cortical processing involving excitation-inhibition interactions and helpful indicators to detect such interactions (Ray and Maunsell, 2015; Bartoli et al., 2020; Ray, 2022). Reports have indicated that gamma oscillations increase in the supragranular layer of V1 neurons under repeated stimulus conditions (Hansen and Dragoi, 2011; Brunet et al., 2014). A study using laminar recordings and examining changes in the information flow due to stimulus repetition through current source density demonstrated that the effects of stimulus repetition (firing reduction) occurring in the supragranular layer due to adaptation subsequently propagated to other layers (Westerberg et al., 2019). This study suggests that the primary origin of repetition-related response modulation in V1 neurons is linked to intracortical processing within the supragranular layers. This does not necessarily exclude the inherited influences of suppression occur in retina. Some of the reduction in firing rate is likely due to a reduction in retinal input (Solomon et al., 2004). Thus, at least some of the modulation is speculated to originate from supragranular layers. In V1 neurons, the horizontal connections are prominent in layers 2/3 (Rockland and Lund, 1983; Lund et al., 1993; Angelucci et al., 2002) and I discussed that one of the circuit responsible for surround modulation is horizontal connections in the previous section. Horizontal connections are patchily distributed in the surrounding functional columns. These observations may suggest that interactions between neurons such as excitation-inhibition leading to gamma oscillations also relate to adaptation and occur primarily in layers 2/3.
As they progress to higher visual areas, gamma oscillations tend to decrease in power (Vinck and Bosman, 2016). The proportion of different inhibitory neuronal types differs across the visual areas (Kondo et al., 1994; Defelipe et al., 1999). Parvalbumin-GABAergic interneurons have been suggested to be involved in the generation of gamma oscillations and are abundant in layers 2/3 (Bartos et al., 2007; Tiesinga and Sejnowski, 2010). Furthermore, the lateral connections in layer 2/3 of the IT cortex exhibit a patchy pattern similar to those in the V1 cortex. However, unlike the V1 neurons, these connections do not decrease with distance, and the patch positions are more random in IT than in V1 (Fujita and Fujita, 1996; Tanigawa et al., 1998; Fujita, 2002). Thus, these differences in the connection patterns might explain the difference between the empirical gamma oscillation and the effect of adaptation across areas such as V1 (Hansen and Dragoi, 2011; Brunet et al., 2014), V4 (Wang et al., 2011), and IT (De Baene and Vogels, 2010). Generating gamma oscillations requires a considerable number of neurons for close synchronization. Synchronization of widely dispersed cells may be challenging to detect. While various insights into Gamma oscillation and adaptation in V1 are intriguing, anatomical circuit differences between V1 and IT may affect their experimental results and interpretation of RS in IT.
4.4 Adaptation in the lower visual area and RS in IT
In this section, I explored the existing hypotheses, experimental results, and implications obtained from the discussion of the adaptation in V1. This examination reveals a profound link between adaptation and context modulation, such as surround modulation. This possibly involves the interactions between inhibition and excitation from the observation of Gamma oscillation modulation in adaptation in V1 layer 2/3. As information processing transitions from V1 to IT, the represented information changes from the one strongly related to the outside environment, thus keeping the configuration of the input image to the one that is more behavior-relevant, abstract, and different from the input image. Thus, we do not have the strict equivalence of V1 surround modulation in IT. Still, modulation mechanisms that exhibit suppression and enhancement, possibly from lateral and feedback connections might be the important players in considering the mechanism of RS in IT.
As the adaptation occurs in the lower visual areas, the RS in the higher visual hierarchy of the IT may be influenced to some extent by the adaptation in the lower visual areas. The same stimuli were presented at non-overlapping locations to discriminate between inherited suppression and suppression occurring in IT (De Baene and Vogels, 2010). If adaptation in IT is inherited from earlier areas, adaptation would be absent when both the adapter and test stimuli do not fall within the same receptive fields at those earlier levels. The result showed suppression, indicating that suppression occurs within IT. However, the degree of suppression was less than at the same location, indicating some extent of suppression was inherited from earlier areas. Therefore, modulation occurs at each stage and is additive, with further modulation occurring in IT. Since modulation at each stage is dependent on the stimulus selectivity of the cells, the accumulated modulation in IT might be complex.
5 Predictive coding and RS
Predictive coding (Rao and Ballard, 1999; Friston, 2005; Spratling, 2008) is an influential proposal for visual processing that has evoked considerable discussion. In an original study (Rao and Ballard, 1999), a hierarchical neural network with neurons carrying prediction and error signals trained using natural images exhibited RF structures similar to those experimentally observed in the V1 neurons. This suggests that the network learns the statistical regularities of the natural images and conveys the signal deviations between the sensory inputs and such regularities (the error) to higher processing hierarchies to update prediction. This processing reduces redundancy by eliminating the predictability of the input signal, and thus, this process is consistent with efficient coding. Additionally, their model can be understood as a Bayesian framework of perception that assumes a generative model to infer the cause of the input. Several excellent and intuitive reviews on predictive coding are available (Kok and de Lange, 2015; Aitchison and Lengyel, 2017; Spratling, 2017; Keller and Mrsic-Flogel, 2018; Walsh et al., 2020; Shipp, 2024). This section will focus on RS and IT neurons in the context of predictive coding.
The most prominent characteristic of predictive coding is the presence of two neuronal types: neurons predicting sensory input and those coding prediction errors (the difference between predicted and observed sensory inputs) (Rao and Ballard, 1999; Friston, 2005; Bastos et al., 2012). The decrease in the response to repeated, thus redundant stimuli in the RS had been considered to be consistent with the behavior of error neurons (Auksztulewicz and Friston, 2016); however, closer examination speaks negatively about the presence of error neurons in the visual cortex of macaques, including the IT neurons (Kaliukhovich and Vogels, 2011; Vinken et al., 2018; Solomon et al., 2021).
In IT studies, following the experimental paradigm of Summerfield et al. (2008) in which alterations in stimulus identity were expected in some blocks and repeats were expected in others, neuronal responses to the stimuli in the expected and unexpected condition were compared (i.e., alterations in repeat blocks or vice versa). The results demonstrated modulation dependent on the repeats, which is presumably RS, but did not show modulation due to expectation violations (Kaliukhovich and Vogels, 2011). However, if the animal was exposed to the fixed ordered stimuli for an extended period (they controlled the probability of the order of the stimulus), the neural response to the stimuli of expected order was shown to decrease (Meyer and Olson, 2011; Meyer et al., 2014; Kaposvari et al., 2018). Such a suppression caused by expectation is called expectation suppression (ES). These studies revealed that ES was observed when the animal is under some belief situations (they believe that a particular stimulus comes after a corresponding particular stimulus); however, RS was observed regardless of the belief, suggesting that different neural mechanisms drive the two phenomena (Vogels, 2016; Vinken et al., 2018). RS is suggested to be the result of relatively low-level automatic processes (Feuerriegel et al., 2021), which differs from ES.
Is RS in IT a process distinct from predictive coding? If predictive coding is the fundamental principle of visual computation, why can it not explain phenomena such as RS, which are broadly observed in the ventral visual area, regardless of additional constraints such as attention, awareness, or exposure time? Keller and Mrsic-Flogel (2018) highlighted the challenges in controlling the expectations of animals or systems and emphasized that researchers can access them at best through certain proxies for them. We lack direct access to elements such as “expectations” and “errors” within the visual hierarchy. It is highly likely that the neural correlate of these concepts is obscured in the complex visual system and not straightforward to comprehend. Another possibility, as highlighted by Cao (2020), is that the information processing structure might not essentially differ from the traditional scheme, where stimulus representation ascends across the hierarchy and is integrated through feedforward adjusted by feedback (Heeger, 2017), thus we may not need to choose or deny either. Some predictive coding models are proposed that do not explicitly assume error neurons (Spratling, 2008; Sihn and Kim, 2022). The evaluation of these models is intriguing. Without identifying specific circuits, the appropriate explanation (Shipp, 2024) or whether they essentially express the same concept remains uncertain.
6 RS in fMRI study
In fMRI studies of visual object perception and recognition, research has been conducted on RS and adaptation from several different perspectives. In this section, I discuss fMRI studies focusing on the property and mechanism of RS and adaptation in higher ventral visual areas such as fusiform face areas (FFA).
fMRI adaptation has been used to investigate area selectivity (Grill-Spector and Malach, 2001; Malach, 2012). The logic of the experiments is that the adaptation causes weakened responses to repeated or prolonged stimuli. If altering the properties of a stimulus causes fMRI responses to recover, this is evidence that a distinct population of neurons has been recruited by the stimulus manipulation. Equivalently, stimulus-specific adaptation effects on fMRI responses indicate the presence of neurons that are selective along the dimension of stimulus manipulation (Larsson et al., 2016). However, there is criticism that the fMRI method, which captures mass activity as changes in BOLD signals, makes it difficult to accurately infer the modulation of neuronal activity that varies in many ways, including increased and decreased activity, fatigue, sharpened tuning, response facilitation, and altered response dynamics (Larsson et al., 2016). In addition, slow response obscures the difference between changes within the area and those coming from other areas.
In other studies, a relationship between Predictive Coding and RS has been investigated. Summerfield et al. (2008) demonstrated that the BOLD signal in FFA is greater for unexpected face stimuli compared to expected face stimuli, suggesting that RS is related to expectation rather than adaptation, thereby supporting predictive coding. However, this type of expectation-related modulation requires attention, while RS occurs even in the absence of attention (Larsson and Smith, 2012). Additionally, electrophysiology studies in monkey IT have shown that neural activity modulation related to expectation is not observed with such short exposures (Kaliukhovich and Vogels, 2011), providing negative evidence against this interpretation. However, by analyzing RS from the perspective of how inter-area relationships change, Ewbank et al. (2013) demonstrated a connection with predictive coding. They recorded the BOLD responses of the FFA and OFA to face stimuli of the same or different sizes. They applied dynamic causal modeling to examine the effects of stimulus repetition. They reported that the repetition of the same face was associated with changes in forward (OFA-to-FFA) connectivity. In contrast, the repetition of a face of a different size was characterized by altered backward connectivity (FFA-to-OFA), insisting the finding is consistent with predictive coding. Another study showed that inter-regional (ACC-FFA) coupling of BOLD signal increases by stimulus repetition, indicating another explanation of the mechanism of RS, based on synchrony (Gotts et al., 2021).
7 Other computational implications for RS
So far, I have explored the neural processes and anatomical structures related to RS, drawing insights from lower visual areas. The RS appears to be intimately connected to the computations in the visual areas. Herein, I further delve into the potential significance of RS in processing visual information in the brain. The models I discuss in this section provide insights into RS. I discuss models that explore the dynamics of neural networks in response to changing inputs, focusing on the explanation of RS, or, in a broader sense, on the visual environment dynamics. The first model examines how neural networks with selective inhibitory connections can achieve inference-based visual processing, which explicitly uses prior and posterior probability, and explains RS (Lochmann et al., 2012; Chalk et al., 2018). The second model incorporates hierarchical processing while integrating efficient coding principles (Snow et al., 2017, Młynarski and Hermundstad, 2018, Park and Pillow, 2024). The final discussion includes spiking neural networks that maintain a tight E/I balance and coding stability (Denève et al., 2017; Gutierrez and Denève, 2021). This network model shows how low energy, thus high-efficiency constraint, can achieve stable population coding.
7.1 Inhibitory connections
Numerous models that incorporate divisive normalization (Heeger, 1992) have been proposed given the wealth of experimental findings suggesting that divisive normalization forms the basis of rich contextual modulation (Northoff and Mushiake, 2020). Among them, Lochmann et al.’s (2012) model distinguishes itself from other models for natural RS reproduction. Their spiking neural network model included feed-forward and competitive inhibitory lateral connections. In their study, the network was a generative model that inferred the probability of the existence of an object (or feature) in a visual image, and the inhibitory connections made the inference of different objects competitive. In repeated-stimulus conditions, the probability of the first object (adaptor stimulus) remains high in the period immediately following its disappearance because the probabilities are updated by the slow integration of the sensory input. This explains why the input of repeated stimuli results in a substantial reduction in the detector gain for that stimulus. Their model is not strictly mapped to real neural circuits but is intriguing in that it explains contextual modulation and RS not as a model describing the phenomenon itself, but as an ingredient for realizing the essential purpose of visual processing, which is the inference of the visual world. A subsequent study (Chalk et al., 2018) demonstrated noise-invariant output in the network that had input-targeted divisive inhibition but less constraint on the stimulus and their neural code than the previous one. Another group showed that the network model trained by natural movies learned the statistics of the movie, which exhibited a response similar to adaptation in V1 via divisive normalization over time (Snow et al., 2017). This model includes a modulation mechanism similar to the V1 surround modulation with divisive normalization, which functions as a statistical inference of the classical receptive field using the surrounding information (Coen-Cagli et al., 2015). The model divisively normalizes the present visual input using past visual inputs only to the extent that they are inferred to be statistically dependent. These models explain the role of inhibitory inputs in visual processing and suggest their potential to simultaneously account for RS.
7.2 Multiple aspects of efficiency
High-fidelity encodings in which precise reconstruction of coded information is possible can be metabolically costly, but low-fidelity encodings can lead to errors in inference. The visual system may experience a tradeoff between these two efficiencies. A model based on input statistics was proposed to balance this tradeoff (Młynarski and Hermundstad, 2018). This model aims to maintain an accurate estimate at a minimal cost. Specifically, the current state (stimulus distribution) is estimated from the response as a firing rate, and this estimation, when fed back into the encoding scheme, adjusts the coding fidelity. In other words, the coding fidelity and metabolic costs are prioritized in situations with high uncertainty and certainty, respectively. A Bayesian observer was used to implement the model. They assumed a visual hierarchy as an encoder and observer being the V1 and V2 neurons, respectively, and demonstrated changes in the firing rates depending on the two scenarios. Bursts occur when the statistical distribution of the stimulus (which is assumed to be coded by the V2 neurons) becomes more uncertain, resembling responses to breaking statistical regularities. Conversely, bursts did not occur when the stimulus distribution was similar to the previous one, qualitatively aligning with response suppression.
Park and Pillow (2024) proposed a framework that combines Bayesian inference and efficient coding (efficient Bayesian coding). The framework includes prior distribution, encoding model, capacity constraint, and loss function, making it possible to compare different types of loss function. The study demonstrates multiple cases suggesting that the original efficient coding (information maximization) may not be relevant. Although these models do not directly predict the underlying computation and a clear connection with the roles of different cells in specific circuits of the visual cortex, the perspective of optimizing multiple objectives may be crucial for comprehensively understanding the visual system and adaptation.
7.3 E/I balance
The final topic is the E/I-balanced network. Excitation and inhibition have been shown to be tightly balanced in the brain (ex. Xue et al., 2014). In the network model of Gutierrez and Denève (2021), the adaptation of the spike-frequency and E/I-balanced recurrent connectivity have emerged as solutions to the global cost-accuracy tradeoff. This network redistributes the sensory responses from highly excitable to less excitable neurons as the cost of neural activity increases. This change does not alter representation at the population level, despite dynamic changes in the individual neurons (Gutierrez and Denève, 2021). The idea of a trade-off between metabolic cost and coding accuracy is similar to that of the aforementioned models. This model is unique in that the conflict is solved using the circuit property of the E/I balancing network, which is a characteristic of the brain. In this model, an autoencoder is used, and the optimization of the coding accuracy involves directly representing the input as the output. Investigating how the hierarchy of visual processing and changes in coding are expressed within this context would be intriguing.
8 Discussion
RS or adaptation has been extensively investigated for a long time, and its significance has been debated across various perspectives. Surprisingly, there are still aspects that remain unclear. Considering the widespread observation of adaptation in visual processing, it is speculated to have deep implications for its computational principles of visual systems. Therefore, elucidating adaptation and RS might be intertwined with understanding the visual computational principles. Insights from electrophysiological studies, anatomical observations, and modeling need to be integrated for a comprehensive understanding beyond what is currently available.
Several studies showed that neurons in lower visual areas efficiently code visual stimuli by maximizing the information that can convey natural image inputs. In contrast, in middle and higher visual areas, optimization may not be for the efficient representation of the input image but for the representation of the output, i.e., various behaviors that can be realized. In the case of the IT cortex, which is closest to the output for behavior, this optimization may strongly depend on the behavior.
The computational principle of the visual system depends on the unique features of the visual system including two-dimensional spread, and the temporal dynamics of input resulting from frequent saccades and fixations. These factors may contribute to the specificity of the visual system, and fundamental commonalities with other modalities are possibly concealed. In recent years, research using optogenetics in rodents has expanded, providing valuable insights. While some findings may be translated, cautious interpretation is essential due to potential differences in the hierarchical structure and anatomical circuit between primates and rodents. Massive and hopefully comprehensive recordings of population activity are necessary to unveil the underlying principles.
In summary, IT neurons dynamically adapt their responses to visual stimuli based on experience, showing robust RS. As a neural activity modulation in which responses change upon repetition of the same stimulus, RS and adaptation is prevalent throughout the visual system and transcends hierarchical levels. RS in IT cannot be explained simply by fatigue as it selectively occurs in response to specific stimuli; instead, it is considered a phenomenon related to the fundamental aspects of visual processing. Insights from studies on similar phenomena in lower-order processing help speculate RS in IT neurons. Adaptation involves inhibitory contextual modulation, which may be related to gamma synchronization. Explaining RS from the efficient or predictive coding perspective may be possible; nevertheless, evaluating efficiency is not straightforward and must be carefully considered. Multiple types of efficiency including metabolic efficiency, read-out efficiency, or coding efficiency can contribute simultaneously to the visual system. In addition, contributions can differ in the visual processing stages. The inhibitory neurons as well as excitatory and inhibitory balance may be critical players in adaptation. RS is an intriguing subject for understanding visual processing and connecting various hypotheses and psychological phenomena, and comparisons across different visual hierarchies and between different methods are essential to understanding its mechanism and role.
Statements
Author contributions
YY: Writing – original draft, Writing – review & editing.
Funding
The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This study was supported by the Naito Foundation Subsidy for Female Researchers after Maternity Leave.
Acknowledgments
I thank Ichiro Fujita, Junji Ito, Ko Sakai, and Kenji Doya for their thoughtful comments on the manuscript, and Editage (www.editage.jp) for English language editing.
Conflict of interest
The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1
AitchisonL.LengyelM. (2017). With or without you: predictive coding and Bayesian inference in the brain. Curr. Opin. Neurobiol.46, 219–227. doi: 10.1016/j.conb.2017.08.010
2
AndersonB.MruczekR. E. B.KawasakiK.SheinbergD. (2008). Effects of familiarity on neural activity in monkey inferior temporal lobe. Cereb. Cortex18, 2540–2552. doi: 10.1093/cercor/bhn015
3
AndersonJ. C.MartinK. A. (2009). The synaptic connections between cortical areas V1 and V2 in macaque monkey. J. Neurosci.29, 11283–11293. doi: 10.1523/JNEUROSCI.5757-08.2009
4
AngelucciA.BijanzadehM.NurminenL.FedererF.MerlinS.BressloffP. C. (2017). Circuits and mechanisms for surround modulation in visual cortex. Annu. Rev. Neurosci.40, 425–451. doi: 10.1146/annurev-neuro-072116-031418
5
AngelucciA.LevittJ. B.LundJ. S. (2002). Anatomical origins of the classical receptive field and modulatory surround field of single neurons in macaque visual cortical area V1. Prog. Brain Res.136, 373–388. doi: 10.1016/s0079-6123(02)36031-x
6
AttneaveF. (1954). Some informational aspects of visual perception. Psychol. Rev.61, 183–193. doi: 10.1037/h0054663
7
AuksztulewiczR.FristonK. (2016). Repetition suppression and its contextual determinants in predictive coding. Cortex80, 125–140. doi: 10.1016/j.cortex.2015.11.024
8
BarlowH. B. (1961). “Possible principles underlying the transformations of sensory messages” in Sensory communication Rosenblith. ed. WalterA. (Cambridge, MA: MIT Press), 217–234.
9
BartosM.VidaI.JonasP. (2007). Synaptic mechanisms of synchronized gamma oscillations in inhibitory interneuron networks. Nat. Rev. Neurosci.8, 45–56. doi: 10.1038/nrn2044
10
BartoliE.BoskingW.FosterB. L. (2020). Seeing visual gamma oscillations in a new light. Trends Cogn. Sci.24, 501–503. doi: 10.1016/j.tics.2020.03.009
11
BastosA. M.UsreyW. M.AdamsR. A.MangunG. R.FriesP.FristonK. J. (2012). Canonical microcircuits for predictive coding. Neuron76, 695–711. doi: 10.1016/j.neuron.2012.10.038
12
BaylisG. C.RollsE. T. (1987). Responses of neurons in the inferior temporal cortex in short term and serial recognition memory tasks. Exp. Brain Res.65, 614–622. doi: 10.1007/BF00235984
13
BrennerN.BialekW.de Ruyter van SteveninckR. (2000). Adaptive rescaling maximizes information transmission. Neuron26, 695–702. doi: 10.1016/s0896-6273(00)81205-2
14
BrincatS. L.ConnorC. E. (2004). Underlying principles of visual shape selectivity in posterior inferotemporal cortex. Nat. Neurosci.7, 880–886. doi: 10.1038/nn1278
15
BrownM. W.WilsonF. A.RichesI. P. (1987). Neuronal evidence that inferomedial temporal cortex is more important than hippocampus in certain processes underlying recognition memory. Brain Res.409, 158–162. doi: 10.1016/0006-8993(87)90753-0
16
BrunetN. M.BosmanC. A.VinckM.RobertsM.OostenveldR.DesimoneR.et al. (2014). Stimulus repetition modulates gamma-band synchronization in primate visual cortex. Proc. Natl. Acad. Sci. USA111, 3626–3631. doi: 10.1073/pnas.1309714111
17
CaoR. (2020). New labels for old ideas: predictive processing and the interpretation of neural signals. Phil. Psycol.11, 517–546. doi: 10.1007/s13164-020-00481-x
18
ChalkM.MarreO.TkačikG. (2018). Toward a unified theory of efficient, predictive, and sparse coding. Proc. Natl. Acad. Sci. USA115, 186–191. doi: 10.1073/pnas.1711114115
19
ChenJ.YangH.WangA.FangF. (2010). Perceptual consequences of face viewpoint adaptation: face viewpoint aftereffect, changes of differential sensitivity to face view, and their relationship. J. Vis.10, 1–11. Published 2010 Mar 26. doi: 10.1167/10.3.12
20
CliffordC. W. (2014). The tilt illusion: phenomenology and functional implications. Vis. Res.104, 3–11. doi: 10.1016/j.visres.2014.06.009
21
Coen-CagliR.DayanP.SchwartzO. (2009) Statistical models of linear and non-linear contextual interactions in early visual processing. Adv. Neur. Inform. Process. Syst.22, 369–377.
22
Coen-CagliR.KohnA.SchwartzO. (2015). Flexible gating of contextual influences in natural vision. Nat. Neurosci.18, 1648–1655. doi: 10.1038/nn.4128
23
CortesJ. M.MarinazzoD.SeriesP.OramM.SejnowskiT. J.van RossumM. C. W. (2012). The effect of neural adaptation on population coding accuracy. J. Comput. Neurosci.32, 387–402. doi: 10.1007/s10827-011-0358-4
24
CrowderN. A.PriceN. S. C.HietanenM. A.DreherB.CliffordC. W. G.IbbotsonM. R. (2006). Relationship between contrast adaptation and orientation tuning in V1 and V2 of cat visual cortex. J. Neurophys.95, 271–283. doi: 10.1152/jn.00871.2005
25
De BaeneW.PremereurE.VogelsR. (2007). Properties of shape tuning of macaque inferior temporal neuron examined using rapid serial visual presentation. J. Neurophysiol.97, 2900–2916. doi: 10.1152/jn.00741.2006
26
De BaeneW.VogelsR. (2010). Effects of adaptation on the stimulus selectivity of macaque inferior temporal spiking activity and local field potentials. Cereb. Cortex20, 2145–2165. doi: 10.1093/cercor/bhp277
27
DefelipeJ.Gonzalez-AlboM. C.Del RioM. R.ElstonG. N. (1999). Distribution and patterns of connectivity of interneurons containing calbindin, calretinin, and parvalbumin in visual areas of the occipital and temporal lobes of the macaque monkey. J. Comp. Neurol.412, 515–526. doi: 10.1002/(sici)1096-9861(19990927)412:3<515::aid-cne10>3.0.co;2-1
28
DenèveS.AlemiA.BourdoukanR. (2017). The brain as an efficient and robust adaptive learner. Neuron94, 969–977. doi: 10.1016/j.neuron.2017.05.016
29
DesimoneR. (1996). Neural mechanisms for visual memory and their role in attention. Proc. Natl. Acad. Sci. USA93, 13494–13499. doi: 10.1073/pnas.93.24.13494
30
DhruvN. T.TailbyC.SokolS. H. (2011). Lennie P multiple adaptable mechanisms early in the primate visual pathway. J. Neurosci.31, 15016–15025. doi: 10.1523/JNEUROSCI.0890-11.2011
31
EsmailpourH.RamanR.VogelsR. (2023). Inferior temporal cortex leads prefrontal cortex in response to a violation of a learned sequence. Cereb. Cortex33, 3124–3141. doi: 10.1093/cercor/bhac265
32
EwbankM. P.HensonR. N.RoweJ. B.StoyanovaR. S.CalderA. J. (2013). Different neural mechanisms within occipitotemporal cortex underlie repetition suppression across same and different-size faces. Cereb. Cortex23, 1073–1084. doi: 10.1093/cercor/bhs070
33
FabbriniF.Van den HauteC.De VitisM.BaekelandtV.VanduffelW.VogelsR. (2019). Probing the mechanisms of repetition suppression in inferior temporal cortex with Optogenetics. Curr. Biol.29, 1988–1998.e4. doi: 10.1016/j.cub.2019.05.014
34
FangF.HeS. (2005). Viewer-centered object representation in the human visual system revealed by viewpoint aftereffects. Neuron45, 793–800. doi: 10.1016/j.neuron.2005.01.037
35
FairhallA. L.LewenG. D.BialekW.de Ruyter van SteveninckR. R. (2001). Efficiency and ambiguity in an adaptive neural code. Nature412, 787–792. doi: 10.1038/35090500
36
FeuerriegelD.VogelsR.KovacsG. (2021). Evaluating the evidence for expectation suppression in the visual system. Neurosci. Biobehav. Rev.126, 368–381. doi: 10.1016/j.neubiorev.2021.04.002
37
FriesP. (2009). Neuronal gamma-band synchronization as a fundamental process in cortical computation. Annu. Rev. Neurosci.32, 209–224. doi: 10.1146/annurev.neuro.051508.135603
38
FriesP.ReynoldsJ. H.RorieA. E.DesimoneR. (2001). Modulation of oscillatory neuronal synchronization by selective visual attention. Science291, 1560–1563. doi: 10.1126/science.1055465
39
FujitaI. (2002). The inferior temporal cortex: architecture computation and representation. J. Neurocytol.31, 359–371. doi: 10.1023/a:1024138413082
40
FujitaI.FujitaT. (1996). Intrinsic connections in the macaque inferior temporal cortex. J. Comp. Neurol.368, 467–486. doi: 10.1002/(SICI)1096-9861(19960513)368:4<467::AID-CNE1>3.0.CO;2-2
41
FristonK. (2005). A theory of cortical response. (2005) Philos. Trans. R Soc. Lond. B Biol. Sci.360, 815–836. doi: 10.1098/rstb.2005.1622
42
GasparM. E.PolackP. O.GolshaniP.LengyelM.OrbanG. (2019). Representational untangling by the firing rate nonlinearity in V1 simple cells. eLife8:e43625. doi: 10.7554/eLife.43625
43
GibsonJ. J.RadnerM. (1937). Adaptation, after-effect and contrast in the perception of tilted lines. I. Qu antitative studies. J. Exp. Psychol.20, 453–467. doi: 10.1037/h0059826
44
GoncharY.BurkhalterA. (2003). Distinct GABAergic targets of feedforward and feedback connections between lower and higher areas of rat visual cortex. J. Neurosci.23, 10904–10912. doi: 10.1523/JNEUROSCI.23-34-10904.2003
45
GottsS. J.MillevilleS. C.MartinA. (2021). Enhanced inter-regional coupling of neural responses and repetition suppression provide separate contributions to long-term behavioral priming. Commun Biol.4:487. doi: 10.1038/s42003-021-02002-7
46
Grill-SpectorK.MalachR. (2001). fMR-adaptation: a tool for studying the functional properties of human cortical neurons. Acta Psychol.107, 293–321. doi: 10.1016/s0001-6918(01)00019-1
47
GrossC. G. (2008). Single neuron studies of inferior temporal cortex. Neuropsychologia46, 841–852. doi: 10.1016/j.neuropsychologia.2007.11.009
48
GutierrezG. J.DenèveS. (2021). Population adaptation in efficient balanced networks. eLife8:e46926. doi: 10.7554/eLife.46926
49
HansenB. R.DragoiV. (2011). Adaptation-induced synchronization in laminar cortical circuits. Proc. Natl. Acad. Sci. USA108, 10720–10725. doi: 10.1073/pnas.1102017108
50
HeegerD. J. (1992). Normalization of cell responses in cat striate cortex. (1992) Vis. Neurosci.9, 181–197. doi: 10.1017/s0952523800009640
51
HeegerD. J. (2017). Theory of cortical function. Proc. Natl. Acad. Sci. USA114, 1773–1782. doi: 10.1073/pnas.1619788114
52
HermundstadA. M.BriguglioJ. J.ConteM. M.VictorJ. D.BalasubramanianV.TkačikG. (2014). Variance predicts salience in central sensory processing. eLife3:e03722. doi: 10.7554/eLife.03722
53
HirabayashiT.MiyashitaY. (2014). Computational pronciples of microcircuits for visual object processing in the macaque temporal cortex. Trends Neurosci.37, 178–187. doi: 10.1016/j.tins.2014.01.002
54
HosoyaT.BaccusS. A.MeisterM. (2005). Dynamic predictive coding by the retina. Nature436, 71–77. doi: 10.1038/nature03689
55
HungC. C.CarlsonE. T.ConnorC. E. (2012). Medial axis shape coding in macaque inferotemporal cortex. Neuron74, 1099–1113. doi: 10.1016/j.neuron.2012.04.029
56
HungC. P.KreimanG.PoggioT.DiCarloJ. J. (2005). Fast readout of object identity from macaque inferior temporal cortex. Science310, 863–866. doi: 10.1126/science.1117593
57
ItoM.TamuraH.FujitaI.TanakaK. (1995). Size and position invariance of neuronal responses in monkey inferotemporal cortex. J. Neurophysiol.73, 218–226. doi: 10.1152/jn.1995.73.1.218
58
JiaX.HongH.DiCarloJ. J. (2021). Unsupervised changes in core object recognition behavior are predicted by neural plasticity in inferior temporal cortex. eLife10:e60830. doi: 10.7554/eLife.60830
59
KarK.KrekelbergB. (2016). Testing the assumptions underlying fMRI adaptation using intracortical recordings in area MT. Cortex80, 21–34. doi: 10.1016/j.cortex.2015.12.011
60
KaliukhovichD. A.VogelsR. (2011). Stimulus repetition probability does not affect repetition suppression in macaque inferior temporal cortex. Cereb. Cortex21, 1547–1558. doi: 10.1093/cercor/bhq207
61
KaposvariP.KumarS.VogelsR. (2018). Statistical learning signals in macaque inferior temporal cortex. Cereb. Cortex28, 250–266. doi: 10.1093/cercor/bhw374
62
KellerG. B.Mrsic-FlogelD. (2018). Predictive processing: A canonical cortical computation. Neuron100, 424–435. doi: 10.1016/j.neuron.2018.10.003
63
KhatriV.HartingsJ. A.SimonsD. J. (2004). Adaptation in thalamic barreloid and cortical barrel neurons to periodic whisker deflections varying in frequency and velocity. J. Neurophysiol.92, 3244–3254. doi: 10.1152/jn.00257.2004
64
KianiR.EstekyH.MirpourK.TanakaK. (2007). Object category structure in response patterns of neuronal population in monkey inferior temporal cortex. J. Neurophysiol.97, 4296–4309. doi: 10.1152/jn.00024.2007
65
KoH.HoferS. B.PichlerB.BuchananK. A.SjöströmP. J. (2011). Mrsic-Flogel TD. Functional specificity of local synaptic connections in neocortical networks. Nature473, 87–91. doi: 10.1038/nature09880
66
KobatakeE.WangG.TanakaK. (1998). Effects of shape-discrimination training on the selectivity of inferotemporal cells in adult monkeys. J. Neurophysiol.80, 324–330. doi: 10.1152/jn.1998.80.1.324
67
KohnA. (2007). Visual adaptation: physiology, mechanisms, and functional benefits. J. Neurophysiol.97, 3155–3164. doi: 10.1152/jn.00086.2007
68
KohnA.MovshonJ. A. (2003). Neuronal adaptation to visual motion in area MT of the macaque. Neuron39, 681–691. doi: 10.1016/s0896-6273(03)00438-0
69
KohnA.MovshonJ. A. (2004). Adaptation changes the direction tuning of macaque MT neurons. Nat. Neurosci.7, 764–772. doi: 10.1038/nn1267
70
KokP.de LangeF. P. (2015). “Predictive coding in sensory cortex” in An introduction to model-based cognitive neuroscience. eds. ForstmannB. U.WagenmakersE.-J. (Berlin: Springer Science + Business Media), 221–244.
71
KondoH.HashikawaT.TanakaK.JonesE. G. (1994). Neurochemical gradient along the monkey occipitotemporal cortical pathway. Neuroreport5, 613–616. doi: 10.1097/00001756-199401000-00020
72
KravitzD. J.SaleemK. S.BakerC. I.UngerleiderL. G.MishkinM. (2013). The ventral visual pathway: an expanded neural framework for the processing of object quality. Trends Cogn. Sci.17, 26–49. doi: 10.1016/j.tics.2012.10.011
73
LarssonJ.SmithA. T. (2012). fMRI repetition suppression: neuronal adaptation or stimulus expectation?Cereb. Cortex22, 567–576. doi: 10.1093/cercor/bhr119
74
LarssonJ.SolomonS. G.KohnA. (2016). fMRI adaptation revisited. Cortex80, 154–160. doi: 10.1016/j.cortex.2015.10.026
75
LeopoldD. A.O’TooleA. J.VetterT.BlanzV. (2001). Protptype-referenced shape encoding revealed by high-level aftereffects. Nat. Neurosci.4, 89–94. doi: 10.1038/82947
76
LiN.DiCarloJ. J. (2010). Unsupervise natural visual experience rapidly reshapes size-invariant object representation in inferior temporal cortex. Neuron67, 1062–1075. doi: 10.1016/j.neuron.2010.08.029
77
LiuY.MurrayS. O.JagadeeshB. (2009). Time course and stimulus dependence of repetition-induced response suppression in inferotemporal cortex. J. Neurophys.101, 418–436. doi: 10.1152/jn.90960.2008
78
LochmannT.ErnstU. A.DenèveS. (2012). Perceptual inference predicts contextual modulations of sensory responses. J. Neurosci.32, 4179–4195. doi: 10.1523/JNEUROSCI.0817-11.2012
79
LogothetisN. K.PaulsJ.PoggioT. (1995). Shape representation in the inferior temporal cortex of monkeys. Curr. Biol.5, 552–563. doi: 10.1016/s0960-9822(95)00108-4
80
LundJ. S.YoshiokaT.LevittJ. B. (1993). Comparison of intrinsic connectivity in different areas of macaque monkey cerebral cortex. Cereb. Cortex3, 148–162. doi: 10.1093/cercor/3.2.148
81
MalachR. (2012). Targeting the functional properties of cortical neurons using fMR-adaptation. NeuroImage62, 1163–1169. doi: 10.1016/j.neuroimage.2012.01.002
82
McGuireB. A.GilbertC. D.RivlinP. K.WieselT. N. (1991). Targets of horizontal connections in macaque primary visual cortex. J. Comp. Neurol.305, 370–392. doi: 10.1002/cne.903050303
83
MeyerT.OlsonC. R. (2011). Statistical learning of visual transitions in monkey inferotemporal cortex. Proc. Natl. Acad. Sci. USA108, 19401–19406. doi: 10.1073/pnas.1112895108
84
MeyerT.RamachandranS.OlsonC. R. (2014). Statistical learning of serial visual transitions by neurons in monkey inferotemporal cortex. J. Neurosci.34, 9332–9337. doi: 10.1523/JNEUROSCI.1215-14.2014
85
MeyerT.WalkerC.ChoR. Y.OlsonC. R. (2014). Image familiarization sharpens response dynamics of neurons in inferotemporal cortex. Nat. Neurosci.17, 1388–1394. doi: 10.1038/nn.3794
86
MillerE. K.LiL.DesimoneR. A. (1991). Neural mechanism for working and recognition memory in inferior temporal cortex. Science254, 1377–1379. doi: 10.1126/science.1962197
87
MiyashitaY. (2004). Cognitive memory: cellular and network machineries and their top-down control. Science306, 435–440. doi: 10.1126/science.1101864
88
MiyashitaY.HayashiT. (2000). Neural representation of visual objects: encoding and top-down activation. Curr. Opin. Neurobiol.10, 187–194. doi: 10.1016/s0959-4388(00)00071-4
89
MoranJ.DesimoneR. (1985). Selective attention gates visual processing in the extrastriate cortex. Science229, 782–784. doi: 10.1126/science.4023713
90
Moreno-BoteR.BeckJ.KanitscheiderI.PitkowX.LathamP.PougetA. (2014). Information-limiting correlation. Nat. Neurosci.17, 1410–1417. doi: 10.1038/nn.3807
91
MłynarskiW. F.HermundstadA. M. (2018). Adaptive coding for dynamic sensory inference. eLife7:e32055. doi: 10.7554/eLife.32055
92
MüllerJ. R.MethaA. B.KrauskopfJ.LennieP. (1999). Rapid adaptation in visual cortex to the structure of images. Science285, 1405–1408. doi: 10.1126/science.285.5432.1405
93
NagelK. I.DoupeA. J. (2006). Temporal processing and adaptation in the songbird auditory forebrain. Neuron51, 845–859. doi: 10.1016/j.neuron.2006.08.030
94
NorthoffG.MushiakeH. (2020). Why context matters? Divisive normalization and canonical microcircuits in psychiatric disorders. Neurosci. Res.156, 130–140. doi: 10.1016/j.neures.2019.10.002
95
Op de BeeckH.WagemansJ.VogelsR. (2001). Inferotemporal neurons represent low-dimensional configurations of parameterized shapes. Nat. Neurosci.4, 1244–1252. doi: 10.1038/nn767
96
PattersonC. A.WissigS. C.KohnA. (2013). Distinct effects of brief and prolonged adaptation on orientation tuning in primary visual cortex. J. Neurosci.33, 532–543. doi: 10.1523/JNEUROSCI.3345-12.2013
97
PattersonC. A.WissigS. C.KohnA. (2014). Adaptation disrupts motion integration in the primate dorsal stream. Neuron81, 674–686. doi: 10.1016/j.neuron.2013.11.022
98
ParkI. M.PillowJ. W. (2024). Bayesian efficient coding. bio Rxiv. doi: 10.1101/178418
99
PriceN. S.BornR. T. (2013). Adaptation to speed in macaque middle temporal and medial superior temporal areas. J. Neurosci.33, 4359–4368. doi: 10.1523/JNEUROSCI.3165-12.2013
100
PriceB.GavornikJ. P. (2022). Efficient temporal coding in the early visual system: existing evidence and future directions. Front. Comput. Neurosci.16:929348. doi: 10.3389/fncom.2022.929348
101
RaoR. P.BallardD. H. (1999). Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat. Neurosci.2, 79–87. doi: 10.1038/4580
102
RingoJ. L. (1996). Stimulus specific adaptation in inferior temporal and medial temporal cortex of the monkey. Behav. Brain Res.76, 191–197. doi: 10.1016/0166-4328(95)00197-2
103
RocklandK. S.LundJ. S. (1983). Intrinsic laminar lattice connections in primate visual cortex. J. Comp. Neurol.216, 303–318. doi: 10.1002/cne.902160307
104
RamachandranS.MeyerT.OlsonC. R. (2016). Prediction suppression in monkey inferotemporal cortex depends on the conditional probability between images. J. Neurophysiol.115, 355–362. doi: 10.1152/jn.00091.2015
105
RamachandranS.MeyerT.OlsonC. R. (2017). Prediction suppression and surprise enhancement in monkey inferotemporal cortex. J. Neurophysiol.118, 374–382. doi: 10.1152/jn.00136.2017
106
RamezanpourH.FallahM. (2022). The role of temporal cortex in the control of attention. Curr. Res. Neurobiol.3:100038. doi: 10.1016/j.crneur.2022.100038
107
RayS. (2022). Spike-gamma phase relationship in the visual cortex. Annu. Rev. Vis. Sci.8, 361–381. doi: 10.1146/annurev-vision-100419-104530
108
RayS.MaunsellJ. H. R. (2015). Do gamma oscillations play a role in cerebral cortex?Trends Cogn. Sci.19, 78–85. doi: 10.1016/j.tics.2014.12.002
109
Sanchez-GiraldoL. G.LaskarM. N. U.SchwartzO. (2019). Normalization and pooling in hierarchical models of natural images. Curr. Opin. Neurobiol.55, 65–72. doi: 10.1016/j.conb.2019.01.008
110
SawamuraH.OrbanG. A.VogelsR. (2006). Selectivity of neuronal adaptation does not match response selectivity: a single-cell study of the fMRI adaptation paradigm. Neuron49, 307–318. doi: 10.1016/j.neuron.2005.11.028
111
ScottJ. W. (1977). A measure of extracellular unit responses to repeated stimulation applied to observations of the time course of olfactory responses. Brain Res.132, 247–258. doi: 10.1016/0006-8993(77)90419-x
112
SchwartzO.HsuA.DayanP. (2007). Space and time in visual context. Nat. Rev. Neurosci.8, 522–535. doi: 10.1038/nrn2155
113
SerenoA. B.LehkyS. R. (2018). Attention effects on neural representations for shape and location are stronger in the ventral than dorsal stream. eNeuro5:e0371. doi: 10.1523/ENEURO.0371-17.2018
114
ShamirM. (2014). Emerging principles of population coding: in search for the neural code. Curr. Opin. Neurobiol.25, 140–148. doi: 10.1016/j.conb.2014.01.002
115
SharpeeT. O.SugiharaH.KurganskyA. V.RebrikS. P.StrykerM. P.MillerK. D. (2006). Adaptive filtering enhances information transmission in visual cortex. Nature439, 936–942. doi: 10.1038/nature04519
116
ShippS. (2024). Computational components of visual predictive coding circuitry. Front. Neural Cir.17:1254009. doi: 10.3389/fncir.2023.1254009
117
SigalaN.LogothetisN. K. (2002). Visual categorization shapes feature selectivity in the primate temporal cortex. Nature415, 318–320. doi: 10.1038/415318a
118
SihnD.KimS. P. (2022). Spatio-temporally efficient coding assigns functions to hierarchical structures of the visual system. Front. Comput. Neurosci.16:890447. doi: 10.3389/fncom.2022.890447
119
SimoncelliE. P.OlshausenB. A. (2001). Natural image statistics and neural representation. Annu. Rev. Neurosci.24, 1193–1216. doi: 10.1146/annurev.neuro.24.1.1193
120
SnowM.Coen-CagliR.SchwartzO. (2017). Adaptation in the visual cortex: a case for probing neuronal populations with natural stimuli. F1000Res6:1246. doi: 10.12688/f1000research.11154.1
121
SolomonS. G.KohnA. (2014). Moving sensory adaptation beyond suppressive effects in single neurons. Curr. Biol.24, R1012–R1022. doi: 10.1016/j.cub.2014.09.001
122
SolomonS. G.PeirceJ. W.DhruvN. T. (2004). Lennie P profound contrast adaptation early in the visual pathway. Neuron42, 155–162. doi: 10.1016/s0896-6273(04)00178-3
123
SolomonS. S.TangH.SussmanE.KohnA. (2021). Limited evidence for sensory prediction error response in visual cortex of macaque and humans. Cereb. Cortex31, 3136–3152. doi: 10.1093/cercor/bhab014
124
SpratlingM. W. (2008). Predictive coding as a model of biased competition in visual attention. Vis. Res.48, 1391–1408. doi: 10.1016/j.visres.2008.03.009
125
SpratlingM. W. (2017). A review of predictive coding algorithms. Brain Cogn.112, 92–97. doi: 10.1016/j.bandc.2015.11.003
126
SrihasamK.VincentJ. L.LivingstoneM. S. (2014). Novel domain formation reveals proto-architecture in inferotemporal cortex. Nat. Neurosci.17, 1776–1783. doi: 10.1038/nn.3855
127
SummerfieldC.TrittschuhE. H.MontiJ. M.MesulamM. M.EgnerT. (2008). Neural repetition suppression reflects fulfilled perceptual expectations. Nat. Neurosci.11, 1004–1006. doi: 10.1038/nn.2163
128
TanigawaH.FujitaI.KatoM.OjimaH. (1998). Distribution, morphology, and γ-aminobutyric acid immunoreactivity of horizontally projecting neurons in the macaque inferior temporal cortex. J. Comp. Neurol.401, 129–143. doi: 10.1002/(sici)1096-9861(19981109)401:1<129::aid-cne8>3.0.co;2-d
129
TesileanuT.PiasiniE.BalasubramanianV. (2022). Efficient processing of natural scenes in visual cortex. Front. Cell. Neurosci.16:1006703. doi: 10.3389/fncel.2022.1006703
130
TiesingaP. H.SejnowskiT. J. (2010). Mechanisms for phase shifting in cortical networks and their role in communication through coherence. Front. Hum. Neurosci.4:196. doi: 10.3389/fnhum.2010.00196
131
ToliasA.KelirisG. A.SmirnakisS. M.LogothetisN. K. (2005). Neurons in macaque area V4 acquire directional tuning after adaptation to motion stimuli. Nat. Neurosci.8, 591–593. doi: 10.1038/nn1446
132
ToveeM. J.RollsE. T.RamachandranV. S. (1996). Rapid visual learning in neurons of the primate temporal visual cortex. Neuroreport7, 2757–2760. doi: 10.1097/00001756-199611040-00070
133
UlanovskyN.LasL.NelkenI. (2003). Processing of low-probability sounds by cortical neurons. Nat. Neurosci.6, 391–398. doi: 10.1038/nn1032
134
VaziriS.CarlsonE. T.WangZ.ConnorC. E. (2014). A channel for 3D environmental shape in anterior inferotemporal cortex. Neuron84, 55–62. doi: 10.1016/j.neuron.2014.08.043
135
VinckM.BosmanC. A. (2016). More gamma more predictions: gamma-synchronization as a key mechanism for efficient integration of classical receptive field inputs with surround predictions. Front. Syst. Neurosci.10:35. doi: 10.3389/fnsys.2016.00035
136
VinkenK.Op de BeeckH. P.VogelsR. (2018). Face repetition probability does not affect repetition suppression in macaque inferotemporal cortex. J. Neurosci.38, 7492–7504. doi: 10.1523/JNEUROSCI.0462-18.2018
137
VogelsR. (2016). Sources of adaptation of inferior temporal cortical responses. Cortex80, 185–195. doi: 10.1016/j.cortex.2015.08.024
138
YamaneY.CarlsonE. T.BowmanK. C.WangZ.ConnerC. E. (2008). A neural code for three-dimensional object shape in macaque inferotemporal cortex. Nat. Neurosci.11, 1352–1360. doi: 10.1038/nn.2202
139
YamaneY.ItoJ.JoanaC.FujitaI.TamuraH.MaldonadoP. E.et al. (2023). Neuronal population activity in macaque visual cortices dynamically changes through repeated fixations in active free viewing. eNeuro10:ENEURO.0086-23.2023. doi: 10.1523/ENEURO.0086-23.2023
140
YangJ.LisbergerS. G. (2009). Relationship between adapted neural population responses in MT and motion adaptation in speed and direction of smooth-pursuit eye movements. J. Neurophysiol.101, 2693–2707. doi: 10.1152/jn.00061.2009
141
YuY.SchmidA. M.VictorJ. D. (2015). Visual processing of informative multipoint correlations arises primarily in V2. eLife4:e06604. doi: 10.7554/eLife.06604
142
WainwrightM. J. (1999). Visual adaptation as optima; infromation transmission. Vis. Res.39, 3960–3974. doi: 10.1016/s0042-6989(99)00101-7
143
WalshK. S.McGovernD. P.ClarkA.O'ConnellR. G. (2020). Evaluating the neurophysiological evidence for predictive processing as a model of perception. Ann. N. Y. Acad. Sci.1464, 242–268. doi: 10.1111/nyas.14321
144
WangY.IliescuB. F.MaJ.JosicL.DragoiV. (2011). Adaptive changes in neuronal synchronization in macaque V4. J. Neurosci.31, 13204–13213. doi: 10.1523/JNEUROSCI.6227-10.2011
145
WesterbergJ. A.CoxM. A.DoughertyK.MaierA. (2019). V1 microcircuit dynamics: altered signal propagation suggests intracortical origins for adaptation in response to visual repetition. J. Neurophysiol.121, 1938–1952. doi: 10.1152/jn.00113.2019
146
WiggsC. L.MartinA. (1998). Martin a properties an mechanisms of perceptual priming. Curr. Opin. Neurobiol.8, 227–233. doi: 10.1016/s0959-4388(98)80144-x
147
WissigS. C.KohnA. (2012). The influence of surround suppression on adaptation effects in primary visual cortex. J. Neurophysiol.107, 3370–3384. doi: 10.1152/jn.00739.2011
148
WoloszynL.SheinbergD. L. (2012). Effects of long-term visual experience on response of fistinct classes of single units in inferior temporal cortex. Neuron74, 193–205. doi: 10.1016/j.neuron.2012.01.032
149
XueM.AtallahB. V.ScanzianiM. (2014). Equalizing excitation-inhibition ratios across visual cortical neurons. Nature511, 596–600. doi: 10.1038/nature13321
Summary
Keywords
repetition suppression, adaptation, efficient coding, predictive coding, inferior temporal (IT) cortex
Citation
Yamane Y (2024) Adaptation of the inferior temporal neurons and efficient visual processing. Front. Behav. Neurosci. 18:1398874. doi: 10.3389/fnbeh.2024.1398874
Received
10 March 2024
Accepted
16 July 2024
Published
26 July 2024
Volume
18 - 2024
Edited by
Yasuko Sugase-Miyamoto, National Institute of Advanced Industrial Science and Technology (AIST), Japan
Reviewed by
Akichika Mikami, Chubu Gakuin University, Japan
Kenji Kawano, Kyoto University, Japan
Updates
Copyright
© 2024 Yamane.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Yukako Yamane, ykyamane@gmail.com
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.