Abstract
Traditionally, functional representations in early visual areas are conceived as retinotopic maps preserving ego-centric spatial location information while ensuring that other stimulus features are uniformly represented for all locations in space. Recent results challenge this framework of relatively independent encoding of location and features in the early visual system, emphasizing location-dependent feature sensitivities that reflect specialization of cortical circuits for different locations in visual space. Here we review the evidence for such location-specific encoding including: (1) systematic variation of functional properties within conventional retinotopic maps in the cortex; (2) novel periodic retinotopic transforms that dramatically illustrate the tight linkage of feature sensitivity, spatial location, and cortical circuitry; and (3) retinotopic biases in cortical areas, and groups of areas, that have been defined by their functional specializations. We propose that location-dependent feature sensitivity is a fundamental organizing principle of the visual system that achieves efficient representation of positional regularities in visual experience, and reflects the evolutionary selection of sensory and motor circuits to optimally represent behaviorally relevant information. Future studies are necessary to discover mechanisms underlying joint encoding of location and functional information, how this relates to behavior, emerges during development, and varies across species.
Introduction
In many parts of the brain, neurons are sensitive to changes in small parts of the visual field, sampled by the retina in the back of the eye. Often, nearby neurons in the brain signal changes in nearby visual field locations. This mapping of nearby locations on the retina onto nearby regions of the brain is referred to as retinotopic organization. Of course, neural circuits encode many other aspects of visual stimuli aside from spatial location, including features critical for different visuo-motor behaviors such as identifying objects, tracking or otherwise interacting with objects, and moving through the world. How are these representations of visual space organized in relation to other stimulus features? In early visual areas that exhibit highly organized retinotopic maps, understanding this relationship has focused on the fine scale organization of cortical circuits that ensures uniform coverage for stimulus features across visual space (Swindale et al., 2000; White et al., 2001; Bosking et al., 2002; Buzás et al., 2003; Yu et al., 2005), rather than how response properties might differ for different regions of visual space. Likewise, in higher visual areas that are specialized to represent specific stimulus properties, the spatial locations associated with the stimulus attributes of a given area are only recently being characterized (Groen et al., 2021).
Recent findings open the door to a new perspective on the encoding of visual space and other features by cortical circuits, one that emphasizes specialization of cortical circuits for processing different stimulus features in different regions of visual space. Here we review the evidence for this location-dependent coding across various stages and scales of the visual system. This includes location-biased sampling of functional features in the output ganglion cell layer of the retina, systematic variation of functional properties within conventional retinotopic maps, and novel periodic retinotopic transforms that dramatically illustrate the tight linkage of feature sensitivity, spatial location, and specialized cortical circuitry. The location-dependent feature sensitivity exhibited at the cortical level appears consistent with the positional regularities in naturally experienced visual input, produced by the statistics of the environment as well as the statistics of bodily movements. Furthermore, a comparative analysis of the published literature supports a species-specific correlation between retinotopic location and functional feature processing across the parallel streams, the fundamental organizing scheme of the visual system originally defined based solely on distinct functional feature sensitivity across areas.
Based on our review and analysis of this evidence, we propose that the joint encoding of ego-centric spatial location and functional features is a canonical organizing principle of the visual system, likely reflecting evolutionary pressures that shape neural circuitry to optimally represent regularities in functionally relevant visual information. One implication of this framework is an increased need to study structural and functional properties of the developing and mature retina in order to understand the location-dependent encoding produced by specialized retinal sampling. Another implication is a need to revise experiments that investigate the representation of spatial location and functional feature sensitivities separately, since neural circuitry imposes inextricable dependencies between these features. We present other implications of this new framework and conclude with a discussion of how retinotopic specializations can be used to establish an ethological understanding of neural encoding, within experimental and computational studies of the visual system.
Visual Inputs Received by the Retina Exhibit Location-Dependent Statistics
The first place to look for evidence of joint encoding of retino-centric location and functional features is obviously the visual inputs to the retina, the first stage of visual processing and the interface between the environment and the visual system. Even without specific measurements, one can safely assume that location and stimulus features are not randomly arrayed in the visual environment, leading to statistical biases in the distribution of visual stimuli that fall on different parts of the retina (Simoncelli and Olshausen, 2001; Geisler, 2008). For instance, predators of mice regularly appear overhead in the blue sky. This produces non-uniform features with distinct statistics and behavioral salience, such that the upper and lower visual field of small rodents exhibit distinct color distributions (Qiu et al., 2021; Figure 1A).
FIGURE 1
Other sources of non-uniform retinal input arise from the animal’s sampling of the environment through bodily structure, bodily movements, and other natural behaviors including social communication. Optic-flow related to self-motion of the eyes and body produces distinct structured patterns that can differ across regions of the retina (Angelaki and Hess, 2005; Calow and Lappe, 2007; Bigge et al., 2021). Similarly, movements of the head, especially when not compensated by stabilizing eye movements (Meyer et al., 2020; Michaiel et al., 2020) can produce different spatiotemporal visual inputs in different locations of each retina, and across the left and right retinae. Social behaviors, such as facing other animals during vocal communication in primates produce a high probability for faces in the central as compared to the peripheral visual field (Figure 1B). The position of the eyes, which are in front of the head in primates and predator animals including cats, or on the side of the head in rodents, tree shrews, and many prey animals including rabbits, influences the relative size of the visual field that is binocular vs. monocular (Figure 1C). In addition, convergence and divergence of the eyes can impact the shape and content of the visual field during natural behaviors (Wallace et al., 2013; Holmgren et al., 2021; Johnson et al., 2021). Altogether, these factors influence the relative structure of visual input across the temporal and nasal regions of the left and right retinae. Many prey animals also tend to have eyes close to the ground, creating differences in visual inputs between their lower and upper visual field that is more exaggerated compared to upright primates, arboreal, or flying animals.
Location-Dependent Feature Sensitivity in the Retina and Early Visual Cortex
Given the positional regularities present in their inputs, it stands to reason that retinal circuits exhibit corresponding specializations and convey these to central visual targets. Here we discuss established and recent evidence in support of topographic specializations in the retina and early visual areas of rodents and primates, the two species with the most published data on this topic.
The mouse retina was long assumed to exhibit topologically uniform functional feature sensitivities, but advanced genetic approaches have revealed distinct retinal specializations in this animal (Figure 2A; Bleckert et al., 2014; El-Danaf and Huberman, 2019; Heukamp et al., 2020). The best studied variation concerns the non-uniform distribution of S and M sensitive cone photoreceptors (Szél et al., 1992; Röhlich et al., 1994; Baden et al., 2013; Nadal-Nicolás et al., 2020). This pattern reflects optimal sampling of environmental statistics from the mouse’s perspective (Baden et al., 2013; Qiu et al., 2021). In addition to environmental statistics, traces of self-motion are also apparent across the mouse retina. Motion direction selectivity in a subset of RGC neurons is tightly related to retinal location in a manner that mirrors the visual inputs during forward self-motion (Sabbah et al., 2017). Presumably, this topographic variation aids the animal in estimating its own motion from visual inputs, but this hypothesis has yet to be tested. Topographic specializations have also been discovered in several retinal ganglion cell (RGC) subtypes with unique functional properties (Bleckert et al., 2014; Warwick et al., 2018; Heukamp et al., 2020), although their contribution to behavior (Johnson et al., 2021) is not yet understood.
FIGURE 2
Early visual areas preserve the global topography of the retina. This implies that topographic variation in feature sensitivity in the output of the retina may also be preserved in early visual areas. This is indeed the case for location-dependent color encoding in the mouse retina, which has been shown (Rhim et al., 2017) to be preserved in mouse V1 (Figure 2A). The cortical representation of the lower visual field in V1 is more sensitive to middle (green) and the representation for the upper visual field is more sensitive to short or UV wavelengths, corresponding not only to upstream retinal specializations but also to behavioral chromatic sensitivity across the visual field (Denman et al., 2018).
Given the joint encoding of location and motion direction in the signal relayed by the retina (Sabbah et al., 2017), one might expect to find location-dependent encoding of motion direction or other features related to self-motion in early visual areas. Several groups (de Malmazet et al., 2018; Li et al., 2020) have recently reported the joint encoding of spatial location and motion direction in the mouse superior colliculus (SC), with structures that could support the encoding of self-motion induced optic flow. However, these reports produced conflicting patterns of the retinotopic specialization and corresponding encoding for optic flow. An exhaustive effort to clarify these findings (Chen et al., 2021) by combining multiple measurement modalities, visual stimuli, behavioral states, and sampling large regions of the retinotopic map found no evidence of correlated spatial location and motion direction encoding in mouse SC. Characterizing species-specific patterns of self-motion (Carriot et al., 2017; Bigge et al., 2021) or experienced optic flow would help to resolve whether positional regularities in direction of motion produced by optic-flow are represented in the visual system beyond the retina (Sabbah et al., 2017). Recording neural activity across multiple regions of freely behaving animals will further help to determine specializations related to visual processing in the context of bodily movements and navigation (Saleem, 2020).
We note that not all location-dependent feature specializations must be inherited from the retina, as precise synaptic wiring in the retina to cortex or retino-tectal pathways can produce these specializations de novo. For instance, by definition, binocular sensitivity in the central visual field (Berman et al., 1982) cannot be inherited from the retina and must be computed by combining information from the two retinae. The source of other location-dependent feature specializations in V1, such as an increased sensitivity to coherent motion in the lower visual field (Sit and Goard, 2020), are harder to parse from basic principles and require careful synaptic examination of topographic variation in cortical circuitry.
Long before the introduction of the mouse as a model organism of the visual system, retinal specializations were noted in larger animals including cats, various non-human primates, and humans. In fact, the most obvious retinal specialization is the fovea of humans, some non-human primates (Bringmann et al., 2018), and some non-mammalian species. The fovea (Figure 2B) is a small retinal area densely packed (Wässle et al., 1989; Curcio and Allen, 1990) with photosensitive cones and retinal ganglion cells (RGCs) with small receptive fields, that provide specialized sensitivity to color and fine features in the most central location of the visual field (Wassle and Boycott, 1991; Sinha et al., 2017).
In addition to cell density and corresponding receptive field size, RGC cell type also varies across the central-peripheral gradient of the primate retina. Midget cells have small and color sensitive receptive fields with slow temporal filtering whereas parasol cells have large, color-blind, receptive fields with fast temporal filtering (Shapley and Hugh Perry, 1986; Kolb and Marshak, 2003). Although the absolute numbers of both cell types decrease with distance from the fovea, their ratio changes as a function of eccentricity (Dacey and Petersen, 1992; Dacey, 1993). Near the fovea, midget cells outnumber parasol cells by several to 1, estimated to be as high as 30 to 1 in humans (Dacey and Petersen, 1992), whereas in the periphery the numbers are nearly balanced. This eccentricity-dependent contribution of midget and parasol cells is preserved along the retino-thalamic-cortical pathway to V1 (Azzopardi et al., 1999). Altogether, such topographic variations in cone and RGC density and cell type create feature-encodings specialized for color and fine details in the central visual field and coarse moving patterns in the peripheral visual field of primates (Heukamp et al., 2020). It is possible that other retinal specializations exist in the primate retina, related to other environmental or self-motion related regularities in visual input. However, the relative lack of modern genetic tools for cell-type specific interrogation of primate retinal circuits has produced a dearth of knowledge on topographic variation in primate (Peng et al., 2019; Yan et al., 2020) vs. rodent retinas.
It is relatively difficult to study topographic variation in primate, vs. rodent, cortex for two reasons. First, the brains of most primates exhibit folds which make optical imaging of the full retinotopic map of a visual area rather difficult. Secondly, the genetic and viral techniques that allow optical imaging in rodents do not necessarily translate to primates, although primate-specific viral vectors have recently proven successful (Seidemann et al., 2016; Li et al., 2017). Nonetheless, topographic organization of feature encoding has been studied using electrophysiology in the folded brains of larger primates, revealing feature encodings that vary primarily across the central-peripheral eccentricity axis (Adams and Horton, 2003; Yu et al., 2015).
In primate V1, as one moves from central to peripheral locations of the retinotopic map, there are major changes in properties related to spatial acuity and binocularity (Daniel and Whitteridge, 1961; Adams and Horton, 2003), and more subtle changes related to motion processing (Orban et al., 1986; Yu et al., 2010, 2015; Yu and Rosa, 2014). This gradient reflects the topographical distribution of the different RGC types in primate retina (Azzopardi et al., 1999; Figure 2B) as well as the position of the eyes. But as discussed above, specializations in cortical areas are not always simply inherited from incoming input and may become exaggerated or develop de novo based on non-uniform anatomical, structural, or molecular properties of neurons in different portions of the retinotopic map. This has been nicely demonstrated in marmoset area MT, which exhibits distinct connections with other visual and non-visual areas in different parts of its retinotopic map (Figure 2B). Regions in the central representation are connected to other visual areas like V1, V2, V3, and V4 whereas regions in the far peripheral representation receive inputs mainly from peripheral V1 and area prostriata in the retrosplenial cortex (Palmer and Rosa, 2006). Zooming in further, remarkable retinotopic specializations exist within the V1-MT projection (Mundinano et al., 2019). MT-projecting neurons, in layer 3C of V1, in the central regions of the retinotopic map are mostly spiny stellate whereas neurons in the peripheral representation are mostly pyramidal. Distinct connectivity between the central and peripheral representation of visual areas and the rest of the brain have also been observed in other cortical areas of the marmoset (Majka et al., 2019) as well as in the visual cortex of humans (Griffis et al., 2017; Sims et al., 2021). Future studies using optical imaging in smooth-brained primates may reveal additional topographic specializations in early cortical areas.
Complex Retinotopic Transforms Highlight Location-Dependent Feature Sensitivity
We have shown that within individual visual areas, functional feature sensitivity exhibits location-dependence in a manner that may reflect efficient representation of positional regularities in visual input (Simoncelli and Olshausen, 2001; Geisler, 2008). In this view, there is no a priori reason why all visual areas should exhibit complete topographic representations of retinal location that preserves the planar layout of the retina. Instead, the coverage and geometry of retinotopic maps is inherently related to the functional features encoded within a visual area. Here we review recent discoveries of retinotopic maps that deviate from conventional topographic representation of retinal location and highlight the ubiquitous location-dependent feature sensitivity in the visual system. We use this data to suggest that functional feature encodings may influence, or co-emerge with, retinotopic representations. Other factors, including the developmental time-course of cortical areas have also been proposed to influence the coverage and geometry of retinotopic maps (Rosa, 2002).
Throughout this and the following sections, we use published definition of visual areas, which are usually demarcated by a combination of anatomy, histochemistry, and functional measurements including visual field coverage. However, we remind readers that the rules of area delineation differ among species as inherent differences in their visual systems prevents application of identical rules. For instance, higher order visual areas in the mouse cover a small region of the visual field whereas extrastriate areas in the primate can cover the entire visual field, although in some cases there are significant biases in coverage. We also caution that reports of limited visual field coverage in any species may be due to methodological limitations or subsampling. However, as we explain in the following section, location-dependent feature sensitivity can be seen across the entire visual system of primates and rodents. Therefore, the precise definition of visual area boundaries does not undermine our central proposal that these features are not independently processed by visual circuits.
The conventional description of retinotopic maps is “conformal”: a point-to-point relationship with the retina that preserves both local relationships as well as the global layout of the retinal image (Schwartz, 1977; Sereno et al., 1994; Adams and Horton, 2003; Garrett et al., 2014), albeit with certain deformations (Wolf et al., 1994; Ta et al., 2014), and magnifications (Figure 3A). In V1 of all studied animals, retinotopic maps are simple in that they preserve the orthogonal axes of the retina. As in tree shrew V1, the azimuth axis of the retina is usually mapped along the medial/lateral axis of the brain, and the orthogonal elevation axis is mapped along the orthogonal anterior/posterior cortical axis (Bosking et al., 2000; Manger et al., 2002; Garrett et al., 2014). Deviations from the conformal mapping in V1 were noted in higher visual areas of primates and cats but were often discussed in terms of a “discontinuity” within an otherwise continuous map of the visual field (Sanides and Albus, 1980; Wolf et al., 1994; Roe and Ts’o, 1995). Our recent study (Sedigh-Sarvestani et al., 2021) of the tree shrew visual system showed that retinotopic maps can exhibit all-together different transformations of the visual field that do not preserve the two-dimensional spatial layout of the retina. A recent study (Yu et al., 2020) in the “third tier” visual cortex of primates also reported a non-conformal map, although the particular retinotopic transform was different from our observations in tree shrew V2.
FIGURE 3
Unlike the conventional retinotopy in V1 of many species, the retinotopic transform in tree shrew V2 can be described as a sinusoidal transform (Figure 3B) that converts the 2D planar layout of the retina to a roughly 1D periodic structure. This transform combines elevation and azimuth by mapping them onto the same elongated axis of V2, no longer preserving their global orthogonal relationship. This produces a simple map of elevation and a periodic map of azimuth, with both maps along the length of V2 (Figure 3C).
The periodic map of retinal location oscillates between representations of the most central regions of the visual field with representation of paracentral regions separated by 10–15 visual degrees. This representation is tightly linked to enhanced sensitivity for binocularity and retinal disparity: small changes in the retinal image viewed by the two eyes that can be used to discriminate distance and depth. This suggests that enhanced binocularity and disparity sensitivity are specialized for the most central regions of the visual field. Furthermore, the representation of this region of the visual field has enhanced interhemispheric connectivity. The correlated periodic pattern in visual field location and feature sensitivity is also reflected in the pattern of callosal terminals, cross-hemispheric projections which carry information about the most central regions of the visual field. Thus, the most central region of visual space specialized with enhanced binocular sensitivity also features enhanced interhemispheric connectivity, which may ensure coherence of bilateral responses in this region of visual space.
Therefore, the structured visual input produced by the position of the eyes in the head, as well as the interhemispheric connectivity, are consistent with the enhanced representation of binocular features in the most central regions of the visual field. However, as discussed in the previous sections (Figure 1), such location-dependent encoding can arise due to several other underlying factors. A remarkably similar pattern has been observed (Manger et al., 2002) in secondary areas of the ferret visual cortex (Figures 4A,B) where the sensitivity to retinotopic location and connectivity with the opposite hemisphere vary in a periodic fashion. Future experiments will determine whether functional feature sensitivity varies in a periodic manner tightly connected to retinotopy in ferret secondary visual areas, as it does in the tree shrew (Manger et al., 2002; Rosa and Manger, 2005; Sedigh-Sarvestani et al., 2021).
FIGURE 4
A similar periodic functional feature map exists in primate V2 (Figure 4C), where the sensitivity to binocularity, disparity, and several other features including color change in a periodic manner along the length of this region (Hubel and Livingstone, 1987; Lu et al., 2010; Hubel et al., 2015). It is not clear whether this functional encoding is correlated with an underlying retinotopic periodicity. However, since callosal projections consistently terminate near the representation of the vertical meridian in multiple species, the presence of periodic bands of callosal terminals in V2/V3 of macaques supports a common periodic retinotopic transform across macaques, tree shrews and ferrets (Rosa and Manger, 2005).
Retinotopy in macaque V2 has been measured using various techniques. Widefield fMRI measurements from V2 that exhibit feature stripes do not exhibit corresponding retinotopic stripes but report a smooth map (Nasr et al., 2016; Li et al., 2019; Figure 4D, middle). However, measurement using single-cell electrode recordings report a non-smooth “ratchet” retinotopy (Roe and Ts’o, 1995; Shipp and Zeki, 2002) at the local scale (Figure 4D, left), where smooth linear projections in spatial location representations are interrupted by sudden “switch backs” at the border of functional feature stripes. We propose that the actual retinotopic map in primate V2 exhibits a sinusoidal structure overlying a smooth map (Figure 4D, right). This sinusoidal retinotopy might appear as a “ratchet” pattern when sampled with single electrodes which lack widefield coverage, and it might appear as a smooth map when measured with fMRI which lacks single-cell resolution. Our proposal, consistent with a previous modeling prediction (Swindale, 2007), can be assessed using optical imaging methods such as calcium imaging that combine high spatial resolution with widefield coverage.
In both tree shrews and ferrets, locations along the two axes of the retina are mapped differently. While the central-peripheral or azimuth axis is mapped in a periodic fashion, the elevation axis is mapped in the conventional smooth manner. Future work may reveal a functionally significant rationale for this mapping. Alternatively, it may be a trivial consequence of a periodic transform optimized for enhanced representation of the central visual field. More broadly, flexibility in the geometry of the retinotopic map may enable the representation of visual information in the most functionally relevant regions of the visual field. Conversely, if parts of the visual field, or one of its two axes, are not utilized in the functional role of a brain area, the retinotopic map should reflect a correspondingly limited representation (Kay et al., 2015).
Parallel Stream Organization Reveals Species-Specific Joint Encoding of Location and Features Across the Entire Visual System
We have discussed evidence for joint representation of retinotopic location and functional features in individual visual areas (Figures 1–4). Here we provide evidence for joint encoding of location and features in higher visual areas along the parallel streams that form the rest of the cortical visual system. The dorsal and ventral streams (Figure 5) are considered the fundamental organizing scheme of the primate (Schneider, 1969; Mishkin and Ungerleider, 1982), and more recently the rodent (Wang et al., 2011; Glickfeld and Olsen, 2017; Bennett et al., 2019), visual system. Each stream is composed of multiple visual areas and is distinguished based on several factors including sensitivity to functional features, anatomical connectivity with visual and non-visual areas, and contribution to visuo-motor behaviors.
FIGURE 5
We further show that, when considered together, dorsal and ventral stream areas exhibit biases in their visual field coverage (Yu et al., 2015; Shetht and Young, 2016) broadly consistent with the functional roles attributed to each stream. Furthermore, we highlight that the location bias is species-specific, with primates exhibiting functional biases along the central-peripheral axis of the visual field and rodents exhibiting biases more pronounced along the lower-upper axis (Figure 5). We begin with a discussion of the primate literature where higher order visual areas and parallel streams were first identified.
The study of parallel streams originated with a focus on functional, not retinotopic, sensitivities. Until recently it was generally assumed that higher visual areas in primates did not exhibit precise retinotopy. Therefore, there is a lack of data on topographic variability in feature sensitivity in higher visual areas. However, recent advances in fMRI technology have revealed ubiquitous location-dependent functional sensitivity in higher order areas of the primate visual system (Hasson et al., 2002; Groen et al., 2021). Multiple retinotopic maps have been reported to correlate with previously known functional specializations in ventral temporal/inferior temporal (IT) cortex for faces, scenes, and buildings. Face-preferring regions tend to overlap the central representation whereas scene and building-preferring regions overlap with the peripheral representation of the visual field (Levy et al., 2001). A similar organization was found in the macaque (Arcaro and Livingstone, 2017a). Some regions also exhibited biases for the upper visual field (Silson et al., 2018), perhaps reflecting a lack of functionally distinct visual information relevant for the detection of faces in the lower visual field. Single neuron recordings in macaques confirmed this central specialization, showing that face-selective neurons had large receptive fields that on average preferred stimuli near the fovea (Gomez et al., 2018). Neurons in area V4, also part of the ventral stream and implicated in color processing, emphasize the “central” 30–40° but have not been found to exhibit representations of the far periphery (Gattass et al., 1988).
In contrast to the central visual field bias noted in several areas of the primate ventral stream, there is some evidence to support a relatively reduced magnification of the fovea and a greater coverage of the peripheral visual field in dorsal stream areas such as MT (Rosa and Elston, 1998), and DM (also known as V6), which exhibits an emphasis of the peripheral visual field (Colby et al., 1988; Galletti et al., 1999; Pitzalis et al., 2006). This is consistent with the distribution of the main ganglion cell types that provide a large fraction of inputs to each stream. Near the fovea, a large proportion of all ganglion cells are midget type whereas in the periphery midget and parasol cells are about equally distributed (Dacey, 1993). This results in the midget-recipient ventral stream receiving inputs biased toward the fovea and the parasol-recipient dorsal stream receiving inputs more evenly distributed across the fovea and periphery.
Thus, the functionally distinct parallel streams of the primate visual system appear to exhibit distinct retinotopic biases consistent with their inputs and functional sensitivities. However, there are notable exceptions to this general trend. One exception, as mentioned above, is the peripheral visual field preference of modules within IT, that respond robustly to scenes or buildings (Hasson et al., 2002; Lafer-Sousa and Conway, 2013). Another exception is the relatively larger representation of the central visual field in area LIP, which belongs to the dorsal stream (Ben Hamed et al., 2001).
The parallel streams in the mouse visual cortex have been largely defined by anatomical features (Wang et al., 2011; Glickfeld and Olsen, 2017), and original studies reported relatively subtle differences in functional feature sensitivity (Andermann et al., 2011; Marshel et al., 2011; Juavinett, 2014), when compared to functional specializations in higher visual areas of primates. More recent work (Sit and Goard, 2020; Yu et al., 2021) has shown some differences in feature encoding, as well as in developmental time courses (Murakami et al., 2017; Smith et al., 2017; Salinas et al., 2020), between areas in the dorsal and ventral streams. Area LM, designated to the ventral stream, exhibits higher fidelity representations of texture compared to area AL in the dorsal stream. AL on the other hand is relatively more sensitive to motion (Yu et al., 2021). As proposed by recent modeling studies (Bakhtiari et al., 2021), such feature differences are broadly consistent with the ventral stream participating in recognition and the dorsal stream participating in movement related behaviors, similar to functions attributed to the primate streams.
Despite the sparsity of current evidence for robust feature specializations, the mouse dorsal and ventral streams exhibit clear and pronounced retinotopic biases (Figure 5B). Areas in the dorsal stream collectively tile the lower visual field whereas areas in the ventral stream tile the upper visual field (Zhuang et al., 2017). This retinotopic bias is consistent with the topographic variations (Heukamp et al., 2020) in the mouse retina (Figure 2A), which largely occur along the elevation axis. It is also consistent with location-dependent encodings observed in individual higher visual areas. Specifically, the representation of the lower visual field across the mouse visual system exhibits greater sensitivity to coherent motion (Sit and Goard, 2020), as well as greater sensitivity for binocular disparity (La Chioma et al., 2019) both within and across individual areas. Both location-dependent representations likely reflect natural image statistics related to visual processing in peri-personal or near visual space, which coincides with the lower field in low-lying animals like mice. Therefore, retinotopic separation of functionally distinct visual circuits appears to be the major organizational scheme of the visual system of both primates and rodents.
But why does the main axis of this joint location-feature encoding differ between species (Figures 5A,B)? First, it reflects the main axis of change in the topographical distribution of RGCs in the retina (Figure 5C). Whereas the foveal specialization in macaques exhibits a gradient largely along the central-peripheral axis, the RGC distribution in the mouse retina has a larger deviation along the elevation axis. More importantly, these axes in the retina and cortex can be explained in terms of the visual environment and behaviors of each animal (Figure 5D). The egocentric visual world of a ground-dwelling prey animal contains distinct functionally relevant information, environmental statistics (Qiu et al., 2021), and distance to objects (La Chioma et al., 2019), compared to the upper visual field. On the other hand, the egocentric visual world of the upstanding and social primate contains detailed cues relevant for social communication and order. Therefore, the organization of the visual system of each species reflects the efficient representation of the geometry and positional regularities of their behaviorally relevant visual experience.
It is important to point out that behaviorally relevant visual experience is not always the most frequent or likely visual experience. Therefore, location-dependent feature selectivity patterns may not reflect the statistical structure of the most likely inputs in order to efficiently represent the most relevant inputs, as seen in grasshopper auditory neurons (Machens et al., 2005). Outside of obvious threat or salient environmental events, it is difficult to know which sensory inputs carry more behavior relevance, but a minimal criterion may be the utility to predict future sensory inputs (Palmer et al., 2015; Chalk et al., 2018). While in many situations the best prediction about the future can be estimated from the past, this is not always the case. Therefore, efficient representation of behaviorally relevant inputs may come at the cost of inefficient representation of more frequent but less relevant inputs. This suggests that the use of behaviorally relevant stimulus paradigms is critical for testing theories regarding efficiency of the neural code.
In addition to efficient encoding of salient visual inputs, another explanation for location-dependent feature sensitivity, and other multiplexed inter-dependent feature encodings, may be efficient decoding of sensory inputs for behavioral control. In other words, encoding the multi-dimensional statistics of salient visual inputs may not only confer energetic advantages, it may also have computational benefits such as increasing the accuracy of information available to downstream sensory-motor circuits (Macellaio et al., 2020).
Implications for Future Studies of the Visual System
The evidence highlighted so far demonstrates a new framework for the organization of the visual system where information about spatial location and functional features are jointly encoded throughout the visual system, in a manner that reflects the specific visual experience of different species. In this view, the geometric distribution of functionally relevant information in the egocentric visual field shapes sensory encoding in the retina and throughout the visual system. A major implication is that our understanding of the structure and function of the visual system is constrained by our understanding of the structural and functional properties of the retina in the context of species-specific visual experience. In the past two decades, several groups have contributed to a growing understanding of the properties of the mouse retina, how these properties reflect the visual experience of mice, and how functionally relevant retinal sampling is communicated to the rest of the visual system (Dhande and Huberman, 2014; Baden et al., 2016; Heukamp et al., 2020). Similar knowledge in other model organisms used in systems-level visual neuroscience pales in comparison. This is partially due to the suite of genetic tools available in mice, but it also reflects a prevailing assumption that functionally simple visual inputs gain complexity only through discrete transitions across each area of the visual cortical hierarchy. Our framework suggests, in addition, that visual information is already functionally specific at the retinal stage and gains further complexity not only across hierarchical areas but also within each area. Consequently, we believe there is a need for greater emphasis on characterization of the visual experience and retinal properties of tree shrews, ferrets, cats, primates, and other model organisms used in vision research.
Specifically, additional research is needed to determine the role of internal and externally driven visual experience in development of retinal specializations. In this review we suggest that retinal specializations, such as topographic biases in the density of specific functional classes of RGC cells, are specific to and consistent with the behaviorally relevant visual experience of animals. However, in most cases we do not know if post-natal visual experience plays any causal role in the formation of retinal specializations in mice or in any other species. A further complication is spontaneously generated “visual experience.” Even before the onset of visual experience during development, activity in different locations of the retina exhibits distinct properties due to the presence of spatiotemporally structured spontaneous activity referred to as “retinal waves” (Feller, 1999; Wong, 1999). It has been suggested (Pratt et al., 2016; Ge et al., 2021) that retinal waves act as a simulacrum of future structured visual experience and serve to scaffold optimal visual circuitry. Cross-species measurement and manipulation of retinal waves will produce further insight into the causal role of this source of early activity in the species-specific organization of the visual system. Further work is also needed to understand the role of post-natal visual experience on the formation of location-dependent specializations in cortical areas (Arcaro and Livingstone, 2021, 2017b).
A second implication is that retinotopic maps can and should be utilized to elucidate the functional properties of visual areas. Most circuit-level studies of the visual system utilize retinotopic measurements primarily for practical matters such as delineating area borders or determining regions of interest for neural recording. However, the retinotopic map itself is rarely considered in attempts to decipher neural computation. Furthermore, many studies report neural encoding in a limited part of the retinotopic representation of a visual area. This practice is problematic in light of retinotopic specializations that exhibit distinct functional encoding, structural connectivity, anatomical, and molecular properties across the retinotopic map of the same visual area. A poor understanding of the retinotopic map, or reliance on measurements from only a single region of the map, could mask potential specializations and produce conflicting reports attributed to a particular cortical region. Furthermore, even at scales far smaller than an entire visual area, neural encoding for location and features needs to be studied together since they are inherently processed together in the brain. For instance, the visual signal received by subcortical and cortical areas will contain correlated information about spatial location and functional features due to non-uniform retinal sampling, even when the externally controlled visual input has uniform features at all locations of the display monitor.
The last implication regards computational models of the visual system, which are increasingly used to test theories about brain and behavior that are currently intractable with experimental approaches (Ponce et al., 2019; Richards et al., 2019; Bakhtiari et al., 2021; Lindsay, 2021; Mineault et al., 2021). The large majority of neural networks used to model the visual system (Schrimpf et al., 2018), including convolutional neural networks specifically modeled after the ventral stream and used to recognize object categories (Lindsay, 2021), lack topographic variation in feature encoding. As we have discussed, location-dependent feature sensitivity appears to be a core organizing principle of the entire visual system, influencing anatomy and connectivity, signal transformations, and ultimately behavior. Therefore, the lack of location-dependent filters in computational models hampers their ability to simulate brain-like cortical architecture and consequently, to contribute to our understanding of behaviorally relevant visual encoding. Incorporating topographic variations in feature encodings may allow neural networks to access priors contained in natural “embodied” visual experience (Mineault et al., 2021), built over million years of species-specific evolution. This would not only bring in silico models closer in line with in vivo visual systems, but it may also bridge the gap between species-specific visual inputs and artificial training sets, which lack the structure of natural embodied visual experience (Straub and Rothkopf, 2021). Alternatively, the use of more naturalistic “animal-view” movies as input (Betsch et al., 2004), coupled with architectural flexibility, may allow neural networks to learn topographic specializations predictive of patterns in the retina or other hierarchical layers of the visual system (Doshi and Konkle, 2021; Blauch et al., 2022). Recent work has shown some progress in this direction, with neural networks explicitly incorporating distinct objectives or constraints that result in the emergence of topographic organization and specializations (Plaut and Behrmann, 2011; Wang and Cottrell, 2017; Lee et al., 2020; Doshi and Konkle, 2021; Zhuang et al., 2021; Blauch et al., 2022; Konkle and Alvarez, 2022), shedding light on the origins of these organizational schemes.
Conclusion
In summary, we have presented evidence for location-dependent feature sensitivity as an organizing principle of the entire visual system. This joint encoding framework contrasts with prevailing views that distinguish between representations of the location vs. features of objects in the visual field. Several studies (Milner and Goodale, 2008; McIntosh and Schenk, 2009; Cloutman, 2013; Milner, 2017) have shown that these pathways interact and the information streams are not segregated in the brain. Nonetheless, the two types of information are thought to be encoded by distinct circuits before they merge downstream. In contrast, our framework suggests that object location and identity are often encoded together by the same neuronal circuits, reflecting their inseparable existence in the visual field. We suggest that this principle reflects the evolutionary selection of sensory and motor circuits to optimally represent behaviorally relevant information suited to an animal’s unique sensory-motor demands. Future studies are necessary to discover mechanisms underlying joint encoding of location and functional information, how this relates to behavior, emerges during development, and varies across species.
Publisher’s Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Statements
Author contributions
MSS and DF conceived of the ideas in the manuscript, wrote the manuscript, and approved the submitted version.
Funding
This work was supported by R01 EY006821 (to DF), and the MPFI.
Acknowledgments
We thank Mai Morimoto, Shahab Bakhtiari, and the reviewers for discussion and comments and Dina Popovkina for original scientific illustrations.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
1
AdamsD. L.HortonJ. C. (2003). A precise retinotopic map of primate striate cortex generated from the representation of angioscotomas.J. Neurosci.233771–3789. 10.1523/jneurosci.23-09-03771.2003
2
AndermannM. L.KerlinA. M.RoumisD. K.GlickfeldL. L.ReidR. C. (2011). Functional specialization of mouse higher visual cortical areas.Neuron721025–1039. 10.1016/J.NEURON.2011.11.013
3
AngelakiD. E.HessB. J. M. (2005). Self-motion-induced eye movements: effects on visual acuity and navigation.Nat. Rev. Neurosci.6966–976. 10.1038/NRN1804
4
ArcaroM. J.LivingstoneM. S. (2017a). Retinotopic organization of scene areas in macaque inferior temporal cortex.J. Neurosci.377373–7389. 10.1523/JNEUROSCI.0569-17.2017
5
ArcaroM. J.LivingstoneM. S. (2017b). A hierarchical, retinotopic proto-organization of the primate visual system at birth.eLife6:e26196. 10.7554/eLife.26196
6
ArcaroM. J.LivingstoneM. S. (2021). On the relationship between maps and domains in inferotemporal cortex.Nat. Rev. Neurosci.22573–583. 10.1038/s41583-021-00490-4
7
AzzopardiP.JonesK. E.CoweyA. (1999). Uneven mapping of magnocellular and parvocellular projections from the lateral geniculate nucleus to the striate cortex in the macaque monkey.Vis. Res.392179–2189. 10.1016/S0042-6989(98)00319-8
8
BadenT.BerensP.FrankeK.Román RosónM.BethgeM.EulerT. (2016). The functional diversity of retinal ganglion cells in the mouse.Nature529345–350. 10.1038/nature16468
9
BadenT.SchubertT.ChangL.WeiT.ZaichukM.WissingerB.et al (2013). A tale of two retinal domains: near-Optimal sampling of achromatic contrasts in natural scenes through asymmetric photoreceptor distribution.Neuron801206–1217. 10.1016/j.neuron.2013.09.030
10
BakhtiariS.MineaultP.LillicrapT.PackC. C.RichardsB. A. (2021). The functional specialization of visual cortex emerges from training parallel pathways with self-supervised predictive learning.bioRxiv [Preprint]. 10.1101/2021.06.18.448989
11
Ben HamedS.DuhamelJ. R.BremmerF.GrafW. (2001). Representation of the visual field in the lateral intraparietal area of macaque monkeys: a quantitative receptive field analysis.Exp. Brain Res.140127–144. 10.1007/S002210100785
12
BennettC.GaleS. D.GarrettM. E.NewtonM. L.CallawayE. M.MurphyG. J.et al (2019). Higher-Order thalamic circuits channel parallel streams of visual information in mice.Neuron102477–492.e5. 10.1016/j.neuron.2019.02.010
13
BermanN.PayneB. R.LabarD. R.MurphyE. H. (1982). Functional organization of neurons in cat striate cortex: variations in ocular dominance and receptive-field type with cortical laminae and location in visual fields.J. Neurophysiol.481362–1377. 10.1152/jn.1982.48.6.1362
14
BetschB.EinhäuserW.KördingK.KönigP. (2004). The world from a cat’s perspective – statistics of natural videos.Biol. Cybern.9041–50. 10.1007/s00422-003-0434-6
15
BiggeR.PfefferleM.PfeifferK.StöcklA. (2021). Natural image statistics in the dorsal and ventral visual field match a switch in flight behaviour of a hawkmoth.Curr. Biol.31R280–R281. 10.1016/J.CUB.2021.02.022
16
BlauchN. M.BehrmannM.PlautD. C. (2022). A connectivity-constrained computational account of topographic organization in primate high-level visual cortex.Proc. Natl. Acad. Sci. U.S.A.119:e2112566119. 10.1073/PNAS.2112566119/-/DCSUPPLEMENTAL
17
BleckertA.SchwartzG. W.TurnerM. H.RiekeF.WongR. O. L. (2014). Visual space is represented by nonmatching topographies of distinct mouse retinal ganglion cell types.Curr. Biol.24310–315. 10.1016/J.CUB.2013.12.020
18
BoskingW. H.CrowleyJ. C.FitzpatrickD. (2002). Spatial coding of position and orientation in primary visual cortex.Nat. Neurosci.5874–882. 10.1038/nn908
19
BoskingW. H.KretzR.PucakM. L.FitzpatrickD. (2000). Functional specificity of callosal connections in tree shrew striate cortex.J. Neurosci.202346–2359. 10.1523/JNEUROSCI.20-06-02346.2000
20
BringmannA.SyrbeS.GörnerK.KaczaJ.FranckeM.WiedemannP.et al (2018). The primate fovea: structure, function and development.Prog. Retin. Eye Res.6649–84. 10.1016/J.PRETEYERES.2018.03.006
21
BuzásP.VolgushevM.EyselU. T.KisvárdayZ. F. (2003). Independence of visuotopic representation and orientation map in the visual cortex of the cat.Eur. J. Neurosci.18957–968. 10.1046/J.1460-9568.2003.02808.X
22
CalowD.LappeM. (2007). Local statistics of retinal optic flow for self-motion through natural sceneries.Network18343–374. 10.1080/09548980701642277
23
CarriotJ.JamaliM.ChacronM. J.CullenK. E. (2017). The statistics of the vestibular input experienced during natural self-motion differ between rodents and primates.J. Physiol.5952751–2766. 10.1113/JP273734
24
ChalkM.MarreO.TkaèikG. (2018). Toward a unified theory of efficient, predictive, and sparse coding.Proc. Natl. Acad. Sci. U.S.A.115186–191. 10.1073/PNAS.1711114115/-/DCSUPPLEMENTAL
25
ChenH.SavierE. L.DePieroV. J.CangJ. (2021). Lack of evidence for stereotypical direction columns in the mouse superior colliculus.J. Neurosci.41461–473. 10.1523/JNEUROSCI.1155-20.2020
26
CloutmanL. L. (2013). Interaction between dorsal and ventral processing streams: Where, when and how?Brain Lang.127251–263. 10.1016/J.BANDL.2012.08.003
27
ColbyC. L.GattassR.OlsonC. R.GrossC. G. (1988). Topographical organization of cortical afferents to extrastriate visual area PO in the macaque: a dual tracer study.J. Comp. Neurol.269392–413. 10.1002/CNE.902690307
28
CurcioC. A.AllenK. A. (1990). Topography of ganglion cells in human retina.J. Comp. Neurol.3005–25. 10.1002/CNE.903000103
29
DaceyD. M. (1993). The mosaic of midget ganglion cells in the human retina.J. Neurosci.135334–5355. 10.1523/JNEUROSCI.13-12-05334.1993
30
DaceyD. M.PetersenM. R. (1992). Dendritic field size and morphology of midget and parasol ganglion cells of the human retina.Proc. Natl. Acad. Sci. U.S.A.899666–9670. 10.1073/PNAS.89.20.9666
31
DanielP. M.WhitteridgeD. (1961). The representation of the visual field on the cerebral cortex in monkeys.J. Physiol.159203–221. 10.1113/jphysiol.1961.sp006803
32
de MalmazetD.KühnN. K.FarrowK. (2018). Retinotopic separation of nasal and temporal motion selectivity in the mouse superior colliculus.Curr. Biol.282961.e–2969.e. 10.1016/j.cub.2018.07.001
33
DenmanD. J.LuvianoJ. A.OllerenshawD. R.CrossS.WilliamsD.BuiceM. A.et al (2018). Mouse color and wavelength-specific luminance contrast sensitivity are non-uniform across visual space.eLife7:e31209.10.7554/ELIFE.31209
34
DhandeO. S.HubermanA. D. (2014). Retinal ganglion cell maps in the brain: implications for visual processing.Curr. Opin. Neurobiol.24133–142. 10.1016/J.CONB.2013.08.006
35
DoshiF.KonkleT. (2021). Organizational motifs of cortical responses to objects emerge in topographic projections of deep neural networks.J. Vis.21:2226. 10.1167/JOV.21.9.2226
36
El-DanafR. N.HubermanA. D. (2019). Sub-topographic maps for regionally enhanced analysis of visual space in the mouse retina.J. Comp. Neurol.527259–269. 10.1002/cne.24457
37
FellerM. B. (1999). Spontaneous correlated activity in developing neural circuits.Neuron22653–656. 10.1016/S0896-6273(00)80724-2
38
GallettiC.FattoriP.GamberiniM.KutzD. F. (1999). The cortical visual area V6: brain location and visual topography.Eur. J. Neurosci.113922–3936. 10.1046/J.1460-9568.1999.00817.X
39
GarrettM. E.NauhausI.MarshelJ. H.CallawayE. M. (2014). Topography and areal organization of mouse visual cortex.J. Neurosci.3412587–12600. 10.1523/JNEUROSCI.1124-14.2014
40
GattassR.SousaA.GrossC. (1988). Visuotopic organization and extent of V3 and V4 of the macaque.J. Neurosci.81831–1845. 10.1523/JNEUROSCI.08-06-01831.1988
41
GeX.ZhangK.GribizisA.HamodiA. S.SabinoA. M.CrairM. C. (2021). Retinal waves prime visual motion detection by simulating future optic flow.Science373:eabd0830. 10.1126/SCIENCE.ABD0830
42
GeislerW. S. (2008). Visual perception and the statistical properties of natural scenes.Annu. Rev. Psychol.59167–192. 10.1146/annurev.psych.58.110405.085632
43
GlickfeldL. L.OlsenS. R. (2017). Higher-Order areas of the mouse visual cortex.Annu. Rev. Vis. Sci.3251–273. 10.1146/annurev-vision-102016-061331
44
GomezJ.NatuV.JeskaB.BarnettM.Grill-SpectorK. (2018). Development differentially sculpts receptive fields across early and high-level human visual cortex.Nat. Commun.9:788. 10.1038/s41467-018-03166-3
45
GriffisJ. C.ElkhetaliA. S.BurgeW. K.ChenR. H.BowmanA. D.SzaflarskiJ. P.et al (2017). Retinotopic patterns of functional connectivity between V1 and large-scale brain networks during resting fixation.Neuroimage1461071–1083. 10.1016/j.neuroimage.2016.08.035
46
GroenI. I. A.DekkerT. M.KnapenT.SilsonE. H. (2021). Visuospatial coding as ubiquitous scaffolding for human cognition.Trends Cogn. Sci.2681–96. 10.1016/j.tics.2021.10.011
47
HassonU.LevyI.BehrmannM.HendlerT.MalachR. (2002). Eccentricity bias as an organizing principle for human high-order object areas.Neuron34479–490. 10.1016/S0896-6273(02)00662-1
48
HeukampA. S.WarwickR. A.Rivlin-EtzionM. (2020). Topographic variations in retinal encoding of visual space.Annu. Rev. Vis. Sci.6237–259. 10.1146/annurev-vision-121219-081831
49
HolmgrenC. D.StahrP.WallaceD. J.VoitK.-M.MathesonE. J.SawinskiJ.et al (2021). Visual pursuit behavior in mice maintains the pursued prey on the retinal region with least optic flow.eLife10:e70838. 10.7554/ELIFE.70838
50
HubelD. H.LivingstoneM. S. (1987). Segregation of form, color, and stereopsis in primate area 18.J. Neurosci.73378–3415. 10.1523/jneurosci.07-11-03378.1987
51
HubelD. H.WieselT. N.YeagleE. M.Lafer-SousaR.ConwayB. R. (2015). Binocular stereoscopy in visual areas V-2, V-3, and V-3A of the macaque monkey.Cereb. Cortex25959–971. 10.1093/cercor/bht288
52
JohnsonK. P.FitzpatrickM. J.ZhaoL.WangB.McCrackenS.WilliamsP. R.et al (2021). Cell-type-specific binocular vision guides predation in mice.Neuron1091527–1539.e4. 10.1016/j.neuron.2021.03.010
53
JuavinettA. L. (2014). Pattern and component motion responses in mouse visual cortical areas.Curr. Biol.11393–395. 10.1038/nmeth.2839.A
54
KayK. N.WeinerK. S.Grill-SpectorK. (2015). Attention reduces spatial uncertainty in human ventral temporal cortex.Curr. Biol.25595–600. 10.1016/j.cub.2014.12.050
55
KolbH.MarshakD. (2003). The midget pathways of the primate retina.Doc. Ophthalmol.10667–81. 10.1023/A:1022469002511
56
KonkleT.AlvarezG. A. (2022). A self-supervised domain-general learning framework for human ventral stream representation.Nat. Commun.13:491. 10.1038/s41467-022-28091-4
57
La ChiomaA.BonhoefferT.HübenerM. (2019). Area-Specific mapping of binocular disparity across mouse visual cortex.Curr. Biol.292954–2960.e5. 10.1016/j.cub.2019.07.037
58
Lafer-SousaR.ConwayB. R. (2013). Parallel, multi-stage processing of colors, faces and shapes in macaque inferior temporal cortex.Nat. Neurosci.161870–1878. 10.1038/NN.3555
59
LeeH.MargalitE.JozwikK.CohenM.KanwisherN.YaminsD.et al (2020). Topographic deep artificial neural networks reproduce the hallmarks of the primate inferior temporal cortex face processing network.bioRxiv [Preprint]. 10.1101/2020.07.09.185116
60
LevyI.HassonU.AvidanG.HendlerT.MalachR. (2001). Center-periphery organization of human object areas.Nat. Neurosci.4533–539. 10.1038/87490
61
LiM.LiuF.JiangH.LeeT. S.TangS. (2017). Long-Term two-photon imaging in awake macaque monkey.Neuron931049–1057.e3. 10.1016/J.NEURON.2017.01.027
62
LiX.ZhuQ.JanssensT.ArsenaultJ. T.VanduffelW. (2019). In Vivo identification of thick, thin, and pale stripes of macaque area V2 using submillimeter resolution (f)MRI at 3 T.Cereb. Cortex29544–560. 10.1093/cercor/bhx337
63
LiY.TuranZ.MeisterM. (2020). Functional architecture of motion direction in the mouse superior colliculus.Curr. Biol.303304–3315.e4. 10.1016/J.CUB.2020.06.023
64
LindsayG. W. (2021). Convolutional neural networks as a model of the visual system: past, present, and future.J. Cogn. Neurosci.332017–2031. 10.1162/JOCN_A_01544
65
LuH. D.ChenG.TanigawaH.RoeA. W. (2010). A motion direction map in macaque V2.Neuron681002–1013. 10.1002/nbm.3066.Non-invasive
66
MacellaioM. V.LiuB.BeckJ. M.OsborneL. C. (2020). Why sensory neurons are tuned to multiple stimulus features.bioRxiv [Preprint]. 10.1101/2020.12.29.424235
67
MachensC. K.GollischT.KolesnikovaO.HerzA. V. M. (2005). Testing the efficiency of sensory coding with optimal stimulus ensembles.Neuron47447–456. 10.1016/J.NEURON.2005.06.015
68
MajkaP.RosaM. G. P.BaiS.ChanJ. M.HuoB. X.JermakowN.et al (2019). Unidirectional monosynaptic connections from auditory areas to the primary visual cortex in the marmoset monkey.Brain Struct. Funct.224111–131. 10.1007/S00429-018-1764-4
69
MangerP. R.KiperD.MasielloI.MurilloL.TettoniL.HunyadiZ.et al (2002). The representation of the visual field in three extrastriate areas of the ferret (Mustela putorius) and the relationship of retinotopy and field boundaries to callosal connectivity.Cereb. Cortex12423–437. 10.1093/cercor/12.4.423
70
MarshelJ. H.GarrettM. E.NauhausI.CallawayE. M. (2011). Functional specialization of seven mouse visual cortical areas.Neuron721040–1054. 10.1016/j.neuron.2011.12.004
71
McIntoshR. D.SchenkT. (2009). Two visual streams for perception and action: current trends.Neuropsychologia471391–1396. 10.1016/j.neuropsychologia.2009.02.009
72
MeyerA. F.OKeefeJ.PoortJ. (2020). Two distinct types of eye-head coupling in freely moving mice.Curr. Biol.302116–2130.e6. 10.1016/j.cub.2020.04.042
73
MichaielA. M.AbeE. T. T.NiellC. M. (2020). Dynamics of gaze control during prey capture in freely moving mice.eLife9:e57458. 10.7554/ELIFE.57458
74
MilnerA. D. (2017). How do the two visual streams interact with each other?Exp. Brain Res.2351297–1308. 10.1007/S00221-017-4917-4
75
MilnerA. D.GoodaleM. A. (2008). Two visual systems re-viewed.Neuropsychologia46774–785. 10.1016/J.NEUROPSYCHOLOGIA.2007.10.005
76
MineaultP. J.BakhtiariS.RichardsB. A.PackC. C. (2021). Your head is there to move you around: goal-driven models of the primate dorsal pathway.bioRxiv [Preprint]. 10.1101/2021.07.19.21260802
77
MishkinM.UngerleiderL. G. (1982). Contribution of striate inputs to the visuospatial functions of parieto-preoccipital cortex in monkeys.Behav. Brain Res.657–77. 10.1016/0166-4328(82)90081-X
78
MundinanoI.-C.KwanW. C.BourneJ. A. (2019). Retinotopic specializations of cortical and thalamic inputs to area MT.Proc. Natl. Acad. Sci. U.S.A.11623326–23331. 10.1073/PNAS.1909799116
79
MurakamiT.MatsuiT.OhkiK. (2017). Functional segregation and development of mouse higher visual areas.J. Neurosci.379424–9437. 10.1523/JNEUROSCI.0731-17.2017
80
Nadal-NicolásF. M.KunzeV. P.BallJ. M.PengB. T.KrisnanA.ZhouG.et al (2020). True S-cones are concentrated in the ventral mouse retina and wired for color detection in the upper visual field.eLife9:e56840. 10.7554/ELIFE.56840
81
NasrS.PolimeniJ. R.TootellR. B. H. (2016). Interdigitated color- and disparity-selective columns within human visual cortical areas V2 and V3.J. Neurosci.361841–1857. 10.1523/JNEUROSCI.3518-15.2016
82
OlavarriaJ. F.AbelP. L. (1996). The distribution of callosal connections correlates with the pattern of cytochrome oxidase stripes in visual area V2 of Macaque monkeys.Cereb. Cortex6631–639. 10.1093/cercor/6.4.631
83
OrbanG. A.KennedyH.BullierJ. (1986). Velocity sensitivity and direction selectivity of neurons in areas V1 and V2 of the monkey: influence of eccentricity.J. Neurophysiol.462462–480. 10.1152/JN.1986.56.2.462
84
PalmerS. E.MarreO.BerryM. J.BialekW. (2015). Predictive information in a sensory population.Proc. Natl. Acad. Sci. U.S.A.1126908–6913. 10.1073/PNAS.1506855112/-/DCSUPPLEMENTAL
85
PalmerS. M.RosaM. G. P. (2006). A distinct anatomical network of cortical areas for analysis of motion in far peripheral vision.Eur. J. Neurosci.242389–2405. 10.1111/j.1460-9568.2006.05113.x
86
PengY. R.ShekharK.YanW.HerrmannD.SappingtonA.BrymanG. S.et al (2019). Molecular classification and comparative taxonomics of foveal and peripheral cells in primate retina.Cell1761222–1237.e22. 10.1016/J.CELL.2019.01.004
87
PitzalisS.GallettiC.HuangR. S.PatriaF.CommitteriG.GalatiG.et al (2006). Wide-Field retinotopy defines human cortical visual area V6.J. Neurosci.267962–7973. 10.1523/JNEUROSCI.0178-06.2006
88
PlautD. C.BehrmannM. (2011). Complementary neural representations for faces and words: a computational exploration.Cogn. Neuropsychol.28251–275. 10.1080/02643294.2011.609812
89
PonceC. R.XiaoW.SchadeP. F.HartmannT. S.KreimanG.LivingstoneM. S. (2019). Evolving images for visual neurons using a deep generative network reveals coding principles and neuronal preferences.Cell177999–1009.e10. 10.1016/J.CELL.2019.04.005
90
PrattK. G.HiramotoM.ClineH. T. (2016). An evolutionarily conserved mechanism for activity-dependent visual circuit development.Front. Neural Circuits10:79. 10.3389/FNCIR.2016.00079
91
QiuY.ZhaoZ.KlindtD.KautzkyM.SzatkoK. P.SchaeffelF.et al (2021). Natural environment statistics in the upper and lower visual field are reflected in mouse retinal specializations.Curr. Biol.313233–3247.e6. 10.1016/j.cub.2021.05.017
92
RhimI.Coello-ReyesG.KoH. K.NauhausI. (2017). Maps of cone opsin input to mouse V1 and higher visual areas.J. Neurophysiol.1171674–1682. 10.1152/jn.00849.2016
93
RichardsB. A.LillicrapT. P.BeaudoinP.BengioY.BogaczR.ChristensenA.et al (2019). A deep learning framework for neuroscience.Nat. Neurosci.221761–1770. 10.1038/s41593-019-0520-2
94
RoeA. W.Ts’oD. Y. (1995). Visual topography in primate V2: multiple representation across functional stripes.J. Neurosci.153689–3715. 10.1523/jneurosci.15-05-03689.1995
95
RöhlichP.van VeenT.SzélÁ. (1994). Two different visual pigments in one retinal cone cell.Neuron131159–1166. 10.1016/0896-6273(94)90053-1
96
RosaM. G. P. (2002). Visual maps in the adult primate cerebral cortex: some implications for brain development and evolution.Braz. J. Med. Biol. Res.351485–1498. 10.1590/S0100-879X2002001200008
97
RosaM. G. P.ElstonG. N. (1998). Visuotopic organisation and neuronal response selectivity for direction of motion in visual areas of the caudal temporal lobe of the marmoset monkey (Callithrix jacchus): middle temporal area, middle temporal crescent, and surrounding cortex.J. Comp. Neurol.393505–527. 10.1002/(sici)1096-9861(19980420)393:4<505::aid-cne9>3.0.co;2-4
98
RosaM. G. P.MangerP. R. (2005). Clarifying homologies in the mammalian cerebral cortex: the case of the third visual area (V3).Clin. Exp. Pharmacol. Physiol.32327–339. 10.1111/j.1440-1681.2005.04192.x
99
SabbahS.GemmerJ. A.Bhatia-LinA.ManoffG.CastroG.SiegelJ. K.et al (2017). A retinal code for motion along the gravitational and body axes.Nature546492–497. 10.1038/NATURE22818
100
SaleemA. B. (2020). Two stream hypothesis of visual processing for navigation in mouse.Curr. Opin. Neurobiol.6470–78. 10.1016/j.conb.2020.03.009
101
SalinasK. J.HuhC. Y. L.ZeitounJ. H.GandhiS. P. (2020). Functional differentiation of mouse visual cortical areas depends upon early binocular experience.J. Neurosci.411470–1488. 10.1523/jneurosci.0548-20.2020
102
SanidesD.AlbusK. (1980). The distribution of interhemispheric projections in area 18 of the cat: coincidence with discontinuities of the representation of the visual field in the second visual area (V2).Exp. Brain Res.38237–240. 10.1007/BF00236745
103
SchneiderG. E. (1969). Two visual systems.Science163895–902. 10.1126/science.163.3870.895
104
SchrimpfM.KubiliusJ.HongH.MajajN. J.RajalinghamR.IssaE. B.et al (2018). Brain-Score: Which artificial neural network for object recognition is most brain-like?bioRxiv [Preprint]. 10.1101/407007
105
SchwartzE. L. (1977). The development of specific visual connections in the monkey and the goldfish: outline of a geometric theory of receptotopic structure.J. Theor. Biol.69655–683. 10.1016/0022-5193(77)90374-5
106
Sedigh-SarvestaniM.LeeK.-S.JaepelJ.SatterfieldR.ShultzN.FitzpatrickD. (2021). A sinusoidal transformation of the visual field is the basis for periodic maps in area V2.Neuron1094068–4079.e6. 10.1016/J.NEURON.2021.09.053
107
SeidemannE.ChenY.BaiY.ChenS. C.MehtaP.KajsB. L.et al (2016). Calcium imaging with genetically encoded indicators in behaving primates.eLife5:e16178. 10.7554/ELIFE.16178
108
SerenoM. I.McdonaldC. T.AllmanJ. M. (1994). Analysis of retinotopic maps in extrastriate cortex.Cereb. Cortex4601–620. 10.1093/cercor/4.6.601
109
ShapleyR.Hugh PerryV. (1986). Cat and monkey retinal ganglion cells and their visual functional roles.Trends Neurosci.9229–235. 10.1016/0166-2236(86)90064-0
110
ShethtB. R.YoungR. (2016). Two visual pathways in primates based on sampling of space: exploitation and exploration of visual information.Front. Integr. Neurosci.10:37. 10.3389/fnint.2016.00037
111
ShippS.ZekiS. (2002). The functional organization of area V2, I: speecializatoiin across layers.Vis. Neurosci.19211–231. 10.1017/s0952523802191176
112
SilsonE. H.ReynoldsR. C.KravitzD. J.BakerC. I. (2018). Differential sampling of visual space in ventral and dorsal early visual cortex.J. Neurosci.382294–2303. 10.1523/JNEUROSCI.2717-17.2018
113
SimoncelliE. P.OlshausenB. A. (2001). Natural image statistics and neural representation.Annu. Rev. Neurosci.241193–1216. 10.1146/ANNUREV.NEURO.24.1.1193
114
SimsS. A.DemirayakP.CedotalS.VisscherK. M. (2021). Frontal cortical regions associated with attention connect more strongly to central than peripheral V1.Neuroimage238:118246. 10.1016/j.neuroimage.2021.118246
115
SinhaR.HoonM.BaudinJ.OkawaH.WongR. O. L.RiekeF. (2017). Cellular and Circuit mechanisms shaping the perceptual properties of the primate fovea.Cell413–426.e12. 10.1016/J.CELL.2017.01.005
116
SitK. K.GoardM. J. (2020). Distributed and retinotopically asymmetric processing of coherent motion in mouse visual cortex.Nat. Commun.11:3565. 10.1038/s41467-020-17283-5
117
SmithI. T.TownsendL. B.HuhR.ZhuH.SmithS. L. (2017). Stream-dependent development of higher visual cortical areas.Nat. Neurosci.20200–208. 10.1038/nn.4469
118
StraubD.RothkopfC. A. (2021). Looking for image statistics: active vision with avatars in a naturalistic virtual environment. Front. Psychol. 12:641471, 10.3389/fpsyg.2021.641471
119
SwindaleN. V. (2007). A model for the thick, thin and pale stripe organization of primate V2.Netw. Comput. Neural Syst.18327–342. 10.1080/09548980701648472
120
SwindaleN. V.ShohamD.GrinvaldA.BonhoefferT.HübenerM. (2000). Visual cortex maps are optimized for uniform coverage.Nat. Neurosci.38822–826. 10.1038/77731
121
Szél RöhlichP.GafféA. R.JuliussonB.AguirreG.Van VeenT. (1992). Unique topographic separation of two spectral classes of cones in the mouse retina.J. Comp. Neurol.325327–342. 10.1002/CNE.903250302
122
TaD.ShiJ.BartonB.BrewerA.LuZ.-L.WangY. (2014). “Characterizing human retinotopic mapping with conformal geometry: a preliminary study,” in Proceedings of the Medical ImagingedsOurselinS.StynerM. A. (San Diego, CA: SPIE Medical Imaging) 10.1117/12.2043570
123
WallaceD. J.GreenbergD. S.SawinskiJ.RullaS.NotaroG.KerrJ. N. D. (2013). Rats maintain an overhead binocular field at the expense of constant fusion.Nature49865–69. 10.1038/nature12153
124
WangP.CottrellG. W. (2017). Central and peripheral vision for scene recognition: a neurocomputational modeling exploration.J. Vis.17:9. 10.1167/17.4.9
125
WangQ.GaoE.BurkhalterA. (2011). Gateways of ventral and dorsal streams in mouse visual cortex.J. Neurosci.311905–1918. 10.1523/JNEUROSCI.3488-10.2011
126
WarwickR. A.KaushanskyN.SaridN.GolanA.Rivlin-EtzionM. (2018). Inhomogeneous encoding of the visual field in the mouse retina.Curr. Biol.28655–665.e3. 10.1016/j.cub.2018.01.016
127
WassleH.BoycottB. B. (1991). Functional architecture of the mammalian retina.Physiol Rev.447447–480. 10.1152/PHYSREV.1991.71.2.447
128
WässleH.GrünertU.RöhrenbeckJ.BoycottB. B. (1989). Cortical magnification factor and the ganglion cell density of the primate retina.Nature341643–646. 10.1038/341643a0
129
WhiteL. E.BoskingW. H.FitzpatrickD. (2001). Consistent mapping of orientation preference across irregular functional domains in ferret visual cortex.Vis. Neurosci.1865–76. 10.1017/S095252380118106X
130
WolfF.BauerH. U.GeiselT. (1994). Formation of field discontinuities and islands in visual cortical maps.Biol. Cybern.70525–531. 10.1007/BF00198805
131
WongR. O. L. (1999). Retinal waves and visual system development.Annu. Rev. Neurosci.2229–47. 10.1146/annurev.neuro.22.1.29
132
YanW.PengY. R.van ZylT.RegevA.ShekharK.JuricD.et al (2020). Cell atlas of the human fovea and peripheral retina.Sci. Rep.101:9802. 10.1038/s41598-020-66092-9
133
YuH.FarleyB. J.JinD. Z.SurM. (2005). The coordinated mapping of visual space and response features in visual cortex.Neuron47267–280. 10.1016/j.neuron.2005.06.011
134
YuH. H.ChaplinT. A.RosaM. G. P. (2015). Representation of central and peripheral vision in the primate cerebral cortex: insights from studies of the marmoset brain.Neurosci. Res.9347–61. 10.1016/j.neures.2014.09.004
135
YuH. H.RosaM. G. P. (2014). Uniformity and diversity of response properties of neurons in the primary visual cortex: selectivity for orientation, direction of motion, and stimulus size from center to far periphery.Vis. Neurosci.3185–98. 10.1017/S0952523813000448
136
YuH.-H.RowleyD.PriceN.RosaM.ZavitzE. (2020). A twisted visual field map in the primate cortex predicted by topographic continuity.Sci. Adv.6:eaaz8763. 10.1101/682187
137
YuH.-H.VermaR.YangY.TibballsH. A.LuiL. L.ReserD. H.et al (2010). Spatial and temporal frequency tuning in striate cortex: functional uniformity and specializations related to receptive field eccentricity.Eur. J. Neurosci.311043–1062. 10.1111/J.1460-9568.2010.07118.X
138
YuY.StirmanJ. N.DorsettC. R.SmithS. L. (2021). Selective representations of texture and motion in mouse higher visual areas.bioRxiv [Preprint]. 10.1101/2021.12.05.471337
139
ZhuangC.YanS.NayebiA.SchrimpfM.FrankM. C.DiCarloJ. J.et al (2021). Unsupervised neural network models of the ventral visual stream.Proc. Natl. Acad. Sci. U.S.A.118:e2014196118. 10.1073/PNAS.2014196118/-/DCSUPPLEMENTAL
140
ZhuangJ.NgL.WilliamsD.ValleyM.LiY.GarrettM.et al (2017). An extended retinotopic map of mouse cortex.eLife6:e18372. 10.7554/elife.18372
Summary
Keywords
retinotopy, cortical maps, topography, visual field specializations, cross-species, tree shrews
Citation
Sedigh-Sarvestani M and Fitzpatrick D (2022) What and Where: Location-Dependent Feature Sensitivity as a Canonical Organizing Principle of the Visual System. Front. Neural Circuits 16:834876. doi: 10.3389/fncir.2022.834876
Received
13 December 2021
Accepted
01 March 2022
Published
12 April 2022
Volume
16 - 2022
Edited by
Edward S. Ruthazer, McGill University, Canada
Reviewed by
Michael Arcaro, University of Pennsylvania, United States; Andrea Benucci, RIKEN Center for Brain Science (CBS), Japan
Updates
Copyright
© 2022 Sedigh-Sarvestani and Fitzpatrick.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Madineh Sedigh-Sarvestani, msarvestani@gmail.com
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.