The Role of Neuronal Oscillations in Visual Active Sensing
- 1Department of Neurological Surgery, College of Physicians and Surgeons, Columbia University, New York, NY, United States
- 2Translational Neuroscience Laboratories, The Nathan S. Kline Institute for Psychiatric Research, Orangeburg, NY, United States
Visual perception is most often studied as a “passive” process in which an observer fixates steadily at point in space so that stimuli can be delivered to the system with spatial precision. Analysis of neuronal signals related to vision is generally keyed to stimulus onset, stimulus movement, etc.; i.e., events external to the observer. In natural “active” vision, however, information is systematically acquired by using eye movements including rapid (saccadic) eye movements, as well as smooth ocular pursuit of moving objects and slower drifts. Here we consider the use of alternating saccades and fixations to gather information from a visual scene. The underlying motor sampling plan contains highly reliable information regarding “where” and “when” the eyes will land, this information can be used predictively to modify firing properties of neurons precisely at the time when this “contextual” information is most useful – when a volley of retinal input enters the system at the onset of each fixation. Analyses focusing on neural events leading to and resulting from shifts in fixation, as well as visual events external to the observer, can provide a more complete and mechanistic understanding of visual information processing. Studies thus far suggest that active vision may be a fundamentally different from that process we usually study with more traditional passive viewing paradigms. In this Perspective we note that active saccadic sampling behavior imposes robust temporal patterning on the activity of neuron ensembles and large-scale neural dynamics throughout the brain’s visual pathways whose mechanistic effects on information processing are not yet fully understood. The spatio-temporal sequence of eye movements elicits a succession of temporally predictable quasi-rhythmic sensory inputs, whose encoding is enhanced by entrainment of low frequency oscillations to the rate of eye movements. Review of the pertinent findings underscores the fact that temporal coordination between motor and visual cortices is critical for understanding neural dynamics of active vision and posits that phase entrainment of neuronal oscillations plays a mechanistic role in this process.
In natural vision, information is nearly always available at the retina. Yet, in primates only the central visual field has sufficient density of retinal photoreceptors to permit high-resolution vision (Curcio et al., 1990). Consequently, both human and non-human primates sample visual space actively by systematically shifting eye gaze, using the fovea (i.e., the central portion of the retina) to sample points of interest (Figures 1A,B) rather than holding gaze constant and simply absorbing a continuous inflow of input (Yarbus, 1967; Gilchrist et al., 1997). Studies in rodents actively sampling the environment by sniffing and whisking prompted the use of the term “active sensing” (Ahissar and Arieli, 2001; Szwed et al., 2003; Kleinfeld et al., 2006). Numerous papers have raised the logical proposition, that this active sensing regime should apply to visual sensing in primates including humans (e.g., Ahissar and Arieli, 2001, 2012; Schroeder et al., 2010), and indeed there is evidence for this view (Purpura et al., 2003; Rajkai et al., 2008; Bartlett et al., 2011; Ito et al., 2011; Hoffman et al., 2013; Jutras et al., 2013; Hamamé et al., 2014; Staudigl et al., 2017; Katz et al., 2018; Barczak et al., 2019). Despite these well-known attributes of natural active visual behavior, experiments designed to study visual sensory and cognitive functions usually require participants to fixate gaze at a central location while visual stimuli are presented. This traditional “passive” approach, has been extremely productive, but it has several major limitations. First, the very idea that eye position is ever static during awake viewing, even when fixation is required, is countered by a wealth of empirical observations (see review by Ahissar and Arieli, 2001). Second, eye movements reflect the observer’s information gathering strategy, and the traditional paradigm disconnects the timing and quality of visual input from this strategy. Finally, because of this disconnection, the ordering of perception and action is obscured; i.e., in natural vision, actions usually precede and follow sensory inputs forming a basic sampling triplet eye movement-perception-eye movement. Because the major sensory input (entering the retina at fixation onset) is in effect “caused” by the eye movement, the timing of the associated visual input is largely predictable at the time of movement initiation. Thus, unlike the traditional approach, in natural vision, the timing of sensory input is controlled by the brain’s motor sampling routine and can be utilized in processing of sensory information (Yarbus, 1967; Bahill et al., 1975). In short, there is a fundamental distinction between visual Active Sensing, and the more passive processes studied with gaze held constant.
Figure 1. Low frequency phase reset as a mechanism for active sensing. (A) An image with overlapped traces from eye tracker showing aggregated eye position across an interval of 1 s (blue line plot). Circle reflects schematic depiction of fovea across three consecutive fixations. (B) Horizontal eye position across three saccades. (C) Schematic depiction of oscillatory field potential phase reset locked to saccade onset in occipital (alpha ∼8–12 Hz) and medial temporal lobe (theta ∼4–7 Hz). Locally dominant oscillations (i.e., alpha in occipital cortex and theta in MTL) are reset at the time of saccade onset.
In this Perspective we highlight the role of active saccadic exploration in visual cognition. In particular, we review evidence supporting a role of predictive phase reset of neuronal oscillations (see Figure 1C) which enhances sensory processing by amplifying responses and by facilitating transfer of information between areas. Next, we consider the idea that this signal, entrained by a motor sampling (saccadic exploration) routine, modulates local neural activity and organizes perception into discrete “events” constraining “sampling rate” of the visual system. Third, we explore the complementarity of our perspective with earlier work on active sensing in vision and somatosensation (Ahissar and Arieli, 2001, 2012). Finally, we outline translational implications of Active Sensing and some future directions.
A Role for Low Frequency Neural Oscillations in Active Visual Sensing
Field potentials (FP) in scalp and intracranial recordings are largely generated by synchronous transmembrane currents occurring in ensemble of neurons around the electrode site (Mitzdorf, 1985; Schroeder et al., 1995). Analysis of the current source density (CSD) profiles across layers of primary sensory cortex revealed that oscillations in FP are generated by rhythmic pulses of transmembrane current flow which reflect shifts between depolarized and hyperpolarized states in the local neuronal ensemble (Lakatos et al., 2005; Womelsdorf et al., 2006; Haegens et al., 2015; see Figure 2). Accordingly, low frequency oscillatory phase modulates neural firing (Buzsáki and Draguhn, 2004; Lakatos et al., 2005; Figure 2), as well as the related broad-band high frequency activity (BHA; Ray et al., 2008; Leszczynski et al., 2019) sampled from the pial surface of the cortex (e.g., Canolty et al., 2006; Leszczyński et al., 2017). Because of this, the phase at which visual input arrives will determine whether it is amplified or attenuated (Schroeder and Lakatos, 2009) and consequently the probability that the input will be perceived. Under the Active Visual Sensing hypothesis, the neural circuitry that generates saccades is the source of signals which predictively reset ongoing oscillations to the high-excitability phase at the time of fixation onset.
Figure 2. (A,i) Theta-band (5–9 Hz band pass) oscillatory activity from a lower supragranular site in primary auditory cortex (asterisks at left) superimposed on the underlying current source density (CSD) profile for the supregranular layers. Net outward transmembrane current flow generates net extracellular current sources (blue), whereas net inward current flow generates current sinks (red). The theta oscillation at this site represents the “underside” of the superficial current dipole so that negative deflections correspond to current sinks and positive deflections reflect current sources, alternating at a theta rhythm. (A,ii) Multiunit activity (MUA) simultaneously recorded from the same site. Drop lines are provided to show the relationship between the initial three negative deflections and sinks at this site and MUA correlates. Note that current sinks and sources correspond to MUA peaks and troughs, indicating alternations in local neuronal excitability. (B) Relation between gamma-band (30–90 Hz) oscillatory phase and neuronal firing (MUA) from a recording in macaque visual area V4. Vertical lines at the bottom represent occurrence of action potentials. Panel (A) reprinted from Schroeder and Lakatos (2009) with permission from Elsevier. Panel (B) reprinted from Womelsdorf et al. (2006) with permission from Springer.
An accumulation of findings across sensory systems (Ahissar and Arieli, 2001; Szwed et al., 2003; Kleinfeld et al., 2006; Schroeder et al., 2010; Morillon et al., 2015, 2016; Merchant et al., 2015; Tort et al., 2018) and even the motor system (Saleh et al., 2010) suggests that active sensing employs entrainment of euronal excitability fluctuations (oscillations) to facilitate information processing. Entraining neuronal oscillations to the rhythm of the sampling strategy can help the observer to reduce and chunk or otherwise structure sensory input streams into sequences of perceptual events. This has two profound consequences for cognition and neural dynamics. First, because only some information is being sampled, chunking limits or “down-samples” the amount of information that must be processed, thereby minimizing computational burden and optimizing use of resources. Second, quasi-rhythmic sampling behavior (i.e., the spatio-temporal pattern of eye movements) with its deterministic properties allows predictive modulation of neuronal excitability in sensory systems, further optimizing sensory encoding. In the context of visual active sensing, neuronal entrainment of sensory cortices is believed to be orchestrated by outputs from motor system which predictively reset oscillations throughout the system to a high excitability phase near the time of fixation onset (Purpura et al., 2003; Rajkai et al., 2008; Bartlett et al., 2011; Ito et al., 2011; Hoffman et al., 2013; Jutras et al., 2013; Staudigl et al., 2017; Katz et al., 2018; Barczak et al., 2019). By aligning ensemble excitability fluctuations with a rhythmic event stream, entrainment can potentially amplify neuronal responses to the events in the stream (Lakatos et al., 2008); in this case, events correspond to inputs that enter at the moment of fixation (Rajkai et al., 2008). Based on earlier work, phase entrainment to the saccade-fixation-saccade cycle could potentially lower perceptual thresholds (VanRullen, 2016), and provide a reference frame for spike-phase coding (Kayser et al., 2009).
Local Neural Activity Reflects an Interaction Between Saccade- and Input-Locked Signals
Every time we move our eyes a volley of visual input is initiated in the retina and this information is then processed by a succession of neural ensembles in areas staged along the brain’s visual pathways (Felleman and Van Essen, 1991). Because the retinal input during each eye movement is suppressed (Ross et al., 2001) as early as lateral geniculate nucleus of the thalamus and V1 (Sylvester et al., 2005), information that propagates through the visual hierarchy are temporally structured forming a sequence of “visual samples.” According to the Active Visual Sensing hypothesis as outlined here, local neural response to visual samples reflect an interaction between retinal “driving” inputs and “modulatory” non-retinal signals. The former carry visual information and can cause (drive) action potentials. The latter signals control (modulate) the probability that the retinal inputs will drive action potentials, and they likely originate in the network of cortical and sub-cortical areas involved in saccade planning (for review see Girard and Berthoz, 2005; Gilbert and Li, 2013). The modulatory signals operate in part by phase-reset of ongoing low frequency oscillations (Ahissar and Arieli, 2001; Szwed et al., 2003; Lakatos et al., 2008; Schroeder and Lakatos, 2009). This should be evident as increased inter-trial phase coherence in oscillatory activity at the rate of saccades (see Figure 1C).
Indeed, signals with such characteristics have been observed in several parts of the primate brain including primary visual cortex (Rajkai et al., 2008; Ito et al., 2011) but also higher order cortices like the temporal lobe and superior temporal sulcus (Purpura et al., 2003; Bartlett et al., 2011), as well as medial temporal lobe and the hippocampus in humans (Hoffman et al., 2013; Staudigl et al., 2017; Katz et al., 2018) and non-human (Jutras et al., 2013) primates. Importantly, stronger phase reset has been associated with saccade onset rather than fixation onset (Ito et al., 2011; Katz et al., 2018) which is in line with its “top-down” and “modulatory” as opposed to “bottom-up” and “driving” characteristics. This non-retinal signal seems to operate by interacting with local population activity in producing excitation (Rajkai et al., 2008) and modulating the timing of visually evoked spiking at fixation onset (Ito et al., 2011). Local activity during natural exploration would therefore reflect an interaction between saccade-locked “modulatory” and input-locked “driving” signals.
Discrete Perception as a Consequence of Quasi-Rhythmic Exploration
According to the Active Sensing hypothesis at each fixation low frequency oscillations are reset to the high-excitability phase modulating neural activity and consequently perceptual threshold. In line with this suggestion attention performance is not uniform across saccade-fixation-saccade cycle but increases close to fixation onset (Jonikaitis et al., 2012; see also Deubel and Schneider, 1996; Rolfs et al., 2011). The current hypothesis suggests further that perceptual threshold should oscillate rhythmically at the rate of exploration with intervals of higher and lower performance (see also VanRullen and Koch, 2003). This hypothesis has gained substantial support particularly in the visual system. For example, the phase of ongoing EEG oscillations in range of alpha/theta was observed to predict near-threshold perception and modulate the likelihood of detecting a target (Busch et al., 2009; see also Mathewson et al., 2009). Similarly, the phase of ongoing oscillations has been observed to influence peri-saccadic mislocalization (McLelland et al., 2016). These effects establish a relation between the phase of ongoing oscillations and perception suggesting that visual inputs are not processed equally across time. Importantly, covert spatial attention, which can shift across a scene independently of eye position, nonetheless shares common control circuits with overt saccadic search (Rizzolatti et al., 1994; Kustov and Robinson, 1996; Corbetta, 1998). Periodicity in visual perception has been shown to originate from rhythmic sampling imposed by attention (Busch and VanRullen, 2010; VanRullen et al., 2014; VanRullen, 2016). This is further supported by work from Fiebelkorn et al. (2013) who sampled near-threshold perception during systematically varied cue-target interval and observed attention cycling at 4–8 Hz (see also Landau and Fries, 2012; Song et al., 2014). This rhythmicity has been also hypothesized to depend on dynamic interplay within the fronto-parietal (“eye movement control”) network in both human (Helfrich et al., 2018) and non-human (Fiebelkorn et al., 2018) primates. Altogether, these studies imply that like perception, attention is periodic rather than continuous with underlying sampling in theta/alpha frequency range.
A blinking or flickering spotlight of attention and perception with repeated fluctuations in perceptual acuity lasting several 100 ms might appear an inefficient mechanism if considered without further assumptions. However, one could argue that such a mechanism is advantageous for survival, as it allows the brain to monitor events outside of the focus of attention (e.g., Landau and Fries, 2012; Fiebelkorn et al., 2013).
Visual Active Sensing Beyond Exploratory Saccadic Sampling
We use large exploratory saccades which predictively relocate the fovea in a search for information as our prime example of active sensing, because they are easy to measure accurately and because vision in human and non-human primates relies heavily on these movements (Yarbus, 1967; Gilchrist et al., 1997; Liversedge and Findlay, 2000; Henderson, 2017). However, similar principles apply to other types of eye and body movements across sensory modalities. For example, the same active sampling strategy is applied in olfactory and somatosensory processing as exemplified by sniffing (Mainland and Sobel, 2005; Wachowiak, 2011) and whisking (Ahissar and Arieli, 2001; Kleinfeld et al., 2006) in rodents. Indeed the conceptual and empirical foundations for our work come from these studies. On the one hand, several parts of our formulation dovetail with earlier formulations of visual active sensing (Ahissar and Arieli, 2001, 2012), particularly as applied to fixational eye movements (FeyeM) in primates. FeyeM are the very small (<<1.0∘) drift, and tremor eye movements that occur during visual fixation (Ahissar and Arieli, 2001). Most have supposed that these, along with “microsaccades” (Martinez-Conde et al., 2013), serve to keep retinal images from fading during fixation, though Ahissar and Arieli (2001, 2012) have proposed that FeyeM in particular reflect an active sensing mechanism that allows fine detail and texture vision at a level of resolution beyond that offered by the density of cone receptors in the retina. They have also speculated that retinal activity driven by FeyeM may drive thalamocortical oscillatory activity loops that help the visual system form a systematic representation of visual form texture across the surface of V1. On the other hand, several of our findings prompt conclusions about visual active sensing that differ distinctly from earlier ones. In particular, activity in the oscillatory thalamocortical loops associated with FeyeM are specifically driven by retinal input, whereas neuronal oscillations tied to larger saccades are ambient dynamics that are predictively phase reset by non-retinal input associated with saccade generation (Rajkai et al., 2008; Barczak et al., 2019). Additionally, we observe that saccades are basically rhythmic or quasi-rhythmic, while that seems less clear with FeyeM. Similarly, in saccades the eyes move conjointly, while this may not be true for FeyeM. Finally, while we would concur on most of the assumptions made by the earlier formulations (Ahissar and Arieli, 2001, 2012), a few (e.g., “processing of the retinal signals evoked immediately after saccades likely involves significant suppression” and “in natural vision there are no strobe flashes”) may be untenable at this point. Regarding the former, it is clear that in natural viewing, V1 neuron excitability is significantly enhanced immediately at saccade offset (Barczak et al., 2019).
There is potential complementarity between saccades and FeyeM that is intriguing. On the one hand, saccades and FeyeM could engage in a basic alternation during natural viewing, with prolonged fixation periods interrupting more rhythmic and rapid saccadic search. On the other hand, with saccadic search operating at 3–4 Hz, and fixations lasting between 200 and 300 ms (e.g., Barczak et al., 2019), there is ample time for drifts (∼10 Hz) and tremors (∼40 Hz) to operate between saccades. This being the case, small rapid FeyeM would be nested within larger slower saccades, much as higher frequency amplitude nests within lower frequency phase in the resting EEG (Lakatos et al., 2005). The proposition that drift-type FeyeM have a tendency to occur at 10 Hz (Ahissar and Arieli, 2012; Herrmann et al., 2017), is also intriguing in light of the finding that in free viewing, phase reset at the rate of saccades is accompanied by phase reset in the higher alpha (8–12 Hz) range. This fits with the idea that some drift FeyeMs are coupled to saccades. Moreover, high amplitude alpha range activity occurs between saccades (Barczak et al., 2019) and alpha oscillations modulate V1 neuronal excitability so strongly that neurons fluctuate between high firing and near zero firing in response to visual input (Haegens et al., 2015). This could in effect produce a “strobe flash-like” modulation that would reduce retinal smearing by FeyeM, as proposed earlier (Ahissar and Arieli, 2012).
Like exploratory saccades, microsaccades have been observed to predictively modulate neural activity and behavior (Rucci et al., 2007; Bosman et al., 2009; Ko et al., 2010; Chen et al., 2015; Bellet et al., 2017; Loughnane et al., 2018; Lowet et al., 2018a, b). For example, using recordings from superior colliculus and frontal eye fields in non-human primates, Chen et al. (2015) observed enhanced neural firing for peripheral stimuli that preceded microsaccades and suppression of neural firing for stimuli presented immediately after microsaccades. This was further supported by research showing that visual behavior in a discrimination task depends on the presence of microsaccade periods with behavioral costs and benefits oscillating relatively to movement onsets (Bellet et al., 2017). These findings suggest that microsaccades are associated with a motor preparatory signal that impacts sensory processing and behavior. Thus, in most respects microsaccades are more like saccades than FeyeM.
As all eye movements in primates involve relocation of the fovea, it is possible that a moving fovea is critical for sensorimotor coupling in vision. Alternatively, sensorimotor coupling in animals with no fovea might rely on other sources of predictive modulation in visual encoding. Is therefore vision of animals with no fovea (e.g., a mouse) less dependent on signals from the motor system? A mouse vision operates at a very low resolution (Prusky and Douglas, 2004) with its entire retina being similar to peripheral retina in primates (Huberman and Niell, 2011). Nevertheless, activity in its primary visual cortex depends on feedback signals from motor cortex and is sensitive to locomotion and mismatch between actual visual information and locomotion-based predictions (Niell and Stryker, 2010; Keller et al., 2012; Saleem et al., 2013; Attinger et al., 2017; Leinweber et al., 2017; for review see Keller and Mrsic-Flogel, 2018; Pakan et al., 2018). Neural oscillations in gamma range and neural firing in mouse V1 are both increased during periods of locomotion as compared to equivalent visual stimulation during stationary intervals (Niell and Stryker, 2010). Importantly, the effect of active (i.e., locomotion-dependent) stimulation has not been observed in the visual thalamus, suggesting it might reflect cortical feedback to V1 rather than changes in peripheral sensory stimulation.
Saccades, albeit demonstrably central to visual information processing in primates, are part of an overall repertoire that includes distinctly different forms of oculomotor behavior like microsaccades, FeyeM, as mentioned above, as well as smooth pursuit (e.g., Lisberger et al., 1987), which were not discussed here. A more detailed review of these movements is beyond the scope of this discussion. It is, however, clear that in all of these cases, sensing depends causally on motor activity (i.e., eye/vibrissal/hand/respiratory movements) sharing the very same assumption that motor output provides information which is fed back to sensory systems where it modulates local neural activity. A complete understanding of Active Sensing needs to include the whole range of eye movements as well as other forms of sensorimotor coupling.
Translational Implications of Active Sensing
Despite theoretical consequences of the acknowledgment that most of sensory input is preceded and followed by a movement, Active Sensing has also translational implications. Autism, schizophrenia, ADHD are only few examples where entrainment to visual or auditory stimuli is impaired (for review see Calderone et al., 2014; Simon and Wallace, 2016). Because Active Sensing involves entrainment of a large and distributed neural network including multiple cortical and subcortical areas which participate in highly specialized functions (like preparing and executing eye movement, integrating information across saccades, etc.). Neurological impairment to any particular part of this network should manifest itself in highly specific disruptions to the dynamics of active exploration and therefore would be diagnostically useful (Thakkar et al., 2017). For example, 2–6-months-old infants later diagnosed with autism spectrum disorders fixate less on the eyes while viewing natural images of faces (Jones and Klin, 2013). Similarly, the temporal pattern of fixations is altered in children with dyslexia even before they develop any reading disability (Benfatto et al., 2016). The visual search pattern is impaired in patients suffering from Alzheimer disease (Molitor et al., 2015) and mild cognitive impairment (Lagun et al., 2011). The Parkinson patients tend to fixate longer while visually exploring complex images. Importantly, the magnitude of this effect scales with disease progression (Archibald et al., 2013). All these findings suggest that active sampling behavior is both selectively and specifically impaired in neurodevelopmental disorders. This does not assume that these disorders are caused by an intrinsic deficit in visual or oculomotor system but rather that they manifest its early onsets in these systems and alter visual sampling behavior.
The studies described above suggest that the active visual exploration elicits neural dynamics in low frequencies (i.e., theta, alpha) similar to multiple cognitive processes like attention (Schroeder and Lakatos, 2009; Saalmann et al., 2012), memory encoding and retrieval (Sederberg et al., 2003; Osipova et al., 2006; Lega et al., 2012), working memory (Jensen and Tesche, 2002; Jokisch and Jensen, 2007; Sauseng et al., 2010; Leszczyński et al., 2015, 2017) as well as cognitive control (for review see Cavanagh and Frank, 2014). Because stimulus complexity, task and subjects’ goal all strongly modulate viewing parameters (Yarbus, 1967; see Voss et al., 2017 for recent review), it remains a possibility that changes in low-frequency neural dynamics currently attributed to particular cognitive functions are confounded with changes in spatio-temporal patterns of active visual exploration. Therefore, future studies rather than attempting to minimize the amount of eye movements should include them as a source of relevant information and study neural dynamics in the context of active sensing strategies adopted by participants. For example, it remains an open question how memory encoding and retrieval depends on individual sampling strategy and how each of these influence neural oscillations (but see Jutras and Buffalo, 2010; Jutras et al., 2013; Staudigl et al., 2017, 2018; see also Voss et al., 2017). In fact, recognition of the critical role of the motor sampling routine in natural active vision raises a host of additional questions such as: (1) How much information is encoded in an individual fixation and how is information accumulated across saccades? (2) Does the phase perturbation observed in a distributed network including early sensory areas as well as the medial temporal lobe and the hippocampus play a direct role in that process? (3) Is the frequency, and duration of phase perturbation constant or does it change from lower to higher visual areas, to medial temporal lobe? (4) What is the role of these distinct nodes during natural exploration; for example, does the low frequency phase reset observed in medial temporal lobe contribute to planning and preparation of the upcoming eye movement? (5) Given that the rate of eye movements changes depending on both external (e.g., scene properties) and internal (e.g., goals) factors, do the parameters of large-scale neural dynamics depend on these changes; for example, do longer fixations lead to phase concentration in lower frequencies which could in turn lead to longer excitability intervals for representing more detailed information encoded by larger ensembles? (6) Related to the last point, do FeyeM and saccades play complementary roles in vision, if not in the larger sense of analyzing “what” vs. “where” information (Ahissar and Arieli, 2001), at least in the sense that FeyeM expand the information gleaned by the saccadic search routines in which they are embedded.
Visual perception is often studied as “passive” process in which an observer fixates steadily at a point in space and information is delivered to the system at times arbitrary defined by an experimenter. In natural “active” vision, however, information is systematically acquired by a motor-sampling plan, programming a sequence of alternating fixations and saccades. This sampling behavior imposes a robust temporal patterning on information flow in the visual pathways. While it is widely acknowledged that visual activity throughout the cortical processing hierarchy depends on the interaction between stimulus qualities and properties of neurons in each cortical area, accumulating evidence suggests that during natural free viewing low frequency neuronal oscillations yoked to eye movements predictively modulate neuronal excitability in order to both amplify the neural representation of visual inputs and enhance their transmission through the visual pathways. This “saccadic” version of the active sensing hypothesis builds on and complements earlier formulations based on patterns of whisking and sniffing in rodents and on smaller, more rapid “fixational” eye movements in primates. The active sensing hypothesis acknowledges that the dynamics of eye movements highly depends on task and stimulus properties and that the pattern and rate of saccades will differ across memory loads, stimuli, and overall task demands. The hypothesis also acknowledges that condition-dependent changes to neural dynamics as typically observed in cognition tasks performed during fixation might in fact parallel those associated with overt sampling behavior (e.g., saccade rate, fixation duration, etc.). In any case, the bulk of the evidence we have reviewed points to the conclusion that visual processing in the context of unconstrained eye movements is fundamentally different from that observed in subjects constrained to fixate steadily during visual stimulation.
Both authors contributed to the writing and editing of this manuscript.
The authors were supported by P50 MH109429 grant.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Barczak, A., Haegens, S., Ross, D. A., McGinnis, T., Lakatos, P., and Schroeder, C. E. (2019). Dynamic modulation of cortical excitability during visual active sensing. Cell Rep. 27, 3447–3459. doi: 10.1016/j.celrep.2019.05.072
Bartlett, A. M., Ovaysikia, S., Logothetis, N. K., and Hoffman, K. L. (2011). Saccades during object viewing modulate oscillatory phase in the superior temporal sulcus. J. Neurosci. 31, 18423–18432. doi: 10.1523/JNEUROSCI.4102-11.2011
Bellet, J., Chen, C. Y., and Hafed, Z. M. (2017). Sequential hemifield gating of α-and β-behavioral performance oscillations after microsaccades. J. Neurophysiol. 118, 2789–2805. doi: 10.1152/jn.00253.2017
Benfatto, M. N., Seimyr, G. Ö., Ygge, J., Pansell, T., Rydberg, A., and Jacobson, C. (2016). Screening for dyslexia using eye tracking during reading. PLoS One 11:e0165508. doi: 10.1371/journal.pone.0165508
Bosman, C. A., Womelsdorf, T., Desimone, R., and Fries, P. (2009). A microsaccadic rhythm modulates gamma-band synchronization and behavior. J. Neurosci. 29, 9471–9480. doi: 10.1523/JNEUROSCI.1193-09.2009
Calderone, D. J., Lakatos, P., Butler, P. D., and Castellanos, F. X. (2014). Entrainment of neural oscillations as a modifiable substrate of attention. Trends Cogn. Sci. 18, 300–309. doi: 10.1016/j.tics.2014.02.005
Canolty, R. T., Edwards, E., Dalal, S. S., Soltani, M., Nagarajan, S. S., Kirsch, H. E., et al. (2006). High gamma power is phase-locked to theta oscillations in human neocortex. Science 313, 1626–1628. doi: 10.1126/science.1128115
Corbetta, M. (1998). Frontoparietal cortical networks for directing attention and the eye to visual locations: identical, independent, or overlapping neural systems. Proc. Natl. Acad. Sci. U.S.A. 95, 831–838. doi: 10.1073/pnas.95.3.831
Fiebelkorn, I. C., Pinsk, M. A., and Kastner, S. (2018). A dynamic interplay within the frontoparietal network underlies rhythmic spatial attention. Neuron 99, 842–853. doi: 10.1016/j.neuron.2018.07.038
Fiebelkorn, I. C., Saalmann, Y. B., and Kastner, S. (2013). Rhythmic sampling within and between objects despite sustained attention at a cued location. Curr. Biol. 23, 2553–2558. doi: 10.1016/j.cub.2013.10.063
Haegens, S., Barczak, A., Musacchia, G., Lipton, M. L., Mehta, A. D., Lakatos, P., et al. (2015). Laminar profile and physiology of the α rhythm in primary visual, auditory, and somatosensory regions of neocortex. J. Neurosci. 35, 14341–14352. doi: 10.1523/JNEUROSCI.0600-15.2015
Hamamé, C. M., Vidal, J. R., Perrone-Bertolotti, M., Ossandón, T., Jerbi, K., Kahane, P., et al. (2014). Functional selectivity in the human occipitotemporal cortex during natural vision: evidence from combined intracranial EEG and eye-tracking. NeuroImage 95, 276–286. doi: 10.1016/j.neuroimage.2014.03.025
Helfrich, R. F., Fiebelkorn, I. C., Szczepanski, S. M., Lin, J. J., Parvizi, J., Knight, R. T., et al. (2018). Neural mechanisms of sustained attention are rhythmic. Neuron 99, 854–865. doi: 10.1016/j.neuron.2018.07.032
Hoffman, K. L., Dragan, M. C., Leonard, T. K., Micheli, C., Montefusco-Siegmund, R., and Valiante, T. A. (2013). Saccades during visual exploration align hippocampal 3–8 Hz rhythms in human and non-human primates. Front. Syst. Neurosci. 7:43. doi: 10.3389/fnsys.2013.00043
Ito, J., Maldonado, P., Singer, W., and Grün, S. (2011). Saccade-related modulations of neuronal excitability support synchrony of visually elicited spikes. Cereb. Cortex 21, 2482–2497. doi: 10.1093/cercor/bhr020
Jokisch, D., and Jensen, O. (2007). Modulation of gamma and alpha activity during a working memory task engaging the dorsal or ventral stream. J. Neurosci. 27, 3244–3251. doi: 10.1523/jneurosci.5399-06.2007
Jutras, M. J., Fries, P., and Buffalo, E. A. (2013). Oscillatory activity in the monkey hippocampus during visual exploration and memory formation. Proc. Natl. Acad. Sci. U.S.A. 110, 13144–13149. doi: 10.1073/pnas.1302351110
Katz, C., Patel, K., Talakoub, O., Groppe, D., Hoffman, K., and Valiante, T. A. (2018). Differential generation of saccade, fixation and image onset event-related potentials in the human mesial temporal lobe. BioRxivGoogle Scholar
Kayser, C., Montemurro, M. A., Logothetis, N. K., and Panzeri, S. (2009). Spike-phase coding boosts and stabilizes information carried by spatial and temporal spike patterns. Neuron 61, 597–608. doi: 10.1016/j.neuron.2009.01.008
Lagun, D., Manzanares, C., Zola, S. M., Buffalo, E. A., and Agichtein, E. (2011). Detecting cognitive impairment by eye movement analysis using automatic classification algorithms. J. Neurosci. Methods 201, 196–203. doi: 10.1016/j.jneumeth.2011.06.027
Lakatos, P., Karmos, G., Mehta, A. D., Ulbert, I., and Schroeder, C. E. (2008). Entrainment of neuronal oscillations as a mechanism of attentional selection. Science 320, 110–113. doi: 10.1126/science.1154735
Lakatos, P., Shah, A. S., Knuth, K. H., Ulbert, I., Karmos, G., and Schroeder, C. E. (2005). An oscillatory hierarchy controlling neuronal excitability and stimulus processing in the auditory cortex. J. Neurophysiol. 94, 1904–1911. doi: 10.1152/jn.00263.2005
Leinweber, M., Ward, D. R., Sobczak, J. M., Attinger, A., and Keller, G. B. (2017). A sensorimotor circuit in mouse cortex for visual flow predictions. Neuron 95, 1420.E5–1432.E5. doi: 10.1016/j.neuron.2017.08.036
Leszczynski, M., Barczak, A., Kajikawa, Y., Ulbert, I., Falchier, A., and Tal, I. et al. (2019). Dissociation of broadband high-frequency activity and neuronal firing in the neocortex. bioRxiv [Preprint]. doi: 10.1101/531368
Lisberger, S. G., Morris, E. J., and Tychsen, L. (1987). Visual motion processing and sensory-motor integration for smooth pursuit eye movements. Ann. Rev. Neurosci. 10, 97–129. doi: 10.1146/annurev.neuro.10.1.97
Loughnane, G. M., Newman, D. P., Tamang, S., Kelly, S. P., and O’Connell, R. G. (2018). Antagonistic interactions between microsaccades and evidence accumulation processes during decision formation. J. Neurosci. 38, 2163–2176. doi: 10.1523/JNEUROSCI.2340-17.2018
Lowet, E., Gips, B., Roberts, M. J., De Weerd, P., Jensen, O., and van der Eerden, J. (2018b). Microsaccade-rhythmic modulation of neural synchronization and coding within and across cortical areas V1 and V2. PLoS Biol. 16:e2004132. doi: 10.1371/journal.pbio.2004132
Lowet, E., Gomes, B., Srinivasan, K., Zhou, H., Schafer, R. J., and Desimone, R. (2018a). Enhanced neural processing by covert attention only during microsaccades directed toward the attended stimulus. Neuron 99, 207–214. doi: 10.1016/j.neuron.2018.05.041
Martinez-Conde, S., Otero-Millan, J., and Macknik, S. L. (2013). The impact of microsaccades on vision: towards a unified theory of saccadic function. Nat. Rev. Neurosci. 14, 83–96. doi: 10.1038/nrn3405
Mathewson, K. E., Gratton, G., Fabiani, M., Beck, D. M., and Ro, T. (2009). To see or not to see: prestimulus α phase predicts visual awareness. J. Neurosci. 29, 2725–2732. doi: 10.1523/JNEUROSCI.3963-08.2009
Merchant, H., Grahn, J., Trainor, L., Rohrmeier, M., and Fitch, W. T. (2015). Finding the beat: a neural perspective across humans and non-human primates. Philos. Trans. R. Soc. Lond. B Biol. Sci. 370:20140093 doi: 10.1098/rstb.2014.0093
Mitzdorf, U. (1985). Current source-density method and application in cat cerebral cortex: Investigation of evoked potentials and EEG phenomena. Physiol. Rev. 65, 37–100. doi: 10.1152/physrev.19126.96.36.199
Morillon, B., Hackett, T. A., Kajikawa, Y., and Schroeder, C. E. (2015). Predictive motor control of sensory dynamics in auditory active sensing. Curr. Opin. Neurobiol. 31, 230–238. doi: 10.1016/j.conb.2014.12.005
Osipova, D., Takashima, A., Oostenveld, R., Fernández, G., Maris, E., and Jensen, O. (2006). Theta and gamma oscillations predict encoding and retrieval of declarative memory. J. Neurosci. 26, 7523–7531. doi: 10.1523/jneurosci.1948-06.2006
Pakan, J. M., Francioni, V., and Rochefort, N. L. (2018). Action and learning shape the activity of neuronal circuits in the visual cortex. Curr. Opin. Neurobiol. 52, 88–97. doi: 10.1016/j.conb.2018.04.020
Purpura, K. P., Kalik, S. F., and Schiff, N. D. (2003). Analysis of perisaccadic field potentials in the occipitotemporal pathway during active vision. J. Neurophysiol. 90, 3455–3478. doi: 10.1152/jn.00011.2003
Rajkai, C., Lakatos, P., Chen, C. M., Pincze, Z., Karmos, G., and Schroeder, C. E. (2008). Transient cortical excitation at the onset of visual fixation. Cereb. Cortex 18, 200–209. doi: 10.1093/cercor/bhm046
Ray, S., Crone, N. E., Niebur, E., Franaszczuk, P. J., and Hsiao, S. S. (2008). Neural correlates of high-gamma oscillations (60–200 Hz) in macaque local field potentials and their potential implications in electrocorticography. J. Neurosci. 28, 11526–11536. doi: 10.1523/jneurosci.2848-08.2008
Saalmann, Y. B., Pinsk, M. A., Wang, L., Li, X., and Kastner, S. (2012). The pulvinar regulates information transmission between cortical areas based on attention demands. Science 337, 753–756. doi: 10.1126/science.1223082
Saleh, M., Reimer, J., Penn, R., Ojakangas, C. L., and Hatsopoulos, N. G. (2010). Fast and slow oscillations in human primary motor cortex predict oncoming behaviorally relevant cues. Neuron 65, 461–471. doi: 10.1016/j.neuron.2010.02.001
Sauseng, P., Griesmayr, B., Freunberger, R., and Klimesch, W. (2010). Control mechanisms in working memory: a possible function of EEG theta oscillations. Neurosci. Biobehav. Rev. 34, 1015–1022. doi: 10.1016/j.neubiorev.2009.12.006
Schroeder, C. E., Steinschneider, M., Javitt, D. C., Tenke, C. E., Givre, S. J., Mehta, A. D., et al. (1995). Localization of ERP generators and identification of underlying neural processes. Electroencephalogr. Clin Neurophysiol. Suppl. 44, 55–75.
Schroeder, C. E., Wilson, D. A., Radman, T., Scharfman, H., and Lakatos, P. (2010). Dynamics of active sensing and perceptual selection. Curr. Opin. Neurobiol. 20, 172–176. doi: 10.1016/j.conb.2010.02.010
Sederberg, P. B., Kahana, M. J., Howard, M. W., Donner, E. J., and Madsen, J. R. (2003). Theta and gamma oscillations during encoding predict subsequent recall. J. Neurosci. 23, 10809–10814. doi: 10.1523/jneurosci.23-34-10809.2003
Song, K., Meng, M., Chen, L., Zhou, K., and Luo, H. (2014). Behavioral oscillations in attention: rhythmic α pulses mediated through θ band. J. Neurosci. 34, 4837–4844. doi: 10.1523/JNEUROSCI.4856-13.2014
Staudigl, T., Hartl, E., Noachtar, S., Doeller, C. F., and Jensen, O. (2017). Saccades are phase-locked to alpha oscillations in the occipital and medial temporal lobe during successful memory encoding. PLoS Biol. 15:e2003404. doi: 10.1371/journal.pbio.2003404
Staudigl, T., Leszczynski, M., Jacobs, J., Sheth, S. A., Schroeder, C. E., and Jensen, O. et al. (2018). Hexadirectional modulation of high-frequency electrophysiological activity in the human anterior medial temporal lobe maps visual space. Curr. Biol. 28, 3325–3329. doi: 10.1016/j.cub.2018.09.035
Sylvester, R., Haynes, J. D., and Rees, G. (2005). Saccades differentially modulate human LGN and V1 responses in the presence and absence of visual stimulation. Curr. Biol. 15, 37–41. doi: 10.1016/j.cub.2004.12.061
Keywords: active sensing, cognition, attention, oscillations, phase reset, eye movement, saccade, predictive coding
Citation: Leszczynski M and Schroeder CE (2019) The Role of Neuronal Oscillations in Visual Active Sensing. Front. Integr. Neurosci. 13:32. doi: 10.3389/fnint.2019.00032
Received: 09 January 2019; Accepted: 03 July 2019;
Published: 23 July 2019.
Edited by:Ehud Ahissar, Weizmann Institute of Science, Israel
Reviewed by:Rossella Breveglieri, University of Bologna, Italy
Yong Gu, Shanghai Institutes for Biological Sciences (CAS), China
Eldad Assa, Weizmann Institute of Science, Israel
Liron Gruber, Weizmann Institute of Science, Israel, in collaboration with reviewer EA
Copyright © 2019 Leszczynski and Schroeder. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.