Abstract
Working memory (WM) is central to human cognition as it allows information to be kept online over brief periods of time and facilitates its usage in cognitive operations (Luck and Vogel, 2013). How this information maintenance actually is implemented is still a matter of debate. Several independent theories of WM, derived, respectively, from behavioral studies and neural considerations, advance the idea that items in WM decay over time and must be periodically reactivated. In this proposal, we show how recent data from intracranial EEG and attention research naturally leads to a simple model of such reactivation in the case of sensory memories. Specifically, in our model the amplitude of high-frequency activity (>50 Hz, in the gamma-band) underlies the representation of items in high-level visual areas. This activity decreases to noise-levels within 500 ms, unless it is reactivated. We propose that top-down attention, which targets multiple sensory items in a cyclical or rhythmic fashion at around 6–10 Hz, reactivates these decaying gamma-band representations. Therefore, working memory capacity is essentially the number of representations that can simultaneously be kept active by a rhythmically sampling attentional spotlight given the known decay rate. Since attention samples at 6–10 Hz, the predicted WM capacity is 3–5 items, in agreement with empirical findings.
Introduction
Working memory (WM) allows us to keep information online over brief periods of time and use it in cognitive operations (Luck and Vogel, 2013). To this day, it has proven difficult to reconcile theories of WM inspired by behavioral data (psychological theories) and those by brain measurements (neural theories), despite clear areas of convergence. We will briefly outline some of the main ideas and difficulties on both sides in the context of WM for sensory information and propose a possible interface between these approaches.
Psychological theories of WM
Despite several differences, most psychological theories of WM agree on the necessity for an active mechanism to prevent spontaneous decay of the representations of sensory items: it is thought that items that are not transferred to a more durable long-term store decay over time (e.g., Raaijmakers and Shiffrin, 1981). Preventing memory decay could occur by placing the items into a special store (e.g., Baddeley, 2003), or by holding them in the focus of attention and periodically reactivating them (e.g., Anderson, 1983; Cantor and Engle, 1993; Cowan, 1995).
The idea that attention is critical for reactivating memory traces has become a central point of discussion. Many reports (e.g., Awh et al., 1998; Downing, 2000; Awh and Jonides, 2001; Jha, 2002; Theeuwes et al., 2011 among others) indicate for instance that attention is preferentially deployed to remembered locations even during the maintenance period, when stimuli are no longer physically present. Similarly, disrupting attention during the maintenance period negatively impacts WM for remembered locations (Awh et al., 1998). However, others disagree with the idea that WM and attention share a common resource (e.g., Woodman et al., 2001; Johnson et al., 2008). Woodman et al. (2001) found that loading WM to its capacity did not affect visual search efficiency, a measure of attention, contradicting a prediction by the common resource hypothesis. However, there might be some methodological issues with these studies. For example, the Woodman et al. (2001) study used clustered stimuli to test their contention that WM and visual attention do not interact. In contrast, Anderson et al. (2013) in a similar dual WM-visual search paradigm, showed that no interference between WM and visual search occurs only if objects in the search task are grouped or clustered stimuli, perhaps because attention can search more efficiently in such situations. In fact, Johnson et al. (2008) themselves found that increasing WM load increases reaction times when participants search for an object that is a combination of two features (i.e., when binding is required) but not when such binding is not required. This suggests that WM load interferes with a task that requires attention such as binding (Treisman and Gelade, 1980). Further, Anderson et al. (2013) tested various kinds of stimuli and showed that visual search efficiency is closely related to WM capacity, indicating that they share a common resource. Other studies have also shown mutual interference between visual search and WM tasks (Oh and Kim, 2003, 2004; Woodman and Luck, 2004).
Based on such findings and their own, Barrouillet et al. (2004), have taken the idea of attentional involvement in WM maintenance one step further. They posit a periodic, rather than continuous, process of reactivation by attention. This is based on four tenets and observations they make: (1) WM processing and maintenance requires the use of a limited attentional resource; (2) when attention switches away, items are subject to decay; (3) attention-demanding cognitive processes therefore prevent refreshing of memory traces; and (4) there is only one attentional process at a time, and thus there should be rapid switching between processing and maintenance aspects of a task (Barrouillet et al., 2007). Taken together, it can be argued that items in WM decay with time, and a resource-limited process, attention, rapidly shifts between these decaying representations in order to reactivate them. In this paper, we expand on this explanation, grounding it in neurophysiological evidence that places constraints on such a theory.
Neuroscience of WM
There is also considerable debate about what neural system underlies WM. Baddeley has associated WM with right fronto-parietal areas for visual memories and with left fronto-parietal areas for verbal memories (Baddeley, 2003). The idea that the frontal lobe, and in particular the prefrontal cortex (PFC), is involved in maintenance has been substantiated by recordings made by Goldman-Rakic and others (Goldman-Rakic, 1995), who observed that some PFC neurons have increased and sustained activity over the course of an interval in which a stimulus had to be maintained. Even recently, Salazar et al. (2012) found clear evidence for item-specific increases in fronto-parietal 14–30 Hz beta oscillatory coherence during WM maintenance. Several other studies have shown that lower-frequency oscillatory synchronization between prefrontal and temporal lobes (Von Stein et al., 1999; Serrien et al., 2004) and between prefrontal and parietal lobes (Lutzenberger et al., 2002; Babiloni et al., 2004; Kopp et al., 2006; Payne and Kounios, 2009) are associated with the maintenance of information during WM tasks. Some of these effects are also dependent on the number of items that participants have to maintain (see Fell and Axmacher, 2011, for a discussion). In addition to involvement of individual oscillatory bands in WM maintenance, phase-amplitude coupling in PFC between low and high frequency oscillatory components has also been observed in the context of WM tasks (Siegel et al., 2009), and in humans one study has further shown that the magnitude of this phase-amplitude coupling is associated with a person’s WM capacity (Sauseng et al., 2009).
Although there is a large body of evidence implicating the PFC in WM maintenance, there is also ample evidence that other regions, including the hippocampus (Axmacher et al., 2007; van Vugt et al., 2010), are involved, especially for visually complex items. However, studies emphasizing the hippocampus typically do not consider the role of attention in WM, which as discussed above, is crucial. If we assume the PFC is involved in (sensory) WM, one central issue is whether PFC neurons actually contain a memory trace of the maintained items, as early studies showing continuous, item-specific activations might have suggested (Goldman-Rakic, 1995). This view is problematic for several reasons: (a) it implies a strong redundancy between PFC and sensory areas, since every item which can be perceived and identified in sensory cortices should once again be represented by a neural group in the PFC, in a one-to-one fashion, during memory maintenance. This duplication seems “metabolically costly” (Sreenivasan et al., 2014); (b) PFC neurons do not exhibit the same degree of selectivity as sensory neurons, which suggests that potential confusions between different, but closely resembling, sensory items could occur in the PFC if memory traces were stored there; that is, if two items which can be differentiated in sensory cortices were maintained by the same pattern of neural activation in the PFC, how could they be differentiated in the recall phase? (e.g., Miyashita and Chang, 1988); and (c) the evidence that PFC supports the representation of items in WM are derived from animal experiments with extensive training and exposure to the same sensory stimuli, which is very unlike the situations in which humans usually use WM. Consequently, it cannot be excluded that specific PFC neural ensembles might specialize over the training phase to store each element of the limited set of items used in the experiment. Obviously, such one-to-one mapping could not occur to store novel and unique sensory items seen only once in real-life situations.
Yet, it is clear that both PFC and sensory areas participate in maintenance of items in sensory WM, but what are their respective contributions? To avoid any loss of specificity and selectivity during the maintenance period, the actual representation of sensory items must necessarily be stored where it was generated in the first place during stimulus presentation: in sensory areas themselves. If so, it immediately raises two (possibly related) questions: (a) how are the representations maintained over time in sensory areas; and (b) what is the role of the PFC? In the following, we propose a model that directly addresses these two questions.
The “hammer and bells” model
Any model of sensory WM must include a mechanism of reactivation to avoid neural representations of memory items to decay to noise level. Such mechanism can either be local (self-maintenance in sensory areas), or global, via long-range projections from distant cortical areas. However, both possibilities have their problems.
Problems with local reactivation
Some authors have proposed that reactivation occurs locally, for instance with the aid of recurrent interactions in the local neural architecture (Lisman and Idiart, 1995; Luck and Vogel, 2013). However, local reactivation by itself does not explain why some items are stored in WM while others are not, usually depending on current behavioral goals and/or task instructions (e.g., Sperling, 1960). Rather, we propose that the special status of remembered items is largely decided by remote cortical areas supporting memory of current behavioral goals (i.e., task sets), namely in the PFC (Sakai and Passingham, 2003), in agreement with Sreenivasan et al. (2014).
Problems with a long-range reactivation
If reactivation of neural assemblies in sensory cortex is directed by distant cortical regions, say the PFC, then PFC neurons that are responsible for the reactivation of the to-be-maintained sensory items must unambiguously identify and connect to the exact set of sensory neurons that support those items, with perfect selectivity. This constraint raises, again, the problem of a possible redundancy between PFC and sensory cortices, wherein each possible sensory item is tagged by a specific neural group in the PFC (see above). Then, the PFC could, by itself, be the substrate of the memory trace, and thus duplicate sensory areas with all associated contradictions (Postle, 2006; Lee et al., 2013). One route of escape from that dead-end, however, is that the neural representations of remembered items in sensory areas have a distinct feature: they are presumably more active than other nearby, sensory neurons, at least before full decay, Therefore, a full redundancy between PFC and sensory areas is only needed to reactivate a memory trace if it has decayed to noise level; until then, PFC neurons can selectively dissociate neurons which participate in the memory trace based on their higher activity. In short, residual activity in the sensory areas could be read by PFC neurons as the signature of to-be-reactivated neurons. A straightforward mechanism, therefore, for WM maintenance, would be to simply reactivate all sensory neurons that are currently active (Figure 1).
Figure 1
A bell analogy
Given the need for periodic reactivation (from both the neuroscientific and psychological theory perspectives), we propose a different mechanism directly inspired by a practical, physical problem (Figure 2). Consider a set of bells. If one decided to hit a subset of them with a hammer, and keep doing so, so that the sound of each bell of that subset never dies, how should one proceed? An obvious solution would be to remember which bells must be hit, and hit them in a sequence, repeatedly. But that would amount to having a separate, independent memory trace of which bells must be hit. If we want to think of the bells as the primary memory traces in the sensory areas, this possibility amounts to the one we discussed above and would raise the obvious objection: why have two memory traces (one, a set of vibrating bells, and the other the memory in the bell-player), why not just the second, “prefrontal” one? To solve that problem without an additional memory trace, we need to find a way to identify which bells must be hit on the sole basis of what distinguishes them from other bells, namely the fact that they are still ringing. The only solution, then, is to hit all the bells that are still ringing, as they would become undistinguishable from the rest once silent. There is no need to remember which specific bell was active. All that is needed is a system that monitors if a bell is still vibrating, and if so, hits it. This means that the number of bells that can be kept ringing simultaneously depends on two parameters: the time it takes to shift the hammer from one bell to another, and the time it takes for the sound to decay, in each bell. We can then predict the maximum number of bells that one can actively maintain. In our model, the bells would be in the sensory areas and PFC neurons would hold the hammer. Our analogy predicts that we can predict the number of sensory items that can be maintained in WM from the time it takes for their neural representation to decay to noise level and from the time necessary for PFC neurons to reactivate one trace after another.
Figure 2
Role for attention
If there is indeed such periodic reactivation, what neural mechanism might reactivate these fading representations? The candidate mechanism should have at least two properties: (a) it should be selective, to enhance active neural representations in sensory cortex while not activating representations of concurrent items; and (b) it should take into account current behavioral goals. These two properties almost define attention, as it is currently understood. In line with that remark, attention is known to be intimately involved in the maintenance of objects in WM (Awh and Jonides, 2001; Jha, 2002; D’Esposito, 2007). Several researchers (Cowan, 1988, 1995; Barrouillet et al., 2004; RepovŠ and Baddeley, 2006; but see Fougnie, 2008) have even posited a cognitive model where spatial WM is merely the information currently present within the focus of attention. When selected, attention enhances activation of that representation (Fries et al., 2002). Attention is also intimately involved in modulating activity based on behavioral goals (e.g., Connolly et al., 2002; Corbetta et al., 2008; Leber, 2010; Chun et al., 2011). Moreover, as desired for periodic reactivation, recent evidence suggests that attention acts in a rhythmic manner (VanRullen et al., 2007; Busch and VanRullen, 2010; Landau and Fries, 2012), sampling the input stream at around 6–10 Hz. Taking all these considerations together, attention could serve as a mechanism by which fading representations can be reactivated.
Maintaining several items simultaneously
This reactivation model based on the bell analogy also solves another problem, that is, the mechanism by which several items can be simultaneously maintained. If there is WM in sensory areas, then apart from the question of reactivating these decaying WM representations, another important question is: how do neural processes maintain multiple items simultaneously in WM? Suppose that in sensory areas each item i has activated, when presented, a neural population Pi. Then the question is how can the neural activity of all populations Pi be maintained during the delay period? One easy solution would be for attention to sustain that activity simultaneously in all Pi, which implies that a PFC-mediated attentional modulation could target all Pi simultaneously. However, there is a long-standing debate about whether attention can select multiple objects simultaneously or whether attention has a unitary focus and switches between objects (LaBerge and Brown, 1989; Posner and Petersen, 1990; Castiello and Umiltà, 1992; Bichot et al., 1999; Müller et al., 2003; Howe et al., 2010). Recent evidence suggests that even when attention appears to be selecting several items simultaneously–the so called “divided attention” model–its is in fact selecting one object at a time and rapidly switching between different objects (VanRullen et al., 2007; Hogendoorn et al., 2010). Based on these findings, we suggest that decaying items are sequentially reactivated by a unitary attentional spotlight.
Given the evidence that items are selected sequentially, what consequences does that have for maintenance in WM? One fallout is that attention would also be processing and hence reactivating items in WM sequentially. Another is that, since attention samples sequentially, all items are loaded into WM sequentially, even if multiple to-be-remembered items are presented simultaneously. This might, to some extent, prevent the representations of multiple active items being “confused” with each other. A neural population that is most active at a given time represents one item. The items are separable because they are all not simultaneously active to the same extent. If multiple representations were simultaneously equally active, then the brain would have no way to distinguish whether those were a single object or two separate ones, since both are likely to be represented in overlapping neural regions. This predicts that high-contrast objects that take longer to decay could potentially be confused with low-contrast objects represented by the same neural population that are presented at a later time (Singer and Gray, 1995).
Various methods to measure the duration of attentional shifts between objects have come up with a number of about 100–300 ms (4–10 Hz). Chakravarthi and VanRullen (2011b) showed that this “shifting” of attention occurs even when a single object is attended–attention samples the object at around 7 Hz (also see VanRullen et al., 2005). That is, attention selects an object and disengages from it periodically, even when sustained attention is required for the task. Recent work has further supported this hypothesis (Landau and Fries, 2012; Song et al., 2014). In summary, this single mechanism of periodically sampling attention could reactivate both single and multiple items to be maintained in sensory WM.
Periodic attention directed by PFC in combination with reactivating decaying memories in sensory areas could therefore explain WM maintenance and impose a limit on the number of items that can be maintained in WM. If we assume that memories are represented by gamma-band synchronized assemblies in sensory areas (Luck and Vogel, 2013) then we should be able to infer from gamma-band recordings the maximal waiting time between two reactivations of the same neural populations. In addition to this, if we have an estimate of the time required by attention to reactivate separate neural populations (that is, the attentional “switch time”), we can compute the maximum number of items that can be stored in WM. Here we will present iEEG (intracranial electroencephalography) data that address this issue.
Empirical data
Intracranial recordings reveal a spontaneous decay of gamma-band neural activity after a presented stimulus is removed. Figure 3A shows the temporal profile of High Frequency Activity (HFA) responses to visual stimuli in six participants in several visual areas during a simple visual oddball task in which participants watched stimuli from different categories (see Vidal et al. (2010) for a detailed description of the protocol). The total duration of HFA, the “response duration”, was calculated as the duration between the peak onset and the last 50-ms window in which amplitude departed significantly (Bonferroni-corrected, p < 0.05) from baseline. The matrix (left) shows the HFA response as a function of time for each site (between blue lines) and for each stimulus category (within blue lines). Each row within a given area represents activity in response to a different stimulus. Amplitude is expressed as a percentage of the response peak. All time points that do not significantly differ from the pre-stimulus baseline have been set to 0. Sites range from early (bottom) to high-level (top) visual areas. This shows that the timing of decay is remarkably consistent across these areas; response duration is, on average, 450 ms (SD 80 ms; range 300–600 ms).
Figure 3
We also observe in these iEEG data a mechanism that prevents decay: attention. Figure 3B shows the HFA response in the fusiform gyrus to attended vs. ignored words during an attentive reading task. The main effect of attention is to prolong response duration even if the visual stimulus triggering that activity is no longer present. Although a local origin of this attentional effect is possible in principle (local recurrent interactions acting to sustain neural activity), it is likely to be triggered directly or indirectly by executive brain areas, presumably in the prefrontal cortex, where task instructions are known to be represented (task-set neural ensembles in the lateral prefrontal cortex, PFC). This is consistent with the findings that top-down influences, such as mental imagery (Hamamé et al., 2012) and attention (Jung et al., 2008), have been shown to sustain or even generate HFA de novo. These findings suggest that attention could reactivate HFA, and thus refresh items in WM.
While Figure 3 shows only one bell strike, Figure 4 shows an example of possible successive bell strikes in the absence of a novel sensory input. HFA was recorded along the dorsal visual stream of an epileptic patient (using iEEG), while she had to maintain in memory a set of six spatial positions (6 dots on a 4 × 4 grid; see panel a). On top of a slower modulation, the trace displays a succession of high-amplitude peaks rhythmically modulated with a period around 150 ms (about 7 Hz). Although we don’t claim that this is a direct proof of our hypothesis, it shows nonetheless that a rhythmic modulation of HFA in the theta/alpha range exists in sensory areas selective to the kind of information stored in that region (spatial positions in the parietal visual pathway). Such modulation might support the periodic reactivation of memory traces. It remains to be determined whether the modulation of gamma peaks corresponds to a top-down modulation by the PFC or potentially other regions such as the hippocampus (Axmacher et al., 2007), and whether it targets specifically the neural assemblies storing items in memory.
Figure 4
The above model implies that WM capacity can be inferred from attention’s discrete sampling rate together with the decay rate of HFA, both of which are measurable. WM capacity is the number of neural assemblies attention can shift to and reactivate before they fully decay. With a sampling rate of 6–10 Hz, 3–5 items can be sequentially reactivated within 450 ms (the duration of the decay), in line with previous findings on WM capacity (Vogel et al., 2005). This model of WM accounts for the asynchronous firing of neural assemblies, the limited WM capacity, and the involvement of attention and PFC in WM, in addition to the explanatory power of previous proposed mechanisms (Luck and Vogel, 2013). Further, it provides clear and testable predictions: (1) participants who can shift their attention more rapidly should have a higher WM capacity. Indeed, this has been shown by Anderson et al. (2013), who found that individuals with higher WM capacity had more efficient search slopes in a visual search task; and (2) the decay rate of HFA in a particular sensory area (say, the Fusiform Face area) should also predict WM capacity of items maintained in that brain region (say, faces) for that individual.
Putting it all together
We have suggested a neural implementation of a periodic reactivation-based theory of WM. The prerequisite neural phenomena of such a model are the following:
precise memory traces of sensory stimuli are stored in cortical regions which can differentiate between nearly identical stimuli (i.e., at the level of different neural representations, to avoid confusion at the time of retrieval)—we argue that this is why traces must be stored in sensory areas.
once a stimulus is no longer physically present, the activity of its neural representation decays progressively with time.
once the activity of a neural representation has decayed down to baseline level, we claim that it can no longer be reactivated—a neural representation is active as long as gamma-band activity is above baseline level in its corresponding neural assembly.
reactivation of decaying memory traces originates from brain regions supporting the instruction to keep those traces active—we propose the dorsal lateral PFC as a likely candidate.
the reactivation mechanism is able to bring back gamma-band activity of an active neural assembly significantly above baseline level.
the reactivation mechanism cannot reactivate distinct neural assemblies simultaneously: there is a minimum shift-time between two assemblies.
This neural implementation of WM is conceptually similar to striking a bell with a hammer. In this analogy, the decaying neural representation of a stimulus, in terms of in gamma-band activity, is periodically reactivated by rhythmic attention. If a bell/stimulus decays within 500 ms (see Figure 3) and the hammer/attention takes 100–200 ms to go from one bell/stimulus to the other, 3–5 bells/stimuli can be kept active at the same time. Our account can thus explain a limited WM capacity through the periodicity of attention. It has also closely related to several existing theories of WM.
Relation to other theories
Another neural model of WM that is similar to our proposed theory is the model developed by Lisman and Idiart (1995), Raffone and Wolters (2001) and Lisman and Jensen (2013). This model assumes that individual items are represented by different cell assemblies that are each active within a different gamma cycle (Fell and Axmacher, 2011). Since gamma cycles have been found to be nested in theta cycles (Lisman and Jensen, 2013) their hypothesis is that the to-be-remembered items are being sequentially reactivated during every theta cycle. The role of gamma is thus to allow alternating assemblies of cells to fire, while the role of theta is to organize these different assemblies sequentially. This theory differs from our proposal in that reactivation of successive assemblies takes about 100–300 ms in our proposal, much slower than the time scale in Lisman et al.’s model, where such reactivations can be estimated to occur every 20–40 ms. The theory by Lisman and Idiart has also been specifically linked to oscillatory coupling between the phase of a lower frequency and the amplitude of a higher frequency (Canolty et al., 2006; Canolty and Knight, 2010). However, our proposal does not make any specific claims about the involvement of theta oscillations or direct coupling between theta and gamma oscillations. It does not need to do so, since the low-frequency modulation refers to a modulation of gamma amplitude itself. Support for this choice comes from earlier studies that have provided some evidence for a lack of phase-amplitude coupling of the sort Lisman-Idiart proposed (Axmacher et al., 2010). Nevertheless, the theta-timescale for reactivation in their model is quite similar to the waxing of waning of attention in our model. The time constant of decay to be used for modeling their data should be empirically testable, by directly looking at decay (and reactivation) rates in successful WM storage of multiple items.
Luck and Vogel (2013), summarizing decades of research, concluded that WM has a “slot-like” nature, where up to four distinct items can be simultaneously stored. They also proposed that, based on Lisman et al.’s work, each item is represented by an active neural ensemble, which is periodically reactivated by local circuits, and hence sustained throughout the maintenance period. However, as we noted, this does not explain how task set can affect which items are stored in WM. Further, Sreenivasan et al. (2014) showed that the PFC is intimately involved in coordinating what items are represented in the sensory areas over time, although it does representing these items itself. This means that local reactivation circuits provide an incomplete picture of WM mechanisms, since they preclude any role for PFC. We combined these two different strands of analyses and suggested that PFC-mediated attention acts on representations in sensory areas and periodically reactivates them, as necessary. Thus, we explain both the role of PFC and the finding that task set can influence WM. Our model is also consistent with the proposal by Barrouillet et al. (2007) which posits that attention plays a crucial role in maintaining items in WM by periodically refreshing them, and thus preventing them from decaying completely. As detailed above, our account also leads to testable predictions about individual differences in WM capacity. Our theory could in the future be tested empirically by measuring the attentional sampling rate for an individual (Chakravarthi and VanRullen, 2011a) at the same time as decay of gamma activity in higher-level sensory areas corresponding to the type of information being maintained. A potentially even stronger test would be a disruption of the proposed refreshing mechanism—which we propose is subserved by interactions between PFC and higher-level sensory areas—for example by stimulation studies.
Assumptions about speed of visual attention
A possible objection against our proposal is that it depends on a speed of visual attention moving at 6–10 Hz, or 100–200 ms per item. However, the visual search literature potentially provides a very different estimate of the speed of visual attention. Visual search, where observers are asked to report the presence of a particular object among distracter objects, has been used as a measure of attentional processing for several decades (e.g., Treisman and Gelade, 1980; Wolfe, 1998; Eckstein, 2011; Nakayama and Martini, 2011). To successfully search for the target, attention has to process several items sufficiently such that it can discard the distracters and select only the target. These studies have provided estimates of how long attention has to process each item to perform the search task effectively. Such estimates of attentional processing time have ranged from 0–10 ms ms (“pop-out” search) to 100 s of ms (“serial” search). Despite the wide range of search “slopes”, in popular literature these results have been summarized as attention requiring about 25 ms per item. In other words this would suggest that attention processes about 40 items a second, which contradicts the proposal provided here. However, there are details of the visual search paradigm that resolve this apparent contradiction.
A search slope of 25 ms/item does not mean that attention samples each item for 25 ms and rapidly switching between items. It merely means that attention can process an object’s features to a sufficient extent within that period for it to make a decision (target/distracter) about the object. Further, it need not be the case that attention processes each object at the rate of 40 Hz. It could be that attention samples a few features of several objects at much slower periods. Say that attention selects and processes four sets of partial features (belonging to four different objects) every 100 ms. This would give a search slope of 25 ms/item, whereas in fact it has sampled and processed features every 100 ms (i.e., a sampling rate of 10 Hz). Indeed, visual search literature is replete with examples where minor changes among one of many target and distracters attributes dramatically changes search slopes, suggesting that attention does not process entire objects, but only the task relevant features of the objects. For example, distracter and target familiarity (Wang et al., 1994) or target-distracter similarity and homogeneity (D’Zmura, 1991) can considerably affect search slopes. In other words, search slopes are not good measures of how long attention processes a single object. Consequently, the 25 ms/item is probably a misleading number. Indeed, there is an ongoing debate about how “serial” visual search is. If search is not exactly serial, then the slopes cannot be used to estimate attentional processing speeds directly. To overcome this limitation in interpretation, several researchers have devised more sophisticated ways of measuring how long attention stays in one place and samples its contents (e.g., Horowitz et al., 2004, 2009; Carlson et al., 2007) and have consistently found that involuntary attention takes about 100–150 ms to switch locations, consistent with the current claims. In summary, we do not see a contradiction between the visual search literature and our model. Visual search indexes a different set of attentional processes and does not provide a direct estimate of attentional sampling times, as one might be tempted to infer.
Conclusion
We have suggested a neural implementation of a periodic reactivation theory of WM in sensory areas. We have shown preliminary evidence that parameters such as decay rates for the to-be-remembered items can be read off from gamma-band activity recorded in iEEG. Coupled with the findings that attention samples items at around 6–10 Hz, we provide physiological grounding to the well-established estimate of WM capacity of 3–5 items and some popular theories of WM.
Statements
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
1
AndersonJ. R. (1983). Retrieval of information from long-term memory. Science220, 25–30. 10.1126/science.6828877
2
AndersonD. E.VogelE. K.AwhE. (2013). Selection and storage of perceptual groups is constrained by a discrete resource in working memory. J. Exp. Psychol. Hum. Percept. Perform.39, 824–835. 10.1037/a0030094
3
AwhE.JonidesJ. (2001). Overlapping mechanisms of attention and spatial working memory. Trends Cogn. Sci.5, 119–126. 10.1016/s1364-6613(00)01593-x
4
AwhE.JonidesJ.Reuter-LorenzP. A. (1998). Rehearsal in spatial working memory. J. Exp. Psychol. Hum. Percept. Perform.24, 780–790. 10.1037/0096-1523.24.3.780
5
AxmacherN.HenselerM. M.JensenO.WeinreichI.ElgerC. E.FellJ. (2010). Cross-frequency coupling supports multi-item working memory in the human hippocampus. Proc. Natl. Acad. Sci. U S A107, 3228–3233. 10.1073/pnas.0911531107
6
AxmacherN.MormannF.FernándezG.CohenM. X.ElgerC. E.FellJ. (2007). Sustained neural activity patterns during working memory in the human medial temporal lobe. J. Neurosci.27, 7807–7816. 10.1523/jneurosci.0962-07.2007
7
BabiloniC.BabiloniF.CarducciF.CincottiF.VecchioF.ColaB.et al. (2004). Functional frontoparietal connectivity during short-term memory as revealed by high-resolution EEG coherence analysis. Behav. Neurosci.118, 687–697. 10.1037/0735-7044.118.4.687
8
BaddeleyA. (2003). Working memory: looking back and forward. Nat. Rev. Neurosci.4, 829–839. 10.1038/nrn1201
9
BarrouilletP.BernardinS.CamosV. (2004). Time constraints and resource sharing in adults’ working memory spans. J. Exp. Psychol. Gen.133, 83–100. 10.1037/0096-3445.133.1.83
10
BarrouilletP.BernardinS.PortratS.VergauweE.CamosV. (2007). Time and cognitive load in working memory. J. Exp. Psychol. Learn. Mem. Cogn.33, 570–585. 10.1037/0278-7393.33.3.570
11
BichotN. R.CaveK. R.PashlerH. (1999). Visual selection mediated by location: feature-based selection of noncontiguous locations. Percept. Psychophys.61, 403–423. 10.3758/bf03211962
12
BuschN. A.VanRullenR. (2010). Spontaneous EEG oscillations reveal periodic sampling of visual attention. Proc. Natl. Acad. Sci. U S A107, 16048–16053. 10.1073/pnas.1004801107
13
CanoltyR. T.EdwardsE.DalalS. S.SoltaniM.NagarajanS. S.KirschH. E.et al. (2006). High gamma power is phase-locked to theta oscillations in human neocortex. Science313, 1626–1628. 10.1126/science.1128115
14
CanoltyR. T.KnightR. T. (2010). The functional role of cross-frequency coupling. Trends Cogn. Sci.14, 506–515. 10.1016/j.tics.2010.09.001
15
CantorJ.EngleR. W. (1993). Working-memory capacity as long-term memory activation: an individual-differences approach. J. Exp. Psychol. Learn. Mem. Cogn.19, 1101–1114. 10.1037/0278-7393.19.5.1101
16
CarlsonT.VanRullenR.HogendoornH.VerstratenF.CavanaghP. (2007). Distinguishing models of multifocal attention: it’s a matter of time. J. Vis.7:641. 10.1167/7.9.641
17
CastielloU.UmiltàC. (1992). Splitting focal attention. J. Exp. Psychol. Hum. Percept. Perform.18, 837–848. 10.1037/0096-1523.18.3.837
18
ChakravarthiR.VanRullenR. (2011a). Attention is a state of mind: phase of ongoing EEG oscillations predicts the timing of attentional deployment. J. Vis.11:192. 10.1167/11.11.192
19
ChakravarthiR.VanRullenR. (2011b). Bullet trains and steam engines: exogenous attention zips but endogenous attention chugs along. J. Vis.11:12. 10.1167/11.4.12
20
ChunM. M.GolombJ. D.Turk-BrowneN. B. (2011). A taxonomy of external and internal attention. Annu. Rev. Psychol.62, 73–101. 10.1146/annurev.psych.093008.100427
21
ConnollyJ. D.GoodaleM. A.MenonR. S.MunozD. P. (2002). Human fMRI evidence for the neural correlates of preparatory set. Nat. Neurosci.5, 1345–1352. 10.1038/nn969
22
CorbettaM.PatelG.ShulmanG. L. (2008). The reorienting system of the human brain: from environment to theory of mind. Neuron58, 306–324. 10.1016/j.neuron.2008.04.017
23
CowanN. (1988). Evolving conceptions of memory storage, selective attention and their mutual constraints within the human information-processing system. Psychol. Bull.104, 163–191. 10.1037/0033-2909.104.2.163
24
CowanN. (1995). Attention and Memory: An Integrated Framework.New York: Oxford University Press.
25
D’EspositoM. (2007). From cognitive to neural models of working memory. Philos. Trans. R. Soc. Lond. B Biol. Sci.362, 761–772. 10.1098/rstb.2007.2086
26
D’ZmuraM. (1991). Color in visual search. Vision Res.31, 951–966. 10.1016/0042-6989(91)90203-h
27
DowningP. E. (2000). Interactions between visual working memory and selective attention. Psychol. Sci.11, 467–473. 10.1111/1467-9280.00290
28
EcksteinM. P. (2011). Visual search: a retrospective. J. Vis.11:14. 10.1167/11.5.14
29
FellJ.AxmacherN. (2011). The role of phase synchronization in memory processes. Nat. Rev. Neurosci.12, 105–118. 10.1038/nrn2979
30
FougnieD. (2008). “The relationship between attention and working memory,” in New Research on Short-Term Memory, ed JohansenN. B. (NY: Nova Science Publishers), 1–45.
31
FriesP.SchröderJ.-H.RoelfsemaP. R.SingerW.EngelA. K. (2002). Oscillatory neuronal synchronization in primary visual cortex as a correlate of stimulus selection. J. Neurosci.22, 3739–3754.
32
Goldman-RakicP. (1995). Cellular basis of working memory. Neuron14, 477–485. 10.1016/0896-6273(95)90304-6
33
HamaméC. M.VidalJ. R.OssandónT.JerbiK.DalalS. S.MinottiL.et al. (2012). Reading the mind’s eye: online detection of visuo-spatial working memory and visual imagery in the inferior temporal lobe. Neuroimage59, 872–879. 10.1016/j.neuroimage.2011.07.087
34
HogendoornH.CarlsonT. A.VanrullenR.VerstratenF. A. J. (2010). Timing divided attention. Atten. Percept. Psychophys.72, 2059–2068. 10.3758/APP.72.8.2059
35
HorowitzT. S.HolcombeA. O.WolfeJ. M.ArsenioH. C.DiMaseJ. S. (2004). Attentional pursuit is faster than attentional saccade. J. Vis.4:6. 10.1167/4.7.6
36
HorowitzT. S.WolfeJ. M.AlvarezG. A.CohenM. A.KuzmovaY. I. (2009). The speed of free will. Q. J. Exp. Psychol. (Hove)62, 2262–2288. 10.1080/17470210902732155
37
HoweP. D. L.CohenM. A.PintoY.HorowitzT. S. (2010). Distinguishing between parallel and serial accounts of multiple object tracking. J. Vis.10:11. 10.1167/10.8.11
38
JhaA. P. (2002). Tracking the time-course of attentional involvement in spatial working memory: an event-related potential investigation. Brain Res. Cogn. Brain Res.15, 61–69. 10.1016/s0926-6410(02)00216-1
39
JohnsonJ. S.HollingworthA.LuckS. J. (2008). The role of attention in the maintenance of feature bindings in visual short-term memory. J. Exp. Psychol. Hum. Percept. Perform.34, 41–55. 10.1037/0096-1523.34.1.41
40
JungJ.MainyN.KahaneP.MinottiL.HoffmannD.BertrandO.et al. (2008). The neural bases of attentive reading. Hum. Brain Mapp.29, 1193–1206. 10.1002/hbm.20454
41
KoppF.SchrögerE.LipkaS. (2006). Synchronized brain activity during rehearsal and short-term memory disruption by irrelevant speech is affected by recall mode. Int. J. Psychophysiol.61, 188–203. 10.1016/j.ijpsycho.2005.10.001
42
LaBergeD.BrownV. (1989). Theory of attentional operations in shape identification. Psychol. Rev.96, 101–124. 10.1037//0033-295x.96.1.101
43
LandauA. N.FriesP. (2012). Attention samples stimuli rhythmically. Curr. Biol.22, 1000–1004. 10.1016/j.cub.2012.03.054
44
LeberA. B. (2010). Neural predictors of within-subject fluctuations in attentional control. J. Neurosci.30, 11458–11465. 10.1523/JNEUROSCI.0809-10.2010
45
LeeS.-H.KravitzD. J.BakerC. I. (2013). Goal-dependent dissociation of visual and prefrontal cortices during working memory. Nat. Neurosci.16, 997–999. 10.1038/nn.3452
46
LismanJ. E.IdiartM. A. P. (1995). Storage of 7 +/−2 short-term memories in oscillatory subcycles. Science267, 1512–1515. 10.1126/science.7878473
47
LismanJ. E.JensenO. (2013). The Theta-Gamma neural code. Neuron77, 1002–1016. 10.1016/j.neuron.2013.03.007
48
LuckS. J.VogelE. K. (2013). Visual working memory capacity: from psychophysics and neurobiology to individual differences. Trends Cogn. Sci.17, 391–400. 10.1016/j.tics.2013.06.006
49
LutzenbergerW.RipperB.BusseL.BirbaumerN.KaiserJ. (2002). Dynamics of Gamma-Band activity during an audiospatial working memory task in humans. J. Neurosci.22, 5630–5638.
50
MiyashitaY.ChangH. S. (1988). Neuronal correlate of pictorial short-term memory in the primate temporal cortex. Nature331, 68–70. 10.1038/331068a0
51
MüllerM. M.MalinowskiP.GruberT.HillyardS. A. (2003). Sustained division of the attentional spotlight. Nature424, 309–312. 10.1038/nature01812
52
NakayamaK.MartiniP. (2011). Situating visual search. Vision Res.51, 1526–1537. 10.1016/j.visres.2010.09.003
53
OhS.-H.KimM.-S. (2003). The guidance effect of working memory load on visual search. J. Vis.3:629. 10.1167/3.9.629
54
OhS.-H.KimM.-S. (2004). The role of spatial working memory in visual search efficiency. Psychon. Bull. Rev.11, 275–281. 10.3758/bf03196570
55
PayneL.KouniosJ. (2009). Coherent oscillatory networks supporting short-term memory retention. Brain Res.1247, 126–132. 10.1016/j.brainres.2008.09.095
56
PosnerM. I.PetersenS. E. (1990). The attention system of the human brain. Annu. Rev. Neurosci.13, 25–42. 10.1146/annurev.ne.13.030190.000325
57
PostleB. (2006). Working memory as an emergent property of the mind and brain. Neuroscience139, 23–38. 10.1016/j.neuroscience.2005.06.005
58
RaaijmakersJ. G.ShiffrinR. M. (1981). Search of associative memory. Psychol. Rev.88, 93–134. 10.1037/0033-295X.88.2.93
59
RaffoneA.WoltersG. (2001). A cortical mechanism for binding in visual working memory. J. Cogn. Neurosci.13, 766–785. 10.1162/08989290152541430
60
RepovŠG.BaddeleyA. (2006). The multi-component model of working memory: explorations in experimental cognitive psychology. Neuroscience139, 5–21. 10.1016/j.neuroscience.2005.12.061
61
SakaiK.PassinghamR. E. (2003). Prefrontal interactions reflect future task operations. Nat. Neurosci.6, 75–81. 10.1038/nn987
62
SalazarR. F.DotsonN. M.BresslerS. L.GrayC. M. (2012). Content-specific fronto-parietal synchronization during visual working memory. Science338, 1097–1100. 10.1126/science.1224000
63
SausengP.KlimeschW.HeiseK. F.GruberW. R.HolzE.KarimA. A.et al. (2009). Brain oscillatory substrates of visual short-term memory capacity. Curr. Biol.19, 1846–1852. 10.1016/j.cub.2009.08.062
64
SerrienD. J.PogosyanA. H.BrownP. (2004). Influence of working memory on patterns of motor related cortico-cortical coupling. Exp. Brain Res.155, 204–210. 10.1007/s00221-003-1720-1
65
SiegelM.WardenM. R.MillerE. K. (2009). Phase-dependent neuronal coding of objects in short-term memory. Proc. Natl. Acad. Sci. U S A106, 21341–21346. 10.1073/pnas.0908193106
66
SingerW.GrayC. M. (1995). Visual feature integration and the temporal correlation hypothesis. Annu. Rev. Neurosci.18, 555–586. 10.1146/annurev.neuro.18.1.555
67
SongK.MengM.ChenL.ZhouK.LuoH. (2014). Behavioral oscillations in attention: rhythmic α pulses mediated through θ band. J. Neurosci.34, 4837–4844. 10.1523/JNEUROSCI.4856-13.2014
68
SperlingG. (1960). The information available in brief visual presentations. Psychol. Monogr. Gen. Appl.74, 1–29. 10.1037/h0093759
69
SreenivasanK. K.CurtisC. E.D’EspositoM. (2014). Revisiting the role of persistent neural activity during working memory. Trends Cogn. Sci.18, 82–89. 10.1016/j.tics.2013.12.001
70
TheeuwesJ.KramerA. F.IrwinD. E. (2011). Attention on our mind: the role of spatial attention in visual working memory. Acta Psychol. (Amst)137, 248–251. 10.1016/j.actpsy.2010.06.011
71
TreismanA. M.GeladeG. (1980). A feature-integration theory of attention. Cogn. Psychol.12, 97–136. 10.1016/0010-0285(80)90005-5
72
VanRullenR.CarlsonT.CavanaghP. (2007). The blinking spotlight of attention. Proc. Natl. Acad. Sci. U S A104, 19204–19209. 10.1073/pnas.0707316104
73
VanRullenR.ReddyL.KochC. (2005). Attention-driven discrete sampling of motion perception. Proc. Natl. Acad. Sci. U S A102, 5291–5296. 10.1073/pnas.0409172102
74
van VugtM. K.Schulze-BonhageA.LittB.BrandtA.KahanaM. J. (2010). Hippocampal Gamma oscillations increase with memory load. J. Neurosci.30, 2694–2699. 10.1523/jneurosci.0567-09.2010
75
VidalJ. R.OssandonT.JerbiK.DalalS. S.MinottiL.RyvlinP.et al. (2010). Category-specific visual responses: an intracranial study comparing Gamma, Beta, Alpha and ERP response selectivity. Front. Hum. Neurosci.4:195. 10.3389/fnhum.2010.00195
76
VogelE. K.McColloughA. W.MachizawaM. G. (2005). Neural measures reveal individual differences in controlling access to working memory. Nature438, 500–503. 10.1038/nature04171
77
Von SteinA.RappelsbergerP.SarntheinJ.PetscheH. (1999). Synchronization between temporal and parietal cortex during multimodal object processing in man. Cereb. Cortex9, 137–150. 10.1093/cercor/9.2.137
78
WangQ.CavanaghP.GreenM. (1994). Familiarity and pop-out in visual search. Percept. Psychophys.56, 495–500. 10.3758/bf03206946
79
WolfeJ. M. (1998). What can 1 million trials tell us about visual search?Psychol. Sci.9, 33–39. 10.1111/1467-9280.00006
80
WoodmanG. F.LuckS. J. (2004). Visual search is slowed when visuospatial working memory is occupied. Psychon. Bull. Rev.11, 269–274. 10.3758/bf03196569
81
WoodmanG. F.VogelE. K.LuckS. J. (2001). Visual search remains efficient when visual working memory is full. Psychol. Sci.12, 219–224. 10.1111/1467-9280.00339
Summary
Keywords
gamma oscillations, working memory, working memory capacity, attention, ECoG
Citation
Van Vugt MK, Chakravarthi R and Lachaux J-P (2014) For whom the bell tolls: periodic reactivation of sensory cortex in the gamma band as a substrate of visual working memory maintenance. Front. Hum. Neurosci. 8:696. doi: 10.3389/fnhum.2014.00696
Received
21 May 2014
Accepted
19 August 2014
Published
04 September 2014
Volume
8 - 2014
Edited by
Srikantan S. Nagarajan, University of California, San Francisco, USA
Reviewed by
Matias Palva, University of Helsinki, Finland; Theodore Zanto, University of California, San Francisco, USA
Copyright
© 2014 Van Vugt, Chakravarthi and Lachaux.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Marieke Karlijn Van Vugt, Artificial Intelligence, Cognitive Modeling Group, University of Groningen, Bernouilliborg, Room 326, Nijenborgh 9, 9712 CP Groningen, Netherlands e-mail: m.k.van.vugt@rug.nl
This article was submitted to the journal Frontiers in Human Neuroscience.
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.