In Figure 2, we consider a neuronal network learning a procedural task. Six neuronal groups (composed of 18 neurons each) are shown and labeled in Figure 2A. The procedural memory task is modeled in such a way that learning happens in a segregated channel, where activity percolates from left to right. To place this experiment within the context of a real-world example, the sequential task can be compared to learning a key progression on a piano.

The network is trained on the sequence A ⇒ B ⇒ C ⇒ D in pair-wise steps. First, the network is repeatedly exposed to the sequence A ⇒ B, then to the sequence B ⇒ C, and finally to the sequence C ⇒ D. This simulates the human behavior where a complex sequence is learned by decomposing it into simple sub-sequences. The network is trained on the correct sequence 90% of the time, while 10% of the time the network is trained on one of two “spurious” sequences (A ⇒ E ⇒ C ⇒ D and A ⇒ B ⇒ F ⇒ D). These erroneous sequences correspond to mistakes (i.e., noise) that may be made when learning a new scale on the piano.

In Figure 3A, we see the spike raster plots of the six neuronal groups during training. From the Figure, we see that the network is trained with A ⇒ B, B ⇒ C, and finally C ⇒ D. Figure 2B shows the changes in connection strength among the neural groups after training in wake. In the figure, the connection color corresponds to the source neuronal group, and the connection width corresponds to synaptic strength. One can see that the strongest connections (A ⇒ B ⇒ C ⇒ D) correspond to the learned sequence. However, other connections have also been strengthened, including those corresponding to “spurious” sequences.

As plasticity leads predominantly to potentiation during wake, the influence of the activation of several neuronal groups has broadened. For example, ideally neuronal group B would have a strong set of synapses to group C. However, due to spurious associations, B has established an effect on three additional neuronal groups. In such a setup, uninterrupted wake can lead to runaway potentiation, resulting eventually in strong connections between all neuronal groups and complete loss of specificity.

Because the neuronal groups corresponding to the signal (A ⇒ B ⇒ C ⇒ D) form stronger inter-group connections as compared to the groups involved only in spurious sequences (groups E and F), they are reactivated at a higher rate in sleep. As a result, the synapses corresponding to the correct procedural memory (A ⇒ B ⇒ C ⇒ D) are protected from depression, whereas the connections corresponding to spurious sequences are not. As renormalization mechanisms during sleep down-select the non-protected synapses, selectivity is enhanced – the connections underlying A ⇒ B ⇒ C ⇒ D remain strong after sleep, while many other connections are depressed. Figure 2C shows the connections strengths between various neuronal groups after a single period of wake-training and sleep. Finally, Figure 2D shows that after many training and sleep sessions, this effect is more pronounced.

After learning in wake, performance and S/N are assessed by evaluating the networks ability to correctly recall the learned sequence in wake. To do so, neuronal group A is clamped in the activated state for 25 ms, while the ensuing activations of other neuronal groups are recorded. A correct recall is counted whenever at least 50% of the neurons in the groups B, C, and D are activated in the correct sequence A ⇒ B ⇒ C ⇒ D within 500 ms of clamping A in the activated state. If 50% of either neuronal group E or F are activated within this time window after stimulus onset, the recall is counted as incorrect. Performance is quantified as the percentage of correct recalls, while S/N is measured as: (10)S∕N=NumberCorrectRecalledSequencesNumberIncorrectRecalledSequences

Recall percentages and S/N values for the experiments described in the next subsections are summarized in Table 1.

Neurons within the same group tend to be active at the same time due to the high degree of intra-group connectivity. As shown in Figure 3B, this activity often propagates between groups, “replaying” the newly learned sequence A ⇒ B ⇒ C ⇒ D. During simulated sleep periods, spontaneous activation of this kind leads to plastic changes mediating synaptic down-selection as described in Section 6. Accordingly, synapses are depressed in a manner inversely proportional to their reactivation frequency, leading to the connectivity shown in Figure 2D.

The baseline network shown in Figure 2A (before training) achieves, as expected, a recall performance of 0% and S/N of 0 (Table 1). Next, the network is trained with the signal (A ⇒ B ⇒ C ⇒ D) and spurious sequences (A ⇒ E ⇒ C ⇒ D and A ⇒ B ⇒ F ⇒ D) as described in Section 1, for a total of 10,000 ms iterations. Immediately after this wake-training period, the network is able to recall the sequence correctly 80% of the time, resulting in a S/N of 4. The corresponding connectivity is shown in Figure 2B.

Subsequently, the network is allowed to undergo synaptic down-selection in the sleep mode for 10,000 ms. The resulting synaptic connectivity is shown in Figure 2C. After sleep, the recall performance of the network improved to 90% and the S/N measure increases to 9 (Table 1). Figure 3C shows a correct recall sequence of the network, after the network had experienced a single wake-sleep cycle.

Finally, the network undergoes 10 wake/sleep cycles. As a result, the synapses corresponding to the correct sequence are significantly strengthened, whereas spurious connections have been down-selected, as shown in Figure 2D. The recall rate is 95% and the S/N measure increased further to a value of 19.

In Figure 4A, the spike-raster plot shows the correct recall of the sequence after a single period of wake followed by sleep-dependent down-selection. To establish whether synaptic down-selection in sleep can be substituted for by additional learning in wake, we simulate an additional wake-training period without intervening sleep. Accordingly, the network is trained in the wake mode for 20,000 ms. Figure 4B shows the spike raster plot of a typical recall sequence after the extended training period. Since the connections to the neuronal groups associated with the spurious sequences (E and F) have been further potentiated (while those associated with the correct sequences were already saturated), the correct sequence is recalled only 20% of the time, corresponding to an S/N measure of 0.25 (Table 1).

Next, we investigate the consequences of permitting synaptic potentiation, rather than down-selection, to occur during the slow oscillations occurring in sleep. For this purpose, the learning rule in the sleep mode is switched from the down-selection rule (Section 3) to the potentiation rule normally associated with wake (Section 2). An example of the networks recall performance is shown in the raster plot of Figure 4C. This example is representative of the fact that, with potentiation occurring during sleep, the neuronal groups involved in the spurious sequences (groups E and F) are activated quite frequently. As a result, the correct sequence was recalled only 25% of the time, corresponding to a S/N value of 0.33.

In Figure 5A, we consider a network composed of 16 neuronal groups. The neuronal groups are configured to identify one of 16 different patterns, A through P. During training in wake, pairs of neuronal groups are simultaneously activated. Each of the pairs is activated for 2,000 ms to induce strong synaptic potentiation.

Figure 5B shows the connectivity established among the neuronal groups once the network has been trained with 8 pairs of associations (e.g., A and L, B and M). In this experiment, we stimulate one neuronal group and observed whether 50% (or more) of the neurons in its associated neuronal group fired within 50 ms of the onset of stimulation. For example, we stimulate neuronal group A and observe whether neuronal group L is correctly recalled (activated). As with the procedural memory task, if any neuronal group outside of the paired associate showed at least 50% activation within this window, the trial is counted as an incorrect recall. The S/N for the declarative memory task is defined as: (11)S/N=PercentageCorrectRecalledPairsPercentageIncorrectRecalledPairs

Initially, the network shown in Figure 5A has no strong associative links, the baseline recall rate is 0%, and the S/N is 0. Immediately after the training phase, the strengthening of connections demonstrated in Figure 5B leads to a recall rate of 87%, corresponding to a S/N of 6.7 (see Table 2). After a period of sleep (see Figure 5C), the network recalls the learned pairs correctly 75% of the time, with a S/N of 2.7. If the network is allowed to undergo 10 wake/sleep cycles, the recall rate improves to 94%, with a S/N of 15.6.

Next, we simulate the standard experimental comparison between the effects of sleep after training versus extra wake. Of the two identical networks trained on the associative recall task, one is allowed to enter the sleep mode for 20,000 ms (as described above), while the other is kept in the wake mode for the same amount of time. To mimic interference due to spurious associations occurring in wakefulness, the second network is exposed to six random associations, different from the ones learned during the initial training. Each of these spurious associations is presented to the network for 300 ms, leading to further synaptic potentiation. As a result, the awake network shows correct recall only 53% of the time, corresponding to a S/N value of 1.12.

The results obtained after declarative learning should be compared to the one obtained with procedural learning above. With the procedural task, recall after training in wake is 80% and after down-selection in sleep it improves to 90%. With the declarative task, recall after training in wake is 87% and after down-selection in sleep it is reduced to 75%, compared to 53% without sleep. This difference between absolute improvement after procedural learning and relative preservation (reduced degradation) after declarative learning is in line with experimental results (2). In the simulations, these results can be explained by considering that, in the procedural task, acquisition of both correct and spurious associations through synaptic potentiation in wake is confined within a single channel, reducing the effects of interference. By contrast, in the declarative task, the all-to-all connectivity that is characteristic of associative matrices is much more prone to interference by spurious associations.

Next, we examine the effects of synaptic potentiation during sleep. For this experiment, the network is trained as before with 8 associated pairs for 2,000 ms. However, instead of synaptic down-selection, the network uses the potentiation rule as in wake (see Section 2.2). As a consequence, the recall rate of the network degrades to 60%, with a S/N value of 1.5. Examining the dynamics of activation and plasticity during sleep revealed that the primary reason for this degradation in performance and S/N is the spurious strengthening of connections between neuronal groups that are spontaneously activated during sleep.

Finally, based on recent experimental results [e.g., Ref. (68, 69)], we also examine the effects of stimulus cuing during sleep. As before, the network shown in Figure 5B is trained in wake to learn 8 associative pairs. After training, the network has a recall rate of 87% and a S/N measure of 6.7. During sleep, 4 of the trained pairs are cued (i.e., strongly activated through external stimuli), while the other 4 pairs are activated as before only through the occurrence of spontaneous slow oscillations. The external cues are presented to the network 3 times each (1 pair at a time) for a period of 500 ms per cue. The total sleep time was again 20,000 ms.

During external cuing in sleep, the plasticity of the network was still governed by synaptic down-selection. Each of the cued neuronal groups spent on average 8% more time in the up-state during slow wave oscillations. Because each of the cued pairs were activated together as a result of the stimulation, the synapses between them were better protected than those between the non-cued pairs, which became activated only through the simulated slow wave oscillations.

Figure 5D shows the state of the network after the cuing in sleep. The color saturation of each neuronal group indicates its intensity of activation during sleep, which has a direct impact on the synaptic strength between the associated pairs. When we measure recall performance for the entire network after 1 sleep cycle, the value is 85%, corresponding to a S/N value of 5.7, higher than without cuing. We then compare directly the cued with the non-cued pairs. As expected, the non-cued pairs have a recall rate of 75% with an S/N of 3, values similar to those observed without cuing (see Section 2). By contrast, the cued pairs have a recall rate of 95% and a S/N measure of 19, an improvement in line with empirical results (68, 69).

Alternatively, if synaptic potentiation occurs during sleep, the network again runs the risk of strengthening spurious connections, as described in Section 4. If potentiation is limited only to cued pairs, while non-cued pairs undergo synaptic down-selection, the results are again in line with those shown in Figure 5D; however, such a result implies that during sleep the brain would have a way to implement different plasticity rules for cued and non-cued pairs and to differentially tag the relevant synapses during wakefulness.

For this purpose, we arrange neuronal groups according to the simple hierarchical organization shown in Figure 6. This network architecture is inspired by the hierarchical organization of anatomical connections in the cerebral cortex and by the physiological evidence showing that neurons in higher regions respond to more invariant stimuli than neurons in lower regions (70). In Figure 6A, each circle represents a neuronal group of 18 interconnected neurons. For this experiment, the first level contains 26 neuronal groups, the middle level 14, and the upper level 3. To ensure that the top level of the network receives feedback activations necessary to trigger plasticity, it is recurrently connected to a fourth level via non-plastic synapses (not shown). As with other experiments, feedforward and feedback connections are initialized to W_max/10 (though feedback connections are not shown to avoid clutter).

Figure 6B shows four different stimuli on which the network is trained during wake, indicated by the colored arrow inputs to the bottom level of the network. Note that the orange, purple, and blue stimuli each partially overlap with the green stimuli. During wake, each of the four stimuli is presented to the network for 500 ms, four times each, with the order chosen randomly. The total training time in wake is 8,000 ms.

Figure 6B shows the feedforward connections after training in wake, where the thickness of the connections corresponds to their synaptic weight. While many connections became stronger, those relaying features that overlapped among the different stimuli – that is, the gist – show the largest increase in strength (at or close to W_max; connections with strength at or less than W_max/10 are omitted for clarity).

In Figure 6C, the color intensity of each of the neuronal groups indicates its frequency of activation during a subsequent period of sleep lasting 10,000 ms. As also shown in the figure, due to down-selection during sleep, many of the synapses are depressed, but those corresponding to the gist shared by the four stimuli are mostly preserved. That is, during down-selection in sleep the connections between level 2 and 3 are comparatively better protected than those between level 1 and 2. Thus, spontaneous activation and down-selection during sleep favor gist extraction.

In the model, an important factor promoting the relative preservation of gist versus detail has to do with the dynamics of spontaneous activation during sleep. The general hypothesis is that neuronal patterns in sleep are more frequently activated top-down (level 3 before 2 before 1) than bottom-up (level 1 before 2 before 3).

To validate this notion, we perform two experiments. Using hierarchical network of neurons similar to that shown in Figure 6A, feedforward and feedback weights are initialized to W_max/5. Furthermore, the timescale of the feedback connections is set equal to the timescale of feedforward connections (1 ms), though feedback connections maintained their voltage-dependent behavior. For this experiment, while the network is engaged in spontaneous slow oscillations through noise injections, synaptic down-selection is turned off. A random neuron is chosen in level 1 and level 3, and we observe the frequency with which the neuron initiated a spike that will percolate through the hierarchy. Percolation is successful if, for example, the neuron in level 3 spikes, then in the next time step a neuron (within the first neuron’s receptive field) in level 2 also spikes, and finally a neuron (within the first neuron’s receptive field) spikes in level 1. Bottom-up sequences are detected the same way, but in the opposite direction (level 1 to 2 to 3). After 10,000 ms of simulated sleep, the number of top-down initiated sequences was nearly 4 times that of bottom-up sequences.

In our second experiment, we take the same network (in simulated slow wave sleep) and impose a steady activation of either 1/3 of the neurons in the top level (level 3), chosen at random, or of 1/3 of the neurons in the bottom level (level 1). For this experiment, the average firing rate of the network is 5% higher when the stimulation is provided to the top level, despite the fact that when the stimulation is provided at the bottom level, substantially more neurons are activated (see Figure 6). In the model, this bias for top-down activation can be explained by the narrower fan-out of feedforward versus feedback connections in a convergent-divergent hierarchical network. In the bottom-up, feedforward direction, it is unlikely that the randomly stimulated neurons fit the receptive field of a higher-level neuron to make it fire. This unlikelihood is multiplied when considering the percolation of activity in a feedforward direction over multiple levels. By contrast, since a level 3 neuron has a broad fan-out to lower levels, it can bias toward firing many compatible patterns of activation in lower levels, so activation is more likely to propagate in top-down direction.

The bias for top-down initiated activity is enhanced further by the longer timescale effects typically associated with top-down connections. By increasing the timescale of feedback connections to 2 ms, the activity of the network is 20% higher when the top level is activated. The bias is even larger if the total number of feedback connections is made to be greater than the number of feedforward connections, as has been suggested for corticothalamic versus thalamocortical connections (71, 72).

Both theoretical considerations (73) and experimental evidence (5, 17) suggest that off-line activation of neural circuits during sleep may provide an ideal setting for the integration of newly acquired associative links with an established body of knowledge within the brain. To assess how sleep-dependent down-selection mechanisms can aid this process of memory integration, we resort again to a hierarchically organized network of neuronal groups (Figure 7). Each of the 3 levels of the network comprised 30 neuronal groups containing 18 neurons each. As with the gist example, the top level of the network receives non-plastic feedback activations from a fourth level in order to initially trigger plasticity (not shown).

The two identical networks are then endowed with “old” memories (shown in red in Figures 7A,D, implemented by a set of strong feedforward and feedback connections between levels (initialized to W_max). Then, both networks are exposed to a stimulus to create a “new” memory. The network in Figure 7B is exposed to a stimulus (shown in green) that has substantial overlap with the old red memories. Conversely, the network in Figure 7E is exposed to a stimulus (shown in blue) that only overlapped with a single neuronal group (per level) associated with the old red memories. To prevent extensive subliminal plasticity through the entire network of associations during wake, we keep the majority of the neuronal groups in the second and third levels of the network in an inactive state (shown in gray). In real brains, global activation and plasticity during experience is ruled out by several mechanisms of intra- and inter-areal competition and inhibition.

As shown in the figure, after a wake-training period of 10,000 ms, both networks has acquired the new memory by strengthening several connections between the levels (shown in blue and green, respectively). Thereafter, both networks are set in the sleep mode for an additional 10,000 ms. During simulated sleep, the new green memory (Figure 7C) is often co-activated with the old red memories, due to the presence of many points of contact. In the figure, the frequency of activation is indicated by the color saturation of the neuronal group. Since connections belonging to the new green and old red memories often terminate on the same synaptic compartments, even partial activations are able to protect synapses from down-selection, as long as activations are conveyed by both feedforward and feedback connections. As a result, the synaptic weights of the new green memory are depressed, on average, by only 2%. Conversely, a significant portion of the new blue memory (Figure 7F) can only be activated by the other neuronal groups within the blue memory. As a result, the blue memory often encountered mismatched feedforward and feedback activations, resulting in substantial depression: synapses between levels 1 and 2 are depressed on average by 28%, and those between level 2 and 3 by 11%. Thus, as expected, spontaneous activation in the sleep mode protected preferentially the new memory that overlapped significantly with old memories, resulting in memory integration.

As mentioned in the Section “Introduction,” down-selection should occur during sleep, when the brain is off-line and it can comprehensively activate its memory bases. By contrast, if the necessary renormalization of synaptic strength are to happen in wake (concurrently with learning by potentiation), it will run the risk of degrading important memories simply because they happen not to be activated during the limited sampling dictated by a day’s wake behavior.

To illustrate this aspect with a simple example, in this final simulation we implement renormalization during wake while a new memory was being formed. The same network shown in Figure 7A is exposed to the green stimulus as in Figure 7B. As in the previous simulation, during wake-training the majority of the neuronal groups in the second and third levels of the network, corresponding to old memories, are kept in an inactive state (shown in gray). This is done to replicate the sampling bias occurring during any particular wake episode: on a given day, the brain is typically faced with some novel stimuli (relating, say, to a new acquaintance), and has no opportunity to activate most of its old memories (including memories related, say, to one’s old friends).

To achieve activity-dependent synaptic renormalization during wake, the plasticity rule is modified by allowing for the depression of synapses that are inactive when the postsynaptic neuron fired, not unlike STDP (62–64). During wake-training, neurons responding to the features of the new stimulus (related, say, to the face, voice, posture etc. of a new acquaintance) are activated often, so the active synapses conveying the new stimulus became progressively stronger, laying down a trace for the new memory (green in Figure 7B). However, on the same day, synapses onto the same neurons originating from neurons involved in old memories (related, say, to the face, voice, posture etc. of old friends) are almost always inactive, and undergo progressive depression (on average, by 90% when using the same learning rates for potentiation and depression). Thus, activity-dependent synaptic renormalization in wake will achieve new learning at the expense of potentially degrading important features of old memories. Clearly, it will not be advisable, when making a new friend, to run the risk that of losing the old ones simply because they were not present on that day.

During simulated sleep, the effects of the synaptic depression in wake are compounded. As the new memory no longer overlaps with the old memory, the advantages of co-activation in sleep are absent. As a result, the connections of the new memory are depressed to the same levels as they would have been were they independent of an old memory (similar to the network shown in Figure 7F.

This last requirement implies that synaptic potentiation should occur primarily in wake, when an organism interacts with its real environment, and not in sleep, when it is disconnected and exposed to its own dreams or “fantasies” (80). While in wake non-declarative skills can be acquired under the control of environmental feedback, new learning during sleep could easily drift onto inappropriate modes. These negative consequences of synaptic potentiation during sleep are schematically illustrated in the simulations of Section 3.2.4 and 3.1.6 for declarative and procedural memories, respectively. The silence of diffuse neuromodulatory systems during most of sleep (50, 51), the associated lack of induction of genes implicated in synaptic potentiation (56, 81), and consistent evidence that true learning is not possible in sleep (82) are in line with the idea that as a rule the acquisition of new memories should be confined to wake.

It must be emphasized that the learning rule used in the present simulations is only meant to be representative of the particular mechanisms implemented by any given neuronal class. In fact, these are likely to differ, even substantially, in different species, brain structures, cell types, and developmental times (40). What the synaptic homeostasis hypothesis explicitly predicts, however, is that wake is bound to lead to a net increase in synaptic strength, which in turn needs then to be renormalized by sleep.

During a typical wake period, when the brain is on-line and learning, it is necessarily faced with a limited sampling of the statistical structure of the environment. For example, one may make a new acquaintance with whom one spends a large portion of the day. As a consequence, many neurons in one’s brain will detect and learn, primarily by synaptic potentiation, several new “suspicious coincidences” having to do with that person’s face, voice, posture, and so on, forming new memories that are potentially very useful. Under these circumstances, however, it would be maladaptive if neurons were to satisfy the requirement for renormalization by depressing those synapses that were not involved in signaling the new acquaintance, just because they were not activated much on that particular day, thus progressively erasing old memories – those of old friends – that are just as useful. These negative consequences of synaptic renormalization in wake, due to the inevitably limited sampling a given waking day affords, were illustrated by a simple example in Section 3.4.2. Conversely, as illustrated in Section 3.4.1, this problem is solved when the brain disconnects from the environment in sleep; decoupled from the requirements of behavior, the brain can afford to perform a comprehensive sampling of all its previous knowledge (old memories). In this way, connections conveying important old memories will be protected, and so will those conveying new memories that fit best with the old ones, whereas only connections conveying spurious coincidences will be at a competitive disadvantage and be depressed.

As was mentioned in the Methods, various synaptic rules enforcing depression during sleep can be envisioned, including a proportional scaling down of all synapses (16) and a rule biasing depression to spare stronger synapses more than weaker ones (48). The down-selection rule implemented here is almost a mirror image of the activity-dependent rule used for synaptic potentiation during wake (Figure 1B). This rule, too, is only meant to be representative, and in fact, as indicated in the Section “Materials and Methods,” it can be modified in various ways while leading to qualitatively similar effects. Altogether, the down-selection process ensures the survival of those circuits that are “fittest,” either because they were strengthened repeatedly during wake, or because they are better integrated with older memories. Instead, synapses involved in circuits that were only occasionally strengthened during wake, or fit less well with old memories, are depressed and eventually eliminated. As shown by the present simulations, the same rule can account for several of the beneficial effects of sleep on memory, including consolidation, gist extraction, and integration.