Autonomous retrieval for continuous learning in associative memory networks

Abstract

The brain's faculty to assimilate and retain information, continually updating its memory while limiting the loss of valuable past knowledge, remains largely a mystery. We address this challenge related to continuous learning in the context of associative memory networks, where the sequential storage of correlated patterns typically requires non-local learning rules or external memory systems. Our work demonstrates how incorporating biologically inspired inhibitory plasticity enables networks to autonomously explore their attractor landscape. The algorithm presented here allows for the autonomous retrieval of stored patterns, enabling the progressive incorporation of correlated memories. This mechanism is reminiscent of memory consolidation during sleep-like states in the mammalian central nervous system. The resulting framework provides insights into how neural circuits might maintain memories through purely local interactions and takes a step forward toward a more biologically plausible mechanism for memory rehearsal and continuous learning.

1 Introduction

Continuous learning (CL) refers to a system's ability to maintain performance across multiple tasks when operating in environments that evolve over time, requiring adaptation to changing data distributions. To do so, the learning mechanism should avoid uncontrolled forgetting of previously acquired knowledge when adapting to new information or contexts. In associative memory networks, this challenge arises when storing new activity patterns sequentially deteriorates existing memory representations, a phenomenon called catastrophic forgetting.

It is important to note that this work specifically addresses catastrophic forgetting in the context of sequential learning. This approach addresses a different challenge than the well-studied spin glass phase transitions that occur in Hopfield networks at high memory loads.

Memory rehearsal is a method that addresses the challenge of catastrophic forgetting by periodically retraining the model on previously stored patterns. This process reinforces older memory representations, preventing their degradation when new information is incorporated (Robins, 1995; McCallum, 1998). In the mammalian nervous system, spontaneous memory replays occur during sleep (Robins, 1995; Louie and Wilson, 2001; Peyrache et al., 2009; Fauth and Van Rossum, 2019; Tononi and Cirelli, 2014), suggesting a biological mechanism analogous to rehearsal techniques in artificial networks (Robins, 1995). This parallel raises a fundamental question: what mechanisms enable these autonomous memory replays in biological systems?

Previous research on bioinspired neural networks demonstrates that short-term synaptic depression can facilitate spontaneous rehearsal of neural assemblies (Fauth and Van Rossum, 2019). However, a significant constraint of this approach is its dependence on minimal overlap between neural assemblies. The neuronal populations in these studies share few neurons, resulting in effectively decorrelated memory representations. By contrast, classical associative memory networks like Hopfield Networks can effectively store uncorrelated memories that share many units. Some learning algorithms even allow the storage of highly correlated patterns that share a majority of their units (Diederich and Opper, 1987). From a biological perspective, understanding how networks implement the rehearsal of correlated populations is crucial as neural representations found in the cortex generally recruit extensively overlapping assemblies (Haxby et al., 2001; Kriegeskorte, 2008). The maintenance of overlapping assemblies is widely considered essential for cortical computation, as it supports stimulus generalization and the emergence of invariant, high-level concepts in which individual neurons participate in multiple but related representations (Haxby et al., 2001; Quiroga et al., 2005; Kriegeskorte, 2008). This problematic is of similar importance in neuromorphic engineering contexts as highly correlated representations are an emerging feature of artificial neural networks (Kothapalli, 2023).

How the rehearsal of correlated memories takes place autonomously on neural substrates remains a largely unaddressed question. In this work, we focus on this issue and explore its potential application for continuous learning (CL). For the sake of bio-plausibility and potential implementation in neuromorphic substrates, we shall demand that our system exhibits the following features: 1) It stores memory states that can be highly correlated; 2) During the pattern recovery, the network does not converge toward strange attractors which would constitute false memories; 3) Plasticity rules are local, meaning that the modification of synaptic efficacy can be computed in terms of its pre- and post-synaptic neuron states; 4) It is autonomous, namely, it should retrieve all previously stored patterns from its own dynamics. The network does not have access to an external list of previously recorded memory states. For the sake of requirements (1) and (3), we shall adopt a perceptron-like algorithm, inspired by the work of (Diederich and Opper 1987). On the other hand, for requirement (2), we shall use continuous Hopfield Networks (CHNs) (Hopfield, 1984). Our work demonstrates that, kept under a certain memory load, CHNs converge exclusively to stored patterns during the retrieval. This approach avoids both the spurious state proliferation common in Discrete Hopfield Networks (DHNs) and the shortcomings of temperature parameter fine-tuning inherent to Stochastic Hopfield Networks (SHNs) (Amit et al., 1985a).

The main contribution of the present work is to introduce an algorithm to address the requirement (4). A crucial feature of our approach is the use of self-inhibition to shrink the basin of attraction of previously visited attractor states, thus allowing for a sequential and thorough search and recovery of all previously stored correlated memory states. This recovery effectively allows the rehearsal of the stored pattern for CL purposes. The dynamic of plastic recurrent inhibition is inspired by computational neuroscience work based on actual neurophysiological data (Vogels et al., 2011).

2 Methods

2.1 Continuous Hopfield Network (CHN)

In contrast with the conventional DHN model (Hopfield, 1982), where neural states are defined as binary variables, in a CHN they are continuous (Hopfield, 1984). The dynamics of the network is defined by a set of differential equations with each neuron unit described as a leaky integration:

where u_i is the membrane potential of neuron i, c is the membrane capacitance, r is the leak resistance of each neuron, W_ij is the synaptic efficacy between neurons j and i, and v_i is the activity (or firing rate) of neuron i that depends solely on the potential as

where σ is a monotonically increasing function of u with saturation to prevent runaway dynamics. We adopt . Therefore, as σ(0) = 0.5, each unit has a positive output at the resting state, allowing the network to have a baseline activity without external input. We shall refer to either the vector v(t) or u(t) as “states”, which should be clear from the context.

The convergence of the flow to stable states for the case of symmetric synaptic weights W_ij has been demonstrated (Hopfield, 1984). Throughout this work, whenever we integrate these equations using Euler method, we do so until the network reaches convergence, defined as , where ϵ = 10⁻⁶.

2.2 Pattern storage and reading

The states v(t) of the CHN evolve on [0, 1]^N through continuous dynamics, with N the number of neurons. Any state in this space may represent a stored pattern. For simplicity, we restrict ourselves to store binary patterns for which active neurons have a high firing rate, v_i≈1, and inactive neurons have a low firing rate, v_i ≈ 0.

Hence, a pattern x^μ is defined as a binary vector such that where for each unit i. A pattern is read from the state of the network at time t using a threshold:

Given a binary pattern x^μ, it is convenient to define target potentials as

We adopt here u_target = 6 to ensure proper pattern reading following the thresholding procedure.

Inspired by previous work on DHN (Diederich and Opper, 1987), we introduce, in Algorithm 1, a perceptron-inspired learning algorithm for efficient storage of correlated patterns in CHNs. The algorithm minimizes the error between the target states and the network's equilibrium states, ensuring that each memory becomes a stable state. Gradient descent methods typically require small step sizes to prevent the optimization process from becoming unstable and to reduce oscillations around local minima. In our implementation, we select α = 0.0001 as the learning rate to ensure stable convergence of weight updates. The derivation of the weight update rule can be found in the Appendix 1. Although a rigorous proof of convergence for the algorithm is beyond the scope of this work, we expect that arguments demonstrating the convergence of the gradient descent algorithm (GDA) in the context of DHN could be adapted for this purpose (Diederich and Opper, 1987). Here, we rely on numerical evidence showing the network's ability to successfully query and revisit stored patterns.

Following the network training with the GDA, each activity vector becomes an attractor. The network can now be queried as the system reliably converges to the nearest stored state from a partial cue. The corresponding binary patterns can then be accurately retrieved by thresholding at time t_f (Equation 3) when the system reaches convergence. The querying procedure is detailed in Algorithm 1 (Appendix 2) and illustrated in Figure 1.

Figure 1

It is worth emphasizing that despite the continuous nature of our model, which theoretically allows for a richer state space, we deliberately restrict ourselves to binary patterns. The interest of using a CHN is the simplicity it allows when implementing our pattern recovery mechanism (Section 2.4), limiting the appearance of false memories as spurious states, which are often encountered in DHNs (Amit et al., 1985b). A variant of the DHN with stochastic units, the SHN (Amit et al., 1985b), would be a possible candidate to implement our algorithm, as they tend to visit only the stored patterns by properly controlling the annealing temperatures. However, the stochastic properties of these networks would require a more complex setup.

2.3 Continuous incorporation of correlated memories

While the GDA effectively enables the storage of correlated memories, its implementation in Algorithm 1 reveals a significant limitation: It requires multiple iterations over the entire set of patterns to achieve convergence. Without the ability to reprocess all patterns, the network would suffer from catastrophic forgetting, where learning a new pattern in isolation rapidly erodes previously stored memories (McCloskey and Cohen, 1989; Robins, 1995; Kirkpatrick et al., 2017; Shen et al., 2023). By repeatedly processing all patterns, the algorithm can find a weight matrix W that properly separates the patterns, despite their correlations (Diederich and Opper, 1987). Adding a new pattern, therefore, requires access to all previously stored patterns from an external source.

This requirement for external access to the complete memory dataset stands in contrast to biological learning systems, which must incorporate new information while maintaining past memories without relying on an explicit external copy of the already stored data. To overcome this external dependency and move toward more biologically plausible learning, a solution is to develop a mechanism that allows the network to internally recover its stored memories.

The development of such an autonomous retrieval mechanism would allow us to exploit the GDA's ability for the continuous incorporation of correlated memories. The continuous learning algorithm is formally defined in Algorithm 2. The act of retraining the network on the whole set is called a rehearsal. Recovering the whole set from the network to allow rehearsal is called retrieval.

2.4 Autonomous retrieval

In this section, we present the autonomous retrieval (AR) mechanism allowing the recovery of stored correlated patterns in a network.

Given a trained network initialized at the “neutral” state, u_i = 0 for each unit i, the network dynamics described by Equation 1 converge deterministically to a given stored attractor, which is thus “retrieved”. This attractor can be seen as the dominant attractor from the neutral state. The goal now is to allow for the exploration of other states to permit a complete recovery of the stored memories. To do so, we introduce the adaptation terms A_i.

with v_i(t_f) the firing rate of neuron i after convergence of the dynamics. β represents the adaptation strength and is chosen to be small, typically below 0.1. Adaptation terms could correspond to various components commonly observed in the mammalian central nervous system. On short time scales, spike frequency adaptation (SFA) of excitatory neurons functions as plastic self-inhibition (Ha and Cheong, 2017; Peron and Gabbiani, 2009). Repeated stimulation progressively reduces firing activity in the neuron, mirroring the dynamics produced by our adaptation A_i. Alternatively, A_i can be interpreted as recurrent inhibition mediated by local inter-neurons, which frequently exhibit Hebbian plasticity (Kodangattil et al., 2013; D'amour and Froemke, 2015).

After each visited attractor, i.e., memory retrieved, its basin of attraction is made smaller by the update of the adaptation term (Equation 5). Once a pattern has been inhibited, the probability for the network to converge into it from the neutral state is reduced. By resetting the network to the neutral state after each convergence-inhibition cycle, we allow the sequential recovery of stored patterns.

Figure 2 illustrates the sequential recovery of two stored patterns and the modification of their attractor basin during the procedure. The geometry of these attractor basins explains why a minimal inhibitory influence, resulting from a small β value, is sufficient to alter the trajectory. The neutral state resides near the separatrix that divides the attractor basins. Consequently, even a slight modification of the separatrix position, caused by inhibitory potentiation, can significantly redirect the network's trajectory.

Figure 2

The increase in adaptation tends to distort the stable states associated with stored patterns. As this distortion is minimal for small β, it is mainly compensated for by the thresholding mechanism used to read the network output (Section 2.2). To further reduce the impact of this distortion, we divide the convergence dynamic into two phases, the “biased” phase and the “free” phase. The biased phase guides the network to converge toward states that have not been retrieved. It corresponds to the simulation of the network with adaptation, Equation 4, until convergence. The optional free phase then allows the network to complete its convergence to an undistorted stored state. It corresponds to the simulation of the network without inhibitory synapses (Equation 1) until convergence. The whole procedure is detailed in Algorithm 3.

Figure 3 provides a visualization of AR for binary-pattern representation of handwritten digits from the MNIST dataset (LeCun et al., 1998). For the first iteration, the inhibitory drive is null as no pattern has been retrieved. The pattern corresponding to the binary picture of a 3 is recovered. The network state is reinitialized with the updated adaptation (A). The network now inhibits the recovery of a 3, which induces convergence toward a second pattern, here a 4. Adaptation is updated again and now inhibits both the 3 and the 4 together. Inhibition of the 3 and the 4 combined allows the recovery of the 5 and so on. For the recovery of MNIST binary digits, as a large inhibitory coefficient β has been chosen, the free phase is mandatory to reduce the distortion of the stored pattern.

Figure 3

The order in which patterns will be visited is an emerging feature of the learning algorithm that has not been studied in this work. With each iteration of the AR algorithm, inhibitory synapses undergo potentiation. This growth is constrained only by the number of iterations and the value of β. Theoretically, such an unbounded growth could cause the adaptation A_i to disrupt the attractor dynamics induced by W. However, in practice, this potential issue can be managed by adjusting the value of β based on the number of iterations. Specifically, when more iterations are required, using a smaller β is sufficient to maintain robust pattern retrieval without compromising attractor dynamics.

In Figure 4, we illustrate the dynamics of the CHN during the retrieval of stored memory patterns in networks with various loads. The load corresponds to M/N, with M the number of stored memories, and N the size of the network. For low loads (see Figure 4 top), the system converges sequentially to the attractors corresponding to the stored patterns. Once every memory has been recovered, if the simulation is not ended, the network falls back into the stored states without showing any false memories. The lower the value of β, the longer this dynamic can proceed without encountering spurious states. The free phase has no utility in this scenario, the attractors are well separated, and the disruption of the energy landscape by adaptation is minimal. For critical loads (see Figure 4 middle), the retrieval process becomes more challenging. The attractors exhibit reduced separation, and the network struggles to converge to the stored states once inhibition is applied. In this scenario, the free phase demonstrates its utility. At the end of the biased phase, during the second iteration of the AR, the network remains in a mixed state characterized by ambiguous correlations with multiple stored patterns. The free phase enables the network to resolve this ambiguity and ultimately converge to a stored state. At high loads (see Figure 4 bottom), the network successfully retrieves some stored states but mostly fall into spurious attractors. Under these conditions, even the free phase cannot rescue the recovery dynamics.

Figure 4

These autonomous recovery dynamics are reminiscent of memory replays observed in the central nervous system of mammals during quiescent states, such as sleep or rest. This process is believed to function as a consolidation mechanism that mitigates or modulates memory forgetting (Robins, 1995; Louie and Wilson, 2001; Peyrache et al., 2009; Fauth and Van Rossum, 2019; Tononi and Cirelli, 2014).

The use of recurrent plastic inhibitory synapses has been tested and shows the same qualitative dynamics as that observed for networks with units that undergo adaptation. The results and the model are detailed in Appendix 4. As implementing adaptation requires less computation and parameters, the following work focuses only on the adaptation model.

3 Results

In this section, we evaluate the ability of our algorithm to retrieve correlated patterns from a given network. We will consider the retrieval of pattern sets with various amounts of correlation. Each set gets assigned a random binary pattern, the parent pattern. Each pattern of a given set is generated by randomly choosing and randomizing a fraction (1−ρ) of bits from the parent pattern as described in Algorithm 2 (Appendix 3). Therefore, a higher ρ induces more correlation, while a lower ρ results in less correlation. Networks with various loads will be tested. All retrieval dynamics are considered without free phases as it has been observed that, for the retrieval of random binary patterns and the use of small inhibitory potentiation values β, the free phase does not significantly improve retrieval performance.

Figure 5 illustrates, for various correlations, the loads for which AR allows systematic recovery of all stored patterns without external cues or memory lists. This property enables the continuous incorporation of new correlated memories, as described in Algorithm 2. The retrieval improves with network size—larger networks can reliably retrieve more stored patterns without encountering false memories. We can observe the rather surprising feature that a moderate amount of correlation (ρ≈0.5) tends to improve pattern retrieval compared to highly correlated and minimally correlated pattern sets. In fact, this finding contrasts with traditional associative memory models, where correlation typically degrades performance (Fontanari and Theumann, 1990).

Figure 5

Our interpretation is that an intermediate amount of correlation helps the system to be driven in the “good” direction in early stages of the evolution, i.e., while still rather close to the neutral state, and the energy landscape is rather featureless. Once driven to a point of the state space proximal to all the stored patterns, the system can finish the convergence. As illustrated in Appendix 5, Appendix Figure 3, this push toward the good direction can be measured through the average correlation between the synaptic drive of each unit and the stored patterns.

As emerges from our simulations, to limit the appearance of false memories, the values of β must be relatively small compared to the target potential u_target. In our case, we found it convenient to keep β smaller than 0.1. By keeping the nudging of the convergence dynamic small, the emergence of false memories that might otherwise result from deformations in the energy landscape is reduced. Figure 6 indicates the existence of a trade-off: Smaller β values require more iterations to retrieve all patterns but provide greater stability, while larger values accelerate pattern retrieval at the cost of increasing the probability of encountering a spurious state. Higher values of β than those considered in this study lead to catastrophic degradation of recovery dynamics for which no stored patterns are recovered. Lower values of β only lead to the need for more iterations when recovering the pattern set.

Figure 6

We shall now describe some qualitative information on the efficiency of our algorithm. As expected, the number of iterations required to successfully store patterns using Algorithm 1 increases with the memory load (Figure 7). Moreover, the higher the correlation between patterns, the higher the number of iterations needed for storage. Overall, our method requires significantly more iterations than previously documented for associative memory tasks in DHNs (Diederich and Opper, 1987). We can argue that this increase in computational cost may come from two factors. First, training a CHN through GDA (Algorithm 1) inherently requires more iterations than training DHNs. Second, we observed that a smaller convergence parameter ϵ in Algorithm 1, while computationally more demanding, yields superior retrieval performance. We hypothesize that this tighter convergence criterion induces stronger competition between pattern attractors at the neutral state. This competition results in enhanced network responsiveness to subtle modifications of attractor basins induced by W' during retrieval. These observations led to the adoption of a very small ϵ = 10⁻⁶, which demanded more iterations when performing the GDA.

Figure 7

4 Discussion

Our work introduces a biologically inspired mechanism for the continuous incorporation of correlated patterns in associative networks. CL is made possible through the autonomous recovery of all stored patterns during a retrieval phase. By systematically retrieving memories, the network can incorporate new patterns while mitigating the forgetting of the ones already stored. Autonomous retrieval is made possible by adaptation, avoiding the necessity of recalling patterns from an external list.

Previous work in computational neurosciences indicates that inhibitory circuits may play a critical role in the regulation of neural activity and plasticity (Vogels et al., 2011; Barron et al., 2016). Here, we demonstrate that inhibitory plasticity or SFA could be one of the key mechanisms that allow the sequential reactivation of memories observed during sleep and resting states (Wilson and McNaughton, 1994; Peyrache et al., 2009). A property of associative networks highlighted by our approach is that subtle changes in self-inhibition can drive substantial shifts in network dynamics without disrupting the fundamental structure of stored attractors. Inhibition, therefore, allows context-dependent activity of the network as observed in experimental setups (Kuchibhotla et al., 2017). Biological neural circuits might, therefore, employ similar mechanisms to navigate complex, correlated, memory spaces. Our results indicate that larger networks experience fewer spurious state visits, suggesting improved reliability with scale. However, understanding how these dynamics extend to networks of biologically relevant sizes would require further investigation.

Traditional associative memory models typically suffer from decreased capacity when storing correlated patterns (Amit et al., 1985b). However, our findings revealed the rather unexpected feature that moderate correlation levels actually improve pattern retrieval in the context of autonomous retrieval. We argue how this result may arise from the way that correlated structures influence the geometry of the attractor basins and the ensuing flow toward them. A more thorough theoretical analysis of this phenomenon could provide information on the factors that influence the robustness of recovery dynamics in both artificial and biological systems.

5 Conclusion

In this work, we demonstrate how adaptation can enable autonomous exploration of attractor landscapes in continuous Hopfield networks. Our key finding reveals that, under a critical load, inhibitory plasticity allows networks to systematically retrieve the entire set of stored memories, even for highly correlated sets. This property allowed us to propose and test an algorithmic scheme for continuous learning leveraging the ability of gradient descent to store correlated patterns. The capacity for self-directed pattern exploration, emerging from inhibitory modulation, offers insights for both biological memory consolidation and neuromorphic computing.

References

• 1

  AmitD. J.GutfreundH.SompolinskyH. (1985a). Storing infinite numbers of patterns in a spin-glass model of neural networks. Phys. Rev. Lett. 55, 1530–1533. 10.1103/PhysRevLett.55.1530

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 2

  AmitD. J.GutfreundH.SompolinskyH. (1985b). Spin-glass models of neural networks. Phys. Rev. A. 32, 1007–1018. 10.1103/PhysRevA.32.1007

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 3

  BarronH. C.VogelsT. P.EmirU. E.MakinT. R.O'SheaJ.ClareS.et al. (2016). Unmasking latent inhibitory connections in human cortex to reveal dormant cortical memories. Neuron90, 191–203. 10.1016/j.neuron.2016.02.031

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 4

  D'amourJ.FroemkeR. (2015). Inhibitory and excitatory spike-timing-dependent plasticity in the auditory cortex. Neuron86, 514–528. 10.1016/j.neuron.2015.03.014

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 5

  DiederichS.OpperM. (1987). Learning of correlated patterns in spin-glass networks by local learning rules. Phys. Rev. Lett. 58, 949–952. 10.1103/PhysRevLett.58.949

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 6

  FauthM. J.Van RossumM. C. (2019). Self-organized reactivation maintains and reinforces memories despite synaptic turnover. Elife8:e43717. 10.7554/eLife.43717

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 7

  FontanariJ. F.TheumannW. K. (1990). On the storage of correlated patterns in Hopfield's model. Journal de Physique. 51, 375–386. 10.1051/jphys:01990005105037500

  • CrossRef
  • Google Scholar
  • View reference in article

• 8

  HaG. E.CheongE. (2017). Spike frequency adaptation in neurons of the central nervous system. Exp Neurobiol. 26, 179–185. 10.5607/en.2017.26.4.179

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 9

  HaxbyJ. V.GobbiniM. I.FureyM. L.IshaiA.SchoutenJ. L.PietriniP.et al. (2001). Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science293, 2425–2430. 10.1126/science.1063736

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 10

  HopfieldJ. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proc. Nat. Acad. Sci. 79, 2554–2558. 10.1073/pnas.79.8.2554

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 11

  HopfieldJ. J. (1984). Neurons with graded response have collective computational properties like those of two-state neurons. Proc. Nat. Acad. Sci. 81, 3088–3092. 10.1073/pnas.81.10.3088

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 12

  KirkpatrickJ.PascanuR.RabinowitzN.VenessJ.DesjardinsG.RusuA. A.et al. (2017). Overcoming catastrophic forgetting in neural networks. Proc. Nat. Acad. Sci. 114, 3521–3526. 10.1073/pnas.1611835114

  • CrossRef
  • Google Scholar
  • View reference in article

• 13

  KodangattilJ. N.DacherM.AuthementM. E.NugentF. S. (2013). Spike timing-dependent plasticity at GABAergic synapses in the ventral tegmental area. J. Physiol. 591, 4699–4710. 10.1113/jphysiol.2013.257873

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 14

  KothapalliV. (2023). Neural collapse: a review on modelling principles and generalization. arXiv [preprint] arXiv:2206.04041. 10.48550/arXiv.2206.04041

  • CrossRef
  • Google Scholar
  • View reference in article

• 15

  KriegeskorteN. (2008). Representational similarity analysis and connecting the branches of systems neuroscience. Front. Syst. Neurosci. 2:2008. 10.3389/neuro.06.004.2008

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 16

  KuchibhotlaK. V.GillJ. V.LindsayG. W.PapadoyannisE. S.FieldR. E. (2017). Parallel processing by cortical inhibition enables context-dependent behavior. Nat. Neurosci. 20, 62–71. 10.1038/nn.4436

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 17

  LeCunY.CortesC.BurgesJ. C. C. (1998). The MNIST Database of Handwritten Digits. Available online at: http://yann.lecun.com/exdb/mnist/

  • Google Scholar
  • View reference in article

• 18

  LouieK.WilsonM. A. (2001). Temporally structured replay of awake hippocampal ensemble activity during rapid eye movement sleep. Neuron29, 145–156. 10.1016/S0896-6273(01)00186-6

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 19

  McCallumS. (1998). Catastrophic forgetting and the pseudorehearsal solution in hopfield-type networks. Conn. Sci. 10, 121–35. 10.1080/095400998116530

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 20

  McCloskeyM.CohenN. J. (1989). Catastrophic interference in connectionist networks: the sequential learning problem. In: Psychology of Learning and Motivation. London: Elsevier (1989). p. 109–165.

  • Google Scholar
  • View reference in article

• 21

  PeronS.GabbianiF. (2009). Spike frequency adaptation mediates looming stimulus selectivity in a collision-detecting neuron. Nat. Neurosci. 12, 318–326. 10.1038/nn.2259

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 22

  PeyracheA.KhamassiM.BenchenaneK.WienerS. I.BattagliaF. P. (2009). Replay of rule-learning related neural patterns in the prefrontal cortex during sleep. Nat. Neurosci. 12, 919–926. 10.1038/nn.2337

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 23

  QuirogaR. Q.ReddyL.KreimanG.KochC.FriedI. (2005). Invariant visual representation by single neurons in the human brain. Nature435, 1102–1107. 10.1038/nature03687

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 24

  RobinsA. (1995). Catastrophic forgetting, rehearsal and pseudorehearsal. Conn. Sci. 7, 123–146. 10.1080/09540099550039318

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 25

  ShenY.DasguptaS.NavlakhaS. (2023). Reducing catastrophic forgetting with associative learning: a lesson from fruit flies. Neural Comput. 35, 1797–1819. 10.1162/neco_a_01615

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 26

  TononiG.CirelliC. (2014). Sleep and the price of plasticity: from synaptic and cellular homeostasis to memory consolidation and integration. Neuron81, 12–34. 10.1016/j.neuron.2013.12.025

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 27

  VogelsT. P.SprekelerH.ZenkeF.ClopathC.GerstnerW. (2011). Inhibitory plasticity balances excitation and inhibition in sensory pathways and memory networks. Science334, 1569–1573. 10.1126/science.1211095

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 28

  WilsonM. A.McNaughtonB. L. (1994). Reactivation of hippocampal ensemble memories during sleep. Science265, 676–679. 10.1126/science.8036517

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article