Innate neural assemblies for Lego memory
- Blue Brain Project, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
A commentary on
A synaptic organizing principle for cortical neuronal groups
by Perin, R., Berger, T. K., and Markram, H. (2011). Proc. Natl. Acad. Sci. U.S.A. 108, 5419–5424.
In a recent study Perin et al. (2011), we used a 12-patch-clamp system to search for evidence of Hebbian assemblies at the most elementary level of the cortex – synaptic clustering of neurons. We found these elusive assemblies, but were surprised to discover that their connectivity and synaptic weights are highly predictable and constrained by a simple common neighbor rule. The data suggests, furthermore, that their basic features are similar for all animals belonging to the same species. These findings imply that experience cannot easily mold the synaptic connections of these assemblies, which are thus less than optimal for memory storage. We speculate that they serve as innate, Lego-like building blocks of knowledge for perception and that the acquisition of memories involves the combination of these building blocks into complex constructs. This Lego theory of memory may explain why animals share common perceptual mechanism, while memories are clearly individual.
In the centuries old debate between nature and nurture, defenders of nurture have portrayed the mind as a blank slate without any innate ideas, whose content is entirely formed by experience (Locke, 1690). Hebb (1949) proposed three postulates that translated these philosophical and psychological concepts into the language of neuroscience. First, learning depends on the strengthening of synaptic connections between neurons when they fire together; second, the operation of this learning rule leads to the formation of assemblies or clusters of synaptically connected neurons whose internal connections are molded by experience to form a percept; third, trains of percepts or thoughts are the result of activation of interlinked assemblies in sequences that are also molded by experience. These ideas have profoundly influenced philosophy, psychology and neuroscience for more than 60 years. As a consequence artificial neural networks have been constructed on the assumption that the initial network is a clean slate.
Functional neuronal assemblies have been reported for decades, but direct evidence of clusters of synaptically connected neurons, beyond what would be expected by simply considering topographic mapping and the position and anatomy of neurons, has been missing. In the work reported in our paper, we performed 270 separate multi-patch experiments, taking repeated measurements from 3446 pairs of layer 5 pyramidal neurons from the somatosensory cortex of 14-day-old rats. When we normalized connection probabilities for known hyper-reciprocity and distance dependency relationships, we found a new form of clustering. Connection probability first increased until neurons were about 100 μm apart before decreasing as they became further apart. We then found that the formation of these clusters was governed by a simple rule. Given any pair of neurons the probability that they were connected was directly proportional to the number of their common neighbors (neurons with which they both had a synaptic connection). In other words, we can estimate the probability that two neurons are connected, simply by counting their common neighbors. Still more surprisingly, the mean weights of the synapses in a group of neurons correlated almost perfectly with the number of connections in the group. In other words, even if activity is required for these assemblies to form, there is little room for experience to uniquely mold connectivity and synaptic weights in these assemblies. Since these assemblies will all be similar in topology and synaptic weights, not molded by any specific experience, we consider these to be innate assemblies.
The correlation between connectivity and synaptic weights was positive and synaptic weights saturate when connectivity reaches only around 20% of the maximum connectivity. Interestingly theorists have criticized Hebb’s synaptic learning rule for doing exactly this – saturating synapses. In this view, if synapses saturated, Hebbian assemblies would be unable to store memories in any meaningful way. The Lego assemblies we found correspond to elementary Hebbian assemblies indicating that the notion of storing acquired memories in Hebbian assemblies will need to be radically revised.
We also found that the variance of synaptic weights is lower in dense clusters than in clusters that are less densely connected. This suggests that mean synaptic weights converge toward the same value in all assemblies with the same level of connectivity – regardless of the animal they come from. This is the opposite of what we would expect if neural assemblies were a store of memory shaped by experience. If we use the common neighbor rule to construct an artificial neural network with Hebb-like assemblies, connectivity within these assemblies is very similar to what we observe in brain slices. This is further evidence that experience plays only a minor role in determining synaptic connections and weights within these assemblies. If John Locke’s clean slate theory were correct, there would be no assemblies at first and no saturation of synaptic weights: every neuron in a network would have the same probability of connecting to every other neuron and experience would be free to drive synaptic weights to any value within their dynamic range. When building artificial neural networks, it will now be necessary to first create these innate assemblies and then only apply a learning rule to associate them that must also support, not break, the common neighbor rule.
Each of the assemblies identified in our studies had about half as many connections between assemblies. Unlike connections within the assemblies, these connections had relatively low mean weights and high variance, a condition believed to make them more amendable to bi-directional plasticity (Sjostrom et al., 2001). This kind of arrangement creates an enormous space of possible connection configurations. At the same time, the existence of multiple links between assemblies could help to stabilize acquired memories stored not in individual synaptic weights but in the mean value and the distribution of the weights (Fusi and Abbott, 2007).
The assemblies we found appear to be interlaced within the same space. This architecture could allow all-to-all functional connectivity between assemblies. While all-to-all connectivity between neurons would eliminate memory capacity, all-to-all connectivity between assemblies could vastly increase it. This is because the association between two assemblies could be defined by the number of inter-connected neurons, the total number of synapses in the connections between them (the strength of the relationship), the specific neurons within each assembly involved, the directions of the synaptic connectivity (the topographic mapping between assemblies), and the dynamics of synaptic transmission (frequency filtering) in each of the connections. The number of ways two assemblies with the same number of neurons can associate topographically is vast and if we also consider the strength of connections and their synaptic dynamics it is clear that the number of possible combinations is practically unlimited.
Our study found evidence that innate Lego-like assemblies of a few dozen neurons are composed of smaller sub-assemblies including the three-cell motifs found earlier (Song et al., 2005) and even four-cell motifs. Connections between assemblies may combine them into super-assemblies within a neocortical layer, then in higher order assemblies in a cortical column, even higher order assemblies in a brain region, and finally in the highest possible order assembly represented by the whole brain.
In this theory, acquiring memories is very similar to building with Lego. Each assembly is equivalent to a Lego block holding some piece of elementary innate knowledge about how to process, perceive, and respond to the world. Activity during development is likely important for these assemblies to mature, but any specific experience is unlikely embedded in the connectivity and synaptic weights. Experience on the other hand could uniquely assign each Lego assembly to a specific task/function/percept within the brain and serve to combine assemblies in a hierarchical manner forming a unique construct representing a unique history of experiences. When different blocks come together, they therefore form a unique combination of these innate percepts that represents an individual’s specific knowledge and experience. Activating any single Lego block could activate other associated blocks creating a rapid cascade-like awakening of the entire super-assembly. This could happen in many different ways reflecting the many different facets of the associations between the elementary assemblies/percepts/Lego blocks. In this way memory may also be retrieved by the activation of any number of assemblies in many different brain regions. This theory also implies an intense competition between specific combinations of assemblies or constructs. Inhibition within and between assemblies may act to dynamically favor which constructs dominate thus playing an important role in perception and memory retrieval. This “Lego theory of memory” could explain why fundamental perceptual mechanisms are similar for all members of a species while memories are clearly individual.
Acknowledgments
The referenced study was carried out as part of the Blue Brain Project, where we aimed to first test the clean slate hypothesis before implementing synaptic learning algorithms in models of neural microcircuits. Rodrigo Perin built the 12-patch system and carried out the 12-patch experiments. Thanks to Ran Levi for ongoing inspiring discussions on combinatorics and algebraic topology. Thanks to Richard Walker for help in editing the manuscript. The study was funded by a grant from the EPFL to the Blue Brain Project.
References
Fusi, S., and Abbott, L. F. (2007). Limits on the memory storage capacity of bounded synapses. Nat. Neurosci. 10, 485–493.
Locke, J. (1690). An Essay Concerning Human Understanding in Four Books. London: Printed for Tho. Basset, and sold by Edw. Mory.
Perin, R., Berger, T. K., and Markram, H. (2011). A synaptic organizing principle for cortical neuronal groups. Proc. Natl. Acad. Sci. U.S.A. 108, 5419–5424.
Sjostrom, P. J., Turrigiano, G. G., and Nelson, S. B. (2001). Rate, timing, and cooperativity jointly determine cortical synaptic plasticity. Neuron 32, 1149–1164.
Citation: Markram H and Perin R (2011) Innate neural assemblies for Lego memory. Front. Neural Circuits 5:6. doi: 10.3389/fncir.2011.00006
Received: 22 March 2011;
Accepted: 19 April 2011;
Published online: 16 May 2011.
Copyright: © 2011 Markram and Perin. This is an open-access article subject to a non-exclusive license between the authors and Frontiers Media SA, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and other Frontiers conditions are complied with.
*Correspondence: e-mail: henry.markram@epfl.ch