Hierarchy and dynamics of neural networks
- 1 School of Computing Science, Newcastle University, Newcastle upon Tyne, UK
- 2 Institute of Neuroscience, Newcastle University, Newcastle upon Tyne , UK
- 3 Department of Brain and Cognitive Sciences, Seoul National University, Seoul, South Korea
- 4 School of Engineering and Science, Jacobs University Bremen, Bremen, Germany
- 5 Department of Health Sciences, Boston University, Boston, USA
- 6 Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, Nijmegen, Netherlands
Hierarchy is a central feature in the organization of complex biological systems and particularly the structure and function of neural networks. While other aspects of brain connectivity such as regional specialization, modularity, or motif composition have already been discussed extensively (Sporns et al., 2004; Bullmore and Sporns, 2009), no comprehensive analysis has been presented so far on the role of hierarchy and its connection to brain dynamics. Nonetheless, recent discussions among many of our colleagues have shown an increasing interest in the subject of hierarchy. This topic is an emerging key question in neuroscience, as well as generally in the field of network science, due to its links with concepts of control, efficiency, and development across scales (Ravasz and Barabasi, 2003; Barthélemy et al., 2004; Breakspear and Stam, 2005; Zhou et al., 2006; Dehmer et al., 2008).
“Hierarchy” may be understood in several different ways, and can apply to topological, spatial, temporal as well as functional properties of neural networks. The papers of this Special Topic on hierarchy and dynamics reflect this conceptual diversity.
One interpretation of hierarchy is that of a processing sequence. For example, the popular organizational scheme of the primate visual system (Felleman and van Essen, 1991) implies a sequential ordering of visual cortical areas from the visual sensory periphery to “higher-level” areas involved in abstract aspects of vision. This hierarchical concept is formalized by Krumnack, et al. (“Criteria for optimizing cortical hierarchies with continuous ranges”) and expanded from a recent paper by Reid et al. (2009). The authors re-analyze the anatomical constraints for the hierarchical sorting of visual areas, using linear optimization and mixed integer programming, and demonstrate that there are multiple optimal solutions for visual hierarchies, as well as several alternative definitions of optimality. For instance, optimal hierarchies can be based on minimizing the number of violated constraints, or minimizing the maximal size of a constraint violation (cf. Hilgetag et al., 2000), broadening the perspective for the interpretation of the anatomical data.
Another widely used definition of hierarchy is that of a repeated encapsulation of smaller elements in larger ones (Kaiser et al., 2007a; Robinson et al., 2009), an organization which may also be characterized as recursive or fractal (Sporns, 2006). In that sense, neural networks show a self-similar hierarchical organization across a wide range of metric or non-metric scales. These scales may be spatial, ranging from the lobes of the brain to cortical mini-columns; temporal, stretching from plasticity and learning processes taking days and longer to neuronal firing at the millisecond scale; or topological, containing small functional elements such as “canonical circuits” (Douglas and Martin, 2004) in larger modules such as the “visual cortex.”
The Special Topic contains several examples of such encapsulated hierarchies.
For instance, Meunier, et al. (“Hierarchical modularity in human brain functional networks”) present techniques for the rapid detection of a hierarchy of encapsulated modules in resting-state fMRI data. They analyzed networks composed of 1,800 regional nodes, extracted from neuroimaging data for 18 human subjects, and found a good degree of similarity between the network hierarchies for different brains. Moreover, out of five modules at the highest level, the occipital modules demonstrated less sub-modular organization than modules comprising regions of multimodal association cortex. Connector nodes and hubs, with a key role in inter-modular connectivity, were also concentrated in cortical association areas. The study demonstrates the feasibility of extracting large-scale hierarchical networks from experimental imaging data, and prepares the ground for characterizing brain function by advanced network analyses.
Modules in hierarchical networks may be overlapping, rather than be cleanly delineated. Moreover, the individual nodes may differ by the topological “reach” that they have across the network. While most nodes have relatively few connections, some regions (such as amygdala and hippocampus in the cat, or the lateral intra-parietal area (LIP) and area 7 in the macaque brain) are connected to many nodes of the network and thus form hubs (Kaiser et al., 2007b). Such hubs can be further distinguished into provincial (intra-modular) hubs or connector (inter-modular) hubs (Sporns et al., 2007). Zamora-López, Zhou, and Kurths (“Cortical hubs form a module for multisensory integration on top of the hierarchy of cortical networks”) expand such previous approaches, and identify a new element in the cat cortical connectivity network, a hub module, which consists of network nodes that possess many connections with the rest of the network as well as each other. This set of nodes forms a topologically central module of the cortex that appears to be essential for integrating multisensory information (Figure 1).
Figure 1. The cartoon illustrates two important features of hierarchical brain networks: modules and hubs. Adapted from Zamora-López et al. (2010).
Several articles in the Special Topic explore the dynamic implications of hierarchical modular network architectures. Kaiser and Hilgetag (“Optimal hierarchical modular topologies for producing limited sustained activation of neural networks”) investigate the influence of the number of hierarchical levels (scales), as well as sub-modules at each level (granularity), on the spreading of activity in hierarchical modular networks of different sizes, using a minimal dynamic node model. In particular, they characterize the conditions leading to the biologically relevant case of limited sustained activity in which activity persists between the extremes of dying out or activating the whole network (Kaiser et al., 2007a). For different network sizes, limited sustained activity is best supported when the average number of connections per node remains similar, and the number of hierarchical levels or modules per level increases. This observation indicates that dynamic constraints may contribute to the evolution of network complexity in brain architecture.
In their work, Jarvis et al. (“Extending stability through hierarchical clusters in Echo State Networks”) also find that increased intricacy of network structure aids network dynamics. They demonstrate that the stability of Echo State Networks (Jaeger and Haas, 2004) is potentially enhanced, as indicated by the range of spectral radius values, when the networks are structured in a multi-modular, hierarchical way. The more clearly the ESNs are structured, the larger the range of spectral radius values, while increasing inter-cluster connectivity decreases the maximal spectral radius. The finding suggests that insights into the organization of biological networks also have the potential to improve the functioning of networks for technical applications.
An important feature of neural systems is the occurrence of sudden changes in their dynamics (a drastic example of such phase transitions is the shift from normal brain activity to epileptic seizures). In a previous paper, Müller-Linow et al. (2008) observed that two different dynamic behaviors may emerge from hierarchical networks: waves propagating from central nodes and module-based synchronization. In the present issue, Hütt and Lesne (“Interplay between topology and dynamics in excitation patterns on hierarchical graphs”) analyze more formally how excitable systems can switch from one of such states to the other. In addition to a mean-field model simulation, a formalism is introduced in which excitation waves are described as avalanches.
Shifting the perspective from network dynamics to network function, the flow of information in hierarchical networks is used in the brain for the processing of external or internal signals. Rodrigues and Costa (“Signal propagation in cortical networks: a digital signal processing approach”) show that signal flow among network nodes can be characterized by a finite impulse response (FIR) filter. With such an approach, filters underlying the cat and macaque cortical organization are found to be low-pass, allowing signal processing to be summarized through respective cut-off frequencies. Furthermore, filtering intensity varies between network modules, and regions involved in object recognition tend to present the highest cut-off frequencies for both the cat and macaque networks.
Hierarchies can also be seen in temporal aspects of brain activity. Natural stimuli, such as speech, possess features at different temporal scales. Therefore, models of speech recognition should be able to represent slowly changing neuronal states that encode trajectories of faster signals. Kiebel et al. (“Perception and hierarchical dynamics”) present a mathematical approach that assumes that sensory input is generated by a hierarchy of attractors in a dynamic system. Future applications of this approach might emerge from modeling perception as non-autonomous recognition dynamics enslaved by autonomous hierarchical dynamics in the sensorium.
The papers presented here offer an exciting glimpse into future directions of the field of hierarchical neural networks. But they also demonstrate that we still need a better understanding of the different kinds of network hierarchies, paralleled by the development of suitable analysis techniques. Most importantly, an improved understanding is required of how the different aspects of topological, spatial, temporal, and functional hierarchy in the brain are related to each other.
References
Barthélemy, M., Barrat, A., Pastor-Satorras, R., and Vespignani, A. (2004). Velocity and hierarchical spread of epidemic outbreaks in scale-free networks. Phys. Rev. Lett. 92, 178701.
Breakspear, M., and Stam, C. J. (2005). Dynamics of a neural system with a multiscale architecture. Philos. Trans. R. Soc. Lond. B Biol. Sci. 360, 1051–1074.
Bullmore, E., and Sporns, O. (2009). Complex brain networks: graph theoretical analysis of structural and functional systems. Nat. Rev. Neurosci. 10, 186–198.
Dehmer, M., Borgert, S., and Emmert-Streib, F. (2008). Entropy bounds for hierarchical molecular networks. PLoS One 3, e3079. doi:10.1371/journal.pone.0003079.
Douglas, R. J., and Martin, K. A. C. (2004). Neuronal circuits of the neocortex. Ann. Rev. Neurosci. 27, 419–451.
Felleman, D. J., and van Essen, D. C. (1991). Distributed hierarchical processing in the primate cerebral cortex. Cereb. Cortex 1, 1–47.
Hilgetag, C. C., O’Neill, M. A., and Young, M. P. (2000). Hierarchical organization of macaque and cat cortical sensory systems explored with a novel network processor. Philos. Trans. R. Soc. Lond. Ser. B 355, 71–89.
Hütt, M.-T., and Lesne, A. (2009). Interplay between topology and dynamics in excitation patterns on hierarchical graphs. Front. Neuroinform. 3:28. doi: 10.3389/neuro.11.028.2009.
Jaeger, H., and Haas, H. (2004). Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304, 78–80.
Jarvis, S., Rotter,S., and Egert, U. (2010). Extending stability through hierarchical clusters in Echo State Networks. Front. Neuroinform. 4:11. doi: 10.3389/fninf.2010.00011.
Kaiser, M., Görner, M., and Hilgetag, C. C. (2007a). Functional criticality in clustered networks without inhibition. New J. Phys. 9, 110.
Kaiser, M., Martin, R., Andras, P., and Young, M. P. (2007b). Simulation of robustness against lesions of cortical networks. Eur. J. Neurosci. 25, 3185–3192.
Kaiser, M., and Hilgetag, C. C. (2010). Optimal hierarchical modular topologies for producing limited sustained activation of neural networks. Front. Neuroinform. 4:8. doi: 10.3389/fninf.2010.00008.
Kiebel, S. J., Daunizeau, J., and Friston, K. J. (2009). Perception and hierarchical dynamics. Front. Neuroinform. 3:20. doi: 10.3389/neuro.11.020.2009.
Krumnack, A., Reid, A. T., Wanke, E., Bezgin, G., and Kötter, R. (2010). Criteria for optimizing cortical hierarchies with continuous ranges. Front. Neuroinform. 4:7. doi: 10.3389/fninf.2010.00007.
Meunier, D., Lambiotte, R., Fornito, A., Ersche, K. D., and Bullmore, E. T. (2009). Hierarchical modularity in human brain functional networks. Front. Neuroinform. 3:37. doi: 10.3389/neuro.11.037.2009.
Müller-Linow, M., Hilgetag, C. C., and Hutt, M. T. (2008). Organization of excitable dynamics in hierarchical biological networks. PLoS Comput. Biol. 4, e1000190. doi:10.1371/journal.pcbi.1000190.
Ravasz, E., and Barabasi, A. L. (2003). Hierarchical organization in complex networks. Phys. Rev. E 67, 026112.
Reid, A. T., Krumnack, A., Wanke, E., and Kotter, R. (2009). Optimization of cortical hierarchies with continuous scales and ranges. Neuroimage 47, 611–617.
Robinson, P. A., Henderson, J. A., Matar, E., Riley, P., and Gray, R. T. (2009). Dynamical reconnection and stability constraints on cortical network architecture. Phys. Rev. Lett. 103, 108104.
Rodrigues, F. A., and da Fontoura Costa, L. (2009). Signal propagation in cortical networks: a digital signal processing approach. Front. Neuroinform. 3:24. doi: 10.3389/neuro.11.024.2009.
Sporns, O. (2006). Small-world connectivity, motif composition, and complexity of fractal neuronal connections. Biosystems 85, 55–64.
Sporns, O., Chialvo, D. R., Kaiser, M., and Hilgetag, C. C. (2004). Organization, development and function of complex brain networks. Trends Cogn. Sci. 8, 418–425.
Sporns, O., Honey, C. J., and Kötter, R. (2007). Identification and classification of hubs in brain networks. PLoS One 2, e1049. doi:10.1371/journal.pone.0001049.
Zamora-López, G., Zhou, C., and Kurths, J. (2010). Cortical hubs form a module for multisensory integration on top of the hierarchy of cortical networks. Front. Neuroinform. 4:1. doi: 10.3389/neuro.11.001.2010.
Citation: Kaiser M, Hilgetag CC and Kötter R (2010) Hierarchy and dynamics of neural networks. Front. Neuroinform. 4:112. doi: 10.3389/fninf.2010.00112
Received: 04 August 2010;
Accepted: 05 August 2010;
Published online: 23 August 2010.
Copyright: © 2010 Kaiser, Hilgetag and Kötter. This is an open-access article subject to an exclusive license agreement between the authors and the Frontiers Research Foundation, which permits unrestricted use, distribution, and reproduction in any medium, provided the original authors and source are credited.
*Correspondence: m.kaiser@ncl.ac.uk