Computing the size and number of neuronal clusters in local circuits

The organization of connectivity in neuronal networks is fundamental to understanding the activity and function of neural networks and information processing in the brain. Recent studies show that the neocortex is not only organized in columns and layers but also, within these, into synaptically connected clusters of neurons (Ko et al., 2011; Perin et al., 2011). The recently discovered common neighbor rule, according to which the probability of any two neurons being synaptically connected grows with the number of their common neighbors, is an organizing principle for this local clustering. Here we investigated the theoretical constraints for how the spatial extent of neuronal axonal and dendritic arborization, heretofore described by morphological reach, the density of neurons and the size of the network determine cluster size and numbers within neural networks constructed according to the common neighbor rule. In the formulation we developed, morphological reach, cell density, and network size are sufficient to estimate how many neurons, on average, occur in a cluster and how many clusters exist in a given network. We find that cluster sizes do not grow indefinitely as network parameters increase, but tend to characteristic limiting values.


INTRODUCTION
Network theory is increasingly applied to better understand the principles of how neurons are interconnected and hence to unravel the networking and topologic mysteries of the brain (Honey et al., , 2010Sporns et al., 2007;Sporns, 2010). Much of this work has been focused on macroscopic network principles such as those connecting brain regions. Increased interest in mesoscopic neuroanatomical connectivity also grew considerably over the years (Bohland et al., 2009). Recently, the discovery of clusters of synaptically connected neurons has opened up investigations of the microscopic network principles. The expression of a number of cellular level motifs of synaptic connectivity was first reported by Song et al. (2005). Subsequent publications on the rat (Perin et al., 2011) and mouse (Ko et al., 2011), hint at a theme shared by different species.
A number of cortical networks that have been described are based on cells with large somata, which are easy to distinguish in electrophysiological experiments. The cell of choice in the neocortex has been the large thick tufted layer 5 pyramidal cell (TTL5-PCs) which, thanks to its size and shape, can be isolated and recorded reliably (Markram et al., 1997). In a multielectrode patch-clamp setup it is possible to record from many such cells simultaneously, determine their synaptic connectivity and, after staining, also obtain their morphological properties. In order to simulate different expanses of axonal and dendritic arborization (Figures 1A,B) we developed the concept of "morphological reach," denoted r, a proportionality factor applied to the decay in connection probability as a function of distance. Our reference, the TTL5-PC (Oberlaender et al., 2011), is assigned a morphological reach of 1. Other cell types, in our simulations, are assigned proportional morphological reaches indicative of the extent of their basal dendritic arborizations as measured using the sum of all branch intersections in Sholl Analyses (Figures 1C,D). TTL5 pyramidal cells have broad dendritic arborization (Markram et al., 1997) and form clusters that may constitute elementary units of information processing within a brain region (medium/regional projections) and between brain regions (long/interregional projections). In previous work we identified a rule, the common neighbor rule, governing the connectivity of these groups of neurons (Perin et al., 2011). The common neighbor rule describes a directly proportional relationship between the connection probability between any two neurons and the number of other neurons in the network connected to both neurons in this pair. In this case the term connected indicates neurons that project to as well as neurons receiving synaptic appositions from the neurons in the pair. When applied, the common neighbor rule produces complex clustered networks that lie between completely random networks (Erdös-Reni-type) (Erdõs and Rényi, 1960) and highly clustered networks with hubs (Barabási-type) (Barabási and Albert, 1999) resembling more closely a Watts-Strogatz-type network (Watts and Strogatz, 1998).
A key distinguishing feature between macro-clustering in the brain (between brain regions) and micro-clustering (between neurons of a local microcircuit) is that there are no hubs at the micro (local) network level within a given cell-type. In other words, each neuron of a given type makes contact with and receives contacts from about the same number of neurons of the same type-in fact a pre-requisite for the occurrence of hubs is that the number of boutons and spines per neuron within the population span some orders of magnitude so that the degree of connectivity might vary accordingly. This form of clustering is quite exceptional amongst biological and social networks in that it contains no hubs and has a degree of separation less than 2.
Since the common neighbor based clustering applies similarly across individuals of the same species, the neuronal groups formed would be quite similar. Bootstrap analysis, in which different subsets of the connectivity data originating from different individuals are used, consistently captures the common neighbor rule (Figures A1, A2). This has led to the suggestion that these clusters are in fact innate (genetically preprogrammed to develop) holding innate knowledge of elementary forms of information processing. These posited innate groups likely require some general activity, rather than experience-dependent activity, during development to form.
It may therefore be of special interest to understand the sizes and numbers of such clusters in a brain region. We therefore constructed networks by applying the common neighbor rule and derived the relationships between cluster sizes and numbers as a function of morphological reach, neuronal density, and network size.  (Figure 2A) covering the range of experimentally observed densities in a neocortical column (Meyer et al., 2010). Uniformly random jitter was then added to each position in three dimensions with amplitude equal to the grid spacing between neurons ( Figure 2B). Since point neurons are used no compensation for extreme proximity between neurons is applied. The simulations used network sizes of 512, 1000, 1728, 3375, 4096, and 5832 cells in cubic arrangements. Each of the three dimensions in this model was made circular so that neurons close to one edge were considered to be close to neurons on the opposite edge of the network ( Figure 2C). Edge effects were countered in this way and every neuron was exposed to a similar number of neighbors and could potentially form the same number of connections.

METHODS
In order to investigate the influence of morphology on clustering, the morphological reach of TTL5 pyramidal cells was multiplied by factors of 0.5, 0.667, 0.8, 1, 1.333, and 2 in networks containing 4096 neurons. We defined a neuron's neighbors as those cells that form synaptic connections onto or that receive synaptic connections from the neuron in question. In our model, the small world clustering of layer 5 pyramidal cells is made directly proportional to the number of common neighbors ( Figure 2D) (Perin et al., 2011). To achieve this goal the connections in the initial network were assigned pseudo-randomly according to inter-somatic distance profiles observed experimentally (Perin et al., 2011) (see also Figure A3). While morphological reconstructions indicate a degree of anisotropy in process arborization such observations indicate that a purely distancebased connection probability function still can capture the trend in connectivity patterns. In the simulations where we were interested in simulating different arborizations we linearly extended or shortened such profiles by dividing the actual distance supplied to these profiles by the "morphological reach" factor. The initial connectivity was then modified according to the number of common neighbors. The pair of neurons sharing the maximal number of common neighbors was assigned a probability of connection of 1. Pairs of neurons sharing no common neighbors we assigned a connection probability of 0. Pairs of neurons sharing intermediate numbers of common neighbors were assigned linearly interpolated connection probability values. This process was iterated until the clustering coefficient of the whole network no longer increased (see Figure A4). The observed clusters consist of a few dozen neurons typically distributed 100-150 µm apart. Based on this study (Perin et al., 2011) we decided to investigate the effects of different network sizes and cell densities as well as the impact of different extents of morphological arborization on the clustering properties of networks. Clusters were identified using the affinity propagation algorithm (Frey and Dueck, 2007) at the end of the simulation and providing the number of common neighbors as the similarity measure ( Figure 2E). Similarity was defined exclusively for connected neuron pairs, being set to zero in pairs that were not connected in any way.
Sets of 10 simulations were performed for each parameter combination. The average numbers of clusters and average cluster sizes were plotted and the data points were fitted with surface fits using the MatLab curve fitting tool.

RESULTS
The goal of our modeling was to better understand the relationship between the different network parameters and the network clustering properties. Six basic relationships were investigated by varying three parameters-network size, cell density, and morphological reach-and analyzing the effects on the two clustering properties-the average number of clusters and the average number of cells contained in clusters (cluster size). From these results we also calculated the number of clusters per unit volume.

NETWORK SIZE AND DENSITY
To investigate the impact of network size on the number and size of clusters we constructed networks ranging from 512 to 5832 neurons. These networks were based on TTL5 Pyramidal Cells and ranged in density from 45554 to 455170 cells/mm 3 . Networks containing different numbers of neurons display different clustering properties. A relatively simple linear relationship exists between the network size and the number of clusters. Simply put, larger networks fit more clusters of similar size ( Figure 3A) as can observed in the curve that fits the surface that describes the number of cluster as a function of networks size and density: where NC stands for the number of clusters, s for network size in number of cells and d for network density in cells/mm 3 . A mild decay in the number of clusters occurs as the network density increases. This is because higher densities lead to larger average cluster sizes ( Figure 3B). Cluster size as a function of network size follows a lognormal profile, first increasing then decreasing. This is a particular feature of networks without boundaries. The fitted surface for cluster sizes as a function of network size and density can be expressed as where CS stands for cluster size, s for network size in number of cells and d for network density in cells/mm 3 . Goodness of fit for the resulting surfaces according to Adjusted R-square values were 0.97 for average number of clusters as a function of network density and size, 0.95 for average cluster sizes as a function of network density and size. The number of clusters per unit volume grows with cell density and displays an inverted relationship to cluster size as a function of network size ( Figure 3C).

NETWORK DENSITY AND MORPHOLOGICAL REACH
Morphological reach is a parameter that reflects the extent of neuronal arborization (Figures 1A,B). The more extensive the arborization the more connections a neuron is likely to form ( Figure 1C). We attribute the value of 1 to the morphological reach characteristic of TTL5 Pyramidal Cells. Cells with more restricted arborization display, therefore, morphological reaches corresponding to some fractional value ( Figure 1D). The density of neurons in a local network, in the following simulations involving 4096 neurons, strongly influences the number of clusters formed, particularly very low densities which cause neurons to be more isolated thus yielding numerous clusters involving few neurons (Figures 3D, A5). As neuronal densities increase the number of clusters quickly becomes less dependent on this parameter reaching a plateau before decreasing again due to increasing cluster sizes (Figures 3D,E). The relationship between the number of clusters in a network and the morphological reach of the cells that form this network follows a similar trend, also forming a plateau before further decreasing. where NC stands for the number of clusters, r for morphological reach and d for network density in cells/mm 3 . Morphological reach, similarly to density, has little effect on the number of clusters in a network except in the case of extremely low values, which lead to nearly isolated neurons ( Figure 3D). Within the physiological range of 0.5-1, growth in morphological reach leads to approximately linear and relatively mild increases in cluster sizes. Increases in morphological reach above 1 tend to have larger effects, especially in the case of networks with large cell density but eventually tend to saturation at extreme values of both. Changes in reach and density led to different cluster sizes ( Figure 3E)  where CS stands for cluster size, d for network density in cells/mm 3 and r for morphological reach. It is important to capture variations in morphological reach and cell density since the combination of both is necessary in order to estimate clustering properties. Goodness of fit for the resulting surfaces according to Adjusted R-square values was 0.97 for average number of clusters as a function of network density and morphological reach, 0.98 for average cluster sizes as a function of network density and morphological reach. The number of clusters per unit volume grows approximately linearly with both cell density and morphological reach ( Figure 3F).

DISCUSSION
The most abundant neuronal type in the mammal neocortex, the pyramidal cell can be found in five of the six neocortical layers and with varying spatial reach of their arbors. This cell retains a remarkable stereotypical shape from mice to man and varies mainly in the length of its axonal and dendritic arbors (Peters and Yilmaz, 1993). Thick tufted layer 5 pyramidal cells are among the largest neurons in the brain and provide the output from the neocortex to subcortical structures and distant brain regions to drive behavior (Morishima and Kawaguchi, 2006). We previously found that these pyramidal cells are networked according to a simple synaptic organization rule; neurons that share more common neighbors are also more likely to be connected (Perin et al., 2011). This rule results in stereotypical clusters of synaptically connected neurons and therefore do not seem to be uniquely shaped by experience. In fact, the synaptic weights of the connections between neurons in these clusters are also on average saturated, which is not ideal for acquired memory storage. We therefore propose that forming synaptic connections according to common neighbors is a pre-programmed rule that drives stereotypical neuronal clustering during development. These clusters may therefore express elementary units of innate knowledge that are combined in an experience-dependent manner to form acquired memories while preserving fundamental perceptual mechanisms. It is likely that at its origin, cluster formation Frontiers in Neuroanatomy www.frontiersin.org February 2013 | Volume 7 | Article 1 | 5 may be linked to the fact that sister cells (i.e., cells that originated from the same progenitor radial glial cell) are more likely to develop strong electrical coupling which in turn favors chemical synapse formation (Yu et al., 2009(Yu et al., , 2012. These preferentially connected sets of neurons may associate to form larger ensembles as those involved in orientation selectivity in the visual cortex (Ko et al., 2011). The common neighbor rule also makes it possible to prespecify the underlying synaptic connectivity in a network of such neurons even before learning rules come into play. The influence from common input leading to greater connection probabilities between neurons, which constituted the central mechanism explored in the current work, seems to apply not only for local but also long-range projections (Otsuka and Kawaguchi, 2008;Brown and Hestrin, 2009). The question we focused on, however, is how the structural features of neuronal arborization further influence the local synaptic connectivity organization. The common neighbor rule, which ensures that important features of synaptic organization are respected, was therefore imposed on the connectivity between neurons with different morphological reach and cell densities. From these networks we derived relationships between morphological reach, cell density and number of cells in a network, in order to determine how many neurons make up the clusters and how many clusters can potentially be formed under different conditions. Experimental evidence from juvenile rat somatosensory cortex (Perin et al., 2011) as well as adult visual, (Ohki et al., 2005;Yoshimura et al., 2005) and auditory (Rothschild et al., 2010) and mouse frontal cortices (Otsuka and Kawaguchi, 2011) supports the occurrence of clusters of excitatory cells in Layers II/III through V, overlapped and interlaced in space, rather than tiled next to each other (Figures 2E,F). This not only allows the packing of many clusters of neurons in the same space but also enables monosynaptic connections between clusters to be adjusted and regulate the relationship between clusters.
Future investigations into this topic should also take into consideration the effects of network boundaries and interactions between different cell types, both constituting factors that further influence cluster formation. x(i, j) once for each pair of neurons i and j. (K) Determine presence and direction of connections based on random numbers that were generated. Reciprocal connections occur where x(i, j) < pr(i, j). Non-reciprocal connections from neuron i to neuron j occur where x(i, j) ≥ pr(i, j) and x(i, j) < pr(i, j) + pnr(i, j)/2. Non-reciprocal connections occur from neuron j to neuron i where x(i, j) ≥ pr(i, j) + pnr(i, j)/2 and x(i, j) < pr(i, j) + pnr(i, j). No connections are formed when x(i, j) ≥ pr(i, j) + pnr(i, j). The overall probability of connection p always respects its distance profile and the equation p(i, j) = pr(i, j) + pnr(i, j)/2.

Frontiers in Neuroanatomy
www.frontiersin.org February 2013 | Volume 7 | Article 1 | 9 FIGURE A4 | Examples of the effect of reorganization on clustering coefficient of a network (multiple thin lines) with a fitted exponential curve (thick blue line). Since the computation of the clustering coefficient is time-consuming we used the number of iterations that best approximated 95% percent convergence to the value projected after infinite iterations (from fitted data). The clustering coefficient is the measure of the probability of connection among each neuron's neighbors calculated for each neuron in the network. Figure 3D, viewed from a different angle.

FIGURE A5 | Number of clusters as a function of network density and morphological reach as in
Frontiers in Neuroanatomy www.frontiersin.org February 2013 | Volume 7 | Article 1 | 10