# Information-geometric measures estimate neural interactions during oscillatory brain states

^{1}Department of Neuroscience, Canadian Centre for Behavioural Neuroscience, University of Lethbridge, Lethbridge, AB, Canada^{2}Department of Psychology, Program in Applied Mathematics, University of Arizona, Tucson, AZ, USA

The characterization of functional network structures among multiple neurons is essential to understanding neural information processing. Information geometry (IG), a theory developed for investigating a space of probability distributions has recently been applied to spike-train analysis and has provided robust estimations of neural interactions. Although neural firing in the equilibrium state is often assumed in these studies, in reality, neural activity is non-stationary. The brain exhibits various oscillations depending on cognitive demands or when an animal is asleep. Therefore, the investigation of the IG measures during oscillatory network states is important for testing how the IG method can be applied to real neural data. Using model networks of binary neurons or more realistic spiking neurons, we studied how the single- and pairwise-IG measures were influenced by oscillatory neural activity. Two general oscillatory mechanisms, externally driven oscillations and internally induced oscillations, were considered. In both mechanisms, we found that the single-IG measure was linearly related to the magnitude of the external input, and that the pairwise-IG measure was linearly related to the sum of connection strengths between two neurons. We also observed that the pairwise-IG measure was not dependent on the oscillation frequency. These results are consistent with the previous findings that were obtained under the equilibrium conditions. Therefore, we demonstrate that the IG method provides useful insights into neural interactions under the oscillatory condition that can often be observed in the real brain.

## Introduction

The dynamics of neural interactions have been conjectured to play an important role in neural information processing. One way to investigate the neural interactions is to record multi-neuronal firing activity from a freely behaving animal, and analyze the correlations between individual units. In past decades, electrophysiological studies have significantly been advanced by the use of multi-electrode recording techniques (Wilson and McNaughton, 1993; Chapin et al., 1999; Kudrimoti et al., 1999; Laubach et al., 2000; Hoffman and McNaughton, 2002; Buzsaki, 2004; Tatsuno et al., 2006; Euston et al., 2007; Davidson et al., 2009; Peyrache et al., 2009b; Dragoi and Tonegawa, 2011, 2013). In order to analyze such high-dimensional multi-neuronal datasets, a number of statistical methods have also been developed (Gerstein and Perkel, 1969; Abeles and Gerstein, 1988; Aertsen et al., 1989; Zhang et al., 1998; Panzeri and Schultz, 2001; Grun et al., 2002a,b; Brown et al., 2004; Fellous et al., 2004; Czanner et al., 2005; Shimazaki and Shinomoto, 2007; Amari, 2009; Gilestro et al., 2009; Peyrache et al., 2009a; Shimokawa and Shinomoto, 2009; Lopes-Dos-Santos et al., 2011). Recently, a method based on information geometry (IG) has been applied to the analysis of neural data (Amari and Nagaoka, 2000; Amari, 2001; Nakahara and Amari, 2002; Amari et al., 2003; Tatsuno and Okada, 2004; Eleuteri et al., 2005; Ikeda, 2005; Miura et al., 2006; Nakahara et al., 2006; Gilestro et al., 2009; Tatsuno et al., 2009; Ince et al., 2010; Lovette et al., 2011; Nie and Tatsuno, 2012). It has been demonstrated that IG provides a powerful statistical tool for analyzing spiking data. Some of the advantages of IG approach include the orthogonal decomposition of neural interactions (Amari, 2001; Nakahara and Amari, 2002), and its direct relationship to underlying connections (Tatsuno and Okada, 2004; Tatsuno et al., 2009; Nie and Tatsuno, 2012); the single-IG measure is related to the amount of external inputs and the pairwise-IG measure is related to the amount of direct neural interactions between two neurons.

These IG properties were often investigated under the assumption that the network is in an equilibrium state. However, in the brain, the equilibrium assumption does not hold true. Instead, the brain undergoes a variety of non-equilibrium states such as oscillations. For example, the slow-wave oscillation (~1 Hz) was discovered during non-REM sleep (Steriade et al., 1993; Crunelli and Hughes, 2010), and evidence suggests that it plays an important role in memory consolidation (Huber et al., 2004; Stickgold, 2005; Diekelmann and Born, 2010). The theta (6–10 Hz) rhythm is a prominent coherent oscillation observed in the hippocampus, and its surrounding area during rat spatial navigation (Vanderwolf, 1969; Bland, 1986; Buzsaki, 2002). The theta rhythm has also been observed in various human neocortical areas during the delay period of working memory tasks (Raghavachari et al., 2001; Meltzer et al., 2008). The beta (15–30 Hz) oscillation is conjectured to play a key role in action preparation and inhibitory control in the motor system (Baker et al., 1997). The gamma (30–80 Hz) oscillation has been shown to play a role in the integration of sensory information (Gray et al., 1989; Singer and Gray, 1995). The fast hippocampal sharp wave ripples (100–200 Hz) were also reported during an animal's awake immobility and slow-wave sleep (Buzsaki et al., 1992). Therefore, it is important to investigate if the IG measures can be applied to neural data under oscillatory conditions.

In this study, we investigated how the single- and pairwise-IG measures are influenced by a network oscillation. Under an equilibrium assumption, previous studies have shown that the single- and pairwise-IG measures provide a robust estimation of the magnitude of external input and direct neural interactions (Tatsuno et al., 2009; Nie and Tatsuno, 2012). We also focused on these IG measures in this study because the external inputs and intrinsic neural interactions are the two main factors for characterizing network dynamics. For the oscillation mechanisms, we have considered two representative cases; one is an external driven oscillation where a network is influenced by external oscillatory inputs. The other is an internally induced oscillation where interactions between excitatory and inhibitory neuron populations produce an oscillation. By computer simulations using simple binary model neurons or more biologically plausible spiking neurons, we investigated whether the properties of the IG measures that were established with the equilibrium condition still hold true under oscillatory network states.

In section Methods, we briefly introduce an information-geometric analysis of neural spikes. In section Results, we describe the model and network structure used in the numerical simulation. In section Discussion, the simulation results for both externally driven and internally induced oscillations are described in detail. In section Acknowledgments, we summarize our findings and discuss future directions of research on this topic.

## Methods

### Information-Geometric Method

We briefly introduce the information-geometric method for spiking data analysis (for details see Amari and Nagaoka, 2000). Generally, in an *N*-neuron system, the state of *i*-th(*i* = 1, …, *N*) neuron is represented by a binary random variable *x*_{i}, where *x*_{i} = 1 or 0 representing neuronal firing or silence, respectively. The joint probability distribution of the *N*-neuron system can be described by a fully expanded *N*-th order log-linear model (LLM)

where θ^{(N, N)}_{ij,… m} (1 ≤ *m* ≤ *N*) represents the *m*-neuron interaction and ψ(θ)^{(N, N)} with θ = {θ^{(N, N)}_{i}, θ^{(N, N)}_{ij}, …, θ^{(N, N)}_{12,…N} } is a normalization constant such that ∑ *p*_{x1x2} … *x*_{N} = 1. The first and the second superscripts in θ^{(N, N)}_{ij, … m} represent the order of the LLM and the number of neurons in the system. We use θ^{(N, N)}_{i}, θ^{(N, N)}_{ij}, and θ^{(N, N)}_{ij, … m} to describe the single-IG measure, the pairwise-IG measure and the *m*-neuron IG measure with the *N*-th order LLM for a *N*-neuron system, respectively, (Nie and Tatsuno, 2012). The joint probability of *N* neurons is calculated by

where *c*_{x1}*x*_{2},…,*x*_{N} is the count of events (*X*_{1} = *x*_{1}, *X*_{2} = *x*_{2}, …,*X*_{N} = *x*_{N}) that occur.

However, in reality, it is difficult to calculate the statistical information from all neurons in a large network. Therefore, the partially expanded LLM is often used for the estimation of neuronal interactions. The partially expanded *k*-th order LLM in an *N*-neuron network is given by

where **θ** = {θ^{(k, N)}_{i}, θ^{(k, N)}_{ij}, …, θ^{(k, N)}_{12,… k}}. The first few terms of **θ** and normalization factor are given as follows:

where ‘* … *′, represents the marginalization over the (*N* − *k*) neurons.

The single-IG measure θ^{(k, N)}_{i} and the pairwise-IG measure θ^{(k, N)}_{ij} are the two main focuses in this study because θ^{(k, N)}_{i} is related to the amount of external inputs and θ^{(k, N)}_{ij} is related to the amount of direct neural interactions between two neurons (Tatsuno and Okada, 2004; Tatsuno et al., 2009; Nie and Tatsuno, 2012). Using a network of simple binary neurons, and the assumption of an equilibrium state, the previous study has shown that the single-IG measure θ^{(2, N)}_{i} and the pairwise-IG measure θ^{(2, N)}_{ij} with the 2nd-order LLM are related to the network parameters as

where *h*_{i} represents the magnitude of constant external input to a neuron *i*, and *J*_{ij} (*J*_{ji}) is the connection weight from a neuron *j* to *i* (from a neuron *i* to *j*), respectively, (Tatsuno et al., 2009). If a network receives correlated inputs, the relationship for θ^{(2, N)}_{ij} in Equation 5 does not hold true anymore. However, we have also shown that θ^{(k, N)}_{ij} with the higher k-th order LLM provides a better estimation of neural interactions (Nie and Tatsuno, 2012). For example, θ^{(4, N)}_{ij} was shown to have the relationship

within approximately a 10% error if the number of neurons *N* is ~10^{3} or larger; a typical size of network in a cortical column (Urban et al., 2001). We have also confirmed that the relationship θ^{(4, N)}_{i} α2*h*_{i} holds true within approximately a 10% error (unpublished data).

These properties could be useful for the field of neuroscience because the IG measures can estimate the changes of underlying network parameters (*h*_{i} and *J*_{ij}) separately, while other correlation measures have not yet been shown to have such a property (Amari, 2009). However, these results were derived under the equilibrium limit, and little is known if the similar relationship holds under the oscillatory condition.

### Neuron Model and Network Structure

#### Neuron model

We investigated the influence of oscillations using a network of simple binary neurons with stochastic dynamics (Ginzburg and Sompolinsky, 1994) and biologically plausible spiking neurons (Izhikevich, 2003). Using simple binary model neurons, we first investigated whether the property of the IG measures that were shown under the equilibrium condition also held true for the oscillatory condition. We then extended our investigation to more realistic spiking neurons.

For a binary model neuron, the transition between the binary states is given by the transition rate *w* as

where τ_{0} is a microscopic characteristic time and *u*_{i} represents the total input to the *i*-th neuron.

is the sigmoidal function in the bounded interval [0, 1] where *m* is a parameter controlling the firing probability of a model neuron.

For a biologically more plausible neuron model, we adopted the Izhikevich model because it is known to be computationally efficient and biologically plausible (Izhikevich, 2003). The Izhikevich model reduced the complex dynamics of the Hodgkin–Huxley (HH) neuronal models to two coupled differential equations as

Here the variable *V*_{i} represents the membrane potential of neuron *i*, and *U*_{i} represents a membrane recovery variable which correlates with the activation of *K*^{+} ionic currents and inactivation of *Na*^{+} [for detail see (Izhikevich, 2003)]. *U*_{i} and *V*_{i} are reset after a spike: if *V*_{i} ≥ 30 mV, then *V _{i}* ←

*c*

_{i},

*U*←

_{i}*U*

_{i}+

*d*

_{i}.

*I*

_{i}represents a total input to neuron

*i*;

*a*

_{i},

*b*

_{i},

*c*

_{i},

*d*

_{i}are dimensionless adjustable parameters which are usually taken as (

*a*

_{i},

*b*

_{i}) = (0.02, 0.2) and (

*c*

_{i},

*d*

_{i}) = (−65, 8) + (15, −6)

*r*

^{2}

_{i}for excitatory neurons, (

*a*

_{i},

*b*

_{i}) = (0.02, 0.25) + (0.08, −0.05

*r*

_{i}) and (

*c*

_{i},

*d*

_{i}) = (−65, 2) for inhibitory neurons.

*r*

_{i}is a uniformly distributed random variable on the interval [0, 1] (Izhikevich, 2003).

#### Network structure

We considered two mechanisms for generating oscillatory network states; one is the oscillation driven by external inputs (Figure 1A), and the other is the oscillation induced by the intrinsic interaction between excitatory and inhibitory neuron populations (Figure 1B). The former mechanism can be a model for hippocampal theta oscillation in which the projections from the medial septum to the hippocampus play a central role (Dragoi et al., 1999). The latter structure where excitatory and inhibitory neuron pools interact is widely observed in cortical areas (Buzsaki and Wang, 2012). It can be a model for cortical oscillations (such as in a gamma-range) that rely on the interplay between excitatory and inhibitory neuron pools.

**Figure 1. A schematic of two mechanisms for generating network oscillations. (A)** Oscillation is generated by an external oscillatory input (externally driven oscillation). A neuron *i* in the network of *N* neurons with recurrent connections *J*_{ij} receives a sinusoidal external input *h*_{i} = *h*_{0} sin(ω*t* + φ_{i}) where *h*_{0}, ω, and φ_{i} represent the amplitude, angular speed, and phase of the sinusoidal input, respectively. **(B)** Oscillation is generated by the interaction between excitatory and inhibitory neuron pools (internally induced oscillation). Excitatory neurons are connected by positive connections *J*_{EE}, inhibitory neurons are connected by negative connections *J*_{II}, inhibitory neurons receive positive connections *J*_{IE} from excitatory neurons, and excitatory neurons receive negative connections *J*_{EI} from inhibitory neurons. In addition, excitatory and inhibitory neurons receive external constant input *H*^{E}_{i} and *H*^{I}_{k}, respectively.

In the first scenario (externally driven oscillation, Figure 1A), a sinusoidal external input *h*_{i}(*t*) = *h*_{0} sin(ω*t* + φ_{i}) for the *i*-th neuron was used to generate oscillatory states in a network, where *h*_{0}, ω, and *upvarphi*_{i} represent the amplitude, angular speed, and phase of sinusoidal signals, respectively. Note that *h*_{0} and ω are common to all neurons, but φ_{i} can be different for individual neurons. The explicit expression of an input signal allows one to produce different network oscillations systematically. For the binary neuron model, the total input to the *i*-th neuron is written as,

where *J*_{ij} represents a connection weight from the *j*-th neuron to the *i*-th neuron. The neuronal state *x*_{i}(*t*) was then updated following the transition rate *w* in Equation 7. Note that model neurons are identical, whether they are excitatory or inhibitory.

For the Izhikevich model in the first scenario, we considered a population of excitatory neurons. Although it has been demonstrated that a network of excitatory neurons can synchronize, a network of Izhikevich neurons that were connected in this particular way cannot produce an intrinsic oscillation (Mirollo and Strogatz, 1990; Hansel et al., 1995). This allows us to investigate the relationship between the IG measures and an externally driven oscillation in a more realistic setting. The total input to an Izhikevich neuron *i* is given by,

where *J*^{EE} represent positive weights between excitatory neurons and *s*^{E}_{j} = δ(*t* − *t*^{f}_{j}) is the delta function representing the existence of a spike emitted from an excitatory neuron *j* at time *t*^{f}_{j}. The neuronal state was then updated by Equation 9 and the associated reset dynamics. In the numerical simulation, we used a small bin width of 1 ms so that it would contain no more than one spike.

Figures 2A–C show example spike trains and multi-unit activity of binary model neurons driven by external sinusoidal inputs of 1, 6, and 100 Hz oscillations, respectively. Izhikevich neurons also exhibited very similar activity (data not shown). It can be clearly seen that neural activity is entrained to external input. Using these two models, we investigated how the IG measures are affected by externally driven oscillations.

**Figure 2. Average firing probability and raster plot of representative oscillatory activity. (A)** Average firing probability of 1000 binary neurons (top panel) and a raster plot of 100 randomly sampled neurons (bottom panel) over 1000 ms under the influence of an external sinusoidal input of 1 Hz (slow oscillation) are shown. **(B)** Average firing probability of 1000 binary neurons (top panel) and a raster plot of 100 randomly sampled neurons (bottom panel) for an external sinusoidal input of 6 Hz (theta oscillation) are shown. **(C)** Average firing probability of 1000 binary neurons (top panel) and a raster plot of 100 randomly sampled neurons (bottom panel) for an external sinusoidal input of 100 Hz (ripple oscillation) are shown. **(D)** 1000 ms of average firing probability of 1250 Izhikevich neurons (top panel) and a raster plot (bottom panel) with approximately a 6-Hz oscillation are shown. In the top panel, spikes from an excitatory neuron and an inhibitory neuron are represented by a black dot and a gray dot, respectively. **(E)** 1000 ms of average firing probability of 1250 Izhikevich neurons (top panel) and a raster plot (bottom panel) with approximately a 40-Hz oscillation are shown.

In the second scenario (internally induced oscillation, Figure 1B), interaction between excitatory and inhibitory neuron pools generates an oscillation. For the binary neuron model, the total input to the *i*-th excitatory neuron and the *k*-th inhibitory neuron are written as,

where *J*^{II}, *J*^{IE}, and *J*^{EI} represent negative weights between inhibitory neurons, positive weights from excitatory neurons to inhibitory neurons, and negative weights from inhibitory neurons to excitatory neurons, respectively. The excitatory and inhibitory neurons receive constant external inputs *H*^{E}_{i} and *H*^{I}_{i}, and maintain sustained oscillatory activity. The neuronal state was updated following the transition rate *w* in Equation 7.

For the Izhikevich model in the second scenario, a similar relationship exists for the total inputs for the *i*-th excitatory neuron and the *k*-th inhibitory neuron,

The neuronal state was then updated following Equation 9 and the associated reset dynamics. Figures 2D,E provide examples of spike trains and multi-unit activity of Izhikevich neurons that exhibited ~6 and 40 Hz oscillations, respectively. Binary neurons also exhibited a very similar activity (data not shown). Neural activity was synchronized, but the degree of entrainment was weaker than the externally driven mechanisms. Using these two models, we investigated how the IG measures were influenced by the internally induced oscillation.

## Results

### Externally Driven Oscillation

We investigated the relationship between the IG measures, θ^{(4, N)}_{i} and θ^{(4, N)}_{ij}, and the connection weights, (*J*_{ij} + *J*_{ji}), using a network of 1000 binary neurons and 1000 Izhikevich neurons. We focused on the IG measures with 4th-order LLM because they have been shown to estimate connection weights (Nie and Tatsuno, 2012) and external inputs (unpublished data) within a 10% error under an equilibrium assumption. In the simulation, we kept the amplitude of external input at a value such that the overall network firing probability is relatively low (*p*_{xi} ~ 0.1). Connection weights were set to the order of 1/*N* to prevent saturation of neuronal activity. For a binary neuron model, we used *J*_{ij} = 1/*N* + ε_{ij} where ε_{ij} is a random variable from a normal distribution *N*(0, 1/*N*) with a mean of 0 and the standard deviation of 1/$\sqrt{{N}}$. For the Izhikevich model, we restricted the simulations to a pool of only excitatory neurons to ensure that no internally induced oscillation occurred. The connection weights were assigned as *J*^{EE}_{Ij} = 1/*N* + ε′_{ij} where ε′_{ij} is a random variable following uniform distribution *U*(0, 1/*N*) within the interval of [0, 1/*N*]. θ ^{(4, N)}_{i} and θ^{(4, N)}_{ij} were calculated by 10^{6} updates of the network. With the time resolution of 1 ms, the simulation corresponds to ~15 min of recordings. To obtain the mean and variances of the IG measures, we performed 100 independent simulations. Error bars in the figure represent the standard error of mean (SEM).

We investigated the oscillation frequencies that have often been observed in the brain; slow oscillation (~1 Hz), theta oscillation (6–10 Hz), and ripple oscillation (100–200 Hz). The left column of (Figures 3A,D,G) shows the results for the slow oscillation. The multi-unit activity of the binary neurons exhibits a slow oscillation of the frequency of external input (Figure 3A) and the neurons were entrained to this frequency (Figure 2A). The spiking activity of Izhikevich neurons also showed almost identical activity (data not shown). To investigate how θ^{(4, N)}_{i} and θ^{(4, N)}_{ij} are related to the change of connection weights, we systematically modified the sum of connection weights between two neurons (1 and 2) from −9/*N* to 9/*N*. Due to the randomness of the connectivity, focusing the neurons (1 and 2) did not affect the generality. We found that θ ^{(4, N)}_{12} was linearly related to the sum of the connection weights, and that the values of θ^{(4, N)}_{12} for both the binary and Izhikevich models were very close (Figure 3D, black line for a binary model and gray line for the Izhikevich model). On the other hand, θ^{(4, N)}_{1} and θ^{(4, N)}_{2} were independent from the change of synaptic weights (Figure 3G). These results are consistent with the previous findings under the equilibrium assumption; showing that IG measures can also provide useful insights in conditions where the network oscillates. The middle and right columns of Figure 3 show the results for theta oscillations (Figures 3B,E,H) and ripple oscillations (Figures 3C,F,I), respectively. We found that the relationship between the IG measures and connection weights was robust against a different frequency of external inputs. This confirmed that the IG measures can also provide useful information for externally driven theta and ripple oscillations.

**Figure 3. Relationship between the IG measures and the sum of connection weights for an externally driven oscillation. (A)** Average firing probability of 1000 binary neurons with a 1-Hz oscillatory modulation is shown. **(B)** Average firing probability of 1000 binary neurons with a 6-Hz oscillatory modulation is shown. **(C)** Average firing probability of 1000 binary neurons with a 100-Hz oscillatory modulation is shown. **(D)** Relationship between the pairwise IG measure (θ^{(4, 1000)}_{12}) and the sum of connection weights (*J*_{12} + *J*_{21}) under a 1-Hz oscillation. Black and gray lines represent the simulation results by binary neurons and Izhikevich neurons, respectively. **(E)** Relationship between the pairwise IG measure (θ ^{(4, 1000)}_{12}) and the sum of connection weights (*J*_{12} + *J*_{21}) under a 6-Hz oscillation. **(F)** Relationship between the pairwise IG measure (θ^{(4, 1000)}_{12}) and the sum of connection weights (*J*_{12} + *J*_{21}) under a 100-Hz oscillation. **(G)** Relationship between the single IG measure (θ^{(4, 1000)}_{1}) and the sum of connection weights (*J*_{12} + *J*_{21}) under a 1-Hz oscillation. Black and gray lines represent the simulation results by binary neurons and Izhikevich neurons, respectively. **(H)** Relationship between the single IG measure (θ^{(4, 1000)}_{1}) and the sum of connection weights (*J*_{12} + *J*_{21}) under a 6-Hz oscillation. **(I)** Relationship between the single IG measure (θ^{(4, 1000)}_{1}) and the sum of connection weights (*J*_{12} + *J*_{21}) under a 100-Hz oscillation.

To further investigate if the robust property of the IG measures for slow, theta, and ripple oscillations holds true for other frequencies, we varied the frequency over 1–200 Hz, the range that can be typically observed in the brain. We set (*J*_{12} + *J*_{21}) = 2*J*. Figure 4 shows that θ^{(4, N)}_{12} and θ^{(4, N)}_{1} did not depend on oscillation frequencies (Figures 4A,C for a binary model, and Figures 4B,D for Izhikevich model). The results confirmed that the IG measures would be useful for neural data analysis when the brain exhibits a variety of oscillations depending on cognitive demands and the sleep stages.

**Figure 4. Dependency of the IG measures for an oscillation frequency for an external drive oscillation**. The values of the single-IG measure θ^{(4, 1000)}_{1} and the pairwise-IG measure θ^{(4, 1000)}_{12} were calculated for different oscillatory frequencies (1, 5, 6, 10, 20, 50, 100, and 200 Hz). Parameters were set as *J*_{12} + *J*_{21} = 2*J, h*_{0} = 0.05, and φ_{i} = 0. **(A)** The relationship between θ^{(4, 1000)}_{12} and the oscillation frequency by binary model neurons. **(B)** The relationship between θ^{(4, 1000)}_{12} and the oscillation frequency by Izhikevich neurons. **(C)** The relationship between θ^{(4, 1000)}_{1} and the oscillation frequency by binary model neurons. **(D)** The relationship between θ^{(4, 1000)}_{1} and the oscillation frequency by Izhikevich neurons.

The previous analyses (Figures 3, 4) were performed under the zero relative phase difference between two neurons *i* and *j*, namely δφ_{ij} = |φ_{i} − φ_{j}| = 0. This corresponds to the synchronous neural firings that were depicted in Figures 2A–C. Neurons can, however, exhibit phase differences. For instance, sequential neural activity was observed in the natural and anesthetized brain states (Lee and Wilson, 2002; Euston et al., 2007; Luczak et al., 2007; Bermudez Contreras et al., 2013). Therefore, we calculated IG measures with phase differences. Figure 5 shows the results of the 6-Hz simulations in which the phase difference between sinusoidal inputs to the neurons 1 and 2 was set to π/6 (Figures 5A,C) and π/2 (Figures 5B,D). The rest of the neuron pairs have random phases in the range of [0, 2π]. Figures 5A,B show that θ^{(4, 1000)}_{12} is linearly related to the sum of synaptic weights, suggesting that the relationship observed in zero phase difference condition also holds for the non-zero phase difference condition. Similarly, Figures 5C,D show that θ^{(4, 1000)}_{1} does not depend on the connection weights, even when neurons fire with phase differences. By comparing these results with Figures 3E,H where there was no phase difference, we also found that phase difference produced the shift of the actual values of IG measures. This suggests that if the phase relationship drastically changes between the two recording epochs, the values of the IG measures cannot be directly comparable. However, if their difference is not large or if phase difference can be estimated beforehand, we could use the information for adjusting the IG values. We also confirmed that these relationships held true for slow (1 Hz) and ripple (100 Hz) frequencies (data not shown).

**Figure 5. Relationship between the IG measures and the sum of connection weights for non-zero phase differences**. An external oscillation mechanism was used, and the oscillation frequency was set to 6 Hz. **(A)** Relationship between the pairwise IG measure (θ^{(4, 1000)}_{12}) and the sum of connection weights (*J*_{12} + *J*_{21}) for the phase difference of δφ_{12} = π/6. Black and gray curves represent the simulation by binary neurons and Izhikevich neurons, respectively. **(B)** Relationship between the pairwise IG measure θ^{(4, 1000)}_{12}) and the sum of connection weights, (*J*_{12} + *J*_{21}) for the phase difference of δφ_{12} = π /2. **(C)** Relationship between the single-IG measure (θ^{(4, 1000)}_{1}) and the sum of connection weights (*J*_{12} + *J*_{21}) for the phase difference of δφ_{12} = π/6. **(D)** Relationship between the single-IG measure (θ^{(4, 1000)}_{1}) and the sum of connection weights (*J*_{12} + *J*_{21}) for the phase difference of δ φ_{12} = π /2.

So far we have focused on the relationship between the IG measures and the connection weights. Another important parameter is the magnitude of sinusoidal input *h*_{0}. Therefore, we have analyzed how θ^{(4, 1000)}_{1} and θ^{(4, 1000)}_{12} are related to *h*_{0}. Figure 6 shows the result when the external sinusoidal input has a frequency of 6 Hz (theta oscillation). We found that θ^{(4, 1000)}_{12} was nearly independent from the change of *h*_{0} (Figure 6A), but θ^{(4, 1000)}_{1} was almost linearly related to it (Figure 6B). We also confirmed that almost identical relationship holds true for other frequencies such as slow oscillation and ripple oscillation if there is no phase difference. For non-zero phases between neurons, we observed that the IG values were shifted, like the case for the IG values and connection weights, but that the same linear and independent relationship in Figure 6 was sustained. The results under the oscillatory condition are consistent with the previous findings under the equilibrium condition; θ^{(4, N)}_{i} was linearly related to the magnitude of the constant input and that θ^{(4, N)}_{ij} was almost independent from it (Nie and Tatsuno, 2012). The investigation here provides further evidence that θ^{(4, N)}_{i} is useful for the estimation of the magnitude of external input.

**Figure 6. Relationship between the IG measures and the amplitude of an external sinusoidal input for an externally driven oscillation**. Oscillation frequency was set to 6 Hz and phase difference was δφ_{12} = 0. **(A)** Relationship between the pairwise-IG measure, θ ^{(4, 1000)}_{12}, and the amplitude of a sinusoidal input, *h*_{0}. Black and gray lines represent the simulation results by binary neurons and Izhikevich neurons, respectively. **(B)** Relationship between the single-IG measure, θ ^{(4, 1000)}_{1}, and the amplitude of a sinusoidal input, *h*_{0}.

In summary, we investigated how the IG measures were influenced by an externally driven oscillation. Using a simple binary neuron model, and a more realistic Izhikevich model, we found that θ ^{(4, N)}_{ij} had a linear relationship with the sum of the connection weights, and that it was almost independent from the magnitude of a sinusoidal input. In contrast, θ^{(4, N)}_{i} was almost independent from the connection weights, but was linearly related to the magnitude of the sinusoidal input. These properties were not affected by the frequency of the oscillations or the relative phase differences between neurons.

### Internally Induced Oscillation

As another mechanism for generating an oscillatory network behavior, we also investigated the interactions between excitatory and inhibitory neuron pools. We analyzed the network structure in Figure 1B by simple binary model neurons and Izhikevich neurons. Unlike the first oscillation mechanism, where an oscillation frequency and phase differences could be explicitly controlled, it was not easy to generate an oscillation with desired parameters. However, we were able to generate two examples that were often observed in the brain. Figures 7A,B show multi-unit activity corresponding to theta frequency (~8 Hz) and gamma frequency (~40 Hz), respectively. The same examples with a raster plot were also depicted in Figures 2D,E. To avoid saturation in neural activity, we have set the connection weights to the order of 1/*N*. For a theta oscillation, we set the connection as *J*^{EE}_{ij} = *J*_{1} · ϵ′_{ij}, *J*^{IE}_{ij} = 5*J*_{1} · ϵ′_{ij}, *J*^{II}_{ij} = −*J*_{2} · ϵ′_{ij} and *J*^{EI}_{ij} = −*J*_{2} · ϵ′_{ij} where *J*_{1} = 1/*N*_{e}, *J*_{2} = 1/*N _{i}*, and ϵ′

_{ij}is a random variable following a uniform distribution

*U*(0, 1) within the interval [0,1]. For a gamma oscillation, we used

*J*

^{EE}

_{ij}=

*J*

_{1}· ϵ′

_{ij},

*J*

^{IE}

_{ij}= 6

*J*

_{1}· ϵ′

_{ij},

*J*

^{II}

_{ij}= −

*J*

_{2}· ϵ′

_{ij}and

*J*= −2

^{EI}_{ij}*J*

_{2}· ϵ′

_{ij}. The stronger

*J*

^{IE}was necessary to induce oscillation (Adini et al., 1997). The external constant inputs to excitatory and inhibitory neurons were set as

*H*

^{E}

_{i}= 0.05 and

*H*

^{I}

_{i}= 0.02, respectively. The simulation of 10

^{6}update was performed with

*N*

_{e}= 1000 excitatory neurons and

*N*

_{i}= 250 inhibitory neurons. The mean and variance was estimated using 100 independent simulations.

**Figure 7. Relationship between the IG measures and the sum of connection weights for an internally induced oscillation**. Both connections, *J*_{12} and *J*_{21}, were positive for the range of (*J*_{12} + *J*_{21}) ≥ 0 and they were negative for (*J*_{12} + *J*_{21}) < 0. **(A)** Average firing probability of 1250 Izhikevich neurons with approximately a 6-Hz oscillatory oscillation is shown. **(B)** Average firing probability of 1250 Izhikevich neurons with approximately a 40-Hz oscillatory oscillation is shown. **(C)** Relationship between the pairwise-IG measure (θ ^{(4, 1000)}_{12}) and the sum of connection weights (*J*_{12} + *J*_{21}) under a 6-Hz oscillation. Black and gray lines represent the simulation results by binary neurons and Izhikevich neurons, respectively. **(D)** Relationship between the pairwise-IG measure (θ^{(4, 1000)}_{12}) and the sum of connection weights (*J*_{12} + *J*_{21}) under a 6-Hz oscillation. **(E)** Relationship between the single-IG measure (θ^{(4, 1000)}_{1}) and the sum of connection weights (*J*_{12} + *J*_{21}) under a 6-Hz oscillation. **(F)** Relationship between the single-IG measure (θ ^{(4, 1000)}_{1}) and the sum of connection weights (*J*_{12} + *J*_{21}) under a 40-Hz oscillation.

To investigate the relationship between the IG measures and the sum of connection weights, without losing the generality, we modified (*J*_{12} + *J*_{21}) between the neurons (1 and 2) in the range of [−9*J*, 9*J*]. Firstly, we focused on the connections within the excitatory neuron population and within the inhibitory neuron population. In other words, both connections, *J*_{12} and *J*_{21}, were positive for the range of (*J*_{12} + *J*_{21}) ≥ 0 and both were negative for (*J*_{12} + *J*_{21}) < 0. Figures 7C,E show the relationship between θ^{(4, 1250)}_{12} and θ^{(4, 1250)}_{1}, and the sum of connection strengths (*J*_{12} + *J*_{21}) under the theta oscillation. The results clearly show that θ^{(4, 1250)}_{12} is linearly related to the sum of the connection weights and that θ ^{(4, 1250)}_{1} was independent from the modulation of the connection weights. Furthermore, the dependency of the IG measures on the connection weights was continuous in both positive and negative ranges. This suggests that IG measures can be applicable to both positive and negative connections. Figures 7D,F show results for gamma oscillation. Similar results were obtained for both θ^{(4, 1250)}_{12} and θ^{(4, 1250)}_{1}.

Secondly, we investigated the interaction between excitatory and inhibitory neurons. Namely, we selected the neuron 1 from the excitatory neuron pool and the neuron 2 from the inhibitory neuron pool. The sum of connection weights was modified from −9*J* to 9*J*. Figures 8A,B are the same with Figures 7A,B, showing the multi-unit activity for theta and gamma oscillation, respectively. Figures 8C,E show the relationship between the IG measures and the sum of the connection weights under the theta oscillation. Similarly, Figures 8D,F are for gamma oscillation. The results show that the linear dependency of θ^{(4, 1250)}_{12} on the sum of the connection weights holds true for an excitatory and inhibitory neuron pair. We also found that θ^{(4, 1250)}_{1} had almost no relationship with the sum of connection weight. For the relationship between the IG measures and the magnitude of constant input *H*^{E}_{i} and *H*^{I}_{i}, we confirmed that θ ^{(4, 1250)}_{1} was linearly related to their magnitude, but θ^{(4, 1250)}_{12} was independent from them (data not shown).

**Figure 8. Relationship between the IG measures and the sum of connection weights for an internally induced oscillation**. Neuron 1 was selected from the excitatory neuron pool and Neuron 2 was selected from the inhibitory neuron pool. In other words, one of the connections in *J*_{12} + *J*_{21} was positive and the other was negative. **(A)** Average firing probability of 1250 Izhikevich neurons with approximately a 6-Hz oscillatory oscillation is shown. **(B)** Average firing probability of 1250 Izhikevich neurons with approximately a 40-Hz oscillatory oscillation is shown. **(C)** Relationship between the pairwise-IG measure (θ^{(4, 1000)}_{12}) and the sum of connection weights (*J*_{12} + *J*_{21}) under a 6-Hz oscillation. Black and gray lines represent the simulation results by binary neurons and Izhikevich neurons, respectively. **(D)** Relationship between the pairwise-IG measure (θ ^{(4, 1000)}_{12}) and the sum of connection weights (*J*_{12} + *J*_{21}) under a 6-Hz oscillation. **(E)** Relationship between the single-IG measure (θ^{(4, 1000)}_{1}) and the sum of connection weights (*J*_{12} + *J*_{21}) under a 6-Hz oscillation. **(F)** Relationship between the single-IG measure (θ^{(4, 1000)}_{1}) and the sum of connection weights (*J*_{12} + *J*_{21}) under a 40-Hz oscillation.

In summary, for internally generated oscillations, we demonstrated that the relationship between the IG measures and the connection weights that were found under equilibrium assumption also held true.

## Discussion

Previous studies have shown that the IG measures provided useful information about network structures (Tatsuno and Okada, 2004; Tatsuno et al., 2009; Nie and Tatsuno, 2012). Specifically, the single-IG measure θ^{(4, N)}_{i} was related to the magnitude of external constant input, and the pairwise-IG measure θ^{(4, N)}_{ij} was related to the sum of the connection strengths. Although these studies were conducted under the equilibrium assumption, the real neural signals exhibit various oscillations depending on cognitive demand of the task or the state of the brain. Therefore, we studied the relationship between the IG measures and the neural network parameters under oscillatory network states.

We have considered two general oscillation mechanisms; one was the oscillation driven by external input, and the other was the oscillation induced internally due to interactions between excitatory and inhibitory neuron pools. Numerical simulation was performed by the network of a simple binary neuron model and the Izhikevich neuron model. The former model was used so as to compare the results with that of previous studies, and the latter was used to investigate the relationship with more realistic model neurons.

For the external oscillation, our investigation showed that θ^{(4, N)}_{ij} was linearly related to the sum of the connection strengths, and that θ^{(4, N)}_{i} was independent from it over a wide range of frequency from 1 to 200 Hz. We also showed that the relationship holds true when there are phase differences between neurons. In addition, we demonstrated that θ^{(4, N)}_{i} was almost linearly related to the magnitude of sinusoidal input, but that θ^{(4, N)}_{ij} was almost independent from it. For the internally induced oscillation, we have also confirmed that θ^{(4, N)}_{ij} was linearly related to the sum of the connection strengths, and that θ^{(4, N)}_{i} was independent from it. We have also shown that the same relationship holds true for any neuron pairs (within excitatory population, within inhibitory population, and across excitatory and inhibitory populations).

In summary, this study and previous studies have demonstrated that the IG measure provides useful information for analyzing neural circuits; not only for the equilibrium condition, but also for the oscillatory condition. The single-IG measure is useful for estimating the relative strength of external inputs. In addition, the single-IG measure is better than using the change in firing rate because the firing rate can be modulated both by the change in synaptic coupling strength and the magnitude of external inputs. Studies show that the appropriately selected single-IG measure is capable of estimating the external inputs with relatively small influence from synaptic interactions. Similarly, the pairwise-IG measure can provide more direct information about the synaptic interactions between neurons than other correlation measures (Amari, 2009). It has been also shown that the pairwise-IG measure is statistically independent from the change in firing rate and that it provides pure neural interactions (Amari, 2001; Nakahara and Amari, 2002). Together with the findings in this study, the pairwise-IG measure is a very useful measure to study direct neural interactions between neurons.

This study suggests that the actual values of the IG measures depend on the mechanisms of oscillation. For an externally driven oscillation, θ^{(4, N)}_{12} ~ 0.2 was obtained for (*J*_{12} + *J*_{21}) ~ 1/*N*. For an internally induced oscillation, the same connection strength produced θ^{(4, N)}_{12} ~ 0.002. Within the same oscillation mechanism, the selection of model neurons (binary model or Izhikevich model), or a small difference in network parameters such as phase differences also produced a difference in the actual value of the IG measures. Nonetheless, as long as the network is in one of the oscillation mechanisms, and the phase difference is kept the same, the IG measures can provide useful insights into network structures regardless of the oscillation frequencies.

In the study of memory consolidation, one of the key questions is to understand how the changes in synaptic connections are related to learning and memory formation. Evidence suggests that neural activity during slow-wave sleep plays an important role in learning (Diekelmann and Born, 2010). Specifically, there is increasing evidence supporting the hypothesis that replay of neural activity during subsequent sleep is positively correlated with memory formation (Pavlides and Winson, 1989; Wilson and McNaughton, 1994; Kudrimoti et al., 1999; Lee and Wilson, 2002; Euston et al., 2007; Girardeau et al., 2009; Peyrache et al., 2009b; Ego-Stengel and Wilson, 2010). However, the direct information about synaptic change is not available from multi-unit recordings of a freely behaving animal because spikes and local field potentials are the two main observables. In this study, we showed that θ^{(4, N)}_{ij} was linearly related to the sum of the connection weights, and that θ ^{(4, N)}_{i} was linearly related to the magnitude of external inputs, even under the oscillatory conditions. We have also verified these relationships not only with a simple binary model neuron, but also with a more realistic spiking model neuron. This finding would allow us to analyze neural activity during slow-wave sleep before and after the task; θ^{(4, N)}_{ij} would be a good measure for the change of connection weight, and θ^{(4, N)}_{i} for the magnitude of background input that would be influenced by local field potentials. By comparing the relative change of θ^{(4, N)}_{ij} between slow-wave sleeps before and after the task, and the strength of memory replay/improvement of behavior performance, the IG measure may provide a way to estimate the relationship between the synaptic modification and memory formation without having direct access to information of synaptic change.

As a related approach to the IG method, the maximum entropy (MaxEnt) has attracted much attention recently (Schneidman et al., 2006; Tang et al., 2008; Tyler et al., 2012). The philosophy of the MaxEnt approach is not to assume anything other than what we know from the data. For example, if firing rate and pairwise correlation are the only information we have, the distribution with maximum entropy is given as the Boltzmann distribution,

where *h*′_{i} is a bias term for the neuron *i*, *J*′_{ij} is the symmetric coupling strength between neurons *i* and *j*, and the partition function *Z* is given by,

We see that the MaxEnt is equivalent to the IG with the 2nd-order LLM,

where the relationship between the parameters are given as,

As was discussed in Tatsuno and Okada (2004) and in Tatsuno et al. (2009), it is possible to relate these IG measures, θ^{(2, N)}_{i} and θ^{(2, N)}_{ij}, to the network structure even for a network with asymmetric connections (Equation 5). However, under the influence of correlated inputs, we have also shown that the relationship in Equation 5 broke down, and that it was necessary to use the IG measures with the higher-order LLM such as the 4th-order (Equation 6) (Nie and Tatsuno, 2012). In other words, it was necessary to take into account neural activity of two additional neurons to estimate the direct neural interaction between neuron *i* and *j*. In summary, we see that the MaxEnt approach and the IG method are closely related. In addition, we also see that the MaxEnt can be considered a part of the IG method that provides a more general analysis framework for the space of the probability distributions.

In this study, we used the synchronous neural activity for estimating the direct neural interaction as the form of (*J*_{ij} + *J*_{ji}). However, in the real learning processes such as sequential learning, it is possible that synaptic modification occurs differently for each direction; e.g., *J*_{ij} increases, while *J*_{ji} decreases. The proposed method is not able to estimate the directed synaptic change. As one possible remedy for this difficulty, calculation of the pairwise-IG measure using the time-lagged spiking activity between neurons was suggested (Tatsuno and Okada, 2004; Nie and Tatsuno, 2012). Another limitation of the present study is not including the effect of delay; e.g., axonal conduction delay or synaptic transmission delay. It is possible that these delays dramatically change the firing patterns as well as increase a variety of coexisting patterns (Izhikevich, 2006). Little is known about the relationship between the IG measures and direct neural interactions with conduction delay. In addition, it has not been clear how IG measures with more neuronal interactions such as triplewise-IG measures θ^{(k, N)}_{ijk} or quadruple-IG measures θ^{(k, N)}_{ijkl} behave under oscillatory conditions. It would be interesting to extend the current study to include more neuronal interactions.

Despite these limitations, the IG method is one of the most promising statistical tools for spike train analysis (Amari, 2001; Nakahara and Amari, 2002). Its direct relationship with the network parameters would provide useful information for the estimation of structural changes (Tatsuno and Okada, 2004; Tatsuno et al., 2009; Nie and Tatsuno, 2012). We hope that an advancement of novel analysis methods including IG will lead to a break-through finding in neuroscience.

## Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

## Acknowledgments

We are grateful to Karim Ali for his support with the numerical simulations and to Amanda Mauthe-Kaddoura for proofreading the manuscript. This work was supported by Alberta Innovates Technology Futures (SCH001), the National Science Foundation (CRCNS-1010172), and Alberta Innovates Health Solutions (Polaris Award).

## References

Abeles, M., and Gerstein, G. L. (1988). Detecting spatiotemporal firing patterns among simultaneously recorded single neurons. *J. Neurophysiol*. 60, 909–924.

Adini, Y., Sagi, D., and Tsodyks, M. (1997). Excitatory-inhibitory network in the visual cortex: psychophysical evidence. *Proc. Natl. Acad. Sci. U.S.A*. 94, 10426–10431. doi: 10.1073/pnas.94.19.10426

Aertsen, A. M., Gerstein, G. L., Habib, M. K., and Palm, G. (1989). Dynamics of neuronal firing correlation: modulation of “effective connectivity.” *J. Neurophysiol*. 61, 900–917.

Amari, S. (2001). Information geometry on hierarchy of probability distributions. *IEEE Trans. Inf. Theory* 47, 1701–1711. doi: 10.1109/18.930911

Amari, S. (2009). Measure of correlation orthogonal to change in firing rate. *Neural Comput*. 21, 960–972. doi: 10.1162/neco.2008.03-08-729

Amari, S., and Nagaoka, H. (2000). *Methods of Information Geometry*. New York, NY: Oxford University Press.

Amari, S., Nakahara, H., Wu, S., and Sakai, Y. (2003). Synchronous firing and higher-order interactions in neuron pool. *Neural Comput*. 15, 127–142. doi: 10.1162/089976603321043720

Baker, S. N., Olivier, E., and Lemon, R. N. (1997). Coherent oscillations in monkey motor cortex and hand muscle EMG show task-dependent modulation. *J. Physiol*. 501(pt 1), 225–241. doi: 10.1111/j.1469-7793.1997.225bo.x

Bermudez Contreras, E. J., Schjetnan, A. G., Muhammad, A., Bartho, P., McNaughton, B. L., Kolb, B., et al. (2013). Formation and reverberation of sequential neural activity patterns evoked by sensory stimulation are enhanced during cortical desynchronization. *Neuron* 79, 555–566. doi: 10.1016/j.neuron.2013.06.013

Bland, B. H. (1986). The physiology and pharmacology of hippocampal formation theta rhythms. *Prog. Neurobiol*. 26, 1–54. doi: 10.1016/0301-0082(86)90019-5

Brown, E. N., Kass, R. E., and Mitra, P. P. (2004). Multiple neural spike train data analysis: state-of-the-art and future challenges. *Nat. Neurosci*. 7, 456–461. doi: 10.1038/nn1228

Buzsaki, G. (2002). Theta oscillations in the hippocampus. *Neuron* 33, 325–340. doi: 10.1016/S0896-6273(02)00586-X

Buzsaki, G. (2004). Large-scale recording of neuronal ensembles. *Nat. Neurosci*. 7, 446–451. doi: 10.1038/nn1233

Buzsaki, G., Horvath, Z., Urioste, R., Hetke, J., and Wise, K. (1992). High-frequency network oscillation in the hippocampus. *Science* 256, 1025–1027. doi: 10.1126/science.1589772

Buzsaki, G., and Wang, X. J. (2012). Mechanisms of gamma oscillations. *Annu. Rev. Neurosci*. 35, 203–225. doi: 10.1146/annurev-neuro-062111-150444

Chapin, J. K., Moxon, K. A., Markowitz, R. S., and Nicolelis, M. A. (1999). Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex. *Nat. Neurosci*. 2, 664–670. doi: 10.1038/10223

Crunelli, V., and Hughes, S. W. (2010). The slow (<1 Hz) rhythm of non-REM sleep: a dialogue between three cardinal oscillators. *Nat. Neurosci*. 13, 9–17. doi: 10.1038/nn.2445

Czanner, G., Grun, S., and Iyengar, S. (2005). Theory of the snowflake plot and its relations to higher-order analysis methods. *Neural Comput*. 17, 1456–1479. doi: 10.1162/0899766053723041

Davidson, T. J., Kloosterman, F., and Wilson, M. A. (2009). Hippocampal replay of extended experience. *Neuron* 63, 497–507. doi: 10.1016/j.neuron.2009.07.027

Diekelmann, S., and Born, J. (2010). The memory function of sleep. *Nat. Rev. Neurosci*. 11, 114–126. doi: 10.1038/nrn2762

Dragoi, G., Carpi, D., Recce, M., Csicsvari, J., and Buzsaki, G. (1999). Interactions between hippocampus and medial septum during sharp waves and theta oscillation in the behaving rat. *J. Neurosci*. 19, 6191–6199.

Dragoi, G., and Tonegawa, S. (2011). Preplay of future place cell sequences by hippocampal cellular assemblies. *Nature* 469, 397–401. doi: 10.1038/nature09633

Dragoi, G., and Tonegawa, S. (2013). Distinct preplay of multiple novel spatial experiences in the rat. *Proc. Natl. Acad. Sci. U.S.A*. 110, 9100–9105. doi: 10.1073/pnas.1306031110

Ego-Stengel, V., and Wilson, M. A. (2010). Disruption of ripple-associated hippocampal activity during rest impairs spatial learning in the rat. *Hippocampus* 20, 1–10. doi: 10.1002/hipo.20707

Eleuteri, A., Tagliaferri, R., and Milano, L. (2005). A novel information geometric approach to variable selection in MLP networks. *Neural Netw*. 18, 1309–1318. doi: 10.1016/j.neunet.2005.01.008

Euston, D. R., Tatsuno, M., and McNaughton, B. L. (2007). Fast-forward playback of recent memory sequences in prefrontal cortex during sleep. *Science* 318, 1147–1150. doi: 10.1126/science.1148979

Fellous, J. M., Tiesinga, P. H., Thomas, P. J., and Sejnowski, T. J. (2004). Discovering spike patterns in neuronal responses. *J. Neurosci*. 24, 2989–3001. doi: 10.1523/JNEUROSCI.4649-03.2004

Gerstein, G. L., and Perkel, D. H. (1969). Simultaneously recorded trains of action potentials: analysis and functional interpretation. *Science* 164, 828–830. doi: 10.1126/science.164.3881.828

Gilestro, G. F., Tononi, G., and Cirelli, C. (2009). Widespread changes in synaptic markers as a function of sleep and wakefulness in Drosophila. *Science* 324, 109–112. doi: 10.1126/science.1166673

Ginzburg, I. I., and Sompolinsky, H. (1994). Theory of correlations in stochastic neural networks. *Phys. Rev. E Stat. Phys. Plasmas Fluids Relat. Interdiscip. Topics* 50, 3171–3191.

Girardeau, G., Benchenane, K., Wiener, S. I., Buzsaki, G., and Zugaro, M. B. (2009). Selective suppression of hippocampal ripples impairs spatial memory. *Nat. Neurosci*. 12, 1222–1223. doi: 10.1038/nn.2384

Gray, C. M., Konig, P., Engel, A. K., and Singer, W. (1989). Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. *Nature* 338, 334–337. doi: 10.1038/338334a0

Grun, S., Diesmann, M., and Aertsen, A. (2002a). Unitary events in multiple single-neuron spiking activity: I. Detection and significance. *Neural Comput*. 14, 43–80. doi: 10.1162/089976602753284455

Grun, S., Diesmann, M., and Aertsen, A. (2002b). Unitary events in multiple single-neuron spiking activity: II. Nonstationary data. *Neural Comput*. 14, 81–119. doi: 10.1162/089976602753284464

Hansel, D., Mato, G., and Meunier, C. (1995). Synchrony in excitatory neural networks. *Neural Comput*. 7, 307–337. doi: 10.1162/neco.1995.7.2.307

Hoffman, K. L., and McNaughton, B. L. (2002). Coordinated reactivation of distributed memory traces in primate neocortex. *Science* 297, 2070–2073. doi: 10.1126/science.1073538

Huber, R., Ghilardi, M. F., Massimini, M., and Tononi, G. (2004). Local sleep and learning. *Nature* 430, 78–81. doi: 10.1038/nature02663

Ikeda, K. (2005). Information geometry of interspike intervals in spiking neurons. *Neural Comput*. 17, 2719–2735. doi: 10.1162/089976605774320593

Ince, R. A., Senatore, R., Arabzadeh, E., Montani, F., Diamond, M. E., and Panzeri, S. (2010). Information-theoretic methods for studying population codes. *Neural Netw*. 23, 713–727. doi: 10.1016/j.neunet.2010.05.008

Izhikevich, E. M. (2003). Simple model of spiking neurons. *IEEE Trans. Neural Netw*. 14, 1569–1572. doi: 10.1109/TNN.2003.820440

Izhikevich, E. M. (2006). Polychronization: computation with spikes. *Neural Comput*. 18, 245–282. doi: 10.1162/089976606775093882

Kudrimoti, H. S., Barnes, C. A., and McNaughton, B. L. (1999). Reactivation of hippocampal cell assemblies: effects of behavioral state, experience, and EEG dynamics. *J. Neurosci*. 19, 4090–4101.

Laubach, M., Wessberg, J., and Nicolelis, M. A. (2000). Cortical ensemble activity increasingly predicts behaviour outcomes during learning of a motor task. *Nature* 405, 567–571. doi: 10.1038/35014604

Lee, A. K., and Wilson, M. A. (2002). Memory of sequential experience in the hippocampus during slow wave sleep. *Neuron* 36, 1183–1194. doi: 10.1016/S0896-6273(02)01096-6

Lopes-Dos-Santos, V., Conde-Ocazionez, S., Nicolelis, M. A., Ribeiro, S. T., and Tort, A. B. (2011). Neuronal assembly detection and cell membership specification by principal component analysis. *PLoS ONE* 6:e20996. doi: 10.1371/journal.pone.0020996

Lovette, I. J., Arbogast, B. S., Curry, R. L., Zink, R. M., Botero, C. A., Sullivan, J. P., et al. (2011). Phylogenetic relationships of the mockingbirds and thrashers (Aves: Mimidae). *Mol. Phylogenet. Evol*. 63, 219–229. doi: 10.1016/j.ympev.2011.07.009

Luczak, A., Bartho, P., Marguet, S. L., Buzsaki, G., and Harris, K. D. (2007). Sequential structure of neocortical spontaneous activity *in vivo*. *Proc. Natl. Acad. Sci. U.S.A*. 104, 347–352. doi: 10.1073/pnas.0605643104

Meltzer, J. A., Zaveri, H. P., Goncharova, I. I., Distasio, M. M., Papademetris, X., Spencer, S. S., et al. (2008). Effects of working memory load on oscillatory power in human intracranial EEG. *Cereb. Cortex* 18, 1843–1855. doi: 10.1093/cercor/bhm213

Mirollo, R. E., and Strogatz, S. H. (1990). Synchronization of pulse-coupled biological oscillators. *SIAM J. Appl. Math*. 50, 1645–1662. doi: 10.1137/0150098

Miura, K., Okada, M., and Amari, S. (2006). Estimating spiking irregularities under changing environments. *Neural Comput*. 18, 2359–2386. doi: 10.1162/neco.2006.18.10.2359

Nakahara, H., and Amari, S. (2002). Information-geometric measure for neural spikes. *Neural Comput*. 14, 2269–2316. doi: 10.1162/08997660260293238

Nakahara, H., Amari, S., and Richmond, B. J. (2006). A comparison of descriptive models of a single spike train by information-geometric measure. *Neural Comput*. 18, 545–568. doi: 10.1162/neco.2006.18.3.545

Nie, Y., and Tatsuno, M. (2012). Information-geometric measures for estimation of connection weight under correlated inputs. *Neural Comput*. 24, 3213–3245. doi: 10.1162/NECO_a_00367

Panzeri, S., and Schultz, S. R. (2001). A unified approach to the study of temporal, correlational, and rate coding. *Neural Comput*. 13, 1311–1349. doi: 10.1162/08997660152002870

Pavlides, C., and Winson, J. (1989). Influences of hippocampal place cell firing in the awake state on the activity of these cells during subsequent sleep episodes. *J. Neurosci*. 9, 2907–2918.

Peyrache, A., Benchenane, K., Khamassi, M., Wiener, S. I., and Battaglia, F. P. (2009a). Principal component analysis of ensemble recordings reveals cell assemblies at high temporal resolution. *J. Comput. Neurosci*. 29, 309–325. doi: 10.1007/s10827-009-0154-6

Peyrache, A., Khamassi, M., Benchenane, K., Wiener, S. I., and Battaglia, F. P. (2009b). Replay of rule-learning related neural patterns in the prefrontal cortex during sleep. *Nat. Neurosci*. 12, 919–926. doi: 10.1038/nn.2337

Raghavachari, S., Kahana, M. J., Rizzuto, D. S., Caplan, J. B., Kirschen, M. P., Bourgeois, B., et al. (2001). Gating of human theta oscillations by a working memory task. *J. Neurosci*. 21, 3175–3183. doi: 10.1152/jn.00409.2005

Schneidman, E., Berry, M. J. 2nd., Segev, R., and Bialek, W. (2006). Weak pairwise correlations imply strongly correlated network states in a neural population. *Nature* 440, 1007–1012. doi: 10.1038/nature04701

Shimazaki, H., and Shinomoto, S. (2007). A method for selecting the bin size of a time histogram. *Neural Comput*. 19, 1503–1527. doi: 10.1162/neco.2007.19.6.1503

Shimokawa, T., and Shinomoto, S. (2009). Estimating instantaneous irregularity of neuronal firing. *Neural Comput*. 21, 1931–1951. doi: 10.1162/neco.2009.08-08-841

Singer, W., and Gray, C. M. (1995). Visual feature integration and the temporal correlation hypothesis. *Annu. Rev. Neurosci*. 18, 555–586. doi: 10.1146/annurev.ne.18.030195.003011

Steriade, M., Nunez, A., and Amzica, F. (1993). A novel slow (< 1 Hz) oscillation of neocortical neurons *in vivo*: depolarizing and hyperpolarizing components. *J. Neurosci*. 13, 3252–3265.

Stickgold, R. (2005). Sleep-dependent memory consolidation. *Nature* 437, 1272–1278. doi: 10.1038/nature04286

Tang, A., Jackson, D., Hobbs, J., Chen, W., Smith, J. L., Patel, H., et al. (2008). A maximum entropy model applied to spatial and temporal correlations from cortical networks *in vitro*. *J. Neurosci*. 28, 505–518. doi: 10.1523/JNEUROSCI.3359-07.2008

Tatsuno, M., Fellous, J. M., and Amari, S. I. (2009). Information-geometric measures as robust estimators of connection strengths and external inputs. *Neural Comput*. 21, 2309–2335. doi: 10.1162/neco.2009.04-08-748

Tatsuno, M., Lipa, P., and McNaughton, B. L. (2006). Methodological considerations on the use of template matching to study long-lasting memory trace replay. *J. Neurosci*. 26, 10727–10742. doi: 10.1523/JNEUROSCI.3317-06.2006

Tatsuno, M., and Okada, M. (2004). Investigation of possible neural architectures underlying information-geometric measures. *Neural Comput*. 16, 737–765. doi: 10.1162/089976604322860686

Tyler, A. L., Mahoney, J. M., Richard, G. R., Holmes, G. L., Lenck-Santini, P. P., and Scott, R. C. (2012). Functional network changes in hippocampal CA1 after status epilepticus predict spatial memory deficits in rats. *J. Neurosci*. 32, 11365–11376. doi: 10.1523/JNEUROSCI.1516-12.2012

Urban, N. N., Henze, D. A., and Barrionuevo, G. (2001). Revisiting the role of the hippocampal mossy fiber synapse. *Hippocampus* 11, 408–417. doi: 10.1002/hipo.1055

Vanderwolf, C. H. (1969). Hippocampal electrical activity and voluntary movement in the rat. *Electroencephalogr. Clin. Neurophysiol*. 26, 407–418. doi: 10.1016/0013-4694(69)90092-3

Wilson, M. A., and McNaughton, B. L. (1993). Dynamics of the hippocampal ensemble code for space. *Science* 261, 1055–1058. doi: 10.1126/science.8351520

Wilson, M. A., and McNaughton, B. L. (1994). Reactivation of hippocampal ensemble memories during sleep. *Science* 265, 676–679. doi: 10.1126/science.8036517

Keywords: information geometry, spikes, spiking neuron model, oscillation, neural networks

Citation: Nie Y, Fellous J-M and Tatsuno M (2014) Information-geometric measures estimate neural interactions during oscillatory brain states. *Front. Neural Circuits* **8**:11. doi: 10.3389/fncir.2014.00011

Received: 09 November 2013; Accepted: 04 February 2014;

Published online: 24 February 2014.

Edited by:

A. Ravishankar Rao, IBM Research, USAReviewed by:

Carmen C. Canavier, LSU Health Sciences Center, USATill R. Schneider, University Medical Center Hamburg-Eppendorf, Germany

Copyright © 2014 Nie, Fellous and Tatsuno. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Masami Tatsuno, Department of Neuroscience, Canadian Centre for Behavioural Neuroscience, The University of Lethbridge, 4401 University Dr W, Lethbridge, AB T1K 3M4, Canada e-mail: tatsuno@uleth.ca