Paired Feed-Forward Excitation With Delayed Inhibition Allows High Frequency Computations Across Brain Regions

The transmission of high frequency temporal information across brain regions is critical to perception, but the mechanisms underlying such transmission remain unclear. Long-range projection patterns across brain areas are often comprised of paired feed-forward excitation followed closely by delayed inhibition, including the thalamic triad synapse, thalamic projections to cortex, and projections within the hippocampus. Previous studies have shown that these joint projections produce a shortened period of depolarization, sharpening the timing window over which the postsynaptic neuron can fire. Here we show that these projections can facilitate the transmission of high frequency computations even at frequencies that are highly filtered by neuronal membranes. This temporal facilitation occurred over a range of synaptic parameter values, including variations in synaptic strength, synaptic time constants, short-term synaptic depression, and the delay between excitation and inhibition. Further, these projections can coordinate computations across multiple network levels, even amid ongoing local activity. We suggest that paired feed-forward excitation and inhibition provide a hybrid signal—carrying both a value and a clock-like trigger—to allow circuits to be responsive to input whenever it arrives.


INTRODUCTION
In digital electronics, computations are synchronized by the presence of a digital clock trigger signal that indicates when each digital component should examine its inputs and perform a computation. The digital clock offers several advantages for digital circuits. First, it allows digital circuits to have short integration times because the components do not need to consider information that has arrived at its inputs long in the past. Second, this short integration time allows the digital circuit to operate at high temporal speeds (many computations per second). Third, the digital clock allows synchronization of computations across different layers of digital components, because the arrival of related inputs is coordinated by the digital clock.
In neuronal circuits, several properties would seem to make coordinated and precise high-speed computation across layers impossible: the apparent lack of an equivalent clock, the electrical filtering that is performed by the cell's membrane, and relatively slow synaptic time constants (von Neumann, 1963). However, we know that neural circuits do operate very quickly, as humans can make decisions within 200 ms of the arrival of a visual stimulus (Thorpe et al., 1996;Sherwin et al., 2012). In the retina, ganglion cells can respond very well to temporal frequencies as high as 50 Hz (Frishman et al., 1987), and LGN neurons can reliably follow the high temporal frequencies of retinal ganglion cell inputs (Movshon et al., 2005), despite postsynaptic membrane filtering and synaptic depression. Further, in natural environments, stimuli may arrive at any time, at slow rates or fast rates, and circuitry must be ready to respond quickly regardless of when a previous stimulus arrived. Therefore, the brain must have mechanisms for quickly following high frequency inputs.
Many long-range connections across brain areas exhibit a peculiar motif: long-range feed-forward excitatory input is paired, after a short delay, with a feed-forward inhibitory copy of the same input. At the retinogeniculate synapse, this inhibitory input is produced at a specialized dendro-dendritic synapse (the triad synapse) so that each feed-forward excitatory input is followed by a very fast and local inhibitory copy (Cox et al., 1998;Chen and Regehr, 2000;Regehr, 2003, 2005). In other connections, such as those from the thalamus to cortex, across regions of the hippocampus, or interareal connections, feed-forward excitatory inputs project to both principal neurons and inhibitory interneurons, and the delayed inhibitory input onto principal neurons arises from the interneurons that received feed-forward excitation (Buzsaki, 1984;Agmon and Connors, 1991;Swadlow and Gusev, 2000;Porter et al., 2001;Swadlow, 2003;Gabernet et al., 2005;Pouille et al., 2009;Yang et al., 2013;Rock and Apicella, 2015;Bhatia et al., 2019).
It has been previously recognized that the coordinated arrival of the excitatory synaptic drive followed by the inhibitory drive can result in very temporally precise postsynaptic action potentials and short integration windows (Pouille and Scanziani, 2001;Wehr and Zador, 2003;Blitz and Regehr, 2005;Mittmann et al., 2005;Higley and Contreras, 2006;Kremkow et al., 2010b;Cardin, 2018). Here in theoretical work, we explore the basic physics of paired feed-forward excitation and inhibition. We show that paired feed-forward excitation and inhibition (FFEI) allows computation at very high temporal frequencies that ''break'' the membrane time constant barrier. Further, we show that this motif can synchronize computations across multiple hierarchical regions even when these regions have their own noisy ongoing local activity. Finally, we compare the limits of the temporal computations that are permitted in a low fan-in situation (1 paired synaptic input, such as at the triad synapse) and a high fan-in situation (such as in input to the cortex).
High temporal frequency transmission was possible over a wide range of synaptic parameters, including variations in synaptic strengths, synaptic time constants, synaptic depression, and delays between excitation and inhibition, suggesting that high temporal frequency transmission should be a characteristic of a variety of circuits that have this input structure. We conclude that paired feed-forward excitatory and delayed inhibitory input is a hybrid signal, providing information about the value of the input while at the same time imposing a clock-like trigger signal that shortens temporal integration in a manner that allows high frequency computation and synchronization across networks or layers.

MATERIALS AND METHODS
Neural circuit simulations were performed in MATLAB (MathWorks, Natick, MA). Leaky integrate and fire neurons were modeled with the following differential equation, using the forward Euler method with a 0.1 ms time step (Lapicque, 1907;Abbott, 1999): where V m is the membrane potential of the neuron, V e is the leak potential, τ m is the membrane time constant, R m is the membrane resistance, P n (t) is the synaptic conductance from synaptic input n, V syn,n is the resting potential at the synapse for input n, and α n (= 1 for excitatory synapses) is a scaling coefficient for the synaptic current from input n. When V m reaches the threshold potential V thresh , the model neuron generates a spike and V m is set to the reset potential V reset (see Tables 1 and 2 for values). Synapses were modeled as a difference of exponentials, with two time constants (Dayan and Abbott, 2001): where P max,n is the maximum conductance of the synapse, B n is a normalization factor to ensure that the peak value of P n (t) is P max,n , ∆t j is the time between t and the jth spike of synaptic input n, τ fall,n is the fall time constant, and τ rise,n is the rise time constant (see Tables 1 and 2 for values).

Simulating Inputs to Neural Circuits
Temporal information was generated using a Poisson process with a rectified sinusoidal firing rate: where PR is the peak firing rate set to 100 Hz for all simulations, F is the input modulation frequency, and rectify(x) = 0 if x < 0, and rectify(x) = x for x ≥ 0. For each set of simulations, temporal information was presented to our simulated neural circuits for a fixed length of time, independent of F.
In some of our simulations, we provided random background activity as an additional excitatory synaptic input. This activity was modeled as N inputs noisy Poisson inputs with a uniform firing rate FR (see Table 1 for values). FR was calculated such that, for  Figure 6 modifications: dt = 0.02 ms Paired feed-forward mixed-sign P max, e = P max, i = 1.31 µS ¶ Feed-forward single-sign P max = 0.066 µS ¶ Figure 6 supplement modifications: dt = 0.02 ms Paired feed-forward mixed-sign P max, e = P max, i = 1.20 µS ¶ Feed-forward single-sign P max = 0.068 µS ¶ Figure 7  Parameters of the single input FFEI / FFE models. † : Blitz and Regehr (2005). ‡ : Chen et al. (2002). & : Chen and Regehr (2000). § : Dayan and Abbott (2001). ¶ : For each set of simulations, synaptic input strengths were chosen to balance the response at F = 5 Hz between different parameter sets. : Excitation at biological triad synapses involves a combination of a fast-decaying AMPA current (Chen et al., 2002;Blitz and Regehr, 2005) and a slow-decaying NMDA current (Chen et al., 2002), resulting in a complex excitatory conductance (Chen et al., 2002). In order to simplify the analysis of this circuit motif, synaptic time constants were chosen to produce a single excitatory current with medium decay. † † : The background noise firing rate was adjusted such that the mean firing rates of the feed-forward inputs and each noisy input were balanced. && : The strength of noisy input was adjusted such that the total noise was comparable to the feed-forward input. § § : GABA A and GABA B channels were both included in triad synapse models with depression (Soltesz and Crunelli, 1992); GABA B time constants were chosen in the range of experimental values (Thompson, 1994;Destexhe and Sejnowski, 1995). ¶ ¶ : Depression parameters were obtained by fitting paired-pulse ratio data from the retinogeniculate synapse (Chen et al., 2002;Blitz and Regehr, 2005) to a depression model with two depressing factors .
each noisy input, the probability of seeing 1 background spike during an input cycle is equal to the probability of a feed-forward input spike over one cycle (that is, the mean firing rate of the feed-forward inputs and each noisy input were balanced):

Calculating the Power of Spiking Responses at a Particular Frequency
Spiking responses were converted to a firing rate per time step (Hz), and the following equation was used to calculate the Fourier coefficient magnitude at a particular frequency (Crawford, 1968): where ∆t is the time step size in seconds, L is the length of the spiking response in seconds, t index is time as an integer index, R(t) is the spiking response in Hz, and F is a particular frequency.
To normalize the power of spiking responses at a particular frequency, we used the average power calculated over all frequencies (FC avg ) as a normalization factor. For a time interval of length L in time steps of ∆t, 1/∆t and 1/L are the maximum and minimum frequencies that can be represented, respectively. Therefore, FC avg was determined by averaging FC F over F = [0,1/∆t] in intervals of 1/L. The normalized frequency response was then expressed as FC F /FC avg . In cases where FC avg = 0, the normalized frequency response was set to 0.
Unless otherwise stated, FC F , FC avg , and FC F /FC avg were each averaged over 10 trials for each model.

Balancing Excitation and Inhibition in Triad Synapse Models
When varying the inhibitory fall time constant (τ fall,i ) in our triad synapse models, we scaled P max,i relative to P max,e , such that the total excitation and inhibition are balanced: The above equation was derived by first integrating P e (t) and P i (t) for a single spike at t = 0 over the interval t = [0,∞], as a measure of the total conductance due to a single spike. P max,i values were then obtained by setting the total excitation equal to the total inhibition,and then solving for P max,i given a set of time constants and excitatory parameters.  Parameter LGN input to (E) LGN input to (I) P max 3 nS ## 4.08 nS ## τ fall 20 ms # 1.6 ms & Parameters of the many input FFEI / FFE models. # : Stratford et al. (1996). ♦ : The firing threshold of inhibitory interneurons was lowered, in order to represent the increased excitability of these neurons compared to other neurons in the cortex (McCormick et al., 1985). ## : Synaptic input strengths were chosen to balance the (I) and (E) response at F = 5 Hz between the new parameter set and the default parameters. See additional annotations for constants chosen in Table 1 caption.

Determining the Expected Output of an Exclusive-or Computation
In Figure 6, we examine the output of a circuit designed to perform an exclusive-or (XOR) computation of two inputs. The expected output of the circuit was determined by performing the XOR computation on sliding windows of length t bin = 10 ms or 5 ms. If only one of the inputs fires during the time interval [t − t bin 2 , t + t bin 2 ], then output is expected at time t.

Simulating Depression at the Retinogeniculate Synapse
Paired-pulse ratio data for AMPA and NMDA (Chen et al., 2002) and for GABA (Blitz and Regehr, 2005) in the retinogeniculate synapse were fitted to the following depression model : In the above equation, A is a scaling factor on the amplitude of the synaptic current generated by each presynaptic spike, A 0 is the undepressed scaling factor, and both D 1 and D 2 are depressing factors with default values of 1. Following each presynaptic spike, D i is set to D i d i , and D i decays back to 1 with a time constant τ d i (see Table 1 for values and Supplementary  Figure 3 for fits).

Sex as a Variable
Not applicable to the computational models studied here.

RESULTS
Our goal was to understand the physics of the configuration of paired feed-forward excitatory and inhibitory (FFEI) connections that are commonly observed in projections across brain regions. We analyzed the problem in two regimes. First, we analyzed a low convergence situation (N inputs = ∼1), inspired by the triad retinogeniculate synapse in the LGN. Here, we found that FFEI projections allowed feed-forward computations at much higher frequencies than if feed-forward excitatory (FFE) projections were employed alone. In order to focus specifically on the difference between FFEI projections and FFE projections, we first explored models that were inspired by   LGN-like neuron (blue). Bottom: A 0.5 s simulation of the model circuit with a rectified sinusoidal input firing rate (yellow) with a peak rate (PR) of 100 Hz and an input modulation frequency (F) of 50 Hz; presynaptic spikes were generated with a Poisson process (dark green). The postsynaptic conductance (light green) and the membrane potential of the LGN cell (blue) are simulated in response to the feed-forward input. Output firing times are marked (red x), and the postsynaptic response is compared to the underlying input characteristics (dashed yellow). (B) Same, but for a paired feed-forward excitatory/inhibitory (E/I) model circuit with a single feed-forward cell (light green) providing excitatory input to an output neuron (blue) and to an interneuron (gray), which in turn provides delayed inhibitory input (1 ms) to the output neuron. (C) For a range of input modulation frequencies F, the Fourier coefficient of the output at F (FC F ) was averaged over 10 trials for triad synapse models with different inhibitory (Continued) (J) For each paired feed-forward E/I model shown in (F), the 1 2 cutoff fold change with respect to the feed-forward E model of matching output power (denoted by line style and FC F at F = 5 Hz) was calculated as FFEI/FFE. For the lowest firing rates, 1 2 cutoff was greater than 1,000 Hz in the FFEI case (out of range). Dashed line indicates a fold change of 1. In all, paired E/I input allowed transmission of higher input modulation frequencies over a wide range of inhibitory synaptic time constants and for a wide range of synaptic input strengths.
the LGN triad synapse but were much simpler, to demonstrate the phenomena and how they depend on the inhibitory delay and properties of the inhibitory synapse. Next, we explored a more realistic model of the triad synapse that includes synaptic depression of excitatory and inhibitory synapses and showed that the principles of the very simple models apply under these conditions. Finally, we examined a high convergence situation (N inputs = ∼50-150) such as is found at thalamocortical and intercortical projections.

Paired Feed-Forward Excitatory and Inhibitory Inputs Allow the Transmission of Temporal Information at High Frequencies
We began with a simple model inspired by the retina-LGN triad synapse that is particularly common at synapses between retinal × cells and LGN × cells (Koch, 1985;Hamos et al., 1987;Lam et al., 2005;Bickford, 2019). At these synapses, excitatory retinal input is provided to an LGN neuron and to an adjacent terminal bouton of the dendrite of an interneuron. The interneuron dendrite, which is electrotonically isolated from its soma (Morgan and Lichtman, 2020), also contacts the LGN cell at very nearly the same location as the excitatory cell, providing an only slightly-delayed inhibitory input to the LGN neuron (Cox et al., 1998). To build our model, we included a feed-forward glutamatergic excitatory synaptic current with temporal dynamics that were set to be between the fast-decaying AMPA current and the slower-decaying NMDA current observed at triad synapses (Chen et al., 2002). Fast GABAergic currents were included with a 1 ms delay, also following experimental measurements Regehr, 2003, 2005). While the total synaptic conductance was varied, the maximal conductances of the excitatory and inhibitory synapses were selected so that the total conductance (the area under the curve) of excitation and inhibition were identical; that is, the total integrated conductance of excitation and inhibition were balanced.
To assess the transmission of temporal information in this model, we simulated the spiking responses of a leaky integrateand-fire neuron to input spikes generated via a Poisson process with a rectified sinusoidal firing rate (Figures 1A,B). The rate at which spikes were generated varied in time sinusoidally from 0 spikes/s to 100 spikes/s and the frequency of the sinewave was termed the modulation frequency. At each 0.1 ms, a random number (0-1) was drawn, and a spike was generated if the random number was less than the product of the rate and the bin size (0.1 ms). While the FFE configuration produces extra spikes that are outside the sinusoidal input modulation, output spikes in the FFEI input configuration are well locked to the sinusoidal input modulation. We examined the transmission of the FFEI projection over a range of inhibitory synapse fall time constants that varied from 20 to 50 ms, and compared its transmission to the case where only feed-forward excitatory (FFE) input was provided. Because changing the fall time constant of the inhibitory synaptic current, while keeping integrated E and I conductance fixed, alters the fraction of the excitatory conductance that arrives earlier in the postsynaptic response, the overall responsiveness of the cell also varies as we adjusted the inhibitory synapse fall time constant. To compare equally responsive models, we adjusted the feed-forward synaptic weight so that each model generated a response of about 75 Hz when driven with an input modulating frequency of 5 Hz.
FFEI projections demonstrated a remarkable ability to transmit high frequency information. To measure transmission quality at each input frequency, we calculated the Fourier coefficient (FC) of the spiking response at the input modulation frequency (FC F ; Figure 1C). The Fourier coefficient is higher when spikes are well locked to the stimulus, and drops as the number of spikes fired per cycle becomes lower or becomes statistical (that is, with some missed cycles). Projections with only excitatory synapses were filtered strongly at input modulation frequencies ranging from 20 to 100 Hz. In contrast, FFEI projections with inhibitory time constants of 20 ms transmitted with nearly equal fidelity from 5 to 100 Hz, and did not reach a 50% reduction in Fourier coefficient responses until the input modulation frequency reached about 400 Hz.
While the Fourier coefficient at the input modulation frequency (FC F ) measures the output transmission at that single frequency, it did not provide information about how much of the total output of the cell was concentrated exclusively at the input modulation frequency. For example, if the output cell increased its overall response only tonically, then FC F would still exhibit an increase. To examine the fraction of the output cell's total frequency response that was at the input modulation frequency, we divided FC F by the average Fourier coefficient calculated over all frequencies, including frequencies much higher and lower than the input modulation frequency ( Figure 1D), to produce a normalized frequency response measure ( Figure 1E). This analysis indicated that FFEI projections transmitted information that was highly specific to the input modulation frequency over a wide frequency range of 5-100 Hz, while the specificity of information transmitted by excitatory-alone projections fell off rapidly with frequency.
The ability to provide specific output at the input modulation frequency was highly related to the inhibitory fall time constant. When the inhibitory fall time constant was short, then the advantage of the FFEI configuration was greater. When the time constant is small (20 ms), then the inhibition comes right on the heels of the excitation so that the excitation cannot drive the cell for a long time. When there is no inhibition, the excitation can drive the cell for a long time, which means less of the response is at the input modulation frequency. When there is a long time constant of inhibition (50 ms), then the inhibition is more spread out in time and the behavior is more intermediate, although more single feed-forward input spikes are missed by the output neuron than in the short time constant case (Supplementary Figure 1).
We plotted the normalized output responses for input modulation frequencies of 50 and 100 Hz in Figure 1G. For an inhibitory fall time constant of 20 ms, the postsynaptic neuron produced output at the input modulation frequency that was more than 12 times the amount of average output at other frequencies. As the inhibitory fall time constant was increased (and weight compensated to retain equal responsiveness), the output neuron behaved more like the excitatory-only configuration, as one would expect. We quantified the ability of the synapse to maintain its response at high temporal frequencies by identifying the temporal frequency where the output was reduced to an arbitrary threshold of 1 2 its value at 5 Hz (a physiologically meaningful threshold would depend on the noise of each circuit). The 1 2 cutoff frequency was also highly dependent on the inhibitory fall time constant ( Figure 1H).
It was possible that the relative advantage of FFEI transmission compared to FFE was not general, but instead was specific to the particular levels of synaptic input strengths that were chosen in order to drive output responses of about 75 Hz at a 5 Hz input modulation frequency. To examine this possibility, we compared FFEI and FFE models that were matched at 5 Hz input modulation but over a wide variety of output response levels ( Figure 1F). In all cases, the FFEI projection responses at the input modulation frequency were much stronger than those of the FFE projection over a wide range of input modulation frequencies. To quantify these changes, we examined the fold change of transmission of FFEI compared to FFE at input modulation frequencies of 50 or 100 Hz for these different levels of synaptic drive ( Figure 1I). For all of these levels of synaptic drive, the FFEI projection produced at least two times as much drive at 50 and 100 Hz compared to FFE alone. The 1 2 cutoff frequency was also more than four times higher for FFEI compared to FFE ( Figure 1J).

Paired Feed-Forward Excitatory and Inhibitory Inputs Allow Effective Transmission at Frequencies That Are Highly Filtered by the Output Cell's Membrane Properties
The ability of the output cell to reliably follow very high input frequencies was surprising because the membrane properties of neurons act as a low-pass filter, and high frequency current fluctuations are not well transmitted to membrane voltage. We compared the ability of the output neuron to follow synaptic input provided through either FFEI or FFE projections with the ability of the output neuron to follow direct sinusoidal membrane current injections of a fixed amplitude (Figure 2). As before, we calibrated the output of the neuron to hit a target rate (75 Hz) at a 5 Hz input modulation frequency. For the FFEI and FFE simulations, we did this by setting the synaptic weight, and for the current injection simulations, we did this by setting the amplitude of the sinusoidal current injection. We compared the raw Fourier coefficient measured at the input modulation frequency (Figure 2B), the overall response across all frequencies (Figure 2C), and the normalized frequency response to measure the specificity of the output response to the input modulation frequency ( Figure 2D). As expected, the cell followed the sinusoidal current injection with substantial fidelity until a critical frequency was reached, after which the input fluctuations became too small to generate action potentials. This resulted in a rapid drop in the ability of the cell to follow the output frequencies. By contrast, FFEI inputs continued to drive responses at high frequencies, well beyond the point at which the sinusoidal current could drive spikes. While the critical frequency at which spiking stops for direct current injection does depend upon the amplitude of the current used, the take-home point here is that FFEI projections drive responses at higher input frequencies when responses to FFEI and direct current injections are matched at low frequencies.
The addition of an inhibitory conductance in the FFEI case will necessarily reduce the effective membrane time constant, and one may ask whether the increased transmission is primarily due to the reduced time constant. To explore this, we modified the membrane time constant of our model neuron over a 10-fold range and examined the influence of the passive membrane time constant on the temporal frequency of transmission (Figures 2E-G). We observed very small influences of modifying the membrane time constant, indicating that a passive adjustment of the membrane time constant cannot account for increased high frequency transmission observed here.
At first glance, it seems almost paradoxical that inputs delivered via FFEI projections can apparently ''break the limit'' of the membrane time constant. To gain intuition into the FIGURE 2 | Paired feed-forward E/I inputs allows transmission of higher input modulation frequencies compared to direct sinusoidal current injection; enhanced high frequency transmission is not attributed to changes in the effective membrane time constant. (A) Left: A model cell receiving direct injections of rectified sinusoidal currents at various frequencies. As input modulation frequency increases, the ability of the cell to follow the input decreases, and the cell is totally unable to spike in response to the input modulation frequency of 100 Hz. (B) Fourier coefficient of spiking output for a range of input modulation frequencies F for direct current injections (magenta), paired feed-forward E/I model with τ fall,I = 20 ms (blue), and for the feed-forward E model (red). (C) Same, except mean responses over all Fourier frequencies are shown (FC avg ). (D) The normalized output response (FC F /FC avg ). The dashed line indicates FC F /FC avg = 1. When firing rates are matched at input modulation frequency of 5 Hz, direct current injections provide more selective transmission for lower frequencies but paired feed-forward E/I connections allow the cell to follow signals at higher frequencies than direct current injection. (E) Adjustment of the membrane time constant over a 10-fold range did not allow high frequency transmission. Fourier coefficient of feed-forward E model output for a range of input modulation frequencies F. Legend entries indicate the membrane time constant τ m for each feed-forward E model. mechanism of action of FFEI projections, we examined synaptic currents generated by a 100 Hz train of input spikes when delivered either by FFE or FFEI projections ( Figure 3A). To study the influence of different amounts of inhibition on transmission, we also varied the relative weight (total area under conductance curve per presynaptic spike) of the inhibitory conductance to the excitatory conductance and called this quantity α.  (G) Inhibitory scaling coefficients α that produced the strongest transmission for each background noise level shown in (F). For these models, moderate inhibition (1.5 ≤ α ≤ 2) produces greater transmission when excitatory background noise is weak. As background noise becomes stronger, the model performs best with slightly scaled inhibition (α = 1.25). (H) The influence of scaling the inhibitory current at 50 Hz when the reversal potential of the leak current was increased, reducing the threshold. Unlike when noise was increased, there is no dependence of the transmission on the parameter α. (I) Optimal values of α for different leak reversal potentials.
The synaptic currents and postsynaptic potentials generated by excitatory-only projections were highly blurred in time and exhibited substantial temporal summation compared to those delivered by FFEI projections. In order for output spike timing to be precise, the input must be very brief in time, and synapses with NMDA receptors cannot support this tight precision. FFEI projections delivered pulse-like current injections that were concentrated in time in a manner that varied with the total amount of inhibition α. However, sharp, high frequency inputs are highly filtered by the neuronal membrane, so the brief input to the neuron must have very high amplitude. This can be produced by strong E and I synaptic weights, and the delay between the two serves to allow enough rapid current to produce spikes without causing a long-lasting trend in the membrane voltage. Therefore, while inputs at high frequencies are highly filtered, the large, sharp current pulses produced by the FFEI input configuration allow spiking transmission at those high frequencies despite the high filtering.
To examine the optimal amount of feed-forward inhibition, we plotted the transmission of feed-forward signals as a function of input modulation frequency (Figures 3B-D). As before, synaptic weights were set so that transmission at an input modulation of 5 Hz was equal to about 75 Hz. Transmission at high frequencies was most efficacious when α was between 1.5 and 3. Larger values of α suppressed transmission because the level of inhibition was too high.
In many neural circuits, the ongoing activity provides a background input that might influence the feed-forward transmission of information. To investigate how these background signals might influence the level of inhibition needed for optimal feed-forward transmission, we modeled this background input as 50 noisy independent Poisson inputs, with varying excitatory synaptic input strengths (Figures 3F,G). As the level of ongoing background input increased, relatively smaller weights were needed on the feed-forward inhibitory synapse for optimal transmission.
One might imagine that the reduced weights needed for optimal transmission in the high noise case could arise from either the mean increase in excitation in the noise case, the variation provided by the noise, or both. To test the dependence on the overall mean excitation, we simulated a family of noiseless models where the reversal potential of the leak current was adjusted, effectively reducing the threshold and increasing the cell's excitability (Figures 3H,I). The optimal weight of the inhibitory input did not change with the change in excitability. This result suggests that the increase in the mean excitability in the noise cases did not influence the optimal value of α, but, instead, that it was the variation in the input that needed to be balanced with less inhibition.

Influence of Inhibitory Delay on Feed-Forward Transmission
Another factor that should have a large influence on the temporal capabilities of feed-forward synapses is the delay between the feed-forward excitatory and inhibitory inputs. To examine this dependency, we varied the delay between excitation and inhibition systematically for a wide range of input modulation frequencies (Figure 4). Shorter delays allowed transmission at much higher temporal frequencies, and a delay of 20 ms produced a transmission profile that was almost as slow as excitatory inputs alone. Therefore, a short delay between feed-forward excitation and feed-forward inhibition is necessary to achieve the high temporal frequency transmission characteristics reported here.

Paired Feed-Forward Excitatory and Inhibitory Inputs Act as a Clock or Trigger That can Organize Feed-Forward Computation Across Layers or Networks
Neural circuits differ from conventional digital electronic circuits in that there is no external clock signal that tells neurons when they should begin evaluating their inputs (von Neumann, 1963). In digital electronics, a gate only updates its output when the clock signal indicates that it should re-scan its inputs. One classic idea suggests that neural circuits operate by attractor dynamics and therefore do not have a need for clocks (Hopfield and Tank, 1986;Miller, 2016). However, attractor dynamics take time to converge, but organisms are able to make extremely fast decisions in a few 100 ms post-stimulus (Thorpe et al., 1996;Sherwin et al., 2012). A feed-forward trigger would be useful for performing such fast computations. Here we examined whether paired feed-forward excitation and inhibition could provide temporal organization of their own input, in essence providing the feed-forward input values (''what to evaluate'') and clock timing information (''when to compute'') in a single mechanism.
We studied transmission across four cells in successive network levels in three different models. One might imagine that these cells represent the same receptive field location across these four network levels, so they have direct connections. However, each cell was also assumed to have local, ongoing excitatory input from its local network that was modeled as 50 noisy Poisson inputs that were not related to the feed-forward input. In one model, these four neurons were not connected at all, so that the baseline activity of these neurons could be studied as a control. In another model, these neurons were connected using FFE projections. These connections represent the simplest feed-forward cross-network computation that we could imagine, which is y = x. This computation was chosen because of its ease of evaluation, but these results should extend for any convergent feed-forward computation where the output is a function of the inputs at a particular time. Finally, we examined a model where cross-network transmission was provided by FFEI synapses across the four model network levels.
We evaluated how well the cells could transmit information forward to their subsequent levels, despite the ongoing local input that was unrelated to the feed-forward inputs. We assessed the temporal fidelity of the computation using the same frequency analysis that we performed in Figure 1. We provided an input firing rate that was modulated at a particular frequency that was varied and examined the Fourier coefficient (Figure 5B), overall activity (Figure 5C), and the normalized Fourier coefficients that indicated how specifically the output response followed the input frequency ( Figure 5D). When FFE projections were used, the network level 4 output neuron did not efficaciously follow the input modulation frequency for frequencies greater than about 5 Hz. However, when FFEI projections were used, the network level 4 output neuron followed the input very well at input modulation frequencies as high as 100 Hz. This difference between FFE and FFEI was robust over a variety of feed-forward synaptic strengths ( Figure 5E). Even when feed-forward firing rates were set so that the output neuron was driven at 20 Hz for 5 Hz input modulation, FFEI projections provided more than an 8-fold increase in transmission compared to FFE inputs modulated at 50 Hz, and differences were more pronounced for higher synaptic strengths ( Figure 5F). The 1 2 cutoff frequency for the same network was more than 5-fold greater ( Figure 5G).
Interestingly, when FFE projections were used, the Fourier coefficient of the network level 4 output peaked several times at higher input frequencies (300-1,000 Hz). This effect can likely be credited to over-excitation of the network level 4 neuron, as seen in Figure 5A. Indeed, at most frequencies, the network level 4 neuron with FFE is likely firing at its maximum rate, due to the amplification of excitatory input over the circuit. As a result, high FC measurements at high input frequencies cannot necessarily be attributed to the circuit following the inputs, but may instead result from the physical features of the neuron allowing for maximal firing at these rates. In either case, the Fourier coefficient peaks at these high input frequencies do not stand out from the overall power of the output response, as shown in Figure 5D.
One may ask how much of these frequency responses could have arisen by chance due to the random activity that is provided to each network level. The fold increase in Fourier coefficient responses at the input modulation frequency is shown in Figure 5H. Even at the lowest synaptic strength tested, FFEI projections provided responses that were at least five times those of the noise-only network, and this difference greatly increased when input strength was increased.
So far, we have only investigated how inputs of fixed modulation frequency are transmitted over multiple network levels. While this addresses how different network levels can exhibit synchronized activity, it does not directly confront how these networks can adapt to changing inputs. We, therefore, constructed a rectified sinusoidal Poisson input with a changing modulation frequency over time ( Figure 5I). After changing the input modulation frequency, we held the input modulation frequency constant for 2 s in order to produce a reliable estimate of the Fourier coefficients. We then provided this input to the FFE and FFEI models and examined the normalized FC spectrum over time (Figures 5J,K), which measured how closely the network level 4 output followed a range of frequencies over the course of the input. With FFE projections, the network level 4 output only noticeably followed the stepping input at low frequencies (5-10 Hz). Meanwhile, with FFEI projections, the network level 4 output followed the entire stepping input fairly well. Furthermore, the network level 4 output did not exhibit significant residual power at frequencies represented earlier in the input, suggesting that the FFEI model was capable of quickly adapting to the changing input frequency. To more closely examine the speed at which the FFEI model transitions between inputs of different modulation frequencies, we provided the model with a Poisson input that increased from F = 10 Hz to F = 50 Hz at time = 1 s, and computed the FC spectrum of the network level 4 output over smaller time intervals (0.2 s; Figure 5L). With this increased temporal resolution, the FFEI model rapidly adapts to an increase in the input frequency.
Taken together, these results indicate that FFEI projections organize multi-level, hierarchical computations that are also being driven by ongoing incidental activity. Further, FFEI projections allow these computations to occur with high fidelity even at high temporal frequencies. In this way, FFEI projections serve to deliver both a signal as well as the timing signal necessary to organize feed-forward computation across layers of processing.

Paired Inputs of Opposite Signs can Organize Computation in Neuromorphic Circuits
We have already noted that paired feed-forward excitation with slightly delayed inhibition is a common circuit motif in the brain, but the concept can be used more generally for computations with either sign. Although most neuronal circuits of which we are aware follow the excitatory-before-inhibitory configuration, there are some places, such as chandelier cells in the somatosensory cortex (Zhu et al., 2004), where inhibitory input precedes excitatory input. The paired E/I or I/E motifs can be used in artificial circuits to organize feed-forward computations at high speeds that exceed the time constant of integration of the individual elements.
We demonstrate the use of mixed feed-forward inputs in a model LIF circuit that performs the exclusive-or (XOR)  computation of two inputs (Figure 6). In this computation, the output neuron should exhibit a response if either of the two inputs is positive, but it should be silent if there is no input or if both inputs are active. To transmit a positive postsynaptic signal, we used FFEI synapses employing excitation that was followed by identical but delayed inhibition ( Figure 6A). To transmit a negative signal, we employed inhibition that was followed by identical but delayed excitation. Input weights for cells in the first layer were set so that one input was positive and the other input was negative. Further, the input with a weight that was positive in the first cell had a negative weight in the second cell. The neuron in the second layer responded if either of the cells in the first layer responded.
Once again, we provided input at different modulation frequencies, although we shifted the two inputs so that they were 180 • out of phase ( Figure 6A). Therefore, the XOR output should be produced at twice the input frequency (the cell should respond strongly when either input is at its positive phase, and weakly or not at all when the inputs are turned off). Once again, we found that the circuit with mixed FFEI projections could follow the XOR computation over high frequencies, while a comparison FFE circuit fell off rapidly with frequency, for both the raw response ( Figure 6B) and for the fraction of the total response ( Figure 6C) that was at the stimulus frequency ( Figure 6D). We compared another input case (with the two inputs shifted by 90 • ) in Supplementary Figure 2.
We conclude that the concept of mirroring a feed-forward input with a delayed negative copy can organize high frequency computations across multi-layer neuromorphic circuits in much the same way that a clock organizes computations across digital electronic circuits.

In Principle, AMPA-Only Currents Allow High Frequency Computations
Most excitatory synapses in the mammalian brain employ a combination of the AMPA and NMDA receptors. AMPA receptors respond rapidly and briefly to neurotransmitters, while NMDA receptors remain open for longer periods of time (50-150 ms). However, there are a few instances of excitatory synapses that primarily involve AMPA receptors. One example is the calyx of Held synapse in the mature mammalian auditory brain stem; notably, this region often processes high frequency temporal information in the form of auditory input (Nakamura and Cramer, 2011). This raises the question of whether neural circuits could, in principle, achieve the same fast computation as FFEI by using feed-forward inputs that were comprised entirely of fast AMPA channels (see also Wolf et al., 2005).
To address this question, we compared feed-forward transmission in synapses that were comprised of AMPA receptors only, AMPA and NMDA receptors only (like FFE), and AMPA, NMDA, and GABA receptors (like FFEI) in Figure 7. We again provided rectified sinusoidal input at an input modulation frequency, and examined the Fourier coefficient of the response at the input frequency (Figure 7B), the average response across all frequencies (Figure 7C), and the normalized output frequency (Figure 7D). Unlike in previous simulations with feed-forward excitatory transmission alone, accurate high frequency transmission was possible when only fast-closing AMPA channels were present. However, when slow-closing NMDA channels were added, the excitatory conductance and the spiking response became blurred, leading to poor transmission of high frequency information. High frequency transmission was restored by inhibition via GABA channels.
To investigate the effects of local background activity on the ability of the synapses to follow high frequency inputs, we provided 50 noisy Poisson inputs, modeled as in Figure 5, and examined the output response (Figures 7E-G). In the presence of excitatory noise, the FFEI synapse overall followed rectified sinusoidal inputs more effectively than the AMPA-only feed-forward synapse. These observations can be attributed to the overall output power, which was somewhat higher for the AMPA-only feed-forward synapse than for the FFEI synapse ( Figure 7F).
These results suggest that AMPA-only feed-forward synapses provide some of the benefits of FFEI with respect to high frequency computation. However, FFEI synapses are more effective than AMPA-only feed-forward synapses when noisy inputs are also provided, suggesting that inhibitory GABA channels may play a role in filtering out excitatory noise and thus overriding ongoing computations.

High Frequency Transmission in a Triad Synapse Model With Synaptic Depression
Up to now, we have not considered an important feature of synaptic transmission in real triad synapses in the LGN: short-term dynamics Tsodyks and Markram, 1997;Varela et al., 1997;Carandini et al., 2002;Swadlow et al., 2002;Boudreau and Ferster, 2005;Higley and Contreras, 2006). The retino-geniculate excitatory and inhibitory synapses, as measured ex vivo in mice, both exhibit substantial short-term depression (Eysel, 1976;Koch, 1985;Chen and Regehr, 1999;Regehr, 2003, 2005). We developed a depression model for excitatory and inhibitory synapses based on experimental data from mice (Chen and Regehr, 1999;Regehr, 2003, 2005) using depression equations from Varela et al. (1997), and added AMPA, NMDA, GABA A, and GABA B currents (Supplementary Figure 3). NMDA and GABA B currents have long time courses, and it is necessary to have compatible time courses for the paired feed-forward input to allow high frequency transmission.
Adding depression dynamics also meant that we needed to consider the state of the synapses at the beginning of the simulation. In vivo in the cat, retinal ganglion cells exhibit relatively high spontaneous firing rates of tens of spikes/sec (Kuffler, 1953), so it is unlikely that retinal ganglion cell synapses would commonly be in a completely undepressed state; the neuron would need to be shown a stimulus that suppresses its firing, such as showing an ON spot to an OFF-center neuron, to achieve such a state. Therefore, although we began our simulations with the synapses in a completely undepressed state, we did not analyze the output until after 1 s had elapsed.
Representative simulations for synapses with AMPA + NMDA (FEI) and AMPA, NMDA, GABA A , and GABA B (FFEI) receptors are shown in Figures 8A,B. When the simulations begin, there is strong excitatory transmission, but excitation and inhibition become more balanced as the simulation runs. Synaptic weights were chosen for excitatory and inhibitory synapses so as to allow moderate firing in the FFEI case after 1 s of simulation; inhibitory inputs were removed for the FFE case, and the excitatory weights were adjusted so that the firing rate for inputs at 5 Hz modulation exhibited 70 Hz after 1 s of simulation, matching the FFEI simulations. With synaptic depression, transmission (Figures 8C-E) is improved for FFEI synapses compared to the simulations without synaptic depression, consistent with previous theoretical analyses . However, transmission at higher temporal frequencies is better with the FFEI synaptic arrangement as compared to FEI.
We modulated the input rate sinusoidally from 0 to 50 spikes/s, a reduction from the 0 to 100 spikes/s we used in the prior examples. Blitz and Regehr (2005) noted that high frequency spike trains (50-100 Hz) caused such substantial depression that the inhibitory synapse effectively stopped transmitting. Here, it is important to note that we are modulating the firing rate probability of the input sinusoidally,  Model circuits utilizing either feed-forward single-sign inputs only (center, top) or paired mixed-sign inputs (center, bottom) to perform an exclusive-or (XOR) computation. The cell in layer A2 fires only when either Poisson input i 1 or i 2 (dark yellow) is active, but not both. A 0.6 s simulation of each model circuit with inputs driven out-of-phase (π) is shown with input parameters F = 5 Hz, PR = 100 Hz. The out-of-phase input should drive the XOR circuit strongly at twice the input modulation frequency because one input is firing when the other is not at two phases in the input wave.The simulated output spikes of cells 1 and 2 in A1 and the cell in A2 are shown to the right for each model circuit (single-sign: red, mixed-sign: blue). The expected output of the XOR computation between i 1 and i 2 (purple), calculated in 10 ms or 5 ms windows, is shown below for comparison with A2 outputs. (B) A2 output Fourier coefficient at 2 × F (FC F ) for a range of input modulation frequencies F. (C) A2 output mean Fourier coefficient. (D) A2 output normalized Fourier coefficient. Dashed line indicates FC F /FC avg = 1. and a 50 Hz peak firing rate probability modulated sinusoidally will rarely result in spikes that have an inter-spike-interval of 20 ms (1/50 Hz). Visual firing rates are typically quantified in a trial-averaged fashion and expressed as the probability of firing in a small time bin, and we are generating spikes according to such a process. At sustained high frequency input rates >50 Hz, it is likely that transmission of both excitation and especially inhibition would be very reduced, although it is unclear how common this situation of sustained high input rates would be in vivo, or if synaptic depression is altered in vivo or across species.

Transmission Is Limited by Inhibitory Neuron Properties When FFEI Is Provided at the Circuit Level
While the FFEI motif is sometimes found in inputs onto individual neurons, such as in the retinogeniculate triad synapse, the FFEI motif is also commonly found in inputs to circuits such as in the hippocampus (Buzsaki, 1984;Pouille and Scanziani, 2001;Bhatia et al., 2019) or the cerebral cortex, both in inputs from the thalamus (Agmon and Connors, 1991;Swadlow, 2003) and interareal projections (Yang et al., 2013). In these networks, excitatory projections from other areas make connections with separate populations of excitatory and inhibitory neurons, and the targeted inhibitory neurons provide rapid but slightly delayed inhibition onto the recipient excitatory neurons. Further, the excitatory and inhibitory neurons typically receive converging inputs from many cells, so that in principle the excitatory and inhibitory inputs to the local circuit may not be entirely matched. That is, the strong 1-to-1 relationship between each feed-forward excitatory postsynaptic potential and each feed-forward inhibitory postsynaptic potential that is found at the triad synapse is not present in circuit-level projections; instead, feed-forward excitation and inhibition are more loosely related. Before we begin studying how FFEI can impact temporal computations at the circuit level, it is worth pausing to consider whether all cortical circuits might take advantage of any properties that we uncover. In the thalamus, it is clear that visual, somatosensory, and auditory neurons can follow sensory inputs at high frequencies. However, in visual (Hawken et al., 1996), somatosensory (Chung et al., 2002), and auditory cortex (Creutzfeldt et al., 1980;FIGURE 8 | In triad synapse models exhibiting synaptic depression, paired feed-forward E/I enhances high frequency transmission. (A,B) Two triad synapse models, in which the LGN neuron receives inputs via different sets of channels exhibiting synaptic depression. Each model is simulated for 2 s with input parameters F = 20 Hz, PR = 50 Hz. Postsynaptic conductance (excitatory: light green, inhibitory: gray) and the membrane potential of the LGN cell (blue) are simulated in response to RG and interneuron input. LGN firing times (red x) and postsynaptic response is compared to the underlying input characteristics (dashed yellow). The output response is divided into 2 components over time: a 1 s period of initial activity following nonphysiological silence (gray highlight) and the subsequent long-term activity (pink highlight).  Wehr and Zador, 2005), cortical neurons primarily respond to stimulation at much lower stimulus repetition frequencies (<20 Hz) than do thalamic neurons. That is, the primary sensory cortex does not seem to segment responses to sensory stimulation at frequencies much higher than 20 Hz. However, in both sensory and higher cortical areas and in the hippocampal formation, 40 Hz activity (gamma activity) is prominent (Fries et al., 2007) and can be induced by driving feed-forward interneurons (Cardin et al., 2009). It has been suggested that in hippocampal circuits, each gamma cycle might encode representations of individual items (Lisman and Idiart, 1995;Fries et al., 2007;Lisman and Jensen, 2013), and 40 Hz sensory stimulation has been reported to be protective against neurodegeneration (Iaccarino et al., 2016). So we feel it is worth considering whether circuit-level FFEI input can allow circuits to follow inputs at such high frequencies.
Here, we examined a model network with 100 feed-forward excitatory inputs that arrived at an LIF excitatory cell and an LIF inhibitory cell ( Figure 9A). Following our previous methods, the firing rate of each input was modulated sinusoidally at an input modulation frequency. In this model, all inputs were driven at the same phase ( Figure 9B). We examined the Fourier coefficient of the output responses at the input modulation frequency (Figure 9C), the average response across all frequencies (Figure 9D), and the ratio of these quantities ( Figure 9E).
The circuit-level FFEI exhibited some similarities to and differences from the single FFEI input situation. One difference is that the circuit's I neuron receives excitatory input without a delayed inhibitory copy and acts just like a neuron receiving FFE input. Therefore, the frequency responses of the inhibitory neuron are the same as the E-only case in Figures 9C-E. If we look at these responses, we see that they fall off much like FFE input to the triad synapse: the response is much attenuated at about 25 Hz, for example. At input modulation frequencies greater than this, the I neuron will not respond strongly to the excitatory input ( Figure 9B, right), so the I neuron's contribution to the circuit is not present at high frequencies, and the circuit behaves like an FFE circuit. This means that the circuit-level FFEI projection has a much lower cut-off response frequency than a similarly-endowed triad-like FFEI projection.
We suspected that the membrane time constant of the inhibitory neuron would be a key parameter in determining the cut-off frequency for the circuit-level FFEI projection, so we compared responses in models where this parameter was varied. In this circuit network, the level of response output is very sensitive to the synaptic strengths of the inputs as well as the strength of the synapse from the inhibitory neuron onto the excitatory neuron. Because there were two non-linear ways to tune the responses to a particular input rate, we simply left the synaptic strengths unchanged while we altered the membrane time constant of the inhibitory neuron so that the models could be compared on the basis of that parameter.
In Figure 9F, we show the normalized Fourier coefficient for these models but with the data scaled so that the output response for an input modulation of 5 Hz is equal to 1. Here it is apparent that the frequency response depends strongly on the inhibitory membrane time constant, with low values corresponding to high transmission at high input modulation frequencies. When the inhibitory time constant was 10 ms, which is consistent with experimental studies of cortical interneurons (Economo and White, 2012), the 1 2 frequency cut off was 40 Hz, indicating that the excitatory cell could follow input responses very effectively up to that frequency ( Figure 9G). This 1 2 cutoff frequency was 70% greater than the 1 2 cutoff frequency for a circuit with FFE projections alone ( Figure 9H).
We have just seen that the properties of the feed-forward inhibitory interneuron are critical to the temporal frequency limits of a circuit with paired feed-forward excitation and inhibition arriving via external excitatory input. In some circuits, the AMPA to NMDA current ratio is larger in feed-forward interneurons than in excitatory neurons (Wang and Gao, 2009). We have seen in Figure 8 that AMPA-only inputs allow high temporal frequency transmission, and we asked whether providing AMPA-only excitatory input to the feed-forward inhibitory neuron would increase the cut-off temporal frequency. In Figure 10, we reduced the synaptic time constant of the excitatory input to the inhibitory neuron from 20 ms to 1.6 ms to reflect an AMPA-dominated synapse. Indeed, the E neuron in the circuit can now follow much higher temporal frequencies, up to about 90 Hz. These results indicate that the properties of the interneuron and its input are the key limiting factor to high temporal frequency transmission in these circuits.

DISCUSSION
We have shown that feed-forward excitation and delayed inhibition can enhance the transmission of high frequency temporal information through neural circuits and that this enhancement occurs for a broad range of synaptic and membrane parameters. Similar to the triad synapses in retinogeniculate projections, FFEI models with low input convergence and tightly paired E/I inputs exhibit robust temporal transmission over a wide range of frequencies up to ∼100 Hz. Meanwhile, in circuits modeled after hippocampal and cortical projections, FFEI enhances high frequency transmission up to about 40-80 Hz, despite high input convergence (N = 100) and a looser association between excitatory and inhibitory inputs. In circuit models of hierarchical networks, we found that FFEI projections reliably transmit high frequency signals and organize activity across network levels over time, suppressing the influence of ongoing activity and essentially acting as a feed-forward clock trigger. Thus, FFEI projections can act as a hybrid input, conveying both information about input value and imposing a clock-like trigger to initiate rapid computation.

FFEI Allows Computation at High Temporal Frequencies That "Break the Limit" of the Membrane Time Constant
Excitation with delayed inhibition produces a narrow time window during which spiking can occur. As a result, FFEI inputs can enhance the temporal fidelity of neural circuit outputs, as shown in experiments in hippocampal CA1 (Pouille and Scanziani, 2001;Bhatia et al., 2019) and at the retinogeniculate synapse (Blitz and Regehr, 2005). Given the high temporal precision of networks containing FFEI projections, Circuit-level feed-forward E/I input allows excitatory neurons to follow higher temporal frequencies than those receiving feed-forward excitatory input alone, but the cut off frequency values are smaller than for the triad synapse, where the interneuron cell body integration does not intervene.
it follows that such networks would accurately transmit high frequency temporal information, as well. This effect can largely be attributed to the narrow spiking time window generated by paired E/I inputs, as previously described (Pouille and Scanziani, 2001;Wehr and Zador, 2003;Blitz and Regehr, 2005;Mittmann et al., 2005;Higley and Contreras, 2006;Cardin, 2018). Here we show that the residual inhibition following each input spike does not suppress the cell for long, and the  FIGURE 10 | AMPA-only input to feed-forward inhibitory neurons increases the temporal frequency capabilities of multi-input models. Panels (A-F) as in Figure 9, except that the inhibitory interneuron receives input with a time constant of 1.6 ms, reflecting AMPA-only input to the I cell, while the E cell receives input with a longer time constant (20 ms) reflecting AMPA and NMDA inputs. Now, the inhibitory neuron can fire with a modulation input of 90 Hz, and the excitatory neuron can achieve good modulation by the input at higher temporal frequencies (green curve in C-F). These experiments show that the properties of the inhibitory input (synaptic time constants or membrane time constants) are a key factor in determining the temporal frequency cut off of the circuit.
cell is quickly ready to respond to subsequent FFEI inputs ( Figure 3A). Networks that propagate signals with high temporal fidelity have been called synfire chains (Abeles, 1991). We have shown here that excitatory-only feed-forward networks can produce reliable computation only at relatively low temporal frequencies.
Excitatory-only feed-forward networks can reliably and quickly transmit firing rate information if each area receives ongoing balanced excitatory and inhibitory internal noisy input (Shadlen and Newsome, 1998;Chance et al., 2002;van Rossum et al., 2002;Murphy and Miller, 2009) or has detailed or loosely balanced strongly recurrent connections (Tsodyks and Sejnowski, 1995;Sompolinsky, 1996, 1998;Hennequin et al., 2017) or balanced connectivity across cortical areas (Joglekar et al., 2018). Here we have shown that this transmission is possible with FFEI projections even if the random local input that a local area experiences or the internal connectivity of the network is not balanced: transmission at high temporal frequencies can be accomplished with balanced feed-forward input alone.
It should be noted that some brain pathways use different strategies to transmit high frequency information. In the auditory brainstem, neurons respond rapidly to a transient input (Schnupp and Carr, 2009), which can be explained by the composition of the intrinsic ionic currents in the cell that allow them to act as ''slope detectors'' (Meng et al., 2012), although they do not respond tonically to constant current, as LGN neurons do (Suzuki and Rogawski, 1989).
The high transmission functions of FFEI could act in concert with another proposed function for the triad synapse: a state-dependent ''veto'' (Koch, 1985). Under this idea, the synapse would act as described here when operating in a feed-forward mode, but the inhibitory neuron could be activated independently in order to suppress feed-forward excitatory input to support functions such as reduced attention or internal processing. A related feed-forward excitatory/inhibitory gate model was demonstrated by Kremkow et al. (2010a). In recurrent, balanced cortical networks, inhibitory synaptic strengths (Vogels and Abbott, 2009) or the correlation between excitatory and inhibitory signals can be used to gate the propagation of tonic activity through downstream networks, and this can be combined with temporal gating (Kremkow et al., 2010a). Learning rules (Vogels et al., 2011;Kleberg et al., 2014) could tune inhibition for ideal propagation at high temporal frequencies (see Figure 3).

FFEI as a Feed-Forward "Clock" Trigger Mechanism
The absence of a synchronizing clock or trigger signal to organize the activity of neurons would seem to imply that neuronal circuits would be limited to performing computations that do not require such a trigger, such as attractor networks or integrators (Hopfield, 1982;Seung et al., 2000;Miller et al., 2003;Miller, 2016). FFEI acts much like a feed-forward synchronizing trigger that forces computation to occur within a short window (Buzsaki, 1984;Agmon and Connors, 1991;Swadlow and Gusev, 2000;Porter et al., 2001;Swadlow, 2003;Rock and Apicella, 2015;Bhatia et al., 2019).
Here we showed that FFEI when used in projections across brain areas or layers, can organize computation across these regions, even when these regions are experiencing ongoing local activity (Figure 5). Unlike a digital computer, where the clock trigger runs at a fixed rate, FFEI allows the trigger signal to be generated at any time ( Figure 5I-K) and at many rates, up to 100 Hz for triad-like synapses and up to about 40-80 Hz for circuit-level FFEI inputs, depending upon the assumptions of the model. In low-level areas, these triggers would be generated when sensory input arrives, and computation could proceed at the speed of the sensory input. In higher-level areas, computation across two areas could proceed whenever the input area generates output and could underlie coordination of high frequency oscillations across areas (Lisman and Idiart, 1995;Lisman and Jensen, 2013).
The degree to which high frequency information actually travels throughout the brain varies depending upon area and channel and has been well studied in the visual system. In recordings that examined velocity tuning to single bar sweeps in both retina and LGN of X and Y cells in the cat, peak velocity tuning was slightly lower in LGN for X cells, but very comparable in retina and LGN for Y cells, with preferred velocities that ranged from about 20 • /s to 100 • /s (Frishman et al., 1983), indicating that spike timing information can be precisely transmitted. Consistent with this idea, retinal inputs activate LGN cells with high efficacy (Usrey et al., 1998), indicating that LGN neurons follow retinal ganglion inputs with high reliability.
Evidence from sensory cortex, however, suggests that the thalamocortical connection does not take full advantage of the potential FFEI speeds. There is a drop in peak temporal frequency tuning between LGN and cortex, with most LGN cells exhibiting strong firing for many frequencies, including those greater than 16 Hz, while few cortical neurons exhibit strong firing at 16 Hz or greater (Hawken et al., 1996;Van Hooser et al., 2003Heimel et al., 2005). If FFEI were the only factor, we would predict that transmission should be quite robust at 20 Hz (Figure 9), but the visual cortex attenuates these frequencies quite strongly. This drop also occurs in the somatosensory (Ahissar et al., 2001;Chung et al., 2002) and auditory systems (Creutzfeldt et al., 1980). Synaptic depression or other cortical circuit mechanisms may underlie this filtering (Chung et al., 2002;Swadlow, 2002;Boudreau and Ferster, 2005;Swadlow et al., 2005;Wehr and Zador, 2005).
Nevertheless, many cortical regions exhibit strong modulation and resonance at around 40 Hz, also known as the gamma frequency (Fries et al., 2007;Cardin et al., 2009;Lisman and Jensen, 2013). If responses at 40 Hz in one area are to be intelligible in another cortical or hippocampal area, then it is necessary for these circuits to be able to follow inputs at these frequencies. Here we showed that circuit-level FFEI inputs can in principle allow such rapid communication.

Predictions
The FFEI models make a simple prediction: the absence of inhibition at cells receiving FFEI inputs should limit the ability of these cells to follow inputs at high temporal frequencies. An optogenetics approach could be used to control the activation of retinal ganglion cells (RGC) with high temporal precision, and the activity of LGN relay cells in response to RGC action potentials could be measured. A GABAzine infusion to block inhibition in the LGN should limit the ability of LGN relay cells to follow RGC inputs at high temporal frequencies. A similar experiment could be performed at thalamocortical slices in the somatosensory cortex (Agmon and Connors, 1991) in order to test the high-convergence regime. While we have noted that there is a high degree of synaptic depression in thalamocortical synapses (Chung et al., 2002;Higley and Contreras, 2006), the data from an experiment should exhibit a strong shift, as in Figure 9, from the FFEI dynamics towards E-only dynamics. One experiment that studied the propagation of signals through the somatosensory cortex in vivo found that spike timing transmission precision decreased at low firing rates (but not for high firing rates) when PV interneurons were silenced (Jang et al., 2020).

CONCLUSION
Feed-forward pairing of excitatory and inhibitory connections provides several functions for neural circuits. It helps to linearize computations that are impacted by rectification in the retina (Werblin, 2010), LGN, and cortex (Hirsch et al., 2015). It sharpens stimulus tuning and spike timing (Pouille and Scanziani, 2001;Wehr and Zador, 2003;Blitz and Regehr, 2005;Mittmann et al., 2005;Higley and Contreras, 2006;Cardin, 2018), and allows neurons to respond to weak inputs without saturating for strong inputs (Pouille et al., 2009). In the hippocampus, the delay between excitation and inhibition shortens with stronger CA3 inputs, producing smaller changes in CA1 membrane potential (Bhatia et al., 2019). Therefore feed-forward excitation and inhibition allow neurons to accommodate a dynamic range of inputs before firing, providing a mechanism of gaining control of these circuits (Pouille et al., 2009;Bhatia et al., 2019). Here we add that it provides a dual signal-the value of its input on the one hand and a clock-like trigger on the other-that allows high frequency feed-forward computation that is synchronized across brain areas.

DATA AVAILABILITY STATEMENT
The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found below: https://github.com/VH-Lab/vhlab-ffei-matlab.

AUTHOR CONTRIBUTIONS
AC performed modeling. AC and SVH designed the models. AC and SVH wrote the article. All authors contributed to the article and approved the submitted version.

FUNDING
This work was funded by NIH EY022122 (SVH) and NIH DA033463 (AC).

ACKNOWLEDGMENTS
We thank Andrew Lipnick, whose undergraduate thesis on velocity tuning led us to consider feed-forward transmission, Paul Miller for useful suggestions, and members of the Van Hooser lab for comments.
SUPPLEMENTARY FIGURE 1 | High frequency transmission differs between excitation-only circuits and circuits exhibiting paired feed-forward excitation and delayed inhibition with long time constants. (A) 0.5 s simulation of a feed-forward E circuit receiving a single rectified sinusoidal Poisson input (yellow) with a peak rate (PR) of 100 Hz and an input modulation frequency (F) of 40 Hz, as in Figure 1A. (B) Same, but for a paired feed-forward E/I model circuit with τ fall, i = 50 ms instead of the baseline 20 ms shown in Figure 1A. Spiking activity in the postsynaptic neuron with τ fall, i = 50 ms is more prolonged than in the 20 ms case in Figure 1A, but the firing is still truncated by the inhibition, as compared to the no-inhibition case in panel (A).
SUPPLEMENTARY FIGURE 2 | Paired feed-forward E/I in an XOR circuit, with π/2 phase-shifted inputs. (A) A simulation of each XOR model circuit described in Figure 6A is shown with inputs modeled as in Figure 6A, but with a π/2 phase shift instead of a π phase shift, which should result in output at the stimulus frequency F (instead of 2 × F when the phase shift is π). (B) For a range of input modulation frequencies F, the Fourier coefficient of the A2 output at F (FC F ) was averaged over 10 trials for the two XOR circuit models shown in (A). (C) For a range of input modulation frequencies F, the mean power of the A2 output over all frequencies (FC avg ). (D) For a range of input modulation frequencies F, the normalized Fourier coefficient of the A2 output is shown. Dashed line indicates FC F /FC avg = 1.
SUPPLEMENTARY FIGURE 3 | Fitting depression at the retinogeniculate synapse. (A-C) The paired-pulse ratio for the AMPAR (A), NMDAR (B), and locked GABAR (C) components of the synaptic current was plotted over a range of interspike intervals. For each channel, raw data (black) obtained from voltage clamp experiments (Chen et al., 2002;Blitz and Regehr, 2005) were fitted to a depression model (see "Materials and Methods" section). Paired-pulse experiments were then simulated using the fitted depression model (red).