Temporal information was generated using a Poisson process with a rectified sinusoidal firing rate:

where PR is the peak firing rate set to 100 Hz for all simulations, F is the input modulation frequency, and rectify(x) = 0 if x < 0, and rectify(x) = x for x ≥ 0. For each set of simulations, temporal information was presented to our simulated neural circuits for a fixed length of time, independent of F.

In some of our simulations, we provided random background activity as an additional excitatory synaptic input. This activity was modeled as N_inputs noisy Poisson inputs with a uniform firing rate FR (see Table 1 for values). FR was calculated such that, for each noisy input, the probability of seeing 1 background spike during an input cycle is equal to the probability of a feed-forward input spike over one cycle (that is, the mean firing rate of the feed-forward inputs and each noisy input were balanced):

Spiking responses were converted to a firing rate per time step (Hz), and the following equation was used to calculate the Fourier coefficient magnitude at a particular frequency (Crawford, 1968):

where Δt is the time step size in seconds, L is the length of the spiking response in seconds, t_index is time as an integer index, R(t) is the spiking response in Hz, and F is a particular frequency.

To normalize the power of spiking responses at a particular frequency, we used the average power calculated over all frequencies (FC_avg) as a normalization factor. For a time interval of length L in time steps of Δt, 1/Δt and 1/L are the maximum and minimum frequencies that can be represented, respectively. Therefore, FC_avg was determined by averaging FC_F over F = [0,1/Δt] in intervals of 1/L. The normalized frequency response was then expressed as FC_F/FC_avg. In cases where FC_avg = 0, the normalized frequency response was set to 0.

Unless otherwise stated, FC_F, FC_avg, and FC_F/FC_avg were each averaged over 10 trials for each model.

When varying the inhibitory fall time constant (τ_fall,i) in our triad synapse models, we scaled P_max,i relative to P_max,e, such that the total excitation and inhibition are balanced:

The above equation was derived by first integrating P_e(t) and P_i(t) for a single spike at t = 0 over the interval t = [0,∞], as a measure of the total conductance due to a single spike. P_max,i values were then obtained by setting the total excitation equal to the total inhibition,and then solving for P_max,i given a set of time constants and excitatory parameters.

In Figure 6, we examine the output of a circuit designed to perform an exclusive-or (XOR) computation of two inputs. The expected output of the circuit was determined by performing the XOR computation on sliding windows of length t_bin = 10 ms or 5 ms. If only one of the inputs fires during the time interval [t−tbin2,t+tbin2], then output is expected at time t.

Paired-pulse ratio data for AMPA and NMDA (Chen et al., 2002) and for GABA (Blitz and Regehr, 2005) in the retinogeniculate synapse were fitted to the following depression model (Varela et al., 1997):

In the above equation, A is a scaling factor on the amplitude of the synaptic current generated by each presynaptic spike, A₀ is the undepressed scaling factor, and both D₁ and D₂ are depressing factors with default values of 1. Following each presynaptic spike, D_i is set to D_id_i, and D_i decays back to 1 with a time constant τ_di (see Table 1 for values and Supplementary Figure 3 for fits).

The code for the article is at https://github.com/VH-Lab/vhlab-ffei-matlab.

Not applicable to the computational models studied here.

We began with a simple model inspired by the retina-LGN triad synapse that is particularly common at synapses between retinal × cells and LGN × cells (Koch, 1985; Hamos et al., 1987; Lam et al., 2005; Bickford, 2019). At these synapses, excitatory retinal input is provided to an LGN neuron and to an adjacent terminal bouton of the dendrite of an interneuron. The interneuron dendrite, which is electrotonically isolated from its soma (Morgan and Lichtman, 2020), also contacts the LGN cell at very nearly the same location as the excitatory cell, providing an only slightly-delayed inhibitory input to the LGN neuron (Cox et al., 1998). To build our model, we included a feed-forward glutamatergic excitatory synaptic current with temporal dynamics that were set to be between the fast-decaying AMPA current and the slower-decaying NMDA current observed at triad synapses (Chen et al., 2002). Fast GABAergic currents were included with a 1 ms delay, also following experimental measurements (Blitz and Regehr, 2003, 2005). While the total synaptic conductance was varied, the maximal conductances of the excitatory and inhibitory synapses were selected so that the total conductance (the area under the curve) of excitation and inhibition were identical; that is, the total integrated conductance of excitation and inhibition were balanced.

To assess the transmission of temporal information in this model, we simulated the spiking responses of a leaky integrate-and-fire neuron to input spikes generated via a Poisson process with a rectified sinusoidal firing rate (Figures 1A,B). The rate at which spikes were generated varied in time sinusoidally from 0 spikes/s to 100 spikes/s and the frequency of the sinewave was termed the modulation frequency. At each 0.1 ms, a random number (0–1) was drawn, and a spike was generated if the random number was less than the product of the rate and the bin size (0.1 ms). While the FFE configuration produces extra spikes that are outside the sinusoidal input modulation, output spikes in the FFEI input configuration are well locked to the sinusoidal input modulation. We examined the transmission of the FFEI projection over a range of inhibitory synapse fall time constants that varied from 20 to 50 ms, and compared its transmission to the case where only feed-forward excitatory (FFE) input was provided. Because changing the fall time constant of the inhibitory synaptic current, while keeping integrated E and I conductance fixed, alters the fraction of the excitatory conductance that arrives earlier in the postsynaptic response, the overall responsiveness of the cell also varies as we adjusted the inhibitory synapse fall time constant. To compare equally responsive models, we adjusted the feed-forward synaptic weight so that each model generated a response of about 75 Hz when driven with an input modulating frequency of 5 Hz.

FFEI projections demonstrated a remarkable ability to transmit high frequency information. To measure transmission quality at each input frequency, we calculated the Fourier coefficient (FC) of the spiking response at the input modulation frequency (FC_F; Figure 1C). The Fourier coefficient is higher when spikes are well locked to the stimulus, and drops as the number of spikes fired per cycle becomes lower or becomes statistical (that is, with some missed cycles). Projections with only excitatory synapses were filtered strongly at input modulation frequencies ranging from 20 to 100 Hz. In contrast, FFEI projections with inhibitory time constants of 20 ms transmitted with nearly equal fidelity from 5 to 100 Hz, and did not reach a 50% reduction in Fourier coefficient responses until the input modulation frequency reached about 400 Hz.

While the Fourier coefficient at the input modulation frequency (FC_F) measures the output transmission at that single frequency, it did not provide information about how much of the total output of the cell was concentrated exclusively at the input modulation frequency. For example, if the output cell increased its overall response only tonically, then FC_F would still exhibit an increase. To examine the fraction of the output cell’s total frequency response that was at the input modulation frequency, we divided FC_F by the average Fourier coefficient calculated over all frequencies, including frequencies much higher and lower than the input modulation frequency (Figure 1D), to produce a normalized frequency response measure (Figure 1E). This analysis indicated that FFEI projections transmitted information that was highly specific to the input modulation frequency over a wide frequency range of 5–100 Hz, while the specificity of information transmitted by excitatory-alone projections fell off rapidly with frequency.

The ability to provide specific output at the input modulation frequency was highly related to the inhibitory fall time constant. When the inhibitory fall time constant was short, then the advantage of the FFEI configuration was greater. When the time constant is small (20 ms), then the inhibition comes right on the heels of the excitation so that the excitation cannot drive the cell for a long time. When there is no inhibition, the excitation can drive the cell for a long time, which means less of the response is at the input modulation frequency. When there is a long time constant of inhibition (50 ms), then the inhibition is more spread out in time and the behavior is more intermediate, although more single feed-forward input spikes are missed by the output neuron than in the short time constant case (Supplementary Figure 1).

We plotted the normalized output responses for input modulation frequencies of 50 and 100 Hz in Figure 1G. For an inhibitory fall time constant of 20 ms, the postsynaptic neuron produced output at the input modulation frequency that was more than 12 times the amount of average output at other frequencies. As the inhibitory fall time constant was increased (and weight compensated to retain equal responsiveness), the output neuron behaved more like the excitatory-only configuration, as one would expect. We quantified the ability of the synapse to maintain its response at high temporal frequencies by identifying the temporal frequency where the output was reduced to an arbitrary threshold of 12 its value at 5 Hz (a physiologically meaningful threshold would depend on the noise of each circuit). The 12 cutoff frequency was also highly dependent on the inhibitory fall time constant (Figure 1H).

It was possible that the relative advantage of FFEI transmission compared to FFE was not general, but instead was specific to the particular levels of synaptic input strengths that were chosen in order to drive output responses of about 75 Hz at a 5 Hz input modulation frequency. To examine this possibility, we compared FFEI and FFE models that were matched at 5 Hz input modulation but over a wide variety of output response levels (Figure 1F). In all cases, the FFEI projection responses at the input modulation frequency were much stronger than those of the FFE projection over a wide range of input modulation frequencies. To quantify these changes, we examined the fold change of transmission of FFEI compared to FFE at input modulation frequencies of 50 or 100 Hz for these different levels of synaptic drive (Figure 1I). For all of these levels of synaptic drive, the FFEI projection produced at least two times as much drive at 50 and 100 Hz compared to FFE alone. The 12 cutoff frequency was also more than four times higher for FFEI compared to FFE (Figure 1J).

The ability of the output cell to reliably follow very high input frequencies was surprising because the membrane properties of neurons act as a low-pass filter, and high frequency current fluctuations are not well transmitted to membrane voltage. We compared the ability of the output neuron to follow synaptic input provided through either FFEI or FFE projections with the ability of the output neuron to follow direct sinusoidal membrane current injections of a fixed amplitude (Figure 2).

As before, we calibrated the output of the neuron to hit a target rate (75 Hz) at a 5 Hz input modulation frequency. For the FFEI and FFE simulations, we did this by setting the synaptic weight, and for the current injection simulations, we did this by setting the amplitude of the sinusoidal current injection. We compared the raw Fourier coefficient measured at the input modulation frequency (Figure 2B), the overall response across all frequencies (Figure 2C), and the normalized frequency response to measure the specificity of the output response to the input modulation frequency (Figure 2D). As expected, the cell followed the sinusoidal current injection with substantial fidelity until a critical frequency was reached, after which the input fluctuations became too small to generate action potentials. This resulted in a rapid drop in the ability of the cell to follow the output frequencies. By contrast, FFEI inputs continued to drive responses at high frequencies, well beyond the point at which the sinusoidal current could drive spikes. While the critical frequency at which spiking stops for direct current injection does depend upon the amplitude of the current used, the take-home point here is that FFEI projections drive responses at higher input frequencies when responses to FFEI and direct current injections are matched at low frequencies.

The addition of an inhibitory conductance in the FFEI case will necessarily reduce the effective membrane time constant, and one may ask whether the increased transmission is primarily due to the reduced time constant. To explore this, we modified the membrane time constant of our model neuron over a 10-fold range and examined the influence of the passive membrane time constant on the temporal frequency of transmission (Figures 2E–G). We observed very small influences of modifying the membrane time constant, indicating that a passive adjustment of the membrane time constant cannot account for increased high frequency transmission observed here.

At first glance, it seems almost paradoxical that inputs delivered via FFEI projections can apparently “break the limit” of the membrane time constant. To gain intuition into the mechanism of action of FFEI projections, we examined synaptic currents generated by a 100 Hz train of input spikes when delivered either by FFE or FFEI projections (Figure 3A). To study the influence of different amounts of inhibition on transmission, we also varied the relative weight (total area under conductance curve per presynaptic spike) of the inhibitory conductance to the excitatory conductance and called this quantity α.

The synaptic currents and postsynaptic potentials generated by excitatory-only projections were highly blurred in time and exhibited substantial temporal summation compared to those delivered by FFEI projections. In order for output spike timing to be precise, the input must be very brief in time, and synapses with NMDA receptors cannot support this tight precision. FFEI projections delivered pulse-like current injections that were concentrated in time in a manner that varied with the total amount of inhibition α. However, sharp, high frequency inputs are highly filtered by the neuronal membrane, so the brief input to the neuron must have very high amplitude. This can be produced by strong E and I synaptic weights, and the delay between the two serves to allow enough rapid current to produce spikes without causing a long-lasting trend in the membrane voltage. Therefore, while inputs at high frequencies are highly filtered, the large, sharp current pulses produced by the FFEI input configuration allow spiking transmission at those high frequencies despite the high filtering.

To examine the optimal amount of feed-forward inhibition, we plotted the transmission of feed-forward signals as a function of input modulation frequency (Figures 3B–D). As before, synaptic weights were set so that transmission at an input modulation of 5 Hz was equal to about 75 Hz. Transmission at high frequencies was most efficacious when α was between 1.5 and 3. Larger values of α suppressed transmission because the level of inhibition was too high.

In many neural circuits, the ongoing activity provides a background input that might influence the feed-forward transmission of information. To investigate how these background signals might influence the level of inhibition needed for optimal feed-forward transmission, we modeled this background input as 50 noisy independent Poisson inputs, with varying excitatory synaptic input strengths (Figures 3F,G). As the level of ongoing background input increased, relatively smaller weights were needed on the feed-forward inhibitory synapse for optimal transmission.

One might imagine that the reduced weights needed for optimal transmission in the high noise case could arise from either the mean increase in excitation in the noise case, the variation provided by the noise, or both. To test the dependence on the overall mean excitation, we simulated a family of noiseless models where the reversal potential of the leak current was adjusted, effectively reducing the threshold and increasing the cell’s excitability (Figures 3H,I). The optimal weight of the inhibitory input did not change with the change in excitability. This result suggests that the increase in the mean excitability in the noise cases did not influence the optimal value of α, but, instead, that it was the variation in the input that needed to be balanced with less inhibition.

Another factor that should have a large influence on the temporal capabilities of feed-forward synapses is the delay between the feed-forward excitatory and inhibitory inputs. To examine this dependency, we varied the delay between excitation and inhibition systematically for a wide range of input modulation frequencies (Figure 4). Shorter delays allowed transmission at much higher temporal frequencies, and a delay of 20 ms produced a transmission profile that was almost as slow as excitatory inputs alone. Therefore, a short delay between feed-forward excitation and feed-forward inhibition is necessary to achieve the high temporal frequency transmission characteristics reported here.

Neural circuits differ from conventional digital electronic circuits in that there is no external clock signal that tells neurons when they should begin evaluating their inputs (von Neumann, 1963). In digital electronics, a gate only updates its output when the clock signal indicates that it should re-scan its inputs. One classic idea suggests that neural circuits operate by attractor dynamics and therefore do not have a need for clocks (Hopfield and Tank, 1986; Miller, 2016). However, attractor dynamics take time to converge, but organisms are able to make extremely fast decisions in a few 100 ms post-stimulus (Thorpe et al., 1996; Sherwin et al., 2012). A feed-forward trigger would be useful for performing such fast computations. Here we examined whether paired feed-forward excitation and inhibition could provide temporal organization of their own input, in essence providing the feed-forward input values (“what to evaluate”) and clock timing information (“when to compute”) in a single mechanism.

We studied transmission across four cells in successive network levels in three different models. One might imagine that these cells represent the same receptive field location across these four network levels, so they have direct connections. However, each cell was also assumed to have local, ongoing excitatory input from its local network that was modeled as 50 noisy Poisson inputs that were not related to the feed-forward input. In one model, these four neurons were not connected at all, so that the baseline activity of these neurons could be studied as a control. In another model, these neurons were connected using FFE projections. These connections represent the simplest feed-forward cross-network computation that we could imagine, which is y = x. This computation was chosen because of its ease of evaluation, but these results should extend for any convergent feed-forward computation where the output is a function of the inputs at a particular time. Finally, we examined a model where cross-network transmission was provided by FFEI synapses across the four model network levels.

We evaluated how well the cells could transmit information forward to their subsequent levels, despite the ongoing local input that was unrelated to the feed-forward inputs. We assessed the temporal fidelity of the computation using the same frequency analysis that we performed in Figure 1. We provided an input firing rate that was modulated at a particular frequency that was varied and examined the Fourier coefficient (Figure 5B), overall activity (Figure 5C), and the normalized Fourier coefficients that indicated how specifically the output response followed the input frequency (Figure 5D). When FFE projections were used, the network level 4 output neuron did not efficaciously follow the input modulation frequency for frequencies greater than about 5 Hz. However, when FFEI projections were used, the network level 4 output neuron followed the input very well at input modulation frequencies as high as 100 Hz. This difference between FFE and FFEI was robust over a variety of feed-forward synaptic strengths (Figure 5E). Even when feed-forward firing rates were set so that the output neuron was driven at 20 Hz for 5 Hz input modulation, FFEI projections provided more than an 8-fold increase in transmission compared to FFE inputs modulated at 50 Hz, and differences were more pronounced for higher synaptic strengths (Figure 5F). The 12 cutoff frequency for the same network was more than 5-fold greater (Figure 5G).

Interestingly, when FFE projections were used, the Fourier coefficient of the network level 4 output peaked several times at higher input frequencies (300–1,000 Hz). This effect can likely be credited to over-excitation of the network level 4 neuron, as seen in Figure 5A. Indeed, at most frequencies, the network level 4 neuron with FFE is likely firing at its maximum rate, due to the amplification of excitatory input over the circuit. As a result, high FC measurements at high input frequencies cannot necessarily be attributed to the circuit following the inputs, but may instead result from the physical features of the neuron allowing for maximal firing at these rates. In either case, the Fourier coefficient peaks at these high input frequencies do not stand out from the overall power of the output response, as shown in Figure 5D.

One may ask how much of these frequency responses could have arisen by chance due to the random activity that is provided to each network level. The fold increase in Fourier coefficient responses at the input modulation frequency is shown in Figure 5H. Even at the lowest synaptic strength tested, FFEI projections provided responses that were at least five times those of the noise-only network, and this difference greatly increased when input strength was increased.

So far, we have only investigated how inputs of fixed modulation frequency are transmitted over multiple network levels. While this addresses how different network levels can exhibit synchronized activity, it does not directly confront how these networks can adapt to changing inputs. We, therefore, constructed a rectified sinusoidal Poisson input with a changing modulation frequency over time (Figure 5I). After changing the input modulation frequency, we held the input modulation frequency constant for 2 s in order to produce a reliable estimate of the Fourier coefficients. We then provided this input to the FFE and FFEI models and examined the normalized FC spectrum over time (Figures 5J,K), which measured how closely the network level 4 output followed a range of frequencies over the course of the input. With FFE projections, the network level 4 output only noticeably followed the stepping input at low frequencies (5–10 Hz). Meanwhile, with FFEI projections, the network level 4 output followed the entire stepping input fairly well. Furthermore, the network level 4 output did not exhibit significant residual power at frequencies represented earlier in the input, suggesting that the FFEI model was capable of quickly adapting to the changing input frequency. To more closely examine the speed at which the FFEI model transitions between inputs of different modulation frequencies, we provided the model with a Poisson input that increased from F = 10 Hz to F = 50 Hz at time = 1 s, and computed the FC spectrum of the network level 4 output over smaller time intervals (0.2 s; Figure 5L). With this increased temporal resolution, the FFEI model rapidly adapts to an increase in the input frequency.

Taken together, these results indicate that FFEI projections organize multi-level, hierarchical computations that are also being driven by ongoing incidental activity. Further, FFEI projections allow these computations to occur with high fidelity even at high temporal frequencies. In this way, FFEI projections serve to deliver both a signal as well as the timing signal necessary to organize feed-forward computation across layers of processing.

We have already noted that paired feed-forward excitation with slightly delayed inhibition is a common circuit motif in the brain, but the concept can be used more generally for computations with either sign. Although most neuronal circuits of which we are aware follow the excitatory-before-inhibitory configuration, there are some places, such as chandelier cells in the somatosensory cortex (Zhu et al., 2004), where inhibitory input precedes excitatory input. The paired E/I or I/E motifs can be used in artificial circuits to organize feed-forward computations at high speeds that exceed the time constant of integration of the individual elements.

We demonstrate the use of mixed feed-forward inputs in a model LIF circuit that performs the exclusive-or (XOR) computation of two inputs (Figure 6). In this computation, the output neuron should exhibit a response if either of the two inputs is positive, but it should be silent if there is no input or if both inputs are active. To transmit a positive postsynaptic signal, we used FFEI synapses employing excitation that was followed by identical but delayed inhibition (Figure 6A). To transmit a negative signal, we employed inhibition that was followed by identical but delayed excitation. Input weights for cells in the first layer were set so that one input was positive and the other input was negative. Further, the input with a weight that was positive in the first cell had a negative weight in the second cell. The neuron in the second layer responded if either of the cells in the first layer responded.

Once again, we provided input at different modulation frequencies, although we shifted the two inputs so that they were 180° out of phase (Figure 6A). Therefore, the XOR output should be produced at twice the input frequency (the cell should respond strongly when either input is at its positive phase, and weakly or not at all when the inputs are turned off). Once again, we found that the circuit with mixed FFEI projections could follow the XOR computation over high frequencies, while a comparison FFE circuit fell off rapidly with frequency, for both the raw response (Figure 6B) and for the fraction of the total response (Figure 6C) that was at the stimulus frequency (Figure 6D). We compared another input case (with the two inputs shifted by 90°) in Supplementary Figure 2.

We conclude that the concept of mirroring a feed-forward input with a delayed negative copy can organize high frequency computations across multi-layer neuromorphic circuits in much the same way that a clock organizes computations across digital electronic circuits.

Most excitatory synapses in the mammalian brain employ a combination of the AMPA and NMDA receptors. AMPA receptors respond rapidly and briefly to neurotransmitters, while NMDA receptors remain open for longer periods of time (50–150 ms). However, there are a few instances of excitatory synapses that primarily involve AMPA receptors. One example is the calyx of Held synapse in the mature mammalian auditory brain stem; notably, this region often processes high frequency temporal information in the form of auditory input (Nakamura and Cramer, 2011). This raises the question of whether neural circuits could, in principle, achieve the same fast computation as FFEI by using feed-forward inputs that were comprised entirely of fast AMPA channels (see also Wolf et al., 2005).

To address this question, we compared feed-forward transmission in synapses that were comprised of AMPA receptors only, AMPA and NMDA receptors only (like FFE), and AMPA, NMDA, and GABA receptors (like FFEI) in Figure 7. We again provided rectified sinusoidal input at an input modulation frequency, and examined the Fourier coefficient of the response at the input frequency (Figure 7B), the average response across all frequencies (Figure 7C), and the normalized output frequency (Figure 7D). Unlike in previous simulations with feed-forward excitatory transmission alone, accurate high frequency transmission was possible when only fast-closing AMPA channels were present. However, when slow-closing NMDA channels were added, the excitatory conductance and the spiking response became blurred, leading to poor transmission of high frequency information. High frequency transmission was restored by inhibition via GABA channels.

To investigate the effects of local background activity on the ability of the synapses to follow high frequency inputs, we provided 50 noisy Poisson inputs, modeled as in Figure 5, and examined the output response (Figures 7E–G). In the presence of excitatory noise, the FFEI synapse overall followed rectified sinusoidal inputs more effectively than the AMPA-only feed-forward synapse. These observations can be attributed to the overall output power, which was somewhat higher for the AMPA-only feed-forward synapse than for the FFEI synapse (Figure 7F).

These results suggest that AMPA-only feed-forward synapses provide some of the benefits of FFEI with respect to high frequency computation. However, FFEI synapses are more effective than AMPA-only feed-forward synapses when noisy inputs are also provided, suggesting that inhibitory GABA channels may play a role in filtering out excitatory noise and thus overriding ongoing computations.

Up to now, we have not considered an important feature of synaptic transmission in real triad synapses in the LGN: short-term dynamics (Abbott et al., 1997; Tsodyks and Markram, 1997; Varela et al., 1997; Carandini et al., 2002; Swadlow et al., 2002; Boudreau and Ferster, 2005; Higley and Contreras, 2006). The retino-geniculate excitatory and inhibitory synapses, as measured ex vivo in mice, both exhibit substantial short-term depression (Eysel, 1976; Koch, 1985; Chen and Regehr, 1999; Blitz and Regehr, 2003, 2005). We developed a depression model for excitatory and inhibitory synapses based on experimental data from mice (Chen and Regehr, 1999; Blitz and Regehr, 2003, 2005) using depression equations from Varela et al. (1997), and added AMPA, NMDA, GABA_A, and GABA_B currents (Supplementary Figure 3). NMDA and GABA_B currents have long time courses, and it is necessary to have compatible time courses for the paired feed-forward input to allow high frequency transmission.

Adding depression dynamics also meant that we needed to consider the state of the synapses at the beginning of the simulation. In vivo in the cat, retinal ganglion cells exhibit relatively high spontaneous firing rates of tens of spikes/sec (Kuffler, 1953), so it is unlikely that retinal ganglion cell synapses would commonly be in a completely undepressed state; the neuron would need to be shown a stimulus that suppresses its firing, such as showing an ON spot to an OFF-center neuron, to achieve such a state. Therefore, although we began our simulations with the synapses in a completely undepressed state, we did not analyze the output until after 1 s had elapsed.

Representative simulations for synapses with AMPA + NMDA (FEI) and AMPA, NMDA, GABA_A, and GABA_B (FFEI) receptors are shown in Figures 8A,B. When the simulations begin, there is strong excitatory transmission, but excitation and inhibition become more balanced as the simulation runs. Synaptic weights were chosen for excitatory and inhibitory synapses so as to allow moderate firing in the FFEI case after 1 s of simulation; inhibitory inputs were removed for the FFE case, and the excitatory weights were adjusted so that the firing rate for inputs at 5 Hz modulation exhibited 70 Hz after 1 s of simulation, matching the FFEI simulations. With synaptic depression, transmission (Figures 8C–E) is improved for FFEI synapses compared to the simulations without synaptic depression, consistent with previous theoretical analyses (Abbott et al., 1997). However, transmission at higher temporal frequencies is better with the FFEI synaptic arrangement as compared to FEI.

We modulated the input rate sinusoidally from 0 to 50 spikes/s, a reduction from the 0 to 100 spikes/s we used in the prior examples. Blitz and Regehr (2005) noted that high frequency spike trains (50–100 Hz) caused such substantial depression that the inhibitory synapse effectively stopped transmitting. Here, it is important to note that we are modulating the firing rate probability of the input sinusoidally, and a 50 Hz peak firing rate probability modulated sinusoidally will rarely result in spikes that have an inter-spike-interval of 20 ms (1/50 Hz). Visual firing rates are typically quantified in a trial-averaged fashion and expressed as the probability of firing in a small time bin, and we are generating spikes according to such a process. At sustained high frequency input rates >50 Hz, it is likely that transmission of both excitation and especially inhibition would be very reduced, although it is unclear how common this situation of sustained high input rates would be in vivo, or if synaptic depression is altered in vivo or across species.

While the FFEI motif is sometimes found in inputs onto individual neurons, such as in the retinogeniculate triad synapse, the FFEI motif is also commonly found in inputs to circuits such as in the hippocampus (Buzsaki, 1984; Pouille and Scanziani, 2001; Bhatia et al., 2019) or the cerebral cortex, both in inputs from the thalamus (Agmon and Connors, 1991; Swadlow, 2003) and interareal projections (Yang et al., 2013). In these networks, excitatory projections from other areas make connections with separate populations of excitatory and inhibitory neurons, and the targeted inhibitory neurons provide rapid but slightly delayed inhibition onto the recipient excitatory neurons. Further, the excitatory and inhibitory neurons typically receive converging inputs from many cells, so that in principle the excitatory and inhibitory inputs to the local circuit may not be entirely matched. That is, the strong 1-to-1 relationship between each feed-forward excitatory postsynaptic potential and each feed-forward inhibitory postsynaptic potential that is found at the triad synapse is not present in circuit-level projections; instead, feed-forward excitation and inhibition are more loosely related.

Before we begin studying how FFEI can impact temporal computations at the circuit level, it is worth pausing to consider whether all cortical circuits might take advantage of any properties that we uncover. In the thalamus, it is clear that visual, somatosensory, and auditory neurons can follow sensory inputs at high frequencies. However, in visual (Hawken et al., 1996), somatosensory (Chung et al., 2002), and auditory cortex (Creutzfeldt et al., 1980; Wehr and Zador, 2005), cortical neurons primarily respond to stimulation at much lower stimulus repetition frequencies (<20 Hz) than do thalamic neurons. That is, the primary sensory cortex does not seem to segment responses to sensory stimulation at frequencies much higher than 20 Hz. However, in both sensory and higher cortical areas and in the hippocampal formation, 40 Hz activity (gamma activity) is prominent (Fries et al., 2007) and can be induced by driving feed-forward interneurons (Cardin et al., 2009). It has been suggested that in hippocampal circuits, each gamma cycle might encode representations of individual items (Lisman and Idiart, 1995; Fries et al., 2007; Lisman and Jensen, 2013), and 40 Hz sensory stimulation has been reported to be protective against neurodegeneration (Iaccarino et al., 2016). So we feel it is worth considering whether circuit-level FFEI input can allow circuits to follow inputs at such high frequencies.

Here, we examined a model network with 100 feed-forward excitatory inputs that arrived at an LIF excitatory cell and an LIF inhibitory cell (Figure 9A). Following our previous methods, the firing rate of each input was modulated sinusoidally at an input modulation frequency. In this model, all inputs were driven at the same phase (Figure 9B). We examined the Fourier coefficient of the output responses at the input modulation frequency (Figure 9C), the average response across all frequencies (Figure 9D), and the ratio of these quantities (Figure 9E).

The circuit-level FFEI exhibited some similarities to and differences from the single FFEI input situation. One difference is that the circuit’s I neuron receives excitatory input without a delayed inhibitory copy and acts just like a neuron receiving FFE input. Therefore, the frequency responses of the inhibitory neuron are the same as the E-only case in Figures 9C–E. If we look at these responses, we see that they fall off much like FFE input to the triad synapse: the response is much attenuated at about 25 Hz, for example. At input modulation frequencies greater than this, the I neuron will not respond strongly to the excitatory input (Figure 9B, right), so the I neuron’s contribution to the circuit is not present at high frequencies, and the circuit behaves like an FFE circuit. This means that the circuit-level FFEI projection has a much lower cut-off response frequency than a similarly-endowed triad-like FFEI projection.

We suspected that the membrane time constant of the inhibitory neuron would be a key parameter in determining the cut-off frequency for the circuit-level FFEI projection, so we compared responses in models where this parameter was varied. In this circuit network, the level of response output is very sensitive to the synaptic strengths of the inputs as well as the strength of the synapse from the inhibitory neuron onto the excitatory neuron. Because there were two non-linear ways to tune the responses to a particular input rate, we simply left the synaptic strengths unchanged while we altered the membrane time constant of the inhibitory neuron so that the models could be compared on the basis of that parameter.

In Figure 9F, we show the normalized Fourier coefficient for these models but with the data scaled so that the output response for an input modulation of 5 Hz is equal to 1. Here it is apparent that the frequency response depends strongly on the inhibitory membrane time constant, with low values corresponding to high transmission at high input modulation frequencies. When the inhibitory time constant was 10 ms, which is consistent with experimental studies of cortical interneurons (Economo and White, 2012), the 12 frequency cut off was 40 Hz, indicating that the excitatory cell could follow input responses very effectively up to that frequency (Figure 9G). This 12 cutoff frequency was 70% greater than the 12 cutoff frequency for a circuit with FFE projections alone (Figure 9H).

We have just seen that the properties of the feed-forward inhibitory interneuron are critical to the temporal frequency limits of a circuit with paired feed-forward excitation and inhibition arriving via external excitatory input. In some circuits, the AMPA to NMDA current ratio is larger in feed-forward interneurons than in excitatory neurons (Wang and Gao, 2009). We have seen in Figure 8 that AMPA-only inputs allow high temporal frequency transmission, and we asked whether providing AMPA-only excitatory input to the feed-forward inhibitory neuron would increase the cut-off temporal frequency. In Figure 10, we reduced the synaptic time constant of the excitatory input to the inhibitory neuron from 20 ms to 1.6 ms to reflect an AMPA-dominated synapse. Indeed, the E neuron in the circuit can now follow much higher temporal frequencies, up to about 90 Hz. These results indicate that the properties of the interneuron and its input are the key limiting factor to high temporal frequency transmission in these circuits.

Excitation with delayed inhibition produces a narrow time window during which spiking can occur. As a result, FFEI inputs can enhance the temporal fidelity of neural circuit outputs, as shown in experiments in hippocampal CA1 (Pouille and Scanziani, 2001; Bhatia et al., 2019) and at the retino-geniculate synapse (Blitz and Regehr, 2005). Given the high temporal precision of networks containing FFEI projections, it follows that such networks would accurately transmit high frequency temporal information, as well. This effect can largely be attributed to the narrow spiking time window generated by paired E/I inputs, as previously described (Pouille and Scanziani, 2001; Wehr and Zador, 2003; Blitz and Regehr, 2005; Mittmann et al., 2005; Higley and Contreras, 2006; Cardin, 2018). Here we show that the residual inhibition following each input spike does not suppress the cell for long, and the cell is quickly ready to respond to subsequent FFEI inputs (Figure 3A).

Networks that propagate signals with high temporal fidelity have been called synfire chains (Abeles, 1991). We have shown here that excitatory-only feed-forward networks can produce reliable computation only at relatively low temporal frequencies. Excitatory-only feed-forward networks can reliably and quickly transmit firing rate information if each area receives ongoing balanced excitatory and inhibitory internal noisy input (Shadlen and Newsome, 1998; Chance et al., 2002; van Rossum et al., 2002; Murphy and Miller, 2009) or has detailed or loosely balanced strongly recurrent connections (Tsodyks and Sejnowski, 1995; van Vreeswijk and Sompolinsky, 1996, 1998; Hennequin et al., 2017) or balanced connectivity across cortical areas (Joglekar et al., 2018). Here we have shown that this transmission is possible with FFEI projections even if the random local input that a local area experiences or the internal connectivity of the network is not balanced: transmission at high temporal frequencies can be accomplished with balanced feed-forward input alone.

It should be noted that some brain pathways use different strategies to transmit high frequency information. In the auditory brainstem, neurons respond rapidly to a transient input (Schnupp and Carr, 2009), which can be explained by the composition of the intrinsic ionic currents in the cell that allow them to act as “slope detectors” (Meng et al., 2012), although they do not respond tonically to constant current, as LGN neurons do (Suzuki and Rogawski, 1989).

The high transmission functions of FFEI could act in concert with another proposed function for the triad synapse: a state-dependent “veto” (Koch, 1985). Under this idea, the synapse would act as described here when operating in a feed-forward mode, but the inhibitory neuron could be activated independently in order to suppress feed-forward excitatory input to support functions such as reduced attention or internal processing. A related feed-forward excitatory/inhibitory gate model was demonstrated by Kremkow et al. (2010a). In recurrent, balanced cortical networks, inhibitory synaptic strengths (Vogels and Abbott, 2009) or the correlation between excitatory and inhibitory signals can be used to gate the propagation of tonic activity through downstream networks, and this can be combined with temporal gating (Kremkow et al., 2010a). Learning rules (Vogels et al., 2011; Kleberg et al., 2014) could tune inhibition for ideal propagation at high temporal frequencies (see Figure 3).

The absence of a synchronizing clock or trigger signal to organize the activity of neurons would seem to imply that neuronal circuits would be limited to performing computations that do not require such a trigger, such as attractor networks or integrators (Hopfield, 1982; Seung et al., 2000; Miller et al., 2003; Miller, 2016). FFEI acts much like a feed-forward synchronizing trigger that forces computation to occur within a short window (Buzsaki, 1984; Agmon and Connors, 1991; Swadlow and Gusev, 2000; Porter et al., 2001; Swadlow, 2003; Rock and Apicella, 2015; Bhatia et al., 2019).

Here we showed that FFEI when used in projections across brain areas or layers, can organize computation across these regions, even when these regions are experiencing ongoing local activity (Figure 5). Unlike a digital computer, where the clock trigger runs at a fixed rate, FFEI allows the trigger signal to be generated at any time (Figure 5I–K) and at many rates, up to 100 Hz for triad-like synapses and up to about 40–80 Hz for circuit-level FFEI inputs, depending upon the assumptions of the model. In low-level areas, these triggers would be generated when sensory input arrives, and computation could proceed at the speed of the sensory input. In higher-level areas, computation across two areas could proceed whenever the input area generates output and could underlie coordination of high frequency oscillations across areas (Lisman and Idiart, 1995; Lisman and Jensen, 2013).

The degree to which high frequency information actually travels throughout the brain varies depending upon area and channel and has been well studied in the visual system. In recordings that examined velocity tuning to single bar sweeps in both retina and LGN of X and Y cells in the cat, peak velocity tuning was slightly lower in LGN for X cells, but very comparable in retina and LGN for Y cells, with preferred velocities that ranged from about 20°/s to 100°/s (Frishman et al., 1983), indicating that spike timing information can be precisely transmitted. Consistent with this idea, retinal inputs activate LGN cells with high efficacy (Usrey et al., 1998), indicating that LGN neurons follow retinal ganglion inputs with high reliability.

Evidence from sensory cortex, however, suggests that the thalamocortical connection does not take full advantage of the potential FFEI speeds. There is a drop in peak temporal frequency tuning between LGN and cortex, with most LGN cells exhibiting strong firing for many frequencies, including those greater than 16 Hz, while few cortical neurons exhibit strong firing at 16 Hz or greater (Hawken et al., 1996; Van Hooser et al., 2003, 2013; Heimel et al., 2005). If FFEI were the only factor, we would predict that transmission should be quite robust at 20 Hz (Figure 9), but the visual cortex attenuates these frequencies quite strongly. This drop also occurs in the somatosensory (Ahissar et al., 2001; Chung et al., 2002) and auditory systems (Creutzfeldt et al., 1980). Synaptic depression or other cortical circuit mechanisms may underlie this filtering (Chung et al., 2002; Swadlow, 2002; Boudreau and Ferster, 2005; Swadlow et al., 2005; Wehr and Zador, 2005).

Nevertheless, many cortical regions exhibit strong modulation and resonance at around 40 Hz, also known as the gamma frequency (Fries et al., 2007; Cardin et al., 2009; Lisman and Jensen, 2013). If responses at 40 Hz in one area are to be intelligible in another cortical or hippocampal area, then it is necessary for these circuits to be able to follow inputs at these frequencies. Here we showed that circuit-level FFEI inputs can in principle allow such rapid communication.

The FFEI models make a simple prediction: the absence of inhibition at cells receiving FFEI inputs should limit the ability of these cells to follow inputs at high temporal frequencies. An optogenetics approach could be used to control the activation of retinal ganglion cells (RGC) with high temporal precision, and the activity of LGN relay cells in response to RGC action potentials could be measured. A GABAzine infusion to block inhibition in the LGN should limit the ability of LGN relay cells to follow RGC inputs at high temporal frequencies. A similar experiment could be performed at thalamocortical slices in the somatosensory cortex (Agmon and Connors, 1991) in order to test the high-convergence regime. While we have noted that there is a high degree of synaptic depression in thalamocortical synapses (Chung et al., 2002; Higley and Contreras, 2006), the data from an experiment should exhibit a strong shift, as in Figure 9, from the FFEI dynamics towards E-only dynamics. One experiment that studied the propagation of signals through the somatosensory cortex in vivo found that spike timing transmission precision decreased at low firing rates (but not for high firing rates) when PV interneurons were silenced (Jang et al., 2020).