Noise-induced synchronization and regularity in feed-forward-loop motifs

In complex systems, the interplay between network structure and noise often leads to emergent phenomena. This study explores the effects of uneven coupling and asymmetric noise on the dynamics of feed-forward loop (FFL) motifs, essential components of networks. We investigate how these variations in network motifs influence noise-induced synchrony and regularity. Using mathematical models and simulations, we uncover that the interaction of asymmetric noise and uneven coupling can significantly impact the behavior of FFL motifs, promoting synchronization and enhancing output regularity. These findings contribute to a deeper understanding of the role of network motifs in shaping network dynamics, shedding light on the intricate interplay between structure, noise, and emergent behavior in complex systems.


Introduction
Network motifs, a fundamental concept in network analysis, are recurring interaction patterns observed in various systems [1].These motifs often retain their specific dynamical functions as well as integrate into the collective behaviors of the broader network structures, thus they are often considered as the building blocks of complex networks [2,3].Among the common three-node network motifs, coherent and incoherent feedforward loops (FFL) have proven to be notably more prevalent than their counterparts [3,4].Here we call them type-1 motif (T1) and type-2 motif (T2) for simplicity and they are demonstrated in Fig. 1.In motif T1, three unidirectional connections are positive (i.e., excitatory coupling).In motif T2, the connections from node 1 to node 2 and from node 1 to node 3 are positive but the connection from node 2 to node 3 is negative (i.e., inhibitory coupling).FFL network motifs have been found in many actual biological networks, such as gene expression in bacteria and yeast [5,6], human and mouse genomes [7,8], cat cortex [9], and the nervous system of roundworm [10].Moreover, both motifs are constructed such that node 1 may be considered an input layer, and node 3 an output layer of the network.
There are two parallel signalling pathways: one direct pathway from the input layer to the output layer, and one indirect pathway from input to output by means of node 2 (the layer 1 arXiv:2310.02427v1[math.DS] 3 Oct 2023 of internodes or interneurons).Such parallel information transmission structure has been discovered in the auditory cortex [11] and electrosensory system [12].
The significance of FFL network motifs leads to the question of how they interact with one another and perform specific information-processing roles in the presence of noise, a ubiquitous factor in real life.Under the conditions of equal coupling and symmetric noise, FFL motifs have been shown to produce coherence [13] or resonance [14].However, to our best knowledge, no prior have clearly demonstrated their performance in a heterogeneous setting: unequal coupling and asymmetric noise.To address this unexplored aspect, our research will focus on noise-induced synchrony and regularity of T1 and T2 motifs in the heterogeneous setting.The remainder of this paper is structured as follows.Sec. 2 introduces the mathematical model and methods.Sec. 2 Mathematical model and methods

Model
We consider a system comprised of three coupled oscillators arranged in FFL patterns (see Fig. 1), where the dynamics of each oscillator are described by the canonical model for the normal form of a Hopf bifurcation (HB), the λ − ω system, with additive noise and diffusive coupling terms.The ith oscillator is modelled by the system of stochastic differential equations (SDEs) where i, j = 1, 2, 3.
The amplitude and phase of ith oscillator can be obtained by r i = x 2 i + y 2 i and ϕ i = tan −1 y i x i , respectively.The function λ(r i ) = λ 0 + αr 2 i + γr 4 i governs the modulation of the amplitude of ith oscillator.λ 0 serves as the control parameter, and the Hopf bifurcation (HB) occurs when λ 0 = 0.The parameters α and γ influence the system's behavior away from this bifurcation point.Additionally, the function ω(r i ) = ω 0 + ω 1 r 2 i controls the modulation of the ith oscillator's frequency, with ω 1 governing how the frequency evolves concerning the amplitude r i .Notably, when ω 1 = 0, changes in amplitude do not directly impact the phase.We consider a supercritical Hopf bifurcation and the corresponding parameter values are α = −0.2,γ = −0.2,ω 0 = 2, and ω 1 = 0.The term δ i dη i (t) represents intrinsic white noise applied to x i , and η i (t) characterizes a Wiener process with zero mean and unit variance.The parameter δ i is the noise intensity.
Additionally, the terms j̸ =i d i,j (x j − x i ) and j̸ =i d i,j (y j − y i ) denote the diffusive coupling that one neuronal oscillator receives, where d i,j signifies the connection strength between oscillators i and j, indicating the signal transmitted from oscillator i to oscillator j.To create the Feed-Forward Loop (FFL) motifs, as illustrated in Figure 1, we configure d j,1 = 0 and d 3,i = 0 for all i and j.This setup ensures that oscillator 1 does not receive input from other oscillators within the FFL, and oscillator 3 does not transmit output to other oscillators within the FFL.In motif T1, all three connections are excitatory, meaning that d 1,2 > 0, d 1,3 > 0, and d 2,3 > 0. Conversely, in motif T2, there are two excitatory connections (d 1,2 > 0 and d 1,3 > 0) along with one inhibitory connection (d 2,3 < 0).

Methods
All computational tasks, including simulation, numerical analysis, and figure generation, are carried out using MATLAB.We employ the Euler-Maruyama method to calculate the numerical solutions to SDEs and use the time interval [50,200] and time step of dt = 0.01.We initiate the simulations with arbitrary, small random initial conditions x i (0), y i (0) ∼ N (0, 0.008 2 ) for i = 1, 2, 3. To address the challenges posed by the highfrequency fluctuations inherent in noise-induced oscillations during numerical analysis, we implement a low-pass filter.This filter is constructed using a Gaussian-weighted moving average spanning a window of 100 data points, ensuring enhanced result consistency.Finally, we compute the coherence measures σ, γ, and R outlined in Equations 4 through 6, averaging these values over a total of N = 200 simulations.

Noise-induced dynamics
We consider the dynamics of our network motifs in the excitable regime but in the vicinity of a supercritical HB at λ 0 = 0 (e.g., λ 0 = −0.3 in Fig. 2).The deterministic systems associated with motifs T1 and T2 (δ i = 0 for i = 1, 2, 3) converge to the stable fixed point (0, 0), as shown in Fig. 2a and 2d, respectively.We also observed that all three oscillators of the T1 motif reach in-phase during the transient time, despite their different initial values (Fig. 2a).The deterministic T2 oscillators 1 and 2 exhibit in-phase, whereas oscillator 3 show anti-phase with the others during the transient time (Fig. 2d).
The presence of intrinsic noise induces sustained limit-cycle oscillations in all oscillators within the T1 and T2 motifs (Fig. 2b and 2e where δ i equals 0.01 for i = 1, 2, 3).This kind of oscillation is commonly referred to as "noise-induced oscillation".Examining the dynamics of the two input oscillators (specifically, oscillator 1) in both T1 and T2 motifs, we observe similarities in their periods and time-varying amplitudes, as indicated by the blue dashed lines in Fig. 2b and 2e.A similar resemblance is found for oscillator 2, as depicted by the gray dotted lines.However, a notable distinction emerges when we consider the output oscillators (oscillator 3) of the T1 and T2 motifs.The output oscillator of the T2 motif (red curve in Fig. 2e) exhibits more regular oscillations compared to its counterpart in the T1 motif (red curve in Fig. 2b).For example, the amplitude of the output oscillator in motif T2 remains relatively stable, characterized by more pronounced peaks and fewer small-amplitude perturbations.
To determine whether this is the case for other noise intensities, we compute the time- However, A 3 differs between the T1 and T2 motifs.In particular, for motif T2, A 3 is greater than its counterpart in motif T1.For instance, when δ 1 , δ 2 , δ 3 = 0.1, A 3 ≈ 0.12 for the T1 motif and A 3 ≈ 0.19 for the T2 motif (see the dashed grey lines in Figs.2c and 2f).
Furthermore, not only does A 3 itself differ across the two FFL motifs, but the relative size of A 3 compared to A 1 and A 2 differs across the two FFL motifs as well.In particular, for

The effects of noise on network synchronization
Next we consider the impact of noise on the synchronization of noise-induced dynamics.To excite oscillations from rest, we fix the noise intensity applied to x 2 and x 3 (e.g., δ 2 , δ 3 = 0.01 in Fig. 3) and vary the noise intensity applied to x 1 (e.g.δ 3 ∈ [0.001, 10 1/2 ] in Fig. 3).
Then, we consider network synchronization from two perspectives.We begin first from a broad perspective, where we analyze the synchronization between every oscillator in each FFL motif.In order to quantify the degree of network synchrony, we use the root mean square deviation [15,16] where and M is the total number of oscillators.Note that σ t is computed using the amplitudenormalized time series, x i (t)/A i , rather than x i (t).This normalization ensures that σ is a measure of temporal synchronization, or phase synchronization, rather than complete synchronization, which depends on alignments in both amplitude and phase [17].Furthermore, since σ quantifies the degree of variability between x 1 , x 2 , and x 3 , it follows that smaller values of σ indicate greater levels of synchrony.
The results of our analysis are presented in Fig. 3a which displays the root mean square deviation, σ, as a function of the driving noise intensity, δ 1 , for the T1 (blue curve) and T2 (black curve) motifs.We observe that both of the T1 and T2 motifs display a resonant response to δ 1 , which indicates the occurrence of coherence resonance (CR) [18,19].That is, when δ 1 is weak (e.g.δ 1 < 0.12 for motif T2 in Fig. 3a), network synchrony increases with the increment of δ 1 , or equivalently, the value of σ decreases.Then, at an intermediate intensity of δ 1 , for example, δ 1 ≈ 0.12 for motif T2 in Fig. 3a, network synchronization reaches an optimal state as σ attains its absolute minimum.Then, as the noise intensity increases further, network synchrony exhibits a sharp decline which suggests that the noise has begun to dominate network dynamics.Indeed, our results in Fig. 3 indicate that the network synchronization (i.e.σ) is the same for the T1 and T2 motifs across the strong intensity range of δ 1 (e.g.δ 1 > 0.5 in Fig. 3).
We observe that both motifs display a resonant character and behave similarly over the strong intensity range of δ 1 , however, they differ considerably over the weak and intermediate intensity ranges (e.g.δ 1 ≤ 0.1 in Fig. 3a).Over the weak intensity range of δ 1 , the σ curve which corresponds to the T1 motif (blue curve in Fig. 3a) is always less than the black curve in Fig. 3a which corresponds to the T2 motif.This suggests that the T1 motif displays comparatively greater network synchrony.Moreover, we see that the optimal driving noise intensity, denoted by δ * 1 , and the absolute minimum which it induces, denoted σ * , are both smaller for the T1 motif than the T2 motif.For example, Fig. 3a shows that the optimal driving noise intensity and its corresponding minimum are located at δ * 1 = 0.07 and σ * = 0.215 for motif T1, and δ * 1 = 0.12 and σ * = 0.218 for motif T2 (see solid squares in Fig. 3a).
A second perspective from which to study network synchronization comes by examining the synchrony solely between the input and output oscillators of the networks.To quantify the synchronization between the input-output pair x 1 and x 3 we analyze the distribution of their phase differences, ∆ϕ = ϕ 1 − ϕ 3 , using a measure known as the mean phase coherence [17,20]: The range of γ is 0 ≤ γ ≤ 1, such that larger values of γ indicate a greater degree of phase synchronization; in particular, γ = 1 indicates a state of perfect synchronization and γ = 0 indicates a completely chaotic state.The results of our analysis are presented in Fig. 3b, where the blue and black curves correspond to the T1 and T2 motifs, respectively.
As in Fig. 2a, γ is computed over the range 0.001 ≤ δ 1 ≤ 10 1/2 with δ 2 and δ 3 fixed at 0.01.The results of Fig. 3b corroborate those in Fig. 3a, and reveal a resonant behaviour in the γ curves which mirrors that in Fig. 3a.That is, for weak noise intensities (e.g.δ 1 < 0.12 for motif T1), increasing δ 1 leads to an increase in network synchronization as indicated by an increase in γ.Then, at an intermediate noise level (e.g.δ 1 = 0.12 for motif T1), synchronization reaches an optimal state, where γ reaches its absolute maximum.
For stronger noise intensities, network synchronization declines which is indicated by a falling γ.This suggests that optimal phase synchrony between x 1 and x 3 is achieved at an intermediate level of δ 1 .
Moreover, we find that the T1 motif requires a lesser intensity of noise to attain an optimal level of synchrony than the T2 motif.For example, in Fig. 3b, the optimal noise intensity for the T1 motif, δ * 1 = 0.12, is smaller than that for the T2 motif which is δ * 1 = 0.188, with a larger corresponding maximum in γ, γ * = 0.728, relative to the T2 motif, for which γ * = 0.713.Furthermore, we note that γ for the T1 motif is consistently higher than that of the T2 motif, and that this difference diminishes as δ 1 increases (as in Fig. 3a).Although the results shown in Figs.3a and 3b have the same qualitative characteristics, there are quantitative differences such as the optimal driving noise intensities.
These differences emerge naturally as a result of the inherent differences in the measures γ and σ defined in Eqs. 5 and 3, respectively.Nevertheless, both measures indicate that the T1 motif exhibits a greater propensity for network synchronization than the T2 motif over the weak to intermediate noise intensity ranges and requires less noise to achieve an optimal level of network synchronization.

The effects of noise on output regularity
In addition to studying the effects of noise on network synchronization, we also consider the effects of noise on output regularity by analyzing the regularity of the time series x 3 .
By regularity, we refer to the degree to which the dynamics of x 3 are periodic and quantify this using the coefficient of variation of the inter-spike intervals (ISIs).The coefficient of variation, R, is defined as the standard deviation of the ISIs divided by the mean of the ISIs [21,22], that is, where ISI k denotes the kth ISI.In order to compute an ISI we mark the occurrence of a spike by a peak in the time series of x 3 (e.g., inset of Fig. 4a) and compute the difference between consecutive spike times.Furthermore, because R is a measure of central variability, larger values of R indicate a more irregular firing pattern and consequently lower regularity, and vice versa.We present the results of our analysis in Fig. 4a, where R is plotted against the driving noise intensity, δ 1 , for the T1 motif (black curve) and T2 motif (blue curve).We find the that both motifs display a resonant character, which indicates the occurrence of CR.That is, R is relatively low at weak intensities of δ 1 (e.g.δ 1 = 0.0023 in Fig. 4a), and with an increase in δ 1 , R decreases, indicating that output regularity is increasing, or, the firing pattern is becoming more regular.Then, as δ 1 surpasses an optimal intensity, R starts to increase again, and the output of our networks becomes more chaotic.Thus, the addition of intrinsic noise can optimize the regularity of the output oscillator, x 3 , of both the T1 and T2 motifs at an intermediate intensity.We find however, that motif T2 exhibits greater regularity than motif T1 over the range 0.001 ≤ δ < 0.16; before the optimal point B in Fig. 4a which occurs at δ 1 = 0.16.That is, when 0.001 ≤ δ < 0.16, the R of the T2 motif is less than the γ of the T1 motif (Fig. 4a).As δ 1 approaches the optimal point δ 1 = 0.16, the difference between the two curves becomes increasingly small.Interestingly, the optimal noise intensity δ * 1 and the corresponding absolute minimum in R, denoted R * , are both the same for motifs T1 and T2.For example, in Fig. 4a,  B is a moderate level of noise with intensity δ 1 = 0.16 and corresponds to the purple curve, and point C is a high level of noise with intensity δ 1 = 1.35 and corresponds to the green curve.First, we find that the peak of the purple density function, which corresponds to the optimal noise intensity, is more pronounced in both panels b and c than those of the orange and green density functions which correspond to noise intensities that are either too weak or too strong.This implies that the ISIs at point B are more tightly grouped, and agrees with our results in Fig. 4a.Furthermore, the orange density function in panel c displays a much more pronounced peak (i.e.larger peak and smaller half-width) than its counterpart, the orange density function in panel b, which has relatively broad bimodal peaks.This suggests that the output of the T2 motif is more regular than that of the T1 motif over the low intensity range of δ 1 .Conversely, the purple and green density functions appear to be equivalent across panel b and c of Fig. 4.This is consistent with Fig. 4a, which suggests that sufficiently strong intensities of the driving noise can eliminate the differences in the output regularity of the T1 and T2 motifs.
Overall, we observe that the regularity of the output of both motifs T1 and T2 increases as the noise intensity increases from low levels until it reaches an optimal point.Before this optimal point, the T2 motif shows greater output reliability (as measured by a lower value of R).And Beyond this optimal point, the regularity of the T1 and T2 motifs is identical, and with the increment of δ 1 , the regularity begins to decline as the noise intensity becomes overpowering.

The effects of network connectivity on network synchrony and output regularity
In Secs.First, we see that there exist threshold values of the common coupling strength, d, denoted d T , such that when d ≤ d T , changes in δ 1 have no significant effect on network synchronization (σ) and output regularity (R).In the case of network synchronization, we find that d T ≈ 0.02 for both T1 and T2 motifs (Fig. 5a and 5c) and for output regularity, d T ≈ 0.03 for both the T1 and T2 motifs (Fig. 5b and 5d).Furthermore, we see that the contour maps in panels c and d, which correspond to motif T2, are notably different than the contour maps in panels a and b, which correspond to motif T1, when the noise intensity is weak (e.g.δ 1 < 0.01 for σ and δ 1 < 0.03 for R in Fig. 5) and the coupling strength is strong (e.g.d > 0.01 for σ and d > 0.02 for R); or simply, the bottom right corners of the contour maps in Fig. 5.We find that the T1 motif has a smaller σ in this region relative to the T2 motif, but a larger R. In other words, the T1 motif shows a greater propensity for network synchrony but a lesser propensity for output regularity than the T2 motif in this region-clearly, the converse holds as well.Moreover, we see that the differences in σ and R disappear as δ 1 increases, which is consistent with our findings in the sections (upper right region of Fig. 5).
The contours in Fig. 5 suggest that the noise intensity required for both motifs to optimize both network synchrony (σ) and/or output regularity (R) is dependent on d.To consider the effects the network connectivity, d, more comprehensively, we compute σ * and we find that σ * is negatively correlated with d.This suggests that the optimal level of network synchrony increases as the network connectivity, d, increases.Additionally, in Fig. 6a, the σ * curve for the T1 motif is less than or equal to the σ * curve for the T2 motif.This is consistent with the results presented in Sec.3.2 which highlight the T1 motif's propensity for greater network synchrony.Furthermore, Fig. 6c reveals that the optimal noise intensity, δ * 1 is negatively correlated with d.That is, as network connectivity increases, the intensity of noise required to optimize network synchrony decreases.In addition, the δ * 1 curve for motif T1 in Fig. 6c (blue curve) is less than or equal to the δ * 1 curve for the motif T2 (black curve).This suggests that the T1 motif requires weaker noise to maximize network synchronization than to the T2 motif (as in Fig. 3a).
From Fig. 6b we see the R * is a non-monotone function of d.For both motifs T1 (black curve) and T2 (blue curve), R * is decreasing when d < 0.05, reaches a minimum at d = 0.05, and then increases for d > 0.05.When d = 0.05, both motifs have the same optimal output regularity: R * = 0.108.Indeed, there is no clear difference between the R * curves for the T1 and T2 motifs in Fig. 6b.This corroborates the findings in Fig. 4, which show that the T1 and T2 motifs have similar levels of optimal output regularity.Fig. 6d, which displays the optimal noise intensities associated with the curves in Fig. 6b suggests δ * 1 decreases as d increases for both types of motifs, and further, that there is no consistent difference between the δ 1 required to minimize R for the T1 motif and T2 motif.This is consistent with our results in Fig. 4. Next we consider how λ 0 affects the optimal driving noise intensity, δ * 1 , needed to minimize σ and R. Figs.2c and 2d display the optimal driving noise intensities which correspond to σ * and R * , respectively, as functions of λ 0 .The T1 motifs are represented by blue curves and the T2 motifs are represented by black curves.We find that the δ * 1 corresponding σ is a constant function of λ 0 for both the T1 and T2 motifs.Namely, δ * 1 ≈ 0.07 for the T1 motif and ≈ 0.12 for the T2 motif (Fig. 7c).These results are consistent with our earlier findings in Fig. 3a, which suggest that the T1 motif requires a lesser intensity of δ 1 to minimize σ relative to the T2 motif.Conversely, in Fig. 7d we find that the δ 1 needed to minimize R is a decreasing function of λ 0 , and as in Fig. 7b, the rate of change of the δ * 1 decreases as λ 0 approaches the excitation threshold.Finally, the noise intensities required to optimize output regularity are approximately the same for both the T1 and T2 type motifs.

Discussion
We explore the impact of noise on network synchronization and output regularity within the context of three-neuron FFL motifs.Our investigation focuses exclusively on two distinct motif types: T1, which has purely excitatory connections (as depicted in Fig. 1a); and T2, which has a combination of both excitatory and inhibitory connections (as illustrated in Fig. 1b).We choose to analyze these specific motifs due to their prevalence in neuronal networks [3,4].Nevertheless, there are other three-neuron motifs that are also prevalent within the brain.More generally, neural motifs (i.e., frequently recurring wiring patterns) are thought to serve important functions within the brain, and may be viewed as the fundamental building blocks of larger, more complex neuronal networks [4,25,26].
We model the dynamics of each oscillator using the λ−ω system, the canonical model for the normal form of a HB, since is a well-established framework that captures the dynamics of the transition from quiescence to oscillations observed in real neurons [27].We consider the excitable regime, which is quiescent in the absence of noise but can be excited by the addition of an intrinsic noise.Our result show that the addition of a pure noise stimulus can maximize both network synchronization and output reliability at an intermediate intensity.
Our findings agree with existing studies which find CR and other noise-induced effects in similar network motifs [28,29,30].
Furthermore, our results indicate that the T1 motif shows a greater propensity for noiseinduced synchronization; it exhibits a greater degree of synchronicity than the T2 motif and requires a comparatively lower noise intensity to optimize network synchrony.Conversely, we found that the T2 motif displays greater output reliability over the weak to intermediate range than the T1 motif but does not require a lesser intensity of noise to reach an optimal point.Importantly, our observations indicate that these differences between the two motifs are robust in that they persist over a wide range of noise intensities and parameter regimes, and only vanish when the intrinsic noise stimulus becomes excessively strong or the network connectivity becomes excessively weak.These results may in part be understood through our results in Fig. 2, which indicate: (i) the T2 motif shows both in-phase and anti-phase synchronization, whereas the T1 motif shows only in-phase synchronization (Fig. 2a, 2d); and (ii) the amplitude of the T2 oscillator is larger and appears to be more stable than the amplitude of the output oscillator of the T1 motif (Fig. 2c, 2f).Since we consider inphase synchronization it is clear why (i) may lead motif T1 to exhibit a greater propensity for network synchrony than the T2 motif.On the other hand, from (ii) we find that the oscillations of x 3 of motif T2 have more pronounced and larger peaks than those which correspond to motif T1.This makes it easier to distinguish between spurious subthreshold oscillations and actual peaks and may therefore promote greater output regularity (or reliability) as we consider regularity in the context of the ISI variability.
Our results further suggest that network connectivity is positively correlated with syn-
as a function of δ i , and average over N = 200 trials.Our results are displayed in Fig.2cfor motif T1 and Fig.2ffor motif T2.The dashed blue lines represent A 1 , the dotted black lines represent A 2 , and the solid red lines represent A 3 .We find that A 1 and A 2 are overlapped for both motifs T1 and T2, implying that oscillators 1 and 2

δ * 1
= 0.162 and R * = 0.116.Moreover, as δ 2 increases further, the blue and black curves in Fig.4achange at the same rate, and therefore the output regularity of motifs T1 and T2 is the same over this region.The density functions of the ISIs presented in panels b and c of Fig.4corroborate our findings in Fig.4a.We consider the density functions of the ISIs for three disparate intensities of the driving noise which correspond to points A, B, and C in Fig.4a.Point A is a low level of noise with intensity δ 1 = 0.0023 and corresponds to the orange curve, point
3.2 and 3.3, we investigated the impact of the driving noise intensity, δ 1 , on network synchronization and output regularity with fixed coupling strengths d 3,1 , d 3,2 , and d 2,1 .Next we will consider the effects of the network connectivity on network synchronization and output regularity by considering a variable coupling strength d, where d = d 3,1 , d 3,2 , d 2,1 for the T1 motif, and d = d 3,1 , −d 3,2 , d 2,1 for the T2 motif.To quantify network synchronization we use the root mean square deviation, σ, in Eq. 3 and to quantify output regularity we use the coefficient of variation, R, in Eq. 6.Our findings are presented in Fig. 5, in the form of contour maps of σ and R as functions of δ 1 and d for the T1 motif (panels a and b, respectively) and T2 motif (panels c and d, respectively).As indicated by the colour bars, warmer colours correspond to larger values of both σ and R.
effects of the bifurcation parameter λ 0 on network synchronization and output regularity.And a discussion is given in Sec. 4.