Discrete Dynamics of Dynamic Neural Fields

Abstract

Large and small cortexes of the brain are known to contain vast amounts of neurons that interact with one another. They thus form a continuum of active neural networks whose dynamics are yet to be fully understood. One way to model these activities is to use dynamic neural fields which are mathematical models that approximately describe the behavior of these congregations of neurons. These models have been used in neuroinformatics, neuroscience, robotics, and network analysis to understand not only brain functions or brain diseases, but also learning and brain plasticity. In their theoretical forms, they are given as ordinary or partial differential equations with or without diffusion. Many of their mathematical properties are still under-studied. In this paper, we propose to analyze discrete versions dynamic neural fields based on nearly exact discretization schemes techniques. In particular, we will discuss conditions for the stability of nontrivial solutions of these models, based on various types of kernels and corresponding parameters. Monte Carlo simulations are given for illustration.

1. Introduction

To model a large amount of cells randomly placed together with a uniform volume density and capable of supporting various simple forms of activity, including plane waves, spherical and circular waves, and vortex effects, Beurle (1956) proposed a continuous model consisting of a partial differential equation, that can capture some neurons behaviors like excitability at the cortical level. Amari (1977) and Wilson and Cowan (1972) subsequent modifications did allow for new features such as inhibition. This essentially gave birth to the theory of dynamic neural fields (DNFs) with far reaching applications. The literature on the DNFs is rather rich, and the trends nowadays are toward cognition, plasticity, and robotics. In cognitive neuroscience for instance, DNFs have enabled analyses of Electroencephalograms (Nunez and Srinivasan, 2006), short term memory (Camperi and Wang, 1998), visual hallucinations (Ermentrout and Cowan, 1979; Tass, 1995), visual memory (Schmidt et al., 2002). Durstewitz et al. (2000) have proposed neurocomputational models of working memory using single layer DNFs and shown that the temporal dynamics they obtained compare well with the in vivo observations. Simmering et al. (2008) and Perone and Simmering (2019) used DNFs to achieve spatial cognition by successfully simulating infants' performance in some tasks. Quinton and Goffart (2017) used nonlinearities in DNFs to propose a unifying model for goal directed eye-movements that allow for the implementation of qualitatively different mechanisms including memory formation and sensorimotor coupling. Plasticity in neuroscience can be thought of as the ability of the brain to adapt to external activities by modifying some of its structure. Connections between plasticity and DNFs have been made in studies on intrinsic plasticity, see Strub et al. (2017) or parallel works such as Neumann and Steil (2011), Pozo and Goda (2010), and the references therein. Robotics has been a great niche for DNFs with the works of Bicho et al. (2000), Erlhagen and Schöner (2001), Erlhagen and Bicho (2006), and Bicho et al. (2010).

To recall the theoretical aspects of DNFs, let Ω ⊆ ℝ^d be a manifold where d is positive integer. In the presence of neurons located on Ω at time t arranged on L layers, the average potential function V_k(x_k, t) is often used to understand the continuous field on the kth layer. V_k(x_k, t) is the average membrane potential of the neuron located at position x_k at time t of the kth layer. When L = 1, V(x_k, t) can also be understood as the synaptic input or activation at time t of a neuron at position or direction x_k. It satisfies the equation (see Amari, 1977) which is given as where W_kl(x_k, x_ℓ) is the intensity of the connection between a neuron located at position x_k on the kth layer with a neuron a position x_ℓ on the ℓth layer, G[V_ℓ(x_ℓ, t)] is the pulse emission rate (or activity) at time t of the neuron located at position x_ℓ on the ℓth layer. G is often chosen as a monotone increasing function. S_k(x_k, t) represents the intensity of the external stimulus at time t arriving on the neuron at position x_k on the kth layer, see Figure 1 below.

Figure 1

The model we propose has the advantage of being general and simple enough to implement. It also captures different generalizations of DNFs such as predictive neural fields (PNFs) and active neural fields (ANFs). We further show that the proposed model is in effect a Mann-type (see Mann, 1953) local iterative process which admits weak-type solutions under appropriate conditions. The model can also be written as a recurrent neural network. Stability analysis shows that in some case, the model proposed can be studied using graph theory.

The remainder of the paper is organized as follows: in section 2, we propose our nearly exact discretization scheme for DNFs. In section 3.1, we make a brief overview of the essential notions for the stability analysis of discrete dynamical systems, in section 3.2, we discuss the existence on nontrivial fixed solutions for DDNFs, in sections 3.3 and 3.4, we discuss stability analysis for a one and multiple layers discrete DNFs. In section 3.5, we propose simulations and their interpretation for neural activity and in section 4, we make our concluding remarks.

2. Materials and Methods

2.1. Discrete Schemes for DNFs

For sake of simplification, we will use the following notations: given a positive integer L, and integers 1 ≤ k, ℓ ≤ L, M_k, M_ℓ, we consider indices 1 ≤ i_k ≤ M_k, 1 ≤ j_ℓ ≤ M_ℓ. We put where h_n = t_n+1 − t_n, and is the time scale of the field. We obviously assume that all the layers of the domain Ω have the same time scale. |Ω| represents the size of the field Ω that henceforth we will assume to be finite. x_i represents a point on the kth layer and y_j a point on the ℓth layer. M_k represents the number of interconnected neurons on the kth layer of the domain Ω, and represents the spatial discretization step on the kth layer of the domain Ω. When L = 1, we will adopt the notations and . Henceforth, for sake of clarity and when needed, we will represent vectors or matrices in bold.

To obtain a discrete model for DNFs, we are going to use nearly exact discretization schemes (NEDS) (see Kwessi et al., 2018). We note that according to this reference, Equation (1) is of type T₁, therefore, the left-hand-side will be discretized as For the right-hand-side, we write a Riemann sum for the integral in Equation (1), over the domain Ω as This means that, for a given L ∈ ℕ, for integers 1 ≤ k ≤ L, M_k, and 1 ≤ i_k ≤ M_k, a discrete dynamic neural field (DDNFs) model can be written as Staying within the confines of the layers' architecture proposed by Amari (1977), in a DDNFs with L ≥ 3, we observe that the activity on neurons located on the bottom (or top) layer depends on the interconnected neurons on that layer and their connections to the layer above (or below). First, we define Middle layers are connected to two layers, one above and one below. This means that at time t_n, a neuron at position x_i on the first layer receives external input , intra-layer signals from M₁ neurons on the first layer, and inter-layer signals from M₂ neurons on the second layer. Likewise, a neuron at position x_i on the Lth layer receives external input , inter-layer signals from M_L−1 neurons on the (L − 1)th layer, and intra-layer signals from M_L neurons on the Lth layer. For 1 < k < L, we have if ℓ ∉ L_k and if ℓ ∈ L_k. This means that at time t = t_n, a neuron at position x_i on the kth layer receives external input , inter-layer signals from M_k−1 neurons on the (k − 1)th layer, intra-layer signals from M_k neurons on the kth layer, and inter-layer signals from M_k+1 neurons on the (k + 1)th layer.

Further, Equation (5) has a matrix notation which helps with the study of stability analysis. Indeed, let and d = L × M, let X = (X₁, X₂, ⋯ , X_L) ∈ Ω, where for 1 ≤ k ≤ L, X_k = {(x_k1, x_k2, ⋯ , x_kM)} represents the positions of M_k neurons on the kth layer. We are assuming without loss of generality that x_ki = 0 if M_k < i ≤ M to have a full L × M matrix. Now consider the matrix formed by the column vectors . In this case, Equation (5) can be written in a matrix form as with an element-wise notation for 1 ≤ i ≤ M and for 1 ≤ k ≤ L. For more complex layers' architectures, the interested reader can refer to De Domenico et al. (2013) and the references therein, where tensor products are used instead of matrices.

2.2. Pulse Emission Rate Functions

Amari (1977) proved the existence of nontrivial solutions for DNFs when G is the Heaviside function. Further studies have shown that in fact this is true for a large classes of functions G. As we stated earlier, functions G satisfying the Hammerstein conditions G(v) ≤ μ₁v + μ₂ are good candidates. However most practitioners use either the Heaviside function or the sigmoid function. For some θ > 0 and v₀ ≥ 0, the sigmoid function, is defined as , and Heaviside function is defined as G₂(v, v₀) = I_{(v0, ∞)}(v), see Figure 2 below. Here I_A is the indicator function and is given as I_A(x) = 1 if x ∈ A and I_A(x) = 0 otherwise. This shows that the assumption that G be nonnegative and increasing is not unrealistic. In fact we can transform many functions to bear similar properties of the Heaviside and sigmoid functions, see for instance Kwessi and Edwards (2020).

Figure 2

Remark 1. We observe that the choice of the sigmoid activation functionis widely preferred in the literature for its bounded nature. Another reason is the fact that it is also suitable when theare binary, that is, they may take values 0, 1, where 0 and 1 represents, respectively active and non active neurons at time n. In this case, would represent the probability that there is an activity on neuron at position x_iat time n + 1.

2.3. Connection Intensity Functions

There are a variety of connection intensities functions (or kernels) that one can choose from. This include a Gaussian kernel , the Laplacian kernel defined as , or the hyperbolic tangent kernel Thyp(x, y, β, r) = 1 + tanh(βx^T · y + r) where ∥·∥ is a norm in ℝ^d. Note that as defined, the hyperbolic tangent kernel is not symmetric, however, for a suitable choice of β and r, one may obtain a nearly symmetric kernel. Indeed, if r < 0 and β is close to zero, we have Thyp(x, y, β, r) ≈ (1 + tanh(r))e^{−β∥x−y∥2}, see for instance Lin and Lin (2003). This kernel is very useful in non-convex problems and support vector machines.

In practice however, it is very common to select the function W(x, y) as the difference between either of the functions above, which results in a “Mexican hat” shaped function, that is, either , or , or . Figure 3 below is an illustration of these kernels for selected parameters. This choice will ensure that neurons within the same cell assembly are connected reciprocally by high synaptic weights σ⁺, whereas neurons not belonging to the same assembly are connected by low synaptic weights σ⁻ (see for instance Durstewitz et al., 2000; Quinton and Goffart, 2017; Wijeakumar et al., 2017; Jin et al., 2021).

Figure 3

3. Existence and Stability of Nontrivial Solutions of the DDNFs

3.1. Stability Analysis

We recall the essential notions for the stability analysis of a discrete dynamical system V_n+1 = F(V_n) where F : ℝ^d → ℝ is continuously differentiable map.

3.2. Existence of Nontrivial Fixed Solutions for the DDNFs Model

A nontrivial solution for the DNFs is a non constant solution for which the left-hand side of Equation (1) is equal to zero. For the DDNFs, it can be formulated in the following way: for a given x_i on the kth layer, the DDNFs in (5) has a nontrivial fixed point if for all n ≥ 1, we have . This is satisfied when The right-hand side of Equation (7) is a Riemann sum for So the existence of a nontrivial fixed point for the continuous DNFs Equation (1) implies the existence of a non trivial fixed solution for the DDNFs. According to Hammerstein (1930), such a nontrivial solution exists if W(x, y) is symmetric positive definite and G satisfies G(v) ≤ μ₁v + μ₂, where 0 < μ₁ < 1, μ₂ > 0 are constants.

We will use a different approach to prove existence of nontrivial solution of DNFs. Let be a measure space. We will consider the space L¹(S) of equivalent classes of integrable functions on S, endowed with the norm , where dx = μ(dx).

Definition 5. A density function f is an element of L¹(S) such that f : S → ℝ such that .

Remark 7.

We note that f ≥ 0 (and Pf ≥ 0) represents equivalent classes of integrable functions on S that are greater or equal to 0 except on a set of measure zero and thus the inequality should be understood almost everywhere.

We also note that when an operator P satisfies the second condition above, it is sometimes referred to as a non-expansive operator.

Theorem 8. (Corollary 5.2.1–Lasota and Mackey, 2004). Letbe measure space and P : L¹(S) → L¹(S) be a Markov operator. If for some density function f, there is g ∈ L¹(S) such that Pⁿf ≤ g, for all n, then Now we use Definitions 4, 5, and 6 and Theorem 8 above to state our result on the existence of nontrivial fixed solution of DNFs.

Theorem 9. Let. Consider a DNFs with no external stimulus [S(x) = 0]. If W(x, y) is positive and bounded, then the DNFs (and consequently the DDNFs) has at least one nontrivial fixed solution V_∗.

Proof. Let be the Borel σ-algebra on S generated by the family and μ be the associated Borel measure. A nontrivial solution V(x) for the DNFs satisfies the equation Consider the linear operator P defined on S as Clearly, P is a Markov operator and we have with W₁(x, y) = W(x, y) and for n ≥ 1. Moreover, by the positivity and boundedness of W, there exists M > 0 such that for a density function f, we have Pⁿf(x) ≤ M, ∀n ≥ 1. By the above Theorem, there exists a density f_∗ such that Pf_∗ = f_∗. We conclude by observing that f_∗(x) = G(V_∗(x)) = V_∗(x) for some function V_∗(x) defined on Ω and thus a nontrivial solution for the DNFs and DDNFs exists. That is, V_∗ is a fixed point of G.

One major limitation of the above theorem is that it only applies to positive connectivity functions W(x, y) which would exclude a wide range of DNFs, for example, competing DNFs use almost exclusively negative weight functions as well as DNFs for working memories, see for instance Zibner et al. (2011). One remedy could be to prove the existence of weak fixed solutions of (5) using iterative processes without a positivity restriction. Indeed, let Ω be a nonempty, closed and convex subset of l², the space of infinite sequences v = (v₁, v₂, ⋯ ) such that . We recall that l² is a Hilbert space with the inner product .

Definition 11. Let T : Ω → l²be either a contraction or a non-expansive map. A Mann-type iterative process is a difference equation on Ω defined as

Theorem 12. Suppose Ω is a subset of l²and W(x, y) is bounded by W₀. If G(v) is Lipschitz with Lipschitz constant K such that W₀C ≤ 1, then the DDNFs (5) is a Mann-type iterative process with at least one weak solution.

Proof. Here, we denote V_n to mean V_n(X) introduced earlier. We start by rewriting certain quantities in vector and matrix form. Let d = L × M where as above, and let . Let and , where as above, we complete the vectors with zeros for M_ℓ<j ≤ M. Then we have the dot product Similarly, let and . We also have the dot product Now we consider the matrix formed by block matrices for 1 ≤ k ≤ L. We observe that (5) is a Mann-type process written as where T is the linear map on Ω defined as T(V_n) = Γ · G(V_n)+S_n. It follows that for U_n, V_n ∈ Ω Since by hypothesis CW₀ ≤ 1, T is either a contraction or a non-expansive map. It is known (see Mann, 1953) that in this case, the Mann-type iterative process (9) has at least one weak solution V_∗, that is, there exists V_∗ ∈ Ω such that for all V ∈ l².

The second equation in (9) is a generalization of Equation (7) in Quinton and Goffart (2017) and according to their description, it can be considered a predictive neural field (PNF) where T(V_n) is an internal projection corresponding to the activity required to cancel the lag of the neural field V_n for a specific hypothetically observed trajectory, by exciting the field in the direction of the motion, ahead of the current peak location, and inhibiting the field behind the peak.

3.3. Stability Analysis of Hyperbolic Fixed Solutions

To discuss stability analysis for the DDNFs in (5), we are going to separate the analysis into a single-layer system and a multiple-layers system. For simplification purposes, we consider the following notations when necessary: If L = 1 and for any M₁ > 0, we will write and .

If L > 1 and M_k = M_ℓ = 1 for all 1 ≤ k, ℓ ≤ L, we will write .

3.3.1. Single-Layer DDNFs Model

Let x_i ∈ Ω. Suppose that there are M neurons interconnected and connected to the neuron located at x_i. Let W_ij be the intensity of the connection between the neuron at x_i and the neuron at x_j. Let be the weighted average rate of change of pulse emissions received or emitted by the neuron at x_i, where is a nontrivial solution of Equation (5) with L = 1. Some authors like Lin et al. (2009) specify the type of connection, by dividing the connectivities or synaptic strengths W_ij into a group of receptive signals called “feed backward” and a group of emission signals called “feed forward.” Here we make no such specification. Recall that . Henceforth, we fix h_n = h > 0 for all n so that .

Theorem 13. Let Ω ⊆ ℝ be a bounded region. Let M be a positive integer. Let x = (x_i)_1≤i≤Mbe a vector of M interconnected points on Ω via a connectivity function W(·, ·).

Let V_∗(x) = (V_i,∗)_1≤i≤Mbe a nontrivial fixed point of the DDNFs on Ω.

Proof. We have . Let x_i ∈ Ω and let V_i,∗ be a nontrivial fixed point of the DDNFs with one layer. Let 1 ≤ j ≤ M and x_j ∈ Ω. The DDNFs with one layer in Equation (5) can be written as and element-wise as The function f : ℝ^M → ℝ is a continuously differentiable function since the function G : ℝ → ℝ is, and given 1 ≤ i, p ≤ M, we have that

where δ_ij is the Kronecker symbol with δ_ij = 1 if i = j and δ_ij = 0 if i ≠ j. It follows that We define the adjacency matrix as the Jacobian matrix A = Jf[V_∗(x)] where A = (a_ij) with . A is a M × M matrix and let ρ(A) be its spectral radius. Therefore, we have

Since 0 < α < 1, the last equality is equivalent to It follows that if , then we have that and consequently, V_∗(x) is asymptotically stable.

Remark 14.

The conditionlinks the size of the domain Ω and other parameters such as the rate of change of the pulse emission function G, the connectivity function W(x, y) and the number of connected neurons M. Indeed, . Therefore, achievingrequires. This means that to ensure stability of the fixed solution V_∗(x), it is enough to have weighted rates of change of pulse emissionbe bounded by 1 for each neuron. This condition is obviously true if the neuron at x_iis not connected with the neuron at x_jsince W_ijwould be zero. In fact the latter corresponds to the trivial fixed solution V_∗(x) = 0.

3.3.2. Special Case of Single Layer Model With a Heaviside Pulse Emission Function

Suppose the pulse emission function G(x) is the Heaviside function G₂(v, v₀). There are two possible approaches to discuss the stability of the nontrivial solution. First, we observe that the derivative of G₂(v, v₀) exists in the sense of distribution and is given as where δ(v₀) is the Dirac measure at v₀. It can be written as if v ≠ v₀ and if v = v₀. This means that for a nontrivial solution, the adjacency matrix A defined above is reduced to the diagonal matrix A = diag(α). It follows that ρ(A) = α < 1 and the nontrivial solution V_∗(x) is asymptotically stable. Another approach is to note that Equation (5) with one layer (L = 1) can be written as where with Moreover, we have Suppose that for all n ∈ ℕ and for all x ∈ Ω, S_n(x) and W(x, y) are bounded by say S and W, respectively. Let Y_m = (1 − α)[S + |Ω|W]. For all 1 ≤ i ≤ M, for x_i ∈ Ω, and n ∈ ℕ we have |Y_i,n+1| ≤ Y_m, and This shows that exists and defines the state of the field overtime. If , then the field settles to an inhibitory phase overtime, and if , the field settles in an excitatory phase overtime. In general S_i,n = ν+U_i,n where ν is a negative real number referred to as the resting level that ensures that the DNFs produce no output in the absence of external input U_i,n. If U_i,n ≡ 0 and V₀(x) < 0 for all x ∈ Ω, then , since λ_i,n−k = 0 for all k = 0, ⋯ , n and for all 1 ≤ i ≤ M. Hence we will have , see section 3.5.1 below for an illustration.

3.4. Multiple-Layers DDNFs Model

Let 1 ≤ k, m ≤ L. Consider the DDNFs with L layers given above and let x_i, x_p on the kth layer. Consider a fixed point solution . Given a layer 1 ≤ k ≤ L and a neuron at position x_i on it, there are two sources of variability for the potential : firstly, the variability due to the M_k neurons on the kth layer connected to the neuron at position x_i, and secondly, the variability due to the neurons located on possibly two layers (k − 1) and (k + 1) connected to the kth layer. Therefore, let 1 ≤ m ≤ L and let x_q be a point on the mth layer. By the chain rule, we have where the variability on the kth layer is From Equation (7), we obtain the variability due to layers k − 1 and k + 1: Let . We recall that . Now we consider a M × M super-adjacency matrix , where A^(k,m) is a M_k × M_m block matrix defined as , where with and Consequently, A^(k,m) = 0 if m ∉ L_k, where 0 represents the zero matrix. Therefore, the super-adjacency matrix can be written as Clearly, A is a block tridiagonal matrix that reduces to a block diagonal matrix if the pulse emission rate function G is the Heaviside function and to single M × M diagonal matrix if L = 1. There is a rich literature on the subject of tridiagonal matrices and the research is still on-going. There is no close form formula for the eigenvalues of A since they are not obvious to calculate, except for some special cases.

3.4.1. Special Case: Single Neuron Multiple-Layers DDNFs

It special case that is worth mentioning since all L layers have exactly one neuron and they are interconnected. In fact in this case, M_k = 1, β_k = |Ω| for 1 ≤ k ≤ L, and the super adjacency matrix reduces to an L × L tridiagonal matrix where from Equations (11) and (12), one has We observe that K^(k,k) is the weighted rate of change of the pulse emission function on the kth layer, and K^(k,k+1) can be regarded as the transitional weighted rate of change of the pulse emission function from layer k to layer k + 1. Likewise, K^(k−1,k) is the transitional weighted rate of change of the pulse emission function from layer k − 1 to layer k. From Molinari (2008), the eigenvalues of A are given by the equation Suppose further that a₁ = a₂ = ⋯ = a_L = a, b₁ = b₂ = ⋯ , b_L−1 = b, and c₁ = c₂ = ⋯c_L−1 = c. This means that the weighted rates of change of the pulse emission function for 1 ≤ k ≤ L, for 1 ≤ k ≤ L − 1, and for 2 ≤ k ≤ L. From Kulkarni et al. (1999), the eigenvalues of A will given by Now for 1 ≤ k ≤ L, put We therefore have the following result.

Proof. By hypothesis, a = α + (1 − α)|Ω|K₀, b = [α + (1 − α)|Ω|K₀]|Ω|K₁, and c = [α + (1 − α)|Ω|K₀]|Ω|K₂, for some K₀, K₁, K₂ > 0. For 1 ≤ k ≤ L, the eigenvalues are given as From Thereom 3, the proof is complete.

Remark 16.

• It is worth observing that in case L = 1 and M = 1, then K₁ = K₂ = 0 and thus μ₁ = α + (1 − α)|Ω|K₀. We see that |μ₁ < 1 if |Ω|K₀ < 1 yielding the condition obtained in the one-layer-model.

• We also note that in factand therefore the stability conditionlinks all the parameters of system.

• This special cases is also important in that we can imagine a large number of neurons supposedly placed on individual layers and interconnected in a forward or backward manner only and each receiving external input. In case the first and last neurons are connected so that the system forms a weighted closed graph, like in Figure 4, where the thickness represents the weights.

  Then the super-adjacency matrix given asAccording to Molinari (2008), the eigenvalues are given by

Figure 4

3.4.2. Special Case: Single Neuron Two-Layer DDNFs

We now discuss a single neuron two-layer DDNFs. In this case, we still have M_k = 1 for k = 1, 2 and L = 2. It follows that the super-adjacency matrix is given as where from Equations (11) and (12), we have Let We have the following stability result.

Proof. By the determinant-trace analysis, see for instance Elaydi (2007), it is enough to prove that the above conditions are equivalent to The determinant of the super-adjacency matrix is Also Therefore since θ < 1 and μ < 1, we have that det(A) < 1. Let . The discriminant of P(α) is . Therefore P(α) is positive if Δ < 0 with θ > 0, that is, with θ > 0.

3.5. Simulations

There are multiple parameters that affect the stability of a fixed point solution to the DDNFs: the choice of the kernel function W(x, y), the pulse emission rate function G, parameters such the size of the field |Ω|, the time scale function ϕ(h), and the number of points per layer M, and the length of time N. In these simulations, we will consider the Ω = [−20, 20], h = 0.8, N = 100, M = 200, and the , with , and σ₂ = 4.5. We select a resting phase ν = −0.5. To initialize our dynamical system, we will choose V_{i, 0} = −1.5 for i = 1, 2, ⋯ , M. In Figures 5–8, below, (a) is the three dimension representation of V_n(x): = V(x, n), (b) represents V_n(x) as a function of n for a given x value, (c) represents V_n(x) as function of x for a given n value, and (d) represents the cobweb diagram for a given x value, namely a plot of V_n+1(x) vs. V_n(x) for which the green dot represents the starting point V₀(x) = −1.5 and the red arrows connecting the points with coordinates [V_n(x), V_n(x)] and [V_n(x), V_n+1(x)] to show the evolution of the dynamical system overtime. Convergence occurs when the arrows stop evolving, and oscillations occurs when they circle around indefinitely. In the multi-layer case, we let L = 250.

Figure 5

3.5.1. Single-Layer With Heaviside Function and No External Input

In this simulation, we consider single-layer DDNFs with U_i,n = 0 with a Heaviside pulse emission function, see Figure 5 below.

3.5.2. Single-Layer With Sigmoid Function and No External Input

In this simulation, we consider single-layer DDNFs with U_i,n = 0 but with a sigmoid pulse emission function, see Figure 6 below.

Figure 6

3.5.3. Single-Layer With Heaviside Function With Gaussian External Input

In this simulation, we consider single-layer DDNFs with a Heaviside Pulse emission function with a Gaussian external input , see Figure 7 below.

Figure 7

3.5.4. Single-Layer With Sigmoid Function With Gaussian External Input

In this simulation, we consider single-layer DDNFs with a Sigmoid Pulse emission function with a Gaussian external input , see Figure 8 below.

Figure 8

3.5.5. Multiple-Layer With Sigmoid Function With No External Input

In this simulation, we consider Multiple-layer DDNFs with a Sigmoid pulse emission rate function with no external input, U_i,n = 0, see Figure 9 below.

Figure 9

3.5.6. Multiple-Layer With Sigmoid Function With Gaussian External Input

In this simulation, we consider Multiple-layer DDNFs with a Sigmoid pulse emission rate function with a Gaussian external input, U_i,n = 0, see Figure 10 below.

Figure 10

3.5.7. Effect of α

In this simulation, we consider V_N(x, α) where N = 100 for different values of α. Since the convergence to the fixed solution is quick, we expect V_N(x, α) to be independent of α at n = N = 100, in that any cross-section of the three dimensional plot V_N(x, α) should be the same. This is shown in Figure 11 below for a G(v) = G₁(v) and .

Figure 11

4. Discussion

In this paper, we have proposed a discrete model for dynamical neural fields. We have proposed another proof of the existence of nontrivial solutions for dynamic neural fields. We have discussed its stability of nontrivial solutions of dynamic neural field for some particular cases. The simulations we propose capture very well the notion of stability on dynamic neural field. Below are some observations drawn from this analysis.

• We observe that the cases of no external input U_i,n with one layer and multiple neurons and multiple layers with one neuron each are very similar, albeit the shape of the potential function . In fact, this is not surprising since multiple layers with one neurons each amount to one layer with neurons connected like in a graph.

• An important observation is that of the stability conditions depend on the knowledge of a nontrivial solution

  V_∗

  for the DDNFs. Two things to note about

  V_∗(X)

  are:

  • We can not obtain V_∗(X) in a close form. We may however rely on estimates like the one proposed in Kwessi (2021). Even in the latter case, using an estimate may proved delicate because of limited accuracy.

  • If one cannot use a true value for V_∗(X), one may use the properties of the pulse emission functions such as the Heaviside or sigmoid functions that are in most cases in applications bounded.

• We note the size of the domain Ω is an important parameter for the shape of .

• As for the types of kernels, we note that Gaussian and Laplacian kernel tends to produce similar results that are sometimes very different from those produced by hyperbolic tangent kernels.

• We note that the choice of α = e^−h is similar to an Euler forward discretization with where dt ≪ τ. This choice is made for simplification purposes and also out of an abundance of caution due to the fact for certain one-dimension systems, the Euler forward discretization have been shown not to always preserve the true dynamics from the ordinary differential equation, see for instance Kwessi et al. (2018).

• Another important note is that the parameter α = 1 was not found to be a bifurcation parameter, thus the bifurcation diagram will not be given for sake of simplicity.

• We acknowledge that in the two special cases discussed for multiple-layers DDNFs, so far, at least theoretically, the results apply only to positive connectivity functions W(x, y) because we rely on results previously established in the current literature. Simulations however suggest that they may also extend to negative connectivity functions and it would a worthwhile exercise to see how to establish these results theoretically.

References

• 1

  AmariS.-I. (1977). Dynamics of pattern formation in lateral-inhibition type neural fields. Biol. Cybern., 27, 77–87. 10.1007/BF00337259

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 2

  BeimP. G.HuttA. (2014). Attractor and saddle node dynamics in heterogeneous neural fields. EPJ Nonlin. Biomed. Phys. EDP Sci.2:2014. 10.1140/epjnbp17

  • CrossRef
  • Google Scholar

• 3

  BeurleR. L. (1956). Properties of a mass of cells capable of regenerating pulses. Philos. Trans. R. Soc. Lond. B240. 55–94.

  • Google Scholar

• 4

  BichoE.LouroL.ErlhangenW. (2010). Integrating verbal and non-verbal communication in adynamic neural field for human-robot interaction. Front. Neurorobot.4:5. 10.3389/fnbot.2010.00005

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 5

  BichoE.MalletP.SchönerG. (2000). Target representation on an autonomous vehicle with low-levelsensors. Int. J. Robot. Res.19, 424–447. 10.1177/02783640022066950

  • CrossRef
  • Google Scholar

• 6

  CamperiM.WangX.-J. (1998). A model of visuospatial short-term memory in prefrontal cortex: recurrent network and cellular bistability. J. Comput. Neurosci.4, 383–405. 10.1023/A:1008837311948

  • CrossRef
  • Google Scholar

• 7

  De DomenicoM.Solé-RibaltaA.CozzoE.KiveläM.MorenoY.PorterM. A.et al. (2013). Mathematical formulation of multilayer networks. Phys. Rev. X3:041022. 10.1103/PhysRevX.3.041022

  • CrossRef
  • Google Scholar

• 8

  DurstewitzD. (2017). Advanced Data Analysis in Neuroscience. Bernstein Series in Computational Neuroscience. Cham: Springer.

  • Google Scholar

• 9

  DurstewitzD.SeamansJ. K.SejnowskiT. J. (2000). Neurocomputational models of working memory. Nat. Neurosci.3(Suppl.), 1184–1191. 10.1038/81460

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 10

  ElaydiS. N. (2007). Discrete Chaos. Chapman & Hall; CRC.

  • Google Scholar

• 11

  ElmanJ. L. (1990). Finding structure in time. Cogn. Sci.14, 179–211.

  • Google Scholar

• 12

  ErlhagenW.BichoE. (2006). The dynamics neural field approach to cognitive robotics. J. Neural Eng.3, R36–R54. 10.1088/1741-2560/3/3/R02

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 13

  ErlhagenW.SchönerG. (2001). Dynamic field theory of movement preparation. Psychol. Rev.109, 545–572. 10.1037/0033-295x.109.3.545

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 14

  ErmentroutG. B.CowanJ. D. (1979). A mathematical theory of visual hallucination patterns. Biol. Cybern.34, 137–150. 10.1007/BF00336965

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 15

  HammersteinA. (1930). Nichtlineare integralgleichungen nebst anwendungen. Acta Math.54, 117–176. 10.1007/BF02547519

  • CrossRef
  • Google Scholar

• 16

  JinD.QinZ.YangM.ChenP. (2021). A novel neural modelwith lateral interactionfor learning tasks. Neural Comput.33, 528–551. 10.1162/neco_a_01345

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 17

  KulkarniD.ScmidtD.TsuiS. K. (1999). Eigen values of tridiagonal pseudo-toeplitz matrices. Linear Algeb. Appl.297, 63–80.

  • Google Scholar

• 18

  KwessiE. (2021). A consistent estimator of nontrivial stationary solutions of dynamic neural fields. Stats4, 122–137. 10.3390/stats4010010

  • CrossRef
  • Google Scholar

• 19

  KwessiE.EdwardsL. (2020). Artificial neural networks with a signed-rank objective function and applications, in Communication in Statistics–Simulation and Computation. 1–26. 10.1080/03610918.2020.1714659

  • CrossRef
  • Google Scholar

• 20

  KwessiE.ElaydiS.DennisB.LivadiotisG. (2018). Nearly exact discretization of single species population models. Nat. Resour. Model.31:e12167. 10.1111/nrm.12167

  • CrossRef
  • Google Scholar

• 21

  LasotaA.MackeyM. C. (2004). Chaos, Fractals, and Noise. Applied Mathematical Sciences. New York, NY: Springer-Verlag.

  • Google Scholar

• 22

  LinH.-T.LinC.-J. (2003). A study on sigmoid kernels for svm and the training of non-psd kernels by smotype methods. Neural Comput.13, 2119–2147.

  • Google Scholar

• 23

  LinK. K.Shea-BrownE.YoungL.-S. (2009). Spike-time reliability of layered neural oscillator networks. J. Comput. Neurosci.27, 135–160. 10.1007/s10827-008-0133-3

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 24

  MannW. R. (1953). Mean value methods in iteration. Proc. Am. Math. Soc.4, 506–510.

  • Google Scholar

• 25

  MolinariL. G. (2008). Determinant of block tridiagonal matrices. Linear Algeb. Appl.429, 2221–2226. 10.1016/j.laa.2008.06.015

  • CrossRef
  • Google Scholar

• 26

  NeumannK.SteilJ. J. (2011). Batch intrinsic plasticity for extreme learning machines, in International Conference on Artificial Neural Networks (Berlin: Springer), 339–346.

  • Google Scholar

• 27

  NunezP. L.SrinivasanR. (2006). Electric Fields of the Brain: The Neurophysics of EEG, 2nd Edn. Oxford University Press. 10.1093/acprof:oso/9780195050387.001.0001

  • CrossRef
  • Google Scholar

• 28

  PeroneA.SimmeringV. R. (2019). Connecting the dots: finding continuity across visuospatial tasks and development. Front. Psychol.2019:1685. 10.3389/fpsyg.2019.01685

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 29

  PozoK.GodaY. (2010). Unraveling mechanisms of homeostatic synaptic plasticity. Neuron66, 337–351. 10.1016/j.neuron.2010.04.028

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 30

  QuintonJ. C.GoffartL. (2017). A unified dynamic neural field model of goal directed eye-movements. Connect. Sci.30, 20–52. 10.1080/09540091.2017.1351421

  • CrossRef
  • Google Scholar

• 31

  SchmidtB. K.VogelE. K.WoodmanG. F.LuckS. J. (2002). Voluntary and automatic attentional control of visual working memory. Percept. Psychophys.64, 754–763. 10.3758/bf03194742

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 32

  SimmeringA. R.SchutteV. R.SpencerJ. P. (2008). Generalizing the dynamic field theory of spatial cognition across real and developmental time scales. Brain Res.1202, 68–86. 10.1016/j.brainres.2007.06.081

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 33

  StrubC.SchönerG.WörgötterF.SandamirskayaY. (2017). Dynamic neural fields with intrinsic plasticity. Front. Comput. Neurosci.11:74. 10.3389/fncom.2017.00074

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 34

  SussiloD.BarackO. (2013). Opening the black box: low-dimensional dynamics in high-dimensional recurrent neural networks. Neural Comput.25, 626–649. 10.1162/NECO_a_00409

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 35

  TassP. (1995). Cortical pattern formation during visual hallucinations. J. Biol. Phys.21, 177–210.

  • Google Scholar

• 36

  WijeakumarS.AmbroseJ. P.SpencerJ. P.CurtuR. (2017). Model-based functional neuroimaging using dynamic neural fields: an integrative cognitive neuroscience approach. J. Math. Psychol.76(Pt B), 212–235. 10.1016/j.jmp.2016.11.002

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 37

  WilliamsR. J.ZipserD. (1990). A learning algorithm for continually running fully recurrent neural networks. Neural Comput.1, 256–263.

  • Google Scholar

• 38

  WilsonH. R.CowanJ. D. (1972). Excitatory and inhibitory interactions in localized populations ofmodel neurons. Biophys. J.12, 1–24. 10.1016/S0006-3495(72)86068-5

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 39

  ZibnerS. K. U.FaubelC.IossifidisI.ShönerG. (2011). Dynamic neural fields as building blocks of a cortex-inspired architecture for robotic scene representation. IEEE Trans. Auton. Ment. Dev.3, 74–91. 10.1109/TAMD.2011.2109714

  • CrossRef
  • Google Scholar