Abstract
Researchers working with neural networks have historically focused on either non-spiking neurons tractable for running on computers or more biologically plausible spiking neurons typically requiring special hardware. However, in nature homogeneous networks of neurons do not exist. Instead, spiking and non-spiking neurons cooperate, each bringing a different set of advantages. A well-researched biological example of such a mixed network is a sensorimotor pathway, responsible for mapping sensory inputs to behavioral changes. This type of pathway is also well-researched in robotics where it is applied to achieve closed-loop operation of legged robots by adapting amplitude, frequency, and phase of the motor output. In this paper we investigate how spiking and non-spiking neurons can be combined to create a sensorimotor neuron pathway capable of shaping network output based on analog input. We propose sub-threshold operation of an existing spiking neuron model to create a non-spiking neuron able to interpret analog information and communicate with spiking neurons. The validity of this methodology is confirmed through a simulation of a closed-loop amplitude regulating network inspired by the internal feedback loops found in insects for posturing. Additionally, we show that non-spiking neurons can effectively manipulate post-synaptic spiking neurons in an event-based architecture. The ability to work with mixed networks provides an opportunity for researchers to investigate new network architectures for adaptive controllers, potentially improving locomotion strategies of legged robots.
1. Introduction
Current research employing neural networks for locomotion control tends to focus on homogeneous networks of neurons communicating through either graded signals (Aoi et al., ) or action potentials (Bing et al., ). However, studies indicate that biological neural networks utilize both communication strategies (Burrows, ) to achieve effective locomotion. Based on this, our research introduces a biologically-inspired non-spiking interneuron (NSI) model into a spiking neural network (SNN) to further increase biological fidelity. In nature, sensor neurons receive information from the external environment and pass it onto NSIs through current injections (Bidaye et al., ). This data is sent onwards by the NSI, affecting the membrane potential of the connected neurons through a graded signal (Burrows and Siegler, ). However, NSIs are not only translational units. They are also found to be the primary neuronal type in some animals such as the C. elegans where communication through graded potentials is the main transmission method (Schafer, ). Thereby, interneurons are computational units in and of themselves. Figure 1 illustrates a simplified neural pathway depicting an analog input from the environment producing movement by an insect (Figure 1A) and the equivalent pathway implemented in this study (Figure 1B).
Figure 1
Individual neurons can be described using different models which try to capture the dynamics of a biological neuron. In this study, we use two spiking neuron models, one works in its intended fashion but the other operates within its sub-threshold range so that the membrane potential never surpasses the spiking threshold. Spiking neuron models try to replicate biological neurons by calculating the membrane potential of the neuron at each time step. The membrane potential is affected by incoming spikes, bias current, and other parameters depending on the model's equation. Classical non-spiking neuron models used in artificial neural networks (ANNs) attempt to replicate neuron dynamics using a transfer function such as the sigmoid function. These non-spiking models are able to map values but they cannot integrate an input over time without a recurrent connection. Therefore, our paper uses a spiking neuron model in a non-spiking “mode” to create the NSI.
SNNs typically communicate through action potentials commonly called spikes. Spikes allow information to be encoded through the frequency of spikes as well as the timing of individual spikes (Bohte, ). This creates the possibility to transfer more information within a spike train as compared to traditional ANNs (Bing et al., ). SNNs are used in our research so future work incorporating data-rich sensory input can take advantage of temporal features for encoding input information.
As SNNs are typically homogeneous, all communication is handled by passing spikes between neurons. If an analog sensor is added to the network, the information must be encoded into spikes for the network to understand. This is also true for an output signal which must be converted from spikes to continuous values to control a motor. The known encoding mechanisms found in nature include individual spike rate (Adrian, ), population activity (Panzeri et al., ), and precise spike timing (Bohte, ). Neural network engineers have applied each of these methods for translating sensory input, grouping the different approaches into three main categories: rate, population, and temporal coding respectively.
Each encoding category has recognized strengths. Population coding is able to relay more information than individual neurons so it can be useful for data-rich inputs (Mallot, ). On the other hand, temporal coding is particularly applicable for encoding streaming data as it quickly processes information (Petro et al., ) while maximizing the amount of information contained within the compressed data (Sengupta and Kasabov, ). Finally, while other methods are able to encode more information, rate coding has been suggested to be the best tool for handling input with high firing rates (Azarfar et al., ). In our work, rate coding is used to filter the output of the network from spikes to a continuous motor signal and we introduce the NSI model as a hybrid encoding method able to directly translate input by itself or work together with rate, population, or temporal methods to increase the amount of information encoded.
1.1. Related Work
ANNs have been shown to effectively manipulate amplitude, frequency, and phase in legged robots to create adaptive controllers (Nachstedt et al., ; Schilling et al., ; Barikhan et al., ; Dürr et al., ; Pitchai et al., ; Thor and Manoonpong, ). Thor and Manoonpong () used an error signal to update synaptic weights for adaptation of frequency to optimize walking, resulting in increased efficiency and reduced tracking error. Pitchai et al. () also created an energy-efficient control mechanism for a legged robot by using an ANN to shape network outputs coupled with a non-spiking central pattern generator (nCPG) to change frequency. Nachstedt et al. () were able to create a self-tuning network using adaptive oscillators which allowed a robot to navigate a more complex environment. Barikhan et al. () showed that a decoupled nCPG network using sensory feedback to adapt to the environment was able to handle changes in robot morphology and could coordinate movement with another robot when working on cooperative tasks. Schilling et al. () used rules for coordination to adapt walking according to sensory input. Similarly, Dürr et al. () developed an ANN to control a hexapod robot which relied on feedback for posture and rules for coordination. Their network produced emergent gaits, adjusting based on the posturing of the robot. The addition of non-spiking interneurons to ANNs has been investigated by Szczecinski et al. (). They reported control of a hexapod robot using interneurons modeled as classical non-spiking neurons to trigger different bio-inspired reflexes. The interneurons were used to control output oscillations, indicating that shaping nCPG outputs via interneurons to reproduce biological behaviors is possible. Our paper combines the use of NSIs and spiking neurons to update amplitude, frequency, and phase, as a step toward creating a more biologically plausible adaptive controller capable of interpreting temporal data.
Pure SNNs are also capable of manipulating output amplitude, frequency, and phase by updating different synaptic and neuronal characteristics (Strohmer et al., ). It was found that frequency could be changed by updating the value of the voltage threshold potential of the spiking central pattern generator (sCPG) neuron populations while amplitude was increased or decreased using the weight of the synaptic conductance to the motor neuron population (MNP). Finally, phase was determined by the network architecture and synaptic delays. However, as input currents to NSIs are known to change firing rates of connected motor neurons, reset rhythmic output (Bidaye et al., ), and update amplitude (von Uckermann and Büschges, ), it is useful to look into how they interact with an sCPG network to shape these outputs.
Woźniak et al. () integrated spiking neurons into an ANN to take advantage of their power-saving potential and temporal data encoding capabilities. They implemented the spiking neurons as a “spiking neural unit” consisting of two non-spiking neurons, one of which handles integration of the membrane potential and the other to emit a spike. The spiking neural unit dynamics were modeled after the leaky integrate-and-fire model of spiking neurons. Their integration of the membrane potential was handled through a recurrent connection while spiking was mimicked using a step function. This approach differs from our implementation as it defined the spiking neurons as a combination of non-spiking neurons whereas we take the dynamics of a spiking neuron model and use it in the sub-threshold region to create a non-spiking neuron model.
Patil et al. () took a similar approach to our research and created a non-spiking neuron modeled as a spiking neuron that communicated via graded potentials. Their work was based on the neural architecture of the C. elegans which is mostly composed of non-spiking neurons though recent research indicates more advanced sensory systems in the worm might use spiking neurons (Liu et al., ). The paper showed that a mixed neural network could be built to mimic the escape response of the C. elegans upon being touched externally. Their mixed network was implemented as neuromorphic hardware, simulating both non-spiking and spiking neurons using analog circuitry, as opposed to our research using simulation software to mathematically model neurons.
Niu et al. () explore first principles of the sensorimotor loop using neuromorphic hardware designed on a field-programmable gate array. They are able to successfully model spiking neurons and muscle physiology in hardware to reproduce stretch reflexes in both an attached cadaveric finger and a robotic finger. This study reinforces the expectation that a simplified model of the sensorimotor loop produces realistic behavior in an anatomical system. Other biological studies investigate the architecture of the central pattern generator (CPG) itself, using multi-layered CPG networks to generate rhythmic output and implementing mutual inhibition on several layers (see review by McCrea and Rybak, ). Markin et al. ()'s CPG network tests the removal of descending signals and is able to regain stable locomotion by increasing synaptic weights associated with sensory feedback. Conversely, Danner et al. () present a network reliant on both descending and ascending signals for coordination of motion. They add interneurons to their network to communicate between legs in addition to relaying descending/ascending signals, finding that some gait patterns were not achievable when certain interneurons were removed. Further work by Mileusnic et al. () suggest a mathematical model for muscle spindles which are the sense organs responsible for relating length and velocity of muscles. The paper confirms the ability to recreate biological data, providing another way to investigate the role of sensory feedback in locomotion. Expanding upon this research, Raphael et al. () use the muscle spindle model in their “spinal-like regulator” capable of processing 184 control inputs, controlling an approximation of the human hand using both descending inputs and sensory feedback. Our work integrates ideas from this biological research by investigating first principles of communication within a mixed neural network.
Our paper presents a novel encoding mechanism, sub-threshold encoding, that uses NSIs to relate values from analog sensory inputs to an SNN. Our proposed method is referred to as sub-threshold encoding because the output is a consequence of membrane potential fluctuations below the spiking threshold. In contrast to using current injections to directly manipulate network output, the momentary membrane potential of the NSI plays a role in how the post-synaptic neurons are affected. The main contribution of this research is the introduction and investigation into how these NSIs can be integrated into an existing SNN to shape network output.
2. Methods
Biological research indicates that sensory input to NSIs affects motor output (Büschges and Wolf, ). Furthermore, depolarizing currents received by NSIs are shown to reset biological central pattern generator (bCPG) rhythms (Bidaye et al., ). We can replicate these behaviors by creating an equivalent neural network representing the sensorimotor neuron pathway. In the neural network, the sensor neuron is replaced by an input bias current to the NSI (see Figure 1B) so that a change in input current represents a change in sensory information from the external environment. The NSI then relates this information to the connected SNN in order to adjust the network output. Figure 2B shows the block diagram of the selected SNN, an sCPG network. The architecture of a pair of mutually inhibitory neuron populations is based on the biology of spiking oscillators driving an antagonistic muscle pair (Bidaye et al., ). The output from the sCPG network is spikes but an analog signal is required for control of a motor. Therefore, Figure 2A demonstrates how these spike events are converted to an analog signal using rate coding. This mimics the low-pass filtering performed by biological muscles (Hooper et al., ) by counting the amount of spikes occurring within a time window to produce an analog value. Figure 2C highlights the characteristics of the rate-coded output to be adjusted by the input to the NSI.
Figure 2
A diagram of the implemented network is shown in Figure 3A. The network consists of an NSI capable of injecting current and manipulating the voltage characteristics of the post-synaptic neurons.
Figure 3
The sCPG populations and MNP consist of 5 neurons each in order to produce a smooth output with a minimum amount of neurons (Strohmer et al., ). The NSI is a single neuron so that all post-synaptic neurons connected to it receive the same inputs. This reduces the complexity of the experiments so that the tests focus on the communication from the NSI. The sCPG populations and MNP are comprised of adaptive exponential integrate-and-fire (AdEx) neurons to allow for bursting behaviors (Brette and Gerstner, ). Neuronal parameters are set based on regular bursting as outlined in Naud et al. () and their dynamics are shown in Equations (1) and (2).
when Vm > 0mV then Vm→Vreset
when Vm > 0mV then w→w + b
where C is the membrane capacitance, Vm is the membrane potential, EL is the resting potential, gL is the leakage conductance, Ie is the bias current plus Gaussian white noise, a is the sub-threshold adaptation conductance, b is the spike-triggered adaptation, ΔT is the sharpness factor, τw is the adaptation time constant, Vth is the voltage threshold potential, Vreset is the reset potential, and w is the spike adaptation current (Naud et al., ). Equation (1) defines the change in membrane potential per time step whereas Equation (2) outlines the current adaptation.
The NSI is simulated as a leaky integrate-and-fire neuron, the dynamics of the neuron are shown in Equation (3). The spiking threshold is set high enough to avoid spiking so there is no reset condition. The simpler neuron model is used for the NSI because the only necessary behavior is the integration of input current and leakage.
where τm = RC is the membrane time constant, Vm is membrane potential, Iinput is input bias current plus Gaussian white noise, and R is membrane resistance.
Yang et al. () show that the magnitude of the NSI output is a graded function of the difference between the interneuron's membrane potential and its rest potential. Additionally, they find a linear correlation between the signal produced by the NSI and the response from the post-synaptic neuron. Based on this knowledge, we construct a relation between the NSI membrane potential and the effect on the post-synaptic neuron (Equations 4 and 5).
where Vcm is voltage characteristic manipulation, Vrest is rest potential, and Vm is membrane potential.
The application of Vcm in the network is further described in section 2.1 outlining experimentation.
Equation (4) provides the offset to the voltage characteristic being adjusted for an sCPG neuron population. Vrest is set to −60mV to stay consistent with biological findings (Graubard, ). The practical implementation of Vrest uses the starting value of the NSI membrane potential as it fluctuates around the desired resting potential due to noise added to the system. The divisor in Equation (4) limits the voltage characteristic offset within a stable range. It is determined by dividing the biologically plausible 15mV NSI membrane potential fluctuation range (Burrows and Siegler, ) with the 5mV voltage characteristic manipulation range known to be stable for the sCPG network (Strohmer et al., ).
Equation (5) calculates the amount of current (Iinjection), in pA, to be added to the original current bias (Ie) of the post-synaptic neuron.
where Iinjection is current injection, w is synaptic conductance weight, Vrest is rest potential, and Vm is membrane potential.
The synaptic conductance weight, w, scales the current injection from the NSI to the post-synaptic neuron. Conductance is measured in Siemens (S), the inverse of Ohms (Ω−1).
2.1. Time-Driven Experimentation
Two main categories of tests are performed on the network, excitatory and inhibitory. The input bias current (Iinput) to the NSI is always positive, this is the input current defined in Equation (3). However, the injection current (Iinjection) from the NSI to the post-synaptic neurons changes sign depending on test type, sending a positive current if excitatory and negative current if inhibitory. The value of Iinjection is determined by Equation (5). Testing is further broken into interactions between the NSI and post-synaptic neuron populations. The overview of these tests is outlined in Table 1 and visualized in Figure 4.
Table 1
| Test no. | Current injection (MNP) | Current injection (sCPG) | Voltage characteristic (sCPG) |
|---|---|---|---|
| NSI Communication Testing | |||
| 1 | Stepping | None | None |
| 2 | None | Stepping | None |
| 3 | Stepping | Stepping | None |
| 4 | Stepping | None | Vth |
| 5 | Stepping | None | Vreset |
| 6 | Stepping | None | Vm |
| 7 | No | None | Vth |
| 8 | Stepping | Stepping | Vth |
| 9 | None | None | Vreset |
| 10 | Stepping | Stepping | Vreset |
| 11 | None | None | Vm |
| 12 | Stepping | Stepping | Vm |
Test code reference.
Figure 4
The tests step through different configurations of injecting current and manipulating voltage characteristics of post-synaptic neuron populations as seen in Figure 4. The voltage characteristics tests separately investigate adjusting voltage threshold potential (Vth), voltage reset (Vreset), and membrane potential (Vm) within Equations (1) and (2). The amount of the offset is determined by the voltage characteristic manipulation Equation (4). The offset generated by Vcm is applied to the individual characteristic being tested. When Vcm is added to the membrane potential (Vm) of the neuron, it is applied after Equation (1) has been calculated, allowing Vcm to be added directly to the Vm at that time step. On the other hand, when Vcm is applied to Vth or Vreset, it is added to the initial values set for each. Therefore, when testing with one of these characteristics the actual value of either is as follows in Equations (6) and (7).
where Vth_initial = −56mV and Vreset_initial = −41mV for excitatory tests of that characteristic; Vth_initial = −51mV and Vreset_initial = −46mV for inhibitory tests of that characteristic.
Each of the voltage characteristic manipulations is tested on individual sCPG neuron populations as well as both simultaneously, these are defined by test subcategories seen in Table 2 and visualized in Figure 5. When a test is running that does not update a particular voltage characteristic, the following values are used: Vth = −51mV and Vreset = −46mV. These values are known to produce a regular bursting pattern by the sCPG network (Strohmer et al., ). The MNP never changes voltage characteristics, it only receives current injections because it is not involved in generating rhythmic patterns, only shaping the output.
Table 2
| Test subcategory | Current injection type | Excitatory sCPG neuron | Inhibitory sCPG neuron |
|---|---|---|---|
| NSI Communication Testing | |||
| _0 | Excitatory | Yes | Yes |
| _1 | Excitatory | Yes | No |
| _2 | Excitatory | No | Yes |
| _3 | Inhibitory | Yes | Yes |
| _4 | Inhibitory | Yes | No |
| _5 | Inhibitory | No | Yes |
Test subcategory code reference.
Figure 5
The test results are compared by plotting the rate-coded output of the MNP. A sliding time window of 5ms is used, counting all spikes from the MNP occurring within each time window to produce an analog value. The output signal from the MNP is considered the output of the network because this would be the signal used to control a motor when testing on a physical robot.
Static Iinput tests are performed first to find the maximum value for the conductance weight, w (Equation 5). These static tests are not outlined in the above tables as they are only for parameter tuning. In order to find w, Iinput is set to the maximum value determined by the change in the NSI membrane potential, restricting it to the biologically plausible range of 15mV (Burrows and Siegler, ). Then w is increased by increments of 10nS until the maximum weight is found. The maximum is accepted as the largest weight value to produce an ideal output signal, this is further discussed in section 3. This value is set as the maximum conductance weight (wmax) for both excitatory and inhibitory tests. After this is found, all test configurations update Iinput at regular intervals to confirm a change in analog input is able to manipulate network output online. These stepping input current trials all use wmax to calculate Iinjection in Equation (5). When the test is excitatory, Iinput starts at 0pA and ends at the determined maximum, the opposite is true when the test is inhibitory. This allows the system to be “excited” by reducing inhibition so the same general behavior can be expected at the output.
Neural Simulation Tool (NEST) (Jordan et al., ) is used to simulate the network and record test results. The simulation is run for 6s for all trials. The input current to the NSI updates every 1 s, allowing the network to settle after initial transients before the input changes again. Iinput starts at 0pA and ends at the maximum, 148pA. Thus, each step adds 29.6pA of input current. Gaussian white noise current is added to all neurons in the network. The standard deviation of the noise current to the NSI is set to 25pA so that the noise current is comparable to the current steps. When testing with larger values than this, the output no longer reliably produces the necessary offsets for the voltage characteristic manipulation. For all other neurons, the standard deviation is 50pA. The standard deviation of the noise currents are selected and have not been tuned. A bash script is used to run tests in a reliable manner. The bash script and the python test script are available on GitLab (see Data Availability Statement).
After performing all of the test combinations for frequency and amplitude manipulation as outlined in Tables 1, 2, one further test is performed to see how output phase is affected by switching between frequencies. The frequency is either held constant for 6s or toggled between 4Hz and 8Hz updating every second for 6s. There is no current injection to the motor population for these tests in order to examine phase as affected solely by frequency. This test differs from previous phase manipulations of an sCPG network where phase was determined by synaptic delay (Strohmer et al., ). Instead of trying to create a specific phase shift, this trial only confirms that phase is affected by changing frequencies and does not attempt to control it.
Insects use proprioceptive sensor neurons to understand the position of their limbs in relation to their body. These internal feedback loops can help an insect engage resistance reflexes to maintain posture (Tuthill and Wilson, ). This knowledge is used to implement a simplified network simulating an internal feedback loop (illustrated in Figure 3B) as a proof of concept application. The rate-coded output from the MNP determines the strength of the synapse connecting the NSI and MNP. A number of trials are run to determine an appropriate scaling factor for excitatory and inhibitory weight calculations.
Varying the synaptic conductance weight, w, will regulate the output amplitude for this internal feedback loop. Equation (8) determines the w to be used in Equation (5) instead of using wmax. The scaling factor is a value which allows for maximum effect without over-exciting or over-inhibiting the output. The desired spike number acts as a set point whereas the current spike number is the actual number of spikes from the MNP occurring in the last time step. In this network an inhibitory weight is applied if the number of spikes in the time window is above the desired number and an excitatory weight if below.
2.2. Event-Driven Experimentation
In order to confirm the NSI's compatibility with a neuromorphic architecture, the network is also simulated on CloudBrain (Larsen et al., ). CloudBrain is a scalable event-based SNN simulation platform utilizing event stream processing technologies to communicate between neurons implemented as microservices on a cluster. The NSI in CloudBrain is implemented as an event-based leaky integrator neuron so that the neuron model is comparable to the NEST simulation. The neuron updates its membrane potential upon receipt of an input event according to the dynamics shown in Equation (9).
Where ΔVm is the change in membrane potential, Vrest is the rest potential, decay is the decay factor simulating leakage, dt is the time since last incoming spike, Iinput is the input current from other neurons, and C is the membrane capacitance.
Equation (9) is the event-driven equivalent of the Equation (3) in the time-driven system. The event-driven simulation does not include noise as the experiment is only designed to confirm functionality of an NSI. In addition to updating the membrane potential, the NSI saves the time stamp of last received input (tupdated).
Each microservice consists of a neuron and all incoming synapses, so the post-synaptic AdEx neurons cannot directly access the voltage of the NSI. Instead, the NSI parameters (V, Vrest, Vm, decay, and tupdated) are sent to the synapses connected to the AdEx neurons each time they are updated. Based on this information, the synapse asynchronously calculates the exact membrane potential of the neuron as needed using Equation (10).
Where t is the current time, Vm is the membrane potential, Vrest is the rest potential, decay is the decay factor simulating the leakage, and tupdated is the time at which the parameters were updated.
A network consisting of an NSI and an sCPG is created in CloudBrain to confirm the NSI is able to update the voltage threshold potential of the post-synaptic neurons in an event-based architecture. In this setup, the sCPG is comprised of two single neurons mutually inhibiting each other. Figure 3C shows the block diagram of the network in CloudBrain. The input to the NSI is spikes since it is an event-based architecture. The input spikes are the equivalent of the stepping input current used in the time-based NEST simulation. A step function is encoded to spikes using Ben's Spiker Algorithm (Schrauwen and Van Campenhout, ) for transmission to the NSI. In turn, the NSI updates the Vth (as described in Equation 1) of the sCPG neurons based on the frequency of spiking input. The Vth of the sCPG neurons is calculated using a linear mapping of the membrane potential. This allows us to control the upper and lower limits of the Vth. The NSI's Vm is recorded for comparison with the Vth and spiking output of the sCPG populations.
3. Results
Simulating with a constant Iinput of 148pA shows the NSI membrane potential starts close to the rest potential, −60mV and ends around −45mV, though these values vary slightly due to noise. This gives us the maximum allowable Iinput that restricts the NSI's membrane potential within a 15mV range.
The maximum synaptic conductance weight for the NSI output synapse is determined to be wmax = 70nS. The effect on amplitude as compared to a lower weight of 2nS is visible in Figure 6. Increasing w to 80nS produces unwanted lifting behavior from the excitatory current injection where the rate-coded output does not always return to zero. This result can be seen in Supplementary Figure 1.
Figure 6
Figure 6 shows that amplitude is affected by a change in Iinjection without altering the frequency or phase of the output for both excitatory (Figure 6A) and inhibitory (Figure 6B) tests. The filled circles on the plots illustrate the peak values (local maxima) of the rate-coded output. The difference in average peak value when comparing 2 vs. 70nS is 28.37 spikes when excitatory and 16.69 when inhibitory.
Table 3 shows the progressive increase in average number of spikes per 5ms time window based on the size of Iinjection to the MNP. This reveals that the change in the number of output spikes due to Iinjection is scalable over a range.
Table 3
| Calculated current | Conductance, | Average peak | Average peak |
|---|---|---|---|
| injection, Iinjection (pA) | w (nS) | (excitatory) | (inhibitory) |
| 30 | 2 | 24.26 | 23.00 |
| 150 | 10 | 26.50 | 20.76 |
| 300 | 20 | 29.33 | 18.10 |
| 450 | 30 | 32.19 | 14.43 |
| 600 | 40 | 36.33 | 12.74 |
| 750 | 50 | 41.14 | 10.64 |
| 900 | 60 | 46.94 | 8.02 |
| 1,050 | 70 | 52.63 | 6.31 |
Average maximum number of spikes per time window when injecting excitatory or inhibitory current to the MNP. Iinjection shown is an approximation, calculated using Equation (5) where Vm = −45mV and w is as defined in the table.
Iinjection is positive when excitatory and negative when inhibitory. The average peak number of spikes increases and decreases with excitation and inhibition respectively.
3.1. Online Amplitude Manipulation
The online manipulation of Iinjection to the MNP results in a change in the number of spikes per time window for both excitatory and inhibitory tests (Figures 7A,B). The addition of Iinjection to both of the sCPG populations changes the behavior of spiking, shifting the phase in comparison to the static input current tests (Figures 7C–F). Additionally, the strong inhibition of both sCPG neuron populations leads to complete suppression of the output.
Figure 7
Injecting current to one sCPG population at a time creates instability in the system. Supplementary Figures 4A,B show that the excitation of the excitatory sCPG population produces lifting behavior when Iinjection reaches a specific threshold. Likewise, an increase in excitation of the inhibitory neuron population dampens MNP spiking.
All figures in the Supplementary Material present both excitatory and inhibitory trials. Excitatory current injections produce a larger change in amplitude whereas inhibition suppresses output up to a certain point in many tests.
3.2. Online Frequency Manipulation
Figure 8 compares frequency manipulation of the output with and without current injection to the MNP. Updating either Vth or Vreset of the sCPG neuron populations points to a linear relationship with frequency.
Figure 8
Based on the voltage characteristic manipulation calculation (Equation 4), Vth and Vreset are either increased or decreased by approximately 1 at each time step. Figures 8A–D show that their manipulation results in an increase in frequency by the addition of approximately 1 peak per second, implying a linear relationship. Figures 9A–C reinforce the expectation of a linear relationship between frequency and the change in Vth or Vreset. By contrast, adding an offset to Vm does not reliably change the frequency and introduces instability as the current injection increases. Figure 9D shows that the frequency does not trend in a particular direction for the Vm manipulation. Additionally, as the inhibitory test's output is suppressed for much of the duration of the trial, there are fewer data points as compared to other tests.
Figure 9
Figure 9A highlights the similarity in the frequency change when comparing Vth and Vreset. This is also true whether the test is excitatory or inhibitory. Supplementary Table 1 shows the exact start and end frequency values for the voltage characteristic manipulation tests. The starting frequency for Vth and Vreset tests is between 2.36 and 2.94Hz, ending between 8.01 and 8.94Hz. This reveals frequency can be affected by a factor of 3.
Amplitude is also affected by frequency, the average number of maximum spikes for each frequency can be seen in Supplementary Table 2. The lower the frequency, the more spikes are counted per time window, resulting in a higher value of the rate-coded output. The relationship does not appear to be linear as the number of spikes is reduced by a larger amount when comparing the lowest 3 frequencies as compared to the highest 3 frequencies.
3.3. Online Phase Manipulation
Phase is affected by frequency manipulation as seen in Figure 10. Figures 10A,B both show the initial 1,000 time steps of the simulation produce the exact same results for amplitude and phase. However, after a frequency change the phase shifts.
Figure 10
3.4. Online Amplitude Regulation Using Internal Feedback
Figure 11 shows the effects of amplitude regulation when updating the frequency of the network by adjusting Vth in Equations (1) and (2). Vth changes based on the offset determined by Equation (4). The scaling factors found to have the most effect without over-inhibition or over-excitation are 50 for inhibition and 2 for excitation (insert into Equation 8).
Figure 11
The amplitude based on a “desired spike value” (see Equation 8) of 10 vs. 50 confirms that the network is monitoring the output and adjusting. This is seen more drastically when comparing the control experiment (originally test 4_0) and the trial with a desired spike value of 10. Regardless of whether the test is increasing or decreasing frequency, the amplitude is adjusted to a similar value. Figure 11 confirms the number of spikes per time window is consistent based on the desired value and the frequency. Furthermore, the phase is unaffected by this amplitude regulation.
3.5. Event-Driven Manipulation
The step input to the NSI resulting in an increase in Vth of the AdEx neuron is seen in Figure 12A. The y-axis is adjusted for each graph so the height of the curves should not be directly compared. However, the time axis is common to both graphs and shows the Vth follows the change in Vm almost immediately.
Figure 12
Figure 12B confirms the output frequency of the AdEx neuron is increased when the input spike bursting frequency increases. The graph shows the increase in input bursting frequency starts ramping up the Vm of the NSI. This promotes an increase in output bursting frequency from the AdEx neurons. The AdEx neurons are confirmed to spike out of phase with each other and increase to the same bursting frequency.
4. Discussion
The results confirm our model of an NSI is capable of shaping output and setting rhythmic patterns of an sCPG network based on a changing analog input. The amount of influence the NSI has on the output is constrained since the change in membrane potential must stay within the biologically plausible 15mV range. However, the range still allows at least a doubling of average output spikes per time window from the MNP and triple the frequency when moving from a low to high input current to the NSI. The output from the MNP also dictates usable parameter ranges. The lifting behavior observed when using a synaptic conductance larger than 70nS means that the output cannot always return to zero due to some neurons always spiking. This is not ideal as an output signal so the maximum conductance to the post-synaptic neurons is limited to 70nS.
The baseline stepping current tests (1_0 and 1_3) confirm amplitude can be manipulated online without changing the behavior of the system by updating the injection current to the MNP. Table 3 shows that the difference in amplitude can be controlled using synaptic conductance. It can logically be concluded that the average number of maximum spikes per time window when no current is injected is between 23 and 24.26. The number of spikes either increases or decreases from this starting point depending on if the connection is excitatory or inhibitory. The difference between maximum spikes over the range of current tested is steady, reinforcing the conclusion that amplitude is reliably altered by current injection. If the synaptic weight is changed in addition to the current injection, the average number of spikes per time window can be regulated from 6.31 to 52.63. This configuration increases the average spike difference to a factor of 8 (52.63/6.31 = 8.34).
Tests involving a current injection to the sCPG neuron populations produce a phase shift. This is expected because they are the populations driving the rhythmic output. The delay generated could be an exploitable feature when coordinating multiple joints. However, the exact effect has not been calculated and the delay changes based on the level of current injection (see Figure 7).
The linear relationship found between frequency and voltage threshold potential seen in Figure 9 is consistent with previous research by Strohmer et al. (). It follows that reset potential has a similar relationship to frequency as voltage threshold potential. Both characteristics change the amount a neuron's membrane potential must change before reaching the spiking threshold. Adding an offset directly to a post-synaptic population's membrane potential introduces discontinuities, creating small “jumps” based on the value of the offset. The frequency does not appear to be linearly correlated with the change in Vm when looking at Figure 9D. However, this might be due to the limited range of the offset tested in this study.
The comparison of frequency to the maximum amount of spikes depends on the size of the time window chosen for rate coding as well as the number of neurons within the motor population. Likewise, a larger number of neurons in the motor population increases the maximum possible spikes. The results shown in this study are based on a time window of 5ms and a MNP size of 5. When comparing the start and end frequencies for Supplementary Table 1 against the average frequencies in Supplementary Table 2, the static tests show a higher starting frequency (both tables are located in the Supplementary Material). The static tests average frequency over a total of 5 s, removing the first second in case of transients. However, Test 4_0 updates the frequency every 1 second so the minimum frequency and maximum frequency shown are not averaged but single calculations. The averaging process most likely accounts for this difference in start and end frequencies. This limits the reliable change in frequency to a difference of 2.8x.
Toggling between a low and high frequency over a single trial as compared to a constant frequency produces a phase shift as expected due to the alteration of the output signal's period. This phase shift is with respect to the sCPG network's own output from the MNP and does not indicate how frequency changes might affect a larger system with connected oscillators. However, these results imply that an NSI is able to reset rhythmic behavior of an sCPG network in accordance with biological research (Bidaye et al., ).
Trials that only affected a single sCPG neuron population at a time created network instability, most likely due to the change in network dynamics. The sCPG populations are mutually inhibitory so the excitation or inhibition of one population affects the balance of the network. This is particularly visible in Supplementary Figure 4 where over-excitation of the excitatory neuron causes lifting and over-inhibition of the excitatory neuron suppresses network output. Based on these results, a stable and predictable output requires updating both sCPG neuron populations with the same current injection or voltage characteristic manipulation. This conclusion can only be inferred for a mutually inhibitory sCPG architecture. Furthermore, this result cannot predict stability over time or with different noise profiles.
Reviewing the frequency tests with and without current injection to the motor population shows that the amplitude adjusts based on the level of current injection without affecting frequency. Furthermore, the frequency is equally affected by a change in Vth or Vreset but biological research shows evidence that voltage threshold adaptation is a strategy used by neurons to change firing rate (Azarfar et al., ). Therefore, based on research findings and biological research, our recommended approach for parameter manipulation of a mutually inhibitory sCPG network is to inject current to the MNP while updating the voltage threshold potential of the sCPG neuron populations.
The regulation of amplitude is a building block for developing closed-loop adaptive controllers with NSIs. The proof of concept network implemented shows that NSIs are capable of controlling output based on live feedback. The method is solely for demonstration purposes and is not necessarily biologically plausible though there is evidence that short-term plasticity affects rhythmic outputs (McDonnell and Graham, ) and that the synaptic weight influences the magnitude of the post-synaptic potential (Burrows, ). Testing the network reveals visible overshoot from the desired set point but this is expected because the spike value must surpass the set point before inhibition initiates. The lower frequencies are also worse at reducing the number of spikes, this is probably because of the significant excitation received from the sCPG populations. As can be seen in the control experiment, the maximum numbers of spikes per time window can reach 100 for the lowest frequency trial. It is more difficult to dampen the effects of these high spike counts than the lower spike counts seen in higher MNP output frequencies.
The results of the event-based CloudBrain simulation are consistent with the time-based NEST simulation showing that output frequency can be manipulated based on input to the NSI. This suggests that the NSI can translate both spiking and analog data into usable information for the network. The ability to receive spiking input increases the usability of the NSI as a possible encoding tool, indicating potential for interfacing with event-driven sensors. Additionally, the ability to run the NSI on an event-based simulation in real-time means that it can be applied in closed-loop control of robots.
5. Conclusion
The method introduced in our research is able to integrate an NSI into an SNN to create a mixed network. Our model NSI is a biologically plausible input value encoder, receiving analog values as Iinput and passing the information to spiking neuron populations which naturally output spikes. This research confirms that the amplitude, frequency, and phase of an sCPG network can be manipulated based on changing input to an NSI, implying that an NSI can function as an encoding mechanism within an SNN. Furthermore, the ability of the network to adjust to an internal signal suggests that this setup could be useful in adaptive controllers.
Our recommended architecture for integrating an NSI with a mutually inhibitory sCPG network is shown in Figure 4, plot 4,5,6, allowing frequency to be adjusted by changing the voltage threshold potential. In this specific setup, the average frequency of the network can be regulated between 3.0Hz and 8.5Hz. Additionally, the regulation of synaptic conductance weight from the NSI allows an average peak output amplitude between approximately 6 and 52 spikes per time window. This assumes a rate-coding time window of 5ms and 5 neurons for the MNP as investigated in this study. The recorded frequency and amplitude ranges are determined by the architecture and parameter ranges so it would be advantageous to look further into the dynamics of this system to maximize flexibility. Additionally, the resolution of attainable frequencies should be evaluated as this will be an important metric for adaptive control.
An implementation of an NSI should be further researched to compare sub-threshold use of existing spiking neuron models to a unique non-spiking model. Further research into both the offset and injection current equations as well as their respective parameters is also necessary. Additionally, the leakage constant of the NSI could be optimized to ensure biological-plausibility and effectiveness. The relationships between frequency and phase as well as frequency and amplitude should be quantified so that these dependencies can be fully exploited.
The finding that amplitude can be scaled based on the value of the current injection to the MNP leads to the possibility of using an NSI to prioritize sensory inputs or inform coordination tasks. Insect inter-leg coordination is known to depend on sensory input not only from the local leg but also neighboring legs (Bidaye et al., ). Therefore, synaptic weights from a single NSI population could be tuned in order to inject more current to the local leg while also providing varying amounts of injection current to neighboring legs.
The event-based demonstration on the CloudBrain platform indicates that implementation on neuromorphic hardware is possible. A closed-loop mixed network should be further studied in CloudBrain since it can communicate with a robot in real-time (Larsen et al., ). This provides the opportunity for investigation of a mixed network adaptive controller using environmental interaction.
The ability of sub-threshold encoding to shape an sCPG network's output opens up possibilities for new approaches to coordination and control tasks. Storchi et al. () report observing the use of combinations of encoding mechanisms in biological systems, indicating that the use of multiple methods within a single robot could be a fruitful investigation direction for adaptive controllers.
Statements
Data availability statement
The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found below: https://gitlab.sdu.dk/stroh/interneuron-encoding.
Author contributions
BS and LL developed the main idea of the paper. BS researched the biological concepts, formulated a theory, and applied it within a time-based simulation tool. LL and RS conceived of the event-based NSI model. RS implemented and validated neuron and synapse implementations. RS implemented the sCPG in CloudBrain. The manuscript was written by BS with support from LL and PM. LL supervised the project and provided the funding. All authors contributed to the article and approved the submitted version.
Funding
This research was funded by the SDU Biorobotics group at the University of Southern Denmark.
Acknowledgments
This paper is available on the BioRxiv pre-print server.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Supplementary material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnins.2021.633945/full#supplementary-material
References
1
AdrianE. D. (1926). The impulses produced by sensory nerve endings: part I. J. Physiol. 61:49. 10.1113/jphysiol.1926.sp002273
2
AoiS.ManoonpongP.AmbeY.MatsunoF.WörgötterF. (2017). Adaptive control strategies for interlimb coordination in legged robots: a review. Front. Neurorobot. 11:39. 10.3389/fnbot.2017.00039
3
AzarfarA.CalciniN.HuangC.ZeldenrustF.CelikelT. (2018). Neural coding: a single neuron's perspective. Neurosci. Biobehav. Rev. 94:238–247. 10.1016/j.neubiorev.2018.09.007
4
BarikhanS. S.WörgötterF.ManoonpongP. (2014). “Multiple decoupled cpgs with local sensory feedback for adaptive locomotion behaviors of bio-inspired walking robots,” in International Conference on Simulation of Adaptive Behavior (Castellón: Springer), 65–75. 10.1007/978-3-319-08864-8_7
5
BidayeS. S.BockemühlT.BüschgesA. (2018). Six-legged walking in insects: how cpgs, peripheral feedback, and descending signals generate coordinated and adaptive motor rhythms. J. Neurophysiol. 119, 459–475. 10.1152/jn.00658.2017
6
BingZ.MeschedeC.RöhrbeinF.HuangK.KnollA. C. (2018). A survey of robotics control based on learning-inspired spiking neural networks. Front. Neurorobot. 12:35. 10.3389/fnbot.2018.00035
7
BohteS. M. (2004). The evidence for neural information processing with precise spike-times: a survey. Nat. Comput. 3, 195–206. 10.1023/B:NACO.0000027755.02868.60
8
BretteR.GerstnerW. (2005). Adaptive exponential integrate-and-fire model as an effective description of neuronal activity. J. Neurophysiol. 94, 3637–3642. 10.1152/jn.00686.2005
9
BurrowsM. (1996). The Neurobiology of an Insect Brain. Oxford University Press. 10.1093/acprof:oso/9780198523444.001.0001
10
BurrowsM.SieglerM. (1978). Graded synaptic transmission between local interneurones and motor neurones in the metathoracic ganglion of the locust. J. Physiol. 285, 231–255. 10.1113/jphysiol.1978.sp012569
11
BüschgesA.WolfH. (1995). Nonspiking local interneurons in insect leg motor control. I. Common layout and species-specific response properties of femur-tibia joint control pathways in stick insect and locust. J. Neurophysiol. 73, 1843–1860. 10.1152/jn.1995.73.5.1843
12
DannerS. M.ShevtsovaN. A.FrigonA.RybakI. A. (2017). Computational modeling of spinal circuits controlling limb coordination and gaits in quadrupeds. eLife6:e31050. 10.7554/eLife.31050
13
DürrV.ArenaP. P.CruseH.DallmannC. J.DrimusA.HoinvilleT.et al. (2019). Integrative biomimetics of autonomous hexapedal locomotion. Front. Neurorobot. 13:88. 10.3389/fnbot.2019.00088
14
GraubardK. (1978). Synaptic transmission without action potentials: input-output properties of a nonspiking presynaptic neuron. J. Neurophysiol. 41, 1014–1025. 10.1152/jn.1978.41.4.1014
15
HooperS. L.GuschlbauerC.von UckermannG.BuschgesA. (2007). Different motor neuron spike patterns produce contractions with very similar rises in graded slow muscles. J. Neurophysiol. 97, 1428–1444. 10.1152/jn.01014.2006
16
JordanJ.MarkH.VennemoS. B.TerhorstD.PeyserA.IppenT.et al. (2019). Nest 2.18.0. 10.5281/zenodo.2605422
17
LarsenL. B.StagstedR. K.StrohmerB.ChristensenA. L. (2021). Cloudbrain: real-time neural computation in the cloud. bioRxiv. 10.1101/2021.01.21.427662
18
LiuQ.KiddP. B.DobosiewiczM.BargmannC. I. (2018). C. elegans awa olfactory neurons fire calcium-mediated all-or-none action potentials. Cell175, 57–70. 10.1016/j.cell.2018.08.018
19
MallotH. A. (2013). “Coding and representation,” in Computational Neuroscience (Springer), 113–129. 10.1007/978-3-319-00861-5_5
20
MarkinS. N.KlishkoA. N.ShevtsovaN. A.LemayM. A.PrilutskyB. I.RybakI. A. (2010). Afferent control of locomotor cpg: insights from a simple neuromechanical model. Ann. N. Y. Acad. Sci. 1198:21. 10.1111/j.1749-6632.2010.05435.x
21
McCreaD. A.RybakI. A. (2008). Organization of mammalian locomotor rhythm and pattern generation. Brain Res. Rev. 57, 134–146. 10.1016/j.brainresrev.2007.08.006
22
McDonnellM. D.GrahamB. P. (2017). Phase changes in neuronal postsynaptic spiking due to short term plasticity. PLoS Comput. Biol. 13:e1005634. 10.1371/journal.pcbi.1005634
23
MileusnicM. P.BrownI. E.LanN.LoebG. E. (2006). Mathematical models of proprioceptors. I. Control and transduction in the muscle spindle. J. Neurophysiol. 96, 1772–1788. 10.1152/jn.00868.2005
24
NachstedtT.WörgötterF.ManoonpongP.AriizumiR.AmbeY.MatsunoF. (2013). “Adaptive neural oscillators with synaptic plasticity for locomotion control of a snake-like robot with screw-drive mechanism,” in 2013 IEEE International Conference on Robotics and Automation (Karlsruhe: IEEE), 3389–3395. 10.1109/ICRA.2013.6631050
25
NaudR.MarcilleN.ClopathC.GerstnerW. (2008). Firing patterns in the adaptive exponential integrate-and-fire model. Biol. Cybernet. 99:335. 10.1007/s00422-008-0264-7
26
NiuC. M.JalaleddiniK.SohnW. J.RocamoraJ.SangerT. D.Valero-CuevasF. J. (2017). Neuromorphic meets neuromechanics, part I: the methodology and implementation. J. Neural Eng. 14:025001. 10.1088/1741-2552/aa593c
27
PanzeriS.MackeJ. H.GrossJ.KayserC. (2015). Neural population coding: combining insights from microscopic and mass signals. Trends Cogn. Sci. 19, 162–172. 10.1016/j.tics.2015.01.002
28
PatilS.ZhouK.ParkerA. C. (2015). “Neural circuits for touch-induced locomotion in Caenorhabditis elegans,” in 2015 International Joint Conference on Neural Networks (IJCNN) (Killarney: IEEE), 1–8. 10.1109/IJCNN.2015.7280408
29
PetroB.KasabovN.KissR. M. (2019). Selection and optimization of temporal spike encoding methods for spiking neural networks. IEEE Trans. Neural Netw. Learn. Syst31, 358–370. 10.1109/TNNLS.2019.2906158
30
PitchaiM.XiongX.ThorM.BilleschouP.MailänderP. L.LeungB.et al. (2019). “CPG driven RBF network control with reinforcement learning for gait optimization of a dung beetle-like robot,” in International Conference on Artificial Neural Networks (Munich: Springer), 698–710. 10.1007/978-3-030-30487-4_53
31
RaphaelG.TsianosG. A.LoebG. E. (2010). Spinal-like regulator facilitates control of a two-degree-of-freedom wrist. J. Neurosci. 30, 9431–9444. 10.1523/JNEUROSCI.5537-09.2010
32
SchaferW. (2016). Nematode nervous systems. Curr. Biol. 26, R955–R959. 10.1016/j.cub.2016.07.044
33
SchillingM.HoinvilleT.SchmitzJ.CruseH. (2013). Walknet, a bio-inspired controller for hexapod walking. Biol. Cybernet. 107, 397–419. 10.1007/s00422-013-0563-5
34
SchrauwenB.Van CampenhoutJ. (2003). “BSA, a fast and accurate spike train encoding scheme,” in Proceedings of the International Joint Conference on Neural Networks, 2003 (Portland: IEEE), 2825–2830. 10.1109/IJCNN.2003.1224019
35
SenguptaN.KasabovN. (2017). Spike-time encoding as a data compression technique for pattern recognition of temporal data. Inform. Sci. 406, 133–145. 10.1016/j.ins.2017.04.017
36
StorchiR.BaleM. R.BiellaG. E.PetersenR. S. (2012). Comparison of latency and rate coding for the direction of whisker deflection in the subcortical somatosensory pathway. J. Neurophysiol.108, 1810–1821. 10.1152/jn.00921.2011
37
StrohmerB.ManoonpongP.LarsenL. B. (2020). Flexible spiking CPGs for online manipulation during hexapod walking. Front. Neurorobot. 14:41. 10.3389/fnbot.2020.00041
38
SzczecinskiN. S.ChrzanowskiD. M.CoferD. W.TerrasiA. S.MooreD. R.MartinJ. P.et al. (2015). “Introducing mantisbot: hexapod robot controlled by a high-fidelity, real-time neural simulation,” in 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (Hamburg: IEEE), 3875–3881. 10.1109/IROS.2015.7353922
39
ThorM.ManoonpongP. (2019). A fast online frequency adaptation mechanism for CPG-based robot motion control. IEEE Robot. Automat. Lett. 4, 3324–3331. 10.1109/LRA.2019.2926660
40
TuthillJ. C.WilsonR. I. (2016). Mechanosensation and adaptive motor control in insects. Curr. Biol. 26, R1022–R1038. 10.1016/j.cub.2016.06.070
41
von UckermannG.BüschgesA. (2009). Premotor interneurons in the local control of stepping motor output for the stick insect single middle leg. J. Neurophysiol. 102, 1956–1975. 10.1152/jn.00312.2009
42
WoźniakS.PantaziA.BohnstinglT.EleftheriouE. (2020). Deep learning incorporating biologically inspired neural dynamics and in-memory computing. Nat. Mach. Intell. 2, 325–336. 10.1038/s42256-020-0187-0
43
YangS. M.VilarchaoM. E.RelaL.SzczupakL. (2013). Wide propagation of graded signals in nonspiking neurons. J. Neurophysiol. 109, 711–720. 10.1152/jn.00934.2012
Summary
Keywords
spiking neural network, non-spiking interneuron, neuromorphic engineering, mixed network, biologically plausible neuron, bio-inspired engineering
Citation
Strohmer B, Stagsted RK, Manoonpong P and Larsen LB (2021) Integrating Non-spiking Interneurons in Spiking Neural Networks. Front. Neurosci. 15:633945. doi: 10.3389/fnins.2021.633945
Received
26 November 2020
Accepted
09 February 2021
Published
05 March 2021
Volume
15 - 2021
Edited by
Alice Cline Parker, University of Southern California, Los Angeles, United States
Reviewed by
János Botzheim, Budapest University of Technology and Economics, Hungary; Suraj Chakravarthi Raja, University of Southern California, Los Angeles, United States
Updates
Copyright
© 2021 Strohmer, Stagsted, Manoonpong and Larsen.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Beck Strohmer stroh@mmmi.sdu.dk
This article was submitted to Neuromorphic Engineering, a section of the journal Frontiers in Neuroscience
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.