Closed-loop Robots Driven by Short-Term Synaptic Plasticity: Emergent Explorative vs. Limit-Cycle Locomotion

We examine the hypothesis, that short-term synaptic plasticity (STSP) may generate self-organized motor patterns. We simulated sphere-shaped autonomous robots, within the LPZRobots simulation package, containing three weights moving along orthogonal internal rods. The position of a weight is controlled by a single neuron receiving excitatory input from the sensor, measuring its actual position, and inhibitory inputs from the other two neurons. The inhibitory connections are transiently plastic, following physiologically inspired STSP-rules. We find that a wide palette of motion patterns are generated through the interaction of STSP, robot, and environment (closed-loop configuration), including various forward meandering and circular motions, together with chaotic trajectories. The observed locomotion is robust with respect to additional interactions with obstacles. In the chaotic phase the robot is seemingly engaged in actively exploring its environment. We believe that our results constitute a concept of proof that transient synaptic plasticity, as described by STSP, may potentially be important for the generation of motor commands and for the emergence of complex locomotion patterns, adapting seamlessly also to unexpected environmental feedback. We observe spontaneous and collision induced mode switchings, finding in addition, that locomotion may follow transiently limit cycles which are otherwise unstable. Regular locomotion corresponds to stable limit cycles in the sensorimotor loop, which may be characterized in turn by arbitrary angles of propagation. This degeneracy is, in our analysis, one of the drivings for the chaotic wandering observed for selected parameter settings, which is induced by the smooth diffusion of the angle of propagation.

• The magenta robot displays the T1 mode, which coexists with the S1 and C1 motion patterns. Bumping into the red robot, first the center of the star-shaped trajectory is shifted. After the second collision it switches to the C1 mode, precessing transiently to the left, while as a result of a third collision it sets into the rightwards circulating mode.
• The light-blue (cyan) colored robot stays in T2 mode even after the collision, since for these parameters (see Fig. 4) other stable motion pattern does not exist.
• The same holds for the C1 mode as well, demonstrated by the orange robot. Note that the radii of the circular patterns generated by the orange and the magenta robots differ significantly, an effect of the significantly different w 0 parameters.
• The S1 mode is visualized by the dark-blue spherical robot. The collision induces an autonomous switch to C1.
• S2 mode is essential for our results, being a singleton locomotive mode in its parameter domain. Hence interactions with other robots or walls can only change the direction of translation. In Suppl. Video 1 one can observe seven such collisions.
Supplementary Video 1. Examples of regular motions patterns T1, T2, C1, S1, S2, S3, generated by identical robots, using U max = 1. The (w 0 , z 0 ) parameter pairs are set for each robots separately, corresponding to the values used for the close-up of the trajectories in Fig. 4 (having initially the same order and color coding). When several modes coexist for the same parameters, robots interacting with each other can change their behavior, choosing an other working mode. See for instance the green robot, changing from S3 to S1 after colliding with the yellow one. See: Video 1.avi.
Supplementary Video 2. Close-up showing the dynamics of the robot in the chaotic mode, using U max = 1 and (w 0 , z 0 ) = (210, 400). The smaller balls fixed to the end of the rods are guides to the eyes, indicating the negative direction along the axes, without any physical influence on the dynamics. The obstacles, having similar masses to the mass of the robot, are pushed apart in different directions. See: Video 2.avi.

Playful behavior
The playful behavior of the spherical robot is shown in Suppl. Video 2 and 3, for the parameter settings used for Fig. 6, U max = 4 and (w 0 , z 0 ) = (210, 400). Autonomous mode switching can be observed between the coexisting chaotic meandering and C1 modes, pushing objects, of masses similar to its own body mass, in a playful manner.

Explorative behavior
In Suppl. Video 4 the two types of chaotic explorations are compared in a maze, also shown in Fig. 10, using U max = 1 and U max = 4. Additional stable modes also arise in environments different from the bare plane ground (see the locomotion along the walls).
Supplementary Video 3. The playful behavior of the robot in the closed playground, viewed from the top. The parameters are set as for Suppl. Video 2, corresponding to the chaotic mode. The blue trajectory, plotted on the ground, traces the C1 locomotion pattern occasionally out, which is unstable for these settings (as discussed in the main text of the paper). See: Video 3.avi.

CHAOTIC MODES
To examine the different modes from a dynamical systems point of view, one can consider the corresponding attractors in the subspace of internal variables (see also (Sándor et al., 2015)). The chaotic attractor corresponding to the explorative behavior with abrupt change of directions, found for U max = 1, has a densely filling structure, as shown in Suppl. Fig. 5. For U max = 4, with the angle of propagation only smoothly diffusing, however, the topology resembles closed braids, a sign of partial predictability, typically found close to period doubling bifurcations (Wernecke et al., 2016). As a result of the attractor's topology the discrete mode switching can never occur for U max = 4 (see Fig. 12).
Note that for U max = 4 there are totally three chaotic attractors in the phase space of internal variables, which are ultimately generating the same type of explorative behavior, being related by symmetry operations (corresponding to the permutation of colors in Fig. 12). Hence, in this case discrete mode switching (between these attractors), similar to the S2 case, can be achieved by adding external noise.