Abstract
The BrainScaleS-2 system is an established analog neuromorphic platform with versatile applications in the diverse fields of computational neuroscience and spike-based machine learning. In this work, we extend the system with a configurable realtime event interface that enables a tight coupling of its distinct analog network core to external sensors and actuators. The 1,000-fold acceleration of the emulated nerve cells allows us to target high-speed robotic applications that require precise timing on a microsecond scale. As a showcase, we present a closed-loop setup for commuting brushless DC motors: we utilize PyTorch to train a spiking neural network emulated on the analog substrate to control an electric motor from a sensory event stream. The presented system enables research in the area of event-driven controllers for high-speed robotics, including self-supervised and biologically inspired online learning for such applications.
1 Introduction
Often perceived to be one of the key features of biological cognitive systems, embodiment has become subject to increasing interest within the neuroscientific community (Kiverstein and Miller, ; Deng et al., ). While state-of-the-art neural networks often outperform humans in cognitive tasks, artificial agents still severely lack behind their mammalian counterparts in applications requiring precise control and coordination of movement. The field of neurorobotics tries to overcome these deficits by tightly interconnecting physical agents to brain-inspired computing solutions. This approach promises multiple advantages compared to the classical separation of control and cognition: on the one hand, it allows research on how to leverage the advantage biological nervous systems show when executing movement in complex environments. On the other hand, our understanding for artificial cognitive systems may improve when their response to a permanent stream of complex real-world stimuli is examined. The required tight coupling can be achieved by linking simulations of biologically inspired neural networks to either real-world robots (Hagras et al., ) or virtual agents in simulated environments (Falotico et al., ). While the latter in principle allows for arbitrary complexity in both, the physics and the neural network simulation, runtime constraints often lead to significant size and fidelity limits for both components. Physical robots, in contrast, inherently experience the rich dynamics of a real-world environment. This poses strong timing requirements on the—simulated—nervous system, which needs to interact with its surroundings in realtime. Especially for biologically inspired spiking neural networks (SNNs), this prerequisite has led to the deployment of specialized neuromorphic accelerators for neurorobotic tasks (Richter et al., ; Blum et al., ; Milde et al., ; Kreiser et al., ; Yan et al., ; DAngelo et al., ). These systems integrate specific analog or digital modules for the efficient simulation—or emulation—of spiking neurons and are thereby capable of running such networks at biological realtime. This makes them a versatile tool for robotic applications operating on time scales similar to those accessible by their biological example.
In contrast to these established neurorobotic platforms, the accelerated BrainScaleS-2 (BSS-2) system operates with time constants that are 1,000-fold accelerated compared to their biological counterparts, allowing the system to tackle robotic tasks far beyond human speed (Schreiber, ). In this work, we present a flexible realtime event interface for BSS-2 and showcase its application with a simple closed-loop controller for brushless electric motors. This problem is well-studied and has been solved with a multitude of different algorithms, both for senosored and sensorless applications (Pillay and Krishnan, ; Ko et al., ; Acarnley and Watson, ; Rodriguez and Emadi, ; Sathyan et al., ; Bosso et al., ). While an applicable agent can be built with affordable off-the-shelf components, it does require precise timing far beyond human performance and—due to the ubiquity of electric motors—still has high technological relevance. To showcase the realtime event interface for BSS-2, we therefore present a basic controller for sensored brushless motors that has been trained to mirror the behavior of a classical controller using standard machine-learning techniques and hardware-in-the-loop training (Schmitt et al., ; Yao et al., ). We envision this setup to be a versatile platform for research on the interaction of biologically plausible spiking neural networks with fast physical systems.
2 A realtime event interface for BrainScaleS-2
BSS-2 is a hybrid neuromorphic system that has primarily been designed to facilitate research of biologically inspired SNNs (Figures 1A, B; Pehle et al., ). In its core, it features 512 individually configurable analog neurons, each of which is capable of reproducing the rich dynamics of the adaptive-exponential (AdEx) neuron model with high fidelity (Brette and Gerstner, ; Billaudelle et al., ). Multiple of these neuron circuits can be combined to implement multi-compartment neurons and thereby resemble the spatial structure of biological dendrites (Kaiser et al., ). The neurons receive input from a total of 131,072 synapses per chip, each featuring a digitally programmable weight of 6 bit resolution. These synapses can be plastic on various different time scales: while short-term plasticity is natively integrated into the analog synapse array, spike-timing-dependent plasticity is implemented through the combination of synapse-local correlation sensors with two embedded SIMD processors. These plasticity processing units (PPUs) can be programmed to execute arbitrary C++ code at runtime and therefore be used to realize user-defined plasticity rules on-chip. To do so, they have access to various different observables in the analog and digital domain, e.g., event counters, membrane potentials or the correlation signals mentioned previously.
Figure 1
Experiments on the BSS-2 system can be formulated using two different software frontends: for SNNs focussing on the complex dynamics found in biological systems, a backend for the PyNN meta-language (Davison et al.,
Due to the tight realtime requirements of the system, these instructions are executed on an intermittent FPGA, which is accessible via Gigabit Ethernet. It buffers the instruction stream compiled on the host computer, streams the data to the BSS-2 ASIC, buffers network responses from the neuromorphic chip and sends them back to the host (Figure 1E, white components). As part of this work, we have extended this FPGA by a second event interface, which allows for event injection and extraction in realtime (Figure 1E, yellow components) and thereby enables robotic applications with the BSS-2 system.
To allow for a substantial amount of (virtual) spike channels without excessive I/O pin demands, we have chosen a serial protocol for receiving and transmitting sensor and motor events. Specifically, we integrate an Universal Asynchronous Receiver/Transmitter (UART), which is widely supported across all classes of embedded devices. Since the realtime nature of the application does not require time-stamped events, any word communicated over this interface represents the channel ID of an event happening at the current time. The word size—and therefore the amount of virtual event channels—can be configured at build time, while the bitrate is a runtime parameter that can be adapted to the requirements of different external spike sources and sinks. For the data link layer we rely on an open-source implementation,1 which uses octuple oversampling to decode events received via the asynchronous interface. With a system clock of 125 MHz and a single start and stop bit, we therefore expect a serialization latency of
where we have set an exemplary interface width of w = 8bit (256 event channels). Correspondingly, the link layer is expected to saturate at . The measurements depicted in Figures 1C, D indicate that these values are reached in the physical implementation: while the measured saturated event rate of 1.5MEvs-1 directly matches the expected Bmax, the measured median loopback latency of results from the sum of tserial and the serialization latency between FPGA and ASIC. The latter is expected to add an additional delay of at least 268 ns in each direction.
For translation between external events and the BSS-2 spike format, individual routing tables for the sending and receiving directions are connected to the UART module. These tables utilize block RAM on the FPGA to provide a dense, runtime-configurable mapping between external event addresses and those used by the neuromorphic system. Users can query the BSS-2 software for the spike labels assigned to their sensor and motor neuron populations. These values can then be used to program an arbitrary mapping to external event channel IDs.
3 Showcase: BLDC motor control
To showcase the application of the presented realtime event interface in a real-world scenario, we have selected the task of commuting an electric brushless DC (BLDC) motor. We have chosen this problem due to the simplicity of the physical setup and its precise timing requirements: BLDC motors—together with sensor and power electronics—are widely available off-the-shelf components, yet they rely on precisely controlled currents to operate reliably. With typical rotational speeds far above 10,000 min–1, the timing requirements for enabling individual coils are in the sub-millisecond regime and therefore go far beyond human capabilities (Kemp,
For the particular demonstration, we use a small gimbal motor with 12N14P configuration and a nominal rotation rate of 205 min-1. Its current rotational angle is read out with 10 bit resolution by a magnetic sensor, which communicates these values via I2C to a microcontroller (Figure 2A, top). The latter serves as a bridge between the sensor and the BSS-2 event interface and is primarily used for encoding the current rotational angle into spikes. Algorithmically, we choose a value-unit representation, for which we pool the 1,024 available raw states into 256 sensor neurons. The resulting events are then injected into the analog network emulation on BSS-2 and processed by a single feed-forward layer of six leaky integrate-and-fire (LIF) neurons. Taking inspiration from biology, we utilize pulse-density modulation for the efferent motor signals. Similar to aforementioned antagonistic muscle pairs in biology, the activity of the output neurons directly corresponds to the state of the six transistors controlling the motor: each of the three phases i can either be driven high (only active), driven low (only active), or left floating. The decoding of the output event stream is handled by a second microcontroller, which momentarily enables the corresponding signal in a 6 bit parallel bus whenever a motor neuron spikes (Figure 2A, bottom). These signals are used by the output stage of a commercial motor driver to control an internal 3-phase H-bridge. Any activity of the motor neurons emulated on BSS-2 thereby creates physical movement, which in turn results in a change of the rotational angle and therefore potentially different inputs to the SNN.
Figure 2

(A) Block diagram of the implemented closed-loop motor controller. The rotational angle is measured, converted to events and sent to the BSS-2 system in realtime. These sensory inputs are then processed by the spiking motor controller. Its motor outputs are converted to control signals for an H-bridge driver, which controls the coils of a BLDC motor. (B) Experimental setup for running the closed-loop experiment. The neuromorphic system is depicted on the right, with the BSS-2 ASIC hidden below the red dust cap. It controlls the small electric motor at the top, which drives a green propeller. The oscilloscope displays the membrane potential of two analog motor neurons on BSS-2. (C) Network weights before (left) and after (right) supervised training. The mapping between sensory input (vertical dimension) to motor output (horizontal dimension) is clearly visible after training. (D) Mapping from rotational angle to motor output. The blue data depicts a measurement of the behavior shown by a classical controller for BLDC motors. The orange curves represent spike rates measured on BSS-2 with simulated input after training. They have been normalized to 1/τref. (E) Rasterplot of the spiking motor controller during the closed-loop experiment. The motor neurons directly map to the six controllables of an H-bridge, while the sensory neurons represent the rotational angle of the motor. The transition phase for a single state shift is highlighted through magnification.
We trained this SNN-based motor controller using standard machine-learning techniques in PyTorch. To do so, we have recorded the behavior of an open-source reference BLDC motor controller (Skuric et al.,
Table 1
| Neuron model | Leaky integrate-and-fire |
| Membrane time constant τmem | 20 μs |
| Refractory time τrefr | 32 μs |
| Synaptic time constant τsyn | 20 μs |
| Network architecture | Single layer feed-forward SNN, 6 units |
| Sensory input coding | Value-unit, rate-based |
| Motoric output coding | Value-unit, pulse-density modulated |
| Training | Stochastic gradient decent |
| Loss function | L2 |
| Optimizer | Adam |
| Learning rate | 0.1 |
| β | 0.9, 0.999 |
| Weight decay | 0 |
| Weight initialization | 0.5 ± 0.1 |
Parameters for the presented SNN-based BLDC controller.
All time constants are given in wall-clock time, illustrating the accelerated nature of the analog substrate.
The full system in operation is depicted in Figure 2B, where the BSS-2 system (right side, with the ASIC hidden under a red dust cap) controls a small BLDC motor. The oscilloscope shows membrane recordings of the two motor neurons and , the areas of high activity are clearly separated from those with strong inhibition. Figure 2E shows a corresponding rasterplot with all involved signals. The linearly ascending sensory inputs indicate the continuous rotational motion of the motor, while the motor neurons show the expected repetitive six-phase commutation pattern.
4 Discussion
This work introduced a realtime spike interface for the accelerated neuromorphic BSS-2 platform, paving the way for high-speed robotic applications. Honoring the system's fast dynamics, it has been optimized for low latency, moderate bandwidth and high compatibility with embedded components for sensor inputs and motor outputs. Similar to Romero Bermudez et al. (
As an initial application, we have shown an SNN-based controller for BLDC motors and thereby join the rank of neuromorphic control units for motor tasks that reaches back multiple decades (DeWeerth et al.,
Statements
Data availability statement
The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.
Author contributions
YS: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Software, Validation, Visualization, Writing – original draft, Writing – review & editing. JS: Conceptualization, Funding acquisition, Methodology, Project administration, Resources, Supervision, Writing – original draft, Writing – review & editing.
Funding
The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. The presented work has received funding from the European Union's Horizon 2020 Research and Innovation Programme under grant agreement nos. 720270, 785907, and 945539 (Human Brain Project, HBP) and Horizon Europe grant agreement no. 101147319 (EBRAINS 2.0), and from the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany's Excellence Strategy EXC 2181/1-390900948 (Heidelberg STRUCTURES Excellence Cluster). For the publication fee the authors acknowledge financial support by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) within the funding programme “Open Access Publikationskosten” as well as by Heidelberg University.
Acknowledgments
The authors express their thanks and gratitude to Sebastian Billaudelle, Joscha Ilmberger, and Philipp Spilger for their consistent support throughout all phases of this study. We extend our gratitude to all members of the Electronic Vision(s) group and thank them for operating the BrainScaleS-2 system.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Footnotes
1.^Alex Forencich: Verilog UART, https://github.com/alexforencich/verilog-uart.
References
1
AcarnleyP. P.WatsonJ. F. (2006). Review of position-sensorless operation of brushless permanent-magnet machines. IEEE Trans. Industr. Electr.53, 352–362. 10.1109/TIE.2006.870868
2
BillaudelleS.StradmannY.SchreiberK.CramerB.BaumbachA.DoldD.et al. (2020). “Versatile emulation of spiking neural networks on an accelerated neuromorphic substrate,” in 2020 IEEE International Symposium on Circuits and Systems (ISCAS) (Seville: IEEE). 10.1109/iscas45731.2020.9180741
3
BillaudelleS.WeisJ.DauerP.SchemmelJ. (2022). “An accurate and flexible analog emulation of AdEx neuron dynamics in silicon,” in 2022 29th IEEE International Conference on Electronics, Circuits and Systems (ICECS) (Glasgow), 1–4. 10.1109/ICECS202256217.2022.9971058
4
BlumH.DietmüllerA.MildeM.ConradtJ.IndiveriG.SandamirskayaY. (2017). “A neuromorphic controller for a robotic vehicle equipped with a dynamic vision sensor,” in Robotics: Science and Systems XIII (Cambridge, MA: Science and Systems Foundation). 10.15607/rss.2017.xiii.035
5
BossoA.ConficoniC.RagginiD.TilliA. (2021). A computational-effective field-oriented control strategy for accurate and efficient electric propulsion of unmanned aerial vehicles. IEEE/ASME Trans. Mechatr.26, 1501–1511. 10.1109/TMECH.2020.3022379
6
BretteR.GerstnerW. (2005). Adaptive exponential integrate-and-fire model as an effective description of neuronal activity. J. Neurophysiol.94, 3637–3642. 10.1152/jn.00686.2005
7
DAngeloG.PerrettA.IaconoM.FurberS.BartolozziC. (2022). Event driven bio-inspired attentive system for the iCub humanoid robot on SpiNNaker. Neuromorph. Comput. Eng. 2:e024008. 10.1088/2634-4386/ac6b50
8
DavisonA. P.BrüderleD.EpplerJ.KremkowJ.MullerE.PecevskiD.et al. (2009). PyNN: a common interface for neuronal network simulators. Front. Neuroinform.2:2008. 10.3389/neuro.11.011.2008
9
DengE.MutluB.MatarićM. J. (2019). Embodiment in socially interactive robots. Found. Trends Robot.7, 251–356. 10.1561/2300000056
10
DeWeerthS. P.NielsenL.MeadC. A.ÅströmK. J. (1991). A simple neuron servo. IEEE Trans. Neural Netw.2, 248–251. 10.1109/72.80335
11
FaloticoE.VannucciL.AmbrosanoA.AlbaneseU.UlbrichS.Vasquez TieckJ. C.et al. (2017). Connecting artificial brains to robots in a comprehensive simulation framework: the neurorobotics platform. Front. Neurorobot.11:2. 10.3389/fnbot.2017.00002
12
HagrasH.Pounds-CornishA.ColleyM.CallaghanV.ClarkeG. (2004). “Evolving spiking neural network controllers for autonomous robots,” in Proceedings ICRA '04 (IEEE) (New Orleans, LA), 4620–4626. 10.1109/ROBOT.2004.1302446
13
KaiserJ.BillaudelleS.MüllerE.TetzlaffC.SchemmelJ.SchmittS. (2022). Emulating dendritic computing paradigms on analog neuromorphic hardware. Neuroscience489, 290–300. 10.1016/j.neuroscience.2021.08.013
14
KempB. J. (1973). Reaction time of young and elderly subjects in relation to perceptual deprivation and signal-on vs. signal-off conditions. Develop. Psychol.8, 268–272. 10.1037/h0034147
15
KingmaD. P.BaJ. (2014). “Adam: a method for stochastic optimization,” in International Conference on Learning Representations (San Diego, CA).
16
KiversteinJ.MillerM. (2015). The embodied brain: towards a radical embodied cognitive neuroscience. Front. Hum. Neurosci.9:237. 10.3389/fnhum.2015.00237
17
KoJ. S.LeeJ. H.ChungS. K.YounM. J. (1993). A robust digital position control of brushless DC motor with dead beat load torque observer. IEEE Trans. Industr. Electr.40, 512–520. 10.1109/41.238020
18
KreiserR.CartigliaM.MartelJ. N.ConradtJ.SandamirskayaY. (2018). “A neuromorphic approach to path integration: a head-direction spiking neural network with vision-driven reset,” in 2018 IEEE International Symposium on Circuits and Systems (ISCAS) (Florence: IEEE). 10.1109/iscas.2018.8351509
19
MildeM. B.DietmullerA.BlumH.IndiveriG.SandamirskayaY. (2017). “Obstacle avoidance and target acquisition in mobile robots equipped with neuromorphic sensory-processing systems,” in 2017 IEEE International Symposium on Circuits and Systems (ISCAS) (Baltimore, MD: IEEE), 1–4. 10.1109/iscas.2017.8050984
20
MüllerE.ArnoldE.BreitwieserO.CzierlinskiM.EmmelA.KaiserJ.et al. (2022). A scalable approach to modeling on accelerated neuromorphic hardware. Front. Neurosci.16:884128. 10.3389/fnins.2022.884128
21
PaszkeA.GrossS.MassaF.LererA.BradburyJ.ChananG.et al. (2019). “Pytorch: an imperative style, high-performance deep learning library,” in Advances in Neural Information Processing Systems 32, eds. H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (Red Hook, NY: Curran Associates, Inc.), 8024–8035.
22
PehleC.BillaudelleS.CramerB.KaiserJ.SchreiberK.StradmannY.et al. (2022). The BrainScaleS-2 accelerated neuromorphic system with hybrid plasticity. Front. Neurosci.16:795876. 10.3389/fnins.2022.795876
23
PillayP.KrishnanR. (1989). Modeling, simulation, and analysis of permanent-magnet motor drives, part II: the brushless DC motor drive. IEEE Trans. Industry Applicat.25, 274–279. 10.1109/28.25542
24
RichterC.JentzschS.HostettlerR.GarridoJ. A.RosE.KnollA.et al. (2016). Musculoskeletal robots: scalability in neural control. IEEE Robot. Automat. Mag.23, 128–137. 10.1109/MRA.2016.2535081
25
RodriguezF.EmadiA. (2007). A novel digital control technique for brushless DC motor drives. IEEE Trans. Industr. Electr.54, 2365–2373. 10.1109/TIE.2007.900312
26
Romero BermudezJ. P.PlanaL. A.RowleyA.HesselM.PedersenJ. E.FurberS.et al. (2023). “A high-throughput low-latency interface board for SpiNNaker-in-the-loop real-time systems,” in Proceedings of the 2023 International Conference on Neuromorphic Systems, ICONS '23 (New York, NY: Association for Computing Machinery), 1–8. 10.1145/3589737.3605969
27
SathyanA.MilivojevicN.LeeY.-J.KrishnamurthyM.EmadiA. (2009). An FPGA-based novel digital PWM control scheme for BLDC motor drives. IEEE Trans. Industr. Electr.56, 3040–3049. 10.1109/TIE.2009.2022067
28
SchmittS.KlähnJ.BellecG.GrüblA.GüttlerM.HartelA.et al. (2017). “Neuromorphic hardware in the loop: training a deep spiking network on the BrainScaleS wafer-scale system,” in Proceedings of the 2017 IEEE International Joint Conference on Neural Networks (IJCNN) (Anchorage, AK), 2227–2234. 10.1109/IJCNN.2017.7966125
29
SchreiberK. (2021). Accelerated Neuromorphic Cybernetics. (Ph.D. thesis). Universität Heidelberg, Heidelberg, Germany.
30
SkuricA.BankH. S.UngerR.WilliamsO.González-ReyesD. (2022). SimpleFOC: a field oriented control (FOC) library for controlling brushless direct current (BLDC) and stepper motors. J. Open Sourc. Softw.7:4232. 10.21105/joss.04232
31
YanY.StewartT. C.ChooX.VoggingerB.PartzschJ.HöppnerS.et al. (2021). Comparing Loihi with a SpiNNaker 2 prototype on low-latency keyword spotting and adaptive robotic control. Neuromorph. Comput. Eng. 1:e014002. 10.1088/2634-4386/abf150
32
YaoP.WuH.GaoB.TangJ.ZhangQ.ZhangW.et al. (2020). Fully hardware-implemented memristor convolutional neural network. Nature577, 641–646. 10.1038/s41586-020-1942-4
Summary
Keywords
analog neuromorphic computing, spiking neural networks, neurorobotics, closed-loop controller, high-speed robotics, motor control
Citation
Stradmann Y and Schemmel J (2024) Closing the loop: High-speed robotics with accelerated neuromorphic hardware. Front. Neurosci. 18:1360122. doi: 10.3389/fnins.2024.1360122
Received
22 December 2023
Accepted
05 March 2024
Published
26 March 2024
Volume
18 - 2024
Edited by
Alex James, Digital University Kerala, India
Reviewed by
Kamilya Smagulova, King Abdullah University of Science and Technology, Saudi Arabia
Chithra Reghuvaran, University College Dublin, Ireland
Updates

Check for updates
Copyright
© 2024 Stradmann and Schemmel.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Yannik Stradmann yannik.stradmann@kip.uni-heidelberg.de
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.