Original Research ARTICLE
Neuromodulated Synaptic Plasticity on the SpiNNaker Neuromorphic System
- 1School of Computer Science, University of Manchester, United Kingdom
- 2School of Engineering and Informatics, University of Sussex, United Kingdom
SpiNNaker is a digital neuromorphic architecture, designed specifically for the low power simulation of large-scale spiking neural networks at speeds close to biological real-time. Unlike other neuromorphic systems, SpiNNaker allows users to develop their own neuron and synapse models as well as specify arbitrary connectivity. As a result SpiNNaker has proved to be a powerful tool for studying different neuron models as well as synaptic plasticity -- believed to be one of the main mechanisms behind learning and memory in the brain. A number of Spike-Timing-Dependent-Plasticity(STDP) rules have already been implemented on SpiNNaker and have been shown to be capable of solving various learning tasks in real-time. However, while STDP is an important biological theory of learning, it is a form of Hebbian or unsupervised learning and therefore does not explain behaviours that depend on feedback from the environement. Instead, learning rules based on neuromodulated STDP (three-factor learning rules) have been shown to be capable of solving reinforcement learning tasks in a biologically plausible manner. In this paper we demonstrate for the first time how a model of three-factor STDP, with the third-factor representing spikes from dopaminergic neurons, can be implemented on the SpiNNaker neuromorphic system. Using this learning rule we first show how reward and punishment signals can be delivered to a single synapse before going on to demonstrate it in a larger network which solves the credit assignment problem in a Pavlovian conditioning experiment. Because of its extra complexity, we find that our three-factor learning rule requires approximately 2x as much processing time as the existing SpiNNaker STDP learning rules. However, we show that it is still possible to run our Pavlovian conditioning model with up to 10000 neurons in real-time, opening up new research opportunities for modelling behavioural learning on SpiNNaker.
Keywords: Neuromodulation, STDP, SpiNNaker, three-factor learning rules, reinforcement learning, behavioural learning
Received: 01 Nov 2017;
Accepted: 12 Feb 2018.
Edited by:Emre O. Neftci, University of California, Irvine, United States
Reviewed by:Doo Seok Jeong, Korea Institute of Science and Technology (KIST), South Korea
Elisabetta Chicca, Bielefeld University, Germany
Copyright: © 2018 Mikaitis, Pineda García, Knight and Furber. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
* Correspondence: Mr. Mantas Mikaitis, University of Manchester, School of Computer Science, Manchester, United Kingdom, firstname.lastname@example.org