Event Abstract

Back to Event

Easy-to-use GPU acceleration of neural network simulations with GeNN

James P. Turner1, Esin Yavuz1 and Thomas Nowotny1*
  • 1 University of Sussex, School of Engineering and Informatics, United Kingdom

General purpose computing with graphics processing units (GPUs) has come a long way since the first pioneers used graphics oriented shader languages to implement their first proof of concept studies. However, even though the CUDA [1] and OpenCL [2] application programming interfaces (APIs) today largely facilitate the use of GPUs for general purpose computing and abstract away much of the complexities of the massively parallel GPU architectures, writing efficient code for GPU accelerators remains a challenge. Many different aspects need to be considered, including the overall strategy of how to parallelize a given computational problem, how and what to communicate between GPU and host and how to use the different memory spaces available to the programmer. This is acerbated by a rapid stream of novel features that improve GPU operation but explode the number of choices, programmers need to make. With the GPU enhanced neuronal networks (GeNN) framework [3, 4, 5] we have provided a middleware that uses heuristics specific to the application domain of spiking neuronal networks to facilitate the work of the scientific programmer. More specficially, in GeNN users define a neuronal network model by providing the following ingredients in a simple C++ API: - Descriptions of neurons models, including code strings for updating the neuron state and detecting spikes; - Descriptions of synapse models including update rules for pre-synaptic spikes, pre-synaptic spike “events”, definitions for these spike “events”, and potentially learning rules - Post-synaptic models that describe how pre-synaptic activation is translated into a post-synaptic current - A definition of neuron populations, based on any of the described neuron types - A definition of synapse populations, complete with the source and target populations of neurons, synaptic model and post-synaptic model Based on this description, GeNN then creates optimised code to run the model on any NVIDIA CUDA enabled hardware accelerators it detects. If several accelerators are present, GeNN will detect the most appropriate one and use it. When generating the code, GeNN will take care of both, the more strategic decisions, such as deciding which device to use, how to choose block sizes, and how to efficiently communicate spike and spike events, and the details of “bean counting” where the sizes of neuron populations don’t match the best block size, and indices need to be tracked for more efficient sparse matrix methods. This frees the computational scientist to focus on the scientific investigation. GeNN is, however, designed for maximal flexibility and transparency for expert users. Using a code generation approach and only providing the critical code for GPU-accelerator and data management related functions, the user is in full control of the remainder of the code. Therefore, GeNN can easily be used in many different scenarios, ranging from large scale simulations on GPU accelerated High Perfomance Computers to embedded controllers used in robotics. For users who are not keen on full programmatic control of the user-side code we have devised higher-level interfaces in the form of Spinecreator/SpineML [6,7] integration and the brian2genn interface [7,8]. In the former, users can use the SpineCreator graphical interface to design models and can then execute them on a GPU accelerator through an interface to GeNN. In the latter, users can define their models fully in the Brian 2 simulator and can then choose the “genn” device to run the model on a GPU accelerator through GeNN. GeNN can for some models achieve speed-ups of more than 100 fold over the speed of a single CPU core [4], but for other models, simple CPU simulation may be faster. When used through brian2genn, the benchmarks are similar, with a wide spread of performances depending on the model. While the devil is in the detail, generally models with sparse activity, sparse, but ordered, connections and a heavy computational load for the neuron dynamics, e.g. in Hodgkin-Huxley models, are fastest. Models with dense random connections and many spikes typically benefit less from the GPU acceleration with GeNN. The GeNN team is now developing methods for distributing computations on several GPUs simultaneously and the support of OpenCL is being initiated in the framework of a Google Summer of Code project. Figure 1: Illustration of the GeNN workflow.

Figure 1
Image

Acknowledgements

This work was partially supported by the EPSRC (Green Brain Project, grant number EP/J019690/1) and a senior research fellowship of the Royal Academy of Engineering and The Leverhulme Trust.

References

1. CUDA [http://www.nvidia.com/object/cuda_home_new.html], accessed 2015-02-25.

2. OpenCL [https://www.khronos.org/opencl/], accessed 2016-04-29.

3. Nowotny, T. (2011). Flexible neuronal network simulation framework using code generation for NVidia® CUDA™. BMC Neuroscience 12(Suppl 1), P239.

4. Yavuz E., Turner J. P., Nowotny, T. (2014). Simulating spiking neural networks on massively parallel graphical processing units using a code generation approach with GeNN. BMC Neuroscience 15(Suppl 1), O1.

5. Yavuz, E., Turner, J. P., Nowotny, T. (2016). GeNN: a code generation framework for accelerated brain simulations. Scientific Reports 6, 18854.

6. Richmond, P., Cope, A., Gurney, K., Allerton, D. J. (2013) From Model Specification to Simulation of Biologically Constrained Networks of Spiking Neurons. Neuroinformatics 12(2), 307-323.

7. Nowotny, T., Cope, A. J., Yavuz, E., Stimberg, M., Goodman, D. F. M., Marshall, J., Gurney, K. (2014) SpineML and Brian 2.0 interfaces for using GPU enhanced Neuronal Networks (GeNN). BMC Neuroscience 15(Suppl 1), P148.

8. Brian2genn [https://github.com/brian-team/brian2genn/], accessed 2016-04-29.

Keywords: GPU computing, spiking neural networks, large scale simulation, computational neuroscience, Numerical modeling, numerical simulation, Hardware Accelerator, Code generation

Conference: Neuroinformatics 2016, Reading, United Kingdom, 3 Sep - 4 Sep, 2016.

Presentation Type: Investigator presentations

Topic: Computing systems

Citation: Turner JP, Yavuz E and Nowotny T (2016). Easy-to-use GPU acceleration of neural network simulations with GeNN. Front. Neuroinform. Conference Abstract: Neuroinformatics 2016. doi: 10.3389/conf.fninf.2016.20.00005

Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.

The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.

Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.

For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.

Received: 29 Apr 2016; Published Online: 18 Jul 2016.

* Correspondence: Prof. Thomas Nowotny, University of Sussex, School of Engineering and Informatics, Brighton, East Sussex, BN1 9QJ, United Kingdom, t.nowotny@sussex.ac.uk