Impact Factor 3.074
2017 JCR, Clarivate Analytics 2018

The world's most-cited Neurosciences journals

Original Research ARTICLE Provisionally accepted The full-text will be published soon. Notify me

Front. Neuroinform. | doi: 10.3389/fninf.2019.00019

Communication sparsity in distributed Spiking Neural Network Simulations to improve scalability

  • 1Department of Automatic Control and Systems Engineering, University of Sheffield, United Kingdom
  • 2University of Sheffield, United Kingdom
  • 3Department of Computer Science, University of Sheffield, United Kingdom

In the last decade there has been a surge in the number of big science projects interested in achieving a holistic understanding of the functions of the brain, using Spiking Neuronal Network (SNN) simulations to aid discovery and experimentation. Such an approach increases the computational demands on SNN simulators: if natural scale brain-size simulations are to be realised, it is necessary to use parallel and distributed models of computing. Communication is recognised as the dominant part of distributed SNN simulations. As the number of computational nodes raises, the proportion of time the simulation spends in useful computing (computational efficiency) is reduced and therefore applies a limit to scalability. This work targets the three phases of communication to improve overall computational efficiency in distributed simulations: implicit synchronisation, process handshake and data exchange. We introduce a connectivity-aware allocation of neurons to compute nodes by modelling the SNN as a \textit{hypergraph}. Partitioning the hypergraph to reduce interprocess communication increases the sparsity of the communication graph. We propose dynamic sparse exchange as an improvement over simple point-to-point exchange on sparse communications. Results show a combined gain when using hypergraph-based allocation and dynamic sparse communication, increasing computational efficiency by up to \replaced[id=CFM]{40.8 percentage points}{20\%} and reducing simulation time by up to \replaced[id=CFM]{73\%}{44\%}. The findings are applicable to other distributed complex system simulations in which communication is modelled as a graph network.

Keywords: Spiking neural network (SNN), distributed simulation, Hypergraph partitioning, Dynamic sparse data exchange, point-To-point (P2P) communication, Complex system simulation, Sparse communication

Received: 06 Jul 2018; Accepted: 11 Mar 2019.

Edited by:

Eilif B. Muller, École Polytechnique Fédérale de Lausanne, Switzerland

Reviewed by:

Hans Ekkehard Plesser, Norwegian University of Life Sciences, Norway
Ivan Raikov, Stanford University, United States  

Copyright: © 2019 Fernandez Musoles, Coca and Richmond. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

* Correspondence: Mr. Carlos Fernandez Musoles, Department of Automatic Control and Systems Engineering, University of Sheffield, Sheffield, S1 3JD, United Kingdom, c.f.musoles@sheffield.ac.uk