Neuroscience Gateway – Cyberinfrastructure Providing Supercomputing Resources for Large Scale Computational Neuroscience Research
-
1
University of California San Diego, San Diego Supercomputer Center, United States
-
2
Yale School of Medicine, Department of Neurobiology, United States
-
3
University College London, Department of Neuroscience, Physiology and Pharmacology, United Kingdom
Past few decades have seen tremendous growth in computational neuroscience. This is reflected in the scientific literature with the emergence of new journals devoted to theoretical and computational neuroscience (e.g. Neural Computation, Journal of Computational Neuroscience) as well as from the publication of modeling papers in the general neuroscience research literature. Similarly prominence of computational modeling in neuroscience is evident from proposals submitted to, and funded by, NSF, NIH and other organizations in the US and in various other countries in Europe, Asia and elsewhere. Proposals submitted by experimentalists contain significant computational research component. During the same time powerful simulation environments such as NEURON [1], NEST [2], Brian [3], PyNN [4], MOOSE [5], GENESIS [6] have evolved along with model sharing and informatics tools such as ModelDB [7], Open Source Brain (OSB) [8], OpenWorm [9], Neuroscience Information Framework (NIF) [10], and NeuroML [11]. The simulation environments have been developed to allow simulation of large parallel neuronal models on high performance computing (HPC) resources. Imaging community is processing larger amount of data coming out of MRI, fMRI, EEG imaging techniques and are in need of HPC for processing. As is well known the past decade or so has seen tremendous growth in cyberinfrastructure (CI), including HPC, data resources, fast networks, software and workflows.
All of these hold the promise for the broader computational neuroscience community to utilize HPC for research which involves complex models (especially large scale networks) or complex protocols (often involving learning rules), or require high-dimensional optimization or parameter space exploration or processing of large amount of imaging data. Such projects have a tremendous potential to use CI, but the broader computational neuroscience community do not have easy access to HPC and in many cases are forced to keep their research “small” such that it can be accommodated within local desktop or departmental computational resources.
Although HPC resources are available at national supercomputer centers in many countries, neuroscience researchers’ access to HPC resources is inhibited by common administrative and technical barriers which include: (i) a process for requesting time on HPC resources every year and that is daunting to most beginners, (ii) difficulty understanding the HPC architecture and complex OS/software environment to optimally install applications on various HPC resources, (iii) having to learn policies and batch system details which are different on each HPC cluster, (iv) challenge of managing workflow that involves multiple remote authentication systems, and (v) figuring out data transfer, output result retrieval and storage issues.
The solution to this problem is building a community infrastructure layer, called a science gateway [12], that abstracts away the need for dealing with the details of underlying hardware and allows neuroscience researchers easy access to best-of-breed neuroscience simulation and analysis packages running on very large HPC resources.
In 2012 the US NSF funded the Neuroscience Gateway (NSG) project [13, 14] which enables individual computational neuroscientists to utilize free supercomputer time on NSF’s Extreme Science and Engineering Discovery Environment (XSEDE) [15] HPC resources. All of the administrative and technical barriers, that otherwise make it difficult for neuroscientists to use HPC resources for computational modeling, are managed by the NSG team. The NSG has been in operation since early 2013 and has over 300 users. As far as we know this is the first and only such open science gateway available to the neuroscience community within US and abroad and which is providing free access to HPC resources for anyone from anywhere in the world. About 60% of the NSG users are from US and the rest are from Europe, Asia and elsewhere.
Since NSG went into production in 2013, NSG users used about 187,000 supercomputer core hour units (SUs) in 2013, about 600,000 SUs in 2014, about 1,844,000 SUs in 2015 and for the current 2016 year we have acquired about 3,700,000 SUs. SUs are acquired every year by submitting an allocation proposal (written by the NSG team on behalf of the NSG user community) to the national XSEDE supercomputer review committee and the proposals are peer reviewed. We have been very successful in acquiring SUs although it is a very competitive peer reviewed process. Number of NSG users have grown steadily – in 2013 we had about 100 users, in 2014 about 200 users, in 2015 about 270 users and by April, 2016 we have about 325 users. Over the last 3.5 years, the average NSG job uses about 200 - 300 cores and larger jobs have used 4096 cores. Close to 10,000 jobs have been run during the past 3.5 years by NSG users.
In 2015 NSF and the BBSRC in UK jointly awarded a follow-on international collaborative grant as a part of which RESTful services will be implemented for the NSG. This will allow researchers to programmatically access HPC resources, for complex computational modeling, both from the familiar environment of neuroscience community projects (such as the OSB, ModelDB, OpenWorm, NIF) and from the familiar work environment as it exists within the laptop or desktop of individual users. Integration with community projects will significantly increase the impact of NSG for the neuroscience community.
All of these demonstrate that the NSG is filling a vacuum that existed for computational neuroscience researchers eager to engage in large scale neuronal simulations and processing of imaging data. This is underscored by the enthusiastic feedback we have received from NSG users. This is also very timely in this context the two large scale, decade long projects that were initiated in 2013 to revolutionize the understanding of the human brain - the BRAIN Initiative [16] in the US and the Human Brain Project [17] in the EU. Developing brain models, running large scale neuronal simulations on supercomputers, analyzing large data sets, and creating new simulation and neuroinformatics tools are central components of both of those initiatives. NSG is already well poised to be a key resource for the new, computationally intensive research that these projects will spawn.
Acknowledgements
The research was supported by the US NSF awards DBI #1458840, DBI #1458495, DBI #1146949, DBI #1146830, ACI #1339856 to University of California San Diego and Yale University researchers, and UK BBSRC award #BB/N005236/1 to University College London researchers.
References
[1] Carnevale, N. T., and Hines, L. M. 1993. The NEURON Book. Cambridge, UK: Cambridge University Press, 2006.
[2] 2. Gewaltig M-O and Diesmann, M.NEST (Neural Simulation Tool) Scholarpedia, 2(4): 1430, 2007.
[3] Stimberg M., Goodman D.F.M, Benichoux, V., and Brette, R.Equation-oriented specification of neural models for simulations, Frontiers Neuroinf, 2014.
[4] Daviaon, A.P., Bruderle, D., Eppler, J.M., Kremkow, J., Muller, E., Pecevski D.A., Perrinet L., and Yger, P. PyNN: a common interface for neuronal network simulators. Front. Neuroinform.2:11, 2008
[5] http://moose.sourceforge.net/.
[6] Bower, J. M, and Beeman, D. The Book of GENESIS: Exploring Realistic Neural Models with the General Neural Simulation System. Second Edition, Springer-Verlag, New York (1998).
[7] Hines ML, Morse T, Migliore M, Carnevale NT, Shepherd GM. ModelDB: A Database to Support Computational Neuroscience. Journal of Computational Neuroscience 2004 Jul-Aug;17(1):7-11.
[8] http://www.opensourcebrain.org
[9] http://www.openworm.org/
[10] http://neuinfo.org/
[11] https://www.neuroml.org/
[12] N. Wilkins‐Diehr, D. Gannon, G. Klimeck, S. Oster, S. Pamidighantam,"TeraGrid Science Gateways and Their Impact on Science", IEEE Computer, Volume 41, Number 11 (November, 2008), pages 32‐41.
[13] www.nsgportal.org
[14] S Sivagnanam, A Majumdar, K Yoshimoto, V Astakhov, A Bandrowski, M. E. Martone, and N. T. Carnevale. Introducing the Neuroscience Gateway, IWSG, volume 993 of CEUR Workshop Proceedings, CEUR-WS.org, 2013.
[15] www.xsede.org
[16] http://www.nih.gov/science/brain/
[17] https://www.humanbrainproject.eu/
Keywords:
neuroscience gateway,
Large scale modeling,
Supercomputing,
parallel brain imaging data processing,
cyberinfrastructure
Conference:
Neuroinformatics 2016, Reading, United Kingdom, 3 Sep - 4 Sep, 2016.
Presentation Type:
Investigator presentations
Topic:
Infrastructural and portal services
Citation:
Majumdar
A,
Sivagnanam
S,
Carnevale
NT,
Yoshimoto
K,
Gleeson
P,
Quintana
A and
Silver
RA
(2016). Neuroscience Gateway – Cyberinfrastructure Providing Supercomputing Resources for Large Scale Computational Neuroscience Research.
Front. Neuroinform.
Conference Abstract:
Neuroinformatics 2016.
doi: 10.3389/conf.fninf.2016.20.00008
Copyright:
The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers.
They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.
The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.
Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.
For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.
Received:
30 Apr 2016;
Published Online:
18 Jul 2016.
*
Correspondence:
Dr. Amitava Majumdar, University of California San Diego, San Diego Supercomputer Center, La Jolla, CA, 92093-0505, United States, majumdar@sdsc.edu
Ms. Subhashini Sivagnanam, University of California San Diego, San Diego Supercomputer Center, La Jolla, CA, 92093-0505, United States, sivagnan@sdsc.edu