A Pythonic Workflow for Automated Large-Scale Parameter Scans
-
1
Norwegian University of Life Sciences, Department of Mathematical Sciences and Technology, Norway
The systematic exploration of the properties of neuronal systems requires parameter scans along several dimensions. The resulting combinatorial explosion in the number of possible parameter combinations entails that hundreds of thousands of parameter sets need to be tested even for a handful of dimenions, with a number of randomized trials for each. While such large-scale scans will remain infeasible for brain-scale models for quite some time, due to overall runtime limitations, they are becoming routine for smaller systems (Nordlie et al., 2010, Heiberg et al., 2013), requiring suitable workflows and software tools for managing automated large-scale parameter scans.
Recent software developments, such as Sumatra (Davison, 2012) and the NeuroTools Parameters package (Muller et al., 2009) have been important steps towards managing research projects involving a large number of simulations. We present here a further step in this direction: a pythonic workflow that allows us to
• generate, aggregate and analyze data from hundreds of thousands of parameter sets and randomized trials;
• progress from coarse-scale to fine-scale scans, continuously monitoring progress and adapting scan resolution;
• avoid re-running any parameter set that has been tested before;
• drop scans along “singular” dimensions (e.g., drop scans along the modulation frequency for those sets with zero modulation amplitude);
• control parameter scans running on large, remote clusters from a personal computer including automated job preparation, submission, and monitoring, and data collection using the Fabric Python library (Hansen and Forcier, 2013);
• and to utilize large clusters with queuing systems efficiently for projects requiring a very large number of very small jobs using distributed shell (dsh) and IPython parallel.
We will discuss our experiences from a real-world project (Heiberg et al., 2013).
Acknowledgements
Partially funded by the Research Council of Norway (Grant 178892/V30 eNeuro) and EU Grant 269921 (BrainScaleS). Simulations were performed using NOTUR resources.
References
Davison AP (2012) Automated capture of experiment context for easier reproducibility in computational research. Computing in Science and Engineering, 14:48–56. doi: 10.1109/MCSE.2012.41.
Hansen CV and Forcier JE (2013). Fabric: Fabric 1.6 Documentation. URL http://fabfile.org.
Heiberg T, Kriener B, Tetzlaff T, Casti A, Einevoll GT and Plesser HE (2013) Firing-rate models
can describe the dynamics of the retina-LGN connection. Submitted.
Muller E, Davison AP, Brizzi T, Bruederle D, Eppler MJ, Kremkow J, Pecevski D, Perrinet L, Schmuker M and Yger P (2009). NeuralEnsemble.Org: Unifying neural simulators in Python to ease the model complexity bottleneck. Front. Neur. Conference Abstract: Neuroinformatics 2009. doi: 10.3389/conf.neuro.11.2009.08.104.
Nordlie E, Tetzlaff T and Einevoll GT (2010) Rate dynamics of leaky integrate-and-fire neurons with strong synapses. Front Comput Neurosci, 4:149. doi: 10.3389/fncom.2010.00149.
Keywords:
Parameter scan,
Simulation Technology,
Modeling and Simulation,
python language,
Parallel Computing
Conference:
Neuroinformatics 2013, Stockholm, Sweden, 27 Aug - 29 Aug, 2013.
Presentation Type:
Poster
Topic:
General neuroinformatics
Citation:
Heiberg
T and
Plesser
H
(2013). A Pythonic Workflow for Automated Large-Scale Parameter Scans.
Front. Neuroinform.
Conference Abstract:
Neuroinformatics 2013.
doi: 10.3389/conf.fninf.2013.09.00027
Copyright:
The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers.
They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.
The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.
Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.
For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.
Received:
25 Apr 2013;
Published Online:
11 Jul 2013.
*
Correspondence:
Dr. Hans Ekkehard Plesser, Norwegian University of Life Sciences, Department of Mathematical Sciences and Technology, Aas, 1432, Norway, hans.ekkehard.plesser@nmbu.no