A testsuite for a neural simulation engine
-
1
Honda Research Institute Europe GmbH, Germany
-
2
Norwegian University of Life Sciences, Norway
-
3
RIKEN Brain Science Institute, Japan
Testing is a standard activity of the software development process [1,2]. However, little research has been carried out on the specific problems of testing neuronal simulation engines. We found that with the growing complexity of the software and the growing number of developers, formalized and systematic testing becomes critical. This is because the changes made to the code for a new feature are more likely to break an existing function and no researcher has a full overview of the code. The rapid growth of neuroscientific knowledge and the changing research directions require an incremental/iterative development style [3] which extends over the full life time of the product. Thus, there is no single testing phase but the same tests need to be carried out repetitively over a time span of many years. Furthermore, there is usually no rigorous way to determine whether the result of a simulation is correct. The confidence of the scientist relies on comparisons of the results of different implementations and the correctness of critical components of the system.
Here we present the architecture of the testing framework we have developed for NEST [4]. The fully automatized testsuite consists of a collection of test scripts which are sequentially executed by a concise shell script. While high expressiveness of the test code is a goal, sometimes more primitive code is used in order not to condition a test on the correctness of a further component. NEST's built-in simulation language (SLI) is the lowest level at which tests are formulated. The C++ classes of the simulation kernel have no code for testing but are equipped with checks of invariants (assertions).
In designing the tests we found the following principles effective: (1) test the ability to report errors (self-test), (2) hierarchically organize tests from simple to complex, (3) test that objects have the expected default values and accept parameter changes, (4) compare simulation results with analytical results for simple scenarios, (5) check correctness of results with simpler algorithms, (6) test the convergence of results with decreasing simulation time step, (7) check for expected accuracy, (8) test the invariance of results with increasing numbers of processors used, (9) create regression tests for fixed problems.
It is essential to combine these principles. For example, a simulation may converge with decreasing time step, but to the wrong result. Convergence to the correct result still does not guarantee correctness of the implementation: if a signal delay is offset by one time step, the simulation may converge, but with unexpectedly large errors at a given resolution.
Acknowledgements: Next-Generation Supercomputer Project of MEXT, EU Grant 15879 (FACETS), BMBF Grant 01GQ0420, Helmholtz Alliance on Systems Biology
References
1. Sommerville I (2007) Software Engineering (8th edn), Harlow, Addison-Wesley
2. Beck K & Andres C (2004) Extreme Programming Explained (2nd edn), Boston, Addison-Wesley
3. Diesmann M and Gewaltig M-O (2002) GWDG Bericht, 58:43-70
4. Gewaltig M-O & Diesmann M (2007) Scholarpedia, 2(4):1430
Conference:
Neuroinformatics 2009, Pilsen, Czechia, 6 Sep - 8 Sep, 2009.
Presentation Type:
Poster Presentation
Topic:
Large scale modeling
Citation:
Eppler
JM,
Kupper
R,
Plesser
HE and
Diesmann
M
(2019). A testsuite for a neural simulation engine.
Front. Neuroinform.
Conference Abstract:
Neuroinformatics 2009.
doi: 10.3389/conf.neuro.11.2009.08.042
Copyright:
The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers.
They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.
The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.
Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.
For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.
Received:
22 May 2009;
Published Online:
09 May 2019.
*
Correspondence:
Jochen M Eppler, Honda Research Institute Europe GmbH, Offenbach, Germany, frontiers@mindzoo.de