AUTHOR=Muller Lorenz K. , Stark Pascal , Offrein Bert Jan , Abel Stefan 

TITLE=Neuromorphic Systems Design by Matching Inductive Biases to Hardware Constraints

JOURNAL=Frontiers in Neuroscience

VOLUME=Volume 14 - 2020

YEAR=2020

URL=https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2020.00437

DOI=10.3389/fnins.2020.00437

ISSN=1662-453X

ABSTRACT=Neuromorphic systems are designed with careful consideration of the physical properties of the computational substrate they use. Specifically neuromorphic engineers often exploit physical phenomena to directly implement a desired functionality, enabled by ``the isomorphism between physical processes in different media'' (Douglas, 1995). This bottom-up design methodology could be described as matching computational primitives to physical phenomena. In this paper we propose a top-down counterpart to the bottom-up approach to neuromorphic design. Our top-down approach, termed `bias matching', is to match the inductive biases required in a learning system to the hardware constraints of its implementation; a well-known example is enforcing translation equivariance by tying weights (replacing vector-matrix multiplications with convolutions). We give numerous examples from the literature and explain how they can be understood from this perspective. Furthermore we propose novel network designs based on this approach in the context of collaborative filtering. Our simulation results underline our central conclusions: Additional hardware constraints can improve the predictions of a Machine Learning system and understanding the inductive biases that mediate these performance gains can be useful in finding applications for a given constraint.