Interleaving cortex-analog mixing improves deep non-negative matrix factorization networks

Mahbod Nouri^1*David Rotermund¹Alberto Garcia-Ortiz²Klaus R. Pawelzik¹

• ¹Universitat Bremen Institut fur Theoretische Physik, Bremen, Germany
• ²University of Bremen Institute of Electrodynamics and Microelectronics (ITEM), Bremen, Germany

Considering biological constraints in artificial neural networks has led to dramatic improvements in performance. Nevertheless, to date, the positivity of long-range signals in the cortex has not been shown to yield improve-ments. While Non-negative matrix factorization (NMF) captures biological constraints of positive long-range interactions, deep convolutional neural networks with NMF modules do not match the performance of conventional neural networks (CNNs) of a similar size. This work shows that introduc-ing intermediate modules that combine the NMF's positive activities, analogous to the processing in cortical columns, leads to improved performance on benchmark data that exceeds that of vanilla deep convolutional networks. This demonstrates that including positive long-range signalling together with local interactions of both signs in analogy to cortical hyper-columns has the potential to enhance the performance of deep networks.