Binary Hidden Variables and Sparse Sensory Coding
-
1
Goethe-Universität Frankfurt am Main, FIAS, Germany
-
2
Honda Research Institute Europe, Honda Research, Germany
Algorithms such as sparse coding and its variants represent one of the most popular approaches to explain simple-cell responses in the mammalian primary visual cortex. Here, we study a sparse coding version that replaces the most commonly used continuous prior with a Bernoulli prior, thus introducing binary hidden variables. We derive a learning algorithm using a novel approximation scheme based on variational EM. In addition to learning the basis functions this training scheme also allows us to learn the noise level and the degree of sparseness, i.e., the average number of hidden causes contributing to each data point. In numerical experiments on artificial visual data, we demonstrate that the algorithm reliably extracts precise estimates of the true underlying basis functions and the true values for noise level and data sparseness. In large-scale applications to acoustic data consisting of one sentence spoken by different speakers (TIMIT) the extracted hidden causes take on the form of elementary waveforms (see Figure 1A). Applying the algorithm to preprocessed natural image data leads to the extraction of Gabor-like generative fields along with a sparseness estimate. For a large number of hidden variables these generative fields take on properties of simple cell receptive fields that classical sparse coding approaches do not reproduce (see Figure 1B). Our results show that typical properties of sensory cells are reproduced assuming binary hidden variables, which has potential implication for the study of neural encoding.
Figure 1: A 72 of a total of 200 learned basis functions extracted in applications to acoustic data (each of which corresponding to a 12.5ms long waveform). B 81 of 700 basis functions of size 26x26 pixels after applying the learning algorithm to natural image data. The globular shape of individual generative fields is a property that is not reported by classical sparse coding approaches.
Keywords:
computational neuroscience
Conference:
Bernstein Conference on Computational Neuroscience, Berlin, Germany, 27 Sep - 1 Oct, 2010.
Presentation Type:
Presentation
Topic:
Bernstein Conference on Computational Neuroscience
Citation:
Bornschein
J,
Henniges
M,
Puertas
G,
Eggert
J and
Lücke
J
(2010). Binary Hidden Variables and Sparse Sensory Coding.
Front. Comput. Neurosci.
Conference Abstract:
Bernstein Conference on Computational Neuroscience.
doi: 10.3389/conf.fncom.2010.51.00114
Copyright:
The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers.
They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.
The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.
Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.
For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.
Received:
06 Sep 2010;
Published Online:
23 Sep 2010.
*
Correspondence:
Dr. Marc Henniges, Goethe-Universität Frankfurt am Main, FIAS, Frankfurt am Main, Germany, henniges@fias.uni-frankfurt.de