Diversity of efficient coding solutions for a population of noisy linear neurons
-
1
New York University , United States
-
2
Columbia University, United States
Efficient coding is a well-known principle for explaining early sensory transformations (Barlow, 1961). But even in the "classical" case of a linear neural population with Gaussian input and output noise, the optimal solution depends heavily on the choice of constraints that are imposed on the problem. These can include constraints on output capacity (which is necessary to prevent the solution from diverging) and number of neurons in the population. With the exception of (Campa et al. 1995), previous literature assumes the number of neurons is equal to the input dimensionality, and furthermore, that the receptive fields are identical (i.e., the population performs a convolution). In addition, previously published examples are based on a single capacity constraint, such as the variance (loosely analogous to average spike rate) of the outputs (Atick & Redlich, 1990, Atick, Li, & Redlich, 1990, van Hateren, 1992), or the norm (analogous to the sum of squared synaptic weights) of filters (Campa et al., 1995). Each of these potential constraints has some approximate mapping onto biologically relevant (i.e., metabolic) costs, implying that a complete formulation should include a generalized cost function that combines power, weights, and population size. Toward this end, we examine a more general formulation of the efficient coding problem. We assume a discrete and finite input signal that is Gaussian with known covariance structure (as in natural signals), corrupted by additive white Gaussian noise. We assume a neural population of arbitrary size with linear receptive fields (RFs), each of whose outputs is corrupted by additive white Gaussian noise. And finally, we assume a cost function that is a linear combination of the number of cells, the L2 norm of the RF weights, and the output power. Given these constraints, we solve for a population of RFs that maximizes the information transmitted about the stimulus. The problem is convex, and thus can be solved with standard optimization methods. We note several important attributes of this formulation. First, it can achieve both over- and under-complete solutions, as is required to explain biological systems. In the retina, for example, previous efforts have assumed a convolutional solution (homogenous population of RFs, one per input cone) (Atick & Redlich, 1990, Atick, Li, & Redlich, 1990, van Hateren, 1992), but the ratio of cones to ganglion cells varies dramatically with eccentricity. Second, the joint cost function allows the theory to automatically select the optimal population size. And finally, even in the case of a convolutional population, the spectral properties of the optimal solution depend critically on the choice of cost function, and can assume shapes ranging from low-pass, to band-pass, to high-pass. For example, although previous literature (Atick & Redlich 1990) obtained low-pass solutions in the case when the input SNR was low (i.e., low contrast stimuli), the inclusion of the penalty for RF weights allows the possibility of low-pass (and highly redundant) solutions even when the input SNR is high. We conclude that identifying the relative significance of different costs in biological systems is critical to testing the efficient coding hypothesis.
Conference:
Computational and Systems Neuroscience 2010, Salt Lake City, UT, United States, 25 Feb - 2 Mar, 2010.
Presentation Type:
Poster Presentation
Topic:
Poster session I
Citation:
Doi
E,
Paninski
L and
Simoncelli
EP
(2010). Diversity of efficient coding solutions for a population of noisy linear neurons.
Front. Neurosci.
Conference Abstract:
Computational and Systems Neuroscience 2010.
doi: 10.3389/conf.fnins.2010.03.00109
Copyright:
The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers.
They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.
The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.
Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.
For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.
Received:
21 Feb 2010;
Published Online:
21 Feb 2010.
*
Correspondence:
Eero P Simoncelli, New York University, New York, United States, eero.simoncelli@nyu.edu