Image classification with complex cell neural networks
-
1
University of Montreal, Canada
Simulations of cortical computation have often focused on networks built from simplified neuron models similar to rate models hypothesized for V1 simple cells. However, physiological research has revealed that V1 cells span a spectrum of behaviour encompassing the roles predicted by traditional simple and complex cell models. Our work looks at some of the repercussions of these modeling choices for learning. What functions can these richer V1 models learn easily that simple V1 models cannot? Are richer V1 models better at learning to categorize objects? In a first step, we implemented a few rate models of V1 neurons: affine-sigmoidal simple cell response, an energy-model, the model of [1], and the canonical circuit of [2]. We randomly initialized the filters in these models and trained a classifier on the V1 representation to categorize images from MNIST, and NORB as well as a synthetic dataset of shapes. Although we used cross-validation to permit each model family as much capacity as it could use, we found that in all cases the complex-cell models generalized significantly better from the limited number of examples used for training. In a second step, we experimented with a slow-features approach to initializing the filters instead of random initialization. This initialization yielded model neurons with additional robustness to edge position in images, and lead to still better generalization. In pursuit of this result, we introduced an approximation of the learning criterion in [3] with a computational cost that is linear (rather than quadratic) in the size of our model V1, and constant (rather than linear) in the number of training examples. In a third step, we experimented with multi-layer convolutional networks built from V1 cell model building blocks. Again, the networks with complex-like response functions outperform simple-cell-like ones. This sort of network achieved the best published test-set performance on MNIST among purely supervised algorithms. Recent complex cell rate models support an improved ability to learn to categorize objects relative to traditional simple-cell models, without incurring extra computational expense. Additional experiments with hybrid models obtained by mixing and matching elements from the V1 models revealed two important mathematical ingredients for significantly better performance over a basic affine-sigmoid response function. Firstly, a sum of at least two squared linear filters brings a signficant gain in statistical efficiency of learning. Secondly, a pointwise non-linearity that saturates polynomially rather than exponentially towards its asymptotic limit is significantly better in the majority of cases. These results underscore the importance of accurate modeling in efforts to understand learning in the visual system.
References
1. Rust NC, Schwartz O, Movshon JA, Simoncelli EP (2005) Spatiotemporal elements of macaque V1 receptive fields. Neuron 46: 945-956.
2. Kouh, M. and T. Poggio (2008) A Canonical Neural Circuit for Cortical Nonlinear Operations. Neural Computation 20(6): 1427-1451.
3. Körding, KP, Kayser, C., Einhäuser, W. and König, P. (2004) How are complex cell properties adapted to the statistics of natural scenes? Journal of Neurophysiology 91(1):206-212.
Conference:
Computational and Systems Neuroscience 2010, Salt Lake City, UT, United States, 25 Feb - 2 Mar, 2010.
Presentation Type:
Poster Presentation
Topic:
Poster session II
Citation:
Bergstra
J,
Bengio
Y,
Lamblin
P,
Desjardins
G and
Louradour
J
(2010). Image classification with complex cell neural networks.
Front. Neurosci.
Conference Abstract:
Computational and Systems Neuroscience 2010.
doi: 10.3389/conf.fnins.2010.03.00334
Copyright:
The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers.
They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.
The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.
Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.
For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.
Received:
08 Mar 2010;
Published Online:
08 Mar 2010.
*
Correspondence:
James Bergstra, University of Montreal, Montréal, Canada, james.bergstra@gmail.com