Hierachical Models of Natural Images
-
1
Max Planck Institute for Biological Cybernetics, Germany
Here, we study two different approaches to estimate the multi-information of natural images. In both cases, we begin with a whitening step. Then, in the first approach, we use a hierarchical multi-layer ICA model [1] which is an efficient variant of projection pursuit density estimation. Projection pursuit [2] is a nonparametric density estimation technique with universal approximation properties. That is, it can be proven to converge to the true distribution in the limit of infinite amount of data and layers. For the second approach, we suggest a new model which consists of two layers only and has much less degrees of freedom than the multi-layer ICA model. In the first layer we apply symmetric whitening followed by radial Gaussianization [3,4] which transforms the norm of the image patches such that the distribution over the norm of the image patches matches the radial distribution of a multivariate Gaussian. In the next step, we apply ICA. The first step can be considered as a contrast gain control mechanism and the second one yields edge filters similar to those in primary visual cortex.
By evaluating quantitatively the redundancy reduction achieved with the two approaches, we find that the second procedure fits the distribution significantly better than the first one. On the van Hateren data set (400.000 image patches of size 12x12) with log-intensity scale, the redundancy reduction in the multi-layer ICA model yields 0.162,0.081,0.034,0.021,0.013,0.009,0.006,0.004,0.003,0.002 bits/pixel after the first, second, third, fourth, …, tenth layer, respectively.( For the training set size used, the performance decreases after the tenth layer). In contrast, we find a redundancy reduction of 0.342 bits/pixel after the first layer and 0.073 bits/pixel after the second layer for the second approach. In conclusion, the universal approximation property of the deep hierarchical architecture in the first approach does not pay off for the task of density estimation in case of natural images.
References
1. Chen and Gopinath. 2001. Proc. NIPS, vol. 13, pp. 423–429.
2. Friedman J. et al. 1984. J. Amer. Statist. Assoc., vol. 71, pp. 599–608.
3. Lyu S. and Simoncelli E. P. 2008. Proc. NIPS, vol. 21, pp.1009–1016.
4. Sinz F. H. and Bethge M. 2008. MPI Technical Report
Conference:
Bernstein Conference on Computational Neuroscience, Frankfurt am Main, Germany, 30 Sep - 2 Oct, 2009.
Presentation Type:
Poster Presentation
Topic:
Abstracts
Citation:
Hosseini
R and
Bethge
M
(2009). Hierachical Models of Natural Images.
Front. Comput. Neurosci.
Conference Abstract:
Bernstein Conference on Computational Neuroscience.
doi: 10.3389/conf.neuro.10.2009.14.123
Copyright:
The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers.
They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.
The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.
Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.
For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.
Received:
27 Aug 2009;
Published Online:
27 Aug 2009.
*
Correspondence:
Reshad Hosseini, Max Planck Institute for Biological Cybernetics, Tubingen, Germany, reshad.hosseini@tuebingen.mpg.de