Online competitive incremental-decremental clustering with the Winner-Kill-Loser rule
-
1
Boston University, Center of Excellence for Learning in Education, Science, and Technology, United States
-
2
Kansai University, Faculty of Informatics, Japan
Incremental clustering techniques are general-purpose methods for clustering where the number of cluster nodes is determined online, and in particular increased, upon presentation of training data. Examples include the Leader algorithm (Hartigan, 1975), and adaptive resonance theory (Carpenter and Grossberg, 2003). Incremental learning rules for clustering are attractive for their simplicity and ease of application, and have generally led to good results on a number of tasks. Recently, a variant of incremental learning which considers online removal of tuned nodes was proposed to train simple cells in a hierarchical network (Fukushima, 2010). The learning rule presented therein, termed winner-kill-loser (WKL), is a kind of competitive rule in which only sufficiently active units enter the competition and where the losing units are effectively removed from the network.
In this work, we extend the WKL learning rule to study the structure and formation of simple cell receptive fields. We start with an analysis of the properties of the WKL rule and demonstrate that, unlike many clustering techniques, in its simplest form the learning rule does not seek to approximate maximum-likelihood estimates of the node parameters. Instead, the WKL learning rule converges asymptotically to a uniform covering of the input space, irrespective of the data probability distribution. Even in the case of a uniform data distribution, our analysis also highlights conditions for the presence of irreducible gaps in this covering. We can compute a lower bound on the size of these gaps, and show how this bound relates to clustering performance in sample simulations. In particular, we investigate through simulation whether this lower bound can account for improvements in performance reported when using the dual-threshold rule, in which the net input activation threshold for each simple cell is lowered after learning (Fukushima, 2010). Based on these results, we propose to complement the WKL learning rule by inserting activity-dependent thresholds that govern the selectivity of learned receptive fields. We show through simulation that the structure of the newly developed receptive fields approximates the input data distribution.
References
1. Hartigan, J.A. (1975). Clustering algorithms. Wiley.
2. Carpenter, G.A. and Grossberg, S. (2003). Adaptive resonance theory. In M.A. Arbib (Ed.) The Handbook of Brain Theory and Neural Networks, Second Edition (pp.87-90). Cambridge, MA: MIT Press.
3. Fukushima, K. (2010). Neocognitron trained with winner-kill-loser rule. Neural Networks, 23, 926-938.
Â
Keywords:
computational neuroscience
Conference:
4th INCF Congress of Neuroinformatics, Boston, United States, 4 Sep - 6 Sep, 2011.
Presentation Type:
Poster Presentation
Topic:
Computational neuroscience
Citation:
Leveille
J,
Hayashi
I,
Fukushima
K and
Versace
M
(2011). Online competitive incremental-decremental clustering with the Winner-Kill-Loser rule.
Front. Neuroinform.
Conference Abstract:
4th INCF Congress of Neuroinformatics.
doi: 10.3389/conf.fninf.2011.08.00142
Copyright:
The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers.
They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.
The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.
Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.
For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.
Received:
17 Oct 2011;
Published Online:
19 Oct 2011.
*
Correspondence:
Dr. Jasmin Leveille, Boston University, Center of Excellence for Learning in Education, Science, and Technology, Boston, United States, jasminl@cns.bu.edu