A delta-rule approximation to Bayesian inference in change-point problems
-
1
Princeton University, United States
-
2
University of Pennsylvania, United States
Whether it is a stock market crash, falling in love, or task switching in controlled behavioral experiment, life is full of "change-points" that cause the rules of the game to change abruptly, often without warning. To thrive in these circumstances, individuals must recognize these events quickly and adapt their behavior accordingly - for example by pulling out their money, putting on cologne, or adopting the appropriate task rules. As with other inference problems, ideal-observer models can help shed light on the computational demands of, and solutions to, change-point problems. Online Bayesian solutions to this problem have been reported [Adams & MacKay, Technical report Cambridge University, 2007; Fearnhead & Liu J. R. Statist. Soc. B 69(4):589, 2007], which can capture many aspects of human behavior on a simple change-point task [Nassar et al., in preparation]. However, despite this success, the computations implied by the Bayesian models seem biologically implausible. These computations require the brain to maintain and update a constantly growing probability distribution (the "run-length distribution") over all possible locations of the last change-point. In this work we address this shortcoming by systematically reducing the full Bayesian solution to show that it can be well approximated by a form of delta rule. This reduced Bayesian algorithm updates its current beliefs about the world based on the difference between the model’s current predictions about the world and the observed reality. Given the substantial evidence for this kind of prediction-error signal in the brain [e.g. Schultz et al., Science 275:1593, 1997], the model is more plausible than the full Bayesian solution. This approach also yields a massive reduction in computational cost, going from O(t) computations per time step at time t, to O(1). Our approach rests on drastically simplifying the representation of the run-length distribution. In particular, we introduce a reduced distribution that is not maintained over all possible run-lengths but instead on just two run-lengths: the first a run-length of zero, which assumes that a change-point just occurred; and the other a non-zero run-length, which represents the expected time since the last change-point. Using this reduced distribution, we derived update equations for the parameters of the model that, for the mean of the predictive distribution, take the form a delta rule. We find that the learning rate depends on both average run-length and the probability of a change-point at the current time, given the latest data. We also show that an extension of this work to deal with hierarchical change-point models [Wilson et al., submitted] leads to a hierarchy of delta-rules. This hierarchical model can adjust to increasingly complex change-point dynamics, including cases not considered by the reported online Bayesian solutions, in which the frequency of change-points is unknown. This work establishes a formal relationship between a biologically plausible algorithm based on the delta rule and ideal-observer Bayesian models. The results allow us to characterize in detail the strengths and limitations of both kinds of model in performing on-line inference for a range of complex change-point problems.
Conference:
Computational and Systems Neuroscience 2010, Salt Lake City, UT, United States, 25 Feb - 2 Mar, 2010.
Presentation Type:
Poster Presentation
Topic:
Poster session III
Citation:
Wilson
RC,
Nassar
M and
Gold
J
(2010). A delta-rule approximation to Bayesian inference in change-point problems.
Front. Neurosci.
Conference Abstract:
Computational and Systems Neuroscience 2010.
doi: 10.3389/conf.fnins.2010.03.00178
Copyright:
The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers.
They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.
The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.
Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.
For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.
Received:
02 Mar 2010;
Published Online:
02 Mar 2010.
*
Correspondence:
Robert C Wilson, Princeton University, Princeton, United States, rcw2@princeton.edu