Bonsai trees: How the Pavlovian system sculpts sequential decisions
-
1
UCL, Gatsby Computational Neuroscience Unit, Wellcome Trust Neuroimaging Centre and Medical School, United Kingdom
-
2
Harvard University, United States
-
3
UCL, Gatsby Computational Neuroscience Unit, United Kingdom
-
4
UCL, Institute of Cognitive Neuroscience, United Kingdom
People face decision problems of gargantuan dimensions daily and happily. Efficient pruning of large decision trees is likely a crucial ingredient of this striking ability. Here, we examine the ability of the Pavlovian system to shape human goal-directed decision making in deep sequential choice scenarios. More specifically, we ask to what extent people optimistically inhibit the evaluation of parts of decision trees that lie below large negative reinforcements. We arrange the cost function so that pruning is counterproductive, thus pitting a reflexive Pavlovian tendency (suppression in the face of punishments) against optimal goal-directed search. Three groups of 15 subjects played a novel computerized task in which they used two buttons to navigate between six states. They first learned that each button led to a particular deterministic transition. For instance, from state 3 button 1 led to state 6 whereas button 2 led to state 4. Subjects then learned the costs of particular transitions. The groups differed in terms of the costs associated with the three most costly transitions (-70, -100 and -140 points respectively). Subjects were then repeatedly dropped in a random state, and asked to produce a varying number of sequential button presses such as to maximise the cumulative rewards earned over the entire sequence. They received a proportion of these points in cash at the end of the experiment. We first compared subjects’ behaviour to optimal choices. Subjects chose optimally on over 70%of the trials when up to three choices remained. They were at chance when 6 or more choices remained. We then fit a model with two discounting parameters. The specific discounting parameter applied to reinforcements on subparts of the decision tree below the large punishments; the general discounting parameter applied to other reinforcements. We find that this model captures subjects’ choices extraordinarily well. Subjects chose the action associated with the higher expectation (after pruning) on close to 90%of the trials, independently of the number of choices remaining. Importantly, subjects pruned after the large negative reinforcements even when it was strongly advantageous not to; that is even when it was profitable to incur large losses because of the large rewards hidden below them. We then compared subjects’ specific pruning to various psychometric variables and found that specific pruning was strongly positively correlated with measures of depression (Beck Depression inventory scores; p=0.0017; r=0.4589) and strongly negatively correlated with a measure of extraverted personality (Revised NEO Personality Inventory, Extraversion factor; p=0.00247, r=-0.4450). (Both significant at a Bonferroni corrected threshold of 0.005). This work provides fundamental new insight into specific interactions between goal-directed, sequential choices and the aversive side of Pavlovian systems. It shows how Pavlovian influences sculpt decision trees, thereby influencing the possible outcomes of a goal-directed tree search; and how this process varies in a predictable manner with standard measures of psychopathology and personality. We believe that this is the first specific demonstration of how lowly Pavlovian systems may fundamentally affect – and facilitate – the function of higher cognitive systems.
Conference:
Computational and Systems Neuroscience 2010, Salt Lake City, UT, United States, 25 Feb - 2 Mar, 2010.
Presentation Type:
Poster Presentation
Topic:
Poster session II
Citation:
Huys
QJ,
Eshel
N,
Dayan
P and
Roiser
JP
(2010). Bonsai trees: How the Pavlovian system sculpts sequential decisions.
Front. Neurosci.
Conference Abstract:
Computational and Systems Neuroscience 2010.
doi: 10.3389/conf.fnins.2010.03.00171
Copyright:
The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers.
They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.
The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.
Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.
For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.
Received:
02 Mar 2010;
Published Online:
02 Mar 2010.
*
Correspondence:
Quentin J Huys, UCL, Gatsby Computational Neuroscience Unit, Wellcome Trust Neuroimaging Centre and Medical School, London, United Kingdom, qhuys@cantab.net