Modulation of dopaminergic activity by the expectation of reward: a computational neural model.
-
1
Chemnitz University of Technology, Department of Computer Science, Germany
-
2
Charité University Hospital, Bernstein Center for Computational Neuroscience, Germany
In classical conditioning experiments, dopaminergic neurons in the ventral tegmental area (VTA) exhibit a reward-prediction error pattern by producing phasic bursts for unexpected rewards (US) and reward-predicting stimuli (CS), and making a pause in firing at reward omission [1]. The pathways subserving time estimation of rewards are not fully determined yet, but the lateral habenula (LHb) has been shown to be excited by small or absent rewards and to selectively inhibit VTA cells. Additionally, cells in the striatum are considered as coincidence detectors able to respond for particular cortical patterns, what can be used for time estimation when cortical oscillations at various frequencies are deterministic [2].
Using mean-firing rate neurons and homeostatic covariance-based learning rules (as in [3]), we propose here a computational model of the interactions during conditioning between the ventromedial prefrontal cortex (vmPFC), the ventral basal ganglia (vBG, including the shell part of the nucleus accumbens NAcc and the ventral pallidum VP), LHb, the amygdala, the lateral hypothalamus (LH) as well as VTA. The basolateral amygdala learns to represent the emotional valence of a rewarding or punishing US with respect to the organism's homeostasis signaled by LH, as well as to associate this valence to the predicting CS. The centromedial amygdala elicits dopaminergic bursts in VTA for both CS and US. The emotional valence of a predicting cue is further transmitted to the shell of NAcc, which opens a recurring loop with vmPFC through VP and the mediodorsal thalamus, allowing vmPFC neurons to exhibit oscillating patterns at various frequencies, but synchronized at CS onset. When the learned delay between the CS and US has passed, the shell detects the corresponding pattern in vmPFC and inhibits VTA, either directly to cancel VTA activation for expected rewards, or indirectly through VP and LHb to pause dopaminergic activation through activation of the inhibitory interneurons of VTA.
This model proposes a functional explanation of dopaminergic activation during conditioning with respect to the known anatomy and explains behavioural data such as extinction or the scalar property of time estimation. It points out the importance of homeostasis in the valuation of rewards and the formation of preferences. It forms a first step towards the comprehension of ventral BG functioning in the learning of incentive values and ultimately decision-making.
Acknowledgements
This work was supported by the DFG grant HA2630/4-2 "The cognitive control of visual perception and action selection".
References
[1] Schultz, W., Dayan, P., and Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275(5306):1593–1599.
[2] Matell, M. S. and Meck, W. H. (2004). Cortico-striatal circuits and interval timing: coincidence detection of oscillatory processes. Brain Res Cogn Brain Res, 21(2):139–170.
[3] Schroll, H., Vitay, J., and Hamker, F. H. (2012). Working memory and response selection: a computational account of interactions among cortico-basalganglio-thalamic loops. Neural Netw, 26:59–74.
Keywords:
Amygdala,
Dopamine,
Homeostasis,
prediction,
Reward,
timing estimation,
ventral basal ganglia
Conference:
Bernstein Conference 2012, Munich, Germany, 12 Sep - 14 Sep, 2012.
Presentation Type:
Poster
Topic:
Attention, reward, decision making
Citation:
Vitay
J and
Hamker
FH
(2012). Modulation of dopaminergic activity by the expectation of reward: a computational neural model..
Front. Comput. Neurosci.
Conference Abstract:
Bernstein Conference 2012.
doi: 10.3389/conf.fncom.2012.55.00123
Copyright:
The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers.
They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.
The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.
Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.
For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.
Received:
11 May 2012;
Published Online:
12 Sep 2012.
*
Correspondence:
Dr. Julien Vitay, Chemnitz University of Technology, Department of Computer Science, Chemnitz, 09107, Germany, julien.vitay@informatik.tu-chemnitz.de