In a recent study, Chen et al. (2019) found that medicated Parkinson's patients were less loss averse than healthy controls in an effort-based decision-making task. Loss aversion refers to a tendency to weight losses (punishments) more heavily than equivalent gains (rewards) (Kahneman and Tversky, 1979; Tversky and Kahneman, 1992), while effort-based decision making refers to tasks in which responding requires the exertion of physical effort (Kurniawan et al., 2011). Chen et al. found that when decisions were framed in terms of maximizing gains, Parkinson's patients and healthy controls were equally motivated to exert effort to obtain reward. However, when framed in terms of minimizing losses, patients were less motivated to exert effort to avoid punishment. Given that many clinical disorders are characterized by aberrant motivational states (Rahman et al., 2001; Cléry-Melin et al., 2011; Baraduc et al., 2013; Chong et al., 2015) and abnormal sensitivity to rewards and punishments (Kobayakawa et al., 2010; Treadway et al., 2012), studying effort-based decision making can provide important insights into the behavioral effects of such disorders and the mechanisms by which they arise.
In Chen et al., participants performed a typical effort-based decision-making task: On each trial they were shown an amount of points and a level of physical force required to execute the decision (using an individually-calibrated dynamometer). In the reward context, participants could choose to either exert the displayed force to gain the points (obtain reward) or skip the trial and receive nothing. In the punishment context, participants could exert the force to receive nothing (avoid punishment) or skip the trial and lose the points. Trial duration was fixed to avoid confounding temporal discounting with effort discounting. To the resulting choice data, the authors fit several computational models of effort discounting, which compute choice utility based on the size of the reward/punishment attenuated by the effort required to respond (Hartmann et al., 2013). For example, Chen et al.'s (best-fitting) parabolic discounting function is given by:
where U represents choice utility, R is the reward/punishment amount, E is the effort required, α is an effort-discounting parameter, and t indexes trials. The probability of choosing to exert effort on a given trial is then computed using a soft-max choice function that scales choice utility by a choice stochasticity parameter, β:
Parameter estimates from these models supported the behavioral results: Patients and controls had similar average effort-discounting parameters in the reward context, but patients had significantly higher effort-discounting parameters in the punishment context, confirming that patients were less motivated to exert effort to avoid punishment.
Although Chen et al. did not explicitly model the latent cognitive processes driving observed choice behavior (nor did they include any neurophysiological measures), the authors speculate that their findings could be due to dopaminergic Parkinson's medication differentially suppressing activity in Basal Ganglia pathways associated with processing punishment (Frank, 2005; Argyelan et al., 2018). This suggestion is consistent with several computational reinforcement learning studies linking reduced Basal Ganglia dopamine activity with impaired learning on punishment-based tasks in medicated Parkinson's patients (Frank et al., 2004; Frank, 2005). However, without additional neurophysiological measures (e.g., fMRI, PET) and an appropriate cognitive model of latent decision processes (e.g., Ratcliff, 1978; Brown and Heathcote, 2008), it is difficult to draw strong conclusions about the latent mechanisms driving the observed loss aversion effects or their neurophysiological basis from Chen et al.'s analyses.
A natural starting point for extending Chen et al.'s modeling to answer such questions would be to augment their soft-max choice rule and utility-based effort-discounting functions with a more comprehensive cognitive process model of decision making, such as a sequential sampling model—the most successful and widely applied class of decision-making models in model-based cognitive neuroscience (for reviews, Mulder et al., 2014; Forstmann et al., 2016). Sequential sampling models treat decision making as a process of accumulating samples of evidence1 from stimuli until a threshold amount is reached, triggering a response (Figure 1).
Figure 1
Crucially, sequential sampling models can be thought of as generalizing soft-max into the time domain, which allows them to simultaneously explain how choices and response times arise from a common set of latent cognitive processes. This provides closer contact between model and data, additional constraint on theory, and more robust inferences. As noted in prior work (Tuerlinckx and De Boeck, 2005; Bogacz et al., 2006; Miletić et al., 2020), soft-max (Equation 2) is formally equivalent to the following sequential sampling choice function, which describes the probability that evidence first reaches the upper threshold (corresponding to an “exert effort” response in Figure 1) as:
where v is the mean rate of information processing (drift rate), A is the response threshold, s is the standard deviation of the evidence accumulation process, and t indexes trials. Equating drift rate with choice utility2 (Equation 1) and substituting β for 2A/s2 yields the original soft-max choice function used by Chen et al. (Equation 2). Integrating Chen's models with Equation 3 would thus yield a standard sequential sampling model with choice utility-based drift rates and an additional effort-discounting parameter. Sequential sampling models explain behavior in terms of psychologically interpretable parameters (e.g., information processing speed, response caution, motor response/encoding time, choice bias), so a combined model would simultaneously decompose observed decision-making behavior into component cognitive processes and quantify effort discounting within a single theoretical framework (for a similar approach to decomposing loss aversion in risk-based decision making using a standard sequential sampling model, Clay et al., 2017).
Such an approach would offer clear benefits for the field of effort-based decision making, improving measurement and facilitating theory development by providing a more detailed characterization of group and individual differences in loss aversion. In particular, this approach could expand on Chen et al.'s results by explaining the relationship between effort discounting and additional Parkinson's-related cognitive and motor deficits known to affect response latency (e.g., longer motor response times, less efficient information processing, impaired cognitive control over thresholds, O'Callaghan et al., 2017; Servant et al., 2018). Patients prone to effort discounting may have difficulty integrating information about effort into the decision process (a drift rate effect) or set more impulsive evidence criteria (a threshold effect) compared with healthy controls. Testing these competing accounts would further our understanding of Parkinson's disease and better titrate individual differences in cognitive and motor processes. Explicit mechanisms to capture additional phenomena relevant to effort-based decision making, such as learning, urgency, and fatigue effects, can also be instantiated within the same sequential sampling framework (Milosavljevic et al., 2010; Miletić et al., 2020), and linking cognitive processes with neurophysiological measures (e.g., by treating model parameters as covariates to neural activity or constructing a joint model, Turner et al., 2017, 2019) would be especially informative regarding Chen et al.'s broader questions about the role of dopamine and Basal Ganglia activity in explaining Parkinson's-related differences in decision making. Overall, moving toward a cognitive process model of effort-based decision making promises a finer-grained mechanistic understanding of aberrant motivational states in neuropsychiatric disorders and more detailed insight into the sources of group and individual differences in effort-based decision making.
Statements
Author contributions
The author confirms being the sole contributor of this work and has approved it for publication.
Acknowledgments
I thank Steven Miletić for his helpful suggestions while preparing this manuscript.
Conflict of interest
The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Footnotes
1.^In the present context, evidence would constitute information about the relative value or utility of exerting effort vs. skipping the trial. This evidence is assumed to be sampled from noisy internal representations, in line with applications of these models to memory- and value-based decisions (e.g., Ratcliff, 1978; Krajbich and Rangel, 2011; Westbrook et al., 2020).
2.^The connection between drift rate and choice utility or value has been well-established in recent work merging reinforcement learning with sequential sampling models in the domain of value-based decision making (for a review, Miletić et al., 2020). Here drift rate is simply set equal to choice utility, however, many other linking functions are possible.
References
1
ArgyelanM.HerzallahM.SakoW.DeLuciaI.SarpalD.et al. (2018). Dopamine modulates striatal response to reward and punishment in patients with Parkinson's disease: a pharmacological challenge fMRI study. NeuroReport29, 532–540. 10.1097/WNR.0000000000000970
2
BaraducP.ThoboisS.GanJ.BroussolleE.DesmurgetM. (2013). A common optimization principle for motor execution in healthy subjects and Parkinsonian patients. J. Neurosci.33, 665–677. 10.1523/jneurosci.1482-12.2013
3
BogaczR.BrownE.MoehlisJ.HolmesP.CohenJ. D. (2006). The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. Psychol. Rev.113, 700–765. 10.1037/0033-295X.113.4.700
4
BrownS. D.HeathcoteA. (2008). The simplest complete model of choice response time: linear ballistic accumulation. Cognit. Psychol.57, 153–178. 10.1016/j.cogpsych.2007.12.002
5
ChenX.VoetsS.JenkinsonN.GaleaJ. M. (2019). Dopamine-dependent loss aversion during effort-based decision-making. J. Neurosci.40, 661–670. 10.1523/JNEUROSCI.1760-19.2019
6
ChongT. J.BonnelleV.ManoharS.VeromannK. R.MuhammedK.et al. (2015). Dopamine enhances willingness to exert effort for reward in Parkinson's disease. Cortex69, 40–46. 10.1016/j.cortex.2015.04.003
7
ClayS. N.ClitheroJ. A.HarrisA. M.ReedC. L. (2017). Loss aversion reflects information accumulation, not bias: a drift-diffusion model study. Front. Psychol.8:1708. 10.3389/fpsyg.2017.01708
8
Cléry-MelinM. L.SchmidtL.LafargueG.BaupN.FossatiP.et al. (2011). Why don't you try harder? An investigation of effort production in major depression. PLoS ONE6:e23178. 10.1371/journal.pone.0023178
9
ForstmannB. U.RatcliffR.WagenmakersE.-J. (2016). Sequential sampling models in cognitive neuroscience: advantages, applications, and extensions. Ann. Rev. Psychol. 67, 641–666. 10.1146/annurev-psych-122414-033645
10
FrankM. J. (2005). Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. J. Cognit. Neurosci.17, 51–72. 10.1162/0898929052880093
11
FrankM. J.SeebergerL. C.O'ReillyR. C. (2004). By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science306, 1940–1943. 10.1126/science.1102941
12
HartmannM. N.HagerO. M.ToblerP. N.KaiserS. (2013). Parabolic discounting of monetary rewards by physical effort. Behav. Process.100, 192–196. 10.1016/j.beproc.2013.09.014
13
KahnemanD.TverskyA. (1979). Prospect theory: an analysis of decision under risk. Econometrica47, 263–291. 10.2307/1914185
14
KobayakawaM.TsuruyaN.KawamuraM. (2010). Sensitivity to reward and punishment in Parkinson's disease: an analysis of behavioral patterns using a modified version of the Iowa gambling task. Parkinsonism Related Disord.16, 453–457. 10.1016/j.parkreldis.2010.04.011
15
KrajbichI.RangelA. (2011). Multialternative drift-diffusion model predicts the relationship between visual fixations and choice in value-based decisions. Proc. Natl. Acad. Sci. U.S.A. 108, 13852–13857. 10.1073/pnas.1101328108
16
KurniawanI. T.Guitart-MasipM.DolanR. J. (2011). Dopamine and effort-based decision making. Front. Neurosci. 5:81. 10.3389/fnins.2011.00081
17
MiletićS.BoagR. J.ForstmannB. U. (2020). Mutual benefits: combining reinforcement learning with sequential sampling models. Neuropsychologia136:107261. 10.1016/j.neuropsychologia.2019.107261
18
MilosavljevicM.MalmaudJ.HuthA.KockC.RangelA. (2010). The drift diffusion model can account for the accuracy and reaction time of value-based choices under high and low time pressure. Judgment Dec. Mak.5, 437–449. 10.2139/ssrn.1901533
19
MulderM. J.Van MaanenL.ForstmannB. U. (2014). Perceptual decision neurosciences - A model-based review. Neuroscience277, 872–884. 10.1016/j.neuroscience.2014.07.031
20
O'CallaghanC.HallJ. M.TomassiniA.MullerA. J.WalpolaI. C.MoustafaA. A.et al. (2017). Visual hallucinations are characterized by impaired sensory evidence accumulation: insights from hierarchical drift diffusion modeling in Parkinson's disease. Biol. Psychiatry Cognit. Neurosci. Neuroimaging2, 680–688. 10.1016/j.bpsc.2017.04.007
21
RahmanS.SahakianB. J.CardinalR. N.RogersR. D.RobbinsT. W. (2001). Decision making and neuropsychiatry. Trends Cognit. Sci.5, 271–277. 10.1016/s1364-6613(00)01650-8
22
RatcliffR. (1978). A theory of memory retrieval. Psychol. Rev.85, 59–108. 10.1037/0033-295X.85.2.59
23
ServantM.van WouweN.WylieS. A.LoganG. D. (2018). A model-based quantification of action control deficits in Parkinson's disease. Neuropsychologia111, 26–35. 10.1016/j.neuropsychologia.2018.01.014
24
TreadwayM. T.BossallerN. A.SheltonR. C.ZaldD. H. (2012). Effort-based decision-making in major depressive disorder: a translational model of motivational anhedonia. J. Abnormal Psychol.121, 553–558. 10.1037/a0028813
25
TuerlinckxF.De BoeckP. (2005). Two interpretations of the discrimination parameter. Psychometrika70, 629–650. 10.1007/s11336-000-0810-3
26
TurnerB. M.ForstmannB. U.LoveB. C.PalmeriT. J.Van MaanenL. (2017). Approaches to analysis in model-based cognitive neuroscience. J. Math. Psychol.76, 65–79. 10.1016/j.jmp.2016.01.001
27
TurnerB. M.ForstmannB. U.SteyversM. (2019). Joint Models of Neural and Behavioral Data: Computational Approaches to Cognition and Perception. Cham: Springer International Publishing. 10.1007/978-3-030-03688-1
28
TverskyA.KahnemanD. (1992). Advances in prospect theory: cumulative representation of uncertainty. J. Risk Uncertainty5, 297–323. 10.1007/978-3-319-20451-2_24
29
VossA.NaglerM.LercheV. (2013). Diffusion models in experimental psychology: a practical introduction. Exp. Psychol.60, 385–402. 10.1027/1618-3169/a000218
30
WestbrookA.van den BoschR.MäättäJ. I.HofmansL.PapadopetrakiD.CoolsR.et al. (2020). Dopamine promotes cognitive effort by biasing the benefits versus costs of cognitive work. Science367, 1362–1366. 10.1126/science.aaz5891
Summary
Keywords
loss aversion, effort-based decision making, sequential sampling models, model-based cognitive neuroscience, Parkinson's disease
Citation
Boag RJ (2020) Commentary: Dopamine-Dependent Loss Aversion during Effort-Based Decision-Making. Front. Neurosci. 14:468. doi: 10.3389/fnins.2020.00468
Received
05 February 2020
Accepted
15 April 2020
Published
13 May 2020
Volume
14 - 2020
Edited by
Walter Adriani, Istituto Superiore di Sanità (ISS), Italy
Reviewed by
Nick Garber Hollon, Salk Institute for Biological Studies, United States
Updates
Copyright
© 2020 Boag.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Russell J. Boag r.j.boag@uva.nl
This article was submitted to Decision Neuroscience, a section of the journal Frontiers in Neuroscience
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.