A commentary on
Differential influence of levodopa on reward-based learning in Parkinson’s disease.
by Graef, S., Biele, G., Krugel, L. K., Marzinzik, F., Wahl, M., Wotka, J., Klostermann, F., and Heekeren, H. R. (2010). Front. Hum. Neurosci. 4:169. doi: 10.3389/fnhum.2010.00169
Parkinson’s disease (PD) and associated dopaminergic medications, such as levodopa and dopamine agonists, have differential effects on cognition. Dopamine medications might either enhance or impair cognition in PD patients depending on type of cognitive task and medication used. In a recent neuropsychological study, Graef and colleagues have tested the effects of PD and levodopa monotherapy on feedback and reversal learning. In feedback learning, subjects learn to select stimuli, based on either rewarding (positive) or punishing (negative) feedback to their responses. Thus, reward learning involves learning to select responses that lead to reward, while punishment learning involves learning to avoid responses that lead to negative outcome. In reversal learning tasks, subjects initially learn to associate different stimuli with different responses, and subsequently learn to associate the same stimuli with the opposite responses (i.e., reversal). Graef et al. have found that levodopa enhances reward learning but impairs reversal learning in PD patients. This finding is consistent with earlier reports that dopamine medications enhances reward learning (Frank et al., 2004; Moustafa et al., 2008a; Bodi et al., 2009; Palminteri et al., 2009) but impairs reversal learning (Swainson et al., 2000; Cools et al., 2001) in PD patients.
One notable aspect of the current study is Graef et al. have tested PD patients on levodopa only. This is contrasted from prior studies that have recruited PD patients on multiple therapies including levodopa and dopamine agonists (Cools et al., 2001; Frank et al., 2004; Moustafa et al., 2008b; Bodi et al., 2009), thus confounding dissociable effects of levodopa vs. dopamine agonists on cognition. Furthermore, the Graef et al. findings are in agreement with other studies showing that the administration of levodopa enhances feedback learning in rats (Pavlis et al., 2006), healthy subjects (Knecht et al., 2004; Pessiglione et al., 2006; Floel et al., 2008; Pleger et al., 2009), stroke patients (Scheidtmann et al., 2001; Rosser et al., 2008), and PD patients (Beeler et al., 2010; de Vries et al., 2010).
Interestingly, Graef et al. have also found that the administration of different doses of levodopa to PD patients does not affect reward learning, that is, levodopa has no dose effect on reward learning. This is in line with the fact that levodopa is a dopamine precursor, taken up by dopamine neurons and converted into dopamine, and thus largely produced in natural conditions. Accordingly, an increase of doses of levodopa may not significantly alter presynaptic dopamine levels in the basal ganglia, and thus may not impact cognition, as reported in the Graef study. This is, however, contrasted with the effects of different doses of dopamine agonists on cognition. For example, studies have found that in healthy subjects, a low-dose (1.25 mg) of the dopamine agonist bromocriptine has no effect or impairs working memory (Gibbs and D’Esposito, 2005) while a high-dose (2.5 mg) of bromocriptine enhances working memory (Luciana et al., 1992; Luciana and Collins, 1997; McDowell, et al., 1998). It is possible that a larger dose of dopamine agonists further increase postsynaptic dopamine in the basal ganglia and prefrontal cortex (Moustafa and Gluck, 2011), and thus explain how different doses of dopamine agonists are associated with dissociable cognitive function.
Another notable finding in the Graef et al. study is in addition to reward learning, levodopa also ameliorates depression and motor performance (as measured by the Beck Depression Inventory and UPDRS scales) but has no effect on Mini-Mental State Exam (MMSE). This is in line with findings that depression and motor performance are related to basal ganglia dopamine function (Walter et al., 2010), while MMSE scores are associated with medial temporal lobe function (Ikeda et al., 2008; Ding et al., 2009), which is thought to be intact in PD patients.
One unresolved issue in the literature is the confounding results regarding the effects of dopamine medications on feedback learning and reversal learning. With regards to feedback learning, unlike Graef et al. results, other studies have found that the administration of dopamine medications to PD patients impairs or has no effect on feedback learning (Czernecki et al., 2002; Shohamy et al., 2006; Jahanshahi et al., 2009). Similarly, studies have shown that the administration of dopamine medications to PD patients either enhances or has no effect on reversal learning (Czernecki et al., 2002; Rutledge et al., 2009), unlike what Graef et al. have found. It is important to note that unlike the Graef study, other studies have tested PD patients on multiple therapies, including levodopa and dopamine agonists, which might explain the different results in these studies. Along the same lines, Feigin et al. (2003) have found that the administration of levodopa slightly (but not significantly) impairs sequence learning in PD patients. Sequence learning is a feedback learning paradigm in which subjects learn to make a “sequence” of motor responses, based on corrective feedback. Feigin et al. have tested PD patients who were on both levodopa and dopamine agonists, but were only on levodopa during the time of testing. Differences between the Graef et al. and Feigin et al. results could be due to long-term effects of the intake of dopamine agonists on cognition in the Feigin et al. study.
How would dopamine replacement therapies enhance reward learning but impair reversal learning? There are two theories that explain Graef et al. results. Frank (2005) argue that the basal ganglia direct pathway (along with dopamine D1 receptors) is required for reward learning, while the indirect pathways (along with dopamine D2 receptors) is essential for punishment learning, which involves learning to avoid responses that lead to negative feedback. Frank further argues that dopamine medications increase dopamine levels in the basal ganglia and enhance reward learning via D1 receptors, but impair punishment learning via D2 receptors. According to Frank, following reversal of reward contingencies in the reversal learning task, subjects are more likely to receive more negative feedback, and thus an inability to learn from negative feedback will lead to reversal learning impairment in medicated PD patients. An alternative theory is proposed by Cools et al. (2001), which argue that while the ventral striatum is intact in PD patients, dorsal striatum is dysfunctional. Cools et al. further argue that reward learning is mediated by the dorsal striatum while punishment learning by the ventral striatum. According to Cools, dopamine medications ameliorate the function of the dorsal striatum, but overdose the ventral striatum and orbitofrontal cortex loop, and thus enhance reward learning but impair reversal learning. Moustafa and Gluck (under review) have recently built a computational model showing how overdosing cortical areas impair reversal learning in medicated PD patients, as reported by Graef et al.
Overall, Graef and colleagues have shown that the administration of levodopa monotherapy has both enhancing and deleterious effects on cognition in PD patients. Future research should investigate the dissociable effects of levodopa vs. dopamine agonists on cognition in PD patients. To my knowledge, studying differential effects of dopamine replacement therapies in PD patients has been only conducted using working memory tasks (Brusa et al., 2003; Costa et al., 2003), but not with reward and reversal learning paradigms.
References
Beeler, J. A., Cao, Z. F., Kheirbek, M. A., Ding, Y., Koranda, J., Murakami, M., and Kang, U. J., Zhuang, X. (2010). Dopamine-dependent motor learning: insight into levodopa’s long-duration response. Ann. Neurol. 67, 639–647.
Bodi, N., Keri, S., Nagy, H., Moustafa, A., Myers, C. E., Daw, N., Dibo, G., Takats, A., Bereczki, D., and Gluck, M. A. (2009). Reward-learning and the novelty-seeking personality: a between- and within-subjects study of the effects of dopamine agonists on young Parkinson’s patients. Brain 132, 2385–2395.
Brusa, L., Bassi, A., Stefani, A., Pierantozzi, M., Peppe, A., Caramia, M. D., Boffa, L., Ruggieri, S., and Stanzione, P. (2003). Pramipexole in comparison to L-DOPA: a neuropsychological study. J. Neural Transm. 110, 373–380.
Cools, R., Barker, R. A., Sahakian, B. J., and Robbins, T. W. (2001). Enhanced or impaired cognitive function in Parkinson’s disease as a function of dopaminergic medication and task demands. Cereb. Cortex 11, 1136–1143.
Costa, A., Peppe, A., Dell’Agnello, G., Carlesimo, G. A., Murri, L., Bonuccelli, U., and Caltagirone, C. (2003). Dopaminergic modulation of visual-spatial working memory in Parkinson’s disease. Dement. Geriatr. Cogn. Disord. 15, 55–66.
Czernecki, V., Pillon, B., Houeto, J. L., Pochon, J. B., Levy, R., and Dubois, B. (2002). Motivation, reward, and Parkinson’s disease: influence of dopatherapy. Neuropsychologia 40, 2257–2267.
de Vries, M. H., Ulte, C., Zwitserlood, P., Szymanski, B., and Knecht, S. (2010). Increasing dopamine levels in the brain improves feedback-based procedural learning in healthy participants: an artificial-grammar-learning experiment. Neuropsychologia 48, 3193–3197.
Ding, B., Chen, K. M., Ling, H. W., Sun, F., Li, X., Wan, T., Chai, W. M., Zhang, H., Zhan, Y., and Guan, Y. J. (2009). Correlation of iron in the hippocampus with MMSE in patients with Alzheimer’s disease. J. Magn. Reson. Imaging 29, 793–798.
Feigin, A., Ghilardi, M. F., Carbon, M., Edwards, C., Fukuda, M., Dhawan, V., Margouleff, C., Ghez, C., and Eidelberg, D. (2003). Effects of levodopa on motor sequence learning in Parkinson’s disease. Neurology 60, 1744–1749.
Floel, A., Vomhof, P., Lorenzen, A., Roesser, N., Breitenstein, C., and Knecht, S. (2008). Levodopa improves skilled hand functions in the elderly. Eur. J. Neurosci. 27, 1301–1307.
Frank, M. J. (2005). Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. J. Cogn. Neurosci. 17, 51–72.
Frank, M. J., Seeberger, L. C., and O’Reilly, R. C. (2004). By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306, 1940–1943.
Gibbs, S. E., and D’Esposito, M. (2005). A functional MRI study of the effects of bromocriptine, a dopamine receptor agonist, on component processes of working memory. Psychopharmacology (Berl.) 180, 644–653.
Ikeda, E., Shiozaki, K., Takahashi, N., Togo, T., Odawara, T., Oka, T., Inoue, T., and Hirayasu, Y. (2008). Total Mini-Mental State Examination score and regional cerebral blood flow using Z score imaging and automated ROI analysis software in subjects with memory impairment. Ann. Nucl. Med. 22, 539–542.
Jahanshahi, M., Wilkinson, L., Gahir, H., Dharminda, A., and Lagnado, D. A. (2009). Medication impairs probabilistic classification learning in Parkinson’s disease. Neuropsychologia 48, 1096–103.
Knecht, S., Breitenstein, C., Bushuven, S., Wailke, S., Kamping, S., Floel, A., Zwitserlood, P., and Ringelstein, E. B. (2004). Levodopa: faster and better word learning in normal humans. Ann. Neurol. 56, 20–26.
Luciana, M., and Collins, P. (1997). Dopaminergic modulation of working memory for spatial but not object cues in normal humans. J. Cogn. Neurosci. 9, 330–347.
Luciana, M., Depue, R. A., Arbisi, P., and Leon, A. (1992). Facilitation of working memory in humans by a D2 dopamine receptor agonist. J. Cogn. Neurosci. 4, 58–68.
McDowell, S., Whyte, J., and D’Esposito, M. (1998). Differential effect of a dopaminergic agonist on prefrontal function in traumatic brain injury patients. Brain 121(Pt 6), 1155–1164.
Moustafa, A. A., Cohen, M. X., Sherman, S. J., and Frank, M. J. (2008a). A role for dopamine in temporal decision making and reward maximization in parkinsonism. J. Neurosci. 28, 12294–12304.
Moustafa, A. A., Sherman, S. J., and Frank, M. J. (2008b). A dopaminergic basis for working memory, learning and attentional shifting in Parkinsonism. Neuropsychologia 46, 3144–3156.
Moustafa, A. A., and Gluck, M. A. (2011). A neurocomputational model of dopamine and prefrontal-striatal interactions during multicue category learning by Parkinson patients. J. Cogn. Neurosci. 23, 151–167.
Palminteri, S., Lebreton, M., Worbe, Y., Grabli, D., Hartmann, A., and Pessiglione, M. (2009). Pharmacological modulation of subliminal learning in Parkinson’s and Tourette’s syndromes. Proc. Natl. Acad. Sci. U.S.A. 106, 19179–19184.
Pavlis, M., Feretti, C., Levy, A., Gupta, N., and Linster, C. (2006). l-DOPA improves odor discrimination learning in rats. Physiol. Behav. 87, 109–113.
Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., and Frith, C. D. (2006). Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 442, 1042–1045.
Pleger, B., Ruff, C. C., Blankenburg, F., Kloppel, S., Driver, J., and Dolan, R. J. (2009). Influence of dopaminergically mediated reward on somatosensory decision-making. PLoS Biol. 7, e1000164. doi: 10.1371/journal.pbio.1000164
Rosser, N., Heuschmann, P., Wersching, H., Breitenstein, C., Knecht, S., and Floel, A. (2008). Levodopa improves procedural motor learning in chronic stroke patients. Arch. Phys. Med. Rehabil. 89, 1633–1641.
Rutledge, R. B., Lazzaro, S. C., Lau, B., Myers, C. E., Gluck, M. A., and Glimcher, P. W. (2009). Dopaminergic drugs modulate learning rates and perseveration in Parkinson’s patients in a dynamic foraging task. J. Neurosci. 29, 15104–15114.
Scheidtmann, K., Fries, W., Muller, F., and Koenig, E. (2001). Effect of levodopa in combination with physiotherapy on functional motor recovery after stroke: a prospective, randomised, double-blind study. Lancet 358, 787–790.
Shohamy, D., Myers, C. E., Geghman, K. D., Sage, J., and Gluck, M. A. (2006). L-dopa impairs learning, but spares generalization, in Parkinson’s disease. Neuropsychologia 44, 774–784.
Swainson, R., Rogers, R. D., Sahakian, B. J., Summers, B. A., Polkey, C. E., and Robbins, T. W. (2000). Probabilistic learning and reversal deficits in patients with Parkinson’s disease or frontal or temporal lobe lesions: possible adverse effects of dopaminergic medication. Neuropsychologia 38, 596–612.
Citation: Moustafa AA (2011) Levodopa enhances reward learning but impairs reversal learning in Parkinson’s disease patients. Front. Hum. Neurosci. 4:240. doi:10.3389/fnhum.2010.00240
Received: 25 October 2010;
Accepted: 30 December 2010;
Published online: 25 January 2011.
Copyright: © 2011 Moustafa. This is an open-access article subject to an exclusive license agreement between the authors and Frontiers Media SA, which permits unrestricted use, distribution, and reproduction in any medium, provided the original authors and source are credited.
*Correspondence: ahmedhalimo@gmail.com