Dopaminergic medication impairs feedback-based stimulus-response learning but not response selection in Parkinson's disease

Cognitive dysfunction is a feature of Parkinson's Disease (PD). Some cognitive functions are impaired by dopaminergic medications prescribed to address the movement symptoms that typify PD. Learning appears to be the cognitive function most frequently worsened by dopaminergic therapy. However, this result could reflect either impairments in learning (i.e., acquisition of associations among stimuli, responses, and outcomes) or deficits in performance based on learning (e.g., selecting responses). We sought to clarify the specific effects of dopaminergic medication on (a) stimulus-response association learning from outcome feedback and (b) response selection based on learning, in PD. We tested 28 PD patients on and/or off dopaminergic medication along with 32 healthy, age- and education-matched controls. In Session 1, participants learned to associate abstract images with specific key-press responses through trial and error via outcome feedback. In Session 2, participants provided specific responses to abstract images learned in Session 1, without feedback, precluding new feedback-based learning. By separating Sessions 1 and 2 by 24 h, we could distinguish the effect of dopaminergic medication on (a) feedback-based learning and response selection processes in Session 1 as well as on (b) response selection processes when feedback-based learning could not occur in Session 2. Accuracy achieved at the end of Session 1 were comparable across groups. PD patients on medication learned stimulus-response associations more poorly than PD patients off medication and controls. Medication did not influence decision performance in Session 2. We confirm that dopaminergic therapy impairs feedback-based learning in PD, discounting an alternative explanation that warranted consideration.


INTRODUCTION
Parkinson's disease (PD) is a common movement disorder, though cognitive abnormalities are now recognized. These nonmotor, cognitive symptoms are a leading cause of poor quality of life in PD (Schrag et al., 2000;Barone et al., 2009). Dopaminergic medications, such as L-3,4-dihydroxyphenylalanine (L-dopa) or dopamine receptor agonists, prescribed to address motor symptoms of tremor, bradykinesia, and rigidity, seem to improve some cognitive functions and to worsen others (Cools, 2006;MacDonald and Monchi, 2011). The paradoxical effects of medication on different aspects of cognition have been explained by differences in endogenous dopamine concentrations in the brain regions that underlie them.
In PD, movement abnormalities appear, and a diagnosis is confirmed, when degeneration of dopamine-producing cells of the substantia nigra (SN) is sufficient to seriously restrict dopamine supply to its efferent, the dorsal striatum (DS) (Kish et al., 1988). In contrast, dopamine-producing cells in the ventral tegmental area (VTA) are relatively spared and dopamine supply to the ventral striatum (VS), along with limbic and frontal cortices, is better preserved (Haber and Fudge, 1997). The striatum is the input structure for a collection of subcortical nuclei that are broadly implicated in movement regulation and increasingly in cognitive functions. The DS includes the bulk of the caudate nucleus and the putamen. The VS, comprising the nucleus accumbens and the most ventral portions of caudate and putamen, is considered separately from the DS because these regions have distinct dopaminergic afferents (Voorn et al., 2004;Wickens et al., 2007), vascular supplies (Feekes and Cassell, 2006), and functions (Cools, 2006;MacDonald and Monchi, 2011). As the pathophysiology would predict, dopaminergic medications substantially improve DS-mediated motor and cognitive symptoms (Cools, 2006;MacDonald and Monchi, 2011;Colzato et al., 2012). However, in PD, these medications appear to worsen cognitive operations performed by VTA-innervated brain regions presumably due to dopamine overdose of these dopamine-replete brain regions (Gotham et al., 1988;Cools et al., 2001;Cools, 2006;MacDonald and Monchi, 2011).
A survey of the literature suggests that learning, in various forms, is the cognitive function most commonly worsened by dopaminergic medication. Studies that have tested PD patients on relative to off medication have reported impairments in probabilistic associative (Torta et al., 2009;Jahanshahi et al., 2010), sequence (Feigin et al., 2003;Ghilardi et al., 2007;Seo et al., 2010;Tremblay et al., 2010), and stimulus-reward reversal learning (Swainson et al., 2000;Cools et al., 2002;Tomer et al., 2007;Graef et al., 2010;MacDonald et al., 2013a), as well as explicit abstract figure and list learning (MacDonald et al., 2013b), stimulus-stimulus facilitation , and learning from negative feedback (Frank et al., 2004). However, investigations often fail to separately assess the acquisition of associations among stimuli, responses, and outcomes, from processes of response selection that rely on these learned associations and that are used to measure new learning (McDonald and White, 1993;Jessup and O'doherty, 2011). For example, typical stimulus-response learning paradigms proceed as follows: (a) a stimulus is presented and participants decide among a set of responses, (b) feedback about the accuracy of the response is provided, through which stimulus-response associations are learned. Stimulus-response association learning is estimated by measuring the accuracy of stimulus-specific response selection and enactment. Impairment in either learning or in using learned associations to decide among responses could yield poor performance in these standard learning paradigms. Atallah et al. (2007) demonstrate this point quite elegantly. An extensive literature existed linking DS to learning associations among stimuli, responses, and rewards (Yin and Knowlton, 2006;Ashby et al., 2007). However, noting the confound outlined above, Atallah and colleagues sought to separate the acquisition of associations from performance based on this learning. In a Y-maze task using odor cues, they observed impairment in rats' abilities to consistently select a rewarded vs. unrewarded arm in animals receiving infusions of inhibitory gamma-amino butyric acid (GABA) agonist to DS compared to a saline solution during the learning phase of the experiment. At first blush, this seemed to suggest that animals receiving inhibitory infusions to DS were learning associations between odor cues and rewards more poorly. When both groups were later tested once the infusions were stopped, however, both experimental and control groups performed the selection task similarly. This demonstrated that associations were learned equally well for both experimental and control (i.e., saline-infused) groups during Session 1 and suggested that inhibition of DS impaired the animal's ability to use learned associations to perform selections reliably. To complement this interesting finding, in another experiment, they found that GABA infusions to DS at test phase resulted in impaired selection performance compared to saline infusions to DS, although both groups had previously shown identical learning of these odor-reward associations during the training phase.
Taken together, these results challenged the widely held notion that DS mediates learning and instead suggested a more specific role for DS in performance based on learning.
We note that the literature implicating dopaminergic medication in learning impairment in PD similarly warrants reconsideration. The specific aim of the present study was therefore to investigate the effect of dopaminergic medication in PD on stimulus-response learning versus response performance processes. In Session 1, PD patients and healthy age-and educationmatched controls first learned to associate abstract images and specific key-press responses through outcome feedback. Session 1 constituted a typical stimulus-response learning study in which processes of stimulus-response association learning and response selection performance were confounded. In Session 2, participants were asked to make the specific key-press responses to abstract images that they had learned in Session 1. However, no feedback regarding the accuracy of responses was provided in Session 2, precluding new feedback-based learning. Figures 1A,B illustrate how trial structures in Sessions 1 and 2 differ from one another in only one regard-the provision of outcome feedback.
Half of the PD patients completed Session 1 on medication, whereas the other half learned stimulus-response associations via feedback off medication. Similarly, half of the PD patients performed Session 2 on and the other half off dopaminergic medication. Because performance in Session 2 depended upon learning in Session 1, and we expected that medication status could influence learning in Session 1, we made two design choices to mitigate carry-over effects. First, we implemented a learning criterion in Session 1 to ensure that all participants achieved a similar level of stimulus-response association strength, without establishing overlearned relations. Second, we ensured that each on and off group in Session 2 was composed of an equal number of participants who had learned stimulus-response associations in Session 1 on compared to off medication. Please see Figure 2 to understand the design of this experiment.

PARTICIPANTS
Thirty-three PD patients and 36 age-and education-matched healthy controls participated in the experiment. All PD patients were previously diagnosed by a licensed neurologist, had no coexisting diagnosis of dementia or another neurological or psychiatric disease, and met core assessment criteria for surgical interventional therapy and the UK Brain Bank for the diagnosis of idiopathic PD (Hughes et al., 1992). No PD patients were treated with deep brain stimulation or other neurosurgeries. Control participants were free of neurological and psychiatric illnesses. All PD patients and no controls were treated with dopaminergic therapy. This study was approved by the Health Sciences Research Ethics Board of the University of Western Ontario and the Ethics Review Board of the Sudbury Regional Hospital. All participants provided informed written consent to the approved protocol before beginning the experiment, according to the Declaration of Helsinki (World Medical Association, 2013).

FIGURE 1 | Example of a single trial in Sessions 1 and 2. (A)
Session 1: PD patients and aged-matched controls learned to associate 9 abstract images with a key-press response. A trial proceeded as follows: (i) a cross appeared in the center of a computer screen for 500 ms; (ii) a blank screen was presented for 500 ms; (iii) an abstract image was presented in the center of the computer screen until the participant entered his or her key-press response using the "1," "2," or "3" numeric keys; (iv) the image disappeared before feedback, either the word "Correct" or "Incorrect," was presented for 1000 ms in the center of the screen; (v) a blank screen was presented for 500 ms before the next trial began. An abstract image was presented and participants provided his or her key-press response before feedback was presented. (B) Session 2: Stimulus-specific key-press responses for stimuli learned in Session 1 were performed in the absence of feedback a day later. The parameters for each trial in Session 2 were otherwise identical to those in Session 1. were excluded from participating. Five PD patients and four control participants did not reach a pre-set learning criterion (see Section Design and Procedure) in Session 1. They were not invited to participate in Session 2 and therefore their data were not included in our analyses. Consequently, we included 28 PD patients (21 males) and 32 control participants (11 males) in our analyses.
The motor sub-scale of the Unified Parkinson's Disease Rating Scale (UPDRS) was scored by a neurologist with sub-specialty training in movement disorders (Penny A. MacDonald) to assess the presence and severity of disease for all PD patients both on and off dopaminergic medication. Control participants were also screened to rule out undiagnosed neurological illness, PD in particular. In addition, all participants completed a battery of standardized cognitive and affective tests to rule out significant cognitive impairments, depression, or anxiety. Table 1 presents mean group demographics, as well as affective and cognitive screening scores for all patients along with these measures for their matched controls. UPDRS motor subscale scores on and off usual dopaminergic medication, daily doses of dopaminergic therapy in terms of L-dopa equivalents, and mean duration of PD are also presented in Table 1. Calculation of daily L-dopa equivalent dose for each patient was based on the theoretical equivalence to L-dopa (Evans et al., 2004) as follows: L-dopa dose + L-dopa × 1/3 if on entacapone + bromocriptine (mg) × 67 + ropinerole (mg) × 20 + pergolide (mg) × 100 + apomorphine (mg) × 8. There were no significant demographic differences between PD patients and controls (Table 1). Screening cognitive measures confirmed that no participants suffered significant cognitive impairment (Table 1).

APPARATUS
The experiment was conducted on a 14.0 widescreen laptop (Lenovo T420) running at a resolution of 1600 × 900 on the Windows 7 operating system. The screen was placed at a distance of approximately 50 cm in front of participants and angled for optimal viewing.

STIMULI
The stimuli used during the experiment consisted of abstract images that were computer-generated with GroBoto (Braid Art Labs, Colorado Springs, USA). Nine abstract images were used in the experiment (Figure 3).

DESIGN AND PROCEDURE
All patients with PD participated in two experimental sessions conducted over consecutive days, as did their age-and educationmatched healthy controls. Half of the PD patients performed Session 1 on and the other half off dopaminergic medication. This assignment was random. Similarly, half of the PD patients performed Session 2 on and the other half off dopaminergic medication. For Session 2, half of the PD patients from the Session 1 ON condition were randomly assigned to the Session 2 ON condition whereas the other half were randomly assigned to the Session 2 OFF condition. Analogously, half of the PD patients from the Session 1 OFF condition were randomly assigned to the Session 2 ON condition whereas the other half were randomly assigned to Session 2 OFF condition. This is illustrated in Figure 2. PD patients took their dopaminergic medication as prescribed by their treating neurologist during ON testing sessions, but abstained from taking L-dopa/carbidopa and entacapone for 12-18 h, and dopamine agonists, such as pramipexole, ropinirole, or pergolide, as well as amantadine, rasagiline, and selegiline for 16-24 h before beginning OFF testing sessions. Although control participants did not take dopaminergic medication during any session, their data were analyzed to correspond to the medication order of the PD patient to whom they were matched. Matching was performed at time of testing, prior to data analysis. This controlled for possible order, fatigue, and practice effects.
Before beginning Session 1, participants received 20 practice trials with different images from those employed during the main experimental sessions. In Session 1, participants performed a task during which they learned to associate abstract images with one of three key-press responses. On each trial, an abstract image appeared and remained in the center of the computer screen until the participant responded with a key press. Outcome feedback (i.e., "Correct" or "Incorrect") was provided after every response and in this way, participants learned to associate each of the abstract images with the appropriate key-press response through trial and error.
All trials proceeded as follows: (i) a cross appeared in the center of a computer screen for 500 ms; (ii) a blank screen occurred for 500 ms; (iii) an abstract image was presented until the participant entered a key-press response, either "1," "2," or "3" keys; (iv) feedback, either "Correct" or "Incorrect," was presented for 1000 ms; (v) a blank screen for 500 ms separated trials ( Figure 1A).
Trials were organized into blocks. Each block consisted of 18 trials, with each of the nine abstract images occurring twice in random order. Three images were assigned to each of the "1," "2," and "3" numeric keys at the top of the keyboard and participants pressed these keys with their index, middle, and ring fingers, respectively. After each block, participants were provided with a percentage score, summarizing their learning performance. A minimum learning criterion of 74% on two successive blocks was required to complete Session 1. This ensured that similar learning was achieved by all participants and ensured that over-learning of associations did not occur.
Session 2 occurred the day after Session 1, approximately 24 h later for each participant. In Session 2, participants performed two blocks of 18 trials, in which the same 9 images studied during Session 1 were presented in random order, twice per block. Participants decided among and selected the key-press response that they had learned for each image in Session 1. No outcome feedback was provided to preclude new feedback-based learning. The parameters for each trial in Session 2 were otherwise identical to those in Session 1 (Figure 1B).
We expected that dopaminergic medication might have an effect on learning in Session 1. Performance in Session 2 depended on how well stimulus-response associations were learned in Session 1. To diminish any carry-over effects from Session 1, we (i) imposed a pre-set learning criterion of 74% in Session 1 and (ii) included an equal number of participants who learned on and an equal number of participants who learned off medication in Session 1, in each of the ON and OFF conditions in Session 2 (Figure 2).

BEHAVIORAL MEASURES
Efficiency of encoding stimulus-response associations across Session 1 was estimated by a mean improvement score that describes the change in percent correct performance per block across Session 1. This score was calculated as follows: In Session 2, decision making based on previously-learned associations was measured with an adjusted-savings score, calculated as follows: average accuracy in Session 2 ÷ accuracy in the last block of Session 1. Anticipating that Group (PD patients vs. Controls) and Medication status (ON vs. OFF) might affect learning, we implemented several measures to reduce the variability in the degree to which stimulus-response associations were learned by all (i.e., compelling all participants to reach a criterion of 74% and ensuring that the groups in Session 2 were entirely balanced with respect to learning conditions in Session 1). In the event that comparable stimulus-response learning was not achieved by all groups, however, the use of an adjusted-savings score was intended to correct for subtle variability among participants in stimulus-response association learning. This allowed us to examine how PD and dopaminergic medication affected recall and enactment of previously-learned stimulus-specific responses independently of how these factors affected stimulus-response association learning per se. Higher improvement scores indicated more efficient learning, and higher adjusted-savings scores indicated superior retention of learned associations and decisionmaking performance based on prior learning. Separate One-Way analyses of variance (ANOVAs) were conducted on stimulusresponse association learning estimates and measures of response selection performance between Groups (PD patients vs. Controls) and across Medication status (ON vs. OFF).

RESULTS
Main findings for Sessions 1 and 2 are presented in Figures 4A,B, respectively. Other behavioral data for Sessions 1 and 2 are presented in Table 2.

SESSION 1: LEARNING STIMULUS-RESPONSE ASSOCIATIONS VIA OUTCOME FEEDBACK
We performed a One-Way ANOVA on improvement scores, with Group (PD patients vs. Control) as the between-subject factor, for ON and OFF medication conditions separately. PD patients learned significantly more poorly than matched control partic- To rule out the possibility that PD patients and controls simply displayed differences in performance on the initial block with similar learning efficiency, we performed One-Way ANOVAs on the first block scores. PD patients did not differ from their matched-control participants for either ON [F (1,29)  Analogously, we performed a One-Way ANOVA on final block scores to ensure that PD patients and controls reached a comparable level of learning performance in Session 1. PD patients did not differ from their matched-control participants for either ON [F (1,29)   Despite measures taken to mitigate the influence of learning in Session 1 on performance in Session 2, there remained the possibility for carry-over effects between Sessions. We therefore performed a 2 × 2 ANOVA on adjusted-savings scores with Group (PD patients vs. Control) and Medication (ON vs. OFF) as the between-subject factors, covaried with Medication during Session 1 (ON vs. OFF). There were no significant main effects of either Group [F (1,56)  These results suggest that the measures taken to reduce the influence of Session 1 learning on Session 2 performance were successful in ensuring that despite any effect of group or medication in Session 1, all participants achieved a similar, pre-asymptotic level of stimulusresponse association learning.

DISCUSSION
In Session 1, participants learned to associate stimuli and responses through outcome feedback. Learning was evidenced when participants correctly recalled the associations, selected, and enacted stimulus-specific responses. In Session 2, feedback was omitted, eliminating the possibility of further feedback-based learning Participants expressed what they had previously learned by enacting the stimulus-specific responses from Session 1 (see Figures 1A,B). Off dopaminergic medication, PD patients performed Sessions 1 and 2 normally compared to age-matched controls. In contrast, PD patients on dopaminergic medication learned stimulus-response relations in Session 1 more poorly than controls. Bolstering this finding further, PD patients on dopaminergic medication achieved the learning criterion in Session 1 less efficiently than PD patients off medication. In Session 2, PD patients on and off dopaminergic therapy performed equivalently. We interpret these results as evidence that PD patients learn stimulus-response associations normally at baseline and that dopaminergic medication impairs feedback-based learning but not recall of these associations, or response selection and enactment.
Stimulus-response learning paradigms often proceed in a single session as follows: (a) a stimulus is presented and participants decide among a set of responses, (b) feedback about the accuracy of the response is provided, through which stimulus-response associations are learned. Stimulus-response association learning is indexed by the accuracy of recalling, selecting, and enacting stimulus-specific responses. Impairment in either learning per se or response selection/enactment based on what has been learned could yield poor performance in this paradigm (Atallah et al., 2007). These data do not suggest that dopaminergic therapy actually impairs performance of stimulus-specific response selection processes, giving the appearance of deficient stimulus-response association learning (Atallah et al., 2007). This confound was addressed by employing a rigorous methodology in which we: (1) tested performance in a session where feedback-based learning was possible relative to one in which it was not, (2) equated trial structures in Sessions 1 and 2 so that they differed only in terms of the provision of outcome feedback, (3) prevented overlearning/ceiling performance by using a 74% accuracy criterion in Session 1 and introduced a 24 h delay between Sessions 1 and 2 to produce comparable average performance in each session in terms of accuracy (see Table 2), (4) mitigated carry-over effects from Session 1 to Session 2 by equating the learning achieved by all participants in Session 1 and including an equal number of PD patients in the ON and OFF conditions in Session 2 from ON and OFF conditions in Session 1 (see Figure 2), and (5) matched PD patients to healthy age-matched controls, analyzing control data to correspond to the ON-OFF order of their PD patient, eliminating the possibility that our findings owed simply to complex order effects.
These results support findings from previous studies in PD that report a medication-associated impairment in various forms of learning (Swainson et al., 2000;Cools et al., 2002;Feigin et al., 2003;Frank et al., 2004;Ghilardi et al., 2007;Tomer et al., 2007;Torta et al., 2009;Graef et al., 2010;Jahanshahi et al., 2010;Seo et al., 2010;Tremblay et al., 2010;but see Shiner et al., 2012Smittenaar et al., 2012). Whereas motor symptoms and some cognitive functions are improved by dopaminergic medication, other cognitive processes are actually worsened (Cools, 2006;MacDonald and Monchi, 2011). These differential effects of dopaminergic therapy on individual cognitive functions likely owe to differences in endogenous dopamine in the brain regions that mediate them (Gotham et al., 1988;Cools et al., 2001;Cools, 2006;MacDonald and Monchi, 2011). At baseline, DS appears to be severely dopamine-depleted whereas VTA-innervated brain regions, including VS, limbic, and prefrontal cortices are relatively dopamine replete. Dopaminergic medication rectifies this dopamine deficit and ameliorates DSmediated motor and cognitive functions (Feigin et al., 2001;Cools et al., 2003;Asanuma et al., 2006;Wu et al., 2009;Colzato et al., 2012). This appears to be at the expense of functions performed by VTA-innervated brain regions, which are worsened by medication, especially at earlier stages of PD (Cools, 2006;MacDonald and Monchi, 2011). Consistent with the dopamine overdose hypothesis and the findings presented here, learning has been ascribed to both VS (O'doherty, 2004;Shohamy et al., 2004Shohamy et al., , 2006Reiss et al., 2005;Cools et al., 2007;Ghilardi et al., 2007;Seo et al., 2010;Tremblay et al., 2010; and limbic cortical regions (McDonald and White, 1993;Rodriguez, 2009). Indeed, in a recent study, we found that VS activity correlated with feedback-based stimulusresponse learning in a procedure that was virtually identical to the one presented here (Hiebert et al., 2014). Finally, and further supporting our interpretation of our findings, even in healthy volunteers, administration of dopaminergic therapy has been shown to worsen learning (Mehta et al., 2001;Breitenstein et al., 2006).
Cognitive dysfunction is now an undisputed non-motor symptom of PD that leads to significant impairment in quality of life (Schrag et al., 2000;Barone et al., 2009). Whereas dopaminergic medication is primarily titrated in response to motor symptoms, it is increasingly understood that some cognitive impairments arise due to the effects of this therapy (Cools, 2006;MacDonald and Monchi, 2011). Clarifying the specific cognitive functions that are helped versus those that are hindered by dopaminergic medication can inform treatment in PD, allowing clinicians to consider cognitive as well as motor complaints in titrating therapy.