Habitual responding for alcohol depends upon both AMPA and D2 receptor signaling in the dorsolateral striatum

Introduction

Growing evidence from studies using both natural and drug rewards suggests that control of reward-seeking behaviors changes over the course of extended training (Adams, 1982; Zapata et al., 2010; Corbit et al., 2012). Early in training, performance is flexible and goal-directed whereas following more extended practice, performance becomes more rigid or habitual. This shift has been demonstrated to occur sooner when a drug serves as the outcome of instrumental responding or when animals have been independently exposed to drugs, including alcohol, amphetamine, and cocaine (Nelson and Killcross, 2006; Corbit et al., 2012, 2014; LeBlanc et al., 2013). In the context of drug use and exposure, this accelerated shift may reflect some of the early stages of loss of control over reward-seeking that is characteristic of substance use disorders and thus, there has been increasing interest in defining the mechanisms of striatal-based learning as well as any neural adaptations that may promote this shift in behavioral control.

The outcome devaluation task can be used to differentiate between goal-directed and habitual performance. In this task, animals are pre-fed their earned outcome, for example, alcohol to satiety, thus decreasing its value relative to a control condition where they are pre-fed a different outcome (e.g., sucrose). Performance of the response that previously earned alcohol is then tested in extinction. If responding tracks the modified value of the outcome, performance is said to be goal-directed. If responding is insensitive to changes in outcome value, this provides evidence for habitual performance.

We have recently demonstrated that following relatively limited (2 weeks) alcohol self-administration; performance is sensitive to devaluation but that following more extended training (8 weeks) performance is no longer sensitive to devaluation. Importantly, animals receiving equivalent training with sucrose reward remained sensitive to outcome devaluation, thus showing evidence of goal-directed responding, unless they were drinking alcohol in the home cage during the training period. These findings provide evidence that alcohol seeking more readily becomes habitual after extended training and/or prolonged exposure to the drug (Corbit et al., 2012).

Notably, findings, from our own and others’ work, support an important role for the dorsal striatum (DS) in drug and alcohol self-administration (Vanderschuren and Everitt, 2005; Jeanblanc et al., 2006; Wang et al., 2007; Belin and Everitt, 2008; Corbit et al., 2012) and indicate that behavioral control shifts from medial to lateral across the course of training. For example, inactivation of the dorsolateral striatum (DLS) has little effect when tests are performed following limited training when performance is goal directed. In contrast, inactivation of the DLS following extended training blocks the expression of habitual alcohol seeking restoring goal-directed control (Corbit et al., 2012). Here we sought to further explore the neuropharmacological basis for this effect.

The striatum is comprised primarily of GABAergic medium spiny projections neurons. These neurons require excitatory glutamatergic input (from cortex, thalamus, etc.) to fire action potentials (Wilson et al., 1983; Wickens et al., 2003). A recent study found that habitual performance increases activity in both the DLS and associated cortical regions (somatosensory and motor cortices), as measured by c-Fos immunohistochemistry (Furlong et al., 2014). Additionally, there are reports that activation of sensorimotor cortex increases following extended training of a motor skill (Karni et al., 1995; Floyer-Lea and Matthews, 2005). Thus the shift in control of responding from DMS to DLS across extended training likely corresponds to increased activation of at least some subsets of medium spiny neurons by glutamatergic afferents and blockade of glutamatergic input to the DLS should attenuate habitual responding for alcohol. Therefore, we examined the role of glutamatergic inputs to the DLS in the expression of habitual alcohol seeking by testing the effects of infusions of the AMPA receptor antagonist, NBQX, prior to devaluation.

Neuronal activity in the DLS is also shaped by dopaminergic inputs from the substantia nigra pars compacta. Dopamine is considered a critical contributor both to learning and expression of instrumental responding (Salamone et al., 1997; Schultz, 2007; Wickens et al., 2007) as well as a mediator of drug seeking characterized as habitual or compulsive (Volkow et al., 2006; Belin and Everitt, 2008). Direct evidence for a role of dopamine in habit learning comes from demonstrations that 6-OHDA lesions of the DA input to the DLS block the development of habitual responding in that well-trained performance remained sensitive to devaluation (Faure et al., 2005). Hence dopamine release may be a critical component in habitual alcohol-seeking behavior mediated by the DLS. Previous studies have generally tested the effects of lesions (Faure et al., 2005) or non-selective dopamine antagonists (e.g., α-flupenthixol; Dickinson et al., 2000; Belin and Everitt, 2008) and so the specific role of different classes of dopamine receptors in habit learning is not known.

Dopamine D2 receptor expression shows a gradient within the DS with these receptors being more prominent in more lateral striatal regions. In contrast, D1 receptors are expressed relatively evenly across the DS (Joyce et al., 1985; Yin et al., 2009). Evidence suggests performance of a newly-learned response is dependent on both D1- and D2-expressing neurons of the direct and indirect pathways (Choi et al., 2005; Yin et al., 2009). Following extended training, performance becomes independent of D1 receptors, while D2 activation remains important for normal performance (Yin et al., 2009). Thus, we hypothesized that habitual instrumental performance following extended training may similarly rely on D2 receptor activation. While Yin et al. (2009) examined the effects of systemic administration of dopamine antagonists, given the demonstrated role of the DLS is habitual performance, we tested the effects of intra-DLS infusions of the selective D2-receptor antagonist, raclopride, prior to devaluation testing in rats with extensive alcohol self-administration training to examine whether D2 receptors specifically within the DLS are necessary for the expression of habitual alcohol-seeking.

Materials and Methods

Subjects and Apparatus

Thirty-four naïve male Long-Evans rats (Harlan, Indianapolis, IN) weighing approximately 350 g at the beginning of the experiment were singly-housed and had free access to food and water in the home cage. All procedures were approved by the Institutional Animal Care and Use Committee of the EGCRC and conform to the standards stipulated by the National Institutes of Health Office of Laboratory Animal Welfare. Training and testing took place in Med Associates (East Fairfield, VT) operant chambers described previously (Corbit et al., 2012).

Alcohol Acclimation in the Home Cage

Because of the aversive taste of alcohol rats are initially relatively reluctant to consume unsweetened alcohol voluntarily. To familiarize them with alcohol, rats initially were given free access to 10% ethanol (10E; v/v) in filtered water in the home cage, for 24 h a day for 14 days, followed by 14 days of 1 h access to 10E at the time that training would subsequently occur. Water was always available in a separate bottle fixed to the home cage. Rats were weighed daily and consumption recorded.

Instrumental Training

Rats underwent a single 30 min magazine training session wherein 10E was delivered under a random time-60 s schedule. Rats were next trained to make a lever-press response to deliver small aliquots (0.1 ml) of 10E in 60 min sessions. The first 2 days of training were under a continuous reinforcement schedule; reinforcement was then shifted to a random ratio (RR) two schedule for 3 days, followed by a RR3 schedule. Animals failing to respond at levels sufficient to achieve alcohol intake of at least 0.3 g/kg for 5 out of 7 days a week were excluded from the study (9 animals excluded according to this criterion leaving 25 animals which were subsequently assigned to either the NBQX (N = 13) or raclopride (N = 12) experiment). The reward receptacle was examined at the end of each session to ensure that the earned rewards were consumed; after the initial 3 training days this was always the case.

Surgery

Surgery was performed after approximately 7 weeks of training. Stereotaxic surgery was conducted under isoflorane anesthesia to implant 26 gauge guide cannulae (Plastics One, Roanoke, VA) targeting the DLS (AP: +1.2 mm, ML: +/−3.4 mm, DV: –1.0 mm; coordinates relative to bregma). Guide cannulae tips were positioned 3 mm dorsal to the intended infusion site; thus, final DV coordinates for the infuser tips was −4.0 ventral to dura. Animals were given a week to recover from surgery and resumed training for 1 week prior to devaluation testing.

Devaluation Testing

For each test, rats were divided into two groups, devalued and non-devalued. For the devalued condition, rats were given 45-min of free access to 10E in the home cage. These parameters typically result in average consumption of 5 ml of 10E and corresponding alcohol levels of 0.85 g/kgs (Corbit et al., 2012). For the non-devalued condition, rats were given 45-min free access to 1% sucrose (wt/vol; 1S; this concentration was chosen as it produces consumption volumes similar to those found with 10E; Corbit et al., 2012). A consumption criterion of 3 ml was required for an animal’s data to be included. Immediately following home-cage pre-feeding, rats received an infusion (as described below) and were tested for lever-press responding in a 10-min extinction test. Following this first test, rats received 2 days of retraining and were tested again such that rats that had received the devaluation treatment now received the non-devalued treatment and vice versa. Additional pairs of tests were completed in the same fashion to allow testing of each rat under each dose.

The Role of DLS AMPA Receptors in Habitual Alcohol Seeking

Each animal underwent a total of six tests to allow testing in both the devalued and non-devalued conditions under saline and two doses of the AMPA-receptor antagonist (order counterbalanced for dose and devaluation condition). The AMPA antagonist used was NBQX (Sigma, St Louis, MO) and 0.3 μl per hemisphere of two doses (0.3 and 1.0 μg/μl) were delivered via infusion cannulae (33 gauge; Plastics One) extending 3 mm below the guide cannula tip at a rate of 0.3 μl/min 10 min prior to test. The infusion cannulae were left in place for at least 1 min after the completion of the infusion to allow for diffusion of the drug away from the tip.

Consumption Tests of the Efficacy of the Devaluation Treatment

It is possible that after 8 weeks of self-administration training the satiety treatment may lose its efficacy due to development of tolerance or other factors. To address this concern, on separate days, rats were given 45-min of free access to either 10E or 1% sucrose in the home cage. These bottles were removed and 10 min later (approximately the time the extinction test would have occurred following infusions) animals were presented with a fresh bottle of 10E. Consumption was recorded. We predicted that if the devaluation treatment was effective, animals would consume less alcohol following previous consumption of alcohol (devalued condition) compared to previous consumption of sucrose (non-devalued condition).

Effects of NBQX on Consumption

To address whether NBQX might have any non-specific effects on alcohol consumption we examined the effects of NBQX treatment on homecage drinking. Rats received an infusion of 0.3 μg/μl NBQX and were returned to their homecage and a bottle of alcohol was provided for 1 h. Bottles were weighed at 30 and 60 min.

The Role of DLS D2 Receptors in the Expression of Habitual Alcohol Seeking

Separate animals underwent acclimation training and surgery as described above. Each animal underwent a total of six tests to allow testing in both the devalued and non-devalued conditions under saline and two doses of the D2 receptor antagonist raclopride (0.2 and 1.0 μg/μl; order of devaluation condition and dose was counterbalanced). Saline or two doses of raclopride (0.3 μl; Sigma, St Louis, MO) were delivered via infusion cannulae 10 min to devaluation testing as described above.

Histology

Coronal sections (50 μm) of formalin-fixed tissue were sliced, mounted, and stained with Nissl stain, to allow verification of cannulae placement.

Data Analysis

Data were analyzed in repeated measures analysis of variance (ANOVA) as appropriate with the within-subjects factors of devaluation (devalued vs. non-devalued) and drug dose.

Results

The Role of DLS AMPA Receptors in Habitual Alcohol Seeking

Histology

Cannula placements for animals included in the behavioral analyses for the two experiments were similar and are summarized in Figure 1. Two rats were excluded from the NBQX experiment for misplaced cannulae leaving 11 rats in the final analyses.

FIGURE 1

Figure 1. Cannula placements within the DLS. Schematic representation of cannulae placements in the DLS for animals included in the NBQX (open circles) and raclopride (black circles) experiments (templates adapted from Paxinos and Watson, 1998). Numbers indicate the distance from bregma in anterior–posterior plane. N = 11 and 12 for NBQX and raclopride experiments, respectively.

Training

The average consumption during the 1 h access in the home cages was 2.65 ml (+/−SEM; 0.35 ml) which produced an average alcohol level of 0.41 g/kg (+/−0.055). Average responding, outcomes earned and g/kg alcohol levels across the 8 weeks of training are presented in Figure 2. Averaging across the last 3 training days before the first test the rats made 121 (+/−12.1) active lever presses, 2.1 (+/−0.7) inactive lever presses, earned 45 (+/−4.3) outcomes producing an average alcohol level of 0.7 (+/−0.07) g/kg.

FIGURE 2

Figure 2. Training data. (A) Mean (+/−SEM) lever presses, (B) outcomes earned, and (C) g/kg alcohol across weeks of instrumental training prior to devaluation testing.

Devaluation testing

Based on previous work with this paradigm we predicted that alcohol self-administration after 8 weeks of training would be habitual and insensitive to devaluation of the alcohol outcome. We tested whether antagonism of AMPA receptors within the DLS, thus blocking fast excitatory transmission to this region, would block the expression of the acquired habit and restore goal directed performance. Data from these tests are presented in Figure 3A. An ANOVA examining sensitivity to devaluation after infusions of saline or NBQX demonstrated a significant effect of devaluation [F_{(1, 10)} = 5.4, p < 0.05], of drug [F_{(2, 20)} = 9.2, p < 0.01] and an interaction between these factors [F_(2,20) = 4.5, p < 0.05]. Simple effects analyses revealed that following an infusion of saline rats were not sensitive to devaluation of alcohol and responded similarly under the devalued and non-devalued conditions [F_(1,10) = 0.3, p > 0.05]. In contrast, infusion of 0.3 μg/μl of NBQX restored sensitivity to devaluation, such that rats decreased responding following devaluation of alcohol compared to the non-devalued condition [F_(1,10) = 9.0, p < 0.05]. Infusion of a higher dose of NBQX (1.0 μg/μl) produced a marginal devaluation effect [F_(1,10) = 4.6, p = 0.057] but this dose also decreased responding overall which may have contributed to this effect. To examine a potential general effect on motor behavior following infusions particularly in the devalued condition where there is potential for an interaction with the pharmacological effects of the alcohol consumed, we examined magazine entries during the tests. These data are presented in Figure 3B. Overall, the magazine response showed the same pattern as the lever-press response with a significant effect of devaluation [F_{(1, 10)} = 6.2, p < 0.05], of drug [F_{(2, 20)} = 10.2, p < 0.01] and an interaction between these factors [F_(2,20) = 3.8, p < 0.05]. Sensitivity to devaluation was only observed following the low dose of NBQX [F_(1,10) = 12.6, p < 0.01]. Following the high dose of NBQX responding did not differ significantly between devalued and non-devalued conditions [F_(1,10) = 1.7, p > 0.05] and responding in the devalued condition was numerically higher than responding following devaluation and the lower dose of NBQX which would not be anticipated if NBQX was producing a deficit because of an interaction with the pharmacological effects of alcohol consumed in the devaluation tests.

FIGURE 3

Figure 3. Effects of NBQX infusion into the DLS on sensitivity to outcome devaluation. (A) Following 8 weeks of ethanol self-administration lever-press responding was insensitive to devaluation following saline infusions. Infusion of 0.3 μg/μl NBQX restored sensitivity to devaluation. Infusion of the higher dose (1.0 μg/μl) produced a marginal devaluation effect but overall responding was also lower. * indicates p < 0.05. (B) Mean magazine entries during the devaluation tests which show a similar pattern to lever-press performance.

To confirm that the devaluation treatment itself was effective following extended alcohol exposure we measured consumption of ethanol following pre-feeding of ethanol (devalued) or sucrose (non-devalued). As shown in Figure 4A, we found a significant devaluation [F_{(1, 10)} = 8.4, p < 0.05] demonstrating that the specific satiety treatment itself was effective, but that this change in outcome value did not direct lever-press performance under control conditions. We also examined whether our most effective dose of NBQX might have any non-specific effects on alcohol consumption by examining the effects of NBQX infusion prior to 1 h access to alcohol in the home cage. Consumption following infusion of either saline or 0.3 μg/μl NBQX is shown in Figure 4B and was equivalent [F_(1,10) = 1.9, p > 0.05] indicating that NBQX treatment itself did not somehow make alcohol aversive or alter satiety or willingness to consume alcohol. Together, these findings suggest that the restoration of sensitivity to devaluation following NBQX was not secondary to altered satiety, intoxication or non-specific motor effects.

FIGURE 4

Figure 4. Effects of NBQX on sensitivity of consumption to outcome devaluation and on home cage drinking. (A) When consumption of ethanol following pre-feeding of ethanol (devalued) or sucrose (non-devalued) was measured, a significant devaluation was detected demonstrating that the specific satiety treatment itself was effective but that this change in outcome value was not translated into lever-press performance under control conditions (see saline condition in Figure 3). (B) Ethanol consumption in the home cage following infusion of either saline or 0.3 μg/μl NBQX was equivalent indicating that NBQX treatment itself did not somehow change the rats’ willingness to consume ethanol. * indicates p < 0.05.

The Role of DLS D2 Receptors in the Expression of Habitual Alcohol Seeking

Histology

All cannulae were placed within the DLS (N = 12).

Training

The average consumption during the 1 h access in the home cages was 3.1 ml (+/−SEM; 0.48 ml) which produced an average alcohol level of 0.50 g/kg (+/−0.08). Average responding, outcomes earned and g/kg alcohol levels across the 8 weeks of training are presented in Figure 2. Averaging across the last 3 training days before the first test the rats made 101 (+/−12.3) active lever presses, 1.4 (+/−1.1) inactive lever presses, earned 37 (+/−43.9) outcomes producing an average alcohol level of 0.58 (+/−0.06) g/kg.

Devaluation testing

Based on the previous demonstration that performance of a well-learned skill relies on D2 receptor activity in the DLS (Yin et al., 2009), we predicted that alcohol self-administration after 8 weeks of training would be habitual and that antagonism of D2 receptors within the DLS, would block the expression of the acquired habit and restore goal-directed performance. Data from the devaluation tests are shown in Figure 5A. An ANOVA examining sensitivity to devaluation after infusions of saline or raclopride demonstrated that overall there was no effect of devaluation [F_{(1, 11)} = 1.9, p > 0.05], but that there was a significant effect of drug [F_{(2, 22)} = 5.2, p < 0.05] and an interaction between these factors [F_(2,22) = 11.9, p < 0.01]. Simple effects analyses revealed that following an infusion of saline rats were not sensitive to devaluation of alcohol and responded similarly under the devalued and non-devalued conditions [F_(1,11) = 0.3, p > 0.05]. In contrast, infusion of 0.2 μg/μl of raclopride restored sensitivity to devaluation and rats decreased responding following devaluation of alcohol compared to the non-devalued condition [F_(1,11) = 23.8, p < 0.01]. Infusion of a higher dose of raclopride (1.0 μg/μl) did not produce a significant devaluation effect [F_(1,11) = 0.3, p > 0.05] but this dose also decreased responding in general. As above, to examine a potential motor effect particularly in the devalued condition where there is potential for an interaction between raclopride and the pharmacological effects of the alcohol consumed, we examined magazine entries during the tests (see Figure 5B). There was an effect of devaluation [F_{(1, 11)} = 10.3, p < 0.05], a significant effect of drug [F_{(2, 22)} = 3.5, p < 0.05] and an interaction between these factors [F_(2,22) = 9.7, p < 0.01]. Simple effects analyses indicated a significant effect of devaluation only after the low dose of raclopride [F_(1,11) = 100.8, p < 0.01]. Following the high dose of raclopride, responding did not differ significantly between devalued and non-devalued conditions [F_(1,11) = 0.4, p > 0.05] and responding in the devalued condition was numerically higher than responding following devaluation and the lower dose which would not be anticipated if raclopride was producing a deficit because of an interaction with the pharmacological effects of alcohol consumed in the devaluation tests.

FIGURE 5

Figure 5. Effects of raclopride infusion into the DLS on sensitivity to outcome devaluation. (A) Following 8 weeks of ethanol self-administration lever-press responding was insensitive to devaluation following saline infusions. Infusion of the 0.2 μg/μl dose of raclopride restored sensitivity to devaluation. After infusion of the higher dose (1.0 μg/μl) sensitivity to devaluation was again lost. * indicates p < 0.05. (B) Mean magazine entries during the devaluation tests.

It is unclear whether the effects of the higher dose are simply due to greater inhibition of D2 receptors or additional recruitment of different classes of D2 receptors. That is, in addition to affecting postsynaptic D2 receptors on medium spiny neurons of the indirect pathway, raclopride could also inhibit autoreceptors and interfere with feedback mechanisms and dopamine levels (Anzalone et al., 2012). The role of these different classes of D2 receptors in habit learning and expression will be an interesting area for future study.

Discussion

The data presented here provide further evidence that with extended training, performance of an alcohol-seeking response becomes habitual and no longer tracks the current value of the outcome it produces. Further, we demonstrate that this habitual performance relies on both AMPA and D2 receptors within the DLS as treatment with either the AMPA-receptor antagonist NBQX or the selective D2-receptor antagonist raclopride suppressed expression of habitual performance thus restoring goal-directed control.

The observation of habitual responding for alcohol can not easily be accounted for by insensitivity to the devaluation manipulation after prolonged alcohol exposure, as rats selectively decrease consumption of alcohol following a previous opportunity to consume alcohol, compared to following consumption of sucrose. Thus, factors such as tolerance which may change across the course of extended alcohol exposure do not readily explain the insensitivity of lever-press performance to devaluation. Rather, it appears that animals no longer flexibly utilize the current value of the earned alcohol to control alcohol-seeking behaviors.

The effects of NBQX confirm that glutamatergic inputs to the DLS are important for driving habitual performance. This is consistent with a recent report that habitual performance increases activity in both the DLS and associated cortical regions (somatosensory and motor cortices) measured by c-Fos immunohistochemistry (Furlong et al., 2014) and reports that activation of sensorimotor cortex increases following extended training of a motor skill (Karni et al., 1995; Floyer-Lea and Matthews, 2005). Thus glutamatergic inputs from cortex likely contribute to habitual performance and these were blocked by NBQX. Similarly, the effects of raclopride suggest that dopaminergic modulation of DLS activity is also important for the expression of habitual responding. One caveat to this conclusion is that it is possible that the raclopride treatment interacted with the pharmacological effects of alcohol in the devaluation tests to exacerbate any intoxicating or motor effects of alcohol. While we did not assess the effects of raclopride on alcohol consumption in the current study, previous work has shown that raclopride administered to the nucleus accumbens core has little effect on ethanol consumption at doses that do impair an alcohol-seeking response (Czachowski et al., 2001) and since the tests of habitual performance were conducted in extinction, without alcohol present, we believe any effects of raclopride related to consumption are unlikely to account for the observed effects. Indeed, magazine approach increased in the devalued condition after the higher dose of raclopride and thus the restoration of a devaluation effect is difficult to explain in terms of non-specific effects of the drug on intoxication or motor performance.

The ability of the D2-receptor antagonist raclopride to block habitual responding is however consistent with the findings of Yin et al. (2009) who demonstrated that systemic administration of a D2 antagonist disrupted performance of a motor skill after extended training. Yin et al. (2009) found that antagonism of D1 receptors during the initial stages of training disrupted rotorod performance but the same treatment was without effect after extended training. This finding is in agreement with previous demonstrations that D1 receptor activation is important for initial learning but plays a decreasing role as training progresses. For example, Choi et al. (2005) found that treatment with a D1 antagonist disrupted both cued and non-cued approach to a food receptacle after limited (3 days) training but was without effect on cued responding following more extended training (16 days) although effects on spontaneous responses remained. This is consistent with the suggestion that compromised dopamine transmission does not produce pure motor deficits but rather impairs the ability to generate voluntary motor acts in the absence of external eliciting stimuli (Jahanshahi and Frith, 1998; Choi et al., 2005). While there was no direct test of whether performance was in fact habitual in the study by Choi and colleagues (i.e., insensitive to outcome devaluation or changes to the response-outcome contingency) the selective effect on cued responding could be explained by a S-R (stimulus-response) learning mechanism like that argued to underlie habitual performance and which the authors suggest may become dopamine-independent over the course of extended training.

Stimuli, and particularly the context in which rats were trained and tested may contribute importantly to the observed effects. There is direct evidence that alcohol-paired environmental stimuli can promote habitual behavior. For example, Ostlund et al. (2010) found that when rats were tested in a saline-paired context they showed a significant devaluation effect. However, when the same rats with the same training history were tested in a context that had been paired with alcohol they were insensitive to outcome devaluation indicating habitual responding and demonstrating that alcohol-predictive cues can disrupt decision making. Similar results have bee reported with more discrete alcohol cues in humans (Hogarth et al., 2013). Discrete cues can also trigger and maintain responding for alcohol and other rewards (Lê and Shaham, 2002; Loeber et al., 2006; Corbit and Janak, 2007a) and many of these stimulus effects rely on the DLS (Vanderschuren et al., 2005; Volkow et al., 2006; Corbit and Janak, 2007b). Thus it is possible that the effects of either NBQX or raclopride relate to an ability to suppress the influence of environmental stimuli, such as the training context, which over the course of training had become paired with alcohol, and thus a remove context-mediated bias towards habitual performance.

It has been suggested that over the course of extended training control of responding is less dependent on dopamine because it shifts to brain regions outside the striatum. For example, single-unit activity in the sensorimotor striatum has been reported to decrease over the course of extended training (Carelli et al., 1997) as does phasic dopamine release in this region (Willuhn et al., 2014). This finding is difficult to reconcile with multiple demonstrations that lesions or inactivation of the DLS abolish habitual performance indicating that the DLS is necessary for the acquisition and expression of response habits (Yin et al., 2004; Corbit et al., 2012). Others have reported decreases in the number of sensorimotor striatal neurons showing task-relevant phasic activity over the course of training, but also that a small group of neurons in this region continue to show task-relevant activity and that the magnitude of neural responses grows with extended training (Barnes et al., 2005; Tang et al., 2007). These demonstrations may reflect weakening of non-task-relevant corticostriatal inputs (through LTD) and/or lateral inhibition of competing striatal neurons by the neurons that have been potentiated over the course of learning (Ashby et al., 2010).

In contrast to the diminishing role of D1 receptor activity following extended training (Choi et al., 2005), several studies have demonstrated that a D1 antagonist (SCH23390) given either during instrumental training (Nelson and Killcross, 2013) or prior to devaluation testing (Furlong et al., 2014) can suppress habitual performance. Notably, in those studies repeated amphetamine or chronic access to a palatable food was given to animals prior to instrumental training and subsequently was found to produce more rapid habit learning than seen in controls. It is not currently known whether the rapid habit learning that follows certain previous experiences including exposure to potent rewards such as drugs rely on the same mechanisms as habit learning following extended training and the role of specific dopamine receptors may differ in these situations. Indeed, in the present study, there are two factors that may contribute to habitual control of performance and consideration of each is important for understanding the role of dopamine in behavioral control; first, rats had chronic exposure to alcohol, and second, they also underwent extensive training. Extended training itself can promote habitual control (Adams, 1982), and performance that is independent of D1 receptor activation (Choi et al., 2005; Yin et al., 2009), and as indicated by the results of Yin et al. this type of learning may recruit D2-containing neurons in the DLS. As noted above, there are now several demonstrations that mere exposure to drugs (Nelson and Killcross, 2006; Nordquist et al., 2007; Corbit et al., 2012, 2014) and other salient events (Dias-Ferreira et al., 2009; Furlong et al., 2014) can lead to subsequent learning falling rapidly under habitual control. Downregulation of D2 receptors and changes in the relative contribution of D1 vs. D2-containing neurons following long-term exposure to drugs or potent rewards (Volkow et al., 1993; Johnson and Kenny, 2010; Park et al., 2013) has been repeatedly demonstrated and may contribute to why D1 receptor antagonists reduce the expression of habitual behaviors following such treatments. Extended training itself may not produce the same adaptations in receptor expression as alcohol exposure and thus, it is possible that dopamine and its receptors will have somewhat different roles in habits produced by drug exposure and extended training. Future studies examining the role of D1 and D2 antagonists in each of these paradigms will be needed to fully understand the mechanisms underlying the shift to habitual control.

In summary, we demonstrate that extended alcohol self-administration leads to alcohol-seeking that is habitual and not sensitive to change in the value of alcohol. Sensitivity is restored following AMPA or D2 receptor antagonism in the DLS. Understanding the cognitive and neural control of alcohol-seeking particularly after extensive experience with the drug and behaviors involved in procuring it will be important for understanding the loss of control that is a hallmark of substance use disorders including alcoholism.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This work was supported by grants from the National Institutes of Health (R01 AA018025; Patricia H. Janak), ABMRF (Laura H. Corbit) and Australian National Health and Medical Research Council (1051037; Laura H. Corbit).

References

Adams, C. D. (1982). Variations in the sensitivity of instrumental responding to reinforcer devaluation. Q. J. Exp. Psychol. 34B, 77–98.

Anzalone, A., Lizardi-Ortiz, J. E., Ramos, M., De Mei, C., Hopf, F. W., Iaccarino, C., et al. (2012). Dual control of dopamine synthesis and release by presynaptic and postsynaptic dopamine D2 receptors. J. Neurosci. 32, 9023–9034. doi: 10.1523/jneurosci.0918-12.2012

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ashby, F. G., Turner, B. O., and Horvitz, J. C. (2010). Cortical and basal ganglia contributions to hait learning and automaticity. Trends Cogn. Sci. 14, 208–215. doi: 10.1016/j.tics.2010.02.001

CrossRef Full Text

Barnes, T. D., Kubota, Y., Hu, D., Jin, D. Z., and Graybiel, A. M. (2005). Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories. Nature 437, 1158–1161. doi: 10.1038/nature04053

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Belin, D., and Everitt, B. J. (2008). Cocaine seeking habits depend upon dopamine-dependent serial connectivity linking the ventral with the dorsal striatum. Neuron 57, 432–441. doi: 10.1016/j.neuron.2007.12.019

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Carelli, R. M., Wolske, M., and West, M. O. (1997). Loss of lever press-related firing of rat striatal forelimb neurons after repeated sessions in a lever pressing task. J. Neurosci. 17, 1804–1814.

Pubmed Abstract | Pubmed Full Text

Choi, W. Y., Balsam, P. D., and Horvitz, J. C. (2005). Extended habit training reduces dopamine mediation of appetitive response expression. J. Neurosci. 25, 6729–6733. doi: 10.1523/jneurosci.1498-05.2005

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Corbit, L. H., Chieng, B. C., and Balleine, B. W. (2014). Effects of repeated cocaine exposure on habit learning and reversal by N-acetylcysteine. Neuropsychopharmacology 39, 1893–1901. doi: 10.1038/npp.2014.37

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Corbit, L. H., and Janak, P. H. (2007a). Ethanol-associated cues produce general pavlovian-instrumental transfer. Alcohol. Clin. Exp. Res. 31, 766–774. doi: 10.1111/j.1530-0277.2007.00359.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Corbit, L. H., and Janak, P. H. (2007b). Inactivation of the lateral but not medial dorsal striatum eliminates the excitatory impact of Pavlovian stimuli on instrumental responding. J. Neurosci. 27, 13977–13981. doi: 10.1523/JNEUROSCI.4097-07.2007

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Corbit, L. H., Nie, H., and Janak, P. H. (2012). Habitual alcohol seeking: time course and the contribution of subregions of the dorsal striatum. Biol. Psychiatry 72, 389–395. doi: 10.1016/j.biopsych.2012.02.024

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Czachowski, C. L., Chappell, A. M., and Samson, H. H. (2001). Effects of raclopride in the nucleus accumbens on ethanol seeking and consumption. Alcohol. Clin. Exp. Res. 25, 1431–1440. doi: 10.1111/j.1530-0277.2001.tb02144.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Dias-Ferreira, E., Susa, J. C., Melo, I., Morgado, P., Mesquita, A. R., Cerqueira, J. J., et al. (2009). Chronic stress causes frontostriatal reorganization and affects decision-making. Science 325, 621–625. doi: 10.1126/science.1171203

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Dickinson, A., Smith, J., and Mirenowicz, J. (2000). Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists. Behav. Neurosci. 114, 468–483. doi: 10.1037/0735-7044.114.3.468

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Faure, A., Haberland, U., Conde, F., and El Massioui, N. (2005). Lesion to the nigrostriatal dopamine system disrupts stimulus-response habit formation. J. Neurosci. 25, 2771–2780. doi: 10.1523/jneurosci.3894-04.2005

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Floyer-Lea, A., and Matthews, P. M. (2005). Distinguishable brain activation networks for short- and long-term motor skill learning. J. Neurophysiol. 94, 512–518. doi: 10.1152/jn.00717.2004

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Furlong, T. M., Jayaweera, H. K., Balleine, B. W., and Corbit, L. H. (2014). Binge-like consumption of a palatable food accelerates habitual control of behavior and is dependent on activation of the dorsolateral striatum. J. Neurosci. 34, 5012–5022. doi: 10.1523/JNEUROSCI.3707-13.2014

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hogarth, L., Field, M., and Rose, A. K. (2013). Phasic transition from goal-directed to habitual control over drug-seeking produced by conflicting reinforcer expectancy. Addict. Biol. 18, 88–97. doi: 10.1111/adb.12009

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Jahanshahi, M., and Frith, C. D. (1998). Willed action and its impairments. Cogn. Neuropsychol. 15, 483–533. doi: 10.1080/026432998381005

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Jeanblanc, J., He, D. Y., McGough, N. N., Logrip, M. L., Phamluong, K., Janak, P. H., et al. (2006). The dopamine D3 receptor is part of a homeostatic pathway regulating ethanol consumption. J. Neurosci. 26, 1457–1464. doi: 10.1523/jneurosci.3786-05.2006

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Johnson, P. M., and Kenny, P. J. (2010). Dopamine D2 receptors in addiction-like reward dysfunction and compulsive eating in obese rats. Nat. Neurosci. 13, 635–641. doi: 10.1038/nn.2519

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Joyce, J. N., Loeschen, S. K., and Marshall, J. F. (1985). Dopamine D-2 receptors in rat caudate-putamen: the lateral to medial gradient does not correspond to dopaminergic innervation. Brain Res. 338, 209–218. doi: 10.1016/0006-8993(85)90149-0

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Karni, A., Meyer, G., Jezzard, P., Adams, M. M., Turner, R., and Ungerleider, L. G. (1995). Functional MRI evidence for adult motor cortex plasticity during motor skill learning. Nature 377, 155–158. doi: 10.1038/377155a0

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Lê, A. D., and Shaham, Y. (2002). Neurobiology of relapse to alcohol in rats. Pharmacol. Ther. 94, 137–156. doi: 10.1016/s0163-7258(02)00200-0

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

LeBlanc, K. H., Maidment, N. T., and Ostlund, S. B. (2013). Repeated cocaine exposure facilitates the expression of incentive motivation and induces habitual control in rats. PLoS One 8:e61355. doi: 10.1371/journal.pone.0061355

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Loeber, S., Croissant, B., Heinz, A., Mann, K., and Flor, H. (2006). Cue exposure in the treatment of alcohol dependence: effects on drinking outcome, craving and self-efficacy. Br. J. Clin. Psychol. 45, 515–529. doi: 10.1348/014466505x82586

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Nelson, A., and Killcross, S. (2006). Amphetamine exposure enhances habit formation. J. Neurosci. 26, 3805–3812. doi: 10.1523/jneurosci.4305-05.2006

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Nelson, A. J., and Killcross, S. (2013). Accelerated habit formation following amphetamine exposure is reversed by D1, but enhanced by D2, receptor antagonists. Front. Neurosci. 7:76. doi: 10.3389/fnins.2013.00076

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Nordquist, R. E., Voorn, P., de Mooij-van Malsen, J. G., Joosten, R. N., Pennartz, C. M., and Vanderschuren, L. J. (2007). Augmented reinforcer value and accelerated habit formation after repeated ampheta- mine treatment. Eur. Neuropsychopharmacol. 17, 532–540. doi: 10.1016/j.euroneuro.2006.12.005

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ostlund, S. B., Maidment, N. T., and Balleine, B. W. (2010). Alcohol-paired contextual cues produce an immediate and selective loss of goal-directed action in rats. Front. Integr. Neurosci. 4:19. doi: 10.3389/fnint.2010.00019

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Park, K., Volkow, N. D., Pan, Y., and Du, C. (2013). Chronic cocaine dampens dopa- mine signaling during cocaine intoxication and unbalances D1 over D2 receptor signaling. J. Neurosci. 33, 15827–15836. doi: 10.1523/JNEUROSCI.1935-13.2013

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Paxinos, G., and Watson, C. (1998). The Rat Brain in Stereotaxic Coordinates. 4th Edn. San Diego: Academic.

Salamone, J. D., Cousins, M. S., and Snyder, B. J. (1997). Behavioral functions of nucleus accumbens dopamine: empirical and conceptual problems with the anhedonia hypothesis. Neurosci. Biobehav. Rev. 21, 341–359. doi: 10.1016/s0149-7634(96)00017-6

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Schultz, W. (2007). Multiple dopamine functions at different time courses. Annu. Rev. Neurosci. 30, 259–288. doi: 10.1146/annurev.neuro.28.061604.135722

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Tang, C., Pawlak, A. P., Prokopenko, V., and West, M. O. (2007). Changes in activity of the striatum during formation of a motor habit. Eur. J. Neurosci. 25, 1212–1227. doi: 10.1111/j.1460-9568.2007.05353.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Vanderschuren, L. J., Di Ciano, P., and Everitt, B. J. (2005). Involvement of the dorsal striatum in cue-controlled cocaine seeking. J. Neurosci. 25, 8665–8670. doi: 10.1523/jneurosci.0925-05.2005

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Vanderschuren, L. J., and Everitt, B. J. (2005). Behavioral and neural mechanisms of compulsive drug seeking. Eur. J. Pharmacol. 526, 77–88. doi: 10.1016/j.ejphar.2005.09.037

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Volkow, N. D., Fowler, J. S., Wang, G. J., Hitzemann, R., Logan, J., Schlyer, D. J., et al. (1993). Decreased dopamine D2 receptor availability is associated with reduced frontal metabolism in cocaine abusers. Synapse 14, 169–177. doi: 10.1002/syn.890140210

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Volkow, N. D., Wang, G. J., Telang, F., Fowler, J. S., Logan, J., Childress, A. R., et al. (2006). Cocaine cues and dopamine in dorsal striatum: mechanism of craving in cocaine addiction. J. Neurosci. 26, 6583–6588. doi: 10.1523/jneurosci.1544-06.2006

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Wang, J., Carnicella, S., Phamluong, K., Jeanblanc, J., Ronesi, J. A., Chaudhri, N., et al. (2007). Ethanol induces long-term facilitation of NR2B-NMDA receptor activity in the dorsal striatum: implications for alcohol drinking behavior. J. Neurosci. 27, 3593–3602. doi: 10.1523/jneurosci.4749-06.2007

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Wickens, J. R., Horvitz, J. C., Costa, R. M., and Killcross, S. (2007). Dopaminergic mechanisms in actions and habits. J. Neurosci. 27, 8181–8183. doi: 10.1523/jneurosci.1671-07.2007

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Wickens, J. R., Reynolds, J. N., and Hyland, B. I. (2003). Neural mechanisms of reward-related motor learning. Curr. Opin. Neurobiol. 13, 685–690. doi: 10.1016/j.conb.2003.10.013

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Willuhn, I., Burgeno, L. M., Groblewski, P. A., and Phillips, P. E. (2014). Excessive cocaine use results from decreased phasic dopamine signaling in the striatum. Nat. Neurosci. 17, 704–709. doi: 10.1038/nn.3694

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Wilson, C. J., Chang, H. T., and Kitai, S. T. (1983). Disfacilitation and long-lasting inhibition of neostriatal neurons in the rat. Exp. Brain Res. 51, 227–235. doi: 10.1007/bf00237198

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Yin, H. H., Knowlton, B. J., and Balleine, B. W. (2004). Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur. J. Neurosci. 19, 181–189. doi: 10.1111/j.1460-9568.2004.03095.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Yin, H. H., Mulcare, S. P., Hilário, M. R., Clouse, E., Holloway, T., Davis, M. I., et al. (2009). Dynamic reorganization of striatal circuits during the acquisition and consolidation of a skill. Nat. Neurosci. 12, 333–341. doi: 10.1038/nn.2261

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Zapata, A., Minney, V. L., and Shippenberg, T. S. (2010). Shift from goal- directed to habitual cocaine seeking after prolonged experience in rats. J. Neurosci. 30, 15457–15463. doi: 10.1523/JNEUROSCI.4072-10.2010

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text