Elevated dopamine alters consummatory pattern generation and increases behavioral variability during learning

The role of dopamine in controlling behavior remains poorly understood. In this study we examined licking behavior in an established hyperdopaminergic mouse model—dopamine transporter knockout (DAT KO) mice. DAT KO mice showed higher rates of licking, which is due to increased perseveration of licking in a bout. By contrast, they showed increased individual lick durations, and reduced inter-lick intervals. During extinction, both KO and control mice transiently increased variability in lick pattern generation while reducing licking rate, yet they showed very different behavioral patterns. Control mice gradually increased lick duration as well as variability. By contrast, DAT KO mice exhibited more immediate (within 10 licks) adjustments—an immediate increase in lick duration variability, as well as more rapid extinction. These results suggest that the level of dopamine can modulate the persistence and pattern generation of a highly stereotyped consummatory behavior like licking, as well as new learning in response to changes in environmental feedback. Increased dopamine in DAT KO mice not only increased perseveration of bouts and individual lick duration, but also increased the behavioral variability in response to the extinction contingency and the rate of extinction.


Introduction
Dopamine, a major neurotransmitter that mainly modulates activity in the basal ganglia, has been implicated in a variety of motivated behaviors (Zhou and Palmiter, 1995;Berridge, 2007;Rossi et al., 2013a,b). Extensive dopaminergic projections target the striatum, the input nucleus of the basal ganglia, where dopamine is released and influences the responsiveness of striatal neurons to glutamatergic cortical inputs (Gerfen and Surmeier, 2011). Dopamine release in the striatum is thought to be critical for the generation of orofacial behavior, including licking and chewing (Fray et al., 1980;Redgrave et al., 1980;Arnt et al., 1987;Delfs and Kelley, 1990;Skitek et al., 1999).
Previous studies have suggested a critical role of dopamine in appetitive and consummatory behaviors for food and water (Szczypka et al., 2001;Salamone and Correa, 2012). For example, systemic administration of dopamine receptor agonists potentiates orofacial movements and induces aberrant licking and chewing (Olpe, 1978;Zarrindast et al., 1992). Oral stereotypy can be blocked by pretreatment with dopamine receptor antagonists (Delfs and Kelley, 1990). Moreover, in dopamine transporter (DAT) knockout (KO) mice, which show prolonged increases in dopamine signaling (Giros et al., 1996), the overall rate of licking for sucrose reward was greatly increased (Costa et al., 2007). Nevertheless, exactly how dopamine can modulate consummatory behaviors remains unclear.
In this study, we used DAT KO mice, an established animal model of hyperdopaminergia (Giros et al., 1996;Rodriguiz et al., 2004), to study the role of dopamine in modulating consummatory behavior. Consummatory licking is a relatively stereotyped behavior in rodents, with tongue protrusions occurring at a rate of 5-9 Hz (Marowitz and Halpern, 1973;Halpern, 1975;Murakami, 1977;Weijnen, 1998). Despite the stereotypical pattern, however, studies have shown that the temporal structure of licking can be modulated by a variety of factors, such as perceptual feedback, satiety, palatability, and so on (Cone, 1974;Cone et al., 1975;Mamedov and Bures, 1984). However, the neural mechanisms that contribute to the patterning of licking behavior are largely unknown.
We quantified the pattern of licking in DAT KO mice and their wild type (WT) littermates using a contact lickometer as they voluntarily licked sucrose solution. We found that, compared to WT mice, DAT KO mice showed higher rates of licking, which were due to increased perseveration of licking in a bout. By contrast, they showed increased individual lick durations, and reduced inter-lick intervals. In addition, they also showed more rapid onset of behavioral variability following the onset of extinction and more rapid extinction of licking behavior.

Subjects
All procedures were conducted in accordance with the National Institutes of Health guidelines for the care and use of animals and in accordance with the Duke University Institutional Animal Care and Use Committee guidelines. DAT KO (n = 11) and WT (n = 8) littermates (8-12 weeks old) were used for experiments as previously described (Giros et al., 1996). During testing, mice were maintained on 23 h water deprivation schedule. Access to water was restricted to 1 h per day. All experiments were conducted during the light phase of the animal's light cycle.

Measuring Licking Behavior
Sucrose (10% w/v) was available from a standard water bottle in one wall of an operant chamber (35 cm × 28 cm × 22 cm). The spout was recessed ∼3 mm within a plastic tube to prevent the mice from contacting the lick sensor with their paws. Licks were recorded using a contact lickometer (Slotnick, 2009). When mice touched the metal spout with their tongues, a circuit was completed, which triggered a voltage change lasting the duration of the contact. Voltage was sampled at 2000 Hz using the Cerebus acquisition system (Blackrock Microsystems).
Testing took place in daily 30-min sessions. Thirsty mice were first acclimated to the testing chamber for 30 min and allowed to freely consume sucrose solution. The following day, mice were tested in the same chamber, and licks were recorded (sucrose condition). The next day, the extinction test was conducted in which the same set up was used, except the spout was empty. On the following day, reinstatement, sucrose solution was once again present in the spout. On the final day, the spout contained water. One KO mouse was omitted from the water test.
Locomotion tests were conducted after completion of the licking tests with a subset of the same mice used in the previous licking experiments (n = 8 WT; n = 9 KO). Mice were placed in a circular chamber for 3 min and video was taken from directly above. Position was tracked frame-by-frame offline using custom software (Matlab) as described previously (Rossi et al., 2013a).

Data Analysis
From the analog voltage signal, we generated timestamps corresponding to the onset and termination of each lick (0.5 ms resolution). From this we calculated the duration (time from onset to termination), duty cycle (the ratio of the duration of the contact with the lickometer to the period for each lick cycle), and inter-lick interval (ILI; time from lick offset to the next lick onset) as well as the number and rate of licks. Bout analysis was used to determine the properties of bouts of licking. Based on previous analysis of rodent licking (Spector et al., 1998), we defined the start of a bout as three or more licks occurring at >3 Hz and being preceded by at least 1 s in which no licks were recorded. The end of the bout was defined as the last lick that was followed by at least 1 s in which no licks were recorded. Power spectral density analysis was used to assess the component frequencies of the lick trains (Neuroexplorer, Nex Technologies). The lick rate was calculated as the inverse of the median ILI of all licks occurring within a bout.
For licking parameters that are not normally distributed (duration, ILI, duty cycle; Pearson omnibus normality test, p < 0.001), we used the median values to compare between groups. Median values were compared with Mann-Whitney test. To compare the number of licks per bout between groups, Welch's t-test was used to correct for unequal variance.  (Figures 1A,B). Overall, KO mice traveled much farther than WT littermates [ Figure 1B; t (15) = 10.33, p < 0.0001]. This is in agreement with previous studies showing hyperlocomotion in hyperdopaminergic mice (Giros et al., 1996).

Results
To assess whether chronically elevated dopamine alters consummatory motor output, we designed a contact lickometer that detected individual licks (Figure 2A; Slotnick, 2009). We found that DAT KO mice had altered lick patterning compared to WT mice when voluntarily licking for sucrose solution (Figures 2B,C). Overall, the lick cycle of DAT KO mice (within bout rate: 7.66 ± 0.13 Hz) appeared slower relative to WT mice (within bout rate: 8.28 ± 0.20 Hz). DAT KO mice showed longer lick durations than WT mice ( Figure 3A; Mann-Whitney U = 10.00, p = 0.0057) and shorter ILIs ( Figure 3B; U = 17.00, p = 0.028). The duty cycle (the ratio of the duration of the contact with the lickometer to the period for each lick cycle) was increased in DAT KO mice relative to WT mice ( Figure 3C; U = 19.00, p = 0.043).

In Extinction, in Which no Solution is Present, DAT KO Mice no Longer Show Altered Lick Patterning Relative to WT Controls
To test whether the differences in the patterning of DAT KO licking were influenced by feedback related to the presence of sucrose in the spout, we conducted licking tests in extinction, when no solution was present. When mice licked Criteria are based on (Spector et al., 1998). *p < 0.05, t-test between KO and WT. the dry spout, the rate of licking was slower than when they licked for sucrose (WT: 6.47 ± 0.21 Hz; KO: 6.32 ± 0.23 Hz).

DAT KO Mice are More Sensitive Than WT Controls to Extinction
Knockout mice were more sensitive to extinction than WT mice (Figure 6). Both groups reduced the rate of licking during extinction [ Figure 6A; two-way ANOVA (Genotype ×  Figure 6C; t (17) = 3.50, p = 0.0028]. The number of licks per bout was greatly reduced for both groups compared to the sucrose condition, yet KO mice still tended to lick more per bout [ Figure 6D; t-test with Welch's correction for unequal variance: t (13) = 3.97, p = 0.0016].
To understand the time course of behavioral adaptation to altered feedback during extinction, we analyzed how lick duration changed over time (Figures 6E-G). DAT KO mice adapted more quickly to the dry spout than WT mice. Because all mice licked different amounts, we divided the licks of each mouse into three equally sized bins to compare lick duration and variability throughout the extinction session. The duration of licks was stable throughout the session, and there was no group difference [ Figure 6F; two wan ANOVA (Genotype × Time): no main effect of Genotype, F (1,34) = 0.75, p > 0.05; no main effect of Time, F (2,34) = 2.98, p = 0.06; no interaction, F (2,34) = 1.21, p > 0.05]. In response to the dry spout, mice increased the variability of lick durations. DAT KO mice, however, did so more quickly than WT mice as measured by the inter quartile range (Figure 6G; no main effect of Genotype, F (1,34) = 1.06, p > 0.05; main effect of Time, F (2,34) = 3.53, p = 0.04; interaction between Genotype and Time, F (2,34) = 4.27, p = 0.02). DAT KO mice had reduced variability late in the extinction test relative to WT mice (p < 0.05).

Lick Patterning is Similar Between KO and WT Mice When Licking Water
To test whether the observed differences in the temporal structure of licking were due to the high incentive value associated with sucrose solution, we recorded licking while mice consumed water. We found that licking was much more similar between DAT KO and WT mice than when they were licking for sucrose ( Figure 9A; WT: 7.67 ± 0.28 Hz; KO: 7.96 ± 0.13 Hz). There was no statistically significant  altered feedback during extinction more quickly than WT mice. The first 50 licks during extinction was considered 'early' and the last 50 licks was considered 'late.' Duration of licks (F) and inter-quartile range of lick durations (G) returns to pre-extinction levels during reinstatement. * p < 0.05. Points represent data from individual mice. Lines represent mean and SEM.
When licking water, the global lick rate was similar between groups [ Figure 10A; no main effect of Genotype,  Figure 10C; unpaired t-test with Welch's correction for unequal variance: t (10) = 4.71, p = 0.0008].

Discussion
The DAT KO is a well-established mouse model of hyperdopaminergia (Giros et al., 1996;Spielewoy et al., 2000). In agreement with previous work, we found general hyperactivity in these mice. Our novel finding is a pronounced change in the pattern and timing of licking. DAT KOs showed increased individual lick duration, reduced ILI, and more licks per bout. In addition, when the feedback was altered during extinction, we found a dramatic difference in the consequent change in licking behavior: whereas controls showed a gradual increase in behavioral variability over time, the DAT KOs were much more sensitive to the change in the feedback function, showing an initial, transient burst of variability following the detection of the extinction contingency ( Figure 6G). This more rapid generation of behavioral variability is also coupled with faster extinction (Figure 6B).

Generation of Licking Pattern
In recent years, licking behavior has increasingly become a focus of neuroscience research because it is highly reliable and stereotyped, permitting convenient characterization of the temporal structure of behavior (Gutierrez et al., 2010;Johnson et al., 2010;Komiyama et al., 2010;Ostlund et al., 2013;Guo et al., 2014).
Although the pattern generator underlying licking is well characterized, the top-down influences, including the role of the basal ganglia, remain largely unknown. Some cortical regions are known to show lick related activity (Gutierrez et al., 2006;Komiyama et al., 2010). Neurons from these regions may drive licking via striatal projections. The neurons within the lateral striatum exhibit oscillatory activity that corresponds to licking behavior (Mittler et al., 1994). Injection of amphetamine in the ventrolateral striatum induces oral stereotypy and voracious licking (Kelley et al., 1988), and unilateral depletion of nigrostriatal dopamine in rats results in impaired licking in which the lick rate slows and tongue force is reduced (Skitek et al., 1999).
Output from the basal ganglia is thought to regulate orofacial movements via projections to the motor regions of the superior colliculus (Taha et al., 1982;Gunne et al., 1988). The superior colliculus, in turn, sends projections to rhythmically active motor and premotor neurons in the reticular formation that control the tongue and jaws (Morimoto et al., 1966;Wiesenfeld et al., 1977;Sparks and Hartwich-Young, 1989;Brozek et al., 1996;Travers et al., 1997).
Licking is often considered a highly stereotyped behavior controlled by a central pattern generator. However, as shown in the present study, the pattern is certainly influenced by feedback from the liquid during licking. DAT KO mice showed higher overall rates of licking for sucrose. This result is not surprising given the known hyperactivity in these mice (Costa et al., 2007;Perona et al., 2008), but we showed for the first time a significant change in the patterning of licking. These changes are characterized by a dramatic increase in the persistence of a bout of licking (more licks per bout), as well as an increase in the individual lick contact duration. Their high rate of licking is primarily due to an increase in the number of licks per bout, i.e., more persistent licking once the bout is initiated. On the other hand, during each bout, their licking is actually slower, characterized by longer duration of contact with the spout and shorter ILIs. This observation can be explained by the hypothesis that DAT KO mice attempt to maximize sucrose intake. In accord with this interpretation, the overall proportion of contact time during a bout (duty cycle) is increased in the DAT KO group.

Extinction
Here we showed that in control mice, the lick duration increased in extinction, when the sucrose reward was no longer delivered. WT mice gradually increased variability of lick duration during extinction, as shown in Figure 6G. Extinction, as a procedure, represents a drastic change in the feedback function, and has long been shown to result in new learning. As shown by the rate of licking, all mice reduced their overall rate of licking over time, but the DAT KO mice showed a more rapid reduction ( Figure 6B). Moreover, they immediately increased behavioral variability, whereas the controls showed a more gradual change.
An increase in behavioral variability is present in initial instrumental learning (Derusso et al., 2010;Costa, 2011;Yin, 2014a). But such variability could be a general feature of learning, including the behavioral adaptations following exposure to the extinction contingency. The mice learned to stop licking, but initially they exhibited "exploration" by varying the pattern and timing of licking. Despite the highly stereotyped licking pattern, both the ILI and the lick duration could be varied, and all mice increased such behavioral variability following the onset of extinction.

The Function of Dopamine
Our results are in accord with previous work showing a critical role of dopamine in the performance of orofacial movements. For example, studies have shown reduced lick frequency in rats with unilateral striatal dopamine depletion (Skitek et al., 1999) and increased frequency of orofacial movements following systemic or intrastriatal administration of dopamine agonists (Fray et al., 1980;Redgrave et al., 1980;Arnt et al., 1987;Delfs and Kelley, 1990). Yet our results suggest that increasing dopamine transmission produces two distinct types of effects-one related to performance and the other related to learning.
The net effect of dopamine is to modulate the gain of the reward seeking system, altering performance to maximize the input by prolonging the contact duration (duty cycle). This suggests that dopamine could be operating at a hierarchically higher level that has access to net sucrose intake. This level, presumably corresponding to the basal ganglia circuits, can simultaneously modulate all three effects (ILI, contact duration, and bout persistence) in the right direction to maximize sucrose intake. As a result, the overall sucrose yield per bout of licking is increased. This interpretation is also supported by the finding that the difference between KO mice and controls is much reduced when they are licking for water, which has lower incentive value (Figure 9).
It has recently been proposed that the basal ganglia networks are closed loop controllers that regulate transition or rate of change in different perceptual variables (Yin, 2014a,b,c;Barter et al., 2015;Rossi et al., 2015). Dopaminergic projections to the striatum can adjust the gain of such a system. The limbic and associative cortico-basal ganglia networks can be especially important for the control of reward rates using diverse behavioral outputs (Yin et al., 2008;Yin, 2014a).
The second effect is related to learning, in this case to behavioral adaptation during extinction or non-reinforcement. As has long been established, extinction can result in new types of learning (Bouton, 2004). Here dopamine appears to modulate the level of behavioral variability in licking pattern generation (Costa, 2011). This effect is in agreement with work on dopaminergic modulation of song variability in song birds (Leblois et al., 2010;Leblois and Perkel, 2012). The prolonged dopamine signaling in DAT KO mice may therefore increase variability depending on the behavioral context, whether it is modulation of song production based on social context or the modulation of licking variability following the onset of extinction. Because the generation of such behavioral variability is critical during the exploratory phase of learning, dopamine can also play a key role in learning.