The habenula governs the attribution of incentive salience to reward predictive cues

The attribution of incentive salience to reward associated cues is critical for motivation and the pursuit of rewards. Disruptions in the integrity of the neural systems controlling these processes can lead to avolition and anhedonia, symptoms that cross the diagnostic boundaries of many neuropsychiatric illnesses. Here, we consider whether the habenula (Hb), a region recently demonstrated to encode negatively valenced events, also modulates the attribution of incentive salience to a neutral cue predicting a food reward. The Pavlovian autoshaping paradigm was used in the rat as an investigative tool to dissociate Pavlovian learning processes imparting strictly predictive value from learning that attributes incentive motivational value. Electrolytic lesions of the fasciculus retroflexus (fr), the sole pathway through which descending Hb efferents are conveyed, significantly increased incentive salience as measured by conditioned approaches to a cue light predictive of reward. Conversely, generation of a fictive Hb signal via fr stimulation during CS+ presentation significantly decreased the incentive salience of the predictive cue. Neither manipulation altered the reward predictive value of the cue as measured by conditioned approach to the food. Our results provide new evidence supporting a significant role for the Hb in governing the attribution of incentive motivational salience to reward predictive cues and further imply that pathological changes in Hb activity could contribute to the aberrant pursuit of debilitating goals or avolition and depression-like symptoms.


INTRODUCTION
Cues and imagery associated with rewards can evoke pleasure, motivate approach, and lead to reward consumption. The integrity of the motivational systems controlling these processes are critical to an individual's functional capacity and quality of life; disruptions can lead to amotivation, avolition, and anhedonia, symptoms that cross the diagnostic boundaries of many neuropsychiatric disorders (Insel et al., 2010;Cohen et al., 2012;Morris and Cuthbert, 2012). While an imbalance in the opposing constructs mediating approach and avoidance could contribute to the development of these clinical symptoms, avoidance fails to faithfully capture the clinical features of anhedonia and avolition. Rather, these maladaptive behaviors could reflect functional changes in systems encoding the incentive salience of stimuli predicting reward. For example, increased attribution of incentive salience to reward cues could lead to reduced impulse control and the aberrant pursuit of rewards while a failure to appropriately assign incentive salience to reward cues is likely to diminish motivation and volition.
The Pavlovian autoshaping paradigm is a method well suited to investigate the circuitry underlying attribution of incentive salience (Brown and Jenkins, 1968;Gamzu and Williams, 1973;Pellegrini et al., 2008). In this procedure, repeated presentation of a stimulus prior to reward delivery results in a quantifiable and progressive increase in conditioned approach toward the goal (e.g., food) as the predictive value of the cue is learned. This conditioned response is termed "goal-tracking." A quantifiable and progressive increase in conditioned approach to the stimuli can also occur as it is increasingly endowed with incentive salience. This conditioned response is termed"sign-tracking." The paradigm offers the possibility of separately assessing predictive value from the motivational salience associated with cues predicting reward delivery. Recent studies demonstrate that dopamine (DA) transmission is required for sign-tracking but not for goal-tracking (Day et al., 2007;Danna and Elmer, 2010;Flagel et al., 2011b).
The circuitry responsible for producing changes in DA cell activity associated with the attribution of incentive salience is incompletely understood. Considerable interest has developed in the role of the lateral habenula (LHb) and rostral medial tegmentum (RMTg) in mediating the transient suppression in DA cell firing associated with negative reward prediction errors (Hong et al., 2011). LHb neurons are coherently activated by the absence of expected rewards (Matsumoto and Hikosaka, 2009) and are presumed to mediate their influence on DA cell firing via connections with GABAergic RMTg neurons (Hong et al., 2011). In addition to its functional role in encoding negatively valenced associations, increased LHb activity during reward anticipation (Bromberg-Martin et al., 2010), and in animals predisposed to assigning motivational properties to neutral cues (Flagel et al., 2011a) suggest a role in governing the attribution of incentive salience. To date, this possibility has not been systematically explored.
In the present series of experiments, electrolytic lesions and electrical stimulation of the fasciculus retroflexus (fr), the sole pathway through which descending LHb efferents are conveyed, Frontiers in Human Neuroscience www.frontiersin.org were used to assess the contribution of this pathway to specific components of the Pavlovian autoshaping procedure. Our results provide new evidence supporting a significant role for the Hb in governing the attribution of incentive motivational salience to reward predictive cues.

SUBJECTS
Adult, male, Sprague-Dawley rats (Charles River Laboratories, Wilmington, MA, USA) weighing approximately 225-250 g at the start of the experiment were used as experimental subjects. Rats were housed in a temperature controlled vivarium under a 12:12 h light:dark cycle and provided unrestricted access to food prior to the start of the behavioral experiments. All studies were conducted in strict accordance with the principals outlined in the NIH Guide for Care and Use of Laboratory Animals and were sanctioned by the Institutional Animal Care and Use Committee of the University of Maryland, Baltimore.

PAVLOVIAN AUTOSHAPING PROCEDURE
Six rat operant chambers (dimensions 25 cm L × 21 cm W × 20 cm H; Med Associates, St. Albans, VT, USA) under the control of Med Associates software were used during the completion of these experiments. Each chamber was equipped with two custom cue lights consisting of a translucent 6 × 6 cm panel positioned on either side of a pellet delivery system. One light in each chamber was covered with translucent blue plastic to enhance the difference between visual cues. A pellet delivery system provided a single 45 mg sucrose pellet to a small recessed receptacle located between the two cue lights. Entrances into the pellet receptacle and approaches to each cue light were detected by separate infrared photobeam detectors. Rats were required to be within 2 cm of the cue light to interrupt the beam. A retractable lever was positioned in the middle panel at the opposite end of the chamber from the stimulus lights and pellet receptacle. Subjects were maintained at 85% of their free feeding weight by rationing daily food allotment. For nine consecutive days, rats were placed in an operant chamber for a single session consisting of 30 trials in which a single CS+ or CS− cue was presented pseudo-randomly for a variable interval of 20 sec (±10 s).
Three experiments were conducted using the same autoshape procedure, two using fr lesioned animals and one using electrical stimulation (see Table 1). fr Lesion. In one group, a full contrast conditioning schedule was used; subjects were reinforced 100% of the time the CS+ was presented and 0% when the CS− was presented. A separate group of fr lesioned rats, a partial contrast conditioning schedule was used (henceforth termed CS 33 ). Subjects were reinforced 100% of the time when the CS+ was presented and 33% of the time when the alternate CS was presented. Electrical stimulation. Only the full contrast schedule was used in the electrical stimulation study. In all conditions, a retractable lever was presented at the rear of the chamber thirty sec after stimulus presentation. The rat was required to press the lever in order to advance to the next trial ensuring that the animal was equidistant from both cues at the time of presentation (Parkinson et al., 2000).

ELECTROLYTIC LESIONS OF THE FASCICULUS RETROFLEXUS
Prior to the start of autoshape training, two groups of rats were anesthetized with ketamine (80 mg/kg, i.p.) and xylazine (10 mg/kg, i.p.) and mounted in a stereotaxic apparatus using atramatic ear bars. The scalp was incised and two burr holes drilled on both sides of the skull overlying the caudal diencephalon (AP: −4.56 mm from Bregma, ML: 0.5 mm from midline, Paxinos and Watson, 2007). A concentric bipolar stimulating electrode (SNEX-100, Rhodes Medical Instruments) was lowered into the region of the fr (7.3 mm below pial surface) and a DC current (0.5 mA) was applied for 15 s using a constant current stimulator. These parameters were empirically derived to produce a lesion restricted to the diameter of the fr. Electrolytic fr lesions were made consecutively on each side of the brain. Sham control animals underwent an identical surgical procedure with the exception that no current was passed through the stimulating electrode. Four days following surgery, rats entered the behavioral arm of the study.

STIMULATION OF THE FASCICULUS RETROFLEXUS
A separate group of rats was prepared for surgery and implanted with stimulating electrodes in the fr in a manner identical to that described above for the sham control group. Once positioned, the electrodes were affixed to the skull using DenMat dental cement (Santa Maria, CA, USA). Four screws, affixed to parietal and frontal skull, served as anchors. Rats recovered in their home cages for 4 days prior to entering the behavioral arm of the study. Prior to the start of each session, rats were connected to a Med Associates PHM-152 stimulator. Electrical stimulations consisted of four 3-s trains of rectangular constant current pulses (0.25 mA; 3 Hz) and were applied during the CS+ presentation (CS+ Stim) or simultaneously with reward delivery (US Stim). A concentric bipolar stimulating electrode (SNEX-100, Rhodes Medical Instruments) was used to deliver stimulation. Stimulation parameters were based upon the approximate spontaneous firing rate of the Hb projection neurons (between 2.2 and 4.3 Hz; Weiss and Veh, 2011) and results demonstrating significant DA inhibition in the VTA and substantia nigra (97%; Ji and Shepard, 2007). A smaller current was used in the present study (0.25 vs. 0.5 mA) to confine the spread of electrical stimulation. Repeated stimulation with a longer train was used to suppress DA neurons for a longer duration of the CS or US presentation. The Sham control rats were Frontiers in Human Neuroscience www.frontiersin.org attached to the swivel but were not connected to the stimulator. CS+ Stim, US Stim, and Sham groups were run concurrently to allow joint comparison with the Sham controls.

Histology
Following the completion of experimental procedures, animals were euthanized and the brains were removed, sectioned, and stained with cresyl violet for verification of lesion and electrode placements. In order to be included in the statistical analysis, lesions were required to ablate at least 25% of the fr on a given side and, at least 75% of the combined bilateral area of the fr.

Analysis
During the session, approach to the reward (US) and cue lights (CS) were quantified by photobeam breaks. Approach to the reward dispenser during the CS+ presentation is a conditioned response that reflects anticipation of reward delivery, referred to herein as goal-tracking. Approach to the CS+ cue during its presentation is a conditioned response that reflects the transfer of incentive motivational value to the predictive cue referred to herein as sign-tracking. The number of approaches to the CS+ and CS− lights during illumination and the difference score (CS+ approaches minus CS− approaches) as well as the number of beam breaks within the reward dispenser during CS+ and CS− presentation, and the corresponding difference score, were used as dependent variables. The number of approaches and difference scores were analyzed using one-, two-, or three-way repeated measures analysis of variance (RMANOVA) as appropriate. Tests for unequal variance and all statistical analyzes were performed using JMP software (Cary, NC, USA).

Full contrast conditioning schedule
Sham and lesioned animals learned the association between stimulus presentation and reward delivery (Figure 1). In both experimental groups, repeatedly presenting the CS+ cue prior to reward delivery resulted in a gradual increase in goal-tracking and sign-tracking during presentation of the CS+ compared to the CS− (F(session × cue) = 9.9; df = 2,21, p = 0.009). Individual subjects were equally likely to exhibit goal-tracking as sign-tracking. Sign-tracking: The effects of fr lesion on conditioned approach to the CS+ and CS− as well as the difference score (CS+ approach minus CS− approach) is presented in Figure 1, panels A and B, respectively. fr lesion significantly increased sign-tracking compared to the sham group (CS+: F(lesion) = 18.44, df = 1,11, p = 0.002). Importantly, the number of approaches to the CS+ cue was similar during the first several sessions; differences between the two groups increased as they gained more exposure to CS+: US pairings (CS+: F(lesion × session) = 10.75, df = 2,10, p = 0.003). The sham and lesion groups did not differ in their approach to the CS−.
Goal-tracking: Both groups of animals learned that the CS+ predicted reward delivery as demonstrated by increased goaltracking during the CS+ presentation across training sessions (F(session) = 11.57; df = 2,17, p = 0.0007; Figures 1C,D). Neither the absolute number of entries into the pellet dispenser during the CS+ or CS− presentation nor difference score was altered by fr lesion. Figure 2 shows a representative photomicrograph of an fr lesion. Additional analysis demonstrates a functional relationship between lesion size and behavioral consequence. When animals that did not meet lesion criteria were considered (partial lesion but less than criteria), there was a significant correlation between number of approaches to the CS+ during the last three sessions of the training period and the extent of the fr lesion (n = 14 total; r 2 = 0.32; p = 0.03). There was no correlation between the extent of the fr lesion and the number of approaches to the CS− or conditioned approach to the pellet receptacle. Overall, histological evidence of lesion, significant behavioral consequence of sham vs. lesion, and significant correlation between lesion size and behavioral are all supportive evidence for the argument that amperages were sufficient to affect the fr. Interestingly, a 25% lesion to the outer portion of the fr would suggest destruction of primarily LHb output since the lateral habenular efferents compose the mantle portion of the bundle (Herkenham and Nauta, 1979). This would explain why a partial fr lesion would be effective since it would constitute a much larger portion of the reward-relevant LHb. Furthermore, since there is contralateral innervation of midbrain structures from the LHb, a partial lesion would have bilateral influence.

Partial contrast conditioning schedule
Introduction of a partially reinforced CS (33%; CS 33 ) had a marked effect on sign-tracking (Figure 2). Unlike the full contrast schedule, sham treated rats were initially unable to distinguish Frontiers in Human Neuroscience www.frontiersin.org

FIGURE 2 | Representative photomicrograph of an fr lesion.
Photomicrograph of section stained with cresyl violet. Electrolytic fr lesions were made consecutively on each side of the brain. Orange shaded region on the right highlights fr location and the arrows on outline the perimeter of the electrolytic region. Sham control animals underwent an identical surgical procedure with the exception that no current was passed through the stimulating electrode. Coronal section from Paxinos and Watson (2007).
between the CS+ and CS 33 , approaching both with equal frequency during the first three sessions of training. However, over the course of the next six sessions, sham-treated rats showed a greater number of approaches to the CS+ than the CS 33 (F(CS+ vs. CS 33 ) = 5.83; df = 1, 12, p = 0.03), reflecting the assignment of increased salience to the cue paired with a more certain outcome (Figures 3A,B). In contrast to the sham-treated rats, fr lesioned animals were clearly able to discriminate between CS+ and CS 33 during the first three sessions of training (F(CS+ vs. CS 33 ) = 7.52; df = 1, 5, p = 0.04), showing a twofold greater increase in the number of approaches to the CS+ than the CS 33 . However, during subsequent sessions, rats in the fr lesioned group approached the CS 33 with increasing frequency while approaches to the CS+ showed a trend toward a decline as demonstrated by a significant reduction in the difference score to near zero (F = 7.81; df = 2,4, p = 0.04; Figure 3B). Divergent response of sham and lesioned groups to partial contrast conditioning was further evidenced statistically by a clear divergence in their difference scores as reflected in a significant interaction as a function of training session (difference score: F(lesion × session) = 6.71; df = 2,10, p = 0.01; Figure 3B). Notably, no group differences were observed in the conditioned approach to the pellet dispenser (Figures 3C,D).

fr STIMULATION
There was no effect of fr stimulation on approach to the CS−, therefore, in order to simplify the presentation, only the difference score (number of CS+ approaches minus the number of CS− approaches) is shown in Figure 4, panels A and B. Sign-tracking: A two way repeated measures ANOVA contrasting the difference score in the three groups (CS-Stim, US-Stim and Sham across training sessions) revealed a significant main effect of stimulation group (F(stimulation) = 10.01, df = 2,20, p = 0.001). Post hoc analysis of pair-wise comparisons (CS+ vs. Sham, CS+ vs. US and Sham vs. US using two way repeated measures ANOVA) shows that stimulation during CS+ decreased sign-tracking significantly from the sham group overall (F(stimulation) = 17.36 df = 1,13, p = 0.001) and as a function of session (F(stimulation × session) = 5.30, df = 2,12, p = 0.022). Stimulation during US, while decreasing sign-tracking less dramatically, also reached statistical signficance (F(stimulation) = 4.87, df = 1,14, p = 0.045) but did Frontiers in Human Neuroscience www.frontiersin.org not differ as a function of training session. This comparison was the only contrast in which statistical analysis of the difference score and the number of conditioned approaches to the CS+ did not reach the same conclusion; conditioned approach to the CS+ was marginally significant between the two groups (F(stimulation) = 3.32, df = 1,13, p = 0.08). The contrast between the CS+ stimulation group and US stimulation group reveals a significant difference between the two conditions (F(stimulation) = 7.35, df = 1,13, p = 0.01). Except where noted above, the statistical analysis of the difference score and conditioned approach to CS+ yeild the same conclusions. Overall, these findings suggest that disruption of incentive valuation was slightly more sensitive to stimulation during the CS+ than during US. Goal-tracking: Conditioned approach to the reward dispenser was not altered by fr stimulation (Figure 4B). Both groups learned the predictive association between CS+ presentation and reward presentation.

DISCUSSION
A solid body of evidence establishes a key role for the Hb in processing aversive events and error signaling (Hikosaka, 2010;Jesuthasan, 2012;Shelton et al., 2012;Brown and Shepard, 2013;Jhou et al., 2013). The capacity of the Hb for bidirectional signaling (Matsumoto and Hikosaka, 2007) and cFos activation during incentive motivational learning (Flagel et al., 2011a) suggested a possible role in encoding positively valenced events such as reward learning and the attribution of incentive salience. This proposition was empirically tested in two complimentary studies. Eliminating the sole FIGURE 4 | Electrical stimulation of the fr during CS+ or US presentation. (A) represents the effects of fr stimulation during CS+ presentation on conditioned approach to the CS+ vs. the CS− as described by the Difference Score (same dependent variable as seen in Figures 1  and 2). (B) represents the effects of fr stimulation during US presentation. n = 8, 7, 8 for the Sham, CS+ Stim and US Stim groups. Error bars represent S.E.M. Difference between experimental groups: ***p < 0.001, **p < 0.01, *p < 0.05. output of the Hb via fr lesion significantly increased the attribution of incentive salience to a reward predictive cue as evidenced by increased sign-tracking. Conversely, producing a fictive Hb signal via fr stimulation significantly decreased the attribution of incentive salience to the reward predictive cue. Neither manipulation altered reward predictive learning as measured by goal-tracking. These results provide new evidence supporting a role for the Hb to serve as a governor, in the sense of automatic control or limitation, under conditions leading to the attribution of incentive salience.
The Hb can negatively modulate phasic DA release in brain areas centrally involved in reward processing (e.g., nucleus accumbens, NAc, medial prefrontal cortex Lecourtier et al., 2008). Therefore, loss of Hb output might be expected to enhance the phasic DA response to CS+ presentation and in turn increase the attribution of incentive salience to reward predictive cues. In agreement with this hypothesis, sign-tracking was nearly doubled when the sole output pathway of the Hb was lesioned. In accordance with the associative opponent model of appetitive-aversive conditioning, in which appetitive and aversive motivational systems are in linked opposition (Solomon and Corbit, 1974), the removal of a tonic Hb-driven inhibitory bias on DA neurons would be expected to diminish constraints on the appetitive salience attributed to reward predicting cues. The role of the Hb in governing the attribution of incentive salience is further supported by the results obtained using a partial-reinforcement schedule that required scaling between two CS's rather than an all or nothing categorization. In the sham group, substituting the CS− with a Frontiers in Human Neuroscience www.frontiersin.org partially reinforced cue (CS 33 ) initially resulted in approach to both the CS+ and CS 33 . However, a clear bias in the sign-tracking toward the CS+ was observed across sessions. Removing Hb output had an unexpected effect on task performance. Lesioned animals initially exhibited a strong preference for the CS+ over the CS 33 , a response pattern very similar to sham rats exposed to the full contrast conditioning schedule. Interestingly, as the conditioning sessions continued, conditioned approach to the CS 33 increased as approaches to the CS+ declined, eventually merging to levels that were not significantly different. Goal-tracking was not altered by the lesion. Thus, removing Hb output eliminated the incentive salience scaling that occurs during conditioning yet left the predictive value of the cues intact. Unpredictable reward delivery requires greater discrimination of incentive value than a full contrast condition and can actually increase sign-tracking to the less-predictable cue (Anselme et al., 2013). On the other hand, it is well established that in order to achieve discriminated approach in an autoshaping paradigm, a cue must be uniquely predictive of the reward Miller, 1962, 1963). By "ungoverning" incentive salience attributed to the CS 33 , lesioned rats would respond as though both cues were equivalent in their incentive value and would not develop conditioned approach to either stimulus, much as we observed in the final sessions of the partial contrast conditioning schedule. In both the full and the partial contrast conditions, the consequences of the lesion grew more prominent as a function of training; this pattern suggests selective involvement in the process that attributes incentive salience. A unique opportunity to parse the differences between goal tracking and sign-tracking was provided by studies in which the Hb was stimulated during discrete epochs of the autoshaping procedure (reward delivery, US vs. cue presentation, CS+). Had Hb activation during reward delivery diminished the hedonic impact of the reward or mimicked the neural signal that encodes 'a less than expected reward' (i.e., negative temporal difference error) a role for the Hb in strict reward prediction learning would have been implied. However, this was not the case as goal-tracking was not altered by stimulation. Rather, fr stimulation during presentation of the CS+ diminished sign-tracking as evidenced by a significant reduction in conditioned approach to the CS+. Previous studies have shown that Hb stimulation suppresses operant responding and induced place avoidance (Friedman et al., 2011;Lammel et al., 2012;Stamatakis and Stuber, 2012), changes consistent with an aversive effect. However, if fr stimulation were acting merely as an aversive stimulus, stimulation during presentation of the US would have led to a decrease in goal tracking.
Previous studies by Day et al. (2007) and Flagel et al. (2011b) provide a neurochemical framework to contextualize the present findings. These investigators have shown that during the initial autoshape training period, DA release in the NAc is phasically increased during all three components of a full contrast autoshape conditioning schedule (i.e., reward delivery, CS+ and CS− presentation). As training progresses however, DA release is limited to animals in which CS+ presentation gains incentive motivational properties. It is the phasic increase in NAc DA release during CS+ presentation that is thought to be principally responsible for the attribution of incentive salience to the cue. Further support for this is notion is the demonstration of a correlation between the development of a phasic DA response to a cocaine cue and the probability of approaching that cue (Aragona et al., 2009) and that pharmacological blockade of the DA receptor in the core of the NAc decreases the performance of an already learned signtracking conditioned response but not goal-tracking (Saunders and Robinson, 2012). It is conceivable that by stimulating Hb efferents during CS+ presentation, we suppressed VTA DA cell firing and by extension, DA release in the NAc (Lecourtier et al., 2008), thereby decreasing the attribution of incentive salience to the CS+. Athough Hb stimulation during CS+ presentation was more effective in disrupting the attribution of incentive salience, smaller changes were also observed in response to stimuli applied during reward delivery. These changes could reflect an attenuation in the increase in DA cell firing needed to stamp-in reward-relevant cues (Wise, 2004) or a decrease in DA-mediated "reboosting" (Berridge, 2007). Overall, the current studies provide evidence to support a role for the Hb and its downstream targets in encoding incentive salience. The relevant circuit likely involves connections with the RMTg, a collection of GABAergic neurons that provide an important source of feedforward inhibitory input to midbrain DA neurons (Jhou et al., 2009).
The attribution of incentive salience is critical for motivation and the pursuit of rewards. Incentive salience attribution endows a reward predicting stimulus with the capacity to initiate goaldirected behavior and, under some conditions, support behavior leading to their presentation (Rescorla and Solomon, 1967;Cools et al., 2007Cools et al., , 2009. Taken together, the results of the present studies support the contention that the Hb is involved in regulating or scaling the relative degree of incentive salience assigned to a given cue. Accordingly, pathological changes in the activity of Hb neurons could result in avolition and depression-like symptoms or the aberrant pursuit of debilitating goals.