Anticipatory pleasure predicts effective connectivity in the mesolimbic system

Convergent evidence suggests the important role of the mesolimbic pathway in anticipating monetary rewards. However, the underlying mechanism of how the sub-regions interact with each other is still not clearly understood. Using dynamic causal modeling, we constructed a reward-related network for anticipating monetary reward using the Monetary Incentive Delay Task. Twenty-six healthy adolescents (Female/Male = 11/15; age = 18.69 ± 1.35 years; education = 12 ± 1.58 years) participated in the present study. The best-fit network involved the right substantia nigra/ventral tegmental area (SN/VTA), the right nucleus accumbens (NAcc) and the right thalamus, which were all activated during anticipation of monetary gain and loss. The SN/VTA directly activates the NAcc and the thalamus. More importantly, monetary gain modulated the connectivity from the SN/VTA to the NAcc and this was significantly correlated with subjective anticipatory pleasure (r = 0.649, p < 0.001). Our findings suggest that activity in the mesolimbic pathway during the anticipation of monetary reward could to some extent be predicted by subjective anticipatory pleasure.


Introduction
Deficits in hedonic capacity, namely anhedonia, are often found in patients with schizophrenia, bipolar disorder, major depression, substance addiction, anxiety, and eating disorders (Shankman et al., 2014). Traditional symptom-based psychiatric diagnosis may not be able to capture these underlying features that cut across diagnostic entities. The recently proposed Research Domain Criteria (RDoC) aims to address this problem (Cuthbert, 2014;Cuthbert and Workgrp, 2014). The RDoC suggests researchers to focus on elemental cognitive and emotional functions, such as hedonic capacity, using various approaches ranging from behavioral performance, through brain circuits, to genes (NIMH, 2008). The Monetary Incentive Delay (MID) task (Knutson et al., 2000) and the Temporal Experience of Pleasure Scale (TEPS; Gard et al., 2006) have been suggested as appropriate instruments to examine the two components of hedonic experience, namely anticipatory and consummatory pleasure. Converging evidence suggests that the nucleus accumbens (NAcc) is a vital hedonic hotspot in anticipatory pleasure (Kringelbach and Berridge, 2009;Berridge and Kringelbach, 2013). Using the MID task, activation of the NAcc has been observed during anticipation of secondary rewards (Knutson et al., 2000(Knutson et al., , 2001. Similar results have been reported in anticipation of primary rewards, such as sucrose solution (O'Doherty et al., 2002) and social rewards (Kohls et al., 2013). The NAcc appears to play a key role in integrating information from the midbrain, the limbic system and the frontal cortex to facilitate appropriate choice and goal-directed behavior (Camara et al., 2009). The substantia nigra/ventral tegmental area (SN/VTA) also plays an important role in reward processing (Horvitz, 2000;Duezel et al., 2009). Anticipation of primary and secondary rewards, taste (O'Doherty et al., 2002), money , and happy faces  activates the SN/VTA. Lastly, the thalamus also plays a role in hedonic experience. Previous studies have reported activation of the thalamus in anticipation of rewards (Knutson et al., 2000(Knutson et al., , 2001Knutson and Greer, 2008). The thalamus integrates messages from the emotional, cognitive, and motor cortices and relays information to the frontal cortex to formulate goal-directed behavior (Haber and Calzavara, 2009). In addition, the thalamus also appears to be important in retrospective and prospective coding for predicted reward (Komura et al., 2001). The NAccnigra-thalamic circuit is involved in the regulatory function of the thalamus in reward processing (Montaron et al., 1996). Our study (Chan et al., in press), adopting the modified MID task, showed that anticipation of monetary gain activated the NAcc, the globus pallidus and the thalamus, whereas anticipation of monetary loss activated the NAcc, the thalamus and the SN/VTA. These findings further support the important roles of the SN/VTA, the NAcc and the thalamus during anticipation of rewards.
Although the functional connectivity between the NAcc, the SN/VTA and the thalamus have been a focus of recent studies in hedonic capacity (Camara et al., 2009;Haber and Calzavara, 2009;Cauda et al., 2011), to the best of our knowledge, few studies had examined the relationships between these three regions, especially the interaction between the SN/VTA and the NAcc during anticipation of rewards. Some studies have used dynamic causal modeling (DCM) to investigate reward related circuits (Alexander and Brown, 2010;Veldhuizen et al., 2011;Gonen et al., 2012;Cho et al., 2013;Yu et al., 2013). Using DCM, Veldhuizen et al. (2011) found that the anterior insular represents breaches of taste identity by receiving afferent connectivity from the ventral striatum and the inferior parietal cortex. Furthermore, by reciprocal connectivity from the amygdala, the ventral striatum plays a role in anticipating the attractability of human faces (Yu et al., 2013). Gonen et al. (2012) found that the VTA and the NAcc are related to the behavioral activation system and the NAcc represents the reward by cooperating with the dorsal medial prefrontal cortex. In addition, brain activity in the substantia nigra was found to be capable of predicting dopamine release in the NAcc during the anticipation of rewards (Schott et al., 2008). These findings suggest the vital role of the VS, especially the NAcc, in the reward circuit. In addition, the SN/VTA may also be engaged during the anticipation of rewards. However, the causal relationship between the SN/VTA and the NAcc is still unknown. In the present study, we constructed nine dynamic causal models between the SN/VTA, the NAcc and the thalamus, which contained reciprocal pathways between the SN/VTA and the NAcc, and between the NAcc and the thalamus and a non-reciprocal pathway from the SN/VTA to the thalamus (Figure 1). Taking into account the complexity of the models, the connectivity from the thalamus to the SN/VTA was excluded from consideration because this anatomical connectivity is not as clear-cut as the other five. Moreover, we measured the subjective anticipatory and consummatory pleasure of participants and correlated them with the parameters of the best-fit model.
Given these and our previous work using the MID task, we examined the reward-related network for anticipating monetary reward. We hypothesized that (1) the NAcc, the SN/VTA and the thalamus would all be activated during anticipation of monetary gain and loss; (2) the SN/VTA would exert a direct effect on the NAcc whereas the thalamus would integrate information from the SN/VTA and the NAcc.

Materials and Methods
Participants Twenty-six (11 females) healthy right-handed adolescents with a mean age of 18.6 years (sd = 1.35), a mean duration of education of 12 years (sd = 1.58) and a mean IQ estimate of 95.38 (sd = 13.56) [estimated by the shot-form of the Chinese version of the Wechsler Adult Intelligence Scale-Revised (WAIS-R; Gong, 1992)] were recruited from the community. Exclusion criteria included: a personal or family history of mental illness, a history of head injury, and a history of substance abuse. The study was approved by the Ethics Committee of the Institute of Psychology, the Chinese Academy of Sciences. Written informed consents were obtained from all participants. Each participant was recompensed with 100 CNY (China yuan) and the monetary rewards they acquired in the MID task after the completion of the study.

Temporal Experience Pleasure Scale
The TEPS was designed to measure anticipatory and consummatory pleasure (Gard et al., 2006). We used the Chinese version of the TEPS which consists of 10 items with a six-point Likert scale measuring anticipatory pleasure and 10 items measuring consummatory pleasure (Chan et al., 2012). The items measuring anticipatory pleasure capture the pleasure experienced during the anticipation of positive events, such as "When I hear about a new movie starring my favorite actor, I can't wait to see it, " whereas the items measuring consummatory pleasure capture the pleasure experienced during the consummation of positive events, such as "A hot cup of coffee or tea on a cold morning is very satisfying to me." Higher score on the TEPS indicates higher hedonic capacity. Both the original and the Chinese version of the TEPS have been shown to have satisfactory reliability and validity (Gard et al., 2006;Chan et al., 2012).

Monetary Incentive Delay Task (MID)
In this study, we used an abbreviated version of the MID task developed by Chan et al. (in press). A cue lasting 250 ms was FIGURE 1 | Candidate dynamic causal models. All nine candidate models are shown. Model 1 contains reciprocal pathways between the substantia nigra/ventral tegmental area (SN/VTA) and the nucleus accumbens (NAcc), reciprocal pathways between the NAcc and the thalamus and a non-reciprocal pathway from the SN/VTA to the thalamus. External inputs, including anticipation of monetary gain and loss, initially impact the SN/VTA, followed by the NAcc and the thalamus.
presented on a projection screen which was reflected in a small mirror fixed on the head coil of the scanner, followed by the first interval and then a blue cross that the participants were asked to quickly hit by pressing a button. Then the second interval was presented which was followed by the monetary stimuli. The cues consisted of a triangle signifying the gain condition, a square signifying the loss condition and a circle signifying the neutral condition. In the gain condition, participants could gain five monetary points if the blue cross was hit. In the loss condition, participants would lose five monetary points if the blue cross was not hit. In the neutral condition, participants would gain or lose nothing whether the blue cross was hit or not. The duration of intervals were randomized to avoid participants from anticipating the blue cross and to maintain the duration of each trial at 12 s. In addition, the duration of the blue cross was jittered around 300 ms according to the performance of each participant to maintain the accuracy at about 66%. Participants were asked to perform two runs of the task. Each run contained 30 trials, 10 gain conditions, 10 loss conditions, 10 neutral conditions, and a blank screen lasting for 8 s presented in the first instance for a dummy scan. The order of the trials was pseudorandom across participants and different between the two runs. Participants were told that the final remuneration would be 100 CNY plus the monetary points they obtained in the task. The average hit rate, the reaction time by condition and the earnings are presented in Supplementary Table S1.

Functional MRI Data Processing
All the fMRI data were analyzed with the free software Statistical Parameter Mapping 8 (SPM8, Wellcome Trust Centre for Neuroimaging, London, UK). Before pre-processing, the first four dummy scans were discarded. After slice timing correction, images were realigned to the twentieth slice of each TR. Then the mean EPI image was normalized to the single person template of the Montreal Neurological Institute. Finally all the images were smoothed with a Gaussian kernel of 5 mm full-width halfmaximum.
Based on the canonical haemodynamic response function (HRF), only the three anticipatory events: the monetary gain, the monetary loss, and the neutral condition, were included in the general linear modeling. Besides, the six parameters of head movement generated in the realignment were included in the modeling as covariates. The contrast 'gain -neutral' was designed to examine brain activities in response to monetary gain, and the contrast 'loss -neutral' was designed to examine brain activities in response to monetary loss. The contrast 'all -neutral' referred to the general effect of monetary stimuli and was used in the DCM. The contrasts of each participant were included in a onesample t-test which was set in the second-level analysis of the SPM8.
Since we aimed to examine the function of the SN/VTA, the NAcc, and the thalamus and their interaction during the anticipation of monetary stimuli, we analyzed brain activation with pre-defined regions of interest (ROI). The ROIs of the NAcc and the thalamus were selected from the Harvard-Oxford subcortical structure atlas. The ROI of the SN/VTA was adopted from a very high resolution subcortical probabilistic atlas which was quantified with a 7T structure MRI (Keuken et al., 2014). The three ROIs, were masked on the contrast 'gain -neutral' and the contrast 'loss -neutral' respectively. Small volume correction (SVC) within an 8-mm radius sphere was applied. The statistical threshold was set as familiar-wise-error (FWE) correction with p < 0.05.

Dynamic Causal Modeling
Before the procedure, the time courses of the SN/VTA, the NAcc, and the thalamus were extracted from the contrast 'all -neutral' of all participants. The ROIs were defined as the overlaps between the masks used in the ROI analysis with an 8-mm radius sphere centered around the peak points activated in the SN/VTA, the NAcc, and the thalamus, respectively. The statistical threshold was set as uncorrected p < 0.05.
We constructed nine dynamic causal models. The complete model, Model 1, contained reciprocal connectivity between the SN/VTA and the NAcc, reciprocal connectivity between the NAcc and the thalamus, and a non-reciprocal connectivity from the SN/VTA to the thalamus. From Model 1, one or two connectivity was subtracted in different ways to form eight other models (Figure 1). The right SN/VTA, the right NAcc, and the right thalamus were included in the model for their stronger activations than their left-sided counterparts ( Table 1). Using the SPM8, a HRF was constructed, which contained the event 'all, ' 'gain, ' and 'loss'. The event 'all' was defined as an input at the SN/VTA which is axiomatically considered a dopamine-rich region projecting to the terminals of the NAcc (Duezel et al., 2009;Haber and Knutson, 2010;Cauda et al., 2011). For the exploratory aim of this study and the unclear effect of valence to the connectivity of the reward circuit, the event 'gain' and 'loss' were, respectively, defined as perturbations to all the intrinsic connectivity, or the edges, of the models. A random-effect analysis of Bayesian Model Selection (BMS) was applied to identify the best-fit model with the highest exceedance probability (Friston et al., 2003;Stephan et al., 2010). Then the endogenous and perturbed parameters of the bestfit model were imported into the Predictive Analytics Software 18.0 (PASW 18.0) for significance testing. Bonferroni correction was applied to correct for multiple comparison. Finally, all the parameters were correlated with the total, the anticipatory and the consummatory subscale scores of the TEPS using Pearson Correlation.

Regions of Interest Analysis
The bilateral NAcc, the bilateral SN/VTA and the bilateral thalamus were all significantly activated to the contrasts 'gainneutral, ' 'loss -neutral, ' and 'all -neutral' ( Table 1).

Dynamic Causal Modeling
The BMS identified Model 8 as the best-fit model with the highest exceedance probability during the anticipation of monetary stimuli (Figures 2 and 3). The endogenous connectivity of Model 8 contained two causal pathways from the SN/VTA to the NAcc   Table 2).
As for the modulation parameters caused by external experimental stimuli in the Model 8, both monetary 'gain' and 'loss' modulated the connectivity from the SN/VTA to the NAcc, from the SN/VTA to the thalamus and from the NAcc to the thalamus ( Table 2).

Correlation between Subjective Pleasure Experience and Modeling Parameters
The mean total TEPS score of the participants was 78.18 ± 12.72, while the mean anticipatory and consummatory subscale scores were 41.78 ± 6.9 and 36.4 ± 7.39, respectively. We found  significant positive correlation between the modulation by 'gain' events on the nigrostriatal pathway from the SN/VTA to the NAcc with anticipatory subscale score (r = 0.649, p < 0.001) and total score on the TEPS (r = 0.555, p = 0.003). The former correlation remained significant after Bonferroni correction (Figure 4).

Discussion
In the present study, using the MID Task, we observed activation of the SN/VTA, the NAcc, and the thalamus in healthy adolescents during anticipation of both monetary gain and loss. We found that the SN/VTA projects two causal pathways to the NAcc and the thalamus. Importantly, the causal connection from the SN/VTA to the NAcc was strengthened by the anticipation of monetary gain. This modulation was also positively correlated with subjective pleasure ratings, especially anticipatory pleasure. Consistent with results from previous fMRI studies (Knutson et al., 2000(Knutson et al., , 2001Breiter et al., 2001;Zink et al., 2004), the NAcc is activated during anticipation of both reward and punishment in the present study. Earlier research in animals highlighted the role of the NAcc in reward anticipation, namely the "wanting" component of reward processing (Berridge and Robinson, 1998;Berridge, 2003). However, other animal studies had stressed the role of the NAcc in reward learning because local dopamine release was associated with novel stimuli and predictive cues indicating forthcoming reward or punishment (Schultz et al., 1997;Schultz, 2007). In human studies, the NAcc is conceptualized as a hedonic hotspot important in pleasure processing (Knutson and Greer, 2008;Kringelbach and Berridge, 2009). Knutson and Greer (2008) reviewed fMRI studies employing the MID task and suggested that the NAcc is involved in anticipating positive events and is associated with subjective pleasure and approaching behavior. Electrical stimulation of the NAcc in rodents and deep brain stimulation of the NAcc in humans both promote approaching behavior (Kringelbach and Berridge, 2009;Berridge and Kringelbach, 2013). Although a previous study had demonstrated that anticipation of reward rather than punishment activated the NAcc (Sabatinelli et al., 2007), activation of the NAcc had also been reported in anticipation of aversive stimuli in another study (Zink et al., 2004). Our findings lend support to the important role of NAcc in the processing of both salient information and pleasure during anticipation of rewards.
The role of the SN/VTA and the thalamus in reward processing is less controversial compared to the function of the NAcc discussed above. O'Doherty et al. (2002) found that the SN/VTA is activated during anticipation of glucose, whereas its role in anticipation of secondary rewards such as monetary stimuli is less clear. While Knutson et al. (2000Knutson et al. ( , 2001 did not observe activation of the SN/VTA during anticipation of monetary stimuli, Breiter et al. (2001) identified activation in the VTA during anticipation of monetary rewards which was similar to our findings. Moreover, the VTA has been found to be activated in response to beautiful faces, which suggested that the midbrain may play an important role in positive social reward . Activation of the SN/VTA to both monetary gain and loss identified in this study stresses the role of SN/VTA in reward processing.
Activation of the thalamus during anticipation of monetary gain and loss in the present study is consistent with results from a previous review (Knutson and Greer, 2008). While the thalamus is regarded as a center for information gathering and integration (Haber and Calzavara, 2009;Kohls et al., 2013), a previous study of electrical recording in rats suggested that the thalamus is also involved in retrospective and prospective coding to reward (Komura et al., 2001). In addition, the thalamus appears to be responsive to salient sensory information during the anticipation of incentives (Cho et al., 2013).
We also found a causal pathway from the SN/VTA to the NAcc during anticipation of monetary stimuli. To the best of our knowledge, few studies have investigated the causal relationship between the midbrain and the VS. A previous study on functional connectivity has reported that spontaneous activation of the NAcc correlated with reward-related brain circuits including the orbitofrontal cortex, the globus pallidus, the thalamus, the midbrain, the amygdala and the insular (Cauda et al., 2011). In addition, connectivity between the mesolimbic system and cortical areas appears to be altered in developmental conditions (Camara et al., 2009). The close relationship between mesolimbic connection and reward processing may be related to local dopaminergic metabolism. Schott et al. (2008) found that activation of the SN/VTA is associated with dopamine release in the NAcc. Moreover, Knutson and Gibbs (2007) identified that dopamine release in the NAcc activates postsynaptic D1 receptors which further induces activation in the NAcc during anticipation of reward. Both studies in functional connectivity and neurotransmitter stressed the role of mesolimbic connection in anticipation of positive events and pleasure, but the causal direction between the midbrain and the VS in this process is not clear. The causal connectivity from the SN/VTA to the NAcc identified in the present study is not only consistent with previous functional connectivity studies, but also provides new information regarding causal relationships in the mesolimbic pathway. Moreover, our findings support the role of the thalamus in integrating information from emotional, cognitive and motor cortical and subcortical areas to facilitate approaching and goal-directed behaviors (Haber and Calzavara, 2009;Haber and Knutson, 2010). In the best-fit model of our study, the SN/VTA projects an excitatory pathway to the thalamus, whereas the NAcc projects an inhibitory pathway to the thalamus. In another previous study which investigated brain circuits during anticipation of monetary incentives using DCM, Cho et al. (2013) found that the thalamus modulated the NAcc through the thalamus-to-NAcc and the thalamus-to-insula-to-NAcc connections which was different from our findings. We believe that the different findings may be related to the choice of the driven regions adopted. In the present study, the SN/VTA was chosen as the driven region, which is upstream to the dopaminergic complex in the NAcc (Schott et al., 2008;Duezel et al., 2009), whereas the thalamus was chosen as the driven region by Cho et al. (2013). However, a connection from the NAcc to the thalamus was also identified in Cho et al.'s (2013) study, which is similar to our findings. We found that both monetary 'gain' and 'loss' induced perturbation in all three connectivities in Model 8, namely the SN/VTA-to-thalamus connectivity, the SN/VTA-to-NAcc connectivity, and the NAccto-thalamus connectivity. In contrast to the study by Cho et al. (2013), which found different patterns of perturbation between monetary 'gain' and 'loss' in the thalamus-insula-NAcc circuit, our findings revealed a similar pattern regardless of valence in the SN/VTA-NAcc-thalamus network. The ROIs adopted and the driving ROI chosen may both cause the difference between the present and previous findings. The SN/VTA adopted here appears to play an elementary role in reward processing which may not be sensitive to the valence of the reward.
More interestingly, the modulation by monetary gain on the connectivity from the SN/VTA to the NAcc was correlated with anticipatory pleasure experience in the present study. The MID task and the TEPS are tools suggested by the RDoC for measuring anticipatory and consummatory pleasure in fMRI and behavioral paradigms, respectively, (NIMH, 2008), but few studies have investigated the underlying neural mechanism of the two instruments. People with schizophrenia spectrum disorders report dampened subjective anticipatory pleasure, while their consummatory pleasure is relatively preserved (Kring and Caponigro, 2010;Kring et al., 2011). In addition, previous studies have identified that people with schizophrenia showed reduced activation in the NAcc when anticipating monetary stimuli (Juckel et al., 2006(Juckel et al., , 2012Walter et al., 2009). These results suggest that the two instruments seem to capture similar underlying neural mechanisms and our findings corroborated this in healthy adolescents. The connection from the SN/VTA to the NAcc in adolescents who reported higher anticipatory pleasure is more easily perturbed by positive events. This phenomenon may reflect inherent dopaminergic metabolism in the mesolimbic system and this hypothesis merits further research.
This study has several limitations. First, we only focused on the causal network in the mesolimbic system and did not investigate other important reward-related circuits such as the mesocortical system. The second limitation is that the adopted dopaminergic midbrain area is the SN rather than the VTA, but the boundary between the SN and the VTA in human is difficult to distinguish (Duezel et al., 2009). Furthermore, laterality was not taken into consideration in this study. We only focused on the right hemisphere which showed higher activation than the left. Finally, only nine models were tested in this study. Future identification of other relevant connectivity or models is needed.

Conclusion
Anticipation of both monetary gain and loss activated the NAcc and other reward-related areas, namely the SN/VTA and the thalamus. The SN/VTA projects causal pathways to the NAcc and the thalamus, while the thalamus integrates information from the SN/VTA and the NAcc. Anticipatory pleasure appears to predict the susceptibility of causal connection from the SN/VTA to the NAcc to positive events. The present findings also lend support to the applicability of the RDoC in research.