The delay effect on outcome evaluation: results from an event-related potential study

Introduction

Time is an important dimension when assessing the value of a reward in a decision-making situation because when delivery of a reward is delayed, an individual’s valuation of a future reward declines (Mazur, 1987). This phenomenon is generally referred to as temporal discounting (Samuelson, 1937; Ainslie, 1975). As temporal discounting is ubiquitous in daily decision-making and impairments in temporal discounting characterize a range of psychiatric conditions (i.e., substance abuse, addiction and attention-deficit hyperactivity disorder), this topic has shown rapid progress in the past few years. Brain research has provided insight into the neural mechanisms underlying temporal discounting. McClure et al. (2004) proposed that two separate neural systems value immediate and delayed rewards. Specifically, a limbic system (β) is thought to place special weight on immediate rewards, whereas a more cognitive, prefrontal-cortex-based system (δ) is more involved in patient choices. However, single valuation account holds that the values of both immediate and delayed rewards are represented in a unitary system encompassing the ventral striatum, medial prefrontal cortex, and posterior cingulate cortex (Kable and Glimcher, 2007, 2010). Nevertheless, in the self-control account, values are assumed to be represented in structures such as the ventromedial prefrontal cortex (vmPFC) but are subject to top-down modulation by prefrontal control regions such as the lateral PFC (Hare et al., 2009; Figner et al., 2010).

However, the above research have been predominantly concerned with discounting future gains rather than on losses, although losses may seem as important as gains, as many of the most-discussed real-world phenomena relating to intertemporal choice involve aversive outcomes (Harris, 2012). Behavioral evidence suggests that people usually discount delayed losses less steeply than delayed gains (Thaler, 1981; Loewenstein, 1987; MacKeigan et al., 1993; Read, 2004). This phenomenon, termed the sign effect (Loewenstein, 1987), is rarely tested with electrophysiology. Furthermore, while fMRI studies have provided abundant evidence of the brain mechanism of temporal discounting, the time course of cortical activation has not been studied precisely.

The event-related potential (ERP) techniques with high temporal resolution have provided critical temporal information for the neural correlates of temporal discounting. However, it remains unclear whether temporal information and valence could be encoded and integrated in the process of outcome evaluation. Feedback-related negativity (FRN) is an important ERP component implicated in reward processing (Gehring and Willoughby, 2002). The FRN, generated by the anterior cingulate cortex (ACC; Gehring and Willoughby, 2002; Holroyd and Coles, 2002), has been conceptualized as a negative deflection around 250 ms post-onset of the feedback stimulus (e.g., Holroyd and Coles, 2002; Hajcak et al., 2006). The FRN is more pronounced for negative feedback associated with unfavorable outcome, such as incorrect response or monetary loss, than for positive feedback (Miltner et al., 1997; Gehring and Willoughby, 2002; Yeung and Sanfey, 2004; Holroyd et al., 2006; Goyer et al., 2008). Weinberg et al. (2012) examined the effect of feedback delay on reward processing, but in their study delay referred to delayed feedback following prior action after a short delay of 6 s, while many of human choices only pay off after months or even years. Blackburn et al. (2012) examined electrophysiological correlates involved in the detection and evaluation of immediate and delayed monetary outcomes. However, in their study participants processed only temporal information in outcome evaluation, while in most environments temporal discounting involves both valence and temporal information. Another study, mainly focused on individual difference, found that FRN classifies outcomes in a binary manner, with immediate non-reward, delayed non-reward, and delayed reward all perceived as unfavorable outcomes (Cherniawsky and Holroyd, 2013). Nevertheless, because they used a non-reward (1 penny) as negative feedback, it is still unclear whether FRN could distinguish losses with different time delays.

The present study comprehensively investigated the neural basis of temporal discounting to verify whether temporal information and valence could be integrated and encoded in the FRN and P300. Using ERP technique, we utilized a relatively straightforward gambling paradigm in which participants attempted to guess which of two pictures hid a monetary reward. Feedback indicating whether participants gain or lose money was presented after each response. There were four kinds of outcome feedback: gain 10 RMB immediately, gain 10 RMB a month later, lose 10 RMB immediately, and lose 10 RMB a month later (RMB, the Chinese currency, is the abbreviation of Ren Min Bi, and rough estimated value for 10 RMB is 1.634 dollars). According to previous research on temporal discounting and the sign effect, the subjective value of immediate gains was larger than that of delayed gains, while for losses, this difference tended to be smaller. Therefore, we hypothesized that FRN would be sensitive to valence and temporal delay. The difference between gain and loss in the time range of the FRN would be substantial for immediate rewards, but the difference would be reduced for delayed rewards. In addition, we examined whether the P300, the most positive peak in the 250–450 ms time window post-onset of feedback, would be impacted by delay.

Materials and Method

Participants

Twenty undergraduates from Beijing Normal University were recruited online. The mean age of the participants was 21 ± 1.63 years, ranging between 19 and 25 years. All participants were right handed and had normal or corrected-to-normal vision, and had no history of neurological, psychiatric, or cognitive disorders. Informed consent was obtained from each participant. This study was approved by the Ethics Committee of State Key Laboratory of Cognitive Neuroscience and Learning at Beijing Normal University.

Stimuli and Procedures

The experiment had a two (temporal delay: immediate vs. delayed) by two (outcome valence: gain vs. loss) factorial design. Four experimental conditions were composed of four types of outcome feedback: gain 10 RMB now, gain 10 RMB a month later, lose 10 RMB now, and lose 10 RMB a month later. Each feedback consisted of a photo of the 10 RMB (The photo was either colored or in black-and-white, indicating gain or loss of money respectively), with the time of reward delivery written below: “Now” or “1 month”.

The time course of a trial is illustrated in Figure 1. At the beginning of each trial, participants were first presented with a red cross for 800 ms at the center of the screen, then two photos of landscapes were presented, and participants were required to select one of them by pressing corresponding keys. The two pictures were presented on the screen until the participant made a choice. They needed to press “F” if they chose the picture on the left and “J” if they chose the picture on the right. The selected photo was highlighted by a yellow border. After a random time interval (500–1000 ms), the feedback of winning or losing was shown for 1000 ms. The next trial began 1 s after the offset of the feedback.

FIGURE 1

FIGURE 1. Time course of stimulus presentation in the gambling task. Participants were informed about the four possible outcomes: gain 10 RMB now, gain 10 RMB a month later, lose 10 RMB now, and lose 10 RMB a month later (bottom panel). They then chose one of two cards by pressing the corresponding button. At the end of each trial, participants were informed about their outcome (top panel).

Before the ERP recordings, participants were provided with verbal instructions and a training session to familiarize with the procedure. The formal experiment consisted of four blocks of 60 trials per block. On each trial, four options were available (i.e., gain money now, lose money now, gain money later, or lose money later). Between the blocks, participants were provided with a self-timed rest period. They were told that they could adopt whatever strategies they wanted to use to maximize their rewards. Unbeknownst to the participants, feedback was predetermined and randomized, with each kind of feedback appearing in equivalent numbers of 60 trials. Participants were assured that they would get 40 RMB as basic payment (20 RMB for today and 20 RMB for 1 month later) and that additional monetary reward would be paid according to their performance, with the immediate rewards to be delivered following completion of the experiment, and the future rewards to be sent via a check from the university 1 month later; however, in the end all participants were paid 50 RMB (about $8.17) immediately in cash.

ERP Recording and Analysis

EEGs were recorded from 32 scalp sites using tin electrodes mounted in an elastic cap (NeuroScan Inc., USA) according to the international 10–20 system. The vertical electrooculogram (VEOGs) were recorded from electrodes located above and below the left eye. The horizontal EOG (HEOG) was recorded by electrodes placed 1.5 cm lateral to the left and right external canthi. All EEGs and EOGs were referenced online to the left mastoid and off-line algebraic re-referenced to the average of the left and right mastoids. A ground electrode was placed on the medial frontal aspect. The interelectrode impedances were maintained below 5 kΩ. The EEG and EOG were amplified (bandpass 0.05–100 Hz) and digitized online with a sampling frequency of 500 Hz.

The EEG data were preprocessed with Brain Vision Analyzer software. Ocular artifacts were corrected with an eye-movement correction algorithm, which employs a regression analysis in combination with artifact averaging (Gratton et al., 1983). A 1000 ms epoch of data, extending from 200 ms prior to 800 ms following the onset of each feedback stimulus, was extracted from the continuous data file for analysis, with the 200 ms pre-stimulus EEG activity used for baseline correction. All trials in which EEG voltages exceeded a threshold of ±90 μV during the recoding epoch were excluded from analysis. The EEG data were low-pass filtered using a 20 Hz low-pass (24 dB octave roll off), and were baseline-corrected by subtracting the average activity of that electrode during the baseline period from each sample.

The ERP components that were analyzed were FRN and P300. Time windows were selected for analysis based on visual inspection of the waveforms and their scalp distributions (Figures 2 and 3). For the FRN, we measured the mean amplitude in the time window of 230–330 ms post-onset of the feedback. To minimize the effects of overlap between ERP components, most notably the P3, we created difference waves by subtracting ERPs elicited by loss feedback from ERPs associated with gain feedback and used the mean values of the difference waves in the 230–430 ms time window as measures of the FRN effect (Holroyd and Krigolson, 2007). For the P300, we took the peak amplitudes in the time window of 250–450 ms. We focused on the four electrode locations in the midline (Fz, FCz, Cz, and Pz), where these components had been most pronounced in previous studies. Separate repeated measures analyzes of variance (ANOVAs) were conducted for the two potentials with three within-participant factors: electrode location (Fz, FCz, Cz, and Pz), temporal delay (immediate vs. delayed) and outcome valance (gain vs. loss). In all analyzes, the Greenhouse-Geisser correction was applied when the assumption of sphericity was violated.

FIGURE 2

FIGURE 2. Grand-averaged ERPs and difference waves for immediate rewards and delayed rewards at Fz, FCz, Cz, and Pz. Difference waves were created by subtracting the feedback-evoked ERP associated with negative feedback from the ERP associated with positive feedback. The solid black line represents difference waves for immediate rewards, while the dotted black line represents difference waves for delayed rewards.

Results

Feedback-Related Negativity

Figures 2 and 3 indicate that, in accordance with previous studies reporting FRN, negative feedback elicited a negative-going waveform that reached its maximum over frontocentral scalp Positions. For the FRN amplitude, a three-way repeated measures ANOVA with factors of feedback valence (loss vs. gain), temporal delay (delayed vs. immediate), and electrode location (Fz, FCz, Cz, and Pz) revealed a significant main effect of electrode location (F_[3,48] = 6.817, p < 0.01). A pair-wise comparison confirmed that FRN at Fz was more negative (10.277 μV ± 1.109) than at other electrode locations. The main effect of feedback valence (F_[1,16] = 30.018, p < 0.001) was also significant, with losing decisions eliciting a more negative FRN (11.125 μV ± 1.162) than winning decisions (13.540 μV ± 1.207). Importantly, there was a significant interaction between valence and delay (F_[1,16] = 7.310, p < 0.05). A simple effect analysis showed that FRN amplitude was more negative for delayed gains (12.834 μV ± 1.193) than for immediate gains (14.247 μV ± 1.285; F_[1,16] = 6.20, p < 0.05), but no such difference was observed in the loss condition (F_[1,16] = 0.55, p = 0.471).

FIGURE 3

FIGURE 3. Scalp topographies of the difference waves between ERP responses to the loss vs. gain outcomes averaged for FRN in the time range of 230–330 ms.

To further investigate this interaction effect, ERP amplitude of difference waves were analyzed on the four electrode locations. A paired samples t-test showed that, at Fz, the difference wave amplitude after immediate outcomes was significantly different from zero (-3.745 μV ± 2.470, t₍₁₆₎ = -6.236, p < 0.001); the FRN effect following delay outcomes was also significant (-1.944 μV ± 2.012, t₍₁₆₎ = -3.984, p < 0.01); and the sizes of the two difference waves were significantly different from each other (t₍₁₆₎ = -2.947, p < 0.01). The FRN effect was more remarkable for immediate outcomes, and delay reduced the valence effect of FRN. Similar results were obtained when we analyzed data from FCz(-3.760 μV ± 2.578, t₍₁₆₎ = -6.013, p < 0.001; -1.818 μV ± 1.945, t₍₁₆₎ = -3.853, p < 0.01; t₍₁₆₎ = -3.100, p < 0.01) and Cz(-3.550 μV ± 2.913, t₍₁₆₎ = -5.025, p < 0.001; -1.445 μV ± 1.990, t₍₁₆₎ = -2.995, p < 0.01; t₍₁₆₎ = -2.825, p < 0.05).

To confirm that the amplitude of the FRN was not confounded by overlap with the P300, we followed the method of Holroyd and Krigolson (2007) to carry out paired samples t-tests on the amplitude of difference wave at Fz and Pz (where the peak of the P300 is usually localized). The result indicated that the difference waves for immediate outcomes were significantly larger at Fz than Pz (-3.735 μV vs. -2.169 μV, t₍₁₆₎ = -4.652, p < 0.001). Similar results were obtained for delayed outcomes (-1.944 μV vs. -0.90 μV, t₍₁₆₎ = -2.867, p < 0.05).

P300

Similar analyzes were conducted for the peak values of the P300, which yielded a significant main effect of electrode location (F_[3,48] = 5.363 p < 0.01). Post-hoc analyzes showed a more positive P300 at Cz (18.904 μV ± 1.294) and Pz (18.310 μV ± 1.334) than that Fz (16.036 μV ± 1.086), p < 0.05. The main effect of valence was significant (F_[1,16] = 37.359, p < 0.001). The amplitude of P300 following gains (19.103 μV ± 1.185) was larger than that following losses (16.781 μV ± 1.142). There were no other significant effects (all ps > 0.239).

Discussion

Despite a wealth of research on temporal discounting, it is still poorly understood whether temporal information and valence could be encoded and integrated in the process of outcome evaluation as reflected by the FRN and P300 component. With the ERP technique, the present study employed a simple gambling task to investigate how temporal delay affects the brain activity in outcome evaluation. Valence and delay time were manipulated to explore how temporal information and valence were integrated and encoded in the FRN and P300. Four types of outcome were presented to participants: immediate gains, immediate losses, delayed gains, and delayed losses.

As expected, FRN was found to be larger in response to unfavorable outcomes. Regardless of time delay, losses were associated with a larger FRN than gains (Nieuwenhuis et al., 2004; Yeung and Sanfey, 2004; Holroyd et al., 2006; Hewig et al., 2007; Goyer et al., 2008). Interestingly, immediate loss and delayed loss elicited comparable FRNs, and delayed loss elicited a larger FRN than delayed gain. Moreover, delayed gain elicited a more negative FRN than immediate gain. This gradually decreasing amplitude of FRN suggests temporal information was integrated and reflected in FRN in the early stage of outcome evaluation. Furthermore, the graded coding of outcome in the FRN shed light on the neural basis of performance monitoring and outcome evaluation. Most previous researchers mainly focused on objective factors, such as the influence of the feedback valence, magnitude, and probability on the amplitude of the FRN (Yeung and Sanfey, 2004; Holroyd et al., 2006; Bellebaum et al., 2010; Kreussel et al., 2012). In recent years, some researchers have taken into account the subjective value of rewards by including social context and personality traits, such as social comparison (Boksem et al., 2011), interpersonal relationship (Fukushima and Hiraki, 2006, 2009; Itagaki and Katayama, 2008; Kang et al., 2010; Ma et al., 2011) and anxiety (Gu et al., 2010a,b). This previous research did indicate that the subjective value of the outcome could modulate FRN in response to our own or others’ performance or monetary outcomes.

Our findings provide further evidence that the subjective value rather than the objective value was encoded by ACC (indexed by the FRN) at an early stage. According to the motivational/affective hypothesis of the FRN (Gehring and Willoughby, 2002; Masaki et al., 2006), FRN reflects the motivational/affective significance of outcomes. If the FRN only encodes objective value, FRN would not be sensitive to time delay. However, we did observe a larger FRN elicited by delayed loss compared to delayed gain, whose amplitude was larger than that of immediate gain, which provided further evidence that reward was encoded by FRN in a fine-grained pattern according to the subjective value of outcome (Hajcak et al., 2007; Holroyd and Krigolson, 2007; Bellebaum and Daum, 2008; Bellebaum et al., 2010; Gu et al., 2011; Luo et al., 2011), rather than a binary pattern according to the good/bad objective value of outcome (Yeung and Sanfey, 2004; Toyomaki and Murohashi, 2005; Hajcak et al., 2006; Holroyd et al., 2006). The observed temporal delay effect is also consistent with previous neuroimaging studies in which the subjective value of delayed monetary rewards was tracked by reward processing areas of the brain, and delays of future rewards decreased activation in mesolimbic dopamine projection areas implicated in reward processing (Kable and Glimcher, 2007; Pine et al., 2009).

Moreover, the results demonstrated a greater FRN effect between gains and losses following immediate outcomes than that following delayed outcomes. The greater FRN effect in the immediate condition is in line with temporal discounting and the sign effect. This suggests the presence of temporal discounting, in which the subjective value of immediate rewards was larger than that of delayed rewards. In the gain conditions, immediate receipt is attractive and delaying receipt needs to be compensated. Thus, immediate gains are more preferable; whereas in loss conditions, immediate receipt is unattractive and people should be willing to pay a premium to put such events off, making delayed losses more preferable. According to model of sign effect, discounting rates for losses are typically far smaller than those for gains (Thaler, 1981). For example, delayed +10 RMB in the current experiment might be valued as +7 RMB immediate. As for future losses, delayed -10 RMB might be valued as -9 RMB immediate. Consequently, the difference between immediate gain and immediate loss is 20, which is more than that between delayed gain and delayed loss (16). Therefore, a larger difference between the subjective value of gain and loss in the immediate condition caused a greater FRN effect.

It is worth noting that we did not observe a differential FRN in the loss condition. One possible reason is the notion of reward positivity. Holroyd et al. (2008) proposed that, rather than a negativity in response to losses, activity in the time range of the FRN may reflect an underlying positivity in response to rewards that is reduced or absent in response to losses. During recent years, more and more studies have found evidence for this notion (Eppinger et al., 2009; San Martin et al., 2010; Foti et al., 2011). Therefore, this hypothesis might help us to explain the existence of modulation associated with gains, rather than losses. Another possible reason might be that the manipulation of time delay (1 month) in the current study is too short, as sign effect predicts a smaller difference between immediate loss and delayed loss, which might not be enough to cause a difference reflected in FRN. More specifically, given a longer time delay, the difference between immediate loss and delayed loss would be larger. Future studies will be needed to verify whether FRN could distinguish immediate losses from delayed losses when the delay time was manipulated gradually.

P300 was found to be modulated by valence, with a larger P300 in response to gains, which replicated previous studies (Hajcak et al., 2005, 2007; Wu and Zhou, 2009). Given that the P300 is widely believed to be related to processes of attentional allocation and to high-level motivational/affective evaluation (Yeung and Sanfey, 2004; Nieuwenhuis et al., 2005), it is possible that more attentional resources (Gray et al., 2004; Linden, 2005) are devoted to outcomes that benefit oneself.

To conclude, the present results are the first demonstration that temporal discounting and sign effect could be encoded in the FRN in the early stage of outcome evaluation, which add important neuroscience evidence of temporal discounting and deepen our understanding of outcome evaluation. The integration of valence and temporal information, which was reflected by FRN, also suggests that FRN works in a graded pattern with regard to subjective value of outcome, rather than a dichotomous pattern with regard to objective value of outcome. Future studies would investigate whether FRN could distinguish immediate losses from delayed losses when the delay time was manipulated gradually.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This work was funded by Open Research Fund of the State Key Laboratory of Cognitive Neuroscience and Learning, by the National Natural Science Foundation of China (31000504,30930031), by the National Basic Research Program of China (No2011CB711000), and the Major Project of National Social Science Foundation (12&ZD228). We sincerely thank Yuru Wang for help in running the experiment and reviewers for their constructive comments on early versions of the manuscript.

References

Ainslie, G. (1975). Specious reward: a behavioral theory of impulsiveness and impulse control. Psychol. Bull. 82, 463–496. doi: 10.1037/h0076860

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Bellebaum, C., and Daum, I. (2008). Learning-related changes in reward expectancy are reflected in the feedback-related negativity. Eur. J. Neurosci. 27, 1823–1835. doi: 10.1111/j.1460-9568.2008.06138.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Bellebaum, C., Polezzi, D., and Daum, I. (2010). It is less than you expected: the feedback-related negativity reflects violations of reward magnitude expectations. Neuropsychologia 48, 3343–3350. doi: 10.1016/j.neuropsychologia.2010.07.023

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Blackburn, M., Mason, L., Hoeksma, M., Zandstra, E. H., and El-Deredy, W. (2012). Delay discounting as emotional processing: an electrophysiological study. Cogn. Emot. 26, 1459–1474. doi: 10.1080/02699931.2012.673478

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Boksem, M. A. S., Kostermans, E., and De Cremer, D. (2011). Failing where others have succeeded: medial frontal negativity tracks failure in a social context. Psychophysiology 48, 973–979. doi: 10.1111/j.1469-8986.2010.01163.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Cherniawsky, A. S., and Holroyd, C. B. (2013). High temporal discounters overvalue immediate rewards rather than undervalue future rewards: an event-related brain potential study. Cogn. Affect. Behav. Neurosci. 13, 36–45. doi: 10.3758/s13415-012-0122-x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Eppinger, B., Mock, B., and Kray, J. (2009). Developmental differences in learning and error processing: evidence from ERPs. Psychophysiology 46, 1043–1053. doi: 10.1111/j.1469-8986.2009.00838.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Figner, B., Knoch, D., Johnson, E. J., Krosch, A. R., Lisanby, S. H., Fehr, E., et al. (2010). Lateral prefrontal cortex and self-control in intertemporal choice. Nat. Neurosci.13,538–539. doi: 10.1038/nn.2516

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Foti, D., Weinberg, A., Dien, J., and Hajcak, G. (2011). Event-related potential activity in the basal ganglia differentiates rewards from nonrewards: temporospatial principal components analysis and source localization of the feedback negativity. Hum. Brain Mapp. 32, 2207–2216. doi: 10.1002/hbm.21182

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Fukushima, H., and Hiraki, K. (2006). Perceiving an opponent’s loss: gender-related differences in the medial-frontal negativity. Soc. Cogn. Affect. Neurosci. 1, 149–157. doi: 10.1093/scan/nsl020

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Fukushima, H., and Hiraki, K. (2009). Whose loss is it? Human electrophysiological correlates of non-self reward processing. Soc. Neurosci. 4, 261–275. doi: 10.1080/17470910802625009

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gehring, W. J., and Willoughby, A. R. (2002). The medial frontal cortex and the rapid processing of monetary gains and losses. Science 295, 2279–2282. doi: 10.1126/science.1066893

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Goyer, J. P., Woldorff, M. G., and Huettel, S. A. (2008). Rapid electrophysiological brain responses are influenced by both valence and magnitude of monetary rewards. J. Cogn. Neurosci. 20, 2058–2069. doi: 10.1162/jocn.2008.20134

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gratton, G., Coles, M. G. H., and Donchin, E. (1983). A new method for off-line removal of ocular artifact. Electroencephalogr. Clin. Neurophysiol. 55, 468–484. doi: 10.1016/0013-4694(83)90135-9

CrossRef Full Text

Gray, H. M., Ambady, N., Lowenthal, W. T., and Deldin, P. (2004). P300 as an index of attention to self-relevant stimuli. J. Exp. Soc. Psychol. 40, 216–224. doi: 10.1016/S0022-1031(03)00092-1

CrossRef Full Text

Gu, R., Ge, Y., Jiang, Y., and Luo, Y.-J. (2010a). Anxiety and outcome evaluation: the good, the bad and the ambiguous. Biol. Psychol. 85, 200–206. doi: 10.1016/j.biopsycho.2010.07.001

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gu, R., Huang, Y. -X., and Luo, Y.-J. (2010b). Anxiety and feedback negativity. Pchophysiology 41, 961–967. doi: 10.1111/j.1469-8986.2010.00997.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gu, R., Lei, Z., Broster, L., Wu, T., Jiang, Y., and Luo, Y.-J. (2011). Beyond valence and magnitude: A flexible evaluative coding system in the brain. Neuropsychologia 49, 3891–3897. doi: 10.1016/j.neuropsychologia.2011.10.006

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hajcak, G., Holroy, C. B., Moser, J. S., and Simons, R. F. (2005). Brain potentials associated with expected and unexpected good and bad outcomes. Psychophysiology 42, 161–170. doi: 10.1111/j.1469-8986.2005.00278.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hajcak, G., Moser, J. S., Holroyd, C. B., and Simons, R. F. (2006). The feedback-related negativity reflects the binary evaluation of good versus bad outcomes. Biol. Psychol. 71, 148–154. doi: 10.1016/j.biopsycho.2005.04.001

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hajcak, G., Moser, J. S., Holroyd, C. B., and Simons, R. F. (2007). It’s worse than you thought: the feedback negativity and violations of reward prediction in gambling tasks. Psychophysiology 44, 905–912. doi: 10.1111/j.1469-8986.2007.00567.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hare, T. A., Camerer, C. F., and Rangel, A. (2009). Self-control in decision-making involves modulation of the vmPFC valuation system. Science 324, 646–648. doi: 10.1126/science.1168450

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Harris, C. R. (2012). Feelings of dread and intertemporal choice. J. Behav. Decis. Mak. 25, 13–28. doi: 10.1002/bdm.709

CrossRef Full Text

Hewig, J., Trippe, R., Hecht, H., Coles, M. G. H., Holroyd, C. B., and Miltner, W. H. R. (2007). Decision-making in blackjack: an electrophysiological analysis. Cereb. Cortex 17, 865–877. doi: 10.1093/cercor/bhk040

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Holroyd, C. B., and Coles, M. G. H. (2002). The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol. Rev. 109, 679–709. doi: 10.1037/0033-295X.109.4.679

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Holroyd, C. B., Hajcak, G., and Larsen, J. T. (2006). The good, the bad and the neutral: electrophysiological responses to feedback stimuli. Brain Res. 1105, 93–101. doi: 10.1016/j.brainres.2005.12.015

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Holroyd, C. B., and Krigolson, O. E. (2007). Reward prediction error signals associated with a modified time estimation task. Psychophysiology 44, 913–917. doi: 10.1111/j.1469-8986.2007.00561.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Holroyd, C. B., Pakzad-Vaezi, K. L., and Krigolson, O. E. (2008). The feedback correct-related positivity: sensitivity of the event-related brain potential to unexpected positive feedback. Psychophysiology 45, 688–697. doi: 10.1111/j.1469-8986.2008.00668.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Itagaki, S., and Katayama, J. (2008). Self-relevant criteria determine the evaluation of outcomes induced by others. Neuroreport 19, 383–387. doi: 10.1097/WNR.0b013e3282f556e8

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kable, J. W., and Glimcher, P. W. (2007). The neural correlates of subjective value during intertemporal choice. Nat. Neurosci. 10, 1625–1633. doi: 10.1038/nn2007

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kable, J. W., and Glimcher, P. W. (2010). An “as soon as possible” effect in human intertemporal decision making: behavioral evidence and neural mechanisms. J. Neurophysiol. 103, 2513–2531 doi: 10.1152/jn.00177.2009

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kang, S. K., Hirsh, J. B., and Chasteen, A. L. (2010). Your mistakes are mine: self-other overlap predicts neural response to observed errors. J. Exp. Soc. Psychol. 46, 229–232. doi: 10.1016/j.jesp.2009.09.012

CrossRef Full Text

Kreussel, L., Hewig, J., Kretschmer, N., Hecht, H., Coles, M. G., and Miltner, W. H. (2012). The influence of the magnitude, probability, and valence of potential wins and losses on the amplitude of the feedback negativity. Psychophysiology 49, 207–219. doi: 10.1111/j.1469-8986.2011.01291.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Linden, D. E. J. (2005). The P300: where in the brain is it produced and what does it tell us? Neuroscientist 11, 563–576. doi: 10.1177/1073858405280524

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Loewenstein, G. (1987). Anticipation and the valuation of delayed consumption. Econ. J. 97, 666–684. doi: 10.2307/2232929

CrossRef Full Text

Luo, Q., Wang, Y., and Qu, C. (2011). The near-miss effect in slot-machine gambling: modulation of feedback-related negativity by subjective value. Neuroreport 22, 989–993. doi: 10.1097/WNR.0b013e32834da8ae

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ma, Q., Shen, Q., Xu, Q., Li, D., Shu, L., and Weber, B. (2011). Empathic responses to others’ gains and losses: an electrophysiological investigation. NeuroImage 54, 2472–2480. doi: 10.1016/j.neuroimage.2010.10.045

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

MacKeigan, L. D., Larson, L. N., Draugalis, J. R., Bootman, J. L., and Burns, L. R. (1993). Time preference for health gains versus health losses. Pharmacoeconomics 3, 374–386. doi: 10.2165/00019053-199303050-00005

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Masaki, H., Takeuchi, S., Gehring, W. J., Takasawa, N., and Yamazaki, K. (2006). Affective motivational influences on feedback-related ERPs in a gambling task. Brain Res. 1105, 110–121. doi: 10.1016/j.brainres.2006.01.022

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Mazur J. E. (1987). “An adjusting procedure for studying delayed reinforcement,” in Quantitative Analyses of Behavior, vol. 5, The Effect of Delay and of Intervening Events on Reinforcement Value, eds M. L. Commons, J. E. Mazur, J. A. Nevin, and H. Rachlin (Erlbaum: Mahwah, NJ), 55–73.

McClure, S. M., Laibson, D. I., Loewenstein, G., and Cohen, J. D. (2004). Separate neural systems value immediate and delayed monetary rewards. Science 306, 503–507. doi: 10.1126/science.1100907

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Miltner, W. H. R., Braun, C. H., and Coles, M. G. H. (1997). Event-related brain potentials following incorrect feedback in a time-estimation task: evidence for a “generic” neural system for error detection. J. Cogn. Neurosci. 9, 788–798. doi: 10.1162/jocn.1997.9.6.788

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Nieuwenhuis, S., Aston-Jones, G., and Cohen, J. D. (2005). Decision making, the p3, and the locus coeruleus–norepinephrine system. Psychol. Bull. 131, 510–532. doi: 10.1037/0033-2909.131.4.510

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Nieuwenhuis, S., Yeung, N., Holroyd, C. B., Schurger, A., and Cohen, J. D. (2004). Sensitivity of electrophysiological activity from medial frontal cortex to utilitarian and performance feedback. Cereb. Cortex 14, 741–747. doi: 10.1093/cercor/bhh034

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Pine, A., Seymour, B., Roiser, J. P., Bossaerts, P., Friston, K. J., Curran, H. V., et al. (2009). Encoding of marginal utility across time in the human brain. J. Neurosci. 29, 9575–9581. doi: 10.1523/JNEUROSCI.1126-09.2009

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Read, D. (2004). “Intertemporal choice,” in Handbook of Judgment and Decision Making, eds D. Koehler and N. Harvey (Oxford: Blackwell), 424–443. doi: 10.1002/9780470752937.ch21

CrossRef Full Text

Samuelson, P. A. (1937). A note on measurement of utility. Rev. Econ. Stud. 4, 155–161. doi: 10.2307/2967612

CrossRef Full Text

San Martin, R., Manes, F., Hurtado, E., Isla, P., and Ibanez, A. (2010). Size and probability of rewards modulate the feedback error-related negativity associated with wins but not losses in a monetarily rewarded gambling task. NeuroImage 51, 1194–1204. doi: 10.1016/j.neuroimage.2010.03.031

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Thaler, R. (1981). Some empirical evidence on dynamic inconsistency. Econ. Lett. 8, 201–207. doi: 10.1016/0165-1765(81)90067-7

CrossRef Full Text

Toyomaki, A., and Murohashi, H. (2005). Discrepancy between feedback negativity and subjective evaluation in gambling. NeuroReport 16, 1865–1868. doi: 10.1097/01.wnr.0000185962.96217.36

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Weinberg, A., Luhmann, C. C., Bress, J. N., and Hajcak, G. (2012). Better late than never? The effect of feedback delay on ERP indices of reward processing. Cogn. Affec. Behav. Neurosci. 1–7. doi: 10.3758/s13415-012-0104-z

CrossRef Full Text

Wu, Y., and Zhou, X. (2009). The P300 and reward valence, magnitude, and expectancy in outcome evaluation. Brain Res. 1286, 114–122. doi: 10.1016/j.brainres.2009.06.032

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Yeung, N., and Sanfey, A. G. (2004). Independent coding of reward magnitude and valence in the human brain. J. Neurosci. 24, 6258–6264. doi: 10.1523/JNEUROSCI.4537-03.2004

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text