Outcome Evaluation Affects Facial Trustworthiness: An Event-Related Potential Study

Introduction

Facial trustworthiness perception is a face-based trait inference process that refers to the evaluation of others’ trustworthiness based on their faces. The perception of facial trustworthiness is a fast process; indeed, individuals can judge the trustworthiness of a face when it appears for 33 ms (Todorov et al., 2009). Judging the trustworthiness of a stranger’s face is a spontaneous process (Klapper et al., 2016) that does not require effort of will (Bonnefon et al., 2013). This perceptual feature of facial trustworthiness is of great significance for the survival and development of humans.

Facial trustworthiness perception provides key information about whether someone should be approached or avoided, which can be influenced by the emotional state of the perceiver and target (Oosterhof and Todorov, 2008; Meconi et al., 2014). A trustworthy face can be considered to convey negative information to an extent; i.e., perception of trustworthiness is positively correlated with judgments of happiness and negatively correlated with the perception of anger from emotionally neutral faces (Oosterhof and Todorov, 2008). Event-related potential (ERP) studies have revealed that untrustworthy faces elicited greater electrical activity in several stages of face processing than did trustworthy faces, as demonstrated in P100, N170, early posterior negativity, feedback negativity (FN), and late positive component (LPC)/late positive potential (LPP; Yang et al., 2011; Marzi et al., 2012; Li et al., 2017; Lischke et al., 2018). Li et al. (2017) found that untrustworthy faces induced more negative FN than trustworthy faces in the late phase of the game and that the anterior cue-elicited FN reflects the reputation appraisal and tracks the reputation learning process in social interactions. People are more inclined to stay away from untrustworthy face individuals, and pay more attention to untrustworthy faces, which induced more positive LPC (Schupp et al., 2000, 2004a,b; Langeslag et al., 2007; Yang et al., 2011). However, studies have also reported that trustworthy faces aroused more positive ERP than did untrustworthy faces, such as an enhanced positivity at approximately 150 ms at frontal sites (Marzi et al., 2012) and during the time window of 200–400 ms over the frontal lobe (Rudoy and Paller, 2009).

Berg et al. (1995) designed a trust game in which the trustor is first given $10 and then must decide how much to “invest” in the trustee. When the amount of investment triples, it is at the discretion of the trustee to decide how much to repay. Facial trustworthiness plays a critical role in the social perception of faces and directly determines the establishment and development of interpersonal trust (Winston et al., 2002; Oosterhof and Todorov, 2008; Van’t Wout and Sanfey, 2008). Individuals prefer to lend money to others who look trustworthy Duarte et al., 2012; Jenq et al., 2015). In trust games, trustors invest more money in trustees with trustworthy faces than in those with untrustworthy faces (Van’t Wout and Sanfey, 2008; Chang et al., 2010; Rezlescu et al., 2012; Tingley, 2014; Ewing et al., 2015; Bailey et al., 2016).

The influence of facial trustworthiness on decision making in trust games may be reduced when participants learn about their partners’ past related behaviors (Chang et al., 2010; Rezlescu et al., 2012). Outcomes (trustees’ feedback) will influence trustors’ investment behavior. With an increase in communication, trustors pay more attention to trustees’ behavior, the influence of the face gradually decreases, and trustors invest more money in partners who repay more money (Chang et al., 2010; Suzuki and Suga, 2010; Rezlescu et al., 2012; Campellone and Kring, 2013; Yu et al., 2014). As an important cognitive function of humans, outcome evaluation is a process in which individuals evaluate results or external feedback caused by their own behaviors (Sun and Luo, 2008). Different feedback leads to distinct emotional experiences. “Gain” or “positive” feedback induces positive emotions, and “loss” or “negative” feedback induces negative emotions (Oosterhof and Todorov, 2008; Todorov et al., 2008b). For outcome evaluation, ERP researchers have focused on feedback-related negativity (FRN), which peaks at frontocentral recording sites between 200 and 350 ms after feedback onset (Holroyd and Coles, 2002).

In daily life, the reality of understanding a person involves integrating information from the observation of their appearances and actual behaviors. For individuals with either concordant or conflicting facial and behavioral trustworthiness, individuals’ reactions and underlying neural substrates warrant further investigation. Previous studies have typically presented faces first followed by feedback. Conversely, less is known about how feedback (outcome evaluation) affects facial trustworthiness judgments. Our study aimed to explore the dissociation between bottom-up and top-down mechanisms in facial trustworthiness processing. Leng et al. (2020) reported a more negative FRN was observed when results were unexpected or negative. Yang et al. (2011) considered that the LPC effect verified the emotion overgeneralization hypothesis of a trustworthy face. Therefore, we hypothesized that trustworthy faces with losses would induce more negative FRN, whereas untrustworthy faces would induce more positive LPC compared with trustworthy faces in the face presentation stage.

Materials and Methods

Participants

We calculated the sample size using G Power software (effect size f = 0.25, α = 0.05, power = 0.80), and the sample size should be more than 24. A total of 28 college students were recruited via advertisements; three participants were excluded due to excessive artifacts in their electroencephalography (EEG) data. Therefore, 25 participants (10 men and 15 women) were included in the final analysis [aged 19–28 years; mean (M) = 23 years, standard deviation (SD) = 3 years]. All participants were right-handed and had normal or corrected-to-normal vision. This study was approved by the institutional research ethics committees of Liaoning Normal University. All participants signed a written informed consent prior to the study.

Materials

The face images (120 trustworthy faces; 120 untrustworthy faces) used in the experiment were the same as those used by Leng et al. (2020). Neutral emotional faces (109 females, 113 males) were selected from the Chinese facial affective picture system (Gong et al., 2011). The photos were processed using Photoshop and edited to be the same size (260 by 300 pixels). Two psychology postgraduates were requested to rate the trustworthiness of faces and observed that the number of trustworthy faces did not meet the goal of this study. Facial features and structure influence the perception of facial trustworthiness (Todorov et al., 2008a, 2015; Stirrat and Perrett, 2010; Sofer et al., 2015). Typical (average) faces are considered more trustworthy (Sofer et al., 2015; Todorov et al., 2015). We thus combined the original faces with average faces to create new faces to improve the trustworthiness of original faces. The typical female face (Figure 1) was developed by a digital averaging process (PsychoMorph Version 5; Tiddeman et al., 2001) of 109 female faces. The typical female face was then combined with the 109 original male faces, and 109 new female faces (50% typical female face, 50% original female face) were obtained. A similar process was used to transform the 113 male faces (Figure 1). Therefore, we obtained 446 face images including the original 222 images, two typical images averaged from the male and female images, and another 222 images by combining the average face with each of the original images. In total, 10 unclear images were removed from 446 images, and six were selected for the pilot experiment. Finally, 430 valid images were selected. These faces were rated for trustworthiness (from 1 = “very untrustworthy” to 7 = “very trustworthy”) by 33 college students. From the 215 images of male faces, 60 faces that accounted for the top 27% of trustworthy scores were selected as male trustworthy faces; 60 faces that accounted for the bottom 27% of trustworthy scores were selected as male untrustworthy faces. This same process was repeated for the female faces, with 60 trustworthy and 60 untrustworthy faces being finally selected. Finally, a total of 240 faces were selected, comprising 120 trustworthy faces and 120 untrustworthy faces. An independent sample t-test revealed a significant difference between the scores for trustworthy and untrustworthy faces (t₍₂₃₈₎ = 31.021, p < 0.001, Cohen’s d = 4.06). The scores for trustworthy faces (4.87 ± 0.48) were significantly higher than those for untrustworthy faces (2.87 ± 0.51).

FIGURE 1

Figure 1. Typical female (left) and male (right) face.

Procedure

E-Prime software (Schneider et al., 2002) was used to compile the program and collect participants’ behavioral data. For the experiment, participants were instructed to imagine that they had invested 100 yuan in each person presented in the image. This money would become 400 yuan in the trustee’s hand, and the trustee had two choices: reciprocate 200 yuan to the participant or not. After the participants acknowledged that they understood the instructions, the experimental trial started with presentation of a fixation point in the center of the screen for 400–600 ms. In order to get better baseline correction, a blank screen was subsequently presented for a random duration of 500–800 ms. The feedback of the trustee was then displayed for 2 s “+100,” and “−100” indicated that the participant had earned and lost 100 yuan, respectively. A second blank screen was then presented for a random duration of 500–800 ms. The trustee’s photos were then displayed for 1 s. Finally, the participants were required to decide whether or not to continue investing. The “F” and “J” keys indicated whether to continue or stop investing, respectively (Figure 2). Mapping of keys was counterbalanced between participants. Each image was shown only once in the experiment. The task consisted of 240 trials divided into four blocks. The trial numbers for each combination of outcome signals (gain and loss) and face cues (trustworthy and untrustworthy) were identical.

FIGURE 2

Figure 2. Experimental flow chart.

Electroencephalography-Event-Related Potential Data Collection and Pre-analysis

An electrode cap (Brain Products GmbH) was used to collect the EEG data. The cap covered 64 scalp sites with tin electrodes arranged according to the 10-20 international placement system. All electrode impedance was maintained below 5 kΩ. EEG signals were sampled at 500 Hz/channel. FCz was used as the reference electrode, and vertical electrooculogram was recorded using an electrode located under the right eye. The bilateral mastoid process was used for re-reference during off-line treatment. The Gratton and Coles ocular correction algorithm of BrainVision Analyzer 2.1 software was used to analyze the EEG data. Channels were marked as artifacts if the signal variation exceeded ±80 μV and were filtered with a low pass of 20 Hz. For analyses of variance (ANOVAs), p-values were corrected via the Greenhouse–Geisser method, and multiple comparisons were corrected with the Bonferroni method when appropriate. The EEG components of feedback presentation and face presentation stage were analyzed. Epochs were extracted from −200 to 1,000 ms around feedback (face) onset. The data were then baseline corrected according to the 200-ms pre-feedback (pre-face) period. Based on previous studies (Yang et al., 2011; Li et al., 2017; Hu et al., 2018; Leng et al., 2020) and the scalp topographies of each component, we conducted statistical analyses on four ERP components (N1, P200, FRN, and LPC). For the N1 component (amplitude: 80–150 ms), data from Fz, FCz, and Cz electrodes were analyzed. For both the P200 (amplitude: 150–250 ms) and FRN (mean amplitude: 250–400 ms) components, data from Fz, FCz, Cz, CPz, and Pz were analyzed. For the LPC component (mean amplitude: 500–700 ms), data from 15 electrodes (F3/Fz/F4, FC3/FCz/FC4, C3/Cz/C4, CP3/CPz/CP4, and P3/Pz/P4) were analyzed.

Results

Behavioral Results

Two-way repeated measures ANOVA was performed with the proportion of participants continuing to invest in four conditions as the dependent variable, [2 (feedback: gain, loss) × 2 (face type: trustworthy face, untrustworthy face)]. Both the main effect (F_(1,24) = 56.537, p < 0.001, ${\eta }_p^{\text{2}}$ = 0.702; F_(1,24) = 139.747, p < 0.001, ${\eta }_p^{\text{2}}$ = 0.853) and the interaction (F_(1,24) = 14.108, p < 0.001, ${\eta }_p^{\text{2}}$ = 0.370) are significant. Simple effects analysis revealed that when the trustee reciprocated, the proportion of participants continuing to invest in trustworthy faces (87.3 ± 12.5%) was significantly higher than that for untrustworthy faces (33.2% ± 22.0%; t₍₂₄₎ = 12.537, p < 0.001, Cohen’s d = 2.506). When the trustee did not reciprocate, the proportion of participants continuing to invest in trustworthy faces (42.6 ± 25.5%) was significantly higher than that for untrustworthy faces (6.6 ± 6.4%; t₍₂₄₎ = 7.661, p < 0.001, Cohen’s d = 1.531).

Event-Related Potential Results

Event-Related Potential Analysis of Feedback

P200 and FRN were induced during the feedback presentation stage. The average trial numbers in the gain and loss conditions were 113 ± 8 and 113 ± 9 (M ± SD), respectively. The number of trials per condition did not differ (t₍₂₄₎ = 0.790, p = 0.437). The ERPs of different feedback results (gain or loss) at electrode sites Fz, FCz, Cz, CPz, and Pz are shown in Figure 3. Scalp topographies of P200 (150–250 ms) and FRN (250–400 ms) are shown in Figure 4.

FIGURE 3

Figure 3. Grand-average event-related potentials (ERPs) of different feedback (gain, loss) at electrode sites Fz, FCz, Cz, CPz, and Pz.

FIGURE 4

Figure 4. Topographic maps of P200 (left) and feedback-related negativity (FRN; right) in different feedback conditions (gain, loss).

P200 and FRN. Two-way repeated measures ANOVA was performed with the average amplitude of P200 (150–250 ms) as the dependent variable [2 (feedback: gain, loss) × 5 (electrode: Fz, FCz, Cz, CPz, Pz)]. A significant main effect of feedback was observed, whereby loss (−0.982 ± 2.624 μV) induced a more positive P200 than that of gain (−1.972 ± 2.492 μV; F_(1,24) = 16.282, p < 0.01, ${\eta }_p^{\text{2}}$ = 0.404). A significant main effect of electrode was noted, whereby the electrode located at the Pz site exhibited a higher amplitude at P200 [Pz (−0.577 ± 2.418 μV) > CPz (−1.445 ± 2.642 μV)/Cz (−1.466 ± 2.631 μV)/FCz (−1.598 ± 2.793 μV) > Fz (−2.299 ± 2.850 μV); F_(4,96) = 7.768, p < 0.01, ${\eta }_p^{\text{2}}$ = 0.245]. No significant interaction between feedback and electrode was observed (F_(4,96) = 1.350, p = 0.268).

Similarly, the ANOVA on the FRN amplitude found that the main effects of feedback were marginal significance, whereby loss (−5.166 ± 2.344 μV) induced a more negative FRN than that of gain (−4.570 ± 2.654 μV; F_(1,24) = 3.15, p = 0.089, ${\eta }_p^{\text{2}}$ = 0.116). A significant main effect of electrode was observed, whereby the electrode located at the FCz site exhibited a higher amplitude in FRN [FCz (−5.253 ± 2.693 μV) > Cz (−4.887 ± 2.725 μV); F_(4,96) = 3.878, p = 0.039, ${\eta }_p^{\text{2}}$ = 0.139]. No significant interaction was observed between feedback and electrode (F_(4,96) = 0.553, p = 0.554).

Event-Related Potential Analysis of Faces

N1, P200, FRN, and LPC were observed during the face presentation stage. The average trial number of trustworthy faces in both the gain and loss condition was 59 ± 2. The average trial numbers of untrustworthy faces in the gain and loss conditions were 59 ± 1 and 59 ± 2, respectively. No significant differences were observed in the average number of trials among the four cases (F_(3,72) = 0.958, p = 0.415).

ERPs induced by trustworthy and untrustworthy faces at electrode sites Fz, FCz, Cz, CPz, and Pz in different feedback conditions (gain or loss) are shown in Figure 5 Scalp topographies of FRN (250–400 ms) and LPC (500–700 ms) are shown in Figure 6.

FIGURE 5

Figure 5. Grand-average ERPs of trustworthy and untrustworthy faces at electrode sites Fz, FCz, Cz, CPz, and Pz in different feedback conditions (gain, loss).

FIGURE 6

Figure 6. Topographic maps of FRN (left) and late positive component (LPC; right) in different feedback (gain, loss) and facial trustworthiness (trustworthy, untrustworthy) conditions.

N1 and P200. A three-way repeated measures ANOVA was performed with N1 amplitude (80–150 ms) as the dependent variable [2 (feedback: gain, loss) × 2 (face type: trustworthy face, untrustworthy face) × 3 (electrode: Fz, FCz, Cz)]. A significant main effect of electrode was noted, whereby the electrode located at the Fz/FCz site exhibited a higher amplitude in N1 [Fz (−5.283 ± 3.030 μV)/FCz (−5.246 ± 3.107 μV) < Cz (−4.516 ± 3.136 μV); F_(2,48) = 10.662, p < 0.001, ${\eta }_p^{\text{2}}$ = 0.308]. A significant interaction between face type and feedback was noted (F_(1,24) = 5.401, p < 0.05, ${\eta }_p^{\text{2}}$ = 0.184). Simple effects analysis revealed that when the feedback was a gain, the N1 amplitude of untrustworthy faces (−5.493 ± 3.042 μV) was significantly more negative than that for trustworthy faces (−4.366 ± 3.331 μV; t₍₂₄₎ = 2.978, p < 0.01, Cohen’s d = 0.595); when the feedback was a loss, the N1 amplitude of trustworthy faces (−5.094 ± 3.254 μV) was not significantly different from that for untrustworthy faces (−5.107 ± 3.438 μV; t₍₂₄₎ = 0.031, p = 0.976). No other significant main effect or interaction was noted. The ANOVA on the P200 amplitude showed that neither the main effect nor interaction was significant.

FRN and LPC. The ANOVA on the FRN amplitude revealed that both the main effect of face type (F_(1,24) = 18.605, p < 0.001, ${\eta }_p^{\text{2}}$ = 0.437) and that of electrode (F_(1,24) = 85.965, p < 0.001, ${\eta }_p^{\text{2}}$ = 0.782) were significant, trustworthy faces (0.946 ± 4.703 μV) induced a more negative FRN than the untrustworthy faces (2.021 ± 5.345 μV), t₍₂₄₎ = −4.313, p < 0.001, Cohen’s d = 0.858, and the electrode located at the Fz/FCz site exhibited a higher amplitude in FRN [Fz (−1.283 ± 4.653 μV)/FCz (−0.816 ± 5.079 μV) < Cz (0.625 ± 5.342 μV) < CPz (3.042 ± 5.519 μV) < Pz (5.849 ± 5.345 μV)].

A significant interaction between face type and feedback was observed (F_(1,24) = 6.496, p < 0.05, ${\eta }_p^{\text{2}}$ = 0.213). No other significant main effect or interactions were noted. Simple effects analysis revealed that when the feedback was a gain, the FRN amplitude of trustworthy faces (1.361 ± 5.186 μV) was not significantly different from that for untrustworthy faces (1.811 ± 5.329 μV; t₍₂₄₎ = −1.366, p = 0.185); when the feedback was a loss, the FRN amplitude of trustworthy faces (0.531 ± 4.355 μV) was significantly lower than that for untrustworthy faces (2.230 ± 5.479 μV; t₍₂₄₎ = −4.614, p < 0.01, Cohen’s d = 0.920). The FRN amplitude for trustworthy faces with losses (0.531 ± 4.355 μV) was significantly lower than that for trustworthy faces with gains (1.361 ± 5.186 μV; t₍₂₄₎ = 2.307, p < 0.05, Cohen’s d = 0.463). No significant difference between the FRN amplitude for untrustworthy faces with losses (2.230 ± 5.479 μV) and untrustworthy faces with gains (1.811 ± 5.329 μV) was observed (t₍₂₄₎ = −1.376, p = 0.181).

Similarly, the ANOVA on the LPC amplitude found that both the main effect of face type (F_(1,24) = 26.048, p < 0.001, ${\eta }_p^{\text{2}}$ = 0.520) and that of electrode (F_(1,24) = 88.961, p < 0.001, ${\eta }_p^{\text{2}}$ = 0.788) was significant; untrustworthy faces (4.393 ± 4.729 μV) induced a more positive LPC than the trustworthy faces (2.553 ± 3.832 μV; t₍₂₄ = −5.104, p < 0.001, Cohen’s d = 1.018), and the electrode located at the Pz site exhibited a higher amplitude in LPC [Pz (7.217 ± 4.412 μV) > CPz (5.314 ± 4.613 μV) > Cz (3.265 ± 4.576 μV) > FCz (1.538 ± 4.343 μV) > Fz (0.031 ± 4.159 μV)].

A significant interaction between face type and feedback was noted, F_(1,24) = 7.637, p < 0.05, ${\eta }_p^{\text{2}}$ = 0.241. No other significant main effect or interactions were noted. Simple effect analysis found that the trustworthy faces with gains (3.138 ± 4.076 μV) induced a more positive LPC than the trustworthy faces with losses (1.967 ± 3.789 μV), t₍₂₄₎ = 3.276, p < 0.01, Cohen’s d = 0.654; there was no significant difference between the LPC amplitude of the untrustworthy faces with gains (4.099 ± 4.616 μV) and the untrustworthy faces with losses (4.688 ± 5.112 μV), t₍₂₄₎ = −1.264, p = 0.219; there was no significant difference between the LPC amplitude of the untrustworthy faces with gains (4.099 ± 4.616 μV) and the trustworthy faces with gains (3.138 ± 4.076 μV), t₍₂₄ = −1.905, p = 0.069; the untrustworthy faces with losses (4.688 ± 5.112 μV) induced more positive LPC than the trustworthy faces with losses (1.967 ± 3.789 μV; t₍₂₄₎ = −5.954, p < 0.001, Cohen’s d = 1.190).

Discussion

The main findings of our study were that facial trustworthiness and feedback information collectively affected trust behavior. Trustors were more inclined to keep investing in trustworthy trustees or those appearing after positive feedback (gains). This is consistent with previous research, i.e., both initial impressions and previous interactions affect people’s trust in their partners (Chang et al., 2010; Suzuki and Suga, 2010; Rezlescu et al., 2012; Campellone and Kring, 2013; Yu et al., 2014).

At the neural level, we observed that losses induced a more positive P200 than that of gains in the feedback presentation stage. P200 reflects the processing of results, and stimuli with negative valence induce a more positive P200 than do those with neutral valence (Carretié et al., 2001, 2005). Prior research suggests that the P200 is positively correlated with the level of risk taking and reward (Kiat et al., 2016). In no response task or observation task, participants can also induce FRN (Leng and Zhou, 2014). FRN can be considered the main ERP component in outcome evaluation. According to reinforcement learning theory, FRN reflects the difference between expected and actual outcomes, with larger FRN amplitude reflecting larger differences (Gehring and Willoughby, 2002). The affective-motivational hypothesis posits that FRN reflects an evaluation of the affective or motivational significance of errors detected by cognitive monitoring processes. FRN is most pronounced following monetary losses as opposed to monetary gains and does not reflect error detection (Holroyd and Coles, 2002). During the early stage of outcome processing, FRN is more negative for losses than for gains (Hu et al., 2018). Some studies used principal components analysis and thought that the FRN may be an artifact positivity enhanced by rewards (Foti et al., 2011; Proudfit, 2015; Wang et al., 2016). Leng et al. (2020) reported that a more negative FRN was observed when results were unexpected or negative. In our study, the main effects of feedback were marginal significant, which might partly be due to the obscuring of FRN effects by P200 (Rigoni et al., 2010). Loss induced a more negative FRN than that of gain, which was consistent with the FRN literature (Gehring and Willoughby, 2002; Yu et al., 2007; Li et al., 2010; Foti et al., 2011).

During the face presentation stage, untrustworthy faces with gains induced a more negative N1 amplitude than that of trustworthy faces with gains. Indeed, negative stimuli induce a more negative N1 amplitude than do positive stimuli (Carretié et al., 2004; Smith et al., 2005; Sun et al., 2012), and early processing bias of threat information biases attention toward threat information (Öhman and Mineka, 2001; Olofsson et al., 2008). This reflects an evolutionary adaptative mechanism to rapidly deal with threats (Carretié et al., 2004, 2006). Untrustworthy faces are more threatening than trustworthy faces, thus inducing more negative N1. The lack of a significant difference in N1 between untrustworthy faces with losses and trustworthy faces with losses could be because participants had already experienced the threat when the feedback was a loss, i.e., they tended to think the subsequent face was untrustworthy and paid less attention to it.

FRN can appear after feedback presentation and after cue stimuli before feedback presentation (Osinsky et al., 2014; Li et al., 2017). In the ultimatum game, unfair results and seeing the face of the unfair proposer before the results are presented will induce FRN. We found that trustworthy faces induced a more negative FRN than the untrustworthy faces, which is consistent with previous research (Chen et al., 2012; Li et al., 2017). Attractive faces are perceived as more trustworthy (Oosterhof and Todorov, 2008; Xu et al., 2012; Lee et al., 2017). Chen et al. (2012) found that attractive faces induced a more negative FRN than unattractive faces; attractive trustees’ betrayal are unexpected for trustors. Li et al. (2017) used multiround trust games to assess how individuals distinguish trustworthiness of others and observed that during the later stages of the game, untrustworthy partners induced larger FN amplitude than did the trustworthy partners.

When feedback is a loss, individuals are more inclined to expect that the subsequent face presented is untrustworthy. When a trustworthy face appears, this violates expectations and elicits greater conflict. Therefore, trustworthy faces induce a more negative FRN than do untrustworthy faces. When feedback is a gain, individuals are more inclined to expect that the subsequent face presented is also trustworthy. Hence, trustworthy faces with losses induce greater negative FRN than do trustworthy faces with gains. If results are inconsistent with one’s expectations, a more negative FRN will be induced. In this regard, a negative result may not necessarily induce a more negative FRN. Expectations for trustworthy faces are generally greater, and the discrepancy between trustworthy faces and associated behavior leads to greater conflicts, thus inducing a more negative FRN. FRN is positively correlated with the level of risk taking and reward (Kiat et al., 2016). Wang et al. (2019) reported that medial frontal negativity (MFN) reflected both probability weight and money magnitude processes, and low-probability options or small magnitude induced a more pronounced MFN. Untrustworthy faces and losses indicate low-probability options or small magnitude. We observed that when the feedback was a gain, the FRN amplitude of trustworthy faces was not significantly different from that of untrustworthy faces. Further, no significant difference was observed between the FRN amplitude of untrustworthy faces with losses and untrustworthy faces with gains. In sum, the effects of conflict on FRN were greater than the effects of low-probability options or small magnitude.

Compared to trustworthy faces, untrustworthy faces induced a more positive LPC, which is consistent with previous research (Yang et al., 2011; Lischke et al., 2018). Yang et al. (2011) considered that the LPC effect was consistent with the prediction of the emotion overgeneralization hypothesis of a trustworthy face. Untrustworthy faces can increase the activity of amygdala, which plays an important role in motivation evaluation (Todorov et al., 2008b). LPC is enhanced by increased motivated attention (Schupp et al., 2000, 2004a,b; Langeslag et al., 2007; Yang et al., 2011) implying that more attention is allocated to untrustworthy faces than to trustworthy faces. We observed an interaction between face type and feedback; trustworthy faces with gains induced a more positive LPC and generated more motivated attention than did trustworthy faces with losses. When feedback was a gain, participants were more inclined to expect that the subsequent face presented would be trustworthy; hence, they paid more attention to trustworthy faces. Conversely, when feedback was a loss, participants were more inclined to expect that the subsequent face to be presented would be untrustworthy; hence, they paid more attention to untrustworthy faces. In sum, trustworthy faces with gains generated more motivated attention in participants than did trustworthy faces with losses.

Bottom–up and top–down processes are integrated for facial trustworthiness perception at different stages. During the face presentation stage, untrustworthy faces with gains induced a more negative N1 compared to that for trustworthy faces with gains, reflecting early perception of threat information. In the loss condition, no differences in N1 amplitude were noted for facial trustworthiness, suggesting that top–down processing (feedback) exerted a greater impact during the early stage. Trustworthy faces with losses induced a more negative FRN compared to that for trustworthy faces with gains and untrustworthy faces with losses. Outcomes that did not meet expectations induced more negative FRN, suggesting that top–down and bottom–up processing are integrated in the medium term. Untrustworthy faces induced more positive LPC compared to that for trustworthy faces and generated more motivated attention; trustworthy faces with gains induced more positive LPC than did trustworthy faces with losses and generated more motivated attention. Regardless of previous feedback, no significant difference was noted in the amplitude of LPC induced by untrustworthy faces, implying that bottom–up processing (facial trustworthiness) exerted a greater impact during the later stage.

With regard to face stimuli, a degree of synthesis was employed even though some original natural faces were utilized. Future experiments should construct larger face stimulus libraries to generate more ecologically valid experimental materials. In our study, participants were instructed to imagine that they had invested 100 yuan. Future studies should also consider actual investment in individuals and present the faces of investors after a period of time. For the ERP analysis, we would try to use principal components analysis to better compare with other studies and increase the sample size to verify the stability of the results.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation, to any qualified researcher.

Ethics Statement

The studies involving human participants were reviewed and approved by institutional research ethics committees of Liaoning Normal University. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

ZJ conceived this study. HL participated in performing the study and writing the manuscript. YL, QL, QW, and DL participated in editing the manuscript. All authors contributed to the article and approved the submitted version.

Funding

This study was funded by the National Natural Science Foundation of China (No. 31271107).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We thank all the students who participated in the research.

References

Bailey, P. E., Szczap, P., McLennan, S. N., Slessor, G., Ruffman, T., and Rendell, P. G. (2016). Age-related similarities and differences in first impressions of trustworthiness. Cogn. Emot. 30, 1017–1026. doi: 10.1080/02699931.2015.1039493

PubMed Abstract | CrossRef Full Text | Google Scholar

Berg, J., Dickhaut, J., and McCabe, K. (1995). Trust, reciprocity and social history. Games Econ. Behav. 10, 122–142. doi: 10.1006/game.1995.1027

CrossRef Full Text | Google Scholar

Bonnefon, J. F., Hopfensitz, A., and De Neys, W. (2013). The modular nature of trustworthiness detection. J. Exp. Psychol. Gen. 142, 143–150. doi: 10.1037/a0028930

PubMed Abstract | CrossRef Full Text | Google Scholar

Campellone, T. R., and Kring, A. M. (2013). Who do you trust? The impact of facial emotion and behaviour on decision making. Cogn. Emot. 27, 603–620. doi: 10.1080/02699931.2012.726608

PubMed Abstract | CrossRef Full Text | Google Scholar

Carretié, L., Hinojosa, J. A., Albert, J., and Mercado, F. (2006). Neural response to sustained affective visual stimulation using an indirect task. Exp. Brain Res. 174, 630–637. doi: 10.1007/s00221-006-0510-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Carretié, L., Hinojosa, J. A., Martín-Loeches, M., Mercado, F., and Tapia, M. (2004). Automatic attention to emotional stimuli: neural correlates. Hum. Brain Mapp. 22, 290–299. doi: 10.1002/hbm.20037

PubMed Abstract | CrossRef Full Text | Google Scholar

Carretié, L., Hinojosa, J. A., Mercado, F., and Tapia, M. (2005). Cortical response to subjectively unconscious danger. NeuroImage 24, 615–623. doi: 10.1016/j.neuroimage.2004.09.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Carretié, L., Mercado, F., Tapia, M., and Hinojosa, J. A. (2001). Emotion, attention and the “negativity bias,” studied through event-related potentials. Int. J. Psychophysiol. 41, 75–85. doi: 10.1016/S0167-8760(00)00195-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Chang, L. J., Doll, B. B., van’t Wout, M., Frank, M. J., and Sanfey, A. G. (2010). Seeing is believing: trustworthiness as a dynamic belief. Cogn. Psychol. 61, 87–105. doi: 10.1016/j.cogpsych.2010.03.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, J., Zhong, J., Zhang, Y., Li, P., Zhang, A., Tan, Q., et al. (2012). Electrophysiological correlates of processing facial attractiveness and its influence on cooperative behavior. Neurosci. Lett. 517, 65–70. doi: 10.1016/j.neulet.2012.02.082

PubMed Abstract | CrossRef Full Text | Google Scholar

Duarte, J., Siegel, S., and Young, L. (2012). Trust and credit: the role of appearance in peer-to-peer lending. Rev. Financ. Stud. 25, 2455–2484. doi: 10.1093/rfs/hhs071

PubMed Abstract | CrossRef Full Text | Google Scholar

Ewing, L., Caulfield, F., Read, A., and Rhodes, G. (2015). Perceived trustworthiness of faces drives trust behaviour in children. Dev. Sci. 18, 327–334. doi: 10.1111/desc.12218

PubMed Abstract | CrossRef Full Text | Google Scholar

Foti, D., Weinberg, A., Dien, J., and Hajcak, G. (2011). Event-related potential activity in the basal ganglia differentiates rewards from nonrewards: temporospatial principal components analysis and source localization of the feedback negativity. Hum. Brain Mapp. 32, 2207–2216. doi: 10.1002/hbm.21182

PubMed Abstract | CrossRef Full Text | Google Scholar

Gehring, W. J., and Willoughby, A. R. (2002). The medial frontal cortex and the rapid processing of monetary gains and losses. Science 295, 2279–2282. doi: 10.1126/science.1066893

PubMed Abstract | CrossRef Full Text | Google Scholar

Gong, X., Huang, Y., Wang, Y., and Luo, Y. (2011). Revision of the chinese facial affective picture system. Chin. Ment. Health J. 25, 40–46. doi: 10.3969/j.issn.1000-6729.2011.01.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Holroyd, C. B., and Coles, M. G. H. (2002). The neural basis of human error processing: reinforcement learning, dopamine and the error-related negativity. Psychol. Rev. 109, 679–709. doi: 10.1037/0033-295X.109.4.679

PubMed Abstract | CrossRef Full Text | Google Scholar

Hu, X., Xu, Z., Li, Y., and Mai, X. (2018). The impact of trust decision-making on outcome processing: evidence from brain potentials and neural oscillations. Neuropsychologia 119, 136–144. doi: 10.1016/j.neuropsychologia.2018.07.036

PubMed Abstract | CrossRef Full Text | Google Scholar

Jenq, C., Pan, J., and Theseira, W. (2015). Beauty, weight and skin color in charitable giving. J. Econ. Behav. Organ. 119, 234–253. doi: 10.1016/j.jebo.2015.06.004

CrossRef Full Text | Google Scholar

Kiat, J., Straley, E., and Cheadle, J. E. (2016). Escalating risk and the moderating effect of resistance to peer influence on the p200 and feedback-related negativity. Soc. Cogn. Affect. Neurosci. 11, 377–386. doi: 10.1093/scan/nsv121

PubMed Abstract | CrossRef Full Text | Google Scholar

Klapper, A., Dotsch, R., van Rooij, I., and Wigboldus, D. H. (2016). Do we spontaneously form stable trustworthiness impressions from facial appearance? J. Pers. Soc. Psychol. 111, 655–664. doi: 10.1037/pspa0000062

PubMed Abstract | CrossRef Full Text | Google Scholar

Langeslag, S. J., Jansma, B. M., Franken, I. H., and Van Strien, J. W. (2007). Event-related potential responses to love-related facial stimuli. Biol. Psychol. 76, 109–115. doi: 10.1016/j.biopsycho.2007.06.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Lee, A. J., Wright, M. J., Martin, N. G., Keller, M. C., and Zietsch, B. P. (2017). Facial trustworthiness is associated with heritable aspects of face shape. Adapt. Hum. Behav. Physiol. 3, 351–364. doi: 10.1007/s40750-017-0073-0

CrossRef Full Text | Google Scholar

Leng, H., Liu, Y., Li, Q., Wu, Q., Yang, Z., and Jiang, Z. (2020). Facial trustworthiness affects outcome evaluation: an event-related potential study. Neuroreport 31, 741–745. doi: 10.1097/WNR.0000000000001483

PubMed Abstract | CrossRef Full Text | Google Scholar

Leng, Y., and Zhou, X. (2014). Interpersonal relationship modulates brain responses to outcome evaluation when gambling for/against others: an electrophysiological analysis. Neuropsychologia 63, 205–214. doi: 10.1016/j.neuropsychologia.2014.08.033

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, D., Meng, L., and Ma, Q. (2017). Who deserves my trust? Cue-elicited feedback negativity tracks reputation learning in repeated social interactions. Front. Hum. Neurosci. 11:307. doi: 10.3389/fnhum.2017.00307

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, P., Jia, S., Feng, T., Liu, Q., Suo, T., and Li, H. (2010). The influence of the diffusion of responsibility effect on outcome evaluations: electrophysiological evidence from an ERP study. NeuroImage 52, 1727–1733. doi: 10.1016/j.neuroimage.2010.04.275

PubMed Abstract | CrossRef Full Text | Google Scholar

Lischke, A., Junge, M., Hamm, A. O., and Weymar, M. (2018). Enhanced processing of untrustworthiness in natural faces with neutral expressions. Emotion 18, 181–189. doi: 10.1037/emo0000318

PubMed Abstract | CrossRef Full Text | Google Scholar

Marzi, T., Righi, S., Ottonello, S., Cincotta, M., and Viggiano, M. P. (2012). Trust at first sight: evidence from ERPs. Soc. Cogn. Affect. Neurosci. 9, 63–72. doi: 10.1093/scan/nss102

PubMed Abstract | CrossRef Full Text | Google Scholar

Meconi, F., Luria, R., and Sessa, P. (2014). Individual differences in anxiety predict neural measures of visual working memory for untrustworthy faces. Soc. Cogn. Affect. Neurosci. 9, 1872–1879. doi: 10.1093/scan/nst189

PubMed Abstract | CrossRef Full Text | Google Scholar

Öhman, A., and Mineka, S. (2001). Fears, phobias and preparedness: toward an evolved module of fear and fear learning. Psychol. Rev. 108, 483–522. doi: 10.1037/0033-295x.108.3.483

PubMed Abstract | CrossRef Full Text | Google Scholar

Olofsson, J. K., Nordin, S., Sequeira, H., and Polich, J. (2008). Affective picture processing: an integrative review of ERP findings. Biol. Psychol. 77, 247–265. doi: 10.1016/j.biopsycho.2007.11.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Oosterhof, N. N., and Todorov, A. (2008). The functional basis of face evaluation. Proc. Natl. Acad. Sci. U S A 105, 11087–11092. doi: 10.1073/pnas.0805664105

PubMed Abstract | CrossRef Full Text | Google Scholar

Osinsky, R., Mussel, P., Öhrlein, L., and Hewig, J. (2014). A neural signature of the creation of social evaluation. Soc. Cogn. Affect. Neurosci. 9, 731–736. doi: 10.1093/scan/nst051

PubMed Abstract | CrossRef Full Text | Google Scholar

Proudfit, G. H. (2015). The reward positivity: from basic research on reward to a biomarker for depression. Psychophysiology 52, 449–459. doi: 10.1111/psyp.12370

PubMed Abstract | CrossRef Full Text | Google Scholar

Rezlescu, C., Duchaine, B., Olivola, C. Y., and Chater, N. (2012). Unfakeable facial configurations affect strategic choices in trust games with or without information about past behavior. PLoS One 7:e34293. doi: 10.1371/journal.pone.0034293

PubMed Abstract | CrossRef Full Text | Google Scholar

Rigoni, D., Polezzi, D., Rumiati, R., Guarino, R., and Sartori, G. (2010). When people matter more than money: an ERPs study. Brain Res. Bull. 81, 445–452. doi: 10.1016/j.brainresbull.2009.12.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Rudoy, J. D., and Paller, K. A. (2009). Who can you trust? Behavioral and neural differences between perceptual and memory-based influences. Front. Hum. Neurosci. 3:16. doi: 10.3389/neuro.09.016.2009

PubMed Abstract | CrossRef Full Text | Google Scholar

Schneider, W., Eschman, A., and Zuccolotto, A. (2002). E-Prime User’s Guide. Pittsburgh, PA: Psychology Software Tools Inc.

Google Scholar

Schupp, H. T., Cuthbert, B. N., Bradley, M. M., Cacioppo, J. T., Ito, T. A., and Lang, P. J. (2000). Affective picture processing: the late positive potential is modulated by motivational relevance. Psychophysiology 37, 257–261. doi: 10.1111/1469-8986.3720257

PubMed Abstract | CrossRef Full Text | Google Scholar

Schupp, H. T., Ohman, A., Junghofer, M., Weike, A. I., Stockburger, J., and Hamm, A. O. (2004a). The facilitated processing of threatening faces: an ERP analysis. Emotion 4, 189–200. doi: 10.1037/1528-3542.4.2.189

PubMed Abstract | CrossRef Full Text | Google Scholar

Schupp, H., Cuthbert, B., Bradley, M., Hillman, C., Hamm, A., Lang, P., et al. (2004b). Brain processes in emotional perception: motivated attention. Cogn. Emot. 18, 593–611. doi: 10.1080/02699930341000239

CrossRef Full Text | Google Scholar

Smith, D. P., Hillman, C. H., and Duley, A. R. (2005). Influences of age on emotional reactivity during picture processing. J. Gerontol. B Psychol. Sci. Soc. Sci. 60, P49–P56. doi: 10.1093/geronb/60.1.p49

PubMed Abstract | CrossRef Full Text | Google Scholar

Sofer, C., Dotsch, R., Wigboldus, D. H. J., and Todorov, A. (2015). What is typical is good: the influence of face typicality on perceived trustworthiness. Psychol. Sci. 26, 39–47. doi: 10.1177/0956797614554955

PubMed Abstract | CrossRef Full Text | Google Scholar

Stirrat, M., and Perrett, D. I. (2010). Valid facial cues to cooperation and trust: male facial width and trustworthiness. Psychol. Sci. 21, 349–354. doi: 10.1177/0956797610362647

PubMed Abstract | CrossRef Full Text | Google Scholar

Sun, J., Sun, B., Wang, B., and Gong, H. (2012). The processing bias for threatening cues revealed by event-related potential and event-related oscillation analyses. Neuroscience 203, 91–98. doi: 10.1016/j.neuroscience.2011.12.038

PubMed Abstract | CrossRef Full Text | Google Scholar

Sun, S., and Luo, Y. (2008). Feedback-related negativity in outcome evaluation with a deception task. Acta Phychol. Sin. 40, 693–700. doi: 10.3724/SP.J.1041.2008.00693

CrossRef Full Text | Google Scholar

Suzuki, A., and Suga, S. (2010). Enhanced memory for the wolf in sheep’s clothing: facial trustworthiness modulates face-trait associative memory. Cognition 117, 224–229. doi: 10.1016/j.cognition.2010.08.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Tiddeman, B., Burt, M., and Perrett, D. (2001). Prototyping and transforming facial textures for perception research. IEEE Comput. Graph. Appl. 21, 42–50. doi: 10.1109/38.946630

CrossRef Full Text | Google Scholar

Tingley, D. (2014). Face-off: facial features and strategic choice. Polit. Psychol. 35, 35–55. doi: 10.1111/pops.12041

CrossRef Full Text | Google Scholar

Todorov, A., Baron, S. G., and Oosterhof, N. N. (2008a). Evaluating face trustworthiness: a model based approach. Soc. Cogn. Affect. Neurosci. 3, 119–127. doi: 10.1093/scan/nsn009

PubMed Abstract | CrossRef Full Text | Google Scholar

Todorov, A., Olivola, C. Y., Dotsch, R., and Mende-Siedlecki, P. (2015). Social attributions from faces: determinants, consequences, accuracy and functional significance. Ann. Rev. Psychol. 66, 519–545. doi: 10.1146/annurev-psych-113011-143831

PubMed Abstract | CrossRef Full Text | Google Scholar

Todorov, A., Pakrashi, M., and Oosterhof, N. N. (2009). Evaluating faces on trustworthiness after minimal time exposure. Soc. Cogn. 27, 813–833. doi: 10.1521/soco.2009.27.6.813

CrossRef Full Text | Google Scholar

Todorov, A., Said, C. P., Engell, A. D., and Oosterhof, N. N. (2008b). Understanding evaluation of faces on social dimensions. Trends Cogn. Sci. 12, 455–460. doi: 10.1016/j.tics.2008.10.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Van’t Wout, M., and Sanfey, A. G. (2008). Friend or foe: the effect of implicit trustworthiness judgments in social decision-making. Cognition 108, 796–803. doi: 10.1016/j.cognition.2008.07.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, G., Li, J., Wang, P., Zhu, C., Pan, J., and Li, S. (2019). Neural dynamics of processing probability weight and monetary magnitude in the evaluation of a risky reward. Front. Psychol. 10:554. doi: 10.3389/fpsyg.2019.00554

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, Y., Zhang, Z., Jing, Y., Valadez, E. A., and Simons, R. F. (2016). How do we trust strangers? The neural correlates of decision making and outcome evaluation of generalized trust. Soc. Cogn. Affect. Neurosci. 11, 1666–1676. doi: 10.1093/scan/nsw079

PubMed Abstract | CrossRef Full Text | Google Scholar

Winston, J. S., Strange, B. A., O’Doherty, J., and Dolan, R. J. (2002). Automatic and intentional brain responses during evaluation of trustworthiness of faces. Nat. Neurosci. 5, 277–283. doi: 10.1038/nn816

PubMed Abstract | CrossRef Full Text | Google Scholar

Xu, F., Wu, D., Toriyama, R., Ma, F., Itakura, S., and Lee, K. (2012). Similarities and differences in chinese and caucasian adults’ use of facial cues for trustworthiness judgments. PLoS One 7:e34859. doi: 10.1371/journal.pone.0034859

PubMed Abstract | CrossRef Full Text | Google Scholar

Yang, D., Qi, S., Ding, C., and Song, Y. (2011). An ERP study on the time course of facial trustworthiness appraisal. Neurosci. Lett. 496, 147–151. doi: 10.1016/j.neulet.2011.03.066

PubMed Abstract | CrossRef Full Text | Google Scholar

Yu, M., Saleem, M., and Gonzalez, C. (2014). Developing trust: first impressions and experience. J. Econ. Psychol. 43, 16–29. doi: 10.1016/j.joep.2014.04.004

CrossRef Full Text | Google Scholar

Yu, R., Luo, Y., Ye, Z., and Zhou, X. (2007). Does the FRN in brain potentials reflect motivational/affective consequence of outcome evaluation. Prog. Nat. Sci. 17, 136–143. doi: 10.1080/10020070612331343232

CrossRef Full Text | Google Scholar