Social and emotional relevance in face processing: happy faces of future interaction partners enhance the late positive potential

Human face perception is modulated by both emotional valence and social relevance, but their interaction has rarely been examined. Event-related brain potentials (ERP) to happy, neutral, and angry facial expressions with different degrees of social relevance were recorded. To implement a social anticipation task, relevance was manipulated by presenting faces of two specific actors as future interaction partners (socially relevant), whereas two other face actors remained non-relevant. In a further control task all stimuli were presented without specific relevance instructions (passive viewing). Face stimuli of four actors (2 women, from the KDEF) were randomly presented for 1s to 26 participants (16 female). Results showed an augmented N170, early posterior negativity (EPN), and late positive potential (LPP) for emotional in contrast to neutral facial expressions. Of particular interest, face processing varied as a function of experimental tasks. Whereas task effects were observed for P1 and EPN regardless of instructed relevance, LPP amplitudes were modulated by emotional facial expression and relevance manipulation. The LPP was specifically enhanced for happy facial expressions of the anticipated future interaction partners. This underscores that social relevance can impact face processing already at an early stage of visual processing. These findings are discussed within the framework of motivated attention and face processing theories.


INTRODUCTION
Humans are intrinsically social. From an evolutionary perspective, social information is critical for survival as it contributes to successful commitment, procreation and preservation (Tooby and Cosmides, 1992;Brothers, 2002). Thus, conspecifics are primary elicitors of emotions designed to promote both affiliation and protection in the face of constantly changing environmental conditions (Keltner and Kring, 1998). Accordingly, viewing facial stimuli is highly informative and mediates perceptual, physiological, and behavioral responses (Hamm et al., 2003;Vuilleumier and Pourtois, 2007).
To investigate the link between social and emotional information processing, the present study focuses on the social relevance of facial pictures. Human faces contain salient social signals mediating information about one's own and the others' identity, emotional state, and intentions (Ekman and Friesen, 1975;Öhman, 1986). The neural signature of face processing has been outlined in recent research (Haxby et al., 2002;Adolphs and Spezio, 2006). Given the crucial importance of being able to efficiently read and understand facial expressions, it has been proposed that distinct brain structures are centrally involved in face processing (e.g., fusiform face area (FFA), superior temporal sulcus (STS); Kanwisher et al., 1997 but see Chao et al., 1999). In addition, research has identified several neural substrates involved in both emotional and social processes (e.g., amygdala, insular, medial prefrontal cortex (MPFC); Gusnard et al., 2001;Norris et al., 2004;Williams et al., 2004;Northoff et al., 2006;Schmitz and Johnson, 2007;Olsson and Ochsner, 2008;Sabatinelli et al., 2011). In order to adequately interact in social situations, observing emotional facial expressions facilitates perceptual, attentional, and behavioral responses (Alpers and Gerdes, 2007;Alpers et al., 2011). For instance, in visual search tasks, threatening (schematic) faces are detected more quickly than friendly or neutral target faces especially among highly anxious participants (Byrne and Eysenck, 1995;Öhman et al., 2001). In line with an evolutionary perspective, this processing advantage has been described specifically for angry and fearful faces mediating potential threat to the observer (Byrne and Eysenck, 1995;Whalen et al., 2001).
Electrophysiological measures are particularly well-suited to investigate the temporal dynamics of face processing. Eventrelated brain potential (ERP) studies have revealed processing differences for facial stimuli within the first 100 ms after stimulus onset. For instance, suggested to reflect attention gain control in extrastriate sources, enhanced P1 amplitudes were observed for fearful compared to neutral faces in visuo-spatial attention tasks (Pourtois et al., 2004). Further, temporo-occipital negativities have been shown to be sensitive to facial stimuli (N170; Bentin et al., 1996) and emotional facial expression (early posterior negativity, EPN; Schupp et al., 2004). The N170 is probably the most frequently investigated ERP component in face processing. It has been primarily related to the structural encoding of faces in temporo-occipital processing areas; for instance, as evidenced by studies manipulating structural features (e.g., face inversion; Itier and Taylor, 2002), presentation of specific face and body parts (e.g., only eye region; Itier et al., 2006), and spatial attention tasks (Holmes et al., 2003;Jacques and Rossion, 2007). Regarding the emotional state and intentions conveyed by facial expression, an early posterior negativity (occipito-temporal EPN; 150-300 ms) and late positive potentials (centro-parietal LPP; 300-700 ms) have been observed for angry as compared to neutral faces, but also for happy faces (Sato et al., 2001;Liddell et al., 2004;Schupp et al., 2004;Williams et al., 2006;Holmes et al., 2008; but see Wangelin et al., 2012). Further, these effects were more pronounced in socially anxious participants (Moser et al., 2008;Sewell et al., 2008; but see Mühlberger et al., 2009) and participants undergoing socially-mediated aversive anticipation (Wieser et al., 2010;Bublatzky and Schupp, 2012). Of particular interest, enhanced LPP amplitudes have been observed for neutral expressions of primed familiar faces (Schweinberger et al., 2002;Kaufmann et al., 2008), and when faces are high in social relevance (e.g., romantic partner, family members; Guerra et al., 2012).
The effects of emotional stimulus content on attention have also been documented with a variety of other visual stimuli (e.g., pictures of naturalistic scenes, words, and hand gestures; Schupp et al., 2006a;Kissler et al., 2007;Flaisch et al., 2009;Schacht and Sommer, 2009). Further, EPN and LPP components were found to vary as a function of emotional arousal (i.e., pronounced EPN/LPP for highly emotional arousing pictures; Schupp et al., 2006a), and the LPP appeared sensitive to emotion regulation (Hajcak et al., 2010;Thiruchselvam et al., 2011). In addition, both ERP components have been observed to occur spontaneously while passive picture viewing and during performance of concurrent explicit attention tasks (Schupp et al., 2006a;Pourtois et al., 2013). These results are in line with those of several neuroimaging studies (i.e., showing increased BOLD responses in distributed occipital, parietal and inferior temporal networks; Junghöfer et al., 2006;Sabatinelli et al., 2011) and studies that have shown clear differences in autonomic and reflex activity for emotional compared to neutral stimuli (e.g., Bradley et al., 2001). In sum, there is ample evidence supporting the notion that EPN and LPP components reflect motivationally guided selective attention to significant stimuli (Schupp et al., 2006a).
Building on these findings, the present study examined the joint effects of social relevance of facial stimuli and the displayed emotional expressions. Using an instructional learning paradigm, participants were informed that they would later be introduced to the person presented in a specific face picture. Thus, these faces acquired social relevance by virtue of being potential interaction partners and were contrasted with other non-relevant face actors. Furthermore, the manipulation of facial expressions (happy, neutral, angry) allowed to model the emotional valence and arousal of the anticipated social situation. In light of previous research on face processing, electrocortical processing was hypothesized to differentiate social relevant from non-relevant faces (enhanced EPN/LPP). Further, valence effects are proposed to account for prioritized emotion processing (e.g., threat-or happy-advantage; Schupp et al., 2004;Williams et al., 2006). For instance, based on higher motivational impact, emotional compared to neutral face processing may benefit from additional social relevance as reflected by enhanced LPP amplitudes (Schupp et al., 2007). Integrating different experimental paradigms and methodologies, the present study constitutes a new experimental approach to examine the mutual impact of social and emotional processes by means of the anticipation of a socially relevant situation. All participants were informed about the study protocol before providing informed consent in accordance with the university's ethics guidelines. Participants received course credits for their participation.

MATERIALS AND PRESENTATION
Happy, neutral, and angry facial expressions of 4 different face actors (2 female) were selected from the Karolinska Directed Emotional Faces (KDEF; Lundqvist et al., 1998). 1 Pictures (1024 × 768 pixels) were randomly presented for 1 s without interstimulus gaps (see Figure 1). The full set of pictures (N = 12) was presented 60 times during two separate blocks, each consisting of 720 trials. The first block served as a control condition without specific instructions (passive viewing task). For the second block (meet task), two specific face actors (1 female and 1 male) were introduced as future interaction partners. Accordingly, two face actors were instructed as relevant whereas the other two face actors were non-relevant with respect to future interaction. Assignment of face stimuli to the relevant/non-relevant condition was counterbalanced across participants. Within blocks, face pictures were presented in a different order for each participant. Accounting for potential repetition effects (see Flaisch et al., 2008), picture randomization was restricted to no more than three repetitions of the same facial expression, equal transition probabilities between facial expressions and face actors, and no immediate repetition of the same actor displaying the same facial expression. Pictures were presented on a 22 inch computer screen located approximately 1 meter in front of the participants.

PROCEDURE
After the EEG sensor net was attached, participants were seated in a dimly-lit and sound-attenuated room. During a practice run (12 picture trials), participants were familiarized with the picture viewing procedure. In the following passive viewing task participants were instructed to attend to each picture appearing on the screen. Before the meet task, instructions were given concerning the relevance of face stimuli by indicating who were FIGURE 1 | Illustration of the experimental procedure. Pictures of 4 face actors (A, B, C, D) displaying happy, neutral, and angry facial expressions were presented randomly (1 s each) in 2 experimental tasks (each 720 trials). Participants were instructed to attend all stimuli in the passive viewing task.
Following picture ratings, the main instruction about social contingencies was given: "You are going to meet one of these two people at the end of the experiment" (here indicated by an arrow). Finally, participants again rated face stimuli and were debriefed.
the relevant and non-relevant face actors. With respect to the kind of interaction situation, the meet instruction was deliberately kept vague and neutral ("You are going to meet one of these two people at the end of the experiment"). After each block, valence and arousal ratings of the picture set was assessed using the paperpencil version of the self-assessment manikin (SAM; Bradley and Lang, 1994). At the end of the experiment, a debriefing interview was completed.

EEG RECORDING
Electrophysiological data were collected using a 64 actiCap system (BrainProducts, Munich, Germany) with Ag/AgCl active electrodes mounted into a cap according to the 10-10 system (Falk Minow Services, Herrsching, Germany). The EEG was recorded continuously with a sampling rate of 500 Hz with FCz as the reference electrode, and filtered on-line from 0.1-100 Hz using VisionRecorder acquisition software and BrainAmp DC amplifiers (BrainProducts, Munich, Germany). Impedances were kept below 10 kΩ. Off-line analyses were performed using VisionAnalyzer 2.0 (BrainProducts) and EMEGS (Peyk et al., 2011) and included low-pass filtering at 30 Hz, artifact detection, sensor interpolation, baseline-correction, and conversion to an average reference (Junghöfer et al., 2000). Stimulus-synchronized epochs were extracted and lasted from 100 ms before to 800 ms after stimulus onset. Finally, separate average waveforms were calculated for the experimental conditions Facial Expression (happy, neutral, angry), Task (passive, meet), and Relevance (relevant, non-relevant), 2 for each sensor and participant.

Self-report data
Valence and arousal ratings were analyzed with repeated measures ANOVAs including the factors Facial Expression (happy, neutral, angry), Task (passive, meet), and Relevance (relevant, non-relevant).

Event-related potentials
To examine the effects of facial expression, instructed task, and relevance on face processing, a two-step procedure was used. As a first step, visual inspection and single sensor waveform analysis were used in concert to identify relevant ERP components. To this end, single sensor waveform analyses were calculated for each time point and each sensor separately (see Peyk et al., 2011) for the factors Facial Expression (happy, neutral, angry), Task (passive, meet), and Relevance (relevant, non-relevant). To correct for multiple testing, effects were only considered meaningful when the effects were observed for at least eight continuous data points and two neighboring sensors (cf., Bublatzky and Schupp, 2012). Supporting this cluster selection procedure, visual inspection helped ensure that no effects relevant to the main hypothesis regarding the interaction between Facial Expression, Task, and Relevance were missed.
Following this, conventional ERP analyses were based on area scores. Repeated measures ANOVAs based on mean activity in selected sensor clusters and time windows were performed. The P1 component was scored over parieto-occipital cluster (left: O1, PO3; right: O2, PO4) within 100 and 140 ms after picture onset. The N170 was scored at P7 and P8 between 150 and 200 ms. The EPN component was scored at bilateral posterior sensors (PO9 and PO10) between 260 and 360 ms after stimulus onset. To account for the broad distribution of the LPP component, mean activity was scored in bilateral centro-parietal clusters (left: FC1, C1, CP1, P1; right: FC2, C2, CP2, P2) in a time window from 450-700 ms.
For effects involving repeated measures, the Greenhouse-Geisser procedure was used to correct for violations of sphericity, and as a measure of effect size the partial η 2 (η 2 p ) are reported. To control for type 1 error, Bonferroni correction was applied for post hoc t-tests.

EVENT-RELATED POTENTIALS
Results indicated that verbal instructions about future interaction partners modulated early and late face processing as revealed by enhanced P1, EPN, and LPP amplitudes (Figures 2, 3). Further, the interaction of social and emotional relevance varied across the visual processing stream. Whereas early components revealed independent main effects of Facial Expression and Task (shown by P1, N170, and EPN), the LPP was markedly augmented for happy faces considered as future interaction partners (Figure 4).

DISCUSSION
The present study examined the impact of instructed social relevance and emotional facial expression on face processing. The main finding was that the mere verbal instruction about social contingencies can modulate early and late face processing as indicated by enhanced P1, EPN, and LPP amplitudes. Importantly, event-related potential measures revealed that the interaction of social and emotional significance varied across visual processing stream. Whereas rather early components revealed independent main effects of facial expression and task instruction (P1, N170, and EPN), the LPP was augmented specifically for happy faces of future interaction partners. These results support the notion of joint impact of emotional and social information mediating face perception. The anticipation of social interaction with another individual is of considerable value. In the present study, social relevance was manipulated by introducing two specific face actors as future interaction partners (meet task). Results indicate that this socioemotional context is associated with specific processing patterns as participants view face pictures. The first ERP component sensitive to both task instruction and emotional facial expression was the P1 component, which was enlarged for angry faces compared to happy and neutral facial expressions. Further, regardless of facial expression, enhanced P1 amplitudes were observed during meet compared to passive viewing task. Thus, several previous findings were replicated: enhanced P1 amplitudes in explicit attention tasks (Hillyard and Anllo-Vento, 1998;Pourtois et al., 2004) and implicit processing biases during self-relevant conditions (e.g., instructed threat or in specific phobia; Michalowski et al., 2009;Bublatzky and Schupp, 2012). Presumably based on intensified visual processing in the extrastriate cortex (Hillyard and Anllo-Vento, 1998;Pourtois et al., 2013), the present P1 effects may indicate enhanced vigilance during task conditions of high self-relevance.
Both N170 and EPN components varied as a function of emotional facial expression. As enhanced negativities have been found for both happy and angry compared to neutral faces, these findings suggest that selective face processing occurs as a function of stimulus arousal. Whereas the N170 has been mostly related to structural encoding of non-affective facial features (Sato et al., 2001;Eimer and Holmes, 2002) within occipitotemporal areas (e.g., STS; Itier and Taylor, 2004), the present data are in line with a growing body of literature showing that the N170 is subject to emotional modulation (Pizzagalli et al., 2002;Batty and Taylor, 2003;Rossignol et al., 2005) similar to the EPN component. Further, indicating the enhanced relevance of facial stimuli for (sub-) clinical populations with high levels of social anxiety, pronounced N170 and EPN amplitudes have been observed for angry facial expression Mühlberger et al., 2009;Wieser et al., 2010). Here, valence specific effects were observed in healthy participants, however, with more pronounced N170/EPN for happy facial expressions. One promising direction for future studies is to manipulate the implicit level of social relevance when examining interindividual differences in emotional face processing (e.g., familiar loved vs. unfamiliar faces displaying emotions; Guerra et al., 2012).
Regarding late positive potentials, face processing was modulated by both task-and emotional relevance. Similar to past research (Schupp et al., 2004), faces displaying angry expressions were associated with enhanced LPP amplitudes, however, this effect was similarly present for happy faces. Of particular interest, the social relevance manipulation revealed an interactive relationship with emotional facial expression. Whereas both happy and angry faces elicited an enhanced late parietal positivity compared to neutral stimuli, this effect was more pronounced when viewing potential interaction partner displaying happy facial expressions. A similar trend was observed for neutral, but not angry, faces of purported interaction partners compared to non-relevant faces. Thus, whereas emotional and social relevance independently modulated early ERP componentsindicating either a threat advantage (P1) or selective emotion processing (EPN)-later processing stage revealed specifically enhanced amplitudes for socially relevant happy faces (LPP). These findings appear in line with the evaluative space model of affective processing (Cacioppo and Berntson, 1994;Cacioppo et al., 1999). Depending on the level of activation, emotional input may provoke different processing and response gradients. For instance, at low activation levels, pleasant stimuli may exert a greater influence than unpleasant stimuli in guiding motivational tendencies (e.g., explorative behavior). Accordingly, in rather lowarousing experimental conditions, happy facial expression may be more efficient in activating the motivational approach system than angry faces fostering avoidance. This hypothesis could be tested with socially relevant faces presented under conditions of low and high arousal (e.g., threat-of-shock paradigm; Grillon and Charney, 2011;Bublatzky et al., 2010Bublatzky et al., , 2013. Importantly, future research is needed to connect findings from the perceptual/attentional domain to the functional level, for instance, by testing approach/avoidance behavior (e.g., decision making; Pittig et al., 2014) to socially relevant happy/angry faces in social phobia (Wangelin et al., 2012).
Over and above the impact of implicit stimulus relevance (i.e., emotional facial expression), explicit instructions about social relevance in the meet task was associated with increased P1, EPN, and LPP amplitudes. These findings may complement recent research utilizing selective attention paradigms (Delorme et al., 2004;Pourtois et al., 2013). For instance, Schupp et al. (2007) observed pronounced EPN and enhanced late parietal positivities for target pictures of different semantic categories. Of particular interest, pictures displaying highly arousing content potentiated attention effects specifically during later processing stages (Schupp et al., 2007). In the present social anticipation task, the emotional facial features were no counting targets and actually rated as little arousing; however, a boost of emotion-focused attention was observable specifically for happy facial expression of purported interaction partner. Here, the reference to neural systems involved in various means of relevance processing based on bottom-up or top-down regulation may be informative (e.g., relevance based on task instruction, emotional, or social information; Schmitz and Johnson, 2007;Pourtois et al., 2013). For instance, paying attention to specific stimulus features modulates BOLD responses in the visual cortex (Kastner and Ungerleider, 2000), and for both emotional scenes and facial expressions a great overlap of neural activity has been demonstrated in the amygdala and medial prefrontal cortex (Sabatinelli et al., 2011); the latter being strongly involved in self-referential processing (Gusnard et al., 2001;Northoff et al., 2006;Olsson and Ochsner, 2008;Schwarz et al., 2013).
Several noteworthy aspects and alternative explanations of the present findings need to be acknowledged and should be addressed in future research. First, the critical test of the interaction between social relevance and facial expression was based on the processing of the same face stimuli that differed only in instructed social relevance. This approach has the advantage of ruling out potential effects due to physical differences, as apparent in comparing "social" vs. "non-social" stimuli, however, required that a fixed order of passive viewing task, followed by social meet instruction, was adopted. Thus, excessive stimulus repetitions may have reduced emotion or task effects.
However, similar to previous research (Codispoti et al., 2006;Schupp et al., 2006b), neither EPN nor LPP components revealed a reduction of selective emotion processing in the later task. On the contrary, the present LPP amplitudes were generally enhanced during social anticipation task. Furthermore, cognitive processes-such as working memory load or implicit emotion regulation-may have contributed to the absence of enhanced LPP to socially relevant angry faces. For instance, recent studies observed reduced LPP amplitudes to aversive stimuli under working memory load, suggesting that threat processing is contingent on available cognitive resources (MacNamara et al., 2011;Van Dillen and Derks, 2012). Alternatively, implicit emotion regulation may have reduced LPP amplitudes to aversive stimuli as shown in previous studies (Hajcak et al., 2010;Thiruchselvam et al., 2011). Here, future research may implement resource competition (e.g., by means of concurrent tasks or distractor stimuli) and active emotion regulation strategies. This could help clarify how social relevance affects emotional and cognitive processes in face perception.
The effects of selective attention elicited by either implicit emotional or explicitly instructed task relevance have been assessed in previous studies (Schupp et al., 2007;Pourtois et al., 2013). Extending this line of research, the present study utilized a novel approach to manipulate stimulus relevance by introducing specific face actors as future interaction partner. Social relevance was found to modulate face processing differently across visual processing stream. Whereas early ERP components revealed independent effects of social and emotional relevance (P1, N170, EPN), later processing stages were associated with specifically enhanced LPP for happy facial expressions when displayed by future interaction partner. Thus, social relevance may facilitate evaluative face processing according to socio-emotional settings (i.e., future interaction; Fischer and van Kleef, 2010).