Original Research ARTICLE
Nonconscious influences from emotional faces: a comparison of visual crowding, masking, and continuous flash suppression
- 1 Laboratoire de Sciences Cognitives et Psycholinguistique, CNRS/EHESS/DEC-ENS, Paris, France
- 2 Computation and Neural Systems, California Institute of Technology, Pasadena, CA, USA
- 3 Laboratoire Interpsy – Université de Lorraine, Nancy, France
In the study of nonconscious processing, different methods have been used in order to render stimuli invisible. While their properties are well described, the level at which they disrupt nonconscious processing remains unclear. Yet, such accurate estimation of the depth of nonconscious processes is crucial for a clear differentiation between conscious and nonconscious cognition. Here, we compared the processing of facial expressions rendered invisible through gaze-contingent crowding (GCC), masking, and continuous flash suppression (CFS), three techniques relying on different properties of the visual system. We found that both pictures and videos of happy faces suppressed from awareness by GCC were processed such as to bias subsequent preference judgments. The same stimuli manipulated with visual masking and CFS did not bias significantly preference judgments, although they were processed such as to elicit perceptual priming. A significant difference in preference bias was found between GCC and CFS, but not between GCC and masking. These results provide new insights regarding the nonconscious impact of emotional features, and highlight the need for rigorous comparisons between the different methods employed to prevent perceptual awareness.
Over the past 20 years, it has been clearly established that stimuli that are inaccessible to conscious reports can still induce behavioral and neural responses (Marcel, 1983; Kouider and Dehaene, 2007). As nonconscious influences are usually of small amplitude, their measurement requires a sensory stimulation of maximum possible energy satisfying the criterion of invisibility. To ensure specificity, they must be associated with a strict control of stimulus visibility to avoid potential confounds with conscious influences. So far, in order to prevent conscious processing, researchers have relied often on two kinds of manipulations, namely visual masking and continuous flash suppression (CFS; see Kim and Blake, 2005, for other techniques relying notably on attentional manipulations). In visual backward masking, a short-lasting stimulus (i.e., below 50 ms) is immediately followed by a visual pattern (i.e., the mask), and becomes impossible to detect or discriminate (Breitmeyer and Öğmen, 2006). In CFS, a stimulus of interest, is presented to one eye while a dynamic stream of salient patterns is presented to the other eye. Due to binocular rivalry, the stimulus is suppressed and remains undetectable even after exposure of several seconds (Tsuchiya and Koch, 2005).
These two approaches have been extremely fruitful to describe the depth of nonconscious cognition (see Kouider and Dehaene, 2007 for a review on visual masking, and Lin and He, 2009 for a review on binocular rivalry). Surprisingly, very few studies compared systematically the fate of stimuli suppressed from awareness by visual masking and CFS (Almeida et al., 2008; Kanai et al., 2010). Such methodological comparisons are of crucial importance in the context of a contrastive research on consciousness (Baars, 1997). Indeed, if one tries to characterize the differences between conscious and nonconscious processing, one has to make sure that these differences are indeed genuine, and do not stem from methodological limitations. As these limitations might be intrinsic to all experimental paradigms, we propose as a heuristic to compare systematically different methods, and disentangle the limits attributed to nonconscious cognition and the limits attributed to the method. Accordingly, this study sets to compare the level of nonconscious processing obtained with visual masking, CFS, and a third, recent alternative involving visual crowding.
In visual crowding, a peripheral stimulus appears jumbled and becomes undiscriminable when surrounded by similar flankers. By contrast to both masking and CFS, the detection of a crowded stimulus is preserved, since observers are able to report its presence. Nonetheless, properties of the crowded stimulus which are shared with the flankers cannot be discriminated. In order to control for stimulus discriminability, we implemented crowding in conjunction with eye-tracking control, resulting in a method we called gaze-contingent crowding (GCC; Faivre and Kouider, 2011a; Kouider et al., 2011). GCC allows for the substitution of the crowded stimulus with an irrelevant content as soon as the observer’s gaze diverges from a defined location. Interestingly, since increasing the duration of a crowded stimulus does not restore discriminability (Kooi et al., 1994), GCC allows for a continuous, long-lasting stimulation (i.e., high energy) with a reliable control of stimulus discriminability (i.e., avoiding potential confounds with conscious influences). Several studies have established the existence of a dissociation between the subjective feeling of a crowded, uninformative percept, and the underlying processing occurring without awareness, both at the level of single features (e.g., line orientation, see He et al., 1996), and multiple features (e.g., facial identity, directional arrow, Faivre and Kouider, 2011a). Recently, we found that peripheral faces whose emotional features were crowded by flanking patterns were processed such as to bias subsequent preference judgments (Kouider et al., 2011). Indeed, participants rated abstract targets (i.e., unknown Chinese pictographs) as more pleasant when previously exposed to happy compared to angry crowded faces. Notably, a similar preference bias was elicited by dynamic (i.e., videos) and static (i.e., pictures) facial expressions. Control conditions revealed that it disappeared when faces were presented upside-down, ruling out an interpretation in terms of low-level stimulus differences.
The purposes of the present study were twofold. First, we aimed at characterizing in more detail the processing of facial expressions during crowding. Extending our previous results, we show that both featural crowding (i.e., crowding of local cues like the shape of the mouth, induced by pattern flankers) and configural crowding (i.e., crowding of local cues, but also metric distances between them, induced by neutral face flankers) preserved the influence from facial expressions on preference judgment (Experiment 1). In addition, replicating our previous study now with a neutral baseline, we showed that the nonconscious preference bias arose specifically from the processing of happy faces, both for static and dynamic stimuli (Experiment 2), whereas both happy and angry faces elicited preference biases only when perceived consciously (Experiment 3).
Second, we performed a systematic comparison of the depth of facial expressions processing in GCC, visual masking (Experiment 4), and CFS (Experiment 5). Because these different paradigms prevent perceptual awareness through different means (e.g., peripheral stimulation and presence of flankers in GCC, duration of the prime and presence of the mask in masking, interocular suppression in CFS), we adapted the experimental parameters of each method such as to reach the sensory stimulation of maximum energy, while remaining below the threshold for perceptual awareness. Respecting the limits entailed by each method, we found qualitatively different results. Indeed, if emotional faces under visual masking and CFS were able to elicit perceptual priming (i.e., they influenced the processing of an identical target face), we could not find evidences that they were processed such as to bias subsequent preference judgments. The discrepancies between the different methods are discussed in light of the neural bases of crowding, masking, and binocular rivalry.
Face perception involves at least two different levels of processing: featural processing, which reflects the encoding of local cues (e.g., geometrical shape of the mouth), and configural processing, which refers to the encoding of metric distances between features (Maurer et al., 2002). Depending on the nature of the flankers, crowding can disrupt either level of processing. When patterns without configural information are used as flankers, they interfere with features of the central face, and thus prevent mainly featural processing (Kouider et al., 2011). On the other hand, in case faces are used as flankers, interferences with the central face involve both featural and configural informations (Louie et al., 2007; Farzin et al., 2009).
Several studies have shown that configural information is more affected by face inversion than the perception of local cues (for review, see Rossion, 2008). In our previous study dealing with featural crowding, emotional inverted faces did not induce a bias in preference judgment, suggesting first that the preference bias we observed had a configural rather than featural origin, and second that this configural processing was preserved despite featural crowding (Kouider et al., 2011). In Experiment 1, we tested whether the nonconscious processing of faces was preserved despite configural crowding. For this purpose, we compared how emotional faces surrounded by patterns (inducing featural crowding) and surrounded by neutral faces (inducing configural crowding) biased preference judgment.
Materials and methods
Participants. Eighteen French college students (age range = 18–35 years) participated in Experiment 1. All reported normal or corrected-to-normal vision and were paid for their participation. All asserted that they were unable to read Chinese pictographs.
Stimuli. We recorded facial expressions from five actresses recruited from a professional acting academy. Each actress’s face was filmed against a black background, in an equally illuminated room. Faces expressed happiness or anger. Random mouth motion was used as a neutral condition. Video clips were slightly sped up or down to compensate for the differences between the timings of the different actors’ expressions. The time of emotional paroxysms during the video clips were defined by two independent observers. All video clips were cropped to show the face only. They were then matched for average luminance (12.7 Cd m−2), contrast, and image size (3.2° × 3.9°). Snapshots were extracted from each video, at t = 0 ms for the neutral expression condition, and t = 1500 ms for the emotional paroxysm condition. Flankers consisted in 2.5° × 2.8° patterns created by overlaying faces and objects, resulting in non-informative objects (de Gardelle and Kouider, 2010). The luminance of the flankers was 40% higher than that of the emotional faces. Targets were 3° × 3° white Chinese pictographs.
Equipment. Eye movements were recorded monocularly with a tower mounted eye tracker (Eyelink 1000 system, SR research, ON, Canada) controlled with the Eyelink toolbox (Cornelissen et al., 2002), with a sampling rate of 1000 Hz and a spatial resolution of 1° or above. Stimuli were all displayed using Matlab with the Psychophysics Toolbox (Brainard, 1997; Pelli, 1997) against a black background on a 22-in. Iiyama Vision master pro 510 screen (frame rate of 85 Hz, resolution of 1024 × 768 pixels, luminance of 0.004 Cd m−2). The participants sat 57 cm from the screen in a dimly lit room. A chin and headrest was used in all experiments. A calibration phase was performed after each block of 90 trials.
Procedure. Experiment 1 included 240 preference judgment trials and 120 visibility trials, equally divided to 4 blocks of 90 trials. In each block, the flankers surrounding the emotional face were displayed in a circular array of either six faces identical to the emotional face but with a neutral expression (f), or in a circular array of six flanker patterns (m). The order of blocks was counterbalanced so that half the subjects went through the (f) block first (i.e., f-m-f-m) and the other half through the (m) block first (i.e., m-f-m-f).
Each trial started with a 0.5° × 0.5° fixation cross that was presented for 300 ms. The emotional face was then displayed for 2500 ms, surrounded by the six flankers (18° eccentricity between the emotional face and fixation cross centered at the quarter top of the screen; 3.1° (center to center distance), or 0.3° (edge to edge distance) between the emotional face and the flankers). Importantly, as soon as the participant ceased to gaze at the fixation area (5° by 5° zone surrounding the fixation cross) the emotional face was substituted by the same face with a neutral expression so to guarantee that the emotional face was never processed foveally (see Figure 1). Then, participants performed one of two tasks (randomly presented); in two thirds of the trials, they were instructed to decide whether a Chinese pictograph displayed for 150 ms at the fixation location was pleasant (right button press) or unpleasant (left button press). They were asked not to make an esthetic judgment, but rather follow a spontaneous intuition, and to provide an answer in less than a second. In the other third of trials, a question concerning the facial expression appeared at the fixation location instead of the Chinese pictograph, stating either “Happiness?” or “Anger?”. Participants answered “yes” by pressing the right button, or “no” by pressing the left button. The question remained on screen until a response was provided1.
Figure 1. Left panel: schematic description of the GCC procedure (Experiment 1). Each trial began with the presentation of a fixation cross, followed by a peripheral emotional face surrounded by flankers. This was followed by either a Chinese pictograph on which participants had to make an evaluative judgment (Preference Task), or by a question concerning the emotion expressed by the prime, on which participants had to answer by yes or no (Visibility Task). Experiment 1: peripheral happy or angry faces were presented statically for 2500 ms, surrounded by flankers that were either patterns or neutral faces. Experiment 2: peripheral happy, neutral, or angry faces were presented statically or dynamically for 1200 ms, surrounded by pattern flankers. Experiment 3: happy, neutral, or angry faces were presented foveally for 75 or 2500 ms, surrounded by pattern flankers. Right panel: schematic description of the gaze-contingent substitution. As long as participants’ gaze was maintained in a 5° × 5° region surrounding the fixation cross (“correct gaze”), the peripheral emotional face was presented among the flankers. As soon as participants’ gaze diverged from this region, the emotional face was replaced by the same face with a neutral expression (“gaze-contingent substitution”). In Experiment 2, which contained a neutral baseline condition, the substitution took place with the same face with a different neutral expression.
Analyses. Trials for which gaze position was correct during 95% of total facial expression duration were analyzed (representing 74.23% of total trials). In order to estimate the preference bias occurring during featural and configural crowding, we ran a 2 × 2 ANOVA with Participant as a random variable, Valence (happy vs. angry), and flanker Type (pattern vs. neutral face) as within-subjects factors and Group [(f-m-f-m) vs. (m-f-m-f)] as a between-subjects factor. The assumptions for use of ANOVAs and t-test were met (preference judgment responses followed a normal distribution (Shapiro–Wilk test, W = 0.98, p > 0.3) and variances were equal between conditions).
Participants rated the target with a mean pleasant response rate of 47.45% (SD = 12.02). The 2 × 2 ANOVA analysis revealed a main effect of valence [F(1,17) = 9.64; p < 0.01], reflecting the fact that participants rated the target as pleasant more frequently after being exposed to a happy compared to an angry face (4.52%, SD = 6.23). This effect of valence did not interact with flanker type nor with group (both F < 1; see Figure 2). All other main effects and interactions did not reach significance. Analysis of the facial expression visibility task revealed that participants performed at chance-level both when flankers were neutral faces [mean d′ = −0.15, t(18) = −0.73, p > 0.4] and patterns [mean d′ = 0.25, t(18) = 1.28, p > 0.2]. No significant difference between the two conditions could be found (p = 0.17). In order to control for the interaction between the preference bias and the discriminability of facial expressions, we relied on Greenwald’s regression method (Greenwald et al., 1995). This complementary analysis revealed both that the amplitude of preference bias did not correlate with discriminability (adjusted R2 = −0.05, p > 0.7), and that the preference bias extrapolated to null visibility was significantly above zero (intercept = 4.4%, p < 0.01), confirming that the effect we observed was genuinely nonconscious.
Figure 2. Comparative results of preference judgment in GCC (Experiment 1). Averaged percentage of pleasant response on the Chinese pictograph, depending on the emotion expressed by angry (dark gray) or happy faces (light gray), when surrounded by pattern flankers or neutral face flankers. Error bars denote one SE.
In sum, Experiment 1 revealed that crowded peripheral facial expressions induced a bias in subsequent preference judgments, both when they were surrounded by patterns (i.e., featural crowding) and by neutral faces (i.e., configural crowding). Complementing our previous study (Kouider et al., 2011), this result suggests that in addition to featural crowding, configural crowding allows for nonconscious configural face processing. However, whether the distinction between featural and configural processing is all or none or only quantitative remains debated (Riesenhuber et al., 2004; Yovel and Kanwisher, 2004). Thus, the existence of nonconscious configural processing should be comforted by other diagnostic tests, like the face composite effect (Young et al., 1987) or the face superiority effect (Tanaka and Farah, 1993).
In our previous study, we found that static and dynamic facial expressions biased preference judgments to the same extent (Kouider et al., 2011). However, when consciously seen, dynamic facial expressions were found to induce increased emotional responses, both behaviorally and physiologically (Sato et al., 2004; Trautmann et al., 2009). The lack of such increased response in our previous study might have resulted from potential differences in the amount of emotional information that could be extracted from static and dynamic stimuli. Indeed, while in the first 500 ms of presentation static faces were already at the paroxysm of emotion, the dynamic faces were still neutral. This resulted in a difference of roughly 20% on a total of 2500 ms of emotional expression, which might explain the similar response to the two types of stimuli.
To examine this interpretation, in Experiment 2 the duration of the videos was shortened to 1200 ms so that the discrepancy between the amount of information in static and dynamic stimuli was reduced to ∼8% (dynamic stimuli starting with a neutral expression of 100 ms, followed by emotional progression until paroxysm at t = 1000 ms, and dynamical maintenance at the peak level during 100 ms). In addition, Experiment 2 included a baseline condition, consisting in dynamic neutral faces moving the mouth randomly, or static neutral faces extracted from these videos (see Materials and Methods). This was done in order to assess which emotion (i.e., happiness or anger, expressed statically or dynamically) was able to modulate preference judgments. Finally, this experiment was conducted within rather than between-subjects, in order to exclude the possibility that the equivalent bias induced by static and dynamic stimuli stemmed from group differences (i.e., in our previous study, groups of participants exposed to static and dynamic stimuli had different mean pleasant response rates).
Materials and methods
Participants. Thirteen French college students (age range = 18–35 years) participated in Experiment 2. All reported normal or corrected-to-normal vision and were paid for their participation. All asserted that they were unable to read Chinese pictographs.
Stimuli. Stimuli were similar to the ones used in Experiment 1.
Procedure. Experiment 2 consisted of 360 priming trials and 120 visibility trials, equally divided to 4 blocks of 120 trials. The procedure was similar to the one used in Experiment 1, except for the following changes: first, only patterns were used as flankers. Second, each block contained either dynamic (d) or static (s) stimuli, which were happy, angry, or neutral faces, presented for 1200 ms. Two counterbalanced groups of participants were defined depending on the sequence of blocks they were assigned to [i.e., either (d-s-d-s) or (s-d-s-d)]. Third, in the visibility task participants had to answer one of these three questions regarding the emotional face: “Happiness?”, “Neutral?”, or “Anger?”.
Analyses. Trials for which gaze position was correct during 95% of total facial expression duration were analyzed (representing 91.18% of total trials). Modulation of the pleasant response rate was analyzed in a 3 × 2 ANOVA, with Participant as a random variable, valence (happy, neutral, angry), stimulus Type (dynamic vs. static) as within-subject factors, and Group [i.e., (d-s-d-s) vs. (s-d-s-d)] as a between-subject factor. A Shapiro–Wilk test assessed that preference judgment responses followed a normal distribution (W = 0.97, p > 0.05). Variances were equal between conditions.
The 3 × 2 ANOVA analysis revealed a main effect of valence [F(1,12) = 5.00; p < 0.05], but no interaction with stimulus type nor group (both F < 1). All other main effects and interactions did not reach significance. We then performed follow-up t-tests to differentiate the impact of happy and angry faces. We found that participants rated the Chinese pictographs as more pleasant when they were previously exposed to crowded happy faces compared with neutral faces [3.7%; t(12) = 2.51; SD = 5.2, p < 0.05]. No overall modulation was observed for angry faces [−1.1%; t(12) = −0.50; SD = 7.96, p > 0.6; see Figure 3].
Figure 3. Comparative results of preference judgment in GCC (Experiment 2). Averaged percentage of pleasant response on the Chinese pictograph, depending on the emotion expressed by angry (dark gray), neutral (gray), or happy faces (light gray). Error bars denote 1 SE. *Denotes a p-value < 0.05.
Analysis of the facial expression visibility task revealed chance-level performance [mean d′ = 0.13; t(12) = 0.77; SD = 0.64, p > 0.4], with no significant differences as a function of valence [F(1,12) = 1.46, p > 0.2] or stimulus type [F(1,12) = 0.78, p > 0.3]. The linear regression between the preference bias and the discriminability of facial expressions confirmed that the preference bias extrapolated to null visibility was significantly above zero in the case of happy expressions (intercept = 3.9%, p < 0.05) but not angry expressions (intercept = −0.9%, p > 0.7). The amplitude of preference bias did not correlate with discriminability in the case of happy expressions (adjusted R2 = −0.06, p > 0.6), nor angry expressions (adjusted R2 = −0.07, p > 0.6).
In sum, Experiment 2 revealed that the nonconscious origin of the preference bias that we obtained previously stems primarily from the processing of facial expressions conveying positive emotions. Their static or dynamic nature did not modulate significantly the preference bias amplitude. We come back to this issue in the discussion.
Experiment 3 aimed at testing whether the restriction to positive biases found in Experiment 2 was specific to nonconscious perceptual processes. We replicated Experiment 2 in condition of full visibility, using static emotional faces presented foveally. Faces were presented either for a long duration (2500 ms), or for a short duration (75 ms), in order to control for the existence of explicit strategies when participants were exposed to long-lasting visible faces (e.g., intentional inhibition of emotional influences). We reasoned that such a short duration will not allow subjects to develop these strategies especially since they were required to provide a preference judgment in less than a second.
Materials and methods
Participants. Twenty-one French college students (age range = 18–35 years) participated in Experiment 3. All reported normal or corrected-to-normal vision and were paid for their participation. All asserted that they were unable to read Chinese pictographs.
Stimuli. Same stimuli were used as in Experiment 1 and 2.
Procedure. Experiment 3 included 120 priming trials and 60 visibility trials, equally divided to 3 blocks of 60 trials. The procedure was similar to the one used in Experiment 2, except for the following changes: First, all stimuli were presented at fixation position, so to enable foveal perception. Second, emotional faces were all static and they were randomly presented either for a short duration (i.e., 75 ms) or for a long duration (i.e., 2500 ms) within a block.
Analyses. Results were analyzed with a 3 × 2 ANOVA, with Participant as a random variable, Valence (happy, neutral, angry) and stimulus Duration (short vs. long) as within-subject factors. A Shapiro–Wilk test assessed that preference judgment responses followed a normal distribution (W = 0.98, p > 0.4). Variances were equal between conditions.
Participants rated the target with a mean pleasant response rate of 59.79% (SD = 14.18). The 3 × 2 ANOVA analysis revealed a main effect of valence [F(1,20) = 17.03; p < 0.001], so that participants rated the target as more pleasant after being exposed to a happy face compared with a neutral one [10.07%; t(20) = 2.78; SD = 16.61; p < 0.05], and as less pleasant after being exposed to an angry face [−7.71%; t(20) = −2.65; SD = 13.34; p < 0.05] (Figure 4). We found no interaction of valence with stimulus duration [F(1,20) = 0.18; p > 0.6], suggesting that the influence of the short-lasting and long-lasting facial expressions were similar. This makes the potential involvement of explicit strategies less plausible. All other main effects and interactions did not reach significance.
Figure 4. Comparative results of conscious preference judgment bias (Experiment 3). Averaged percentage of pleasant response on the Chinese pictograph, depending on the emotion expressed by angry (dark gray), neutral (gray), or happy faces (light gray). Short stimulus duration and long stimulus duration conditions are averaged. Error bars denote 1 SE. *Denotes a p-value < 0.05.
Analysis of the visibility task revealed a main effect of stimulus valence [F(1,20) = 24.10; p < 0.001], showing that participants discriminated more accurately happy faces (mean accuracy = 92.86%) compared to angry faces (mean accuracy = 80.24%) and to neutral expressions (mean accuracy = 66.91%). There were significant discriminability differences between happy and neutral faces [paired t-test, t(20) = 8.18, p < 0.001], angry and neutral faces [paired t-test, t(20)= 3.60, p < 0.01], and happy and angry faces [paired t-test, t(20) = 4.90, p < 0.001]. Visibility was clearly above chance-level performance for all the three facial expressions (all p < 0.001). Furthermore, a main effect of stimulus duration [F(1,20) = 16.38; p < 0.001] revealed a greater accuracy for long-lasting compared to short-lasting stimuli (i.e., 84.13 vs. 75.87%). In sum, Experiment 3 revealed that when facial expressions are consciously visible, they can bias preference judgments toward negativity when they represent anger, and toward positivity when they represent happiness, this effect occurring regardless of stimulus duration.
Comparison with Visual Masking
The large majority of research on nonconscious perception has relied on the use of very brief stimulus durations (i.e., typically less than 50 ms) together with stimulus degradation through backward masking (Breitmeyer and Öğmen, 2006; Kouider and Dehaene, 2007). For instance, facial expressions presented very briefly and backward masked have been argued to affect brain regions dealing with emotional information (Morris et al., 1998; Whalen et al., 1998). Yet, such findings have proven difficult to replicate when visibility is stringently controlled (Pessoa, 2005). Here, we relied on a recent face priming paradigm that carefully controlled for stimulus visibility yet still demonstrated behavioral, electrophysiological, and hemodynamic responses to invisible stimuli, at least at the perceptual level (Henson et al., 2008; Kouider et al., 2009). In Experiment 4a, we adapted this masked face priming method to the preference judgment task, creating a masking equivalent to GCC. Experiment 4b was aimed at assessing whether the masked facial expressions were perceptually processed. There, participants had to indicate whether a target face was happy or angry, in case it was preceded by an identical masked face, and in case it was preceded by a masked face of different identity that expressed the same emotion.
Materials and methods
Participants. Eighteen French college students (age range = 18–35 years) participated in Experiment 4a, and 10 in Experiment 4b. All reported normal or corrected-to-normal vision and were paid for their participation. All asserted that they were unable to read Chinese pictographs.
Stimuli. Emotional faces were the same static pictures of happy or angry expressions used in Experiment 1. In order to produce a robust masking effect, faces luminance was 7.8 Cd m−2, 40% lower than for the masks, and peripheral facial attributes (e.g., hair, ears) were cropped (final size of both faces and masks was 3.2° × 3.9°). In Experiment 4a, targets were 3° × 3° white Chinese pictographs. In Experiment 4b, targets were either identical to the masked face, or different faces expressing the same emotion. Targets were 10% larger compared to the masked faces, in order to avoid physical overlap.
Equipment. Stimuli were displayed on a 17-in. Compaq Trinitron P700 monitor with a frame rate of 60 Hz and a resolution of 1024 × 768 pixels.
Procedure. Experiment 4a included 120 priming trials and 60 visibility trials. Each trial started with a 0.5° × 0.5° fixation cross presented for 300 ms, followed by a forward mask that was presented for 500 ms, an emotional face that appeared for 33 ms and then a backward mask presented for 50 ms (see Figure 5). In the preference task, a Chinese Pictograph appeared centrally for 150 ms, while in the visibility task, a question regarding the facial expression remained on screen until a response was provided. Instructions were identical to those used previously in Experiment 1.
Figure 5. Schematic description of the masking procedure (Experiment 4a). Each trial began with the presentation of a fixation cross, followed immediately by a forward mask pattern, and then by the emotional face accompanied by the backward mask pattern. This was followed by a Chinese pictograph (Preference Task), or a question concerning the emotion expressed by the face (Visibility Task). Experiment 4b employed the same procedure, except that the preference task was replaced by a perceptual priming task, in which the target was a face which was either identical or expressing the same emotion as the masked face.
Experiment 4b included 160 priming trials and 60 visibility trials. Each trial started with a 0.5° × 0.5° fixation cross for 300 ms. The masked faces were displayed in the same way as in Experiment 4a. In priming trials, the task was to decide, as quickly as possible, whether an emotional target face displayed for 200 ms at the fixation location expressed anger or happiness. Participants indicated their response with their left index finger if they thought the face was angry or right index finger if they thought it was happy. They were instructed to answer as fast as possible. In visibility trials the question concerning the masked face (“Happiness?” or “Anger?”) appeared at the fixation location instead of the target face. Participants answered “yes” by pressing the right button or “no” by pressing the left button. The occurrence of a visibility task was chosen randomly.
Analyses. A Shapiro–Wilk test assessed that preference judgment responses followed a normal distribution (W = 0.98, p > 0.1). Variances were equal between conditions.
In Experiment 4a, we failed to observe a significant preference bias of happy compared to angry masked faces [2.5%; t(18) = 1.22; SD = 9.12; p > 0.2; see Table 1]. Importantly however, the repetition priming experiment (Experiment 4b) that focused on perceptual processing of the emotional faces, revealed a significant effect of priming [8 ms; t(10) = 2.15; p < 0.05; see Table 2]. In both experiments, the visibility task confirmed that backward masking prevented conscious access to the facial expressions, as performance was at chance for both Experiment 4a (mean d′ = −0.05; t(18) = −0.67; SD = 0.3, p > 0.5) and Experiment 4b (mean d′ = 0.11; t(10) = 0.32; SD = 1.15, p > 0.7).
Performing a linear regression between the preference bias and the discriminability of facial expressions did not reveal a significant correlation (adjusted R2 = 0.02, p > 0.2). Furthermore, the preference bias extrapolated to null visibility was non-significantly different from zero (intercept = 2.9%, p = 0.18). In sum, the absence of effect in Experiment 4a suggest that the masking procedure we employed does not allow the emotional contents from biasing preference judgments. Yet, perceptual priming effects in Experiment 4b show that masking still allows for the extraction of perceptual information. We come back to this issue in the discussion.
Comparison with Continuous Flash Suppression
Another alternative for rendering stimuli invisible that has recently attracted attention is CFS (Tsuchiya and Koch, 2005). This technique, which is an extension of binocular rivalry, consists in presenting a dynamic stream of contour-rich, high-contrast patterns to one eye, in order to suppress the stimulus presented to the other eye from awareness. As in GCC, this method allows for long stimulus presentations, thus potentially increasing the strength of invisible signals. However, in CFS, the largest suppression appears to occur relatively early in the visual system (i.e., in sub-cortical and striate areas), and might thus reduce signal processing in visual ventral areas dealing with object and face recognition (see Lin and He, 2009, for a review). Here, we tested whether static facial expressions manipulated by CFS could bias preference judgments (Experiment 5a) and elicit repetition priming (Experiment 5b). In Experiment 5c, we extended the measure of biasing preference judgment to dynamic facial expressions.
Materials and methods
Participants. Fifteen French college students (age range = 18–35 years) participated in Experiment 5a, 9 in Experiment 5b, and 12 in Experiment 5c. All reported normal or corrected-to-normal vision and were paid for their participation. All asserted that they were unable to read Chinese pictographs.
Stimuli. Static stimuli in Experiment 5a and 5b were the same as those used in the static condition of Experiment 1. Dynamic stimuli in Experiment 5c were the same as those used in the dynamic condition of Experiment 2.
Equipment. Observers viewed the display through an adjustable mirror stereoscope (ScreenScope Mirror Stereoscope) on a 17′′. Compaq Trinitron P700 monitor with a frame rate of 60 Hz and a resolution of 1024 × 768 pixels.
Procedure. A 8.7 × 6.7° frame, composed of textured black and white bars (0.7° width) was presented in each eye to facilitate stable convergence of the two images. Mondrian patterns consisted of arrays of randomly generated shapes of different color and form (i.e., changing every 125 ms, 8 Hz), surrounded by a squared border corresponding to the area of stimuli presentation (8° × 6°). They were presented to left or right eye (“non-suppressed eye,” randomly chosen at each trial). Emotional faces were displayed for 2500 ms, either statically (Experiments 5a and b) or dynamically (Experiment 5c; see Figure 6). In order to keep the suppressed faces invisible, their luminance was decreased by a factor of 70% compared to the targets. Targets were 10% larger compared to the suppressed faces, in order to avoid physical overlap.
Figure 6. Schematic description of the CFS procedure (Experiment 5a). Each trial began with the presentation of a fixation cross, followed by a dynamic stream of alternating patterns (Mondrian) presented to one eye, while a static emotional face was presented to the other (suppressed eye). When the Mondrian stream stopped, either a Chinese pictograph (Preference Task) or a question concerning the emotion expressed by the face (Visibility Task) was presented to the suppressed eye. In Experiment 5b, except that the preference task was replaced by a perceptual priming task, in which the target was either an identical face expressing the same emotion as the suppressed face or a different face expressing the same emotion. Experiment 5c was identical to Experiment 5a, yet with dynamic faces.
Preference bias experiments (Experiments 5a and c) included 120 priming trials and 60 visibility trials, randomly intermixed. The preference valence task and the visibility task were the same as those used previously in Experiment 1.
The perceptual priming experiment (Experiment 5b) included 160 priming trials and 60 visibility trials, randomly intermixed. Targets were either identical to the suppressed emotional face, or consisted of different faces expressing the same emotion. The perceptual priming task and the visibility task were the same as those previously used in Experiment 4b. In each trial, both target and suppressed faces were always presented to the same eye.
Analyses. A Shapiro–Wilk test assessed that preference judgment responses followed a normal distribution in Experiment 5a (W = 0.96, p > 0.05) and 5c (W = 0.96, p > 0.05). Variances were equal between conditions.
In Experiment 5a, no significant preference bias from facial expressions was found [0.6%; t(14) = 0.24; SD = 9.14; p > 0.8; see Table 1]. However, in the repetition priming experiment (5b) a significant effect of repetition priming revealed that subjects identified the emotional expression of the target slower when primed with an identical emotional face [−18 ms; t(8) = −2.56, p < 0.05; see Table 2]. This is congruent with previous reports by our group of longer reaction times following prolonged stimulus exposure in either CFS (Barbot and Kouider, 2011) or crowding (Faivre and Kouider, 2011b). Indeed, we recently found that while short stimulus exposures (e.g., 200 ms), lead to positive repetition priming, longer exposures (e.g., 1000 ms) actually induce negative priming. This inversion of priming effects might be related to the fact that visual stimulation in the absence of perceptual awareness does not allow for a sustained representation of the encoded features and result in sensory habituation (see Faivre and Kouider, 2011b).
Stimulus visibility was not significantly better than chance, in both Experiment 5a [mean d′ = 0.15; t(14) = 1.98; SD = 0.29, p < 0.1] and Experiment 5b [mean d′ = 0.09; t(8) = 1.35; SD = 0.20, p > 0.2]. However, there was a marginal trend for visibility in Experiment 5a, which could indicate that some participants might have been aware of the facial expressions. In order to address this possibility, we relied once more on Greenwald’s regression method (Greenwald et al., 1995), which revealed that the preference bias extrapolated to null discriminability did not differ from zero (intercept = 0.9%, p > 0.7), and that the preference bias was not modulated by discriminability (adjusted R2 = −0.07, p > 0.7).
Finally, in Experiment 5c, we tested whether dynamic stimuli under CFS could bias preference judgments. No significant preference bias was found [−1.1%; t(11) = −0.5; SD = 8.82; p > 0.6], with chance-level performance in the visibility task (mean d′ = 0.19; SD = 0.48, p > 0.1). The linear regression between the preference bias and the discriminability of facial expressions revealed that the intercept did not differ significantly from zero (intercept = −1.5%, p > 0.5), and that the preference bias was not modulated by discriminability (adjusted R2 = −0.07, p > 0.6).
In sum, our results reveal that, as with masking, perceptual information from suppressed stimuli in CFS is extracted up to the point of inducing perceptual priming. As we did not find significant effect, we can not conclude that processing of static and dynamic facial expressions under CFS is deep enough to bias subjects’ subsequent preference judgments.
Face Perception under Visual Crowding
In a series of eight experiments, we characterized face processing outside perceptual awareness using three different techniques: GCC, visual masking, and CFS. In addition to providing further evidence regarding the nonconscious processing of emotional faces (see Tamietto and de Gelder, 2010, for a review), this comparative approach provide grounds for better understanding of the different suppression mechanisms, hereby allowing a more founded interpretations of findings regarding the depth of nonconscious processing.
First, we showed with GCC that peripheral stimuli conveying emotions (i.e., happy compared to angry faces) could bias subsequent preference judgments, both when featural and configural information processing was impeded by crowding2. In a second experiment which included a neutral baseline, we showed that this approach could be applied to dynamic stimuli (videos) as well, and that the preference bias arose specifically from faces expressing happiness. This restriction to positive preference bias was not observed in Experiment 3, in which facial expressions were presented foveally and were thus consciously visible. The existence of an influence from angry faces during foveal but not peripheral vision was confirmed by post hoc analyses, which revealed a significant difference in negative preference bias between Experiment 2 and 3 [Welch’s t-test, t(31.99) = 1.81, p < 0.05]. The difference in positive preference bias between Experiment 2 and 3 did not reach significance (p > 0.1; see Figure 7).
Figure 7. Comparative results of preference judgment bias (Experiments 1, 2, 3, 4a, and 5a,c). Averaged percentage of preference bias from the comparison of angry vs. neutral faces (dark gray, Experiments 2 and 3), happy vs. neutral faces (light gray, Experiments 2 and 3), and happy vs. angry faces (white, Experiments 1, 4a, and 5a,c). GCC stands for gaze-contingent crowding. CFS stands for continuous flash suppression. Horizontal bars denote interactions between negative preference bias in Experiment 2 and 3, and interactions between positive preference bias in Experiments 1, 2, and 4a, 5a,c (see meta-analyses in the discussion). Error bars denote 1 SE. *Denotes a p-value < 0.05, **p-value < 0.01.
Possibly, under visual crowding the lower signal from angry faces was not sufficient to bias preference judgments. This is corroborated by previous results showing that peripheral presentation of faces was found to globally impair expression recognition, except for happy expressions (Goren and Wilson, 2006). This apparent advantage for happy faces might be related to the fact that the diagnostic features for the recognition of happiness involve mouth in the low spatial frequency spectrum, while the recognition of anger is rather driven by the eyes (i.e., frown) through higher spatial frequencies (Smith and Schyns, 2009). This is further supported by the higher recognition rate of happiness compared to anger during foveal viewing (see Experiment 3). However, this advantage might have also resulted from the higher similarity between the negative expressions and the neutral ones, compared with the positive expressions. Such similarity in our stimuli might have led to an underestimation of preference bias when contrasting neutral and angry expressions.
Our findings shed new light on the phenomenon of crowding itself, whose origins remain highly controversial (Levi, 2008). Crowding occurs when an object and its flankers are integrated in the same integration field, resulting in a jumbled percept in which the object cannot be discriminated. It has been shown that the size of the integration field increases with eccentricity in the visual field, following an empirical psychophysical law (Bouma law, Bouma, 1970). However, the reason for this widening remains highly debated. According to bottom-up proposals, integration fields are materialized in the primary visual cortex (Pelli, 2008), which consequently implies a loss of information for crowded stimuli at and beyond V1. Our results tend to argue against this view, and rather suggest that the information from crowded stimuli is still present in the visual system, but remains inaccessible to conscious awareness (thus being able to influence later preference judgments). From this perspective, crowded perception would only reflect a partial conscious read-out of information, while further processing occurs in a nonconscious manner.
Consistent with a belated occurrence of crowding along the visual pathways, several studies have shown that single-feature crowded stimuli (e.g., line orientation, see He et al., 1996; Montaser-Kouhsari and Rajimehr, 2005; Faivre and Kouider, 2011b) and multifeature crowded objects (e.g., faces or directional arrows, see Faivre and Kouider, 2011a) are processed despite the fact that they remain consciously undiscriminable. In other words, during featural crowding, a feature is nevertheless processed despite featural interactions with the flankers. Finally, this study shows that such processing can also occur despite configural crowding, when face flankers interfere with configural processing. This finding extends our previous study in which no bias of preference judgments was found for inverted as opposed to up-right faces, suggesting that the bias stems from the faces’ configural information. Here, our results suggest that crowding allows for a nonconscious configural face processing despite configural interactions with the flankers. Clearly, other pieces of evidence for the configural processing of crowded faces will be necessary, like the face superiority effect (Tanaka and Farah, 1993), or the face composite effect (Young et al., 1987).
In Experiment 2, we found that GCC could be used to demonstrate the influence of static (i.e., pictures) and dynamic (i.e., videos) happy faces on subsequent preference judgment. Notably, the amplitude of preference bias was not significantly different between the two types of stimuli. This seems to contradict the findings that dynamic facial expressions are better recognized during conscious processing than static ones. Surely, it remains possible that the behavioral measure we employed was not sensitive enough to detect a differential effect. Nevertheless, these results might also imply that nonconscious processing does not allow the temporal binding of dynamic stimuli into unified perceptual representation. In that respect, GCC seems to offer good means for the study of nonconscious temporal integration, a fundamental property that has far not been investigated so far.
Comparing Different Methods for Preventing Perceptual Awareness
Our study provides unique insights regarding the level of processing of facial expression under three different techniques preventing perceptual awareness. While GCC allows for the processing underlying a bias in subsequent preference judgments, only lower-level repetition priming effects were found under visual masking and CFS. Post hoc analyses confirmed this finding, by showing a significant difference regarding the preference bias from happy faces under GCC (i.e., difference in preference judgment after exposition to happy vs. angry faces in Experiments 1 and 2) and masking/CFS [i.e., difference in preference judgment after exposition to happy vs. angry faces in Experiments 4a, 5a, and 5c; 6.10 vs. 0.96%, Welch’s t-test, t(55.73) = 2.20, p < 0.05; see Figure 7]. Separate analyses revealed a significant difference between GCC (Experiments 1 and 2) and CFS [Experiments 5a,c; Welch’s t-test, t(50.21) = 2.41, p < 0.05], but not between GCC (Experiments 1 and 2) and masking (Experiment 4; p > 0.3).
The absence of significant preference bias with masking might arise from a lack of statistical power (19 participants, bias = 2.5%, SD = 9.12, resulting in a power d = 0.21; see Cohen, 1988). On the other hand, the drastic degradation required in order to prevent perceptual awareness might dramatically reduce stimulus strength and notably the reentrant activity associated with higher-level visual processes, so that the masked stimulus might not reach the level of processing necessary for biasing preference judgments (Lamme and Roelfsema, 2000). The results we found in Experiment 4a and 4b are consistent with previous rigorous analyses (Pessoa, 2005), and question the existence of influences from emotional faces beyond perceptual levels.
Regarding CFS, the lack of significant preference bias is in line with theories of binocular rivalry that hold drastic suppression to occur early along the visual pathways (i.e., in sub-cortical and striate areas), hereby impeding processing in higher visual regions of the ventral stream (Fang and He, 2005; Jiang and He, 2006). Along the same lines, categorically congruent stimuli suppressed using CFS were found to facilitate the categorization of tools (assumed to be processed in the dorsal visual pathway), yet not that of non-manipulable objects that are assumed to be processed in the ventral visual pathway (Almeida et al., 2008). We should mention, on the other hand, evidences for high-level processes during CFS in case of faces (Jiang et al., 2007; Adams et al., 2010) or complex visual scenes (Mudrik et al., 2011).
Drawing conclusions from the results obtained with masking or CFS as taken separately, one might be prone to infer that nonconscious processing of facial expressions is essentially limited to perceptual rather than higher emotional levels. Yet, our results with GCC argue otherwise, since the biasing of subsequent preference judgment induced by crowded stimuli demonstrates that high-level processing is possible despite perceptual unawareness. This suggests that the absence of significant results with masking and CFS do not reflect a theoretical limitation of nonconscious cognition, but rather a methodological one. In this respect, the set of data we report provides a good example of the importance of methodological comparisons in the context of a contrastive study of consciousness. It also highlights the need for a rigorous methodological inquiry of the mechanisms underlying the different methods of rendering stimuli invisible, and for a cautious interpretation of results when relying on single methods.
One interesting feature of crowding compared to masking and interocular suppression is its ecological relevance. Indeed, outside a laboratory, one rarely faces a stimulus for a few tens of milliseconds, immediately followed by another backward masking stimulus sharing some physical properties (i.e., same position, size, spatial frequency spectrum). Similarly, one rarely receives a stimulus in one eye, while the other eye is bombarded with rapidly changing flashes. By contrast, crowding is ubiquitous in spatial vision (e.g., while reading, walking among a crowd), and peripheral stimuli are of indubitable relevance as we evolve in cluttered environments. For obvious reasons, it remains necessary to adapt crowding to a well-controlled environment (e.g., in GCC, substituting the stimulus as soon as the observer’s gaze diverges from a defined location). Nevertheless, relying on a perceptual modality which occurs during natural vision may not be trivial to uncover the depth of nonconscious cognition. The present demonstration of high-level nonconscious effects which are unique for the GCC technique, makes it a valuable alternative for the study of the depth of nonconscious processing. While crowding is considered as a bottleneck to object recognition (Whitney and Levi, 2011), the amount of environmental information that can be consciously accessed is thus reduced, while informational encoding is maintained to a degree that suffices to influence subjects’ behavior.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The authors thank the two reviewers for their helpful comments. Special thanks go to Claire Bradley, Isabelle Brunet, Anne-Caroline Fievet, and Arnaud Piketty for technical assistance, and to Julien Dubois and Liad Mudrik for their help with the manuscript. This work was supported by a starting grant from the European Research Council to Sid Kouider.
- ^It is of note that this discrimination measure offers two main advantages. First, the use of a meta-cognitive discrimination task (Kouider et al., 2011; Faivre and Kouider, 2011b) in which a probe question about the stimulus is presented prevents potential influences of the crowded stimulus on the objective visibility measure itself (Cheesman and Merikle, 1986). Indeed, while meta-cognitive access is thought to be specific to conscious cognition (e.g., Lau and Rosenthal, 2011), more classical discrimination tasks (e.g., always pressing the right button for happy faces vs. the left button for angry faces; see Kouider and Dehaene, 2007) may be resolved through nonconscious stimulus-response mapping, which would lead to an overestimation of stimulus visibility (Abrams and Greenwald, 2000; Damian, 2001). Here, since the question is chosen randomly on each trial, a meta-cognitive access to the facial expression is necessary in order to perform a correct answer (e.g., in case a happy face was presented, being able to answer “yes” to the question “Happiness ?” or “no” to the question “Anger ?”). In addition, because the upcoming task (i.e., preference judgment or visibility task) is determined randomly on each trial, it allows measuring visibility on-line, in alternation with the preference bias. The visibility measure is then sensitive to potential changes in perceptual thresholds occurring during the experiment (e.g., training or fatigue effects). Furthermore, as participants cannot predict which task they will encounter, they must attend to the peripheral face as well for the two tasks. This attentional amplification has even been shown to increase nonconscious effects (Faivre and Kouider, 2011a). Overall, we believe that this meta-cognitive discrimination task intermixed with preference judgment task, provides a more accurate estimate of conscious visibility over the whole experiment.
- ^It could be argued that the preference bias arises from a change in the appearance of the flankers due to the flanked facial expression. If it was the case, participants would probably be able to discriminate this change, as crowding is less important for the flankers than for the central stimulus. As we showed that participants did not perform above chance-level during the visibility task, this makes this hypothesis unlikely. Yet, if the appearance of flankers changed but remained unnoticed due to crowding, this would reflect a nonconscious influence from the flanked facial expression, which is compatible with our claim.
Baars, B. (1997). “Contrastive phenomenology: a thoroughly empirical approach to consciousness,” in The Nature of Consciousness: Philosophical Debates, eds N. Block, O. Flanagan, and G. Guzeldere (Cambridge: MIT Press), 187–201.
Greenwald, A. G., Klinger, M. R., and Schuh, E. S. (1995). Activation by marginally perceptible (“subliminal”) stimuli: dissociation of unconscious from conscious cognition. J. Exp. Psychol. Gen. 124, 22–42.
Kanai, R., Walsh, V., and Tseng, C. (2010). Subjective discriminability of invisibility: a framework for distinguishing perceptual and attentional failures of awareness. Conscious. Cogn. 19, 1045–1057.
Sato, W., Kochiyama, T., Yoshikawa, S., Naito, E., and Matsumura, M. (2004). Enhanced neural activity in response to dynamic facial expressions of emotion: an fMRI study. Brain Res. Cogn. Brain Res. 20, 81–91.
Trautmann, S. A., Fehr, T., and Herrmann, M. (2009). Emotions in motion: dynamic compared to static facial expressions of disgust and happiness reveal more widespread emotion-specific activations. Brain Res. 1284, 100–115.
Whalen, P. J., Rauch, S. L., Etcoff, N. L., McInerney, S. C., Lee, M. B., and Jenike, M. A. (1998). Masked presentations of emotional facial expressions modulate amygdala activity without explicit knowledge. J. Neurosci. 18, 411–418.
Keywords: consciousness, face processing, emotion, crowding, GCC, CFS, masking
Citation: Faivre N, Berthet V and Kouider S (2012) Nonconscious influences from emotional faces: a comparison of visual crowding, masking, and continuous flash suppression. Front. Psychology 3:129. doi: 10.3389/fpsyg.2012.00129
Received: 13 March 2012; Paper pending published: 26 March 2012;
Accepted: 11 April 2012; Published online: 03 May 2012.
Edited by:Morten Overgaard, Aalborg University, Denmark
Reviewed by:Kristian Sandberg, Aarhus University Hospital, Denmark
Talis Bachmann, University of Tartu, Estonia
Copyright: © 2012 Faivre, Berthet and Kouider. This is an open-access article distributed under the terms of the Creative Commons Attribution Non Commercial License, which permits non-commercial use, distribution, and reproduction in other forums, provided the original authors and source are credited.
*Correspondence: Nathan Faivre, Laboratoire de Sciences Cognitives et Psycholinguistique, Ecole Normale Supérieure, 29 rue d’Ulm, 75005 Paris, France. e-mail: firstname.lastname@example.org