This study involved two experiments, each with 15 undergraduate students. In experiment 1, there were 10 female participants and 5 male participants, all with a mean age of 24 years. Experiment 2 consisted of 12 female participants and 3 male participants, with a mean age of 25 years. All participants were psychology majors at the California State University of San Bernardino, and they received class credit in appreciation of their participation. Prior to their involvement, each student provided informed consent, and no student participated in more than one experiment.

To generate the image dataset used for Experiment 1, original face images from 26 white and 26 black models (equally distributed across gender) were selected from the RADIATE database (Conley et al., 2018). This database provides open-access face stimuli, featuring racially and ethnically diverse models displaying various emotional expressions. Two emotional expressions, namely “happy” and “angry,” were chosen for each model, resulting in a set of 104 unique face images. To expand the dataset, an AI aging filter (FaceApp) was employed to create an “older version” of each model, effectively doubling the number of stimuli. Subsequently, the dataset was doubled again by digitally adding a facemask to each face image, employing Adobe Photoshop. As a result, the dataset was expanded to include a total of 416 face images. For Experiment 2, the initial image dataset from Experiment 1 was halved by removing all “angry” faces. Then, the dataset was brought back to a total of 416 images by adding an inversed version of the remaining faces. Throughout both experiments, all face images were thus equally divided based on several attributes, including race (black/white), gender (male/female), age (young/old), use of face mask (mask/no mask), and emotional expression (happy/angry). In Experiment 2, an additional attribute, face orientation (upright/inverted), was considered, in place of emotional expression. Figure 1A illustrates examples of the stimuli used for experiments 1 and 2, featuring one black and one white male and female model and the stimuli derived from it.

The experiment involved participants wearing a 64-channel EEG-cap (BrainVision), with only 32 channels effectively utilized (sampling rate of 500 Hz) and the reference electrode placed at the FCz location, following the standard 10–20 EEG system. The EEG electrodes were connected to a BrainVision actiCHamp active channel amplifier (BrainVision) and checked for proper conductivity (impedance below 5kΩ for each electrode) before starting the recording. During the recording session, participants were seated in a dimly lit, quiet room, in front of a 19-inch Dell monitor, positioned 50 cm away from their heads. The experiment presented 416 face images (see “Stimuli”) sequentially, each displayed at the center of the screen with a visual angle of 17° × 23°. The order of the images was randomized to minimize any potential biases. To ensure participants’ attention, they were instructed to indicate the gender of each face using a button box (see Figure 1B). Each face was displayed for at least 1 s, and disappeared from the screen as soon as it was evaluated, followed by a 1-s inter-trial interval displaying a cross at the center of the screen before the next face appeared (see Figure 1B). Participants were encouraged to respond as quickly and accurately as possible. Any images not evaluated within 4 s disappeared from the screen to maintain the experimental flow. The entire experimental paradigm was created using Experiment Builder by SR Research.

Participants viewed 416 images, which were presented one by one in random order. Their sole task was to identify whether each face image depicted a female or a male by using a button box (see Figure 1B).

The 416-image stimulus set was purposely designed to be split into two sets of 208 images each, based on five distinct face attributes (see Figure 1A for examples). Consequently, this division resulted in five pairs of grand averaged ERP waveforms. These pairs were categorized as follows: “young” and “old” (see Figure 2A), “white Caucasian” and “Black African American” (see Figure 2B), “male” and “female” (see Figure 2C), “mask” and “no mask” (see Figure 2D), and “angry” and “happy” (see Figure 2F). Additionally, for each participant, the trials were divided into two groups: those presented first (trials 1 to 208) and those presented last (trials 209–416) (see Figure 2E).

To visualize and analyze the differences in the measured EEG signals between the selected pairs of conditions (see Materials and methods), the following features were included in each panel: a shaded error band representing ± standard error for each waveform, short horizontal lines under each pair of waveforms to indicate the timepoints at which the waveforms statistically differed from each other (with alpha = 0.01), and asterisks (see caption Figure 2) to indicate differences in amplitudes between the waveforms for the key ERP components (P100, N170, and P200) that are relevant to this study, based upon a non-parametric test (see Materials and methods).

Results, obtained through a 3-way ANOVA, aimed at exploring interactions among the binary dimensions of “age,” “emotion,” and “mask,” as well as “gender,” “race,” and “mask,” and their impact on the amplitudes of the P100, N170, and P200 components, are presented in Supplementary Tables S1, S2. Complementing these findings, Supplementary Figures S1, S2 show topographic maps depicting amplitude distributions for these components under various conditions.

Following experiment 1, a group of 15 new participants was recruited to replicate the study, incorporating notable modifications. Specifically, the “emotional expression” factor was omitted, and a new “stimulus orientation dimension” was introduced. These adjustments were informed by the findings of experiment 1, which highlighted substantial differences, particularly in N170 and P200 amplitudes, between masked and unmasked faces. In contrast, no discernible distinctions were observed between “angry” and “happy” faces. This alignment with the concept that emotional effects on early vision may not necessarily signify an influence of cognitive processes (Raftopoulos, 2023). Because similar amplitude differences have been reported for inverted faces (Rossion et al., 2000), the introduction of face stimulus orientation provided an opportunity to compare and study the effect of both modulations, and hence learn more about the underlaying neural mechanisms. For examples of stimuli used for Experiment 2, and how they differ compared with Experiment 1, see Figure 1A.

Similar to the results shown for Experiment 1 (see Figure 2), the panels in Figures 3A–E illustrates the results obtained from splitting the data based on one pair of experimental conditions. To examine the effect of inverting the stimuli, upright (left subpanels) and inverted (right subpanels) stimuli were analyzed separately. Additionally, to compare the effects of both masks and stimulus orientation, the data was split into four groups: no masks and masks, either upright or inverted (see Figure 3F).

The results, derived from a 3-way ANOVA, were directed toward investigating interactions among the binary dimensions of “stimulus orientation” and “mask,” along with either “age” (Supplementary Table S3), “gender” (Supplementary Table S4), or “race” (Supplementary Table S5), and their influence on the amplitudes of the P100 and N170 components (see Supplementary material). To complement these findings, Supplementary Figures S3–S5 present topographic maps illustrating amplitude distributions for these components under conditions involving masked vs. unmasked stimuli and upright vs. inverted faces.

Table 1 provides a comprehensive summary of amplitude differences for all ERP components considered in both experiment 1 and experiment 2. A non-parametric randomization test was employed to compare amplitudes between any two experimental conditions (see Materials and methods). Additionally, this test helped identify latency shifts (not shown in Table 1) between different conditions for different ERP components. When analyzing the EEG response separately for inverted or upright face image stimuli, no latency shift was observed between masked and unmasked images. However, the response to inverted unmasked faces showed a significant delay compared to upright unmasked faces for the N170 (6 ms, p = 0.017) and the P200 (20 ms, p < 0.00001). Inverting masked faces also exhibited a delay, albeit not statistically significant (4 ms, p = 0.08), but a significant delay for the P200 (20 ms, p < 0.00001) compared to upright masked faces. For a more detailed interpretation of these findings, please see the “Discussion” section below.

This study found that N170 amplitudes increased with appearent complexity of processing faces. Older faces, for instance, known to be more challenging in terms of emotional expression recognition (Grondhuis et al., 2021), elicited larger N170 responses, noted in both Experiment 1 and 2, consistent with prior research (Wiese et al., 2008). Interestingly, the inversion of faces abolished this age-related difference (see Figure 3), although a separate study observed larger inversion effects for young faces compared to old faces (Wiese et al., 2008). The presence of a face mask, which also imposes additional processing demands, was also found to increase N170 amplitudes and P200 responses in both experiments, aligning with recent studies reporting similar N170 increments due to face masks (Prete et al., 2022; Proverbio and Cerri, 2023). It’s noteworthy, however, that another study (Żochowska et al., 2022) did not observe changes in N170 amplitudes with face masks. Remarkably, none of these studies noted a corresponding increase in P200 amplitude, possibly indicative of cognitive task variations.

Another critical aspect impacting face processing is facial inversion, a phenomenon evident across all conditions in our study except for the “masked face condition” (see below). This effect resulting in larger N170s is well-documented (Rossion et al., 2000; Rousselet et al., 2004; Sadeh and Yovel, 2010), been observed to impact the N250 (Hashemi et al., 2019; Abreu et al., 2023), an ERP component that falls outside the scope of this study. The absence of familiarity with faces (Ito and Urland, 2003), giving rise to the “own race bias effect,” has also been linked to heightened challenges in processing faces, resulting in larger N170 responses to other-race faces (Sun et al., 2014; Yao and Zhao, 2019). However, it’s important to note that the current study did not identify any sensitivity of early ERP components to race or skin color. This absence of sensitivity can be attributed to the predominant representation of Latino participants in the sample, which may not offer the requisite diversity to thoroughly investigate own-race effects. Furthermore, gender-based differences may affect N170 amplitudes, as evidenced by larger N170s for male faces, primarily due to the gender imbalance in our participant pool. Nevertheless, this finding warrants further investigation with a more balanced participant pool.

One plausible explanation for the enhanced N170 response to inverted faces lies in the early recruitment of additional neural mechanisms, rather than a simple increase in activity within existing neural populations during the N170 time-window. These findings align with Rossion’s hypothesis, which posits the involvement of object-sensitive neurons in augmenting the N170 amplitude observed for inverted faces (Rossion et al., 2000). This hypothesis may also be extended to elucidate the increased N170 response observed in our study for masked faces, as masks themselves can be considered objects. Intriguingly, the study demonstrates that the addition of face masks has a comparable effect on N170 amplitude as inverting the maskless face stimulus. However, combining a mask and inversion did not lead to an additive increase, suggesting a potential neural saturation point.

The delay in the N170 response to inverted faces is often linked to alterations in the spatial relationships among facial features. Additionally, an amplification of the N170 component and a corresponding shift in latency have been associated with a reduced ability to recognize faces. For instance, a study demonstrated that gradually rotating facial images from an upright to an upside-down position resulted in a declining ability to identify faces (Jacques and Rossion, 2007). However, it’s important to emphasize that the reduced face recognition ability alone may not completely explain the observed delay in both the N170 and P200 components. This becomes evident in the present study, where no such delay was observed for faces with face masks compared to unmasked faces, despite face masks also significantly impeding face identification (Freud et al., 2022).

The P200 component is well-established as being linked to configural face encoding, with more typical faces consistently yielding larger P200 amplitudes (Wuttke and Schweinberger, 2019). Conversely, as deviations from the norm increase, relatively smaller P200 amplitudes are typically observed (Halit et al., 2000; Latinus and Taylor, 2006; Schweinberger and Neumann, 2016). An intriguing revelation from our study is that both masked faces, as observed in our current investigation, and scrambled faces (Latinus and Taylor, 2006), elicit substantial P200 amplitudes. In contrast, a similar robust increase in the P200 component was not observed in response to facial inversion. This intriguing finding suggests that faces lacking spatial and configurational information may be processed as highly typical or in a default state, with the amplitude modulation being influenced by the presence of spatial face information.

Moreover, the inversion of faces was found to induce delays in both N170 and P200 latencies. This phenomenon has been noted previously (Latinus and Taylor, 2006), as well as in the case of Mooney faces compared to other face types. However, it is crucial to highlight that, unlike in our current study, previous research primarily emphasized the delay in N170 latency, as indicated by N170-to-P200 peak analyses. In contrast, this study revealed that the observed latency for inverted faces increased from 4–6 ms (N170) to approximately 20 ms (P200), demonstrating that this delay is not limited to one specific processing stage.

The investigation of diminished ERP components resulting from repetition effects typically involves comparing responses to two identical faces or faces sharing common attributes (e.g., identity) vs. responses to two distinct faces. For an in-depth review, see (Schweinberger and Neumann, 2016). Previous studies (Amihai et al., 2011; Walther et al., 2013) have demonstrated that the N170 component is influenced when preceded by another face, regardless of whether the sequentially presented faces represent the same individual or different individuals. The present study provides further insight by showing that the categorical face adaptation effect accumulates over the duration of a session, leading to a gradual decline in N170 amplitude throughout the experimental session, as depicted in Figures 2E, 3E. Even when upright and inverted face stimuli were interleaved and separately analyzed, the apparent “neural adaptation effect” remained consistent across sessions. It’s noteworthy that this phenomenon has also been observed in data collected from the occipital lobe of rhesus macaque monkeys using non-face stimuli (Brunet et al., 2014). Therefore, researchers utilizing block designs to compare experimental conditions should consider this effect to ensure the reliability and validity of their interpretations.

It is essential to acknowledge certain limitations in our study. The homogeneity of our participant pool, primarily consisting of college students identifying as Hispanic/Latine, may influence our findings. Furthermore, the gender distribution in the sample was skewed toward females, suggesting the need for a more balanced participant pool in future investigations.