Visual Mismatch Negativity Reflects Enhanced Response to the Deviant: Evidence From Event-Related Potentials and Electroencephalogram Time-Frequency Analysis

Introduction

One of the important functions of the human cognitive system is to detect changes in the environment and make a response. Evidence that the brain can rapidly process discrete and unusual changes in events has been widely confirmed by studies using the event-related potentials (ERPs) technique (Kenemans et al., 2003). One crucial finding is that mismatch negativity (MMN), an ERP component, emerges due to violation of the probability in a sequence of sensory stimuli, reflecting the brain’s automatic detection of information changes (Kimura et al., 2011; Czigler, 2014). In early research, MMN was usually studied in the auditory domain (aMMN, Näätänen, 1990; Näätänen et al., 2001, 2005). However, many subsequent studies (Kimura, 2012; Stefanics et al., 2014b) have shown that the phenomenon also exists in the visual system, where it is known as visual mismatch negativity (vMMN). The vMMN is an ERP index of automatic processing of unattended visual information, which has been widely used to study the neural mechanism underlying a visual change detection process (Kimura et al., 2011; Stefanics et al., 2011; Li et al., 2012; Czigler, 2014; Kremláèek et al., 2016; Chen et al., 2020).

In vMMN studies, the passive oddball paradigm is often used to present events with high or low probability. In a sequence of visual stimuli (e.g., OOOXOOXOOO…), the frequently presented visual stimulus (e.g., O) is called the standard stimulus and the infrequent stimulus (e.g., X) is the deviant stimulus. In a reverse oddball sequence, the roles of the stimuli are reversed, i.e., the deviant becomes the standard, and the standard becomes the deviant. Comparison of the visual stimulus-evoked ERPs reveals more negative-going waveforms induced by deviant stimuli than those by standard stimuli at the posterior scalp electrodes from 150 to 400 ms following the stimulus onset (Kimura, 2012). The difference of the ERPs between the deviant and the standard stimuli is called vMMN. However, because of the differences in the presentation probability of deviant-standard stimulus pairs (D-S pairs), standard stimuli with high probability are more likely to cause refractoriness, such as S in SSSSD. It may also be D in DDDS. S is more likely to appear; S, therefore, establishes more repetitions. The difference wave may be affected by stimulus-specific refractoriness (Kimura et al., 2010; Stefanics et al., 2014b; Ruusuvirta, 2021).

Three hypotheses have been developed to explain vMMN: the sensory fatigue (or refractoriness) hypothesis, the memory trace hypothesis, and the predictive coding hypothesis. The sensory fatigue hypothesis holds that when stimuli are repeated, neural activity is usually weakened (Näätänen and Picton, 1987; Grill-Spector et al., 2006; Ruusuvirta, 2021). This repetition effect on neural activity has been confirmed using many methods, including the recording of individual cortical neurons (Caggiano et al., 2013; Kilner et al., 2014), the use of functional magnetic resonance imaging (Heleven and Overwalle, 2019; Rostalski et al., 2019), and the use of EEG/ERPs (Jonas et al., 2014; Stefanics et al., 2020). As mentioned above, when using the oddball paradigm to study the automatic processing of visual changes, the standard stimulus is repeatedly presented. The earliest visual refractoriness can occur 200 ms after the onset of the stimulus (Shen et al., 2017; Dravida et al., 2018; Feuerriegel et al., 2019). According to the sensory fatigue hypothesis, the vMMN effect is caused by decreased ERPs of the standard stimuli, not by increased responses to the deviant one (Nelken and Ulanovsky, 2007; May and Tiitinen, 2009).

The second hypothesis, known as the memory trace, holds that the vMMN is the result of automatic comparison between current and previous stimuli (Näätänen and Picton, 1987). The theory is that repetition of the standard stimuli would leave memory traces (or templates), and the brain would automatically compare the neural representation of the subsequent stimuli and discern the differences (Näätänen, 1990). While the brain would not respond to standard stimuli that match the traces, it would respond to a mismatch between deviant stimuli and the memory traces, resulting in deviant-related mismatch negativity. Therefore, from the perspective of memory traces, the vMMN effect is caused by the enhancement of the ERPs elicited by the deviant stimuli rather than the weakening ERPs elicited by the standard ones (Winkler, 2007; Paavilainen, 2013).

The memory trace hypothesis was interpreted as a “regularity violation” hypothesis by later researchers (Winkler, 2007). That is, the vMMN signal shows the difference between the current stimuli and the expectation based on previous information. It not only represents the memory traces but also the relationship between the previous stimuli (Stefanics et al., 2009, 2011; Paavilainen, 2013). Building on this, Garrido et al. (2008) posited a hierarchical predictive coding framework to explain the mismatch process. The predictive coding hypothesis assumes that the human brain is a nested system of different levels in which error signals are passed up and predictions are passed down. Repeated or interactive information transferred between different levels enables the model to be optimized or adjusted to select the optimal interpretation for the current sensory information input (Friston, 2005, 2010). According to the predictive coding hypothesis, the vMMN effect is caused by both the enhancement of the prediction error to the deviant stimuli and the weakening of the prediction error to the standard stimuli (Friston, 2005; Stefanics et al., 2014a,2018).

In the present study, we used the concept of “probability” rather than “violation of regularity” because “probability” is the description of the whole sequence, while “violation of regularity” is the microscopic description. The fact that stimuli with low probability violate the regularity established by stimuli with high probability is more likely to occur.

Some researchers have used the equiprobable paradigm to study vMMN and to control the refractoriness on vMMN (Czigler et al., 2002; Kimura et al., 2009, 2011; Kimura, 2012; Li et al., 2012; Kovarski et al., 2017). There are two kinds of sequences in the equiprobable paradigm, one oddball and one equiprobable. The oddball sequence is similar to the one in the oddball paradigm, including the deviant (with a probability of 20% occurrence, for example) and the standard stimuli (e.g., 80%). In the equiprobable sequence, the same deviant and standard stimuli as in the oddball sequence are used, and some other different stimuli are randomly intermixed, each with an equal probability (e.g., 20%), matching the probability of the deviant in the oddball sequences. The difference wave is obtained by subtracting the ERPs elicited by the deviant stimuli in the equiprobable sequence from the ERPs elicited by the same deviant stimuli in the oddball sequence, and these two ERPs are evoked by the same physical stimuli with the same probability. There is supposedly no regularity in the equiprobable sequence. Regularity in the sequence is the only difference between the deviant in the oddball sequence and control in the equiprobable sequence. Thus, the equiprobable vMMN is considered a genuine vMMN without stimulus-specific refractoriness effects (Stefanics et al., 2014b; Ding et al., 2020).

Previous studies of vMMN have used simple visual stimuli to study the effect of visual refractoriness on differential waves (Czigler et al., 2002; Kimura et al., 2009). Other studies used complex visual information (such as facial expression) as experimental materials, but the conclusion has been controversial (Li et al., 2012; Kreegipuu et al., 2013; Kovarski et al., 2017). For example, Li et al. (2012) and Kovarski et al. (2017) used the equiprobable paradigm to study the emotion-related vMMN effect. However, the results were partly inconsistent. Both studies presented equiprobable sequences and oddball sequences to the subjects. In the equiprobable sequence, faces with varied emotions were presented with equal probability, while in the oddball sequence, neutral faces (frequently presented) and fearful faces (or angry faces, infrequently presented) were presented. The ERPs elicited by the standard stimuli in the oddball sequence and the control stimuli in the equiprobable sequence were each subtracted from the ERPs elicited by the deviant stimuli in the oddball sequence, resulting in the oddball vMMN and control vMMN, respectively. Li et al. (2012) found that the early oddball vMMN (110–210 ms from stimulus onset) was more negative than the control vMMN suggesting that the oddball vMMN in the early stage was based on visual refractoriness, while the late oddball vMMN (210–310 ms) was not significantly different from the control vMMN suggesting that the oddball vMMN in the late stage was without refractoriness effects. However, Kovarski et al. (2017) found that the vMMN effect had no significant difference in both early and late stages. The deviant-standard stimulus pairs of the oddball vMMN showed differences in both presentation probability and low-level features, while the deviant-control stimulus pairs of the control vMMN had no differences in both dimensions. That is, it is unclear whether the difference between oddball vMMN and control vMMN is caused by the presentation probability (similar to refractoriness) or by the difference in low-level features. Therefore, further study is needed to explore this issue.

Our study adopted a variant of the oddball paradigm to explore the effects of visual refractoriness and low-level features on the vMMN effect. Specifically, participants were presented with an oddball sequence, a reverse oddball sequence, and an equiprobable sequence. The ERPs elicited by the standard stimuli in the oddball sequence, the standard stimuli in the reverse oddball sequence, and the deviant/control stimuli in the equiprobable sequence were each subtracted from the ERPs elicited by the deviant in the oddball sequence to obtain oddball vMMN (labeled vMMN1), reverse oddball vMMN (labeled vMMN2), and equiprobable, or control vMMN (labeled vMMN3), respectively. Previous studies used only two of these sequences, so the effect of visual refractoriness or low-level features could only be excluded. In the equiprobable paradigm, there were differences in low-level features and probability between the deviant and the standard stimuli in the oddball sequence (vMMN1), while in the equiprobable sequence, the pair had neither probability nor low-level feature differences (vMMN3). However, in the reverse oddball paradigm, the pair in vMMN2 had only probability difference. vMMN1 and vMMN3 were compared to verify whether visual refractoriness affected vMMN. In contrast, the deviant-standard stimulus pairs in vMMN1 and vMMN3 were different not only in the presentation probability (visual refractoriness) but also in low-level features. The pairs in vMMN2 and vMMN3 were different only in the presentation probability (i.e., visual refractoriness) but the same in low-level features, which allows a direct comparison of the true effects of visual refractoriness excluding low-level features.

Based on the memory trace hypothesis, we aimed to show that vMMN is not affected by ERP weakening of standard stimuli (visual refractoriness), so that there would be no significant difference between vMMN2 and vMMN3. Otherwise, based on the view of refractoriness (sensory fatigue hypothesis), the difference between vMMN2 and vMMN3 would be significantly different. In addition, our study focused on whether low-level features affect the vMMN effect, that is, whether there is a significant difference between vMMN1 and vMMN2.

Additionally, we investigated the effects of low-level features and refractoriness in the neural sources of vMMN. It has been suggested that the sources of vMMN are localized in the superior frontal gyrus, middle frontal gyrus, middle occipital gyrus, inferior parietal gyrus, medial temporal lobe, superior temporal gyrus, cingulate gyrus, insula, and precuneus (Li et al., 2012; Xu et al., 2013; Vogel et al., 2015; Chen et al., 2020). We anticipated that differences in low-level features and refractoriness would affect the sources in these regions of the brain. We also explored the visual mismatch neural oscillatory responses (vMORs). Presently, few studies have focused on vMORs. Task-irrelevant deviant stimuli induce greater alpha oscillation (Tugin et al., 2016) or greater theta oscillation (Hesse et al., 2017; Yan et al., 2017) compared to the standard stimuli. The evidence so far is mixed (Stothart and Kazanina, 2013; Hesse et al., 2017; Yan et al., 2017; Chen et al., 2020). Most studies focus on the power of vMORs in the alpha band (8–13 Hz) and theta band (4–7 Hz). Therefore, our study focused on the oscillatory response of vMMN in both alpha and theta bands.

Materials and Methods

Participants

Twenty-one students were recruited from Guangzhou University and participated for payment or course credit. One participant’s data were not used because the impedance was difficult to drop below 5,000 Ω. The final sample comprised 20 participants (10 males, 10 females, mean age = 19.30 years, SD = 1.20 years). All participants were right-handed, with normal or corrected-to-normal vision. They were naive about the purpose of the experiment and all participants signed informed consent before the experiment. After the experiment, we told them that the sequence of faces was made up of different expressions. The study was approved by the local institutional research board (IRB) of the Department of Psychology, School of Education, Guangzhou University.

Materials and Procedure

The stimuli consisted of 10 human faces of different identities (5 males, 5 females). The emotions include fearful, anger, happy, surprise, and neutral. All face pictures were selected from the NimStim Database (Tottenham et al., 2009). According to NimStim, fearful face are correctly identified by 72% of observers. Other rates of correct identification are listed as 87% for angry faces, 98% for happy faces, 82% for surprise faces, and 57% for neutral faces. Each gray picture extended to a visual angle of approximately 6°horizontally and 6°vertically.

Participants sat comfortably in a dimly lit and electrically shielded room, at a viewing distance of approximately 60 cm. The experiment was implemented using E-Prime software (version 3.0, Psychology Software Tools, Inc., Pittsburgh, United States). The stimuli were presented on a 17-inch cathode ray tube monitor with a refresh rate of 85 Hz and a resolution of 1,024 × 768 pixels. At the beginning of each trial, two faces identical in features and emotions with dimensions of 0.5° × 0.5°were presented to the left and the right of the fixation cross. A distance of 6°separated the center of the face picture and the fixation cross. In each trial, the face image disappeared after 300 ms and then the screen presented only the fixation cross for 400–700 ms (Figure 1). As in previous vMMN studies (Stefanics et al., 2012; Chen et al., 2020), participants were asked to focus on the fixation cross during the display of faces and to discriminate and respond to changes in the fixation cross. The frequency of change of the fixation cross was 0.15. When it changed, the horizontal or vertical line could grow from 0.5 to 1.0° for 300 ms.

FIGURE 1

Figure 1. A task schematic of the sequence shows the time course of the stimulus presented for 300 ms, followed by the cross displayed on a gray screen for 400–700 ms.

The experimental procedure consisted of three sequences. The oddball sequence comprised 576 trials of neutral faces (standard stimulus, presented at 80% probability) and 144 trials of fearful faces (deviant stimulus, 20%). The reverse oddball sequence comprised 576 trials presenting fearful faces and 144 trials of neutral faces. The equiprobable sequence comprised 144 trials (20%) each for fearful, angry, happy, surprised, and neutral faces. To exclude the influence of differences in low-level features from the influence of emotions, the faces in each trial were randomly presented, and the order of the trials also was randomly determined. At least one standard stimulus was presented between two deviant stimuli. The participants were asked not to blink while viewing the sequence in the experiment. To prevent fatigue, each sequence was divided equally into 6 blocks, each lasting about 1.5 min, during which the participants could determine the duration of the rest time. The presentation order of the three sequences among the participants was balanced.

Recordings

Electroencephalogram (EEG) recordings were done with Ag/AgCl electrodes placed at 64 locations on an elastic electrode cap (NeuroScan, Texas, United States). The nose tip served as the online reference electrode. The horizontal electrooculogram was recorded by placing two electrodes 1 cm from the external canthi of the left and right eye. The vertical electrooculogram was recorded by placing two electrodes 2 cm above and below the left eye. The EEG data were sampled at a rate of 1,000 Hz and were filtered online (0.05–100 Hz bandpass). The impedance of all electrodes was maintained at a level below 5,000 Ω throughout the recording session.

The EEG data were bandpass filtered offline at 0.1–30 Hz with a Hamming window finite impulse response filter (24 dB/oct slope). Epochs from −200 to 699 ms after stimulus onset were extracted from the continuous EEG data. Data in the interval from −200 ms to stimulus onset served as the baseline. Independent component analysis was used to identify artifact components related to eye movements. The following epochs were excluded: (1) epochs that contained changes in the fixation cross, (2) epochs that contained participants’ responses, (3) epochs with potential values exceeding ± 75 μV on an EEG channel. The frequency of rejected epochs was 28.4% (SD = 5.8%). There were 299–470 effective standard trials and 65–115 effective deviant trials remaining.

Analysis

Behavior Data

To test whether the participants were in a state of inattention to faces, this study analyzed the response accuracy in behavioral data. If the subject paid attention to the fixation cross, high percentage changes in the fixation cross gaze should be captured, otherwise, the hit rate would be low or the false alarm rate would be high. We did not analyze response time because we were most interested in accuracy rather than reaction time.

Event-Related Potentials and Difference Waves

The EEG signals of six kinds of conditions, characterized by the combination of stimulus type (deviant and standard) and sequence type (oddball sequence, reverse oddball sequence, and equiprobable sequence), were superimposed and averaged, and the corresponding ERP waveforms were obtained within the time interval of −200 through 699 ms.

In vMMN studies, the difference wave is obtained by subtracting the ERP of the standard stimulus (or the ERP of the corresponding face in the equiprobable sequence) from the ERP of the deviant stimulus. Previous studies have shown that the differential wave generally peaks in the parietal occipital region within 100–400 ms (Kimura et al., 2011; Kimura, 2012). Three kinds of vMMN were calculated by subtracting the ERP of the standard (or control) stimulus in different sequences from that of the deviant stimulus in the oddball sequence. Here we use the shorthand Fearful 1, Fearful 2, and Fearful 3 and Neutral 1, Neutral 2, and Neutral 3 in describing the values for different emotion types. Specifically, Neutral 1 (80%, the standard) in the oddball sequence was subtracted from Fearful 1 to obtain the fearful face vMMN1. To obtain vMMN2, Fearful 2 (80%, the standard) was subtracted from Fearful 1. For vMMN3, Fearful 3 (20%, the equiprobable stimuli) was subtracted from Fearful 1. According to the MMN calculation method, the neutral face in the reverse oddball sequence (Neutral 2) can also be used as the deviant stimulus. Therefore, neutral face vMMN1, neutral face vMMN2, and neutral face vMMN3 were obtained by subtracting the amplitude for Neutral 2 from Fearful 2, Neutral 1, and Neutral 3, respectively.

In the subsequent discussion in this manuscript, the six vMMNs are abbreviated as FvMMN1, FvMMN2, FvMMN3 for fearful face-related vMMNs, and NvMMN1, NvMMN2, and NvMMN3 for neutral faces.

Based on the data from previous vMMN studies and the present study, we selected as the analysis object the mean ERP amplitude within the 101–350 ms range in the parietal-occipital region (labeled PO3, PO5, PO7 in the left hemisphere and PO4, PO6, PO8 in the right hemisphere). We used repeated-measure analysis of variance (ANOVA) to examine the main effects and interactions of different factors. Factors we considered were stimulus type (deviant, standard), emotion type (fearful face, neutral face), sequence of standard stimulus (vMMN1, vMMN2, vMMN3), and hemisphere (left hemisphere, right hemisphere). According to previous studies, vMMN generally appears in the range of 100–350 ms. In this study, a flexible time processing method with 5 50-ms-long time intervals was adopted to measure the mean amplitude of ERP in the range of 101–350 ms, and the time interval was taken as a factor in the ANOVA. Therefore, a five-factor repeated-measure ANOVA was performed for the ERP amplitude, which included stimulus type × emotion type × sequence × hemisphere × time interval. The p-value was corrected using the Greenhouse-Geisser method when necessary, but the degree of freedom was reported as the uncorrected value. LSD correction was used for post hoc comparisons. Time-frequency analysis uses the same method. Since the object of this study was the mismatched negative wave, we considered only the main effect or interaction of the stimulus type.

Source Localization Analysis

Source localization can identify and map brain regions associated with specific stimulus responses or behaviors, further improving spatial resolution from the analysis of sensor patterns (Baillet et al., 2001). In this study, we used standardized low-resolution electromagnetic brain tomography to analyze the source locations of three kinds of vMMN with different emotions (Pascual-Marqui, 2002). Using the ICBM152 template to compute the head model (Fonov et al., 2009), we calculated the cortical three-dimensional distribution of the scalp EEG current source density by constrained solution. This method provides a standardized, discrete, three-dimensional, distributed, linear, minimum-norm inverse solution to the inverse problem of the brain source location. However, it locates only the gray matter of the brain and does not take into account the depth of the source.

The source analysis started from the trial level. The noise covariance matrix was calculated within the range of −200 to −1 ms, and the source activity from 0 to 699 ms was calculated using a unified head model. Data for the source activity of each subject under each condition were obtained by stacking the average of each trial. Data for source activity in the time range −200 to −1 ms were taken as the baseline, and the source activity data of 0–699 ms were converted into a Z-value, then spatial smoothing was performed with a resolution of 3 mm. To examine the source locations of different vMMNs, the source in the given time interval would be averaged to a normalized value (Z-score transformation, the source activity of −200 to −1 ms would be served as baseline). The paired permutation t-test was carried out between the normalized map induced by the deviant and the normalized map induced by the standard. To test the difference of source locations of vMMN with different emotions and different sequences, the difference between the source induced by the deviant and the standard was calculated, using the formula Source_vMMN=Z_D−s. Paired permutation t-test was used to compare the sources of vMMN1, vMMN2, and vMMN3 with different emotions.

The statistical significance test was conducted using the Monte-Carlo approach in the paired permutation tests, using 5,000 random samples. The results of multiple comparisons between source signals were corrected for false discovery rate, and the critical threshold of a two-tailed test with an alpha level of 0.05 was used to determine whether the difference was significant.

Time-Frequency Analysis

The Morlet wavelet transform method was used for time-frequency decomposition (Bertrand and Tallon-Baudry, 2000). First, to obtain the time-frequency average of each trial, we calculated the non-standardized time-frequency power within the time range -200 to 699 ms of the trial level, using the time resolution of 2 s and the center frequency of 0.5 Hz, and averaged the result. To keep the time-frequency signal stable, the average value of all tests was calculated, and the average value of each test was subtracted before calculating the time-frequency power. Then, using the range of -200 to 0 ms as the baseline, we used decibel conversion to normalize the time-frequency power data within the 0–699-ms range. The conversion formula is dB = 10 × lg (energy after baseline / average energy baseline), and we obtained the time-frequency energy graphs of the standard and the deviant stimuli. Then, we calculated the difference between the deviant (D) and the standard (S) values to obtain the visual mismatch oscillatory responses (vMORs), using the formula vMORs = dB_D−dB_S.

Similar to the ERP analysis, our study focused on the dB values of the theta band (4–7 Hz) and alpha band (8–13 Hz) in the parietal-occipital region, labeled PO3, PO5, and PO7 in the left hemisphere and PO4, PO6, and PO8 in the right (Chen et al., 2020). Data from different electrode points in the same hemisphere were averaged point-to-point. The main effects and interactions of different factors were examined by repeated-measure ANOVA, with the factors of the frequency band (theta, alpha), emotion type (fearful, neutral), sequence (vMORs1, vMORs2, and vMORs3), hemisphere (left, right), time interval (7–100-ms-long consecutive intervals in the 0–699-ms range) and the type of stimulus (deviant, standard). We aimed to study the vMORs, therefore we focused only on the main effect of the stimulus type or its interaction.

Repeated-measure ANOVA was performed using SPSS 23.0 software. EEGLAB (Delorme and Makeig, 2004) and ERPLAB (Calderon and Luck, 2014) were used to preprocess EEG and extract ERP values. Source location analysis, time-frequency analysis, and permutation tests were performed with the Brainstorm3 toolbox (Tadel et al., 2011). The EEGLAB, ERPLAB, and Brainstorm3 functions were accessed in MATLAB 2016b (MathWorks).

Results

Behavioral Data

The behavioral data in the experiment showed that the average correct rate of fixation change detection was 97% (85–99%, SD = 2.8%), suggesting that participants focused on the fixation cross, leaving task-irrelevant emotional faces unattended.

Event-Related Potentials and Difference Waves

Figure 2A shows the ERP waveforms of vMMN1, vMMN2, and vMMN3 for fearful and neutral emotion types in the parietal-occipital region, and the topographic map of the average amplitude within the 100–350-ms range. Consistent with previous studies on vMMN, the difference wave emerged within that time range.

FIGURE 2

Figure 2. (A) The vMMN scalp maps and waveforms in parieto-occipital areas. (B) The significant source activation of vMMN and significant source activation of paired permutation t-test of different vMMN (no significant region was found).

The ANOVA revealed that the main effect of stimulus type was significant $F(1,19)=4.80,p=0.041,{\mathrm{\eta }}_p^2=0.202$. The ERP amplitude induced by the deviant was significantly larger than that of the standard, indicating a significant vMMN in the parietal-occipital region during the time range 101–350 ms. The interaction between stimulus type and time interval was significant, $F(4,76)=2.76,p=0.047,{\mathrm{\eta }}_p^2=0.127.$ Post hoc analysis found that the interaction was mainly caused by vMMN within 251–300 ms [$F(1,19)=8.18,p=0.010,{\mathrm{\eta }}_p^2=0.301$], 201–250 ms [$F(1,19)=4.75,p=0.042,{\mathrm{\eta }}_p^2=0.200$], and 301–350 ms [$F(1,19)=4.57,p=0.046,{\mathrm{\eta }}_p^2=0.194$], and 0.0.164 ps = 0.634 in other time intervals. The interaction of stimulus type × hemisphere was significant, $F(1,19)=8.11,p=0.010,{\mathrm{\eta }}_p^2=0.299$. Post hoc analysis showed that the interaction was mainly caused by the vMMN effect of the right hemisphere [$F(1,19)=6.75,p=0.018,{\mathrm{\eta }}_p^2=0.262$], indicating that vMMN had right hemisphere dominance.

In addition, the interaction of stimulus type × emotion type × time interval × hemisphere was marginally significant, $F(4,\mathrm{\hspace{0.17em}76})=2.66,p=0.057,{\mathrm{\eta }}_p^2=0.123$, indicating that the hemisphere effect of emotional vMMN varied by time interval. Since the emotional effect of vMMN was one of our research issues, we conducted data analysis in five time windows, respectively, and conducted a three-factor repeated-measure ANOVA of stimulus type × emotion type × hemisphere (for scalp distribution, see Figure 3).

FIGURE 3

Figure 3. From top to bottom: The vMMN scalp maps of mean amplitude in the time range of 101–150, 151–200, 201–250, 251–300, and 301–350 ms.

In the range of 101–150 ms, the interaction between stimulus type × emotion type was significant $[F(1,19)=5.54,p=0.030,{\mathrm{\eta }}_p^2=0.226)]$, and the vMMN effect of fearful emotion $[F(1,\mathrm{\hspace{0.17em}19})=5.05,p=0.037,{\mathrm{\eta }}_p^2=0.210]$ was significantly greater than that of neutral emotion (p = 0.302). From 151 to 200 ms, the interaction of stimulus type × hemisphere was significant $[F(1,\mathrm{\hspace{0.17em}19})=7.62,p=0.012,{\mathrm{\eta }}_p^2=0.286]$. The vMMN effect of the right hemisphere $[F(1,19)=5.15,p=0.035,{\mathrm{\eta }}_p^2=0.213]$ was significantly greater than that of the left hemisphere (p = 0.564). The vMMN effect was significant $[F(1,19)=4.75,p=0.042,{\mathrm{\eta }}_p^2=0.200]$within 201–251 ms. The interaction of stimulus type × hemisphere was significant $[F(1,\mathrm{\hspace{0.17em}19})=4.77,p=0.042,{\mathrm{\eta }}_p^2=0.201]$. The vMMN effect of the right hemisphere $[F(1,\mathrm{\hspace{0.17em}19})=6.24,p=0.022,{\mathrm{\eta }}_p^2=0.247]$was significantly greater than that of the left hemisphere (p = 0.099). The interaction of stimulus type × emotion type × hemisphere was marginally significant $[F(1,19)=3.89,p=0.063,{\mathrm{\eta }}_p^2=0.170]$, and the negative bias of the left hemisphere $[F(1,\mathrm{\hspace{0.17em}19})=4.18,p=0.055,{\mathrm{\eta }}_p^2=0.180]$was significantly higher than that of the right hemisphere (p = 0.412). In the time range 251–300 ms, only the main effects of stimulus type were significant $[F(1,\mathrm{\hspace{0.17em}19})=8.18,p=0.010,{\mathrm{\eta }}_p^2=0.301]$. From 301 to 350 ms, the vMMN effect was significant $[F(1,\mathrm{\hspace{0.17em}19})=4.57,p=0.046,{\mathrm{\eta }}_p^2=0.194]$, stimulus type × hemispheric interaction was significant $[F(1,\mathrm{\hspace{0.17em}19})=7.47,p=0.013,{\mathrm{\eta }}_p^2=0.282]$, and the vMMN effect of the right hemisphere $[F(1,19)=6.60,p=0.019,{\mathrm{\eta }}_p^2=0.258]$ was greater than that of the left hemisphere (p = 0.141).

A post-hoc test of four-factor repeated-measure ANOVA mentioned above found negative bias and right hemisphere dominance of the vMMN effect, which is typical in emotional vMMN studies. The remaining five-factor repeated-measure ANOVA showed no significant interaction with the type of stimulus (0.102 < ps < 0.713).

Source Localization Analysis

Source location and source location differences (Figure 2B) were tested for average vMMN activity within 100–350 ms. Sources of FvMMN1 were located in the lateral occipital lobe, inferior parietal lobe, superior parietal lobe, inferior temporal lobe, cuneus, and precuneus on both sides of the brain. Sources of FvMMN2 were located in the lateral occipital lobe, inferior temporal lobe, inferior parietal lobe, fusiform gyrus, glossal gyrus, precuneus, and right cuneus. The source of NvMMN1 was located in the left fusiform lobe. FvMMN3, NvMMN2, and NvMMN3 had no significant activation regions.

An examination of source location differences revealed that no differences were found in vMMN1 vs. vMMN2 nor vMMN1 vs. vMMN3 nor vMMN2 vs. vMMN3, regardless of emotion (Figure 2B), indicating that, according to the average activity in the range of 100–350 ms, the low-level features and repetition effect of the standard (or visual refractoriness) did not affect the source of difference of vMMN.

The paired permutation t-test showed no significant differences in FvMMN1 vs. NvMMN1, FvMMN2 vs. NvMMN2, and FvMMN3 vs. NvMMN3 across the whole brain range, suggesting no significant differences in the sources of different emotion-related vMMN.

Time-Frequency Analysis

Figure 4 shows the time-frequency maps of different vMORs in the parietal-occipital region. Repeated-measure ANOVA for time-frequency data showed that the main effect of stimulus type was not significant [$F(1,\mathrm{\hspace{0.17em}19})=0.244,p=0.627,{\mathrm{\eta }}_p^2=0.013$]. The interaction of stimulus type × frequency band was significant [$F(1,\mathrm{\hspace{0.17em}19})=12.30,p=0.002,{\mathrm{\eta }}_p^2=0.393$]. Post hoc analysis showed that the vMORs of the theta frequency band (mean 0.148, 95%CI: [0.009 0.287], p = 0.038) were significantly larger than that in the alpha band (-0.091, 95%CI: [-0.231 0.048], p = 0.187). The interaction of stimulus type × frequency band × time interval was significant [$F(6,\mathrm{\hspace{0.17em}114})=2.71,p=0.048,{\mathrm{\eta }}_p^2=0.125]$. Post hoc analysis showed that the vMORs in the theta band were positive (ps = 0.052) in the 100–199 and 200–299-ms ranges, while the vMORs in the alpha band were not significant (ps = 0.222). The vMORs of the alpha band were negative (ps = 0.015) and the theta band was not significant (ps = 0.193) in the 400–499 and 500–599-ms ranges. The interaction of stimulus type × emotion × time interval was significant [$F(6,\mathrm{\hspace{0.17em}114})=2.84,p=0.050,{\mathrm{\eta }}_p^2=0.130$]. Post hoc analysis showed that within 100–199 ms, the vMORs of the fearful face were significantly positive [$F(1,\mathrm{\hspace{0.17em}19})=5.76,p=0.027,{\mathrm{\eta }}_p^2=0.233]$, while those of neutral face were not (p = 0.860). The vMORs of the fearful face (ps = 0.074) and neutral face (ps = 0.307) in other time intervals were not significant. The interaction of the six factors was marginally significant $[F(12,\mathrm{\hspace{0.17em}228})=2.17,p=0.066,{\mathrm{\eta }}_p^2=0.103]$, and the results were no longer analyzed.

FIGURE 4

Figure 4. Visual mismatch oscillatory responses in different emotion and sequence conditions at electrode sites of parieto-occipital areas.

Discussion

The results showed significant vMMN in the time range of 100–350 ms and vMORs effects in the theta band, as well as a negative bias. However, there were no significant differences among the three kinds of responses measured by the oddball (vMMN1/vMORs1), the reverse oddball (vMMN2/vMORs2), and the equiprobable paradigms (vMMN3/vMORs3), regardless of emotion type. The results suggest that low-level physical information and the repetition effect (i.e., refractoriness) did not affect vMMN (vMORs). In addition, the source localization showed consistent results.

Different Paradigms in the Visual Mismatch Negativity Study

We found that there was no significant difference in the effects of vMMN among different sequences, indicating that vMMN from all three paradigms can be used as an indicator of automatic detection.

Previous studies on automatic processing of visual information generally used the oddball paradigm (Zhao and Li, 2006; Astikainen and Hietanen, 2009), the reverse oddball paradigm (Stefanics et al., 2012; Chen et al., 2020), or the equiprobable paradigm (Czigler et al., 2002; Kimura et al., 2009; Li et al., 2012; Kovarski et al., 2017). In the oddball paradigm, there are differences in low-level features between the deviant and the standard stimuli, and the probability of these two kinds of stimuli also is different. With the reverse oddball paradigm, only the difference in probability is present. In the equiprobable paradigm, there are neither differences in probability nor differences in the low-level features. Our evidence showed no significant differences in the vMMN effects among the three paradigms. The differences of low-level features and the differences of probability in the deviant-standard stimulus pair seemed not to affect the vMMN effects, suggesting that the use of vMMN is an effective and robust index of automatic detection.

The current study obtained a significant vMMN effect, which was not affected by low-level physical features. This result was contrary to the conclusion of a recent study. Male et al. (2020) repeated the experiment (orientation vMMN) of Kimura et al. (2009) and studied the vMMN effects of contrast, phase, and spatial frequency. There was no significant deviant-minus-same standard vMMN effect, but a significant early ERP that might index low-level visual irregularities, i.e., N1 and P1, was found. A potential possibility is that, as suggested by Male et al. (2020), vMMN cannot reflect the automatic processing of low-level visual features. If so, facial emotion-related vMMN obtained in this study should not belong to this category. In fact, the latency of ERP induced by face are later, such as N170 and N250. The time window of these components is very consistent with that of vMMN effect. However, this conclusion needs more targeted research to confirm.

Memory Trace Rather Than Refractoriness Accounts for Visual Mismatch Negativity

Our results suggest that the vMMN effects in the time range of 100–350 ms are not affected by the repetition effect, indicating that vMMN index is an enhanced response to the deviant.

This is consistent with the findings of Kovarski et al. (2017) but does not fully support the findings of Kimura et al. (2009) and Li et al. (2012), which concluded that the early (before 210 ms) vMMN is a repetition effect. These studies compared responses to oddball (vMMN1) and control (vMMN3) sequences in the same group of participants. However, the studies seemed to confuse the low-level information effect with the repetition effect, because the deviant-standard stimuli pairs were not different in the presentation probability (e.g., 20 and 80% in vMMN1, 20 and 20% in vMMN3) and also in the low-level information (e.g., anger face for deviant and neutral face for standard in vMMN1, and anger face for both deviant and standard in vMMN3). In the current study, the deviant stimulus in vMMN2 (vMORs2) and vMMN3 (vMORs3) was the fearful face in the oddball sequence (20%), while the standard stimulus was the fearful face in the reverse oddball sequence (80%) and the fearful face in the equiprobable sequence (20%), which differed only in probability. Therefore, the differences between vMMN2 (vMORs2) and vMMN3 (vMORs3) can be assumed to be the result of the repetition effect. According to the view of refractoriness (Nelken and Ulanovsky, 2007; May and Tiitinen, 2009), repetition of the standard stimulus weakens neural activity, resulting in the emergence of vMMN. However, the current study did not find that repetition of the standard stimulus had a significant influence on vMMN. Our results indicate that the vMMN effect is not due to refractoriness of repeated stimuli, but rather an enhanced response to rare stimuli, which supports the memory trace hypothesis (Winkler, 2007; Paavilainen, 2013). The results favor theoretical accounts that describe vMMN as an indicator of “memory trace” (Winkler, 2007; Paavilainen, 2013) rather than refractoriness accounts (Nelken and Ulanovsky, 2007; May and Tiitinen, 2009) or predictive coding view (Stefanics et al., 2014a,2018).

A recent study has also examined the effect of repetition on visual mismatch responses (VMRs, (Feuerriegel et al., 2021). Feuerriegel et al. (2021) used the fast visual stimulus presentation paradigm. In the sequence, the base face stimuli appeared periodically at a frequency of 6 Hz. Either the expected face (standard stimulus) or a surprise face (deviant stimulus) appeared every 7 faces. The probability of the surprise stimulus in each sequence varied from 10 to 90% and that of the expected stimulus varied from 90 to 10%. Therefore, there were 50% neutral stimuli (corresponding to the equiprobable stimuli). The results showed that in the parieto-occipital region, ERPs induced by the surprise stimuli were significantly more negative than those induced by the expected stimuli, meaning that VMRs were observed. On the other hand, there were no significant differences between the neutral (50%) and expected stimuli (90%), suggesting that VMRs reflect an enhanced response to the surprise stimuli rather than a decreased response to the expected stimuli. In this study, we compared ERPs of the deviant (20%) with those of the equiprobable (20%) and also showed a significant vMMN, which was consistent with Feuerriegel et al. (2021). To be noted, the control of visual refractoriness was stricter in the current study.

Several studies have shown that task-irrelevant deviant stimuli (e.g., line orientation or facial emotion) can lead to greater alpha oscillations (Tugin et al., 2016; Chen et al., 2020). Some researchers considered that the enhancement of the alpha band was the inhibition of response to stimuli (Klimesch, 2012). Some studies have also found that deviant stimuli induced larger theta-band oscillations (Stothart and Kazanina, 2013; Hesse et al., 2017; Yan et al., 2017). Our results were similar to Stothart and Kazanina (2013) in that a deviant emotional face induced greater theta oscillation and smaller alpha oscillation.

However, vMORs were not affected by the presentation probability of the standard stimuli, indicating that vMORs reflect increased oscillation with the deviant stimuli rather than reduced oscillation with the standard stimuli. Combined with the results of ERP, we found that the changes in visual information caused the enhancement of both amplitude and neural oscillation. Some researchers believe that theta-band oscillation is related to facilitatory and inhibitory attention (Karakas, 2020), or an inherent feature of the attention network in support of top-down guided attention (Helfrich et al., 2019). For the deviant stimuli, a stronger inhibition mechanism of the brain was required, compared to frequently presented standard stimuli.

In addition, the results of source analysis showed that the sources of different vMMN had no significant difference in the whole brain, which was consistent with the results of time-frequency analysis and ERP. In general, we got relatively consistent results that the repetition effect of standard stimuli did not affect vMMNs/vMORs, suggesting that vMMNs/vMORs index the deviant stimuli, supporting the memory trace view of vMMNs/vMORs.

The Negative Bias of Emotion-Related Visual Mismatch Negativity

We found that the vMMN effect of fearful faces was significantly greater than that of neutral faces within 100–150 ms, indicating that the brain has a negative bias toward the automatic processing of emotions.

Negative emotion is related to a negative internal state, and the emergence of negative emotion may be related to the brain’s perception of dangerous information. Recently, studies have shown that relative to positive emotions, negative emotions may be processed automatically at a lower level of consciousness (Chen et al., 2020) and may more easily break through the threshold of unconsciousness (Jiang et al., 2018), and may more quickly affect the amygdala (Méndez-Bértolo et al., 2016). We found that among the emotion-related vMMN studies, three studies used fearful- or angry-neutral face stimuli pairs (Kovarski et al., 2017; Rosburg et al., 2019; Kask et al., 2021). Consistent with these studies, the fearful-related vMMN effect is significantly greater than the neutral-related one. In addition, we analyzed the neural oscillatory response and found that vMORs were also sensitive to facial emotion. Specifically, vMORs in the theta band were significantly positive for the fearful faces, and neutral emotion was not significant from zero, indicating that deviant fearful faces induced greater theta-band oscillation compared with the standard fearful face.

A meta-analysis of the P3 amplitude showed that pictorial stimuli were more likely to induce negative biases than text stimuli (Yuan et al., 2019), implying that negative bias exists widely, not only in the automatic processing of facial emotion. Infrequently presented negative emotions can activate more intense brain activity than infrequent non-negative emotions. This may be related to the important function of the brain in detecting threats in the environment, thus increasing the chance of survival.

Limitations and Future Directions

One limitation of our study is that we used fearful-neutral stimulus pairs as deviant-standard stimulus pairs. The conclusion of negative bias may not be convincing enough compared with the study of fearful-happy pairs, because the intensity of response to neutral emotion may be different from that to negative or positive emotion. A majority of studies involving emotion-related vMMN use negative emotion and positive emotion as stimuli. The stimuli we used had correct identification rates of only 57 and 72% for neutral and fearful emotions, respectively. Emotional faces with a higher discrimination rate might produce more reliable results in a future study.

Recently, some studies compared the differences between color change automatic processing and face automatic processing and found that the traditional vMMN and computational modeling methods reached a consistent conclusion (Stefanics et al., 2018). Future research could combine low-level visual stimuli (such as color) with high-level visual stimuli (such as facial emotion) to explore vMMN variations of different sequences.

Summary

Our study analyzed the oddball-vMMN, reverse oddball-vMMN, and equiprobable-vMMN. We found that the vMMN effect was significant within the range of 100–350 ms from stimulus onset, but differences between the three vMMN effects were not significant. The source localization showed consistent results that the source of different sequence-vMMN had no significant difference in the whole brain. In addition, we performed time-frequency decomposition on the visual mismatch oscillations (vMORs) and found that the theta wave vMORs were significantly enhanced, but as with the vMMN effect, there were no significant differences among the three vMORs. Low-level features and visual refractoriness did not affect the vMMN/vMORs, indicating enhanced response to a rare visual stimulus rather than inhibited response to the repeated standard. Additionally, we found that the emotion-related vMMN/vMORs effect had a negative bias. These conclusions suggest that vMMN/vMORs are a good electrophysiological index to study unattended visual information processing and support the notion that vMMN is memory-trace based.

Data Availability Statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.

Ethics Statement

The studies involving human participants were reviewed and approved by the institutional Research Board (IRB) of the School of Education, Guangzhou University, China. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

XZ and SF designed the experiment. XZ, YL, and YZ collected and analyzed data. XZ, LJ, YL, and YZ wrote the draft. LJ and SF contributed to the conceptualization and the editing of the manuscript. SF provided funding and resources for this research. All authors contributed to the article and approved the submitted version.

Funding

This research was supported by the National Natural Science Foundation of China (31970993).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Astikainen, P., and Hietanen, J. K. (2009). Event-related potentials to task-irrelevant changes in facial expressions. Behav. Brain Funct. 5:30. doi: 10.1186/1744-9081-5-30

PubMed Abstract | CrossRef Full Text | Google Scholar

Baillet, S., Mosher, J. C., and Leahy, R. M. (2001). Electromagnetic brain mapping. IEEE Signal Process. Mag. 18, 14–30. doi: 10.1109/79.962275

CrossRef Full Text | Google Scholar

Bertrand, O., and Tallon-Baudry, C. (2000). Oscillatory gamma activity in humans: a possible role for object representation. Int. J. Psychophysiol. 38, 211–223. doi: 10.1016/S0167-8760(00)00166-5

CrossRef Full Text | Google Scholar

Caggiano, V., Pomper, J. K., Fleischer, F., Fogassi, L., Giese, M., and Thier, P. (2013). Mirror neurons in monkey area F5 do not adapt to the observation of repeated actions. Nat. Commun. 4:1433. doi: 10.1038/ncomms2419

PubMed Abstract | CrossRef Full Text | Google Scholar

Calderon, J. L., and Luck, S. J. (2014). ERPLAB: an open-source toolbox for the analysis of event-related potentials. Front. Hum. Neurosci. 8:213. doi: 10.3389/fnhum.2014.00213

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, B., Sun, P., and Fu, S. (2020). Consciousness modulates the automatic change detection of masked emotional faces: evidence from visual mismatch negativity. Neuropsychologia 144:107459. doi: 10.1016/j.neuropsychologia.2020.107459

PubMed Abstract | CrossRef Full Text | Google Scholar

Czigler, I. (2014). Visual mismatch negativity and categorization. Brain Topogr. 27, 590–598. doi: 10.1007/s10548-013-0316-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Czigler, I., Balazs, L., and Winkler, I. (2002). Memory-based detection of task-irrelevant visual changes. Psychophysiology 39, 869–873. doi: 10.1111/1469-8986.3960869

PubMed Abstract | CrossRef Full Text | Google Scholar

Delorme, A., and Makeig, S. (2004). EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods 134, 9–21. doi: 10.1016/j.jneumeth.2003.10.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Ding, X., Liu, J., Wang, Y., Kang, T., and Dang, C. (2020). The automatic processing of changes in emotion: implications from EMMN. Adv. Psychol. Sci. 28, 85–97. doi: 10.3724/SP.J.1042.2020.00085

CrossRef Full Text | Google Scholar

Dravida, S., Noah, J. A., Zhang, X., and Hirsch, J. (2018). Comparison of oxyhemoglobin and deoxyhemoglobin signal reliability with and without global mean removal for digit manipulation motor tasks. Neurophotonics 5:011006. doi: 10.1117/1.NPh.5.1.011006

CrossRef Full Text | Google Scholar

Feuerriegel, D., Churches, O., Coussens, S., and Keage, H. A. D. (2019). Temporal expectations modulate face image repetition suppression of early stimulus evoked event-related potentials. Neuropsychologia 122, 76–87. doi: 10.1016/j.neuropsychologia.2018.11.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Feuerriegel, D., Yook, J., Quek, G. L., Hogendoorn, H., and Bode, S. (2021). Visual mismatch responses index surprise signalling but not expectation suppression. Cortex 134, 16–29. doi: 10.1016/j.cortex.2020.10.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Fonov, V. S., Evans, A. C., Mckinstry, R. C., Almli, C. R., and Collins, L. (2009). Unbiased nonlinear average age-appropriate brain templates from birth to adulthood. Neuroimage 47(Suppl. 1):S102. doi: 10.1016/S1053-8119(09)70884-5

CrossRef Full Text | Google Scholar

Friston, K. J. (2005). A theory of cortical responses. Philos. Trans. R. Soc. B Biol. Sci. 360, 815–836. doi: 10.1098/rstb.2005.1622

PubMed Abstract | CrossRef Full Text | Google Scholar

Friston, K. J. (2010). The free-energy principle: a unified brain theory? Nat. Rev. Neurosci. 11, 127–138. doi: 10.1038/nrn2787

PubMed Abstract | CrossRef Full Text | Google Scholar

Garrido, M. I., Friston, K. J., Kiebel, S., Stephan, K. E., Baldeweg, T., and Kilner, J. M. (2008). The functional anatomy of the MMN: a DCM study of the roving paradigm. Neuroimage 42, 936–944. doi: 10.1016/j.neuroimage.2008.05.018

PubMed Abstract | CrossRef Full Text | Google Scholar

Grill-Spector, K., Henson, R., and Martin, A. (2006). Repetition and the brain: neural models of stimulus-specific effects. TRENDS Cogn. Sci. 10, 14–23. doi: 10.1016/j.tics.2005.11.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Heleven, E., and Overwalle, F. V. (2019). The neural representation of the self in relation to close others using fMRI repetition suppression. Soc. Neurosci. 14, 717–728. doi: 10.1080/17470919.2019.1581657

PubMed Abstract | CrossRef Full Text | Google Scholar

Helfrich, R. F., Breska, A., and Knight, R. T. (2019). Neural entrainment and network resonance in support of top-down guided attention. Curr. Opin. Psychol. 29, 82–89. doi: 10.1016/j.copsyc.2018.12.016

PubMed Abstract | CrossRef Full Text | Google Scholar

Hesse, P. N., Schmitt, C., Klingenhoefer, S., and Bremmer, F. (2017). Preattentive processing of numerical visual information. Front. Hum. Neurosci. 11:70. doi: 10.3389/fnhum.2017.00070

PubMed Abstract | CrossRef Full Text | Google Scholar

Jiang, Y., Wu, X., Saab, R., Xiao, Y., and Gao, X. (2018). Time course of influence on the allocation of attentional resources caused by unconscious fearful faces. Neuropsychologia 113, 104–110. doi: 10.1016/j.neuropsychologia.2018.04.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Jonas, J., Rossion, B., Krieg, J., Koessler, L., Colnat-Coulbois, S., Vespignani, H., et al. (2014). Intracerebral electrical stimulation of a face-selective area in the right inferior occipital cortex impairs individual face discrimination. Neuroimage 99, 487–497. doi: 10.1016/j.neuroimage.2014.06.017

PubMed Abstract | CrossRef Full Text | Google Scholar

Karakas, S. (2020). A review of theta oscillation and its functional correlates. Int. J. Psychophysiol. 157, 82–99. doi: 10.1016/j.ijpsycho.2020.04.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Kask, A., Põldver, N., Ausmees, L., and Kreegipuu, K. (2021). Subjectively different emotional schematic faces not automatically discriminated from the brain’s bioelectrical responses. Conscious. Cogn. 93:103150. doi: 10.1016/j.concog.2021.103150

PubMed Abstract | CrossRef Full Text | Google Scholar

Kenemans, J. L., Grent-’t-Jong, T., and Verbaten, M. N. (2003). Detection of visual change: mismatch or rareness? Neuroreport 14, 1239–1242. doi: 10.1097/01.wnr.0000081871.45938.c4

CrossRef Full Text | Google Scholar

Kilner, J. M., Kraskov, A., and Lemon, R. N. (2014). Do monkey F5 mirror neurons show changes in firing rate during repeated observation of natural actions? J. Neurophysiol. 111, 1214–1226. doi: 10.1152/jn.01102.2012

PubMed Abstract | CrossRef Full Text | Google Scholar

Kimura, M. (2012). Visual mismatch negativity and unintentional temporal-context-based prediction in vision. Int. J. Psychophysiol. 83, 144–155. doi: 10.1016/j.ijpsycho.2011.11.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Kimura, M., Katayama, J. i., Ohira, H., and Schröger, E. (2009). Visual mismatch negativity: new evidence from the equiprobable paradigm. Psychophysiology 46, 402–409. doi: 10.1111/j.1469-8986.2008.00767.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Kimura, M., Schröger, E., and Czigler, I. (2011). Visual mismatch negativity and its importance in visual cognitive sciences. Neuroreport 22, 669–673. doi: 10.1097/WNR.0b013e32834973ba

PubMed Abstract | CrossRef Full Text | Google Scholar

Kimura, M., Widmann, A., and Schröger, E. (2010). Human visual system automatically represents large-scale sequential regularities. Brain Res. 1317, 165–179. doi: 10.1016/j.brainres.2009.12.076

PubMed Abstract | CrossRef Full Text | Google Scholar

Klimesch, W. (2012). Alpha-band oscillations, attention, and controlled access to stored information. TRENDS Cogn. Sci. 16, 606–617. doi: 10.1016/j.tics.2012.10.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Kovarski, K., Latinus, M., Charpentier, J., Cléry, H., Roux, S., Houy-Durand, E., et al. (2017). Facial expression related vMMN: disentangling emotional from neutral change detection. Front. Hum. Neurosci. 11:18. doi: 10.3389/fnhum.2017.00018

PubMed Abstract | CrossRef Full Text | Google Scholar

Kreegipuu, K., Kuldkepp, N., Sibolt, O., Toom, M., Allik, J., and Näätänen, R. (2013). vMMN for schematic faces: automatic detection of change in emotional expression. Front. Hum. Neurosci. 7:714. doi: 10.3389/fnhum.2013.00714

PubMed Abstract | CrossRef Full Text | Google Scholar

Kremláèek, J., Kreegipuu, K., Tales, A., Astikainen, P., Põldver, N., Näätänen, R., et al. (2016). Visual mismatch negativity (vMMN): a review and meta-analysis of studies in psychiatric and neurological disorders. Cortex 80, 76–112. doi: 10.1016/j.cortex.2016.03.017

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, X., Lu, Y., Sun, G., Gao, L., and Zhao, L. (2012). Visual mismatch negativity elicited by facial expressions: new evidence from the equiprobable paradigm. Behav. Brain Funct. 8:7. doi: 10.1186/1744-9081-8-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Male, A. G., O’Shea, R. P., Schröger, E., Müller, D., Roeber, U., and Widmann, A. (2020). The quest for the genuine visual mismatch negativity (vMMN): event-related potential indications of deviance detection for low-level visual features. Psychophysiology 57:e13576. doi: 10.1111/psyp.13576

PubMed Abstract | CrossRef Full Text | Google Scholar

May, P., and Tiitinen, H. (2009). Mismatch Negativity (MMN), the deviance-elicited auditory deflection, explained. Psychophysiology 47, 66–122. doi: 10.1111/j.1469-8986.2009.00856.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Méndez-Bértolo, C., Moratti, S., Toledano, R., Lopez-Sosa, F., Martínez-Alvarez, R., Mah, Y. H., et al. (2016). A fast pathway for fear in human amygdala. Nat. Neurosci. 19, 1041–1049. doi: 10.1038/nn.4324

PubMed Abstract | CrossRef Full Text | Google Scholar

Näätänen, R. (1990). The role of attention in auditory information processing as revealed by event-related potentials and other brain measures of cognitive function. Behav. Brain Sci. 13, 201–288. doi: 10.1017/S0140525X00078407

CrossRef Full Text | Google Scholar

Näätänen, R., and Picton, T. W. (1987). The N1 wave of the human electric and magnetic response to sound: a review and an analysis of the component structure. Psychophysiology 24, 375–425. doi: 10.1111/j.1469-8986.1987.tb00311.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Näätänen, R., Jacobsen, T., and Winkler, I. (2005). Memory-based or afferent processes in mismatch negativity (MMN). Psychophysiology 42, 25–32. doi: 10.1111/j.1469-8986.2005.00256.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Näätänen, R., Tervaniemi, M., Sussman, E., Paavilainen, P., and Winkler, I. (2001). ‘Primitive Intelligence’ in the auditory cortex. Trends Neurosci. 24, 283–288. doi: 10.1016/S0166-2236(00)01790-2

CrossRef Full Text | Google Scholar

Nelken, I., and Ulanovsky, N. (2007). Mismatch negativity and stimulus-specific adaptation in animal models. J. Psychophysiol. 21, 214–223. doi: 10.1027/0269-8803.21.34.214

CrossRef Full Text | Google Scholar

Paavilainen, P. (2013). The mismatch-negativity (MMN) component of the auditory event-related potential to violations of abstract regularities: a review. Int. J. Psychophysiol. 88, 109–123. doi: 10.1016/j.ijpsycho.2013.03.015

PubMed Abstract | CrossRef Full Text | Google Scholar

Pascual-Marqui, R. D. (2002). Standardized low-resolution brain electromagnetic tomography (sLORETA): technical details. Methods Find. Exp. Clin. Pharmacol. 24, 5–12. doi: 10.1002/med.10000

PubMed Abstract | CrossRef Full Text | Google Scholar

Rosburg, T., Weigl, M., and Deuring, G. (2019). Enhanced processing of facial emotion for target stimuli. Int. J. Psychophysiol. 146, 190–200. doi: 10.1016/j.ijpsycho.2019.08.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Rostalski, S.-M., Amado, C., and Kovacs, G. (2019). Repetition suppression for noisy and intact faces in the occipito-temporal cortex. Front. Psychol. 10:1348. doi: 10.3389/fpsyg.2019.01348

PubMed Abstract | CrossRef Full Text | Google Scholar

Ruusuvirta, T. (2021). The release from refractoriness hypothesis of N1 of event-related potentials needs reassessment. Hear. Res. 399:107923. doi: 10.1016/j.heares.2020.107923

PubMed Abstract | CrossRef Full Text | Google Scholar

Shen, C., Stasch, J., Velenosi, L., Madipakkam, A. R., Edemann-Callesen, H., and Neuhaus, A. H. (2017). Face identity is encoded in the duration of N170 adaptation. Cortex 86, 55–63. doi: 10.1016/j.cortex.2016.10.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Stefanics, G., Astikainen, P., and Czigler, I. (2014a). Visual mismatch negativity (vMMN): a prediction error signal in the visual modality. Front. Hum. Neurosci. 8:1074. doi: 10.3389/fnhum.2014.01074

PubMed Abstract | CrossRef Full Text | Google Scholar

Stefanics, G., Kremláèek, J., and Czigler, I. (2014b). Visual mismatch negativity: a predictive coding view. Front. Hum. Neurosci. 8:666. doi: 10.3389/fnhum.2014.00666

PubMed Abstract | CrossRef Full Text | Google Scholar

Stefanics, G., Csukly, G., Komlósi, S., Czobor, P., Czigler, I., and Czigler, I. (2012). Processing of unattended facial emotions: a visual mismatch negativity study. Neuroimage 59, 3042–3049. doi: 10.1016/j.neuroimage.2011.10.041

PubMed Abstract | CrossRef Full Text | Google Scholar

Stefanics, G., Háden, G. P., Sziller, I., Balázs, L., Beke, A., and Winkler, I. (2009). Newborn infants process pitch intervals. Clin. Neurophysiol. 120, 304–308. doi: 10.1016/j.clinph.2008.11.020

PubMed Abstract | CrossRef Full Text | Google Scholar

Stefanics, G., Heinzle, J., Czigler, I., Valentini, E., and Stephan, K. E. (2020). Timing of repetition suppression of event-related potentials to unattended objects. Eur. J. Neurosci. 52, 4432–4441. doi: 10.1111/ejn.13972

PubMed Abstract | CrossRef Full Text | Google Scholar

Stefanics, G., Heinzle, J., Horvath, A. A., and Stephan, K. E. (2018). Visual mismatch and predictive coding: a computational single-trial ERP study. J. Neurosci. 38, 4020–4030. doi: 10.1523/jneurosci.3365-17.2018

PubMed Abstract | CrossRef Full Text | Google Scholar

Stefanics, G., Kimura, M., and Czigler, I. (2011). Visual mismatch negativity reveals automatic detection of sequential regularity violation. Front. Hum. Neurosci. 5:46. doi: 10.3389/fnhum.2011.00046

PubMed Abstract | CrossRef Full Text | Google Scholar

Stothart, G., and Kazanina, N. (2013). Oscillatory characteristics of the visual mismatch negativity: what evoked potentials aren’t telling us. Front. Hum. Neurosci. 7:426. doi: 10.3389/fnhum.2013.00426

PubMed Abstract | CrossRef Full Text | Google Scholar

Tadel, F., Baillet, S., Mosher, J. C., Pantazis, D., and Leahy, R. M. (2011). Brainstorm: a user-friendly application for MEG/EEG analysis. Comput. Intell. Neurosci. 2011:879716. doi: 10.1155/2011/879716

PubMed Abstract | CrossRef Full Text | Google Scholar

Tottenham, N., Tanaka, J. W., Leon, A. C., McCarry, T., Nurse, M., Hare, T. A., et al. (2009). The NimStim set of facial expressions: judgments from untrained research participants. Psychiatry Res. 168, 242–249. doi: 10.1016/j.psychres.2008.05.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Tugin, S., Hernandez-Pavon, J. C., Ilmoniemi, R. J., and Nikulin, V. V. (2016). Visual deviant stimuli produce mismatch responses in the amplitude dynamics of neuronal oscillations. Neuroimage 142, 645–655. doi: 10.1016/j.neuroimage.2016.07.024

PubMed Abstract | CrossRef Full Text | Google Scholar

Vogel, B. O., Shen, C., and Neuhaus, A. H. (2015). Emotional context facilitates cortical prediction error responses. Hum. Brain Mapp. 36, 3641–3652. doi: 10.1002/hbm.22868

PubMed Abstract | CrossRef Full Text | Google Scholar

Winkler, I. (2007). Interpreting the mismatch negativity. J. Psychophysiol. 21, 147–163. doi: 10.1027/0269-8803.21.34.147

CrossRef Full Text | Google Scholar

Xu, Q., Yang, Y. P., Wang, P., Sun, G., and Zhao, L. (2013). Gender differences in preattentive processing of facial expressions: an ERP study. Brain Topogr. 26, 488–500. doi: 10.1007/s10548-013-0275-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Yan, T., Feng, Y., Liu, T., Wang, L., Mu, N., Dong, X., et al. (2017). Theta oscillations related to orientation recognition in unattended condition: a vMMN study. Front. Behav. Neurosci. 11:166. doi: 10.3389/fnbeh.2017.00166

PubMed Abstract | CrossRef Full Text | Google Scholar

Yuan, J., Tian, Y., Huang, X., Fan, H., and Wei, X. (2019). Emotional bias varies with stimulus type, arousal and task setting: meta-analytic evidences. Neurosci. Biobehav. Rev. 107, 461–472. doi: 10.1016/j.neubiorev.2019.09.035

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhao, L., and Li, J. (2006). Visual mismatch negativity elicited by facial expressions under non-attentional condition. Neurosci. Lett. 410, 126–131. doi: 10.1016/j.neulet.2006.09.081

PubMed Abstract | CrossRef Full Text | Google Scholar