Recent neurotechnology has developed various methods for neurofeedback (NF), in which participants observe their own neural activity to be regulated in an ideal direction. This neuromodulation through long-term training could improve participants’ cognitive performance or even some clinical traits for people with mental disorders (see for review, Watanabe et al., 2017; Lubianiker et al., 2019). These previous findings suggest our mental adaptability, where the NF system serves to assist our innate ability to learn new mental states. Since neural activity itself does not produce any sensory feedback within the brain, the system can “externalize” that activity to be controlled by the participants themselves. This strategy has been technically developed in terms of brain-computer-interface (BCI), where participants can control the externalized “sensory feedback” of their neural activity through training.

Regarding the implementation of the NF system, there are two options: spatial or temporal priority over the neural representations of the brain. The former is achieved by functional magnetic resonance imaging (fMRI)-NF, while the latter is mainly achieved by electroencephalogram (EEG)-NF. In terms of clinical application, resting-state functional connectivity (FC) in fMRI analysis can be a target of fMRI-NF (i.e., FC-NF), since FC is assumed to be a biomarker for some mental disorders (e.g., Yamada et al., 2017). More recently, however, resting-state EEG signals have also been actively used in line this this, taking advantage of its high temporal resolution to complement the disadvantages of fMRI, which has a resolution of only a few seconds, making it difficult to provide immediate feedback. One useful EEG measure in the context of NF is the EEG microstate. EEG-microstates (EEGms) are spatially featured expressions that can be regulated through NF training (Diaz Hernandez et al., 2016) since recent studies have also suggested EEGms as biomarkers (e.g., da Cruz et al., 2020; Murphy et al., 2020).

Accordingly, the current study was conducted to develop an EEG-NF system for detecting “canonical EEGms” in real time (see Figure 1 for an overview). Once simple resting-state EEG data are collected from a sufficient participant population, group-level common EEG states can be obtained. Previous studies have repeatedly reported that there are only four representative EEG states, regardless of the number of channels, preprocess procedures, or participants (Khanna et al., 2015). Although the optimal number of canonical states remains debatable (Seitzman et al., 2017; Michel and Koenig, 2018), the four most agreed-upon spatial patterns are often called msA, B, C, and D (Figure 2). Therefore, in this study, our 10 Hz NF system was implemented to detect all four msABCD and audio-visually inform participants of its detection, unlike a previous NF system for a specific microstate (Diaz Hernandez et al., 2016).

The EEGms are observed as quasi-stable potentials lasting 60–120 ms that represent whole-brain network activity (Michel and Koenig, 2018). The suggested procedure for EEGms analysis consists of two main stages: clustering and labeling. At the clustering stage, the standard deviation (global field power: GFP) among all electrodes is first calculated for each recording session. The time points of the greatest strength in the neuroelectric field, namely GFP peaks, are well-reasoned to best represent periods of momentary stability in the voltage topography (Zanesco, 2020). Consequently, it is preferable to select GFP peak points to achieve a high signal-to-noise ratio. Every spatial pattern at the local maximum of each GFP time series is accumulated as the GFP peak dataset for a participant group as a whole. An unsupervised learning algorithm, such as K-means, conducts clustering of the GFP peak dataset into optimal classes. Finally, group-wise common topographies are obtained as templates (i.e., centroid) for each class (typically, msA, B, C, and D). After that, the labeling procedure calculates the similarity between the templates obtained and every time point (including the GFP peak points) of the participants and labels one of them (e.g., msA) for all data points in a so-called “winner-take-all” manner (see section “Discussion”, Mishra et al., 2020). Since each topography should last for a while in a millisecond order, the original EEG data with multiple channels are now simply seen as a state transition pattern among four states, such as C, D, B, A, D, etc. As a result, the EEGms analysis typically reveals the frequency of the state (“occurrence”), the lasting of the state (“duration”), and transition patterns among the states (“transition probability”) for each recording session or each participant. These depicted features of EEGms can be compared among participant groups as potential biomarkers (de Bock et al., 2020; Perrottelli et al., 2021).

The current study examined the real-time effect of the developed NF system, which has been implemented to detect canonical msABCD (group-wise common topographies) and audio-visually inform participants of its detection. The audio-visual NF, however, was intentionally delayed for participants to prevent their control in learning (i.e., hit the targeted state; see section “Materials and Methods”). We hypothesized that an inserted delay would affect participants’ learning through the NF system due to their inability to utilize the feedback. Although EEG-NF is advantageous in terms of its temporal resolution in comparison with fMRI-NF, the real-time effect has not been examined well in the literature, especially for EEGms. Given that the EEGms might be assumed as basic components of consciousness, often referred to as “atoms of thoughts” (Koukkou and Lehmann, 1987; Lehmann, 1992; Lehmann et al., 1998; Changeux and Michel, 2006), and last for a short period of time, EEGms-NF should become less effective with a certain delay (e.g., over some hundreds of milliseconds). Aside from EEGms-NF, several studies have attempted to implement real-time EEG feedback (Zich et al., 2015; Pei et al., 2020). A recent study examined the effect of latency in visual NF that falls within the range of 300 to 1000 ms in terms of a parietal alpha rhythm (Belinskaya et al., 2020), and concluded that the delay is a crucial parameter that must be minimized to achieve the desired NF effect. This policy is also motivation to enhance the “sense of agency” for participants in their NF training: a feeling of “I am the origin of the sensory feedback” (Gallagher, 2000). In addition to many psychophysical studies that have demonstrated that the feedback delay clearly reduces both participants’ sense of agency (Asai and Tanno, 2007; Asai, 2016) and their performance in motor control tasks (Tanaka et al., 2011), Evans et al. also suggested that an inserted delay resulted in a reduced sense of agency over the externalized feedback through a motor imagery-based BCI system (Evans et al., 2015).

The aim of the current study was to develop a closed-loop NF system and validate it as a function of the inserted delay. Participants were instructed to attempt to make more “Hits” with audio-visual feedback (see Figures 3, 4 for the definition of “Hits”). However, in some conditions (i.e., “sessions” in the current case) the additional delay was inserted between participants’ neural activity and its externalized feedback in a secret manner (Figure 5 for experimental design). If real-time NF is effective for learning, participants’ Hit performance should be increased from the baseline period and from delayed conditions. In particular, given that each EEGms could be a different cognitive unit, as previously discussed, the controllability of each microstate in the real-time NF situation and robustness of the controllability to a feedback delay may be different depending on its cognitive responsibility. Therefore, we also compared the real-time effects of msA, B, C, and D in terms of each cognitive functionality.

The templates we obtained are shown in Figure 2, where the labels of maps were named according to previously established studies (i.e., typically msA, B, C and D; Lehmann et al., 1987; Koenig et al., 2002; Khanna et al., 2014, 2015; Diaz Hernandez et al., 2016; Milz et al., 2016; Michel and Koenig, 2018). The topography showing a left posterior to right anterior orientation was determined as msA, the right posterior-left anterior orientation was msB, the anterior-posterior orientation was msC, and the fronto-central extreme location was msD. The spatial configurations of these four maps were highly similar to those previously described and frequently cited in EEG microstate research (Koenig et al., 2002) (Figure 2B). Comparing each two maps with the corresponding label assignment (Brodbeck et al., 2012), we achieved high spatial correlation coefficients (0.85–0.97, Figure 2C). Furthermore, we calculated the global explained variance (GEV) of the templates. The GEV is one of the parameters that can be used to evaluate whether this set of maps is reasonable as a canonical template. It provides a metric of how well the selected template maps account for the variance of the entire dataset (Poulsen et al., 2018). The higher the GEV, the better the explanation of the entire dataset. As a result, our template of four cluster maps explained 75% of the variance. Referring to previous studies (Figure 2B), the GEV was 71% in Diaz Hernandez et al. (2016), who recorded and preprocessed EEG signals with the same number of electrodes and filter settings as ours, 70% in Khanna et al. (2014) (they used a wider bandpass filter than ours), and 77% in Milz et al. (2016) (they used a larger number of electrodes than ours). Therefore, we obtained canonical microstates that were congruent with previous observations. These templates were used in our NF system to be matched with participants’ EEG states in real time.

In the current study, we developed a new EEGms-NF system and validated it by examining the decline in participants’ performance as a function of delay. Although participants were presumably totally naive about the inserted delay as the result of the agency questionnaire also implies (Figure 6A), the real-time NF successfully increased their Hit performance in total (Figure 7A), even within a single-day experiment. This suggests that by using our NF system, participants could implicitly learn through a supervised manner how to control their own neural state, spatially represented in EEG channels, immediately during each no-delay session. Contrastingly, participants’ subjective agency was not modulated (i.e., reduced) even by a 20-s–delay, unlike many behavioral studies, given that they should not have been aware of the inserted delay. This indicates that we do not have an internal model of our own neural state (EEG state in our case) by default. Without this type of externalizing BCI system, it is difficult for us to estimate our own neural state or to compare the actual and desired states in forward modeling. This should be an essential factor why we do not feel a sense of control over BCI interactions in real time, where only outcome-dependent agency may be elicited (Evans et al., 2015). However, the current study further indicates that the use of an NF system that implements real-time sensory feedback like ours in long-term training for some months (such as daily motor behaviors), for example, an internal model over that system with subjective online agency can be developed. If the NF system represents our neural dynamics in nature, controlling or internalizing the system would mean that we could regulate our own neural state through the developed internal model (possibly even without the NF system). The EEGms, in this sense, are candidates for representing natural dynamics in the brain. Therefore, developing a BCI-based NF system in terms of online agency (or internal model) should be an important framework for future studies. For that purpose, our results suggest that minimizing feedback delays could be a worthwhile implementation for improving task learning (Belinskaya et al., 2020) such as motor learning (Tanaka et al., 2011).

In comparison with the preceding EEGms-NF system (Diaz Hernandez et al., 2016), there are several points to be discussed here. First, the templates in the system as the desired target state are group-wise topographies in our case, while in the previous study they were individually defined. This difference could be expected to result in different training outcomes. In our case, the spatial pattern itself was the target for participants’ learning, given that the individual EEGms must be slightly different from the group-wise EEGms (the “canonical” EEGms). We may observe the changes for individual EEGms moving toward group-wise EEGms through long-term training in future studies. However, individually defined templates should be easier for participants to induce, because they exhibit these patterns by default in nature, although the canonical EEGms are reportedly common across participants. Another difference in the implementation is sensory feedback. For supervised learning, participants were visually aware of their current and targeted state simultaneously. In addition to the visual feedback used in the previous study, the auditory feedback was provided to indicate Hits (successes in minimizing a prediction error) and to reward participants, as well. This audio-visual BCI monitors and externalizes whole-brain mass neural activity as EEGms to be regulated (Diaz Hernandez et al., 2016). This implementation enables us to treat four targets simultaneously (see Figure 1). In addition, the optimal frequency of NF is an important factor for participants’ learning so that the definition of EEGms has been further parameterized in terms of its duration and topographical robustness. Therefore, the difference to note is our definition of the state. Our system detects only spatio-temporally robust EEGms (approximately 100 ms for duration and r_thr = 0.8 for spatial similarity) so that strict online denoising was not necessary, which is always a nuisance in the “winner-take-all” definition of EEGms. Both noisy states and spatio-temporally less-robust states are ignored in our system. This largely helped the real-time implementation. However, if a polarity reversal or rapid transition is observed during the 100 ms (e.g., msA followed by msC and msD), the system ignores that state (i.e., as a mixture of A, C, and D, see Figure 3B). This is contrasted to a conventional offline EEG microstate analysis that has been implemented to handle this with polarity-ignored labeling (msABCD) and final temporal smoothing among labels (Poulsen et al., 2018). Recently, this classical approach of microstate labeling (“winner-take-all”) has been controversial (e.g., Shaw et al., 2019; Mishra et al., 2020). Some previous studies have intentionally introduced “unlabeled” states for this reason (e.g., Zanesco et al., 2021). In this sense, the current system was not developed to be matched with the classical labels, and there should be non-small differences between them depending on the analysis parameters, especially temporal smoothing of labels. Accordingly, our conservative system was not designed to detect every robust state but to not falsely detect them. We can imagine that 1-ms or 10-ms–averaged state is less reliable for learning and needs further temporal smoothing during a certain period. In this sense, we may update to a 100-Hz system using overlapping 100-ms blocks in a future study.

Regarding the potential differences among the four EEGms in responsible cognitive functionality, our results suggest some results congruent with previous studies as well as some new findings, especially in terms of neural controllability. First, for msD, the most frequently examined among the four EEGms, was reportedly increased through NF training for 10 days (Diaz Hernandez et al., 2016). Although that study was the first attempt to modify EEGms, msD is a specific target for future clinical application, since msD may be a biomarker for schizophrenia. Therefore, a link between schizophrenic symptomatology and the function of msD has often been discussed. Previous studies indicate that msD is functionally related to attentional control or central executive function that the frontoparietal network underpins. Simultaneous EEG-fMRI recordings suggest a link where the fluctuation of msD (time-series of spatial correlation to msD template) explains BOLD signals in the right-lateralized dorsal and ventral areas of the frontoparietal cortex (Britz et al., 2010). More recently, a simple mental arithmetic calculation increased participants’ duration of msD (Seitzman et al., 2017; Bréchet et al., 2019). If msD is responsible for this attentive cognitive load in general, our results are consistent with this. Even during 20-s–delay NF sessions (participants were not aware of this), participants tried to make more Hits so that the msD was increased compared to baseline. However, msC has often been discussed in contrast to msD as some previous studies indicate that the increased msD coincides with the decreased msC (e.g., Seitzman et al., 2017; Bréchet et al., 2019). Indeed, simultaneous recordings suggested that msC is neurally related to the default mode or saliency network, which is the functionally opposite network to the frontoparietal network (Britz et al., 2010; as a review, Seitzman et al., 2017; Michel and Koenig, 2018). In the current study, msC was the most delay-sensitive and controllable in real time. In this sense, saliency-related cognitive function may be a candidate for msC.

In contrast to the msC-D axis for controllable cognitive states, msA and B have been discussed as more perceptual functionalities. Simultaneous recordings suggest that msA is neurally related to the auditory region, while msB is related to the visual region (Britz et al., 2010). Recent studies also indicate msA for auditory perception as well as msB for visual perception, depending on the stimuli of modality or perceptual experiences (e.g., Cai et al., 2018; Bréchet et al., 2019; D’Croz-Baron et al., 2021). On the other hand, our results suggest that msA or B did not increase, even with real-time NF. These results indicate that the functionality of msA and B (or msA-B axis) is more perceptual; therefore, it might be passive and difficult to control intentionally. Although a more comprehensive understanding of four canonical EEGms is needed, since previous findings were given in a relatively sporadic matter, our results discriminated between msC-D and msA-B in terms of real-time controllability with NF. Our experiment did not examine each functionality directly and we do not assume that cognitive functionality has a 1:1 correspondence to each microstate, rather, microstate “dynamics” could be responsible to our cognition and perception (Michel and Koenig, 2018; Ruggeri et al., 2019, 2020). However, the result is interpretable in line with or extending the existing understanding of EEGms where the functionality of msA and B are perceptual (auditory for msA and visual for msB), and therefore, may be less controllable, while msC and D are cognitive (saliency or default mode for msC and attentive or executive for msD), and therefore, more controllable.

This study has some limitations for future applications. The current study was conducted within a single day for each participant. Accordingly, the effects of long-term training using the system are unknown, especially for the target specificity. Even within a session, the total Hit performance (and msC and msD) increased above the “task-ready” baseline (Figure 9A), suggesting that long-term training is predicted. If msC or msD is a candidate biomarker for some diseases (for example, schizophrenia), our system is clinically applicable for future studies as it is (without the inserted delay, of course). Although msA and msB appear more difficult to control according to the current results, they are still useful for examining the effect of long-term training, since a non-significant tendency for increasing was observed (Figures 7, 9). Another possible future approach would be to manipulate a parameter of threshold (spatial correlation r) individually, as well as msABCD, respectively. The former serves to absorb individual differences in an EEG cap condition, head fitness with a cap, and innate similarity with canonical templates. The latter serves to equalize the feedback frequency among msABCD because msC might occur more often innately. Examining thresholds could improve the balance between discriminability and detectability among msA, B, C, and D (Figures 3A,B). For this purpose, pre-training recording is required to determine the optimal thresholds and objective procedure. Whether this parameter should be changed daily through long-term training should also be considered. Even in this case, our system has already parameterized the threshold value. This should prove helpful, especially for specific participants with disorders who might have a reduced msD, for example, the system with a regular parameter should be less responsive to them than to healthy participants. Finally, the definition of a state as a canonical EEGms and the number of microstates are controversial (Michel and Koenig, 2018; Tarailis et al., 2021). Further studies should address this issue, since the current study used traditional definitions. Because our system was developed to make the template easily replaceable, new state definitions can be readily applied for participant learning (e.g., a 7-EEGms model).

In conclusion, the aim of the current study was to develop a closed-loop NF system and validate it as a function of the inserted delay. If real-time NF is effective for learning, the participants’ Hit performance should increase above baseline and from delayed conditions. Our results suggest that this is the case, although the target specificity is still unclear. Future studies can examine long-term training with this NF system, even for a specific population.

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

The studies involving human participants were reviewed and approved by the Ethics Committee of ATR (reference number is 20–144). The patients/participants provided their written informed consent to participate in this study.

TA mainly designed the experiments, analyzed the data, and wrote the manuscript. TH and SK partially analyzed the data and wrote the manuscript. TA, TH, SK, and HI discussed the results and reviewed the manuscript. All authors approved the final version of the manuscript for submission.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.