Influence of the Number of Channels and Classification Algorithm on the Performance Robustness to Electrode Shift in Steady-State Visual Evoked Potential-Based Brain-Computer Interfaces

Introduction

The brain-computer interface (BCI) allows the users to communicate with the external world using their brain activities without standard communication methods, such as speaking and through gestures (Wolpaw et al., 2002). Various recording modalities have been used to implement BCIs, such as electroencephalography (EEG) (Lotte et al., 2018), magnetoencephalography (Mellinger et al., 2007), near-infrared spectroscopy (Hwang et al., 2014, 2016), electrocorticography (Hill et al., 2006; Schalk et al., 2007), and local field potential (Asgharpour et al., 2021). Among these modalities, EEG has been the most widely used owing to its advantages, such as non-invasiveness, excellent temporal resolution, convenience of use, and cost-effectiveness (Han et al., 2013; Hwang et al., 2013; Oikonomou et al., 2016; Lotte et al., 2018).

Electroencephalography-based BCIs convert electrical potentials measured on the scalp of the user while they perform a specific mental task, into designated commands. These commands are applied to various applications such as mental spellers (Hwang et al., 2012; Lim et al., 2015; Speier et al., 2017), assistive technology for patients (Hwang et al., 2017; Lim et al., 2017), and online home appliance control (Park et al., 2020). EEG-based BCIs can be classified based on the paradigms to elicit neural activities (Hwang et al., 2013) as motor imagery (MI)-based BCI (Acqualagna et al., 2016; León et al., 2020), visual P300-based BCI (Gu et al., 2019), steady-state visual evoked potential (SSVEP)-based BCI, non-motor mental imagery-based BCI (Kristensen et al., 2020), auditory BCI (Kim et al., 2011; Simon et al., 2015), and hybrid BCI (Jalilpour et al., 2020). Among these, SSVEP-based BCIs have the advantage of higher accuracy and higher information transfer rate (ITR) and generally require no/short training time (Tello et al., 2014; Nakanishi et al., 2015; Hwang et al., 2017; Xing et al., 2018).

Steady-state visual evoked potential-based BCIs recognize the intentions of the users by identifying the frequency of the visual stimulus. SSVEP is a quasi-periodic brain signal elicited through visual stimulus flickering or reversing at a specific frequency, generally recorded in the occipital region of the brain (Vialatte et al., 2010; Xu et al., 2014). Power spectral density analysis has traditionally been used to classify the frequency of SSVEP (Cheng et al., 2002; Cao et al., 2015). Recently, various algorithms have been proposed to enhance the classification accuracy and ITR of SSVEP-based BCIs, such as canonical correlation analysis (CCA) (Lin et al., 2007), least absolute shrinkage and selection operator (Zhang et al., 2012), multivariate synchronization index (MSI) (Zhang et al., 2014), filter bank CCA (FBCCA) (Chen et al., 2015), and task-related component analysis (Nakanishi et al., 2018).

Although elevating the classification accuracy and ITR of SSVEP-based BCIs is important, maintaining consistently high performance in the BCIs is an equally important factor to be considered for long-term and/or daily-life BCI applications. One factor that degrades the consistency of performance in the SSVEP-based BCI is the electrode dislocation from the optimal electrode locations, e.g., O1, O2, and Oz locations were generally used for a three-electrode BCI. It is generally not possible to reattach the electrodes to an identical location on the scalp for every usage in a BCI system. In addition, the locations of the electrodes can be slightly modified when the EEG device is concurrently used with external devices, such as a virtual reality head-mounted display (HMD) headset. However, to the best of our knowledge, the influence of electrode shift on the performance of SSVEP-based BCIs has not yet been quantitatively evaluated.

Park et al. (2013) evaluated feature extraction methods for MI-based BCIs in terms of performance robustness to slight changes in the electrode location and recommended the cross-correlation method as a promising feature extraction method. Except for this study, the robustness of performance against the electrode location shift has not been studied for EEG-based BCIs. On the contrary, the robustness of performance to the electrode shift was mainly investigated for electromyography (EMG)-based myoelectric interfaces. Boschmann and Platzner (2012) showed that the degradation of pattern recognition performance due to electrode shift could be improved by increasing the number of electrode channels and expanding the recording locations. Pan et al. (2015) demonstrated that the performance robustness against electrode shift in myoelectric prostheses based on high-density EMG could be improved by employing the multiclass common spatial pattern feature. Fan et al. (2016) proposed a novel CCA-based method to eliminate the reduction in performance in a surface EMG-based motion classification system. The recent work of Lv et al. (2018) proposed a novel neural network model using an autoencoder to extract the robust features against electrode shift.

With the recent growing interest in the practical use of SSVEP-based BCIs, there is a growing need for studies evaluating the performance robustness under various conditions, such as the number of channels and classification algorithms. This study aimed to investigate the performance variation in SSVEP-based BCIs owing to minor electrode shifts and to discover the conditions that minimize the variations in classification accuracy while maintaining high classification accuracy. The stimulus frequencies used for SSVEP-based BCIs can be classified into three frequency bands: low (~12 Hz), medium (12–30 Hz), and high (>30 Hz) frequency bands (Regan, 1989). The majority of SSVEP-based BCI studies were based on low-frequency and medium-frequency bands, since SSVEP responses are stronger in low-frequency bands than in high-frequency bands (Zhu et al., 2010; Liang et al., 2019). Therefore, in this study, we primarily investigated the performance robustness of the low-frequency SSVEP-based BCIs. To this aim, an experiment eliciting the SSVEP was conducted with 21 healthy participants and the EEG signals were recorded from the electrodes densely attached to their occipital area. Then, the performance robustness of SSVEP-based BCIs to the electrode shift with respect to the numbers of channels and classification algorithms was quantitatively investigated.

Materials and Methods

Participants

In this study, 25 healthy volunteers (15 men and 10 women; age 23.52 ± 2.73 years) with normal or corrected-to-normal vision had participated. None of them had any history of neurological, psychiatric, or other severe diseases that could affect the experimental results. The participants signed an informed consent form and were familiarized with the detailed protocol of the experiment. After completion of the experiment, each participant received a monetary reimbursement. Data from four participants were excluded from further analyses because of the non-existence of spectral peaks. The remaining data (14 men and 7 women; age 23.9 ± 2.79 years) were used in this study. This experimental study was approved and reviewed by the Institutional Review Board (IRB) of Hanyang University, Korea (HYI-14-167-11).

Experiments and Data Acquisition

Four 6 × 6 pattern-reversal checkerboard stimuli reversing at different frequencies were used to elicit the SSVEP responses. The four frequencies were set to 6 Hz (top left), 6.66 Hz (top right), 7.5 Hz (bottom left), and 10 Hz (bottom right), considering the intrinsic monitor refreshing rate. Figure 1A illustrates the configuration of the visual stimuli. In each trial, four visual stimuli were presented for 5 s with a 2 s inter-stimulus interval. The participant gazed at one of the four stimuli without blinking the eyes and making no body movements according to the instructions provided immediately before the stimulus presentation. One session consisted of 20 trials, and five sessions were conducted for each participant with a variable time interval between the sessions. Note that the participants could take a rest as long as they wanted during the inter-session intervals for reducing visual fatigue. The order of stimulus presentation was randomized and counterbalanced to avoid the potential habituation effects. The visual stimuli were displayed on a 24-in LCD monitor screen with a 60 Hz refresh rate, placed approximately 70 cm away from the eyes of the participant. The visual stimuli were generated and controlled using the Cogent-2000 toolbox (www.vislab.ucl.ac.uk/cogent.php; Wellcome Trust Centre for Neuroimaging and Institute of Cognitive Neuroscience, UCL, London).

FIGURE 1

Figure 1. Experiment setup and channel combination (A) four 6 × 6 pattern-reversal checkerboard stimuli [frequencies of the four stimuli were set to 6 Hz (top left), 6.66 Hz (top right), 7.5 Hz (bottom left), and 10 Hz (bottom right)] (B) fabricated electrode array pad with dense electrode configuration (C) array pad [distance between the two neighboring electrodes was set to 1.3 cm,; center of the array pad was placed at Oz and according to the extended international 10–20 system; 21 channels were categorized into three categories depending on the location of the channel: left (channel 1–9), middle (channel 7–15), and right (channel 13–21)] (D) possible channel combinations with respect to the number of channels used for classification [for example, when two EEG signals recorded at two channels were used to classify the frequency of SSVEP, the classification accuracy was calculated in a total 81 channel combinations by combining the left 9 channels (channel 1–9) and right 9 channels (channel 13–21)].

Electroencephalography signals were recorded using a commercial biosignal recording system (ActiveTwo; Biosemi, Amsterdam, the Netherlands) from the 21 electrodes at a sampling rate of 2,048 Hz. A lab-made electrode pad was used to record the EEG signals from the 21 electrodes attached to the occipital area of the participants. Figure 1B shows the fabricated electrode array pad with a sports headband that mounts the electrodes densely on the occipital area. Figure 1C shows the detailed configuration of the electrodes on the pad, which were arranged in three rows and seven columns, and the distance between two neighboring electrodes was set to be 1.3 cm. EEG electrodes used for measuring the SSVEPs are usually attached to O1, Oz, and O2 according to the extended international 10–20 system (Meng et al., 2011; Zhang et al., 2012; Lim et al., 2015, 2017; Hwang et al., 2017). Therefore, the center of the array pad (Channel-11) was placed at Oz, when Channel-5 and Channel-17 corresponded to O1 and O2, respectively. A common mode sense active electrode and a driven right leg passive electrode were attached to the left and right mastoids, respectively, to form a feedback loop for the amplifier reference (www.biosemi.com/faq/cms&drl.htm).

Analysis

All analyses, including the signal preprocessing, classification, and statistical analysis, were performed using MATLAB R2019b (Mathworks, Natick, MA, USA).

Preprocessing

Recorded EEG signals were downsampled at 256 Hz considering the computational efficiency and were then band-pass filtered using a third-order zero-phase Butterworth filter with cutoff frequencies of 4 and 52 Hz. We investigated the SSVEP responses elicited by flickering stimuli using EEG signals recorded from Channels 5, 11, and 17. Please note that we excluded the data from four participants whose SSVEP responses were not observed, as mentioned in section Participants. Supplementary Figure 1 shows the average amplitude spectra averaged across 21 subjects and 25 trials for each stimuli frequency at channels O1, Oz, and O2, where clear SSVEP peaks were observed at the target frequencies (6, 6.66, 7.5, and 10, which corresponds to 60/10, 60/9, 60/8, and 60/6) and their harmonics. For each trial, the visual stimulus that the participant was staring at was identified using five different classification algorithms: CCA, extended CCA (ECCA), FBCCA, MSI, and extended MSI (EMSI) (Zhang et al., 2017). The preprocessed EEG datasets are available at the following link: https://figshare.com/s/cd46e1531d67f3023fa6.

Classification Algorithms

Canonical correlation analysis is a multivariate statistical method used to determine the linear combinations, which maximize the correlation between the two data sets X and Y (Lin et al., 2007; Bin et al., 2009). The weight vectors W_X and W_Y that maximize the canonical correlation between ${x=X^{T}W}_X$ and ${y=Y^{T}W}_Y$ were determined by solving the following optimization problem:

$\begin{array}{ll}{\displaystyle \underset{W_x,W_y}{max}}\rho \left(x,y\right)=\frac{W_X^{T}X{Y}^T{W}_Y}{\sqrt{W_X^{T}X{X}^T{W}_X·W_Y^{T}Y{Y}^T{W}_Y}},& (1)\end{array}$

where ρ is the canonical correlation coefficient. In the SSVEP frequency classification, X and Y are the EEG signals and SSVEP reference signals, respectively. The reference signals for each stimulus frequency were computed as follows:

$\begin{array}{ll}Y=\left[\begin{array}{c}sin\left(2\pi f\frac{t}{F_s}\right)\\ cos\left(2\pi f\frac{t}{F_s}\right)\\ ⋮\\ sin\left(2\pi N_{h}f\frac{t}{F_s}\right)\\ cos\left(2\pi N_{h}f\frac{t}{F_s}\right)\end{array}\right],t=1,2,\cdots ,M,& (2)\end{array}$

where N_h is the number of harmonics, F_s is the sampling rate, M is the number of signal samples, and f is the base frequency of each stimulus frequency. The number of harmonics for the five classification algorithms was set to four for achieving the highest average classification accuracy for the optimal electrode positions. The canonical correlation coefficients were calculated for all the stimulus frequencies. Then, the frequency of the largest coefficient was selected.

Multivariate synchronization index measures the synchronization between the two datasets, similar to the coefficient of CCA for classifying the stimulus frequency (Zhang et al., 2014). Let X and Y denote the EEG and reference signals, respectively. The size of matrix X is N × M and the size of Y is 2N_h × M. Here, N is the number of channels in the EEG signals, N_h is the number of harmonics, and M is the number of samples. The correlation matrix of X and Y was calculated as follows:

$\begin{array}{ll}C=\left[\begin{array}{cc}{C}_{11}& C_{12}\\ C_{21}& C_{22}\end{array}\right],& (3)\end{array}$

where $C_{11}=\frac{1}{M}X{X}^T$, $C_{22}=\frac{1}{M}Y{Y}^T$, $C_{12}=C_{21}=\frac{1}{M}X{Y}^T$, and M is the number of samples. Because the synchronization measure can be influenced by autocorrelation, the following linear transformation was applied to matrix C (Joudaki et al., 2012).

$\begin{array}{ll}U=\left[\begin{array}{cc}{C_{11}}^{-\frac{1}{2}}& 0\\ 0& {C_{22}}^{-\frac{1}{2}}\end{array}\right]& (4)\end{array}$

The transformed matrix was given by

$\begin{array}{ll}R=UC{U}^T,& (5)\end{array}$

where the dimension of R is N + 2N_h. Let λ₁, λ₂, ⋯ , λ_P, where P = N+2N_h, be the Eigenvalues of matrix R. The Eigenvalues were normalized as follows: ${\lambda }_i^{{}^{\prime }}=\frac{{\lambda }_i}{\sum _{i=1}^P{\lambda }_i}$. Then, the synchronization index between X and Y was calculated as follows:

$\begin{array}{ll}S=1+\frac{\sum _{i=1}^P{\lambda }_i^{{}^{\prime }}log{\lambda }_i^{{}^{\prime }}}{logP}& (6)\end{array}$

The synchronization indices were calculated for all the stimulus frequencies and the frequency of the largest synchronization index was selected.

Extended MSI is an algorithm that extends the MSI by incorporating time-delayed EEG signals during the calculation of the synchronization index (Zhang et al., 2017). ECCA is an algorithm that extends the CCA similar to the EMSI. Let X be the EEG signal. The delayed signals, X^τ, were appended to X as

$\begin{array}{ll}\overline{X}=\left[\begin{array}{c}X\\ X^{\tau }\end{array}\right],& (7)\end{array}$

where τ is the number of delayed samples (Zhang et al., 2017). In this study, we generated the delayed signals using the circular shift method that moves the final sample of a signal to the first position while shifting all the other samples of the signal to the next position. We shifted the circular samples of the EEG signals once, i.e., τ was set to 1. Then, the new synchronization index and the new canonical correlation coefficient using $\overline{X}$ were calculated using Equations (2)–(6) and (1)–(2), respectively. The target frequency was then estimated in a manner similar to the MSI and CCA.

Filter bank CCA decomposes the EEG signals into multiple sub-band components and calculates the canonical correlation coefficient of each sub-band component. The weighted sum of the squared CCA coefficients corresponding to all the sub-band components was calculated as follows:

$\begin{array}{ll}{\rho }_f={\displaystyle \sum _{n=1}^{N_{sb}}}w\left(n\right)·{\left({\rho }_f^n\right)}^2,& (8)\end{array}$

where N_sb is the number of sub-bands and n is the index of the sub-band. The weights of the sub-band components were defined as follows:

$\begin{array}{ll}w(n)=n^{-a}+b,& (9)\end{array}$

where a and b are constants. In this study, three parameters were determined based on the previous studies (Chen et al., 2015) (a = 1.25, b = 0.25, and N_sb = 5). The lower and upper cutoff frequencies of the five sub-bands were designed as follows: 4–52 Hz, 8–52 Hz, 12–52 Hz, 16–52 Hz, and 20–52 Hz. The weighted sum of the squared CCA coefficients (ρ_f) was calculated for all the stimulus frequencies. The frequency with the maximum ρ_f was then selected as the target frequency.

Performance Evaluation Measures

Classification accuracies were calculated for five training-free SSVEP classification algorithms for all the possible electrode shift cases with respect to different numbers of channels (1–3 channels). As shown in Figures 1C, 21 channels were divided into three-channel groups: the left 9 channels (Channels 1–9), middle 9 channels (Channels 7–15), and right 9 channels (Channels 13–21). In each channel group, the central channels (left: Channel 5, middle: Channel 11, right: Channel 17) corresponded to the original channel positions (O1, Oz, and O2, respectively), and the remaining eight channels in each group corresponded to the position where the electrodes could be shifted from their original positions.

There can be various channel combinations when the number of channels is 1, 2, and 3, as illustrated in Figure 1D. When the number of channels was 1, the frequency of SSVEP was classified using only one EEG signal recorded from one channel. In this case, the classification accuracies were calculated for each of the middle nine channels (Channels 7–15). When two EEG signals recorded at two different channels were used for the classification, the classification accuracies were calculated for each of 81 channel combinations by combining the left nine channels (Channels 1–9) and right nine channels (Channels 13–21). Finally, when the number of channels was 3, classification accuracies were calculated for each of 675 channel combinations by combining the left 9 channels, middle 9 channels, and right 9 channels. In the combination of these three groups, the combinations with specific channels that overlapped with each other were excluded, e.g., Channels 7, 7, and 13.

To compare the changes in the classification performance owing to the electrode shift, two measures were calculated for each subject: (1) average classification accuracy across the electrode shift (ACA) and (2) robustness to the electrode shift (RES). The ACA was calculated by averaging the classification accuracies for all the channel combinations. The RES was evaluated by calculating the coefficient of variation (CV) of the classification accuracies for all the channel combinations, with the RES value being defined as 1 – CV. CV is a standardized measure of dispersion, which is defined as the ratio of the standard deviation to the mean. A low-RES value indicates that the classification performance is affected significantly by the changes in the electrode locations, whereas a high-RES value indicates that the classification performance is not affected significantly by the electrode shift.

Statistical Analysis

A two-way analysis of variance with repeated measures based on two factors (the number of channels and the classification algorithm) was used to statistically compare the ACA and RES. A paired t-test with Bonferroni correction was used for the post-hoc analyses.

Results

Statistical Comparison

First, we statistically compared the ACA and RES values with respect to the number of channels and classification algorithms for window lengths of 2–5 s with intervals of 0.5 s. In Table 1, the two-way repeated-measures ANOVA for ACA showed a statistically significant main effect on the number of channels and the algorithms for all the window lengths. The interaction between the number of channels and the algorithm was statistically significant for each window length. In contrast, the two-way repeated-measures ANOVA for RES showed a statistically significant main effect only for the classification algorithms for every window length, when the interaction between the number of channels and the algorithm was statistically significant for each window length. While the main effect of the number of channels on the RES value was not significant, we statistically compared the ACA and RES of the three levels of the number of channels and five levels of algorithms using a paired t-test as a post-hoc analysis because the interaction of ACA and RES was found to be statistically significant for every window length.

TABLE 1

Table 1. Results of the two-way repeated-measures ANOVA with two factors of “Number of channels” and “Algorithm” for various window lengths.

Average Classification Accuracy Across the Electrode Shift

Figure 2 and Supplementary Table 1 show the average and SD of the ACA across the 21 subjects. Table 2 shows the results of the paired t-test for the number of channels with respect to the algorithm and window length. When classifying the frequency of SSVEP using the CCA, ECCA, and FBCCA, there was no statistically significant difference between the ACA for one channel and that for two channels in each window length. However, the ACA for three channels was significantly higher than those for one channel and two channels (CCA: 4.5 and 5 s; ECCA: 3.5, 4, 4.5, and 5 s; FBCCA: all window lengths). When the frequency of SSVEP was classified using MSI and EMSI, the ACA was significantly increased with an increase in the number of channels in every window length. In short, the ACA for three channels was higher than that for one and two channels regardless of the algorithms. The ACA for two channels increased significantly compared to that for one channel in MSI and EMSI.

FIGURE 2

Figure 2. Mean and SD of the average classification accuracy across the electrode shift (ACA) across the subjects with five SSVEP classification algorithms in the case of 3 channels and seven window lengths (wl) (black, gray, and white bars denote the mean accuracies of 1 channel, 2 channels, and 3 channels, respectively; the error bars represent the SD ACA).

TABLE 2

Table 2. Bonferroni corrected p-values of post-hoc analysis among the average classification accuracies across the electrode shift (ACAs) of three different numbers of channels for classification algorithm and window length.

Table 3 shows the results of the paired t-test among the algorithms with respect to the number of channels and window lengths. In the case of one channel, the ACA of EMSI was significantly higher than that of CCA (window length: 4, 4.5, and 5 s) and MSI (window length: 2.5, 4, 4.5, and 5 s). The ACA of ECCA was significantly higher than that of MSI in the case with a window length of 2.5 s. In the case of two channels, the ACAs of ECCA, MSI, and EMSI were significantly higher than that of CCA (ECCA: every window length except 3.5 s; MSI and EMSI: every window length). The ACAs of MSI and EMSI were significantly higher than that of ECCA (MSI: 2.5, 3, and 4 s; EMSI: every window length). The ACA of EMSI was significantly higher than that of MSI (window length: 3.5, 4, 4.5, and 5 s). Finally, in the case of three channels, the ACAs of ECCA, MSI, and EMSI were significantly higher than that of CCA in each window length. The ACAs of MSI and EMSI were significantly higher than that of ECCA (MSI: 3 s; EMSI: each window length except 2 s). The ACA of EMSI was significantly higher than that of MSI (window length: 3.5, 4, 4.5, and 5 s). Taken together, ACAs of MSI and EMSI were significantly higher than that of CCA and ECCA in the multichannel condition. The ACA of EMSI was significantly higher than that of the other algorithms except for FBCCA regardless of the number of channels.

TABLE 3

Table 3. Bonferroni corrected p-values of post-hoc analysis among the average classification accuracies across the electrode shift (ACA)s in five different algorithms for the number of channels and window length.

Robustness to the Electrode Shift

Figure 3 and Supplementary Table 2 show the average and standard deviation of RES across the 21 subjects with respect to the number of channels and classification algorithm. Table 4 shows the results of the post-hoc analysis among the RESs in the case of three channels for each classification algorithm. There was no significant difference among the RESs of the three different numbers of channels for CCA and ECCA. In the case of FBCCA, the RES for three channels was significantly higher than that for two channels (window length: 4, 4.5, and 5 s). For MSI and EMSI, the RESs increased with an increase in the number of channels, and the RES for three channels was significantly higher than those for one channel and two channels in every window length. To sum up, the RES for three channels was higher than that for both one and two channels when MSI and EMSI were used.

FIGURE 3

Figure 3. Mean and standard deviation of the robustness to the electrode shift (RES) across the subjects with five SSVEP classification algorithms in the case with the number of channels as 3 and window length (wl) as 7 (black, gray, and white bars denote the mean accuracies in the cases with the number of channels as 1, 2, and 3, respectively; the error bars represent standard deviations of RES).

TABLE 4

Table 4. Bonferroni corrected p-values of post-hoc analysis among the robustness to the electrode shift (RES) in the cases with the number of channels as 1, 2, and 3 for classification algorithm and window length.

Table 5 shows the results of the post-hoc analysis of the RESs of five classification algorithms for the number of channels. In the single-channel, there was no statistically significant difference among the RESs of the five algorithms. The RESs of MSI and EMSI were significantly higher than that of CCA, ECCA, and FBCCA for every window length when using multiple channels (two channels and three channels). In addition, in the case of three channels, the RES of ECCA was significantly higher than that of CCA (window length: 4 and 5 s), and the RES of EMSI was significantly higher than that of MSI (window length: 3.5, 4.5, and 5 s). In summary, the RESs of the five algorithms were similar under the single-channel condition. However, the RESs of MSI and EMSI were significantly higher than those of the other algorithms under multichannel conditions. In the case of three channels, the RES of EMSI was significantly higher than that of MSI.

TABLE 5

Table 5. Bonferroni corrected p-values of post-hoc analysis among the robustness to electrode shift (RES) in four different algorithms for the number of channels and window lengths.

Optimal Configuration for the Robustness to the Electrode Shift

Comparisons of two measures, i.e., the ACA and RES, with respect to the number of channels showed that the ACA increased significantly regardless of the algorithms and the RES increased significantly when MSI and EMSI were used in the case of three channels. Comparisons of two measures with respect to the algorithms showed that the ACA and RES of MSI and EMSI were superior to those of the CCA and ECCA under multichannel conditions. In particular, in the case of three channels, the RES of EMSI was superior to that of the other four algorithms. Therefore, the optimal conditions that make the SSVEP-based BCI robust to the electrode shift were the multichannel configurations (especially three channels) and the employment of the EMSI algorithm.

Discussion

The classification accuracy of SSVEP-based BCI is closely related to the signal-to-noise ratio (SNR) (Zhang et al., 2014; Kumar and Reddy, 2018). With an increase in the window length, the SNR of the SSVEP components increases, thereby leading to a higher classification accuracy (Xing et al., 2018). Similarly, as the number of channels increases, SNR of the SSVEP components can be enhanced by employing spatial filtering, also resulting in higher classification accuracy (Bin et al., 2009). Therefore, the variation in the accuracy owing to the electrode shift could decrease as the number of channels increases. In the MSI and EMSI, the S-estimator algorithm was used to measure the synchronization between the EEG signals and reference signals. Because the S-estimator is a non-linear measure of synchronization, both MSI and EMSI are known to have a lower probability of losing some useful information compared with CCA, ECCA, and FBCCA (Zhang et al., 2014; Zerafa et al., 2018). Additionally, the S-estimator has been proven robust to dynamic noises (Boccaletti et al., 2002; Carmeli et al., 2005; Jalili et al., 2007). Therefore, it is possible that MSI and EMSI based on S-estimator could classify the frequency of SSVEP more accurately at the surrounding positions of the original position compared with the other three algorithms.

Under the single-channel condition, the RES values of the five algorithms were similar as shown in Table 5. This result implies that the spatial filtering effect of five algorithms for strengthening the SSVEP components and suppressing the noises (Wong et al., 2020) was similar to each other in the case of a single channel. However, under the multichannel conditions, the RES values of MSI and EMSI were significantly higher than those of the other CCA-based algorithms.

Interestingly, we found that the variation in the classification performance due to the electrode shift increased as the number of channels increased when CCA and its variants (i.e., ECCA and FBCCA) were employed. As shown in Figures 2, 3, the ACA increased as the number of channels increased, but the RES for a single channel was higher than that for a multichannel. Particularly, this trend became more distinct for shorter window lengths with lower SNR. Bin et al. (2009) reported that the CCA weights from the parietal region and occipital region had the opposite sign, and thus the noise that these signals commonly have could be eliminated by employing the CCA weights. On the contrary, common noises may be enhanced if the CCA weights from different channels have the same sign. When two or three electrodes were shifted simultaneously, the SNR of SSVEP components could be either increased or decreased depending on the signs of the CCA weights, making the variation of the classification performance increased. As shown in Tables 2, 4, in the case of CCA-based algorithms, the ACA for three channels was significantly higher than those for single and two channels, while the RES did not increase as the number of channels increased. These results suggest that the CCA-based algorithms are not suitable for reducing the variation of classification performance due to the electrode dislocation.

Recently, BCI studies with portable EEG devices have been actively investigated with the aim of enhancing the practical usability of BCIs (Casson, 2019). When using portable EEG devices, it is generally difficult to place the EEG electrodes consistently on the designated locations because most portable devices are not customized to be fitted to the head shape of a specific user. Because the distance between adjacent electrodes in portable EEG devices is usually fixed, we further compared the classification accuracies when all the electrodes moved in the same direction. Figure 4 shows the nine-channel combinations corresponding to the cases in which all the electrodes shifted in the same direction, where the numbers in each element represent the EEG channel numbers. The three rows with colored matrices show the mean classification accuracies across the subjects for the nine-channel combinations with respect to different window sizes. The accuracies were calculated using EMSI, and the nine mean accuracies were min-max normalized. Notably, it could be seen from the figures that horizontal electrode shifts resulted in a relatively smaller decrease in the classification accuracy than vertical electrode shifts.

FIGURE 4

Figure 4. Nine channel combinations corresponding to the cases, in which all the electrodes shift to the same direction and the mean accuracies across the subjects of the nine-channel combinations (accuracies were calculated using EMSI and the nine mean accuracies all the cases with various number of channels and each window length were min–max normalized).

One of the limitations of our study is that we investigated the performance robustness to a slight electrode shift only for training-free SSVEP detection algorithms. According to a previous study (Zerafa et al., 2018), algorithms detecting SSVEP can be categorized based on training requirements into three categories: training-free, subject-specific, and subject-independent. Training-free algorithms are more practical than the ones included in the other two categories because they do not require any training data for SSVEP classification. However, classification algorithms using training data generally outperform the training-free algorithms because the inter-subject variability of the EEG activity is reflected in the classification. Algorithms belonging to the subject-independent category are expected to be a satisfactory compromise between performance and training effort. Therefore, evaluating the RES of various subject-independent algorithms would be one of the promising topics that we plan to pursue in future studies. Recently, SSVEP-based BCIs employing high-frequency bands have been actively studied because they could effectively reduce visual fatigue, compared to those employing low- and mid-frequency bands (Liang et al., 2019; Hsu et al., 2020). Despite its improved usability, however, its performance was not as high as that of the low- and mid-frequency SSVEP-based BCIs. Therefore, it would be a promising topic to investigate the influence of slight electrode displacement on the performance of high-frequency SSVEP-based BCIs.

Conclusion

In this study, we investigated the changes in the performance of SSVEP-based BCIs by minor electrode shifts to discover the optimal conditions that minimize the variations in classification accuracy while maintaining high classification accuracy. The performance robustness of SSVEP-based BCIs to the electrode shift with respect to the numbers of channels and classification algorithms was quantitatively investigated using two measures (ACA and RES). Our results suggested that the use of multichannel electrode configurations and employment of the EMSI algorithm could make the performance of SSVEP-based BCIs robust to the electrode shift from the optimal electrode locations. To the best of our knowledge, this was the first study that quantitatively evaluated the influence of electrode shift on the performance of SSVEP-based BCIs.

Recently, commercial wearable EEG devices that can record SSVEP responses in a more convenient manner have been released to the market. The representative examples include NextMind™ (https://www.next-mind.com) and BrainBit (https://www.brainbit.com), both of which employ dry electrodes and elastic headbands. Therefore, investigation on the robustness of BCI performance to the minor electrode shift for commercial wearable EEG devices would be necessary to assess the practicality of these devices. Additionally, it would be an interesting future topic to investigate optimal electrode configurations that are most robust to electrode shift in SSVEP-based BCIs.

Data Availability Statement

The preprocessed EEG datasets are available at the following link: https://figshare.com/s/cd46e1531d67f3023fa6.

Ethics Statement

The studies involving human participants were reviewed and approved by Institutional Review Board (IRB) of Hanyang University. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

HK: conceptualization, data curation, formal analysis, investigation, methodology, validation, visualization, and writing-original draft preparation. C-HI: conceptualization, investigation, supervision, writing-review, and editing. All the authors have reviewed and approved the final manuscript.

Funding

This work was supported in part by the National Research Foundation of Korea (NRF) (Grant No. 2019R1A2C2086593) and in part by the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korean government (MIST) (No. 2017-0-00432).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fninf.2021.750839/full#supplementary-material

References

Acqualagna, L., Botrel, L., Vidaurre, C., Kübler, A., and Blankertz, B. (2016). Large-scale assessment of a fully automatic co-adaptive motor imagery-based brain computer interface. PLoS ONE 11:e0148886. doi: 10.1371/journal.pone.0148886

PubMed Abstract | CrossRef Full Text | Google Scholar

Asgharpour, M., Foodeh, R., and Daliri, M. R. (2021). Regularized Kalman filter for brain-computer interfaces using local field potential signals. J. Neurosci. Methods 350:109022. doi: 10.1016/j.jneumeth.2020.109022

PubMed Abstract | CrossRef Full Text | Google Scholar

Bin, G., Gao, X., Yan, Z., Hong, B., and Gao, S. (2009). An online multi-channel SSVEP-based brain-computer interface using a canonical correlation analysis method. J. Neural Eng. 6:046002. doi: 10.1088/1741-2560/6/4/046002

PubMed Abstract | CrossRef Full Text | Google Scholar

Boccaletti, S., Kurths, J., Osipov, G., Valladares, D., and Zhou, C. (2002). The synchronization of chaotic systems. Phys. Rep. 366, 1–101. doi: 10.1016/S0370-1573(02)00137-0

CrossRef Full Text | Google Scholar

Boschmann, A., and Platzner, M. (2012). “Reducing classification accuracy degradation of pattern recognition based myoelectric control caused by electrode shift using a high density electrode array,” in 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (San Diego, CA: IEEE), 4324–4327.

PubMed Abstract | Google Scholar

Cao, L., Ju, Z., Li, J., Jian, R., and Jiang, C. (2015). Sequence detection analysis based on canonical correlation for steady-state visual evoked potential brain computer interfaces. J. Neurosci. Methods 253, 10–17. doi: 10.1016/j.jneumeth.2015.05.014

PubMed Abstract | CrossRef Full Text | Google Scholar

Carmeli, C., Knyazeva, M. G., Innocenti, G. M., and De Feo, O. (2005). Assessment of EEG synchronization based on state-space analysis. Neuroimage 25, 339–354. doi: 10.1016/j.neuroimage.2004.11.049

PubMed Abstract | CrossRef Full Text | Google Scholar

Casson, A. J. (2019). Wearable EEG and beyond. Biomed. Eng. Lett. 9, 53–71. doi: 10.1007/s13534-018-00093-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, X., Wang, Y., Gao, S., Jung, T.-P., and Gao, X. (2015). Filter bank canonical correlation analysis for implementing a high-speed SSVEP-based brain–computer interface. J. Neural Eng. 12:046008. doi: 10.1088/1741-2560/12/4/046008

PubMed Abstract | CrossRef Full Text | Google Scholar

Cheng, M., Gao, X., Gao, S., and Xu, D. (2002). Design and implementation of a brain-computer interface with high transfer rates. IEEE Trans. Biomed. Eng. 49, 1181–1186. doi: 10.1109/TBME.2002.803536

PubMed Abstract | CrossRef Full Text | Google Scholar

Fan, Z., Wang, Z., Li, G., and Wang, R. (2016). “A canonical correlation analysis based EMG classification algorithm for eliminating electrode shift effect,” in 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (Orlando, FL) 867–870.

PubMed Abstract | Google Scholar

Gu, Z., Chen, Z., Zhang, J., Zhang, X., and Yu, Z. L. (2019). An online interactive paradigm for P300 brain-computer interface speller. IEEE Trans. Neural Syst. Rehabil. Eng. 27, 152–161. doi: 10.1109/TNSRE.2019.2892967

PubMed Abstract | CrossRef Full Text | Google Scholar

Han, C.-H., Hwang, H.-J., and Im, C.-H. (2013). Classification of visual stimuli with different spatial patterns for single-frequency, multi-class SSVEP BCI. Electron. Lett. 49, 1374–1376. doi: 10.1049/el.2013.2792

CrossRef Full Text | Google Scholar

Hill, N. J., Lal, T. N., Schroder, M., Hinterberger, T., Wilhelm, B., Nijboer, F., et al. (2006). Classifying EEG and ECoG signals without subject training for fast BCI implementation: comparison of nonparalyzed and completely paralyzed subjects. IEEE Trans. Neural Syst. Rehabil. Eng. 14, 183–186. doi: 10.1109/TNSRE.2006.875548

PubMed Abstract | CrossRef Full Text | Google Scholar

Hsu, C.-C., Yeh, C.-L., Lee, W.-K., Hsu, H.-T., Shyu, K.-K., Li, L. P.-H., et al. (2020). Extraction of high-frequency SSVEP for BCI control using iterative filtering based empirical mode decomposition. Biomed. Signal Process. Control 61:102022. doi: 10.1016/j.bspc.2020.102022

CrossRef Full Text | Google Scholar

Hwang, H.-J., Choi, H., Kim, J.-Y., Chang, W.-D., Kim, D.-W., Kim, K., et al. (2016). Toward more intuitive brain–computer interfacing: classification of binary covert intentions using functional near-infrared spectroscopy. J. Biomed. Opt. 21:091303. doi: 10.1117/1.JBO.21.9.091303

PubMed Abstract | CrossRef Full Text | Google Scholar

Hwang, H.-J., Kim, S., Choi, S., and Im, C.-H. (2013). EEG-based brain-computer interfaces: a thorough literature survey. Int. J. Hum. Comput. Interact. 29, 814–826. doi: 10.1080/10447318.2013.780869

CrossRef Full Text | Google Scholar

Hwang, H.-J., Lim, J.-H., Jung, Y.-J., Choi, H., Lee, S. W., and Im, C.-H. (2012). Development of an SSVEP-based BCI spelling system adopting a QWERTY-style LED keyboard. J. Neurosci. Methods 208, 59–65. doi: 10.1016/j.jneumeth.2012.04.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Hwang, H.-J., Lim, J.-H., Kim, D.-W., and Im, C.-H. (2014). Evaluation of various mental task combinations for near-infrared spectroscopy-based brain-computer interfaces. J. Biomed. Opt. 19:077005. doi: 10.1117/1.JBO.19.7.077005

PubMed Abstract | CrossRef Full Text | Google Scholar

Hwang, H. J., Han, C. H., Lim, J. H., Kim, Y. W., Choi, S. I., An, K. O., et al. (2017). Clinical feasibility of brain-computer interface based on steady-state visual evoked potential in patients with locked-in syndrome: case studies. Psychophysiology 54, 444–451. doi: 10.1111/psyp.12793

PubMed Abstract | CrossRef Full Text | Google Scholar

Jalili, M., Lavoie, S., Deppen, P., Meuli, R., Do, K. Q., Cuenod, M., et al. (2007). Dysconnection topography in schizophrenia revealed with state-space analysis of EEG. PLoS ONE 2:e1059. doi: 10.1371/journal.pone.0001059

PubMed Abstract | CrossRef Full Text | Google Scholar

Jalilpour, S., Sardouie, S. H., and Mijani, A. (2020). A novel hybrid BCI speller based on RSVP and SSVEP paradigm. Comput. Meth. Programs Biomed. 187:105326. doi: 10.1016/j.cmpb.2020.105326

PubMed Abstract | CrossRef Full Text | Google Scholar

Joudaki, A., Salehi, N., Jalili, M., and Knyazeva, M. G. (2012). EEG-based functional brain networks: does the network size matter? PLoS ONE 7:e35673. doi: 10.1371/journal.pone.0035673

PubMed Abstract | CrossRef Full Text | Google Scholar

Kim, D.-W., Hwang, H.-J., Lim, J.-H., Lee, Y.-H., Jung, K.-Y., and Im, C.-H. (2011). Classification of selective attention to auditory stimuli: toward vision-free brain–computer interfacing. J. Neurosci. Methods 197, 180–185. doi: 10.1016/j.jneumeth.2011.02.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Kristensen, A. B., Subhi, Y., and Puthusserypady, S. (2020). Vocal imagery vs intention: viability of vocal-based EEG-BCI paradigms. IEEE Trans. Neural Syst. Rehabil. Eng. 28, 1750–1759. doi: 10.1109/TNSRE.2020.3004924

PubMed Abstract | CrossRef Full Text | Google Scholar

Kumar, G. K., and Reddy, M. R. (2018). Periodic component analysis as a spatial filter for SSVEP-based brain-computer interface. J J. Neurosci. Methods 307, 164–174. doi: 10.1016/j.jneumeth.2018.06.003

PubMed Abstract | CrossRef Full Text | Google Scholar

León, J., Escobar, J. J., Ortiz, A., Ortega, J., González, J., Martín-Smith, P., et al. (2020). Deep learning for EEG-based motor imagery classification: accuracy-cost trade-off. PLoS ONE 15:e0234178. doi: 10.1371/journal.pone.0234178

PubMed Abstract | CrossRef Full Text | Google Scholar

Liang, L., Yang, C., Wang, Y., and Gao, X. (2019). “High-frequency SSVEP stimulation paradigm based on dual frequency modulation,” in 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (Berlin: IEEE), 6184–6187.

PubMed Abstract | Google Scholar

Lim, J.-H., Lee, J.-H., Hwang, H.-J., Kim, D. H., and Im, C.-H. (2015). Development of a hybrid mental spelling system combining SSVEP-based brain–computer interface and webcam-based eye tracking. Biomed. Signal Process. Control 21, 99–104. doi: 10.1016/j.bspc.2015.05.012

CrossRef Full Text | Google Scholar

Lim, J. H., Kim, Y. W., Lee, J. H., An, K. O., Hwang, H. J., Cha, H. S., et al. (2017). An emergency call system for patients in locked-in state using an SSVEP-based brain switch. Psychophysiology 54, 1632–1643. doi: 10.1111/psyp.12916

PubMed Abstract | CrossRef Full Text | Google Scholar

Lin, Z., Zhang, C., Wu, W., and Gao, X. (2007). Frequency recognition based on canonical correlation analysis for SSVEP-based BCIs. IEEE Trans. Biomed. Eng. 54, 1172–1176. doi: 10.1109/TBME.2006.889197

PubMed Abstract | CrossRef Full Text | Google Scholar

Lotte, F., Bougrain, L., Cichocki, A., Clerc, M., Congedo, M., Rakotomamonjy, A., et al. (2018). A review of classification algorithms for EEG-based brain–computer interfaces: a 10 year update. J. Neural Eng. 15:031005. doi: 10.1088/1741-2552/aab2f2

PubMed Abstract | CrossRef Full Text | Google Scholar

Lv, B., Sheng, X., and Zhu, X. (2018). “Improving myoelectric pattern recognition robustness to electrode shift by autoencoder,” in 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (Honolulu, HI: IEEE), 5652–5655.

PubMed Abstract | Google Scholar

Mellinger, J., Schalk, G., Braun, C., Preissl, H., Rosenstiel, W., Birbaumer, N., et al. (2007). An MEG-based brain–computer interface (BCI). Neuroimage 36, 581–593. doi: 10.1016/j.neuroimage.2007.03.019

PubMed Abstract | CrossRef Full Text | Google Scholar

Meng, L., Jin, J., and Wang, X. (2011). “A comparison of three electrode channels selection methods applied to SSVEP BCI,” in Biomedical Engineering and Informatics (BMEI), 2011 4th International Conference on IEEE (Shanghai), 584–587.

Google Scholar

Nakanishi, M., Wang, Y., Chen, X., Wang, Y.-T., Gao, X., and Jung, T.-P. (2018). Enhancing detection of SSVEPs for a high-speed brain speller using task-related component analysis. IEEE Trans. Biomed. Eng. 65, 104–112. doi: 10.1109/TBME.2017.2694818

PubMed Abstract | CrossRef Full Text | Google Scholar

Nakanishi, M., Wang, Y., Wang, Y.-T., and Jung, T.-P. (2015). A comparison study of canonical correlation analysis based methods for detecting steady-state visual evoked potentials. PLoS ONE 10:e0140703. doi: 10.1371/journal.pone.0140703

PubMed Abstract | CrossRef Full Text | Google Scholar

Oikonomou, V. P., Liaros, G., Georgiadis, K., Chatzilari, E., Adam, K., Nikolopoulos, S., et al. (2016). Comparative evaluation of state-of-the-art algorithms for SSVEP-based BCIs. arXiv preprint arXiv:1602.00904. Available online at: https://arxiv.org/abs/1602.00904

Google Scholar

Pan, L., Zhang, D., Jiang, N., Sheng, X., and Zhu, X. (2015). Improving robustness against electrode shift of high density EMG for myoelectric control through common spatial patterns. J. NeuroEng. Rehabil. 12, 1–16. doi: 10.1186/s12984-015-0102-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Park, S. A., Hwang, H. J., Lim, J. H., Choi, J. H., Jung, H. K., and Im, C. H. (2013). Evaluation of feature extraction methods for EEG-based brain-computer interfaces in terms of robustness to slight changes in electrode locations. Med. Biol. Eng. Comput. 51, 571–579. doi: 10.1007/s11517-012-1026-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Park, S., Cha, H.-S., Kwon, J., Kim, H., and Im, C.-H. (2020). “Development of an online home appliance control system using augmented reality and an ssvep-based brain-computer interface,” in 2020 8th International Winter Conference on Brain-Computer Interface (BCI) (Gangwon: IEEE), 1–2.

Google Scholar

Regan, D. (1989). “Evoked potentials and evoked magnetic fields in science and medicine,” in Human Brain Electrophysiol (New York, NY: Elsevier), 59–61.

Google Scholar

Schalk, G., Kubanek, J., Miller, K., Anderson, N., Leuthardt, E., Ojemann, J., et al. (2007). Decoding two-dimensional movement trajectories using electrocorticographic signals in humans. J. Neural Eng. 4:264. doi: 10.1088/1741-2560/4/3/012

PubMed Abstract | CrossRef Full Text | Google Scholar

Simon, N., Käthner, I., Ruf, C. A., Pasqualotto, E., Kübler, A., and Halder, S. (2015). An auditory multiclass brain-computer interface with natural stimuli: usability evaluation with healthy participants and a motor impaired end user. Front. Hum. Neurosci. 8:1039. doi: 10.3389/fnhum.2014.01039

PubMed Abstract | CrossRef Full Text | Google Scholar

Speier, W., Deshpande, A., Cui, L., Chandravadia, N., Roberts, D., and Pouratian, N. (2017). A comparison of stimulus types in online classification of the P300 speller using language models. PLoS ONE 12:e0175382. doi: 10.1371/journal.pone.0175382

PubMed Abstract | CrossRef Full Text | Google Scholar

Tello, R. M., Muller, S. M., Bastos-Filho, T., and Ferreira, A. (2014). “A comparison of techniques and technologies for SSVEP classification,” in Biosignals and Biorobotics Conference (2014): Biosignals and Robotics for Better and Safer Living (BRC), 5th ISSNIP-IEEE (Salvador: IEEE), 1–6.

PubMed Abstract | Google Scholar

Vialatte, F.-B., Maurice, M., Dauwels, J., and Cichocki, A. (2010). Steady-state visually evoked potentials: focus on essential paradigms and future perspectives. Prog. Neurobiol. 90, 418–438. doi: 10.1016/j.pneurobio.2009.11.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., and Vaughan, T. M. (2002). Brain–computer interfaces for communication and control. Clin. Neurophysiol. 113, 767–791. doi: 10.1016/S1388-2457(02)00057-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Wong, C. M., Wang, B., Wang, Z., Lao, K. F., Rosa, A., and Wan, F. (2020). Spatial filtering in SSVEP-based BCIs: unified framework and new improvements. IEEE Trans. Biomed. Eng. 67, 3057–3072. doi: 10.1109/TBME.2020.2975552

PubMed Abstract | CrossRef Full Text | Google Scholar

Xing, X., Wang, Y., Pei, W., Guo, X., Liu, Z., Wang, F., et al. (2018). A high-speed SSVEP-based BCI using dry EEG electrodes. Sci. Rep. 8:14708. doi: 10.1038/s41598-018-32283-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Xu, M., Chen, L., Zhang, L., Qi, H., Ma, L., Tang, J., et al. (2014). A visual parallel-BCI speller based on the time–frequency coding strategy. J. Neural Eng. 11:026014. doi: 10.1088/1741-2560/11/2/026014

PubMed Abstract | CrossRef Full Text | Google Scholar

Zerafa, R., Camilleri, T., Falzon, O., and Camilleri, K. P. (2018). To train or not to train? A survey on training of feature extraction methods for SSVEP-based BCIs. J. Neural Eng. 15:051001. doi: 10.1088/1741-2552/aaca6e

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhang, Y., Guo, D., Yao, D., and Xu, P. (2017). The extension of multivariate synchronization index method for SSVEP-based BCI. Neurocomputing 269, 226–231. doi: 10.1016/j.neucom.2017.03.082

CrossRef Full Text | Google Scholar

Zhang, Y., Jin, J., Qing, X., Wang, B., and Wang, X. (2012). LASSO based stimulus frequency recognition model for SSVEP BCIs. Biomed. Signal Process. Control 7, 104–111. doi: 10.1016/j.bspc.2011.02.002

CrossRef Full Text | Google Scholar

Zhang, Y., Xu, P., Cheng, K., and Yao, D. (2014). Multivariate synchronization index for frequency recognition of SSVEP-based brain-computer interface. J. Neurosci. Methods 221, 32–40. doi: 10.1016/j.jneumeth.2013.07.018

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhu, D., Bieger, J., Garcia Molina, G., and Aarts, R. M. (2010). A survey of stimulation methods used in SSVEP-based BCIs. Comput. Intell. Neurosci. 2010:702357. doi: 10.1155/2010/702357

PubMed Abstract | CrossRef Full Text | Google Scholar