A Synchronous Motor Imagery Based Neural Physiological Paradigm for Brain Computer Interface Speller

Brain Computer Interface (BCI) speller is a typical BCI-based application to help paralyzed patients express their thoughts. This paper proposed a novel motor imagery based BCI speller with Oct-o-spell paradigm for word input. Furthermore, an intelligent input method was used for improving the performance of the BCI speller. For the English word spelling experiment, we compared synchronous control with previous asynchronous control under the same experimental condition. There were no significant differences between these two control methods in the classification accuracy, information transmission rate (ITR) or letters per minute (LPM). And the accuracy rates of over 70% validated the feasibility for these two control strategies. It was indicated that MI-based synchronous control protocol was feasible for BCI speller. And the efficiency of the predictive text entry (PTE) mode was superior to that of the Non-PTE mode.

Cecotti proposed a SSVEP-based speller with five stimulus buttons including character groups (Cecotti, 2010). To spell a letter, user should focus on the corresponding stimuli. Volosyak et al. developed a SSVEP speller controlled using a virtual keyboard (Volosyak et al., 2011). Five commands were used to move the cursor and select the character in a virtual keyboard. Speed of the SSVEP-based speller is reduced due to the limitation FIGURE 1 | In the first layer, 26 letters, 10 digits, and 6 symbols are divided into eight blocks. In the second layer, 7 different interfaces are designed to unfold the blocks in the above layer. Only two symbols are connected to the third layer. The subject can select the associating word by numeric option layer by layer. of control commands. To solve this problem, 30 flicking stimulus with different frequencies were designated to the speller (Hwang et al., 2012). The average speed was 9.39 letter per minute (LPM), and the average rate of data transfer was 40.72 bits per minute. One disadvantage for these spellers is visual fatigue caused by prolonged gazing.
The Berlin BCI research group has proposed a novel speller paradigm called "Hex-o-Spell, " with which subjects spell a character using two-step commands in a hexagon interface (Blankertz et al., 2006). For most subjects, the speed of their performance in this two-class MI-based BCI speller was between 4.6 and 7.6 LPM. However, the performances of existing FIGURE 2 | Framework for asynchronous and synchronous control protocols. First, EEG signals are band pass filtered at 5-30 Hz for removing background noise. Then, common spatial pattern (CSP) and support vector machine (SVM) are combined to discriminate three-class (left hand, right hand, foot) MI patterns. For asynchronous control protocol, a brain switch is used for turn on/off this BCI speller by MI tasks.
MI-based spellers are not comparable with visual stimuli-based speller (Blankertz et al., 2006;D'albis et al., 2012). Recently, the efficiency of MI-based BCI speller has been greatly improved by the Oct-o-spell paradigm in our previous work . We developed a 2-D cursor control strategy using the modified Hex-o-Spell paradigm and an asynchronous control switch was used for starting the system.
In this paper, synchronous protocol is presented for BCI speller. Furthermore, an intelligent input method is used for improving the performance of the BCI speller. The online experiment is conducted involving three subjects. The comparable result is showed to validate the feasibility of this BCI speller.

Oct-o-Spell Paradigm
In this paper, a three-layer interface of Oct-o-Spell paradigm is proposed for character input (Figure 1). In the first layer, 26 letters, 10 digits and 6 symbols ("Quit, " "=, " "←, " "., " "F1, " "Chn/Fn") are divided into eight blocks. In the second layer, 8 different interfaces are designed to unfold the blocks in the first layer. Only two symbols are connected to the third layer. One is "F1, " including 6 punctuation: " * , " "@, " "?, " "+, " "!, " and"#." The other one is 'Quit' , which is used for quitting the system. After the user inputs several characters, associative words will be listed by numeric labels. The user can select it by entering the corresponding number. Then, he/she doesn't need to type all letters for saving considerable time. The function is adopted for predictive text entry (PTE) to improve the efficiency of character input. The monitor resolution is 1,440 * 838 pixels and the ratio of the size of cursor target circle and the screen is 0.0003:0. 0065:1. The classification probabilities are marked as P1, P2, P3 which is mapped to three vectors. The angle between each two vectors is 120 • and the length of the vector is equal to the value of the probability.

Motor Imagery-Based Synchronous Control Protocol
In the previous study , we presented a motor imagery-based BCI for asynchronous 2D cursor control. Compared with this method, synchronous control remove the brain-actuated switch to improve the efficiency. These two control protocols are illustrated in Figure 2. First, EEG signals are band pass filtered at 5-30 Hz for removing background noise. Then, common spatial pattern (CSP) and support vector machine (SVM) are combined to discriminate three-class (left hand, right hand, foot) MI patterns. The CSP method is useful for discriminating two populations of EEG. Based on the dataset, a transformation matrix is calculated for a spatial filter. The first and last rows of the CSP transformation matrix could maximize the difference of two groups of data. In our algorithm, the first and last two rows are selected to construct the transformation matrix for further data analysis. On the other hand, the probabilities from SVM classifier are three decimals ranged from 0 to 1 and sum of them equals to 1. The feature vectors are fed into the SVM classifier, provided by LIBSVM FIGURE 4 | The user performs MI tasks to drive the cursor to the circle of the target block. The corresponding character will be selected for output.
toolbox (Chang and Lin, 2011), to predict the class label and probability estimate for multi-class task. The radial basis function (RBF) is selected as the kernel function and a five-fold crossvalidation in training data is used for choosing suitable SVM parameters. As shown in Figure 3, they are marked as P1, P2 and P3 which are mapped to three vectors. The angle between each two vectors is 120 • and the length of the vector is equal to the value of the probability. To hit the target located between two vectors, the user performs two corresponding motor imagery tasks simultaneously to generate a resultant vector, by which the cursor is moved directly to the target.
On the other hand, the moving distance is computed by the classification result. We define the moving step as L, and the horizontal and vertical moving distances, d x and d y , are calculated as follows: where the negative value means that the cursor moves left and vice visa.
To spell character or exit this system, the subject needs to move the cursor to the target circle of the block in each layer. At the first layer, the user can move cursor to reach the target circle of the characters block (containing the target character). Then the interface will extend to the second layer according to the previous selection. Then, the target character can be targeted in the second layer and the speller will output this letter. Only "Quit" or "F1" is chosen in this layer, the interface would extend to the next layer. And the target character can be typed in the third layer or quit the system. To hit a target block (Figure 4), the user should perform MI tasks to drive the cursor to the circle of the target block. If a  wrong block is selected, the "Back" command in the next layer can be selected to return to the previous interface. If a wrong character is chose, the subject can select the "Delt" command to delete it in the second layer. The MI BCI system will produce projection values for cursor moving every 0.5 s. Typically, the subject needs to take about 3 s (6 trials) to conduct MI task for targeting the circle of a block in one certain layer. And the maximum time consumption will be given in the description of experimental settings.

Subjects and Experimental Settings
Six healthy subjects (4 males, 2 females) aged between 19 and 26, took part in our experiment. All of them were naive in BCI and consent to be involved in the study. Firstly, three-class (left hand imagery, right hand imagery and feet imagery) MI sessions were conducted for training in one day. Each training session included 20 trials for each class and each trial last 2 s. Subjects had a short break between sessions. In the experiment, each subject was seated in front of a notebook computer, with arms on the chair arms and hands relaxed. An arrow pointing three different directions was used to indicate one class MI. Finally, three subjects who had achieved performance rates of over 70% (criterion level) were selected for subsequent spelling experiment.
In the spelling experiment, all previous train data were used for training SVM classifiers. EEG signals were recorded by a 16-channel g.USBamp amplifier, and 13 electrodes were placed according to the international 10-20 system (FC3, FCZ, FC4, C5, C3, C1, CZ, C2, C4, C6, CP3, CPZ, and CP4).The ground and reference electrodes were placed on the FZ position and the right earlobe. EEG signals were sampled at 256 Hz and band-pass filtered between 5 and 30 Hz. Online copy spelling experiment: In this experiment, the subject used the above synchronous method for MI-based BCI task. This BCI system could produce three projection values in corresponding directions every 0.5 s for cursor moving. The BCI classifier model was trained by previous MI training sessions.  In order to help the subjects with the speller interface, each subject performed 20 free spell familiarize runs in which subject was asked to type characters from A to Z (26 letters) for 2 days. Then the subjects participated in an online copy spelling experiment on the third day. Firstly, 3 subjects spelled 15 English words by PTE and non-PTE modes. The words were selected from reference (Hwang et al., 2012): "WOMEN, DESK, WATER, HAND, MEMORY, ZONE, BABY, FACE, TAXI, JUNE, QUICK, VIDEO, GOLF, HOUR, PENCIL" for synchronous paradigms testing. There was a 3-s interval between trials. If the subject couldn't spell the word correctly in 40 s, this trial would be ended and another trial started. Wrongly typed letters had to be corrected. Furthermore, all subjects performed Chinese character input by PTE mode. Ten common words were used for this test ( Table 1). And the rule was the same as English character input.

RESULTS
In our study, online English and Chinese words spelling experiments were respectively conducted to evaluate our synchronous BCI system. Conventionally, some typical indicators were selected to assess the systematical efficiency.
(1) Accuracy rate (ACC): the percentage of successful classification result.
(2) Information transmission ratio (ITR): the bit rate of information communicated per unit time.
(3) Letter per minute (LPM). Table 2 shows the performance for 3 subjects. The comparison between PTE and non-PTE methods didn't show significant differences. For the English word spelling experiment, we compared synchronous control with previous asynchronous control under the same experimental condition (Figure 5). There were no significant differences between these two control methods in ACC (paired t-test: t = 0.4125, p > 0.05), ITR (paired t-test: t = 2.3053, p > 0.05) or LPM (paired t-test: t = 0.2428, p > 0.05). Even so, the synchronous mode had better performance in ITR and LPM. It was verified that this strategy had higher efficiency in BCI output than that of asynchronous mode.
Chinese spelling tasks were performed by all subjects. The performance is presented in Table 3. All subjects achieved sufficient performance rate of over 70% (criterion level). It demonstrated that our proposed BCI speller was feasible for both English and Chinese.

DISCUSSION
For 3 subjects, experimental results showed that motor imagery was feasible for common communication and expression in BCI. All subjects gave positive feedback to the manipulation of the speller after experiments. No pronounced difference was observed between our MI-based speller and other BCI spellers (e.g., SSVEP, P300) (Cecotti, 2010;Jin et al., 2011a,b;Xu et al., 2014). Moreover, the accuracy and ITR of our MI-based speller are approximately equal to those of hybrid BCIs (Xu et al., 2013;Yin et al., 2013). This MI-based control strategy is useful for Octo-Spell paradigm. The result of LPM implied that our MI-control strategy had better output performance than those of previous BCI spellers (Blankertz et al., 2006;D'albis et al., 2012). Moreover, the number of candidate characters was 36 for our BCI system. And previous MI-based BCI spellers had few target characters for user selection. It indicated that our BCI speller was more practical than other MI-based BCI spellers. In the experiment, subjects performed two motor imagery tasks simultaneously. Compared with traditional single modalitybased MI control task, it was more difficult to obtain accurate results in this parallel control system because of simultaneous multi modality -based MIs. We reported the number of errors for all motor imagery tasks (Figure 6). The error was defined as the incorrect block selection resulted from cursor moving, which indicated by the use of "Back" and "Delt." It was speculated that, before these two characters were used for correcting a mistake, the user didn't move the cursor along with the target direction, caused by poor performance in MI tasks. It indicated that synchronous motor imagery tasks (i.e., left hand and feet motor imagery, right hand and feet motor imagery, and left hand and right hand motor imagery) were harder to be mentally controlled. Hand-based motor imagery with two hands was the most difficult task for moving the cursor to the target point. Therefore, characters used frequently must be located in the position where the subject is able to move the cursor by single modality-based MI.
Although the precision and ITR of Non-PTE mode was slightly higher than PTE mode, no significant differences were observed in terms of performance between the PTE and the Non-PTE mode. The detailed process of character input for all subjects was reported at Tables 4-6. Although there were more spelling errors in the PTE mode, which was probably due to distraction caused by word selection, the number of required characters by the PTE mode was less than that for the Non-PTE mode. Nevertheless, this disadvantage was compensated by the reduction of required spelling letters. It was reflected in the average number of spelling letters per word. That was the reason why the LPM of the PTE mode was higher than that for the Non-PTE mode. Moreover, the efficiency of PTE mode could be improved by raising the precision of character selection.
The ITRs for the synchronous mode were higher than those for the asynchronous mode for all the subjects. The asynchronous BCI took some time to start the speller, which reduced the transmission efficiency in a trial. Therefore, the number of switch control was decreased for improving the efficiency of the BCI speller. Therefore, subjects needed to reduce the use of switch control for improving the efficiency of the BCI speller. For instance, when the user activated the switch consciously in the first layer, the switch would be active automatically for target selection in the second and third layer. Because the user intended to spell something after activating the speller in the first layer. The switch was redundant to detect mental state when the user was conscious to manipulating. Thus, the time consumption would be reduced for better ITR and LPM. Meanwhile, we could add a selective function for switching off the system in the second and third layers. It was used for correcting the false positive selection in the first layer. This control protocol that possess the superiority of both patterns, would be developed in further work. The number of electrodes of our BCI speller was higher than SSVEP-based BCI speller (Xu et al., 2014). As mentioned, fewer channels were beneficial for wearable portable BCI devices. And the selection of electrodes would be considered for real conditions and user experience in future. Furthermore, optional mode-based BCI speller should be developed to account for individual differences.

CONCLUSIONS
This paper proposed a novel MI-based BCI speller with Octo-spell paradigm for word input. The 3D motor imagery was used for synchronous control. The detailed analysis was given in the discussion. Our results showed that MI-based synchronous protocol was feasible for BCI speller. And the efficiency of the PTE mode was superior to that of the non-PTE mode.

ETHICS STATEMENT
This study was carried out in accordance with the recommendations of "name of guidelines, name of committee" with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the "name of committee."

AUTHOR CONTRIBUTIONS
LC and BX contributed to the conception of the study, as well as performed the data analyses and wrote the manuscript. OM, JL, and HX contributed significantly to analysis and manuscript preparation. And NB helped revising the manuscript critically for important intellectual content.

ACKNOWLEDGMENTS
The work was supported by the Fundamental Research Funds for International Young Scientists Title of the awarded research project (Grant No. 61550110252) and Shanghai NSF research project (Grant No. 16JC1401300).