How capable is non-invasive EEG data of predicting the next movement? A mini review

1. Introduction

The concept of prediction in motor tasks was first introduced by Helmholtz in 1867 in an attempt to explain how humans localize objects. Later on, the concept of efference copies was published by Von Holst and Sperry in the fifties stating that motor commands create an internal copy which reflects the predicted movement and its resulting sensations (Blakemore et al., 1998; Wolpert and Flanagan, 2001). Since then, the idea that humans predict the consequences of their motor tasks has appeared as a prominent theory in all aspects of sensorimotor commands. Generally, prediction refers to estimating future states of a system.

Moreover, researches show great potential in the analysis of brain rhythms and event-related potentials (ERP) recorded by electroencephalography (EEG). Through EEG data acquisition, the current state of the brain can be understood in real time. As also mentioned above, part of this current state is a prediction of the next motor task. Therefore, by understanding and analyzing the brain rhythms and ERPs, the future motor commands can be predicted. This combination can lead to better rehabilitation tools for people with physical challenges. It might even have a potential to be used as supportive gadgets for healthy individuals.

In general, EEG recordings can be divided into two categories: non-invasive EEG recordings obtained from electrodes attached to the scalp surface and invasive EEG recorded from electrodes implanted inside the cranium. Since implanted electrodes in invasive EEG are closer to the brain than scalp electrodes, they can record the brain signals with higher amplitudes and smaller spatial scales ranging from a single neuron cell to distributed cell groups. Additionally, it is assumed that invasive EEG recording does not suffer from major artefacts that compromise non-invasive EEG, such as eye blinks. However, invasive EEG suffers from some technical difficulties and have significant clinical risks; because recording electrodes are implanted in the cortex and are required to function well for a long time, there is a risk of infections and other damage to the brain (Engel et al., 2005; Ball et al., 2009). Many studies have shown that although non-invasive EEG is less accurate in comparison with invasive EEG, it still contains enough real-time information to be used as a source for different applications and even in real-time brain computer interface (BCI) machines oriented to tasks such as word processing, Internet browsing or controlling a two-dimensional movement (Wolpaw and McFarland, 2004; Müller and Blankertz, 2006; Citi et al., 2008). Thus, in this article, the main focus is on non-invasive EEG single-trial analysis which from now on is shortened to EEG.

Another concept to address is the meaning of “before the movement” or “pre-movement” activity. This phenomenon refers to the time when no muscle movement is detectable or irrelevant if it occurs, but the subject is completely aware of the action that he is going to perform in the near future and it is also referred to as planning/preparation of the movements (Crammond and Kalaska, 2000; Toni and Passingham, 2003). In this time interval, which ranges from 500 ms to 2 s before movement onset, the cortex is preparing for action execution. As will be seen in section 2, there is a noticeable change in brain waves that only happens before movement.

This article aims to investigate the combination of the concepts of EEG and prediction, to see, based on today's knowledge and previous experiments, if it is possible to predict the next motor movement using current EEG recording. In the next section, we describe the main changes in the EEG data reported just before the voluntary movement. Then, in section 2 we see how the knowledge of these changes can be used to extract information about the upcoming movement. In each case, we address the EEG setup and protocol used and discuss the main foundation of the study. Finally, in the discussion and conclusion section we summarize the main ideas with the hope to draw more attention to the richness of pre-movement and pre-motor imagery EEG.

2. Changes in Brain Rhythm Reported Just Before the Movement

In this section, we address the changes detectable in EEG before the onset of the movement or motor imagery. All the following phenomena have been reported both when the movement is actually executed and when it is imagined as a part of the motor preparation procedure. One or a combination of these changes are the main focus of the studies trying to extract features from pre-movement or pre-motor imagery period discussed in the next section.

2.1. Bereitschaftspotential or Readiness Potential

Bereitschaftspotential (BP) or readiness potential (RP) is a negative cortical potential which starts to develop around 1.5 to 1 s before the onset of a voluntary movement and has two main components. The first component, also called “early BP,” is a slow-rising negative segment which begins about 1.5 s prior to the movement onset and is more prominent in the central-medial scalp. The other component, the “late BP,” has a steeper slope and occurs around 400 ms prior to the movement onset, having maximum amplitude over the primary motor cortex (M1). BP is an ERP, since its onset is time-locked to an event such as movement (Jahanshahi and Hallett, 2003; Shibasaki and Hallett, 2006). BP is proven to be evoked not only when a movement is performed by the subject, but also when the execution of an action by others is observed or even when the movement is imagined (Pineda et al., 2000; Kilner et al., 2004). Two of the most recent studies which focus on detection of BP are Ahmadian et al. (2012) and Ahmadian et al. (in press).

2.2. Alpha and Beta Event-Related Desynchronization

A short-lasting block/decrease of frequency power or event-related desynchronization (ERD) in the alpha band (about 8–12 Hz) and in the central beta band (about 16–24 Hz) has been reported beginning about 2 s before self-paced movement or motor imageries (Pfurtscheller and Neuper, 2003).

2.3. Contingent Negative Variation

Contingent negative variation (CNV) is a slow negative wave that develops in the interval between a “Warning” and a “Go” stimulus and shows anticipation for a forthcoming signal and preparation for execution of a response. In other words, CNV reflects preparation for signaled movements and is an index for expectation. The earlier segment of the CNV has maximum amplitude over the frontal cortex and is generated in response to a “Warning” cue. The later or terminal CNV (tCNV) begins around 1.5 s before the “Go” cue, it reflects preparation for motor response and has maximum amplitude over the motor cortex (M1) (Rohrbaugh et al., 1976; Brunia, 2003).

CNV potential has not been the main focus of the studies, although some studies have used a cue-based delayed protocol which can trigger this potential and thus affect the results reported (Hammon et al., 2008; Wang and Makeig, 2009; Lew et al., 2012). The one study that actually focused on detecting this potential states that it was not identified as the main pre-movement/pre-motor imagery feature in most of the subjects (Morash et al., 2008).

3. Studies

In this section we consider the studies that extract information from the pre-movement or pre-imageries EEG data. Some studies show the effectiveness of data acquired in this period in the real-time BCIs. Each study used a different EEG data acquisition including the electrodes montage and signal enhancement which is also discussed, since it is directly related to the results reported. We organized the studies in three groups based on the aims and findings of authors: predicting the onset of the next movement, the direction of next movement and also the limb engaged in the movement. These studies prove that with right EEG setting and signal processing, significant information can be extracted about the movement yet to come (see Table 1).

TABLE 1

Table 1. Studies on Pre-movement or Pre-imagery motor task categorized by the main findings.

3.1. Predicting the Intention of Movement

Studies carried out in Haw et al. (2006); Bai et al. (2011); Niazi et al. (2011) and Lew et al. (2012) try to answer the question whether or not the subject wants to move in the short future. They do not try to determine the kind or the direction of movement but to detect the upcoming of any movement. In these studies, the movement onset is not used as a prior in the detection method in order to simulate the full meaning of prediction.

In Haw et al. (2006) the authors implemented a user-specific template matching structure as part of a method to detect movement planning. Here, the focus was more to detect the movement rather than to predict it. A BP waveform recorded via only one electrode was used to build the template. The results showed detection of movement onset with an average accuracy of 70% and a low false positive rate.

Authors in Bai et al. (2011) focused on prediction. In their experiment, subjects were asked to perform three sessions in each of which the only task was to perform a right-hand wrist extension movement whenever they wanted, without any cue (self-paced movement). EEG data from the first two sessions were used for training and applied to the third session to test the predictions. The work was based on detection of the ERD or power decrease in alpha and beta bands in single trials. The study reported an average true positive rate of 75 ± 10% of total predictions, about 0.62 s on average before the movement onset.

In Niazi et al. (2011), the authors detected the onset of the movement as early as 187 ms before the movement with an average accuracy of 82.5%. The subjects were asked to perform a self-paced ankle dorsiflexion. Each subject performed 5 runs: 2 were used for training and the rest for the testing phase. The detection method was based on movement-related cortical potentials such as BP; a template was extracted from the training data sets using a spatial filter. The detection decision was based on the correlation computed between spatially filtered channels of the test data sets and the template in a sliding window.

In Lew et al. (2012), the authors showed that it is possible to predict the movement 500 ms prior to its occurrence. Here, the subjects were asked to move their hands at least 2 s after hearing an auditory cue. In the training phase, the signal preceding the movement by 500 ms was used in comparison with 500 ms before the auditory cue. For the testing phase, a shifting window was implemented. Their results showed maximum average true-positive rate of 81% peaking 140 ms before movement onset across subjects.

3.2. Predicting the Direction of the Movement

In studies done by some researchers (Lakany and Conway, 2007; Hammon et al., 2008; Wang and Makeig, 2009) it was shown that it is possible to determine the direction of the moving limb prior to movement onset. In Lakany and Conway (2007), the authors analyzed 3 s of data before movement onset. The subjects were asked to move a manipulandum placed in their hands toward the direction indicated by a cue on the screen. For EEG analysis, spatio-temporal features were extracted via continuous wavelet transform (CWT). Then, they performed a wrapper method based on support vector machines (SVM) for feature selection and classification and reported an average accuracy of 81.5% during testing for two different directions.

In the study reported in Hammon et al. (2008), the subjects were asked to reach and touch the corner of the screen indicated by a directional cue. The delay between the direction cue and go cue was randomized between 750 and 1500 ms. Only the EEG data of the first 500 ms of the delay period were analyzed by three classifiers: a four-class classifier for reaching the target, and two binary classifiers of left vs. right reaches and top vs. bottom reaches. The results of the binary classifiers showed that distinguishing left from right targets is more effective than discriminating top from bottom targets.

In a follow-up work in Wang and Makeig (2009) the authors showed that EEG signals obtained from the posterior parietal cortex before the movement onset carry information about the direction of the intended movement. A delayed movement protocol was designed where the movement consisted of tracking the target with gaze, reaching for it with one hand without eye movement, or doing both actions. It had to be performed 700 ms after the subject was made aware of the task direction by a cue; this was the period which was then used for data analysis. For EEG analysis, they used independent component analysis (ICA) performed by the Extended Infomax algorithm. Two lateralized temporal-parietal components were identified in each subject's EEG data time locked to the onset of the movement cue. For a better understanding of direction coding, the parietal ICs were back-projected onto the scalp and visualized. The result produced a clear contralateral negativity and positivity with respect to the intended movement direction (see Figure 1). Moreover, using SVM, the authors obtained an average accuracy of 80.25% for single-trial classification of right vs. left.

FIGURE 1

Figure 1. Scalp maps and ERP waveforms of the back-projected parietal ICs for one subject in three conditions (left, center, and right) 320 ms after the direction cue. The ERP waveforms were from the two lateral parietal electrodes with the highest amplitude projection of cortex. The picture is taken from Wang and Makeig (2009).

3.3. Predicting the Type of the Movement

In this section, we consider the studies which try to determine the type of the movement or the body part that will move in a short future. In the previous section the movement was carried out by one limb and the direction of the movement was in question. However, some studies go beyond that to see whether it is possible to predict the future moving limb. One of the earliest work in this area is the work reported in Blankertz et al. (2001). In this work, using the BP features, authors managed to discriminate between left and right-hand finger pre-movement in keyboard typing on average 100–230 ms before the key was pressed. The actual time at which the key was pressed was determined by electromyography (EMG). The subject task was to press the computer keyboards with the index or little fingers of both hands in a self-paced manner, resulting in an average speed of 1 key every 2.1 s. Based on this research, BCI competition 2003, data set IV, self-paced tapping, was released (Blankertz, nd). Using one subject executing the same task, 416 epochs of 500 ms EEG were provided, each ending 130 ms before an actual key press. The epochs were randomly shuffled and split into 316 labeled epochs for the training set and 100 unlabeled epochs for the test set. Obviously, the goal of the competition was to submit the estimated labels for the test set with the smallest number of misclassifications. Needless to say, this data set attracted a lot of researchers to find a solution for the problem of predicting the next moving limb. A total of 15 groups submitted their results for this data set, 4 of which had a performance close to chance level (the error rate was higher than 43%) (Blankertz et al., 2004). The best submission was by Zhang and colleagues (Zhang et al., 2004) with an error rate of 16%. The use of this data set did not stop after the end of the competition and the release of the results; since this dataset is still available for download, it has become a benchmark and has been used by other research groups such as (Congedo et al., 2006).

In Blankertz et al. (2003), the authors used the concept of bit transfer to evaluate the speed and accuracy of their method in discriminating between left and right-hand figure movements in keyboard typing. The subjects were asked to press one of the four keys, using the fingers of the right or left hand, in a self-chosen order and timing. For data analysis, after preprocessing the data, a classifier based on Fisher's Discriminant was applied to the mean and covariance matrices calculated from the training data. Their results showed that not only could the discrimination between different limb movements be detected as early as 120 ms before their onset but that this was also achieved in motor sequences as fast as 2 taps per second.

In another study reported in Morash et al. (2008), the authors used the EEG signals preceding movement and motor imagery to predict which of the movements/motor imageries of different body parts (right hand, left hand, tongue, and right foot) was about to occur. The subjects were informed about the type of the movement by a cue and performed it at least 2 s after being informed by another “Go” cue. The time window of 1.5 s before the “Go” cue was used in the analysis of data which correspond to ERD/ERS and CNV oscillation. The results showed that the preceding movement and motor imagery ERD/ERS can be used to predict which of the four movements/imageries is about to occur. Prediction accuracy depended on signal quality. However, the highest average testing accuracies reported for the two-movement/imagery categories were related to right-foot and left-hand responses. The study also tried to identify CNV and ERD/ERSs on motor areas in single trials and concluded that, compared to CNV, the ERD/ERS is the most specific pre-movement/pre-motor imagery signal with respect to the movement/imagery that is about to be performed.

4. Discussion

In the studies above there is a close link between one or more brain rhythms and ERPs and the preprocessing step. In particular, the choice of the filter and of the placement of the electrodes depends on the brain features on which the method is based. For instance, (Blankertz et al., 2003) and (Lew et al., 2012), which were based on BP, used a bandpass filter with upper cutoff frequency of less than 4 Hz and on electrodes attached to the motor cortex. Consequently, more studies to further investigate the pre-movement period can improve the BCIs and, thus, the prediction of the next motor task.

This article is not a completely exhaustive review of the capability of EEG in predicting the next motor task. It is mainly aimed at providing examples of the progress in the field as well as stimulating ideas for new research proposals.

5. Conclusion and Future Work

In conclusion, the EEG data gathered before the forthcoming movement which corresponds to motor preparation and planning period of the brain show significant prediction potentials. The evidence from both Neuroscience and Engineering research support this hypothesis, even though this field has not been vastly explored. Further studies in both fields in this short pre-movement time can extend exploration of the frontiers of motor preparation and decrease the response time of BCIs. To know beforehand whether the subject is willing to move, in what direction, or using which limb can make it possible to determine the next course of actions in rehabilitation procedures more effectively. It can also decrease the response time of BCIs and let them appear more natural to their users. In some applications, such as driving, it may even have a more crucial role. Study reported in Haufe et al. (2011) showed that it is possible to predict an upcoming emergency brake up to 130 ms earlier than pedal response. At driving speed of $100\frac{\text{km}}{\text{h}}$, this time reduces the braking distance by 3.66 m, which can be very effective in terms of survival. They also showed that levels of predictive accuracy using EEG worked faster than EMG, suggesting the superiority of using EEG in such a context.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Funding

Pouya Ahmadian is funded by the European Commission (Marie Curie ITN MIBISOC, FP7 PEOPLE-ITN-2008, GA n. 238819).

References

Ahmadian, P., Sanei, S., Ascari, L., Gonzalez-Villanueva, L., and Umilta, M. A. (in press). Constrained blind source extraction of readiness potentials from EEG. IEEE Trans. Neural Syst. Rehabil. Eng.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ahmadian, P., Sanei, S., Mussi, L., Ascari, L., and Umiltà, M. A. (2012). “Automatic detection of readiness potential,” in Proceedings of the 9th IASTED International Conference on Biomedical Engineering (BioMed 2012) (Innsbruck).

Bai, O., Rathi, V., Lin, P., Huang, D., Battapady, H., Fei, D.-Y., et al. (2011). Prediction of human voluntary movement before it occurs. Clin. Neurophysiol. 122, 364–372.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ball, T., Kern, M., Mutschler, I., Aertsen, A., and Schulze-Bonhage, A. (2009). Signal quality of simultaneously recorded invasive and non-invasive EEG. Neuroimage 46, 708–716.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Blakemore, S. J., Goodbody, S. J., and Wolpert, D. M. (1998). Predicting the consequences of our own actions: the role of sensorimotor context estimation. J. Neurosci. 18, 7511–7518.

Pubmed Abstract | Pubmed Full Text

Blankertz, B. (n.d.). Data set self-paced 1s. BCI Competition 2003. Retrieved 5 March 2013. Available online at: http://www.bbci.de/competition/ii/berlin_desc.html

Blankertz, B., Curio, G., and Mller, K.-R. (2001). “Classifying single trial EEG: towards brain computer interfacing,” in Neural Information Processing Systems: Natural and Synthetic (NIPS 2001), eds T. G. Dietterich, S. Becker, and Z. Ghahramani (Vancouver, BC: MIT Press), 157–164.

Blankertz, B., Dornhege, G., Schäfer, C., Krepki, R., Kohlmorgen, J., Müller, K.-R., et al. (2003). Boosting bit rates and error detection for the classification of fast-paced motor commands based on single-trial EEG analysis. IEEE Trans. Neural Syst. Rehabil. Eng. 11, 100–104.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Blankertz, B., Mller, K.-R., Curio, G., Vaughan, T. M., Schalk, G., Wolpaw, J. R., et al. (2004). The BCI competition 2003: progress and perspectives in detection and discrimination of EEG single trials. IEEE Trans. Biomed. Eng. 51, 1044–1051.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Brunia, C. H. M. (2003). “CNV and SPN: indices of anticipatory behavior,” in The Bereitschaftspotential: Movement-Related Cortical Potentials, eds M. Jahanshahi and M. Hallett (New York, NY: Kluwer Academic/Plenum), 283–320.

Citi, L., Poli, R., Cinel, C., and Sepulveda, F. (2008). P300-based BCI mouse with genetically-optimized analogue control. IEEE Trans. Neural Syst. Rehabil. Eng. 16, 51–61.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Congedo, M., Lotte, F., and Lecuyer, A. (2006). Classification of movement intention by spatially filtered electromagnetic inverse solutions. Phys. Med. Biol. 51, 1971–1989.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Crammond, D. J., and Kalaska, J. F. (2000). Prior information in motor and premotor cortex: activity during the delay period and effect on pre-movement activity. J. Neurophysiol. 84, 1645–1655.

Pubmed Abstract | Pubmed Full Text

Engel, A. K., Moll, C. K. E., Fried, I., and Ojemann, G. A. (2005). Invasive recordings from the human brain: clinical insights and beyond. Nat. Rev. Neurosci. 6, 35–47.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hammon, P. S., Makeig, S., Poizner, H., Todorov, E., and de Sa, V. R. (2008). Predicting reaching targets from human EEG. IEEE Signal Process. Mag. 25, 69–77.

Haufe, S., Treder, M. S., Gugler, M. F., Sagebaum, M., Curio, G., and Blankertz, B. (2011). EEG potentials predict upcoming emergency brakings during simulated driving. J. Neural Eng. 8:056001. doi: 10.1088/1741-2560/8/5/056001

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Haw, C., Lowne, D., and Roberts, S. (2006). “User specific template matching for event detection using single channel,” in Graz 2006 Conference on Brain Computer Interface (Graz).

Jahanshahi, M., and Hallett, M. (2003). “The bereitschaftspotential: What does it measure and where does it come from,” in The Bereitschaftspotential: Movement-Related Cortical Potentials, eds M. Jahanshahi and M. Hallett. (New York, NY: Kluwer Academic/Plenum), 1–17.

Kilner, J. M., Vargas, C., Duval, S., Blakemore, S.-J., and Sirigu, A. (2004). Motor activation prior to observation of a predicted movement. Nat. Neurosci. 7, 1299–1301.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Lakany, H., and Conway, B. (2007). Understanding intention of movement from electroencephalograms. Expert Syst. 24, 295–304.

Lew, E., Chavarriaga, R., Silvoni, S., and Millan, J. R. (2012). Detection of self-paced reaching movement intention from EEG signals. Front. Neuroeng. 5:13. doi: 10.3389/fneng.2012.00013

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Morash, V., Bai, O., Furlani, S., Lin, P., and Hallettd, M. (2008). Classifying EEG signals preceding right hand, left hand, tongue, and right foot movements and motor imageries. Clin. Neurophysiol. 119, 2570–2578.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Müller, K.-R., and Blankertz, B. (2006). Towards non-invasive brain-computer interfaces. IEEE Signal Process. Mag. 23, 125–128.

Niazi, I. K., Jiang, N., Tiberghien, O., Nielsen, J. F., Dremstrup, K., and Farina, D. (2011). Detection of movement intention from single-trial movement-related cortical potentials. J. Neural Eng. 8:066009. doi: 10.1088/1741-2560/8/6/066009

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Pfurtscheller, G., and Neuper, C. (2003). “Movement and ERD/ERS,” in The Bereitschaftspotential: Movement-Related Cortical Potentials, eds M. Jahanshahi and M. Hallett (New York, NY: Kluwer Academic/Plenum), 283–320.

Pineda, J. A., Allison, B. Z., and Vankov, A. (2000). The effects of self-movement, observation, and imagination on mu rhythms and readiness potentials (RP's): toward a brain-computer interface (BCI). IEEE Trans. Rehabil. Eng. 8, 219–222.

Pubmed Abstract | Pubmed Full Text

Rohrbaugh, J. W., Syndulko, K., and Lindsley, D. B. (1976). Brain wave components of the contingent negative variation in humans. Science 191, 1055–1057.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Shibasaki, H., and Hallett, M. (2006). What is the bereitschaftspotential? Clin. Neurophysiol. 117, 2341–2356.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Toni, I., and Passingham, R. E. (2003). “Movement preparation: neuroimaging studies,” in The Bereitschaftspotential: Movement-Related Cortical Potentials, eds M. Jahanshahi and M. Hallett (New York, NY: Kluwer Academic/Plenum), 1–17.

Wang, Y., and Makeig, S. (2009). “Predicting intended movement direction using EEG from human posterior parietal cortex,” in Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience (HCII 2009), eds D. D. Schmorrow, I. V. Estabrooke, and M. Grootjen (Berlin: Springer), 437–446.

Wolpaw, J. R., and McFarland, D. J. (2004). Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. Proc. Natl. Acad. Sci. U.S.A. 101, 17849–17854.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Wolpert, D. M., and Flanagan, J. R. (2001). Motor prediction. Curr. Biol. 11, R729–R732.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Zhang, Z., Wang, Y., Li, Y., and Gao, X. (2004). BCI competiton 2003 data set IV. IEEE Trans. Biomed. Eng. 51, 1081–1086.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text