Can electrocorticography (ECoG) support robust and powerful brain–computer interfaces?
- Brain–Computer Interface R and D Program, Wadsworth Center, New York State Department of Health, Albany, NY, USA
A commentary on
Long-term asynchronous decoding of arm motion using electrocorticographic signals in monkey.
by Zenas C. Chao, Yasuo Nagasaka and Naotaka Fujii (2010). Front. Neuroeng. 3:3.
Brain–computer interfaces (BCIs) use brain signals to communicate a user’s intent (Wolpaw et al., 2002 ). Because these systems directly translate brain activity into action, without depending on peripheral nerves and muscles, they can be used by people with severe motor disabilities. Successful translation of BCI technology from the many recent laboratory demonstrations into widespread and valuable clinical applications is currently substantially impeded by the problems of traditional non-invasive or intracortical signal acquisition technologies.
Non-invasive BCIs use electroencephalographic (EEG) activity recorded from the scalp (Birbaumer et al., 1999 ; Pfurtscheller et al., 2000 ; Wolpaw et al., 2002 ; Millan Jdel et al., 2004 ; Wolpaw and McFarland, 2004 ; Blankertz et al., 2007 ; McFarland et al., 2008 ). While non-invasive BCIs can support much higher performance than previously assumed (Wolpaw and McFarland, 2004 ; Müller and Blankertz, 2006 ; McFarland et al., 2008 , 2010 ), such performance typically requires extensive user training and can also be variable. Intracortical BCIs use action potential firing rates or local field potential activity recorded from individual or small populations of neurons within the brain (Serruya et al., 2002 ; Taylor et al., 2002 ; Carmena et al., 2003 ; Shenoy et al., 2003 ; Santhanam et al., 2006 ; Donoghue et al., 2007 ; Velliste et al., 2008 ). Signals recorded within cortex may encode more information and might support BCI systems that require less training than EEG-based systems. However, clinical implementations are impeded mainly by the problems in achieving and maintaining stable long-term recordings from individual neurons and by the high variability in neuronal behavior (Shain et al., 2003 ; Donoghue et al., 2004 ). Despite encouraging evidence that BCI technologies can serve useful functions for severely disabled individuals (Kübler et al., 2005 ; Hochberg et al., 2006 ; Nijboer et al., 2008 ), these issues of non-invasive and action potential-based techniques in acquiring and maintaining robust recordings and BCI control remain crucial obstacles that currently impede widespread clinical use in humans.
In consequence, a critical challenge in designing BCI systems for widespread clinical application is the identification and optimization of a BCI method that combines good performance with robustness. In the current absence of robust techniques to extract high-fidelity signals from EEG or to record activity from within the brain over prolonged periods, the use of electrocorticographic (ECoG) activity recorded from the cortical surface could be a powerful and practical alternative. ECoG has higher spatial resolution than EEG (i.e., tenths of millimeters vs. centimeters, Freeman et al., 2000 ; Slutzky et al., 2010 ), broader bandwidth (i.e., 0–500 Hz, Staba et al., 2002 , vs. 0–40 Hz), higher amplitude (i.e., 50–100 μV maximum vs. 10–20 μV), much greater signal-to-noise ratio (Ball et al., 2009 ), and far less vulnerability to artifacts such as EMG (Freeman et al., 2003 ). In addition to these superior general characteristics, a number of human studies (Schalk et al., 2007 ; Ball et al., 2008 ; Pistohl et al., 2008 ; Sanchez et al., 2008 ; Waldert et al., 2008 ; Gunduz et al., 2009 ; Kubanek et al., 2009 ) have recently shown that ECoG can provide information about movements that far exceeds that provided by EEG. Other studies (Leuthardt et al., 2004 ; Wilson et al., 2006 ; Schalk et al., 2008 ) demonstrated that this information in ECoG can be used to provide one- or two-dimensional BCI control with little training. In summary, these (predominantly human) studies have produced great excitement for ECoG recordings, because they demonstrate that ECoG can provide information about movements and other aspects of behavior that is in aspects relevant to BCI performance on par with, and can even exceed, the information provided by single-neuron recordings.
While these studies demonstrated ECoG’s impressive capabilities, and while several other studies suggested that ECoG may have long-term robustness (Loeb et al., 1977 ; Bullara et al., 1979 ; Yuen et al., 1987 ; Pilcher and Rusyniak, 1993 ; Margalit et al., 2003 ), concrete quantitative evidence for ECoG’s long-term stability has been missing. The recent study by Chao et al. (2010) provided this critical piece of information. This study evaluated ECoG-based decoding of hand position and arm joint angles during reaching movements. Data were recorded in two monkeys over a period of several months. This study confirmed and extended the previous finding that local field potentials recorded from the surface of the brain can be used to accurately decode different kinematic parameters of limb movements. More importantly, it also provided two other pieces of information. First, the authors showed that decoding performance does not significantly degrade with time, which suggests that the signal-to-noise ratio of ECoG recordings is robust over many months. Second, the authors also showed that there is no negative correlation between decoding performance and the time between model generation and model testing, which suggests that the neural representations that encode kinematic parameters of reaching movements are stable across the months of study.
In conclusion, the study by Chao and colleagues is of critical importance to the whole field of BCI research. It justifies previous excitement for ECoG recordings, and more forcefully suggests a realistic trajectory toward robust, powerful, and widespread clinical applications of BCI technology.
References
Ball, T., Demandt, E., Mutschler, I., Neitzel, E., Mehring, C., Vogt, K., Aertsen, A., and Schulze-Bonhage, A. (2008). Movement related activity in the high gamma range of the human EEG. Neuroimage 41, 302–310.
Ball, T., Kern, M., Mutschler, I., Aertsen, A., and Schulze-Bonhage, A. (2009). Signal quality of simultaneously recorded invasive and non-invasive EEG. Neuroimage 46, 708–716.
Birbaumer, N., Ghanayim, N., Hinterberger, T., Iversen, I., Kotchoubey, B., Kubler, A., Perelmouter, J., Taub, E., and Flor, H. (1999). A spelling device for the paralysed. Nature 398, 297–298.
Blankertz, B., Dornhege, G., Krauledat, M., Muller, K. R., and Curio, G. (2007). The non-invasive Berlin brain–computer interface: fast acquisition of effective performance in untrained subjects. Neuroimage 37, 539–550.
Bullara, L. A., Agnew, W. F., Yuen, T. G., Jacques, S., and Pudenz, R. H. (1979). Evaluation of electrode array material for neural prostheses. Neurosurgery 5, 681–686.
Carmena, J. M., Lebedev, M. A., Crist, R. A., O’Doherty, J. A., Santucci, D. M., Dimitrov, D. F., Patil, P. G., Henriquez, C. S., and Nicolelis, M. A. L. (2003). Learning to control a brain–machine interface for reaching and grasping by primates. PLoS Biol. 1, 1–16. doi: 10.1371/journal.pbio.0000042.
Chao, Z. C., Nagasaka, Y., and Fujii, N. (2010). Long-term asynchronous decoding of arm motion using electrocorticographic signals in monkeys. Front. Neuroengineering 3:3. doi: 10.3389/fneng.2010.00003.
Donoghue, J. P., Nurmikko, A., Black, M., and Hochberg, L. R. (2007). Assistive technology and robotic control using motor cortex ensemble-based neural interface systems in humans with tetraplegia. J. Physiol. (Lond.) 579, 603–611.
Donoghue, J. P., Nurmikko, A., Friehs, G., and Black, M. (2004). Development of neuromotor prostheses for humans. Suppl. Clin. Neurophysiol. 57, 592–606.
Freeman, W. J., Holmes, M. D., Burke, B. C., and Vanhatalo, S. (2003). Spatial spectra of scalp EEG and EMG from awake humans. Clin. Neurophysiol. 114, 1053–1068.
Freeman, W. J., Rogers, L. J., Holmes, M. D., and Silbergeld, D. L. (2000). Spatial spectral analysis of human electrocorticograms including the alpha and gamma bands. J. Neurosci. Methods 95, 111–121.
Gunduz, A., Sanchez, J. C., Carney, P. R., and Principe, J. C. (2009). Mapping broadband electrocorticographic recordings to two-dimensional hand trajectories in humans motor control features. Neural. Netw. 22, 1257–1270.
Hochberg, L. R., Serruya, M. D., Friehs, G. M., Mukand, J. A., Saleh, M., Caplan, A. H., Branner, A., Chen, D., Penn, R. D., and Donoghue, J. P. (2006). Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature 442, 164–171.
Kubanek, J., Miller, K. J., Ojemann, J. G., Wolpaw, J. R., and Schalk, G. (2009). Decoding flexion of individual fingers using electrocorticographic signals in humans. J. Neural Eng. 6, 66001.
Kübler, A., Nijboer, F., Mellinger, J., Vaughan, T. M., Pawelzik, H., Schalk, G., McFarland, D. J., Birbaumer, N., and Wolpaw, J. R. (2005). Patients with ALS can use sensorimotor rhythms to operate a brain–computer interface. Neurology 64, 1775–1777.
Leuthardt, E. C., Schalk, G., Wolpaw, J. R., Ojemann, J. G., and Moran, D. W. (2004). A brain–computer interface using electrocorticographic signals in humans. J. Neural Eng. 1, 63–71.
Loeb, G. E., Walker, A. E., Uematsu, S., and Konigsmark, B. W. (1977). Histological reaction to various conductive and dielectric films chronically implanted in the subdural space. J. Biomed. Mater. Res. 11, 195–210.
Margalit, E., Weiland, J., Clatterbuck, R., Fujii, G., Maia, M., Tameesh, M., Torres, G., D’Anna, S., Desai, S., Piyathaisere, D., Olivi, A., de Juan, E. J., and Humayun, M. (2003). Visual and electrical evoked response recorded from subdural electrodes implanted above the visual cortex in normal dogs under two methods of anesthesia. J. Neurosci. Methods 123, 129–137.
McFarland, D. J., Krusienski, D. J., Sarnacki, W. A., and Wolpaw, J. R. (2008). Emulation of computer mouse control with a noninvasive brain–computer interface. J. Neural Eng. 5, 101–110.
McFarland, D. J., Sarnacki, W. A., and Wolpaw, J. R. (2010). Electroencephalographic (EEG) control of three-dimensional movement. J. Neural Eng. 7, 036007.
Millan, J. R., Renkens, F., Mourino, J., and Gerstner, W. (2004). Noninvasive brain-actuated control of a mobile robot by human EEG. IEEE Trans. Biomed. Eng. 51, 1026–1033.
Müller, K. R., and Blankertz, B. (2006). Toward noninvasive brain–computer interfaces. IEEE Signal Process. Mag. 23, 126–128.
Nijboer, F., Sellers, E. W., Mellinger, J., Jordan, M. A., Matuz, T., Furdea, A., Halder, S., Mochty, U., Krusienski, D. J., Vaughan, T. M., Wolpaw, J. R., Birbaumer, N., and Kubler, A. (2008). A P300-based brain–computer interface for people with amyotrophic lateral sclerosis. Clin. Neurophysiol. 119, 1909–1916.
Pfurtscheller, G., Guger, C., Muller, G., Krausz, G., and Neuper, C. (2000). Brain oscillations control hand orthosis in a tetraplegic. Neurosci. Lett. 292, 211–214.
Pilcher, W., and Rusyniak, W. (1993). Complications of epilepsy surgery. Neurosurg. Clin. N. Am. 4, 311–325.
Pistohl, T., Ball, T., Schulze-Bonhage, A., Aertsen, A., and Mehring, C. (2008). Prediction of arm movement trajectories from ECoG-recordings in humans. J. Neurosci. Methods 167, 105–114.
Sanchez, J. C., Gunduz, A., Carney, P. R., and Principe, J. C. (2008). Extraction and localization of mesoscopic motor control signals for human ECoG neuroprosthetics. J. Neurosci. Methods 167, 63–81.
Santhanam, G., Ryu, S. I., Yu, B. M., Afshar, A., and Shenoy, K. V. (2006). A high-performance brain–computer interface. Nature 442, 195–198.
Schalk, G., Kubanek, J., Miller, K. J., Anderson, N. R., Leuthardt, E. C., Ojemann, J. G., Limbrick, D., Moran, D., Gerhardt, L. A., and Wolpaw, J. R. (2007). Decoding two-dimensional movement trajectories using electrocorticographic signals in humans. J. Neural Eng. 4, 264–275.
Schalk, G., Miller, K. J., Anderson, N. R., Wilson, J. A., Smyth, M. D., Ojemann, J. G., Moran, D. W., Wolpaw, J. R., and Leuthardt, E. C. (2008). Two-dimensional movement control using electrocorticographic signals in humans. J. Neural Eng. 5, 75–84.
Serruya, M. D., Hatsopoulos, N. G., Paninski, L., Fellows, M. R., and Donoghue, J. P. (2002). Instant neural control of a movement signal. Nature 416, 141–142.
Shain, W., Spataro, L., Dilgen, J., Haverstick, K., Isaacson, M., Saltzman, M., and Turner, J. N. (2003). Controlling cellular reactive responses around neural prosthetic devices using peripheral and local intervention strategies. IEEE Trans. Neural Syst. Rehabil. Eng. 11, 186–188.
Shenoy, K. V., Meeker, D., Cao, S., Kureshi, S. A., Pesaran, B., Buneo, C. A., Batista, A. P., Mitra, P. P., Burdick, J. W., and Andersen, R. A. (2003). Neural prosthetic control signals from plan activity. Neuroreport 14, 591–596.
Slutzky, M. W., Jordan, L. R., Krieg, T., Chen, M., Mogul, D. J., and Miller, L. E. (2010). Optimal spacing of surface electrode arrays for brain-machine interface applications. J. Neural Eng. 7, 26004.
Staba, R. J., Wilson, C. L., Bragin, A., Fried, I., and Engel, J. (2002). Quantitative analysis of high-frequency oscillations (80–500 Hz) recorded in human epileptic hippocampus and entorhinal cortex. J. Neurophysiol. 88, 1743–1752.
Taylor, D. M., Tillery, S. I., and Schwartz, A. B. (2002). Direct cortical control of 3D neuroprosthetic devices. Science 296, 1829–1832.
Velliste, M., Perel, S., Spalding, M. C., Whitford, A. S., and Schwartz, A. B. (2008). Cortical control of a prosthetic arm for self-feeding. Nature 453, 1098–1101.
Waldert, S., Preissl, H., Demandt, E., Braun, C., Birbaumer, N., Aertsen, A., and Mehring, C. (2008). Hand movement direction decoded from MEG and EEG. J. Neurosci. 28, 1000–1008.
Wilson, J. A., Felton, E. A., Garell, P. C., Schalk, G., and Williams, J. C. (2006). ECoG factors underlying multimodal control of a brain–computer interface. IEEE Trans. Neural Syst. Rehabil. Eng. 14, 246–250.
Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., and Vaughan, T. M. (2002). Brain–computer interfaces for communication and control. Clin. Neurophysiol. 113, 767–791.
Wolpaw, J. R., and McFarland, D. J. (2004). Control of a two-dimensional movement signal by a noninvasive brain–computer interface in humans. Proc. Natl. Acad. Sci. U.S.A. 101, 17849–17854.
Citation: Schalk G (2010) Can electrocorticography (ECoG) support robust and powerful brain–computer interfaces? Front. Neuroeng. 3:9. doi: 10.3389/fneng.2010.00009
Received: 10 May 2010;
Accepted: 27 May 2010;
Published online: 17 June 2010
Copyright: © 2010 Schalk. This is an open-access article subject to an exclusive license agreement between the authors and the Frontiers Research Foundation, which permits unrestricted use, distribution, and reproduction in any medium, provided the original authors and source are credited.
*Correspondence: schalk@wadsworth.org