The Frontiers in Neuroscience journal series is the 1st most cited in Neurosciences

Original Research ARTICLE

Front. Neuroeng., 08 August 2012 | https://doi.org/10.3389/fneng.2012.00015

Decoding onset and direction of movements using Electrocorticographic (ECoG) signals in humans

  • 1Department of ECSE, Rensselaer Polytechnic Institute, Troy, NY, USA
  • 2J Crayton Pruitt Family Department of Biomed Engineering, University of Florida, Gainesville, FL, USA
  • 3BCI R&D Program, Wadsworth Center, New York State Department of Health, Albany, NY, USA
  • 4Department of Neurology, Albany Medical College, Albany, NY, USA

Communication of intent usually requires motor function. This requirement can be limiting when a person is engaged in a task, or prohibitive for some people suffering from neuromuscular disorders. Determining a person's intent, e.g., where and when to move, from brain signals rather than from muscles would have important applications in clinical or other domains. For example, detection of the onset and direction of intended movements may provide the basis for restoration of simple grasping function in people with chronic stroke, or could be used to optimize a user's interaction with the surrounding environment. Detecting the onset and direction of actual movements are a first step in this direction. In this study, we demonstrate that we can detect the onset of intended movements and their direction using electrocorticographic (ECoG) signals recorded from the surface of the cortex in humans. We also demonstrate in a simulation that the information encoded in ECoG about these movements may improve performance in a targeting task. In summary, the results in this paper suggest that detection of intended movement is possible, and may serve useful functions.

1. Introduction

Brain-computer interfaces (BCIs) aim to translate a person's intentions into meaningful computer commands using brain activity alone (Wolpaw et al., 2002; Mak and Wolpaw, 2009). In particular, determining when and where a person intends to move would have important clinical applications for those suffering from neuromuscular disorders (Sejnowski et al., 2007; Tan and Nijholt, 2010). For example, a BCI that detects intended movement onset in absence of actual movements could restore grasp function in people with chronic stroke (Buch et al., 2008; Daly and Wolpaw, 2008; Wisneski et al., 2008; Muralidharan et al., 2011; Yanagisawa et al., 2011). Also, a BCI that predicts intended movement onset prior to actual movements would have many practical applications in everyday life. For example, it may support faster braking during vehicle operation (Haufe et al., 2011) or more rapid targeting in military applications (Gunduz and Schalk, 2011).

The first step in this direction is to establish whether it is possible to detect the onset of actual movements from brain signals. Several previous studies have shown that intracortical activities recorded in primates over the premotor or parietal cortices are related to the onset of movements (Achtman et al., 2007; Lebedev et al., 2008; Hwang and Andersen, 2009; Hasan and Gan, 2011; Mirabella et al., 2011), but access to intracortical activity in humans has been scarce (e.g., Hochberg et al., 2006; Simeral et al., 2011). Other studies have investigated movement onset using electroencephalographic (EEG) signals in humans (Mason and Birch, 2000; Millan and Mouriño, 2003; Borisoff et al., 2004; Leeb et al., 2007; Bai et al., 2008; Hasan and Gan, 2011; Muralidharan et al., 2011), but accurately detecting the corresponding EEG signatures in single trials has proven difficult. Electrocorticographic (ECoG) signals are recorded directly from the surface of the cortex, and thus have a higher signal-to-noise ratio compared to EEG (Ball et al., 2009). They also readily support detection of certain physiological phenomena, such as high gamma activity (>70 Hz), that is largely inconspicuous on the scalp. The ability to detect high gamma activity is an important advantage, since many ECoG studies (e.g., Miller et al., 2007, 2009; Kubánek et al., 2009; Chao et al., 2010) demonstrated that spatially focused high gamma activity correlates closely with specific aspects of motor functions. Yet, no previous study comprehensively studied the possibility that movement onset can be detected using ECoG signals.

The second step in this direction is to determine whether ECoG also holds information about movement direction prior to the actual movement. Several studies (Schalk et al., 2007; Pistohl et al., 2008; Gunduz et al., 2009) showed that ECoG signals collected during movements hold information about two-dimensional trajectories of hand movements, and Leuthardt et al. (2004) and Schalk et al. (2008) demonstrated one- and two-dimensional real-time control of a computer cursor using ECoG, respectively. However, there has only been scarce evidence that brain signals recorded in humans give information about movement direction prior to the movement (Leuthardt et al. (2004) using ECoG, Wang et al. (2010) using MEG), and that this information may be useful.

In this paper, we investigate whether ECoG holds information about the onset and direction of hand movements in a center-out task. Specifically, we use a support vector machine (SVM) classifier to determine, at each step in time, the probability that the subject initiated a hand movement at that particular time. We also characterize the ECoG features that hold the most information about movement onset. We then use a novel implementation of a time-varying dynamic Bayesian network (TVDBN), which was designed to take advantage of the spatio-temporal dynamics of ECoG features, to determine the direction of the intended movement using ECoG signals prior to the movement. Finally, we simulate a targeting application in which brain signals prior to the movement are combined with the actual movement signals. In this simulation, the time-to-target reduces by up to 150 ms when we use the directional information captured in ECoG signals. Overall, our results contribute to our understanding of the neural representation of intended movements and suggest that integrating information from brain signals and motor execution may eventually lead to systems that can improve a user's performance.

2. Materials

2.1 Human Subjects

Five subjects participated in this study. The subjects were patients with intractable epilepsy who underwent temporary implantation of subdural electrode arrays for the localization of seizure foci prior to surgical resection. Table 1 summarizes the subjects' clinical profiles. All of the subjects had normal cognitive capacity and were functionally independent. The study was approved by the Institutional Review Board of Albany Medical College as well as by the Human Research Protections Office of the US Army Medical Research and Materiel Command, and the subjects gave informed consent. The implanted electrode grids (Ad-Tech Medical Corp., Racine, WI) consisted of platinum-iridium electrodes that were 4 mm in diameter (2.3 mm exposed) and were configured with an inter-electrode distance of 1 cm. Subject E was implanted with a higher density (6 mm inter-electrode distance) grid with 68 contacts (PMT Corp., Chanhassen, MN) over the temporal lobe. Each subject had postoperative anterior-posterior and lateral radiographs (see Figure A1), as well as computer tomography (CT) scans to verify grid location. The number of implanted electrodes varied between 58 and 120 contacts across subjects (Table 1). We excluded data collected over the occipital strips (in Subjects A and B) from the analyses to minimize the potential impact of visual stimulation on the results.

TABLE 1
www.frontiersin.org

Table 1. Clinical profiles of the subjects that participated in the study.

2.2 Cortical Mapping

We used Curry software (Compumedics, Charlotte, NC) to create subject-specific 3D cortical brain models from high resolution pre-op MRI scans. We co-registered the MRIs with post-op CTs and extracted the stereotactic coordinates of each grid electrode. We identified the cortical areas underneath each electrode using an automated Talairach Atlas (Lancaster et al., 2000) (http://www.talairach.org/daemon.html) for functional mapping. We also projected the electrodes onto the reconstructed brain models (see Figure 1) and generated activation maps using custom Matlab software to delimit the cortical areas involved in prediction of movement onset and direction.

FIGURE 1
www.frontiersin.org

Figure 1. Subject-specific brain models and projected electrode locations for Subjects A–E. The brain template on the bottom right, indicated with a star, depicts the location of the central sulcus, Sylvian fissure, and important Brodmann areas. The areas most relevant to our task are Brodmann areas 6 (premotor), 4 (primary motor), 1–3 (sensory motor), and 7 (posterior parietal cortex).

2.3 Data Collection

We recorded ECoG signals at the bedside using eight 16-channel g.USBamp biosignal acquisition devices (g.tec, Graz, Austria) at a sampling rate of 1200 Hz. Electrode contacts distant from epileptic foci and areas of interest were used for reference and ground. In addition to recording brain activity, we also recorded the subjects' eye gaze using a monitor with a built-in eye tracking system (Tobii Tech., Stockholm, Sweden) positioned 54–60 cm in front of the subjects, and the movements of a joystick. The eye tracker was calibrated to each subject at the beginning of the experimental session using custom software that invoked standard calibration functions provided by Tobii. Data collection from the biosignal and behavioral acquisition devices (g.USBamp, eye tracker, and joystick, respectively), as well as control of the experimental paradigm and stimulus presentation, were accomplished simultaneously using BCI2000 software (Schalk et al., 2004; Schalk and Mellinger, 2010). BCI2000 provides a flexible general-purpose software platform that consists of modules that realize signal acquisition, signal processing, user feedback, and an operating protocol. BCI2000 facilitates the implementation of any BCI or related system, and is used in hundreds of laboratories for this purpose.

2.4 Experimental Paradigm

ECoG signals were collected while the subjects performed an 8-target center-out cursor movement task (Georgopoulos et al., 1982) while fixating their eye gaze at a central fixation cross. Eye gaze fixation was enforced online by BCI2000: a trial was aborted if the subject looked away from the center for more than 5° for more than 500 msec. Each trial started with the presentation of a target in one of eight possible locations. A cursor appeared 1 s later at the center of the screen. The subjects' task was to use their hand contralateral to the implant(s) to control a joystick so as to move the cursor into the target. (Only Subject D used the non-dominant hand.) We positioned the subjects such that the joystick movements were mainly restricted to the wrist (see Figure A2). The subjects were instructed to make exaggerated movements and achieve maximal radial extension of the joystick to hit the targets. Once the target was hit, the next trial started after an inter-trial interval of 1 s. Figure 2 gives a simple illustration of the stages of the task. Trials aborted by the eye tracker, trials in which joystick movement preceded the presentation of the cursor, and trials in which subjects failed to hit the correct target were omitted from further analyses. The total number of remaining valid trials were 394, 584, 258, 398, and 305 for Subjects A through E, respectively.

FIGURE 2
www.frontiersin.org

Figure 2. Illustration of all eight possible target locations (left) and the five experimental stages.

3. Methods

The primary goal of this study was to determine whether ECoG may be used to detect the onset and direction of an intended movement, and whether this information could be useful to reduce the time-to-target in a simulated targeting application. In the following sections, we describe our methods for ECoG feature extraction, movement onset and direction prediction, and the simulation of the targeting application.

3.1 Feature Extraction

We first re-referenced the raw ECoG signals (excluding occipital channels) using a common average reference (CAR) spatial filter to remove spatial noise (Kubánek et al., 2009). For each 100 ms time step and each channel, we converted 300 ms windows (i.e., 200 ms overlap) of ECoG time series into the frequency domain using an autoregressive model of order 25 (Marple, 1986). Using this model, we derived frequency amplitudes between 0 and 200 Hz in 1 Hz bins. Figure 3 shows an example of ECoG activity at different frequencies during the preparation for and execution of the movement task with respect to rest averaged across trials. Spectral amplitudes were divided by an average spectrum of the rest condition. A normalized amplitude of 1 suggests no task-related modulations at a particular time and frequency, whereas a value of 2, for instance, suggests that the spectral amplitude of interest doubled during the task. ECoG features were attained by averaging these frequency amplitudes across three frequency bands: mu (8–12 Hz), beta (18–26 Hz), and high gamma activity (70–170 Hz). In addition to these three spectral features, we also calculated the local motor potential (LMP) (Schalk et al., 2007; Kubánek et al., 2009) by averaging the raw time-domain signal at each channel over each 300 ms time window (also 200 ms overlap). This process resulted in a total of four features from each ECoG channel at each (100 ms) time step.

FIGURE 3
www.frontiersin.org

Figure 3. Exemplary time-frequency plots from Subject B of normalized spectral amplitude (indicated with a linear colorscale) for an electrode over the premotor cortex (indicated by the blue star in the brain model on top) and an electrode over primary motor cortex (yellow hexagon). The horizontal axis gives time (the target was presented at time 0; the cursor was presented at 1 s). The vertical axis gives frequency.

To remove features unrelated to movement onset and direction prediction, we performed feature selection via forward search on five cross-validation folds (i.e., dividing the number of total trials into five and using four-folds for training and one-fold as the novel testing set, five times). The algorithm started with an empty feature set and, at every iteration, added a new feature to the set to generate best classification accuracy across each of the five testing folds. We chose the size of features as 20 for both movement onset prediction and movement direction prediction.

3.2 Prediction of Movement Onset

We downsampled the joystick data to 10 Hz using a moving average filter (300 ms window, 200 ms overlap) to align with the ECoG features. In each trial, we defined the actual movement onset as the time sample when the joystick was pushed beyond one eight of its maximum radial extension from its rest position. For each trial, there was a single onset sample and all time samples from the beginning of the trial up to this onset were labeled as not onset.

We then designed a detector that accumulated 1 s of ECoG features into a first-in-first-out (FIFO) buffer and determined from a full buffer whether the subsequent time step would be the onset of a movement. Each trial started with an empty buffer which was updated with new features every 100 ms. Once the buffer was full, a prediction was made every 100 ms via a weighted SVM (Huang and Du, 2005). We opted for a weighted SVM as it overcomes the classification bias that results from the unbalanced nature of the data (i.e., the class not onset' is much more likely than the class onset) by setting the ratio of penalties to the inverse ratio of the class sizes. We configured the weighted SVM to use a radial basis function as the kernel. The labeled joystick data was divided into 5-folds; four of these folds were used for training the weighted SVM, and one fold was allocated for testing. We repeated this process five times until each fold was used for testing1.

The output of the weighted SVM classifier yielded the probability of an onset as a function of time. Time points were classified as movement onsets if their probability values were greater than an empirically determined threshold of 0.3. We chose the F1-score as our accuracy metric as it is preferable over percent accuracy or error rate for highly unbalanced classes (van Rijsbergen, 1979). F1-score is defined as:

F1=2TP2TP+FP+FN(1)

where TP, FP, and FN are the occurrences of true positive, false positive, and false negative predictions. F1-score is particularly befitting for our onset predictor, as the metric is not influenced by true negative (TN) predictions. Given the imbalance of our two classes, any random classifier is likely to yield high TN rates. F1-scores take on values between 0 and 1, with the latter corresponding to a perfect classifier.

We computed the F1-scores of onset predictions for all subjects. We were also interested in determining the ECoG features (i.e., spectral bands and spatial locations) that were most predictive of movement onset. To do this, we calculated F1-scores separately for each ECoG feature using the methods described above. We ran the 5-fold cross validation 20 times; each time there was a random division of the folds. A series of high F1-scores suggests that the prediction was highly accurate, whereas ratios close to zero indicate a poor predictor. Hence, we tested whether the F1-scores in these 20-folds were significantly different than zero via a t-test. The corresponding p-values represent the significance of the F1-scores, and thus the accuracy of the classifier. We converted these p-values into indices of confidence [(i.e., −log(p))], and mapped those confidence indices on the cortex models of the individual subjects.

3.3 Prediction of Direction of Intended Movement

We predicted movement direction using all ECoG features from 1s prior to the actual movement onset (i.e., the same window used to predict the movement onset). Routinely in such multivariate prediction problems, multichannel time series are re-arranged into a vector to be used as input features to train a classifier, e.g., a neural network or SVM. However, the shortcoming of this approach is that it generally ignores the spatial and temporal structure of the multidimensional time series. Because we felt that such structure was likely important for directional classification, we opted to implement a novel modified time-varying dynamic Bayesian network (MTVDBN) that can capture the spatial and temporal dependency of the ECoG signals across both domains.

A Bayesian network is a probabilistic graphical model that represents a set of variables and their conditional dependencies via directed acyclic graphs, in which nodes represent random variables and edges represent conditional dependencies. A Bayesian network is an established method for modeling dependency structures in complex multivariate systems. An extension of Bayesian networks that models time series is called a dynamic Bayesian network (DBN). However, an important assumption underlying DBNs is that the time series are generated by a stationary process, which generally does not hold for neural signals. TVDBN (Song et al., 2009) introduced non-stationary temporal transitions, but did not describe spatial dependency between variables. In this paper, we extend the TVDBN to the modified TVDBN (MTVDBN) in which the non-stationary temporal and spatial dependencies of ECoG signals are modeled simultaneously. Figure 4 depicts a block diagram of a MTVDBN in which (vertical) arrows within each time slice (i.e., across rows of a column) represent spatial dependencies, while (horizontal) arrows across time slices (i.e., across columns) describe the temporal dependencies. Regular TVDBNs lack the former spatial structure, i.e., the (vertical) arrows within each time slice.

FIGURE 4
www.frontiersin.org

Figure 4. Block diagram of a modified time-varying dynamic Bayesian network (MTVDBN). The shaded nodes are the parents of the node *.

Let Xt = (Xt1, …, XtM)T be a vector representing the ECoG features (mu, beta, high gamma bands, and LMP) from all channels at time t (i.e., M is four times the number of channels for each subject). The joint likelihood of the time sequence of length T can be expressed as:

P(X1, ..., XT)=t=1Tm=1MP(Xmt|Xπmt1, Xπmt),(2)

where Xt−1πm and Xtπm denote the parents of input feature Xtm at time (t−1) and t, respectively (see Figure 4). Since we use 1 s of ECoG features, we are interested in a time sequence of length T = 10 samples. Note that the parent Xt−1πm represents temporal dependencies, where as parent Xtπm represents spatial dependencies. An equivalent form of representing P(Xtm|Xt−1πm, Xtπm) is the following linear model (Duda et al., 2008):

Xmt=amt1Xπmt1+amtXπmt+ ε, where ε~N(0, 1),(3)

at−1m and atm are row vectors of the coefficients of parents of feature Xtm at times (t−1) and t, respectively, and the variable ε is Gaussian noise with mean zero and unit standard deviation. The coefficients a represent the structure of the network. The zero elements of a represent the missing links within the structure, whereas the non-zero elements stand for the dependence strength. The coefficients are learned through maximizing the likelihood of Xtm across all training samples. To prevent overfitting and to encourage sparse structures, we learn a through an ℓ1 penalty. Specifically, the coefficients at−1m and atm are learned by:

minamt1, amt1Nn=1N(Xmn, tamt1Xπmn, t1amtXπmn, t)2     +λ(||amt1||1+||amt||1)(4)

where N is the size of training vector samples, Xn,tm represents the data at the feature m at time t in the nth training vector. Similarly, Xn,tπm is the parent of Xn,tm at time t in the nth training sample. Parameter λ is the penalty coefficient, which controls the sparsity of the structure and it is identified by cross validation. Equation (4) is solved by least angle regression and shrinkage (LARS) (Efron et al., 2004), which has a computational complexity of O(N(2M−1)3) time. Equation (4) was applied to each node to select the potential parents. However, this preprocessing does not automatically result in acyclic graphs at each time slice. A hill-climbing algorithm was then used to greedily construct an acyclic graph in which the edges were restricted by the selected potential parents (Schmidt et al., 2007).

We built a MTVDBN for each direction trained using only the trials toward that direction. For each direction d = {1, 2, …, 8}, the set of parameters atd,m for m = {1, 2, …, M} and t = {1, 2, …, T} were trained. Each MTVDBN thus learned the patterns of the input ECoG features associated with planning a movement in their assigned direction (Equation 3). After these model parameters were trained, given a test sample of ECoG features, the classification was done by:

d*=argmindt=1Tm=1MlogP(Xm*, t|Xπm*, t1, Xπm*, t)     =argmindt=1Tm=1M(Xm*, tad, mt1Xπm*, t1ad, mtXπm*, t)2.(5)

In other words, every time an onset was detected, 1 s of input features (i.e., T = 10 samples) were fed into eight MTVDBNs. Each of these eight MTVDBNs yielded the joint likelihood of X* for one particular direction, d. In each trial, we then chose the direction that yielded the highest likelihood (i.e., lowest negative log-likelihood) as the predicted direction.

3.4 Integration of ECoG-Based Predictions and Task

In additional offline analyses, we simulated the integration of the directional prediction using ECoG signals with the task. We placed the cursor in the predicted direction at a distance α from the center. In other words, if the directional prediction was accurate, the cursor would be placed closer to the target, resulting in a decreased time needed to hit the target. Figure 5 gives a simple illustration of re-positioning the cursor toward the predicted target direction. In this figure, the black box is the target and the predicted direction is (somewhat inaccurately) towards the right. If the radius of the circle in Figure 5 is R, to decrease movement time, the distance between the re-positioned cursor and the target should be less than R. The dashed circle is the locus of all points at a distance R to the target. Thus, to be closer to the target, after direction prediction the cursor should lie within the dashed circle. The area enclosed by the two circles is 39% of the area of the solid circle, implying that if we place the cursor in a random direction, then there would be a 61% chance that the cursor would be farther away from the target compared to if were left in the center of the circle. In other words, if we randomly placed the cursor within the solid circle, on average it would take the subject longer to hit the target. Conversely, if the output prediction is the correct target or at least one of its neighbors, this would bring the cursor closer to the target and may allow the user to reduce the time it would take to hit the target. The angle between the imaginary line that passes through the center and the actual target and the imaginary line that passes through the center and one of the intersection points between the two circles is 60°. Hence, we have an error margin of ±60° to improve the performance.

FIGURE 5
www.frontiersin.org

Figure 5. An illustration of the simulated brain-assisted targeting system. The black box depicts the current target and the white boxes represent all other possible target locations. The arrow shows the incorrectly predicted direction. The dashed circle is the locus of all points at a distance R away from the target. When the cursor is re-positioned in this circle, the distance to the target is decreased, as R′ < R.

4. Results

4.1 Prediction of Movement Onset

As described in the Methods, we computed from ECoG signals (i.e., amplitudes in the mu, beta, and gamma band, as well as the LMP) the likelihood of an occurrence of a movement onset. Figure 6 gives an exemplary resulting time course of the likelihood values of an onset over 20 s for Subject A, along with arrows that indicate the actual and predicted movement onsets. We observe that the likelihood curves exhibit sharp peaks around the actual movement onset, which demonstrates that the classifier can accurately detect movement onsets using ECoG signals. In each subject, we used these likelihood values to classify each time point t as “onset” if the likelihood value exceeded an empirically attained threshold value of 0.3. The confusion matrices, evaluated during actual movement, for all five subjects are shown in Table 2 in percentages, along with the total number of events and the resulting F1-scores. Most of the false positives occur 200 ms around the movement onset (i.e., the peak of the probability function). The predictor yields high detection performance for Subjects A and B. The high occurrence of false negatives decreased the F1-score for Subjects D and E. The high number of false negatives and false positives led to reduced performance in Subject C.

FIGURE 6
www.frontiersin.org

Figure 6. Time course of likelihood of movement onset for Subject A. The dashed line represents a probability level of 0.3, which was used as a threshold to predict the movement onset.

TABLE 2
www.frontiersin.org

Table 2. Confusion matrices of movement onset prediction for Subjects A–E. The values shown in parentheses are the false positives that occur more than 200 ms away from the actual movement onset. The bottom row gives the F1-score.

Figure 7 shows the confidence indices of the ECoG features that are most predictive of the movement onset. (Note that a significance level of p < 0.05 corresponds to a confidence index of −log(p) > 3). The two larger brain models in the top row show the confidence indices for the high gamma band and LMP, the most predictive of the features, accumulated across all subjects. The bottom two rows show the results of the high gamma band and LMP for individual Subjects A–E. These results indicate that the brain areas yielding prediction of movement onset are centered on hand representations of motor cortical areas for the high gamma band and extend beyond the hand representations for the LMP. This finding is in agreement with results from a previous report that investigated finger flexions (Kubánek et al., 2009). However, areas other than the sensorimotor cortex were used in the onset prediction as selected by the feature selection algorithm (see Methods). On the other hand, mu and beta bands did not yield statistically significant activations for prediction of movement onset. Although desynchronization of these bands has been posited to relate to local increases in high gamma amplitude in the motor cortex (i.e., as a gating mechanism) (Miller et al., 2009), the corresponding brain signals are known to be spatially widespread and slowly evolving, and thus may not be the best indicators for the exact timing of movement onset.

FIGURE 7
www.frontiersin.org

Figure 7. Relationship of brain signals with movement onset. This figure shows the spatial distribution of accumulated confidence indices −log(p) for the 70–170 Hz high gamma band (top left) or the LMP (top right) across all subjects. The spatial distribution of the confidence indices for each subject are depicted in the bottom rows.

4.2 Prediction of Direction of Intended Movement

We computed the absolute value of the angular error of the directional predictions obtained by the MTVDBNs for each trial and each subject, and calculated their mean and standard deviation across all trials. This yielded the following single-trial angular error statistics for Subjects A–E, respectively: 55.29° ± 52.61°, 46.28° ± 57.80°, 68.95° ± 55.16°, 70.55° ± 53.78°, and 87.8° ± 57.9°. A single-sided t-test revealed that the accuracy of our results was better than chance (i.e., 90.0 ± 57.36°) at a significance level of 5% (p < 0.05) for all subjects. Our method yielded single-trial angular errors that were smaller than 90° for all subjects, and as low as 46° in one subject. Given that the targets are separated by 45°, the results indicate that the classifier was able to infer the direction of intended movements within less than two targets in single trials.

4.3 Integration of ECoG-Based Predictions and Task

We further studied whether integrating information extracted from ECoG signals would improve performance in a simulated targeting task. Every time a movement onset and direction were predicted, we placed the cursor in the predicted direction. As described in the Methods, this stimulation will result in improved performance if the single-trial error of the predicted direction is less than 60°. Figure 8 shows the positions of the cursor averaged across trials for each target for Subject A. It is clear that, after the re-positioning of the cursor based on the predicted directions, on average the subject would need to move a shorter distance to hit each target, and therefore should complete the task in less time assuming he/she would not have been distracted by the changing position of the cursor. After the cursor was placed closer to the decoded target, we simulated the cursor movement toward the target at each subject's average moving speed.

FIGURE 8
www.frontiersin.org

Figure 8. Results for movement direction prediction results for Subject A. The symbols represent the eight different targets. The symbols in bold depict the actual positions of the targets and the thinner symbols show the averaged pre-movement positions of the cursor across all trials in each direction. (Note that these are average cursor positions. Values for single-trial angular errors are given in the text.)

The distance α at which we place the cursor toward the predicted target is an important parameter that needs to be optimized. Note that α is a factor of the distance to the target (i.e., α = 0.5 places the cursor halfway between the center and the predicted target) and thus 0 < α ≤ 1. With either too small or too large of an α (α > 1), we might not take full advantage of the prediction results. The time needed to hit the target as a function α is plotted in Figure 9 for Subject A, which suggests an optimal value of 0.8 for α. The optimal α values for other subjects are listed in Table 3, which also compares the average movement time with and without the assistance of ECoG signals. The results in this table also demonstrate that the movement time of the majority of trials would have improved up to 150 ms. These encouraging results suggest that at least under certain assumptions2, ECoG signals may improve a user's performance in a targeting task.

FIGURE 9
www.frontiersin.org

Figure 9. Time to hit the target as a function of α for Subject A, where α is a scaling factor of the distance from the center to the targets. The dashed line gives the average time to hit the target without the assistance of ECoG signals. Error bars give the standard deviation of the mean. Note that predictions from the brain signals do not have any effect when α is zero.

TABLE 3
www.frontiersin.org

Table 3. Statistics and parameters of the simulated performance: optimal α values, original average movement times, average simulated movement times achieved by integrating ECoG, percentage of trials with improved movement time, percentage of trials with degraded movement time, and the statistical significance of performance improvement (in trials with reduced movement time).

5. Discussion

In this paper, we demonstrated that it is possible to use ECoG signals to detect the onset and the direction of an intended movement. We also demonstrate in a simulation that it may be possible to use these predictions to reduce the time to complete a targeting task by pre-positioning the cursor using ECoG signals acquired only prior to the movement. We achieved detection of movement onset by continually analyzing the incoming ECoG signals using a SVM classifier, and predicted the intended movement direction using a novel variant of a dynamic Bayesian network (i.e., the MTVDBN algorithm) that captures the spatial and temporal structure of the ECoG features.

5.1 Optimal Integration of Brain Signals for Improvement of Performance

The distance α from the center at which the cursor is repositioned affects the improvement in performance as shown in Figure 9. Moreover, we see in Table 3 that the optimal α values ranges from 0.4 to 1.0 across subjects. It can also be observed that a higher α value is reflected in higher percentage of trials with improved movement time. This does not imply that a higher α value improves performance, which is not the case for Subjects D–E. Rather, it implies that if the directional classifier yields accurate outputs, we can take better advantage of the system (with a high α) for improving performance. Hence, improving classifier performance is crucial for the optimal integration of information from brain signals with external (e.g., joystick) control.

5.2 Relevance for Asynchronous BCIs

Synchronous BCIs restrict the user to communicate in predefined time frames. Asynchronous BCIs, which allow the user to communicate spontaneously, may support more powerful practical applications as they are self-initiated and self-paced systems. However, asynchronous BCIs require the detection of the event in addition to identifying the properties of the event. The results presented in this paper may prove useful as the basis for an asynchronous BCI. While data collection was achieved using cued external events, our decoder was ambivalent to these events and processed the incoming data asynchronously. That is, the only input to the decoder in our experiment was the ECoG time sequence. At each time t, the decoder detected whether the user was beginning to move the joystick, and also predicted the intended movement. Future research could explore similar capacities in completely uncued situations, and in people who attempt but do not actually execute movements, such as people with chronic stroke.

5.3 Relevance for Performance Augmentation

We showed in a simulation that the time required to perform a directional motor task can be reduced by up to 150 ms. Such improvements in human performance should have a number of important applications. As an example, a reduction of the time to acquire a target may increase the probability to come out ahead in tactical combat. To investigate this possibility, the methods presented in this paper could be readily transferred to real-time testing. In this scenario, model parameters are estimated once initial training data are collected. The computational complexity of the MTVDBN directional classifier during the online testing session is O(dTM2), where d is the number of directions, T is the memory depth (i.e., input time samples), and M is the number of input features (i.e., spectral bands times the number of channels). This relatively modest computational requirement should readily support real-time testing once the MTVDBNs are trained. Nevertheless, real-time implementation and testing is necessary to determine whether the time to hit the target will also decrease in actual online experiments.

5.4 Experimental Limitations

While the signal characteristics of ECoG are attractive, the acquisition and study of ECoG have several important limitations. Foremost, to record cortical signals subdurally, a craniotomy and dural incision must be performed. Hence, the implantation of the grids is associated with infrequent, but serious risks, such as inflammation or death. Moreover, the extent of grid coverage and its placement is not standardized across subjects and is determined by the clinical needs of the patients. For instance, the subject with the smallest fraction of trials with improved movement time (i.e., Subject D) had grids implanted ipsilateral to her dominant (left) hand. Thus, she was asked to use her non-dominant hand (i.e., contralateral to her implants) during the experiment, which might have contributed to reduced information about movement direction, and hence the low percentage of trials with improved movement times. Next, the physical and cognitive condition and level of cooperation of each subject are variable. Moreover, ECoG experiments are for practical reasons performed in uncontrolled noisy environments (i.e., hospital rooms). Furthermore, the subjects in the study suffered from epilepsy, and thus may have some degree of functional or structural reorganization compared to healthy individuals. Despite these limitations, the results presented in this and other ECoG studies are usually consistent with expectations based on human neuroanatomy.

While we controlled for important variables in this study (such as eye gaze), the experimental setup in any ECoG study is necessarily somewhat less controlled than that in the typical animal or human neuroscientific study. However, real-world environments are typically very uncontrolled as well. This circumstance strengthens our claim that our results may translate into benefits in real-life scenarios.

In its present design, the onset predictor is based on ECoG signals from the previous 1s. In other words, at least 1s of data need to be available to make a prediction about movement onset. In addition, the directional classifier is designed for prediction of discrete directions at the time of movement onset. Moreover, it takes advantage of the ±60° error margin to bring the cursor closer to the target. Hence, it is unclear to what extent the information in ECoG would generalize to reliable predictions during continuous cursor control. Finally, real-time implementation of the proposed system (i.e., re-positioning of the cursor) is required for evaluating the proposed system, as unpredicted jumps in the cursor, whether closer to or farther from the target, might affect the performance of the user.

5.5 Future Directions

The work presented in this paper focused on detecting the onset and direction of movements using ECoG signals. The methodologies for initiation detection presented in this paper may also be extended to the detection of inhibition of a movement. Volitional inhibition is the process of adapting to sudden changes in the surrounding environment by stopping or modifying an action. Thus, future work may include testing countermanding of initiated motor responses, e.g., using a stop-signal paradigm (Logan et al., 1984). Inhibiting a response has been suggested to recruit a fronto-basal ganglia-thalamic network, including the right inferior frontal gyrus (Aron et al., 2003, 2007; Rubia et al., 2003; Chambers et al., 2006) and pre-supplementary motor cortical areas (Sumner et al., 2007; Chen et al., 2010; Scangos and Stuphorn, 2010). These areas are thought to influence the cortical areas underlying limb movement preparation and initiation, i.e., dorsal premotor (Mirabella et al., 2012) and primary motor cortices (Coxon et al., 2006; Swann et al., 2009), through the subthalamic nucleus (Aron and Poldrack, 2006; van den Wildenberg et al., 2006; Mirabella et al., 2011). Applying the methodological framework described in this study to ECoG signals collected over the right inferior gyrus, motor and pre-supplementary motor cortices during a countermanding task may thus allow for the detection of the onset of corresponding inhibitory processes. Using such an approach, it may be possible for stopping the cursor from moving in the wrong direction in cases when the directional classifier failed.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This work was supported in part by grants from the US Army Research Office [W911NF-08-1-0216 (Gerwin Schalk) and W911NF-07-1-0415 (Gerwin Schalk) and the NIH EB006356 (Gerwin Schalk) and EB000856 (Gerwin Schalk)]. The authors would like to thank the subjects for their participation, the staff of AMC Epilepsy Monitoring Unit for their assistance, and Griffin Milsap for technical support.

Footnotes

  1. ^In ancillary analyses, we performed feature selection on 1/5th of the data, which were subsequently not used in the training and testing of the onset and direction predictors. The results of this more conservative analysis were very similar to the results reported in the paper.
  2. ^One assumption here is that reduced distance-to-target will reduce movement time or at least will have other distinct benefits.

References

Achtman, N., Afshar, A., Santhanam, G., Yu, B. M., Ryu, S. I., and Shenoy, K. V. (2007). Free-paced high-performance brain-computer interfaces. J. Neural Eng. 4, 336–347.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Aron, A. R., Behrens, T. E., Smith, S., Frank, M. J., and Poldrack, R. A. (2007). Triangulating a cognitive control network using diffusion-weighted magnetic resonance imaging (MRI) and functional MRI. J. Neurosci. 27, 3743–3752.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Aron, A. R., Fletcher, P. C., Bullmore, E. T., Sahakian, B. J., and Robbins, T. W. (2003). Stop-signal inhibition disrupted by damage to right inferior frontal gyrus in humans. Nat. Neurosci. 6, 115–116.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Aron, A. R., and Poldrack, R. A. (2006). Cortical and subcortical contributions to Stop signal response inhibition: role of the subthalamic nucleus. J. Neurosci. 26, 2424–2433.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Bai, O., Lin, P., Vorbach, S., Floeter, M. K., Hattori, N., and Hallett, M. (2008). A high performance sensorimotor beta rhythm-based brain-computer interface aociated with human natural motor behavior. J. Neural Eng. 5, 24–35.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ball, T., Kern, M., Mutschler, I., Aertsen, A., and Schulze-Bonhage, A. (2009). Signal quality of simultaneously recorded invasive and non-invasive EEG. Neuroimage 46, 708–716.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Borisoff, J. F., Mason, S. G., Bashashati, A., and Birch, G. E. (2004). Brain-computer interface design for asynchronous control applications: improvements to the LF-ASD asynchronous brain switch. IEEE Trans. Biomed. Eng. 51, 985–992.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Buch, E., Weber, C., Cohen, L. G., Braun, C., Dimyan, M. A., Ard, T., Mellinger, J., Caria, A., Soekadar, S., Fourkas, A., and Birbaumer, N. (2008). Think to move: a neuromagnetic brain-computer interface (BCI) system for chronic stroke. Stroke 39, 910.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Chambers, C. D., Bellgrove, M. A., Stokes, M. G., Henderson, T. R., Garavan, H., Robertson, I. H., Morris, A. P., and Mattingley, J. B. (2006). Executive “brake failure” following deactivation of human frontal lobe. J. Cogn. Neurosci. 18, 444–455.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Chao, Z. C., Nagasaka, Y., and Fujii, N. (2010). Long-term asynchronous decoding of arm motion using electrocorticographic signals in monkeys. Front. Neuroeng. 3:3. doi: 10.3389/fneng.2010.00003

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Chen, X., Scangos, K. W., and Stuphorn, V. (2010). Supplementary motor area exerts proactive and reactive control of arm movements. J. Neurosci. 30, 14657–14675.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Coxon, J. P., Stinear, C. M., and Byblow, W. D. (2006). Intracortical inhibition during volitional inhibition of prepared action. J. Neurophysiol. 95, 3371–3383.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Daly, J. J., and Wolpaw, J. R. (2008). Brain-computer interfaces in neurological rehabilitation. Lancet Neurol. 7, 1032–1043.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Duda, R. O., Hart, P. E., and Stork, D. G. (2008). Pattern Classification. New York, NY: Wiley-Interscience.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Efron, B., Hastie, T., Johnstone, I., and Tibshirani, R. (2004). Least angle regression. Ann. Stat. 32, 407–499.

Georgopoulos, A. P., Kalaska, J. F., Caminiti, R., and Massey, J. T. (1982). On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J. Neurosci. 2, 1527–1537.

Pubmed Abstract | Pubmed Full Text

Gunduz, A., Sanchez, J. C., Carney, P. R., and Principe, J. C. (2009). Mapping broadband electrocorticographic recordings to two-dimensional hand trajectories in humans. Neural Netw. 22, 1257–1270.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gunduz, A., and Schalk, G. (2011). “Defense-related insights and solutions from neuroscience and neuroengineering,” in Proceedings of SPIE, Volume 8058 of Independent Component Analyses, Wavelets, Neural Networks, Biosystems and Nanoengineering, ed H. Szu (Orlando, FL, USA: International Society of Optics and Photonics).

Hasan, B. A. S., and Gan, J. Q. (2011). Temporal modeling of EEG during self-paced hand movement and its application in onset detection. J. Neural Eng. 8, 056015.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Haufe, S., Treder, M., Gugler, M., Sagebum, M., Curio, G., and Blankertz, B. (2011). EEG potentials predict upcoming emergency brakings during simulated driving. J. Neural Eng. 8, 056001.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hochberg, L., Serruya, M., Friehs, G., Mukand, J., Saleh, M., Caplan, A., Branner, A., Chen, D., Penn, R., and Donoghue, J. (2006). Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature 442, 164–171.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Huang, Y. M., and Du, S. X. (2005). “Weighted support vector machine for classification with uneven training class sizes,” in Proceedings of International Conference on Machine Learning and Cybernetics, Vol. 7, (Guangzhou, China), 4365–4369.

Hwang, E. J., and Andersen, R. A. (2009). Brain control of movement execution onset using local field potentials in posterior parietal cortex. J. Neurosci. 29, 14363–14370.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kubánek, J., Miller, K. J., Ojemann, J. G., Wolpaw, J. R., and Schalk, G. (2009). Decoding flexion of individual fingers using electrocorticographic signals in humans. J. Neural Eng. 6, 066001.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Lancaster, J. L., Woldorff, M. G., Parsons, L. M., Liotti, M., Freitas, C. S., Rainey, L., Kochunov, P. V., Nickerson, D., Mikiten, S. A., and Fox, P. T. (2000). Automated Talairach atlas labels for functional brain mapping. Hum. Brain Mapp. 10, 120–131.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Lebedev, M. A., O'Doherty, J. E., and Nicolelis, M. A. L. (2008). Decoding of temporal intervals from cortical ensemble activity. J. Neurophysiol. 99, 166–186.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Leeb, R., Friedman, D., Müller-Putz, G. R., Scherer, R., Slater, M., and Pfurtscheller, G. (2007). Self-paced (asynchronous) BCI control of a wheelchair in virtual environments: a case study with a tetraplegic. Comput. Intell. Neurosci. 2007, 7:1–7:12.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Leuthardt, E. C., Schalk, G., Wolpaw, J. R., Ojemann, J. G., and Moran, D. W. (2004). A brain-computer interface using electrocorticographic signals in humans. J. Neural Eng. 1, 63–71.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Logan, G. D., Cowan, W. B., and Davis, K. A. (1984). On the ability to inhibit simple and choice reaction time responses: a model and a method. J. Exp. Psychol. Hum. Percept. Perform. 10, 276–291.

Pubmed Abstract | Pubmed Full Text

Mak, J. N., and Wolpaw, J. R. (2009). Clinical applications of brain-computer interfaces: current state and future prospects. IEEE Rev. Biomed. Eng. 2, 187–199.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Marple, S. L. (1986). Digital Spectral Analysis: With Applications. Upper Saddle River, NJ: Prentice Hall.

Mason, S. G., and Birch, G. E. (2000). A brain-controlled switch for asynchronous control applications. IEEE Trans. Biomed. Eng. 47, 1297–1307.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Millan, J., and Mouriño, J. (2003). Asynchronous BCI and local neural classifiers: an overview of the adaptive brain interface project. IEEE Trans. Neural Syst. Rehabil. Eng. 11, 159–161.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Miller, K. J., Leuthardt, E. C., Schalk, G., Rao, R. P., Anderson, N. R., Moran, D. W., Miller, J. W., and Ojemann, J. G. (2007). Spectral changes in cortical surface potentials during motor movement. J. Neurosci. 27, 2424–2432.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Miller, K. J., Zanos, S., Fetz, E. E., den Nijs, M., and Ojemann, J. G. (2009). Decoupling the cortical power spectrum reveals real-time representation of individual finger movements in humans. J. Neurosci. 29, 3132.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Mirabella, G., Iaconelli, S., Romanelli, P., Modugno, N., Lena, F., Manfredi, M., and Cantore, G. (2012). Deep brain stimulation of subthalamic nuclei affects arm response inhibition in Parkinson's patients. Cereb. Cortex 22, 1124–1132.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Mirabella, G., Pani, P., and Ferraina, S. (2011). Neural correlates of cognitive control of reaching movements in the dorsal premotor cortex of rhesus monkeys. J. Neurophysiol. 106, 1454–1466.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Muralidharan, A., Chae, J., and Taylor, D. M. (2011). Early detection of hand movements from electroencephalograms for stroke therapy applications. J. Neural Eng. 8, 046003.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Pistohl, T., Ball, T., Schulze-Bonhage, A., Aertsen, A., and Mehring, C. (2008). Prediction of arm movement trajectories from ECoG-recordings in humans. J. Neurosci. Methods 167, 105–114.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Rubia, K., Smith, A. B., Brammer, M. J., and Taylor, E. (2003). Right inferior prefrontal cortex mediates response inhibition while mesial prefrontal cortex is responsible for error detection. Neuroimage 20, 351–358.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Scangos, K. W., and Stuphorn, V. (2010). Medial frontal cortex motivates but does not control movement initiation in the countermanding task. J. Neurosci. 30, 1968–1982.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Schalk, G., Kubanek, J., Miller, K. J., Anderson, N. R., Leuthardt, E. C., Ojemann, J. G., Limbrick, D., Moran, D., Gerhardt, L. A., and Wolpaw, J. R. (2007). Decoding two-dimensional movement trajectories using electrocorticographic signals in humans. J. Neural Eng. 4, 264.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Schalk, G., McFarland, D. J., Hinterberger, T., Birbaumer, N., and Wolpaw, J. R. (2004). BCI2000: a general-purpose brain-computer interface (BCI) system. IEEE Trans. Biomed. Eng. 51, 1034–1043.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Schalk, G., and Mellinger, J. (2010). A Practical Guide to Brain-Computer Interfacing with BCI(2000). New York, NY: Springer.

Schalk, G., Miller, K. J., Anderson, N. R., Wilson, J. A., Smyth, M. D., Ojemann, J. G., Moran, D. W., Wolpaw, J. R., and Leuthardt, E. C. (2008). Two-dimensional movement control using electrocorticographic signals in humans. J. Neural Eng. 5, 75–84.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Schmidt, M., Niculescu-Mizil, A., and Murphy, K. (2007). “Learning graphical model structure using L1-regularization paths,” in Proceedings of the 22nd National Conference on Artificial Intelligence, Vol. 2, (AAAI Press), 1278–1283.

Sejnowski, T. J., Dornhege, G., del Millán, J. R., Hinterberger, T., McFarland, D. J., and Müller, K. (2007). Toward Brain-Computer Interfacing (Neural Information Processing). Cambridge, MA: MIT Press.

Simeral, J. D., Kim, S.-P., Black, M. J., Donoghue, J. P., and Hochberg, L. R. (2011). Neural control of cursor trajectory and click by a human with tetraplegia 1000 days after implant of an intracortical microelectrode array. J. Neural Eng. 8, 025027.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Song, L., Kolar, M., and Xing, E. P. (2009). “Time-varying dynamic Bayesian networks,” in Proceedings of the 23rd Neural Information Processing Systems (NIPS), (Vancouver, Canada).

Sumner, P., Nachev, P., Morris, P., Peters, A. M., Jackson, S. R., Kennard, C., and Husain, M. (2007). Human medial frontal cortex mediates unconscious inhibition of voluntary action. Neuron 54, 697–711.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Swann, N., Tandon, N., Canolty, R., Ellmore, T. M., McEvoy, L. K., Dreyer, S., DiSano, M., and Aron, A. R. (2009). Intracranial EEG reveals a time- and frequency-specific role for the right inferior frontal gyrus and primary motor cortex in stopping initiated responses. J. Neurosci. 29, 12675–12685.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Tan, D. S., and Nijholt, A. (2010). Brain-Computer Interfaces. New York, NY: Springer Press.

van Rijsbergen, C. J. (1979). Information Retrieval. Cambridge, MA, USA: Butterworths.

van den Wildenberg, W. P. M., van Boxtel, G. J. M., van der Molen, M. W., Bosch, D. A., Speelman, J. D., and Brunia, C. H. (2006). Stimulation of the subthalamic region facilitates the selection and inhibition of motor responses in Parkinson's disease. J. Cogn. Neurosci. 18, 626–636.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Wang, W., Sudre, G. P., Xu, Y., Kass, R. E., Collinger, J. L., Degenhart, A. D., Bagic, A. I., and Weber, D. J. (2010). Decoding and cortical source localization for intended movement direction with MEG. J. Neurophysiol. 104, 2451–2461.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Wisneski, K. J., Anderson, N., Schalk, G., Smyth, M., Moran, D., and Leuthardt, E. C. (2008). Unique cortical physiology associated with ipsilateral hand movements and neuroprosthetic implications. Stroke 39, 3351.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., and Vaughan, T. M. (2002). Brain-computer interfaces for communication and control. Clin. Neurophysiol. 113, 767–792.

Pubmed Abstract | Pubmed Full Text

Yanagisawa, T., Hirata, M., Saitoh, Y., Goto, T., Kishima, H., Fukuma, R., Yokoi, H., Kamitani, Y., and Yoshimine, T. (2011). Real-time control of a prosthetic hand using human electrocorticography signals. J. Neurosurg. 114, 1715–1722.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Appendix

FIGURE A1
www.frontiersin.org

Figure A1. Example of an implanted subdural grid (Subject A). (A) Lateral radiograph indicating grid position. (B) Subdural grid placed over left fronto-parietal and temporal lobes.

FIGURE A2
www.frontiersin.org

Figure A2. The experimental setup at Albany Medical College. Joystick control mostly entailed wrist movements.

Keywords: brain computer interface, ECoG, movement direction prediction, movement onset prediction, neurorehabilitatation, performance augmentation

Citation: Wang Z, Gunduz A, Brunner P, Ritaccio AL, Ji Q and Schalk G (2012) Decoding onset and direction of movements using Electrocorticographic (ECoG) signals in humans. Front. Neuroeng. 5:15. doi: 10.3389/fneng.2012.00015

Received: 29 January 2012; Paper pending published: 02 March 2012;
Accepted: 28 June 2012; Published online: 08 August 2012.

Edited by:

Giovanni Mirabella, University of La Sapienza, Italy

Reviewed by:

Giovanni Mirabella, University of La Sapienza, Italy
Mikhail A. Lebedev, Duke University, USA

Copyright © 2012 Wang, Gunduz, Brunner, Ritaccio, Ji and Schalk. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.

*Correspondence: Gerwin Schalk, Wadsworth Center, New York State Department of Health, C650 Empire State Plaza, Albany, NY, USA. e-mail: schalk@wadsworth.org

Co-first authors.