Error-Related Neural Responses Recorded by Electroencephalography During Post-stroke Rehabilitation Movements

Introduction

Stroke is the second leading cause of death and the third leading cause of disability in the world (Johnson et al., 2016). Depending on the extent and the location of damage due to stroke, stroke survivors suffer from a various degree of body functionality impairment (Kalaria et al., 2016). The most notable impairments include cognitive impairment, dementia, and limb movement impairment (Kalaria et al., 2016). Nevertheless, recovery from the disability due to stroke is possible through neuroplasticity (Basteris et al., 2014). Neuroplasticity is brain’s natural process aimed to reorganize itself by forming new neural connections, especially in response to learning or experiences that result in partial recovery from the disability. Post-stroke rehabilitation accelerates this process of neurological changes and ultimately helps in attaining a higher rate of recovery (Liu et al., 2017).

Motor function impairment affects the patient’s activities of daily living (ADLs) (Basteris et al., 2014). Therefore, recovery from motor disabilities is one of the primary objectives of the post-stroke rehabilitation program. Historically, in post-stroke rehabilitation, the patients are asked to perform certain physical exercises using their affected limb (Liu et al., 2017). Previous studies have described the role of intensive and repetitive rehabilitation exercises in promoting the rate of recovery from motor disabilities (Grosmaire and Duret, 2017; Liu et al., 2017; Tacchino et al., 2017). In addition, several studies have highlighted the significance of the active participation of the patient in performing rehabilitation exercises to promote motor recovery (Grosmaire and Duret, 2017; Tacchino et al., 2017). However, the key issue is that stroke patients cannot perform the rehabilitation exercises repetitively and actively due to their motor impairment (Basteris et al., 2014; Yue et al., 2017).

Assist-as-needed (AAN) robot therapy-based rehabilitation program helps in eliminating the issues mentioned above and allows the patients to perform exercises repetitively and actively (Basteris et al., 2014; Grosmaire and Duret, 2017). In AAN based robot therapies, assistance is provided to the patient in performing the rehabilitation exercise when he/she is unable to perform it on his/her own and vice versa. A few well-known EEG based BCI approaches, including movement-related cortical potentials (MRCP) (Liu et al., 2017), event-related synchronization/desynchronization (ERS/ERD) (Liu et al., 2017) and surface-electromyographic (sEMG) signals (Basteris et al., 2014), are most popular in developing assistive exoskeletons that assist stroke survivors in performing physical exercises. Error-related potential (ErrP) is another event-related potential (ERP) signal which is gaining popularity recently in BCI research due to its inherent intrinsic human feedback mechanism (Chavarriaga et al., 2014). It is an ERP that is elicited when a human perceives an error (Chavarriaga et al., 2014).

The existing body of research on ErrP suggests that ErrP signal is elicited under certain task situations. Response ErrP occurs when the subject is asked to respond as quickly as possible (e.g., choice reaction task) (Olvet and Hajcak, 2009). Feedback ErrP occurs when the subject realizes an error upon given feedback of the task (Lopez-Larraz et al., 2010). Interaction ErrP occurs when the subject is interacting with a machine, and the machine misinterpreted an instruction given (Ferrez and Del R. Millán, 2008). Observation ErrP occurs when the subject recognizes an error made by a machine or external system (Roset et al., 2014). Recently, three new types of errors, namely target, outcome, and execution ErrPs have been reported (Milekovic et al., 2013). To date, a number of studies have investigated the ErrP signal and its applicability in the EEG-based BCIs (Olvet and Hajcak, 2008; Chavarriaga et al., 2014; Weinberg et al., 2015; Kim et al., 2017). In Omedes et al. (2018), authors developed a 3D virtual interface that simulated participants’ hand to reach and grasp different virtual objects and found that an ErrP evokes in erroneous commands. In Kim et al. (2017), ErrP was employed to detect if a robot has made an error in recognizing the gesture made by the participant. In Yazmir and Reiner (2017), authors developed a virtual tennis game in which a difference in EEG response was observed when the participant successfully hit to that of miss the target.

Although significant research has been carried out on ErrP, there is no single study which discusses the feasibility of ErrP in implementing assist-as-needed robot therapy approach. With the aim of filling the gap mentioned above in BCI based methods, in this study we investigated if an ErrP signal is evoked when a participant is unable to perform a physical exercise. Fifteen stroke patients participated in this EEG experiment in which they performed a standard rehabilitation exercise. We hypothesized that a difference similar to error-related potentials will exist when the stroke patients are unable to perform a rehabilitation exercise (failure trial) to that of when they complete the same rehabilitation exercise (success trial) in a given time. Results of this study will provide a novel basis for the BCI based methods to be implement in the AAN based robot therapy, hence forming a foundation of a new type of task situation in which the ErrP brain signal can be elicited. This will assist in differentiating a failure trial against a success trial in performing a rehabilitation exercise. The developed ErrP based brain-in-the-loop approach is expected to enhance the efficiency of robot-based stroke rehabilitation programs.

Materials and Methods

Participants

Fifteen stroke patients (five females, mean age: 57.5 ± 11.3 years) participated in the experiment (see Table 1). The experiment was conducted in collaboration with the 2nd Hospital of Jiaxing, China. All participants voluntarily took part in the experiment and provided their written informed consent before commencement of experiments. The stroke patients’ affected limb’s impairment level was assessed using Brunnstrom stages of recovery by an expert panel from the 2nd Hospital of Jiaxing, China prior to experimentation (see Table 1). The study was approved by the Ethics Committee of the 2nd Hospital of Jiaxing, and the experiments were conducted in accordance with the declaration of Helsinki. The data of Patient 1 and 9 had to be excluded from the analysis because patient 1 aborted the experiment at the start and patient 9 was not performing the experiment as per the instructions given.

TABLE 1

Table 1. Stroke patients’ information.

Task Description

Participants sat on a comfortable chair facing the LCD monitor (resolution of 1280 × 720 and refresh rate of 100 Hz) that delivered task instructions to the participants.

The experiment required participants to perform a standard Bobath’s rehabilitation exercise: shoulder flexion-extension while adjoining both hands. Already recorded video of a therapist performing the exercise was used to illustrate the know-how of the exercise to the participants. The video was collected from the 2nd Hospital of Jiaxing, China. Each participant was asked to participate in two sessions of the experiment including 24 trials each; however, patient 2, 3, 7, and 14 participated only in one session due to muscle fatigue. All participants’ data satisfied the minimum number of trials required to conduct an ErrP based ERP study, i.e., at least six trials of each class, as outlined in the previous studies (Olvet and Hajcak, 2009; Meyer et al., 2013). A fixation cross, as shown in Figure 1A appeared at the start of a trial. Followed by the fixation mark, the exercise video was played (see Figure 1B). Participants were asked to observe the exercise being described in the video. Afterward, a 3-2-1 timer (see Figure 1C) starts and the participants were asked to perform the exercise (accurately as depicted in the video) once the timer finishes and Figure 1D appears. Participants were given 2–15 s to complete the exercise. Roughly, in 35% of the trials, 2–4 s, in 10% of the trials 9–10 s and in the rest of the trials 15 s were given to complete the exercise in a pseudo-random fashion. Participants were asked to complete the exercise before the ‘Time’s up!’ screen (see Figure 1E) has appeared. Additionally, a beep sound was played on the onset of ‘Time’s up!’ screen for about 30 ms to inform the participants that the time has expired. The accuracy was preferred over the speed as the ErrP amplitudes are notably larger when the accuracy is emphasized over the speed (Gehring et al., 1993).

FIGURE 1

Figure 1. Description of the task’s trial and visual stimuli. Instructions were delivered in the following order. (A) A fixation cross marking the start of the trial, (B) exercise video, (C) a 3-2-1 timer, (D) image depicting instruction to start performing the exercise. The time given to perform the exercise varied between 2 and 15 s in a pseudo-random fashion and it was based on a preliminary investigation. (E) Participants were asked to complete the exercise before this screen appeared.

The time durations given to complete the exercise were based on a preliminary experiment such that the patients will not be able to complete the exercise in the short 2–4 s trials (named failure trials onwards) and will be able to successfully complete the exercise in the long 15 s trials (named success trials onwards). A complete exercise was marked in a trial when the participant started performing the exercise on the onset of the screen as in Figure 1D and execute the exercise depicted in the exercise video, and his/her arms come to rest before the onset of the screen as in Figure 1E. The medium 9–10 s trials were used to add additional uncertainty so that participants cannot anticipate trial durations. The medium 9–10 s trials were not included in the data analysis. Participants were asked to take rest for about 20 min in between two sessions to avoid muscle fatigue, and thus, the participants could keep full concentration in the experiment.

Experiment procedure was explained to the participants, and sufficient time was given for practice before the start of the experiment. The visual stimuli of the experiment were designed using Presentation software (version 20.2, Neurobehavioral Systems, Inc., Berkeley, CA, United States).

Data Acquisition

Fifty-eight channels of monopolar EEG were recorded using an active EEG electrode system and g. HIamp amplifier (g. tec GmbH, Austria) as per the international 10–20 system. Electro-oculogram (EOG) signals were recorded from the electrodes placed above and below the left eye (FP1, VEOG) and on the outer canthi of both eyes (F9, F10). The EOG data were recorded to remove the eye-blinks artifacts from the EEG signals. Channels were referenced to the right earlobe and grounded to AFz location. Bipolar sEMG signals were recorded from the deltoid posterior muscle. All electrodes impedance was kept below 10 kΩ. All three physiological signals were sampled at 2400 Hz with a 50 Hz notch filter. EEG and EOG data were additionally filtered with 0.01–1000 Hz band-pass filter, and EMG data were additionally filtered with a 10–500 Hz band-pass filter to remove the slow drift and high-frequency noises in the signal (Farquhar and Hill, 2013).

Data Analysis

To prepare the raw data for further analysis, all the recorded physiological signals were preprocessed offline. Data preprocessing was performed in MATLAB and EEGLAB toolbox (Delorme and Makeig, 2004). The preprocessing steps are as follows:

(1) High-pass filtered the data to remove low-frequency drifts using Windowed sinc finite impulse response filter (zero phase shift), 0.1 Hz cut-off frequency, as per the parameters recommended in Widmann et al. (2015).

(2) Low pass filtered the data using Windowed sinc finite impulse response filter (zero phase shift), 128 Hz cut-off frequency, as per the parameters recommended in Widmann et al. (2015).

(3) Data were down-sampled to 512 Hz.

(4) Artifactual channels were removed with manual inspection of the data.

(5) Artifact subspace reconstruction (ASR) (Mullen et al., 2013) method was used to remove transients, stereotypical and non-stereotypical high amplitude artifacts from the continuous EEG data with parameters [standard deviation (SD) 20, sliding window length 500 ms and correlation coefficient 0.8] as suggested in Chang et al. (2018).

(6) Independent component analysis was used to remove any artifactual component left in the data (Delorme and Makeig, 2004). The components that comprised of ocular, cardiac and muscular artifacts were removed. A hybrid approach constituting ICLabel (EEGLAB) and visual inspection was used to filter out the artifactual components.

(7) Data were segmented into epochs ranging from −200 to 800 ms relative to the onset of Figure 1E. Further analysis was performed on the extracted epochs.

(8) Data epochs with maximum signal amplitude exceeding ± 75 μV and improbability exceeding 6 SD for single channels and 2 SD for all channels were flagged as outliers and were removed; about 5% of the trials were removed (Sörnmo and Laguna, 2005; Delorme et al., 2007).

Teager Kaiser energy operator (TKEO) method is a powerful method to detect the onset and offset of the muscle activity from the sEMG signal (Li et al., 2007). The discrete TKEO operator ψ is defined as:

$$\mathrm{\Psi }\left[x\left(n\right)\right]=x^2\left(n\right)-x\left(n+1\right)x\left(n-1\right)$$

where x is the sEMG value and n is the sample number (Li et al., 2007).

Muscle activity onset was considered when the TKEO operator ψ value exceeds a fixed threshold α for more than 25 consecutive samples, and similarly muscle activity offset was considered when the ψ value remains below the α threshold for more than 25 consecutive samples after the onset of the muscle activity (Solnik et al., 2008). The threshold α is defined as:

$$\mathrm{\alpha }={\mathrm{\mu }}_0+j{\mathrm{\delta }}_0$$

where μ₀ and δ₀ are mean and SD of the background sEMG noise in the TKEO domain, whereas j is a scale factor for δ₀. Value of j was chosen to be 7 based on the recommendation in Li et al. (2007). In a trial, if the participant started performing the exercise before Figure 1D screen appeared or in other words if the onset of the muscle activity was before the Figure 1D screen appeared, it was removed. The trials in which patients failed to observe the onset of Figure 1D screen and consequently delayed the execution of exercise were removed as well. Furthermore, the authenticity of failure and success trials were confirmed with the offset of the muscle activity, i.e., in failure trials, muscle activity offset should occur after the onset of the ‘Time’s up!’ screen and in success trials it should occur before the onset of the ‘Time’s up!’ screen. All the trials that had ambiguity in being a success or failure trials and the trials in which the participant did not perform the exercise at all were removed. Overall less than 20% of the trials were rejected. The artifacts free unambiguous success and failure trials were considered for further analysis.

Individual failure and success related ERP waveforms were calculated by averaging over the success and failure epochs of the same individual. Grand-average success and failure ERP waveforms were calculated by averaging over individual success and failure ERPs. Difference of individual as well as grand-average failure and success ERP waveforms were calculated further and were smoothed out with 25 Hz low-pass FIR filter for illustration purpose and statistical analysis. Topographical scalp maps provide reference-bias-free multi-channel information of the EEG signals (Murray et al., 2008). Therefore, scalp maps were also calculated using the grand-average success and failure EEG signals on the most prominent peaks.

In order to gauge induced and evoked power in the ERP signal, time-frequency analysis was performed (Dickter and Kieffaber, 2014). Evoked power is phase-locked to the stimulus presented; however, induced power (induced oscillatory activities) is not phase-locked and hence get canceled out in averaging (Dickter and Kieffaber, 2014). Nevertheless, both evoked and induced power are time-locked to the stimulus event (Dickter and Kieffaber, 2014). Event-related spectral perturbations (ERSPs) represents amplitude dynamics in a broad range of frequencies as a function of time furnishing information of both evoked and induced power unlike the grand-average ERPs that only provide information of the evoked power (Dickter and Kieffaber, 2014). On the other hand, Inter-trial coherence (ITC) computes phase synchronization across trials at each time point in a broad range of frequencies (Dickter and Kieffaber, 2014). ITC ranges between zero and one where zero suggests random phases and one suggests perfectly coherent phases across trials (Dickter and Kieffaber, 2014). Thus, localized changes in ERSP with a high value of ITC at that particular time and frequency represents evoked power whereas localized ERSP changes with low values of ITC represents induced power (Dickter and Kieffaber, 2014). Jointly with the ERSPs and the ITC, the power of the evoked response, as well as the induced oscillatory activities, can be gauged (Dickter and Kieffaber, 2014).

In result, time-frequency analysis was performed in EEGLAB to calculate the ERSPs and ITC with parameters as recommended in Delorme and Makeig (2004) and Dickter and Kieffaber (2014). ERSPs and ITC were calculated on full epoch length with a resolution of 4 ms and frequency resolution of 1 Hz approximately. Fast Fourier transform (FFT) with Hanning tapering was used for decomposition. The ERSP spectrogram post-0 ms was divided by the spectrogram before 0 ms for normalization. It is to be noted that the onset of Figure 1E screen is taken as 0 ms reference throughout the analysis. The significance of ERSPs deviation from the baseline was assessed using the bootstrap method (α = 0.05).

Statistical analysis was conducted using IBM SPSS Statistics 25 to estimate the statistical significance of the results. Statistical significance of the difference of failure and success ERP peaks’ amplitude (named difference ERP onwards) for each of the 13 participants were tested against zero with one-sample t-test (two-tailed). These individual difference ERP amplitudes were computed by averaging each patients’ failure and success trials separately and, then, extracting the amplitudes of the most prominent peaks and then taking a difference of that. The t-test null-hypothesis was that there is zero mean amplitude difference in the ERPs of success and failure trials, whereas the alternative hypothesis was non-zero mean amplitude difference. Previous studies have suggested that the ErrP signals have the highest activity around Cz and FCz electrodes (Chavarriaga and Millán, 2010; Spüler and Niethammer, 2015; Yazmir and Reiner, 2017). Therefore, the analysis was carried out on the most prominent peaks in the difference ERP at the Cz electrode location.

Shapiro–Wilk test (α = 0.05) was used to confirm the normal distribution of the mean samples before the statistical analysis (Shapiro and Wilk, 1965). The null-hypothesis of Shapiro–Wilk assumes a normal-distribution while alternative hypothesis denies that. Skewness and Kurtosis z-value were used as an additional measure to confirm the normality of the mean data samples (Cramer, 1998).

Results

Strong evidence of differences in the EEG response of the success and the failure trials has been found. The obtained ErrP has some unique features pertaining to the novel task situation besides some features similar to the previously reported ErrP signal.

The result of the grand average ERPs of the failure and success trials and their difference at Cz location has been shown in Figure 2A. Three-time windows (W1: 80–120 ms, W2: 120–160 ms, and W3: 380–440 ms) were chosen for analysis based on the most prominent peaks. Failure events ERP has a positive component with a peak at 98 ms and an amplitude of −1.139 μV and also a negative component with a peak at 131 ms and an amplitude of −5.236 μV. In addition, a negative component related to failure events has been observed with a peak at 414 ms and an amplitude of −1.528 μV. Similarly, success events ERP has a negative component with a peak at 102 ms with an amplitude of −5.043 μV and also a positive component with a peak at 137 ms with an amplitude of 0.714 μV. Likewise, a positive component related to success events has been observed with a peak at 400 ms and an amplitude of 2.55 μV. Figure 2A also presents the results of the difference of ERPs of failure and success events (difference ERP) in the three time-windows along with its standard deviation. As can be seen in Figure 2A, in the difference ERP, a positive peak appears at 100 ms with an amplitude of 3.611 μV pursued by a negative peak at 135 ms with an amplitude of −5.827 μV followed by a prominent negative peak at 410 ms with an amplitude of −4.687 μV. The scalp maps of the grand-average success and failure ERPs have been showing a centro-parietal scalp distribution mainly as can be seen in Figure 2B.

FIGURE 2

Figure 2. (A) The grand average ERPs of the failure events, success events, difference ERP along with its standard deviation at Cz electrode location. W1, W2, and W3 represent three times windows selected for analysis based on the most prominent peaks. (B) Scalp distribution maps of the most prominent peaks.

Figure 3 depicts individual waveforms of difference ERP. What stands out in Figure 3 is the repeating pattern of the individual difference ERPs. Results of only first six patients have been shown to prevent over-burdening the figure, while we got similar outcomes for each of the 13 patients, which can also be deduced from the SD shape exhibited in Figure 2A. Individual waveforms were based on all available epochs for each patient.

FIGURE 3

Figure 3. Individual difference ERPs at the Cz electrode for six patients. Results of only first six patients have been shown to prevent over-burdening the figure, while we got similar outcomes for each of the 13 patients, which can also be deduced from the SD shape exhibited in Figure 2A.

Event-related spectral perturbations and ITC of success and failure events have been depicted in Figure 4. The results showed in all four Figures 4A–D are statistically significant (α = 0.05); the insignificant areas are shown in the green color. Failure events associated evoked power is significantly high in the time-period 96–470 ms and 3–12 Hz, and ITC reaches up to 0.44 in this period as shown in Figure 4B. Induced power is significantly high in the time period 150–700 ms and 42–50 Hz, 640–700 ms and 11–24 Hz, 15–34 Hz at various times as shown in Figure 4A. Success events associated evoked power is significantly high in the time-period 100–460 ms and 3–10 Hz, 20–170 ms and 10–21 Hz and ITC reaches up to 0.46 in this period as shown in Figure 4D. Induced power is significantly high in the time period 550–700 ms and 3–22 Hz, 30–650 ms and 18–30 Hz, 34–38 Hz and 42–50 Hz at various times as shown in Figure 4C.

FIGURE 4

Figure 4. Grand average ERPs’ Event-related spectral perturbations (ERSPs) and inter-trial coherence (ITC). All colors except green shows statistically significant values at α = 0.05. (A,B) Depict ERSPs and ITC patterns for the failure events, respectively. (C,D) Depict ERSPs and ITC patterns for the success events respectively.

Shapiro–Wilk test results show that the assumption of normal distribution was not rejected for difference ERPs in all three-time windows at α = 0.05 (see Table 2). In addition, skewness and kurtosis z-value was within ± 1.96 for the three-time windows’ difference ERP peaks’ amplitudes that further strengthen the assumption of normal distribution.

TABLE 2

Table 2. Shapiro–Wilk test p-value, mean, standard deviation, t-test statistics (two-tailed) result of the difference in difference ERP’s peak amplitudes against zero in the three time-windows.

One-sample t-test (two-tailed) was applied to examine if the amplitude of the difference ERP peak in the time window W1, W2, and W3 (see Figure 2A) significantly different against zero or not. The results of the statistical analysis are shown in Table 2. In window W1, the difference ERP’s positive peak amplitude is significantly higher than zero with M = 4.509, SD = 3.55, t(12) = 4.580, p = ≤ 0.001. In window W2, the difference ERP’s negative peak amplitude is significantly lower than zero with M = −6.556, SD = 4.706, t(12) = − 5.023, p = ≤0.001. In window W3, the difference ERP’s negative peak amplitude again significantly lower than zero with M = −6.406, SD = 7.242, t(12) = −3.190, p = 0.008.

The literature on ErrP has highlighted several factors that affect its amplitude, such as speed and accuracy of task execution (Gehring et al., 1993, 2018). Thus, a step further, it has been investigated if the amount of exercise executed can affect the amplitude of failure events peaks in the three-time windows. It is understood that the failure trials in which 4 s were given to complete the exercise (named 4-s failure trials onwards), the amount of exercise executed was higher before the onset of ‘Time’s up!’ screen as there was more time in comparison to the failure trials in which 2 s were given to complete the exercise (named 2-s failure trials onwards). Therefore, the 2-s failure trials and the 4-s failure trials were separated and compared. The grand-average ERPs with its SD as well as the individual ERPs of 2- and 4-s failure trials at the Cz electrode location have been shown in Figures 5A,B respectively. Results of only first six patients have been shown to prevent over-burdening the figure, while we got similar outcomes for each of the 13 patients. Paired-samples t-test has been performed to evaluate the significance of the difference in the amplitudes of 2- and 4-s trials in the three-time windows. Shapiro–Wilk test (α = 0.05) was used to confirm the normal distribution of the mean samples before the statistical analysis, and the results have been shown in Table 3. Also, skewness and kurtosis z-value of the mean samples was within ± 1.96 for the three-time windows except for the 4-s failure trials skewness z-value in the window W2 which was −2.028. However, paired-samples t-test was used in this case as well to check any significant difference. The results of the statistical analysis are shown in Table 3. In window W1, the 2-s trials M and SD are −0.502 and 6.980 respectively and 4-s trials M and SD are 2.166 and 7.280 respectively; however, the difference is statistically insignificant with t(12) = −0.832, p = 0.422. In window W2, the 2-s trials M and SD are −8.366 and 7.258 respectively and 4-s trials M and SD are −3.679 and 10.801 respectively; nevertheless, the difference is statistically insignificant with t(12) = −1.161, p = 0.268. In window W3, the 2-s trials M and SD are −6.020 and 8.020 respectively and 4-s trials M and SD are −2.887 and 7.348 respectively; however, again the difference is statistically insignificant with t(12) = − 0.970, p = 0.351.

FIGURE 5

Figure 5. (A) The grand-average ERPs with its SD as well as individual ERPs of the 2-s failure trials. (B) The grand-average ERPs with its SD as well as individual ERPs of the 4-s failure trials. Results of only the first six patients have been shown to prevent over-burdening the figure, while we got similar outcomes for each of the 13 patients.

TABLE 3

Table 3. Shapiro–Wilk test p-value, mean, standard deviation, paired samples t-test statistics (two-tailed) results of the difference in 2-s failure events’ peak amplitudes and 4-s failure events’ peak amplitudes in the three time-windows.

Single-trial EEG responses of patient 2 have been shown in Figure 6. To assess the feasibility of detecting ErrP signals in a single-trial, average ITC of the most prominent peaks of the single-trial failure and success responses of patient 2 were calculated, in the three-time windows W1, W2, and W3. For the failure trials, the average ITC has been observed to be 0.3486, and for the success responses, it has been observed to be 0.4187. To make a comparison, Zhang et al. (2014) reported a maximum ITC 0.36 in their ErrP study involving a monetary gambling task and Yazmir and Reiner (2017) reported a maximum ITC 0.3 in their virtual throwing game ErrP study. It indicates that the novel ErrP signal of this study can be detected in a single-trial with good accuracy using state-of-the-art machine learning methods.

FIGURE 6

Figure 6. Single-trial EEG responses of patient 2. (A) Failure trials, (B) Success trials. In the three-time windows W1, W2 and W3, the average ITC of the failure trials has been observed to be 0.3486, and for the success trials, it has been observed to be 0.4187.

Discussion

The goal of this study was to check whether an error related potential can be elicited following a failure trial in a post-stroke rehabilitation movement against the success. The obtained results of this study are promising. Overall, it has been found that the ERP, associated with the failure related events and the ERP associated with the success related events, differ from each other significantly and the resulting difference ERP resembles the previously reported ErrP signals and has some unique features which are novel to this new ErrP signal.

In the grand average difference ERP, first positive peak exhibits at 100 ms post the ‘Time’s up!’ screen appeared. After that, a negative peak occurred at 135 ms and finally, a broader negative peak appeared at about 410 ms. These findings are consistent with that of Ferrez and Del R. Millán (2008) who reported a series of positive and negative peaks related to interaction ErrP around 200 to 450 ms post a reference event in a pressing button computer game. A comparison of previous studies has shown that the ErrPs is elicited sooner in the tasks that demand participants to respond quickly and accurately (Vocat et al., 2008; Chavarriaga and Millán, 2010; Ladouceur et al., 2018; Omedes et al., 2018) which is in accordance with the present results. Furthermore, Omedes et al. (2018) also reported a N400 peak in the context of observation of erroneous action in a 3D reaching task, which is consistent with our studies 410 ms negative peak. Prior studies (Olvet and Hajcak, 2008; Weinberg et al., 2015; Rosburg et al., 2018) that have observed the difference ERPs (error/failure minus correct/success trials) noted a negative peak followed by positive peak post-reference event in error trials ERPs. Surprisingly, in this study, reverse polarities have been observed in the failure trial ERP, i.e., a positive peak followed by a negative peak which could be unique to the novel task methodology opted in this experiment. Interestingly, the difference ERP’s final negative peak around 410 ms corroborates the earlier findings (Ferrez and Del R. Millán, 2008; Chavarriaga and Millán, 2010; Omedes et al., 2018). Although the final negative peak around 410 ms popularly, known as N400 is believed to be linked with semantic mismatch (Kutas and Federmeier, 2011), however, N400 existence was also observed during erroneous actions when goal-oriented actions were involved (Balconi and Vitaloni, 2014).

Event-related potentials of both success and failure trials have three prominent peaks. The scalp distribution of the first peak at 100 ms has fronto-central distribution whereas the peaks at 135 and 410 ms have mainly centro-parietal scalp distribution. This finding is consistent with that of Omedes et al. (2018), who also observed fronto-central and centro-parietal scalp distribution of ErrP peaks.

The success events induced power was significant in 3–22, 18–30, 34–38, and 42–50 Hz frequency ranges at various times. On the other hand, the failure events induced power was significant in 11–24, 16–34, and 42–50 Hz at various times. The alpha-band power induced following the success events is in accords with the earlier observations of Carp and Compton (2009), which showed that alpha-band power in the range 8–14 Hz increase and then decrease following a correct response in a six-choice Stroop task; the trend was majorly absent in erroneous response which is consistent with our findings. A possible explanation for this is that in the success response patients completed the exercise before the onset of the ‘Time’s up!’ screen. Therefore, the inter-trial resting interval may have disengaged the patients, and notably, alpha power is linked with the alertness (Carp and Compton, 2009). On the contrary, following the failure response patients’ alertness may have been maintained throughout that caused absence of the quadratic pattern of the induced alpha power. Furthermore, following a failure response, a periodic rebound has been observed in the beta activity that may have been caused due to inhibition of the ongoing motor activity — a trend which has been absent following a success response. A similar activity was observed by Koelewijn et al. (2008) following an erroneous execution of a button-pressing task. It suggests that higher order metal-functions are involved in the evaluation of motor activities even after the execution has stopped.

No significant difference has been observed in the 2- and 4-s failure trials. These findings support our methodology of a direct comparison of 2- and 4-s failure trials taken collectively with the success trials can be made.

Implications for BCI

In the form of a new task situation that elicits ErrP, the results of this study have demonstrated a clear difference in the failure responses from the success responses. We expect that the results from this study can be transferred to a assist-as-needed (AAN) robot-therapy based rehabilitation program and may be used to improve the current state-of-the-art methods in recognizing the point when the patient has failed in performing the rehabilitation exercise and needed assistance.

The insights gained from this study will also assist developing an adaptive algorithm for AAN methods that can increase/decrease the assistance level on detecting the failure/success related EEG response, respectively. This may make the rehabilitation program continually challenging that may result in an increasingly engaged rehabilitation program that ultimately motivates stroke patients to train or practice longer (Wang et al., 2017).

As the ErrP response associated with the success and the failure events of this study do not demand any mental workload on the participants, therefore, it is expected that the findings of this study can be integrated with existing AAN modalities that may improve the efficacy of the overall rehabilitation program.

Study Limitations and Future Work

One can argue that the ErrP signals observed in this study are possibly due to the surprise of seeing the finished time-period instead of the failure on executing the rehabilitation exercise. However, it is notable that a surprise forms an integral part of the action-monitoring system of the human brain (Hayden et al., 2011). The experience of error in previous ErrP studies’ tasks such as in gesture controlled robot (Kim et al., 2017), virtual 3D reaching task (Omedes et al., 2018), monitoring cursor on a screen (Chavarriaga and Millán, 2010) had come with a surprise as well. A possible method to reduce this confounding factor is to run the experiment in an asynchronous manner in which the patient will have autonomy of performing rehabilitation exercises, which identifies an area of further research.

Another limitation of the study is the diversity in the participants in terms of days since they suffered a stroke and Brunnstrom stage, which can be possible sources of variability in the EEG signal. The effects of patients’ age, Brunnstrom stage, recovery days, and interest on the amplitude of the ErrP signal elicited in our novel task condition can be studied in the future work.

Another natural progression of this study is to work on detecting the failure and success trials in a single-trial, including against resting periods, so that the ErrP signal can be used in assist-as-needed robot-therapy in real-time. This is an important area for future research.

In EEG studies, reference selection during recording and analysis has always been a source of bias, which can potentially affect the shape of the EEG signals (Murray et al., 2008). Several studies advocate the use of bipolar electrodes to reduce the noise level instead of unipolar references when neural signals only from the cortical sources are of interest (Yao et al., 2019). Several other studies suggest the use of average reference and reference electrode standardization technique (REST) (Yao et al., 2019). Multiple reference techniques should be employed, and their results should be compared to get a reference-bias-free estimate of EEG responses.

Conclusion

In this paper, we have reported the existence of the error-related potential signal in a new task condition that is when a participant is unable to complete a physical movement in a given time. We conducted experiments on the stroke patients that involved performing a physical rehabilitation exercise. All patients had various levels of limb function impairment. We applied MATLAB and EEGLAB based signal processing techniques to analyze the EEG, EOG, and sEMG data. We used parametric statistical methods to evaluate the statistical significance of the findings. None of the previously reported task conditions that elicit ErrP involve continuous physical exercise movements. To the best of authors’ knowledge, for the first time, an ErrP has been observed while performing rehabilitation exercise. The research has also shown that the reported ErrP has reverse polarity characteristics for the first two peaks of failure trial ERP which is unique to this new task condition ErrP and a final negative peak which is consistent with the results reported in the literature. In summary, the findings reported here shed light on a new type of task that evokes ErrP using which failure events from success events while performing a physical exercise can be distinguished.

Data Availability Statement

The datasets generated for this study are available on request to the corresponding author.

Ethics Statement

The experimental protocol was approved by the Ethics Committee of 2nd Hospital of Jiaxing and in accordance with Declaration of Helsinki. All participants were informed about the experiment, and they provided their oral and written consents before the start of the experiment.

Author Contributions

AK contributed to the experiment design, data collection and analysis, and manuscript preparation. QF contributed to idea formation, experiment design, results discussion, and manuscript preparation. JF and XG contributed to ethics approval, patient recruitment, data collection, and medical guidance. EP contributed to results discussion and manuscript preparation.

Funding

This work was supported by the National Key R&D Program of China (Grant No. 2018YFC2001600).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors would like to thank Mr. Guojun Yu from Shantou University and staff of 2nd Hospital of Jiaxing, China, for assistance in the data collection.

References

Balconi, M., and Vitaloni, S. (2014). Dorsolateral pFC and the representation of the incorrect use of an object: the transcranial direct current stimulation effect on N400 for visual and linguistic stimuli. J. Cogn. Neurosci. 26, 305–318. doi: 10.1162/jocn_a_00500

PubMed Abstract | CrossRef Full Text | Google Scholar

Basteris, A., Nijenhuis, S. M., Stienen, A. H. A., Buurke, J. H., Prange, G. B., and Amirabdollahian, F. (2014). Training modalities in robot-mediated upper limb rehabilitation in stroke: a framework for classification based on a systematic review. J. Neuroeng. Rehabil. 11:111. doi: 10.1186/1743-0003-11-111

PubMed Abstract | CrossRef Full Text | Google Scholar

Carp, J., and Compton, R. J. (2009). Alpha power is influenced by performance errors. Psychophysiology 46, 336–343. doi: 10.1111/j.1469-8986.2008.00773.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Chang, C. Y., Hsu, S. H., Pion-Tonachini, L., and Jung, T. P. (2018). “Evaluation of artifact subspace reconstruction for automatic EEG artifact removal,” in Proceesdings of the 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (Piscataway, NJ: IEEE), 1242–1245.

Google Scholar

Chavarriaga, R., and Millán, J. D. R. (2010). Learning from EEG error-related potentials in noninvasive brain-computer interfaces. IEEE Trans. Neural Syst. Rehabil. Eng. 18, 381–388. doi: 10.1109/TNSRE.2010.2053387

PubMed Abstract | CrossRef Full Text | Google Scholar

Chavarriaga, R., Sobolewski, A., and Millán, J. R. (2014). Errare machinale est: the use of error-related potentials in brain-machine interfaces. Front. Neurosci. 8:208. doi: 10.3389/fnins.2014.00208

PubMed Abstract | CrossRef Full Text | Google Scholar

Cramer, D. (1998). Fundamental Statistics for Social Research: Step-by-Step Calculations and Computer Techniques Using SPSS for Windows. Abingdon: Routledge.

Google Scholar

Delorme, A., and Makeig, S. (2004). EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods 134, 9–21.

PubMed Abstract | Google Scholar

Delorme, A., Sejnowski, T., and Makeig, S. (2007). Enhanced detection of artifacts in EEG data using higher-order statistics and independent component analysis. Neuroimage 34, 1443–1449.

PubMed Abstract | Google Scholar

Dickter, C. L., and Kieffaber, P. D. (2014). EEG Methods for the Psychological Sciences. Thousand Oaks: SAGE Publications Inc.

Google Scholar

Farquhar, J., and Hill, N. J. (2013). Interactions between pre-processing and classification methods for event-related-potential classification: best-practice guidelines for brain-computer interfacing. Neuroinformatics 11, 175–192. doi: 10.1007/s12021-012-9171-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Ferrez, P. W., Del, R., and Millán, J. (2008). Error-related EEG potentials generated during simulated brain-computer interaction. IEEE Trans. Biomed. Eng. 55, 923–929. doi: 10.1109/TBME.2007.908083

PubMed Abstract | CrossRef Full Text | Google Scholar

Gehring, W. J., Goss, B., Coles, M. G. H., Meyer, D. E., and Donchin, E. (1993). A neural system for error detection and compensation. Psychol. Sci. 4, 385–390. doi: 10.1118/1.4868459

PubMed Abstract | CrossRef Full Text | Google Scholar

Gehring, W. J., Goss, B., Coles, M. G. H., Meyer, D. E., and Donchin, E. (2018). The error-related negativity. Perspect. Psychol. Sci. 13, 200–204.

Google Scholar

Grosmaire, A. G., and Duret, C. (2017). Does assist-as-needed upper limb robotic therapy promote participation in repetitive activity-based motor training in sub-acute stroke patients with severe paresis? Neurorehabilitation 41, 31–39. doi: 10.3233/NRE-171454

PubMed Abstract | CrossRef Full Text | Google Scholar

Hayden, B. Y., Heilbronner, S. R., Pearson, J. M., and Platt, M. L. (2011). Surprise signals in anterior cingulate cortex: neuronal encoding of unsigned reward prediction errors driving adjustment in behavior. J. Neurosci. 31, 4178–4187. doi: 10.1523/JNEUROSCI.4652-10.2011

PubMed Abstract | CrossRef Full Text | Google Scholar

Johnson, W., Onuma, O., Owolabi, M., and Sachdev, S. (2016). Stroke: a global response is needed. Bull. World Health Organ. 94, 634A–635A.

Google Scholar

Kalaria, R. N., Akinyemi, R., and Ihara, M. (2016). Stroke injury, cognitive impairment and vascular dementia. Biochim. Biophys. Acta Mol. Basis Dis. 1862, 915–925.

Google Scholar

Kim, S. K., Kirchner, E. A., Stefes, A., and Kirchner, F. (2017). Intrinsic interactive reinforcement learning-Using error-related potentials for real world human-robot interaction. Sci. Rep. 7:17562.

Google Scholar

Koelewijn, T., Van Schie, H. T., Bekkering, H., Oostenveld, R., and Jensen, O. (2008). Motor-cortical beta oscillations are modulated by correctness of observed action. Neuroimage 40, 767–775. doi: 10.1016/j.neuroimage.2007.12.018

PubMed Abstract | CrossRef Full Text | Google Scholar

Kutas, M., and Federmeier, K. D. (2011). Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP). Ann. Rev. Psychol. 62, 621–647. doi: 10.1146/annurev.psych.093008.131123

PubMed Abstract | CrossRef Full Text | Google Scholar

Ladouceur, C. D., Tan, P. Z., Sharma, V., Bylsma, L. M., Silk, J. S., Siegle, G. J., et al. (2018). Error-related brain activity in pediatric anxiety disorders remains elevated following individual therapy: a randomized clinical trial. J. Child Psychol. Psychiatry 59, 1152–1161. doi: 10.1111/jcpp.12900

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, X., Zhou, P., and Aruin, A. S. (2007). Teager-kaiser energy operation of surface EMG improves muscle activity onset detection. Ann. Biomed. Eng. 35, 1532–1538.

PubMed Abstract | Google Scholar

Liu, D., Chen, W., Pei, Z., and Wang, J. (2017). A brain-controlled lower-limb exoskeleton for human gait training. Rev. Sci. Instrum. 88:104302. doi: 10.1063/1.5006461

PubMed Abstract | CrossRef Full Text | Google Scholar

Lopez-Larraz, E., Iturrate, I., Montesano, L., and Minguez, J. (2010). Real-time recognition of feedback error-related potentials during a time-estimation task. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2010, 2670–2673. doi: 10.1109/IEMBS.2010.5626623

PubMed Abstract | CrossRef Full Text | Google Scholar

Meyer, A., Riesel, A., and Proudfit, G. H. (2013). Reliability of the ERN across multiple tasks as a function of increasing errors. Psychophysiology 50, 1220–1225.

PubMed Abstract | Google Scholar

Milekovic, T., Ball, T., Schulze-Bonhage, A., Aertsen, A., and Mehring, C. (2013). Detection of error related neuronal responses recorded by electrocorticography in humans during continuous movements. PLoS One 8:e55235. doi: 10.1371/journal.pone.0055235

PubMed Abstract | CrossRef Full Text | Google Scholar

Mullen, T., Kothe, C., Chi, Y. M., Ojeda, A., Kerth, T., Makeig, S., et al. (2013). Real-time modeling and 3D visualization of source dynamics and connectivity using wearable EEG. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2013, 2184–2187. doi: 10.1109/EMBC.2013.6609968

PubMed Abstract | CrossRef Full Text | Google Scholar

Murray, M. M., Brunet, D., and Michel, C. M. (2008). Topographic ERP analyses: a step-by-step tutorial review. Brain Topogr. 20, 249–264. doi: 10.1007/s10548-008-0054-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Olvet, D. M., and Hajcak, G. (2008). The error-related negativity (ERN) and psychopathology: toward an endophenotype. Clin. Psychol. Rev. 28, 1343–1354. doi: 10.1016/j.cpr.2008.07.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Olvet, D. M., and Hajcak, G. (2009). The stability of error-related brain activity with increasing trials. Psychophysiology 46, 957–961. doi: 10.1111/j.1469-8986.2009.00848.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Omedes, J., Schwarz, A., Müller-Putz, G. R., and Montesano, L. (2018). Factors that affect error potentials during a grasping task: toward a hybrid natural movement decoding BCI. J. Neural Eng. 15:046023. doi: 10.1088/1741-2552/aac1a1

PubMed Abstract | CrossRef Full Text | Google Scholar

Rosburg, T., Deuring, G., Boillat, C., Lemoine, P., Falkenstein, M., Graf, M., et al. (2018). Inhibition and attentional control in pedophilic child sexual offenders – An event-related potential study. Clin. Neurophysiol. 129, 1990–1998. doi: 10.1016/j.clinph.2018.06.029

PubMed Abstract | CrossRef Full Text | Google Scholar

Roset, S. A., Gant, K., Prasad, A., and Sanchez, J. C. (2014). An adaptive brain actuated system for augmenting rehabilitation. Front. Neurosci. 8:415. doi: 10.3389/fnins.2014.00415

PubMed Abstract | CrossRef Full Text | Google Scholar

Shapiro, S. S., and Wilk, M. B. (1965). An analysis of variance test for normality (complete samples). Biometrika 52, 591–611.

Google Scholar

Solnik, S., Devita, P., Rider, P., Long, B., and Hortobágyi, T. (2008). Teager-Kaiser operator improves the accuracy of EMG onset detection independent of signal-to-noise ratio. Acta Bioeng. Biomech. 10, 65–68.

PubMed Abstract | Google Scholar

Sörnmo, L., and Laguna, P. (2005). Bioelectrical Signal Processing in Cardiac and Neurological Applications. Cambridge: Elsevier.

Google Scholar

Spüler, M., and Niethammer, C. (2015). Error-related potentials during continuous feedback: using EEG to detect errors of different type and severity. Front. Hum. Neurosci. 9:155. doi: 10.3389/fnhum.2015.00155

PubMed Abstract | CrossRef Full Text | Google Scholar

Tacchino, G., Gandolla, M., Coelli, S., Barbieri, R., Pedrocchi, A., and Bianchi, A. M. (2017). EEG analysis during active and assisted repetitive movements: evidence for differences in neural engagement. IEEE Trans. Neural Syst. Rehabil. Eng. 25, 761–771. doi: 10.1109/TNSRE.2016.2597157

PubMed Abstract | CrossRef Full Text | Google Scholar

Vocat, R., Pourtois, G., and Vuilleumier, P. (2008). Unavoidable errors: a spatio-temporal analysis of time-course and neural sources of evoked potentials associated with error processing in a speeded task. Neuropsychologia 46, 2545–2555. doi: 10.1016/j.neuropsychologia.2008.04.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, Q., Markopoulos, P., Yu, B., Chen, W., and Timmermans, A. (2017). Interactive wearable systems for upper body rehabilitation: a systematic review. J. Neuroeng. Rehabil. 14:20. doi: 10.1186/s12984-017-0229-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Weinberg, A., Dieterich, R., and Riesel, A. (2015). Error-related brain activity in the age of RDoC: a review of the literature. Int. J. Psychophysiol. 98, 276–299. doi: 10.1016/j.ijpsycho.2015.02.029

PubMed Abstract | CrossRef Full Text | Google Scholar

Widmann, A., Schröger, E., and Maess, B. (2015). Digital filter design for electrophysiological data - a practical approach. J. Neurosci. Methods 250, 34–46. doi: 10.1016/j.jneumeth.2014.08.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Yao, D., Qin, Y., Hu, S., Dong, L., Bringas Vega, M. L., and Valdés Sosa, P. A. (2019). Which reference should we use for EEG and ERP practice? Brain Topogr. 32, 530–549. doi: 10.1007/s10548-019-00707-x

PubMed Abstract | CrossRef Full Text | Google Scholar

Yazmir, B., and Reiner, M. (2017). I act, therefore I err: EEG correlates of success and failure in a virtual throwing game. Int. J. Psychophysiol. 122, 32–41. doi: 10.1016/j.ijpsycho.2017.02.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Yue, Z., Zhang, X., and Wang, J. (2017). Hand rehabilitation robotics on poststroke motor recovery. Behav. Neurol. 2017:3908135. doi: 10.1155/2017/3908135

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhang, D., Gu, R., Broster, L. S., Jiang, Y., Luo, W., Zhang, J., et al. (2014). Linking brain electrical signals elicited by current outcomes with future risk decision-making. Front. Behav. Neurosci. 8:84. doi: 10.3389/fnbeh.2014.00084

PubMed Abstract | CrossRef Full Text | Google Scholar