Efficacy and Brain Imaging Correlates of an Immersive Motor Imagery BCI-Driven VR System for Upper Limb Motor Rehabilitation: A Clinical Case Report

Introduction

Worldwide, stroke is a leading cause of adult long-term disability (Mozaffarian et al., 2015). From those who survive, an increased number is suffering with severe cognitive and motor impairments, resulting in loss of independence in their daily life such as self-care tasks and participation in social activities (Miller et al., 2010). Rehabilitation following stroke is a multidisciplinary approach to disability which focuses on recovery of independence. There is increasing evidence that chronic stoke patients maintain brain plasticity, meaning that there is still potential for additional recovery (Page et al., 2004). Traditional motor rehabilitation is applied through physical therapy and/or occupational therapy. Current approaches of motor rehabilitation include functional training, strengthening exercises, and range of movement exercises. In addition, techniques based on postural control, stages of motor learning, and movement patterns have been proposed such as in the Bobath concept and Bunnstrom approach (amongst others) (Bobath, 1990). After patients complete subacute rehabilitation programs, many still show significant upper limb motor impairment. This has important functional implications that ultimately reduce their quality of life. Therefore, alternative methods to maximize brain plasticity after stroke need to be developed.

So far, there is growing evidence that action observation (AO) (Celnik et al., 2008) and motor imagery (MI) improve motor function (Mizuguchi and Kanosue, 2017) but techniques based on this paradigm are not widespread in clinical settings. As motor recovery is a learning process, the potential of MI as a training paradigm relies on the availability of an efficient feedback system. To date, a number of studies have demonstrated the positive impact of virtual-reality (VR) based on neuroscientific grounds on recovery, with proven effectiveness in the stroke population (Bermúdez i Badia et al., 2016). However, patients with no active movement cannot benefit from current VR tools due to low range of motion, pain, fatigue, etc. (Trompetto et al., 2014). Consequently, the idea of directly training the central nervous system was promoted by establishing an alternative pathway between the user’s brain and a computer system.

This is possible by using electroencephalography (EEG)-based Brain-Computer Interfaces (BCIs), since they can provide an alternative non-muscular channel for communication and control to the external world (Wolpaw et al., 2002), while they could also provide a cost-effective solution for training (Vourvopoulos and Bermúdez, 2016b). In rehabilitation, BCIs could offer a unique tool for rehabilitation since they can stimulate neural networks through the activation of mirror neurons (Rizzolatti and Craighero, 2004) by means of action-observation (Kim et al., 2016), motor-intent and motor-imagery (Neuper et al., 2009), that could potentially lead to post-stroke motor recovery. Thus, BCIs could provide a backdoor to the activation of motor neural circuits that are not stimulated through traditional rehabilitation techniques.

In EEG-based BCI systems for motor rehabilitation, Alpha (8–12 Hz) and Beta (12–30 Hz) EEG rhythms are utilized since they are related to motor planning and execution (McFarland et al., 2000). During a motor attempt or motor imagery, the temporal pattern of the Alpha rhythms desynchronizes. This rhythm is also named Rolandic Mu-rhythm or the sensorimotor rhythm (SMR) because of its localization over the sensorimotor cortices. Mu-rhythms are considered indirect indications of functioning of the mirror neuron system and general sensorimotor activity (Kropotov, 2016). These are often detected together with Beta rhythm changes in the form of an event-related desynchronization (ERD) when a motor action is executed (Pfurtscheller and Lopes da Silva, 1999). These EEG patterns are primarily detected during task-based EEG (e.g., when the participant is actively moving or imagining movement) and they are of high importance in MI-BCIs for motor rehabilitation.

A meta-analysis of nine studies (combined N = 235, sample size variation 14 to 47) evaluated the clinical effectiveness of BCI-based rehabilitation of patients with post-stroke hemiparesis/hemiplegia and concluded that BCI technology could be effective compared to conventional treatment (Cervera et al., 2018). This included ischemic and hemorrhagic stroke in both subacute and chronic stages of stoke, between 2 to 8 weeks. Moreover, there is evidence that BCI-based rehabilitation promotes long-lasting improvements in motor function of chronic stroke patients with severe paresis (Ramos-Murguialday et al., 2019), while overall BCI’s are starting to prove their efficacy as rehabilitative technologies in patients with severe motor impairments (Chaudhary et al., 2016).

The feedback modalities used for BCI motor rehabilitation include: non-embodied simple two-dimensional tariffs on a screen (Prasad et al., 2010; Mihara et al., 2013), embodied avatar representation of the patient on a screen or with augmented reality (Holper et al., 2010; Pichiorri et al., 2015), neuromuscular electrical stimulation (NMES) (Kim et al., 2016; Biasiucci et al., 2018). and robotic exoskeletal orthotic movement facilitation (Ramos-Murguialday et al., 2013; Várkuti et al., 2013; Ang et al., 2015). In addition, it has been shown that multimodal feedback lead to a significantly better performance in motor-imagery (Sollfrank et al., 2016) but also multimodal feedback combined with motor-priming, (Vourvopoulos and Bermúdez, 2016a). However, there is no evidence which modalities are more efficient in stroke rehabilitation are.

Taking into account all previous findings in the effects of multimodal feedback in MI training, the purpose of this case study is to examine the effect of the MI paradigm as a treatment for post-stroke upper limb motor dysfunction using the NeuRow BCI-VR system. This is achieved through the acquisition of clinical scales, dynamics of EEG during the BCI treatment, and brain activation as measured by functional MRI (fMRI). NeuRow is an immersive VR environment for MI-BCI training that uses an embodied avatar representation of the patient arms and haptic feedback. The combination of MI-BCIs with VR can reinforce activation of motor brain areas, by promoting the illusion of physical movement and the sense of embodiment in VR (Slater, 2017), and hence further engaging specific neural networks and mobilizing the desired neuroplastic changes. Virtual representation of body parts paves the way to include action observation during treatment. Moreover, haptic feedback is added since a combination of feedback modalities could prove to be more effective in terms of motor-learning (Sigrist et al., 2013). Therefore, the target of this system is to be used by patients with low or no levels of motor control. With this integrated BCI-VR approach, severe cases of stroke survivors may be admitted to a VR rehabilitation program, complementing traditional treatment.

Methodology

Patient Profile

In this pilot study we recruited a 60 years old male patient with left hemiparesis following cerebral infarct in the right temporoparietal region 10 months before. The participant had corrected vision through eyewear, he had 4 years of schooling and his experience with computers was reported as low. Moreover, the patient was on a low dose of diazepam (5 mg at night to help sleep), dual antiplatelet therapy, anti-hypertensive drug and metformin. Hemiparesis was associated with reduced dexterity and fine motor function; however, sensitivity was not affected. Other sequelae of the stroke included hemiparetic gait and dysarthria. Moreover, a mild cognitive impairment was identified which did not interfere with his ability to perform the BCI-VR training. The patient had no other relevant comorbidities. Finally, the patient was undergoing physiotherapy and occupational therapy at the time of recruitment and had been treated with botulinum toxin infiltration 2 months before due to focal spasticity of the biceps brachii.

Intervention Protocol

The patient underwent a 3-weeks intervention with NeuRow, resulting in 10 BCI sessions of a 15 min of exposure in VR training per session. Clinical scales, motor imagery capability assessment, and functional -together with structural- MRI data had been gathered in three time-periods: (1) before (serving as baseline), (2) shortly after the intervention and (3) one-month after the intervention (to assess the presence of long-term changes). Finally, electroencephalographic (EEG) data had been gathered during all sessions, resulting in more than 20 datasets of brain electrical activity.

The experimental protocol was designed in collaboration with the local healthcare system of Madeira, Portugal (SESARAM) and approved by the scientific and ethic committees of the Central Hospital of Funchal. Finally, written informed consent was obtained from the participant upon recruitment for participating to the study but also for the publication of the case report in accordance with the 1964 Declaration of Helsinki.

Assessment Tools

A set of clinical scales were acquired including the following:

1. Montreal Cognitive Assessment (MoCA). MoCA is a cognitive screening tool, with a score range between 0 and 30 (a score greater than 26 is considered to be normal) validated also for the Portuguese population, (Nasreddine et al., 2005).

2. Modified Ashworth scale (MAS). MAS is a 6-point rating scale for measuring spasticity. The score range is 0, 1, 1+, 2, 3, and 4 (Ansari et al., 2008).

3. Fugl-Meyer Assessment (FMA). FMA is a stroke specific scale that assesses motor function, sensation, balance, joint range of motion and joint pain. The motor domain for the upper limb has a maximum score of 66 (Fugl-Meyer et al., 1975).

4. Stroke Impact Scale (SIS). SIS is a subjective scale of the perceived stroke impact and recovery as reported by the patient, validated for the Portuguese population. The score of each domain of the questionnaire ranges from 0 to 100 (Duncan et al., 1999).

5. Vividness of Movement Imagery Questionnaire (VMIQ2). VMIQ2 is an instrument that assess the capability of the participant to perform imagined movements from external perspective (EVI), internal perspective imagined movements (IVI) and finally, kinesthetic imagery (KI) (Roberts et al., 2008).

NeuRow BCI-VR System

EEG Acquisition

For EEG data acquisition, the Enobio 8 (Neuroelectrics, Barcelona, Spain) system was used. Enobio is a wearable wireless EEG sensor with 8 EEG channels for the recording and visualization of 24-bit EEG data at 500 Hz and a triaxial accelerometer. The spatial distribution of the electrodes followed the 10–20 system configuration (Klem et al., 1999) with the following electrodes over the somatosensory and motor areas: Frontal-Central (FC5, FC6), Central (C1, C2, C3, C4), and Central-Parietal (CP5, CP6) (Figure 1A). The EEG system was connected via Bluetooth to a dedicated desktop computer, responsible for the EEG signal processing and classification, streaming the data via UDP through the Reh@Panel (RehabNet Control Panel) for controlling the virtual environment. The Reh@Panel is a free tool that acts as a middleware between multiple interfaces and virtual environments (Vourvopoulos et al., 2013).

FIGURE 1

Figure 1. Experimental setup, including: (A) the wireless EEG system; (B) the Oculus HMD, together with headphones reproducing the ambient sound from the virtual environment; (C) the vibrotactile modules supported by a custom-made table-tray, similar to the wheelchair trays used for support; (D) the visual feedback with NeuRow game. A written informed consent was obtained for the publication of this image.

Head Mounted Display

For delivering the visual feedback to the user, the Oculus Rift DK1 HMD was used (Oculus VR, Irvine, CA, United States). The HMD is made of one 7″ 1280 × 800 60 Hz LCD display (640 × 800 resolution per eye), one aspheric acrylic lens per eye, 110^o Field of View (FOV), internal tracking through a gyroscope, accelerometer, and magnetometer, with a tracking frequency of 1000 Hz (Figure 1B).

Haptic Feedback

For delivering vibrotactile feedback, a custom module was used with out-of-the-box components including an Arduino Mega 2560 board and vibrating motors. The vibrating motors (10 mm diameter, 2.7 mm thick) performed at 11000 RPM at 5 V and were mounted inside cylindrical tubes -using 3D printed casing- which act as grasping objects for inducing the illusion of movement during the BCI task. In our setup, a pair of tubes with 12 cm (4.7 inches) of length and 3 cm (1.2 inches) diameter were used (Figure 1C).

VR Feedback

The BCI-VR task involved the use of NeuRow, a first-person BCI game (or neurogame). NeuRow uses a self-paced BCI paradigm and not a cue-based in order to increase the ecological validity of the training task. The actions are triggered whenever the user intends to move, like one would do in a real-life scenario. The BCI-VR task involves a boat rowing task through mental imagery with the goal of collecting as many flags as possible in a fixed amount of time. The auditory feedback involved two types of sounds. (1) background and ambient sounds of the water including rowing movement for increased realism and (2) event sounds for when the player was achieving a score by capturing a flag. NeuRow is a multiplatform virtual environment developed in Unity game engine (Unity Technologies, San Francisco, CA, United States). Finally, NeuRow was used under Windows OS, although it is optimized also for Android and Web browser by using the Reh@Panel (Vourvopoulos et al., 2016b).

The in-game interface includes time indication, score, navigational aids and first-person perspective of a virtual avatar representation of upper limbs rowing. NeuRow can be customized with different settings, depending on the experimental setup, BCI paradigm and running platform. NeuRow has two operating modes: (1) MI training and (2) online control. During training, the navigational arrow and the targets are removed to focus user’s attention only on the MI BCI-VR task. During online mode, the behavior of the boat can be changed by setting the heading speed, turn speed and cut-off angle of 45°. The cut-off angle is the allowed angle that the boat can be off-course with respect to the target flag before stopping. This serves as an additional safety feature to ensure that the user does not deviate from the target since the virtual environment is procedurally generated (Figure 1D).

NeuRow BCI-VR Protocol

Training Session

The first step of the training consisted on the acquisition of the raw EEG data to train a linear classifier to distinguish between Right and Left imagined hand movements. Throughout the training session, the user performed mental imagery of the corresponding hand rowing (left or right) at the same pace as the movement presented in VR. For each hand, the user is stimulated visually (VR action observation), auditorily, and haptically through the vibration on the corresponding hand. The training session was configured to acquire data in 24 blocks (epochs) per class (left or right hand imagery) in a randomized order (Figure 2).

FIGURE 2

Figure 2. BCI Protocol: (A) Intervention stages including the setup, training, resting period and finally the BCI task. (B) The training stages. (C) Training feedback distributed in 24 epochs per class (left|right)

Online EEG Data Processing

Following training, the acquired signals were processed with a bandpass filter (8–30 Hz), epoched into 4 s chunks following a stimulation event and used to compute a Common Spatial Patterns (CSP) filter. CSP was used in order to maximize the difference between the signals of the two classes (left vs right) for increased performance, and has become a standard tool in the use of MI-based BCIs (Lotte, 2014). Further, the training EEG data through the spatial filter feature vector were used to train a Linear Discriminant Analysis (LDA) classifier. LDA computed a vector which best discriminates between the two classes (left or right motor-imagery). Finally, the LDA output (hyperplane distance) was used as an input to the Adaptive Performance Engine (APE) (Ferreira et al., 2015). The APE module is composed a Bayesian Inference Layer (BIL) and a Finite State Machine (FSM) and it was added to smoothen binary decisions in a self-paced BCI scenario. BIL was computed as the likelihood of a specific LDA output belonging to each MI class with the following formula:

$$\text{P}(\text{i}|\text{LDA}\cdot \text{output}=\frac{{\text{MI}}_{\text{i}}(\text{LDA}\cdot \text{output},{\mathrm{\mu }}_{\text{j}},{\mathrm{\sigma }}_{\text{j}})*{\text{P}}_{\text{i}}}{{\displaystyle {\sum }_{\text{j}}{\text{MI}}_{\text{j}}(\text{LDA}\cdot \text{output},{\mathrm{\mu }}_{\text{j}},{\mathrm{\sigma }}_{\text{j}})}}\left(1\right)$$

where P_i indicates the prior probability of action i (0.5 for left vs. right MI). μ and σ are updated at each iteration, taking into account all previous history of the user for the given i MI action. LDA output indicates the output value of the LDA classifier.

Next, the posterior probabilities of each class (left or right MI) were sent to the FSM. Each state in the FSM represents the class, but also the confidence level associated to them.

EEG Offline Analysis

For the offline analysis, EEG signals were processed in MATLAB^® (The MathWorks, Natick, MA, United States) with the EEGLAB toolbox (Delorme and Makeig, 2004). After importing the data together with the channel info, a high-pass filter at 1 Hz was used to remove the “baseline drift” followed by line-noise and harmonics removal at 50 Hz. Furthermore, bad channels were rejected, and the data were subsequently re-referenced to average. Any potential missing channels had been interpolated to minimize a potential bias in the re-referencing stage through the pop_interp() function of EEGLAB which uses the spherical spline algorithm (Perrin et al., 1989). Next, an Independent Component Analysis (ICA) was performed for removing eye blinking, and movement artifacts (Makeig et al., 1996). For the independent components (IC) labeling, we performed manual artifact recognition by inspecting the different components both in the time and frequency domain but using also the ICLabel plugin from EEGLAB. The ICLabel plugin includes a trained classifier for EEG independent component which provides us with the probabilities that a component is being in any of the seven categories: brain; muscle; eye; heart; line noise; channel noise; other. The ICLabel classifier is trained by using crowd sourced data labeling or crowd labeling (Pion-Tonachini et al., 2017).

EEG Spectral Power

The Welch’s method for Power Spectral Density (PSD) of the power spectrum (Welch, 1967) was used for computing the average spectral power across the following frequency bands during the training task: Alpha (8–12 Hz), Beta (12–30 Hz), Theta (4–7 Hz), and Gamma (35–90 Hz). In addition, Alpha PSD was computed during resting-state before the trials.

Event-Related Desynchronization

In addition, the event-related synchronization/desynchronization (ERS/ERD) was extracted following the standard ERS/ERD method (Pfurtscheller and Aranibar, 1979) across the Mu band (8–12 Hz) and the Beta band (12–30 Hz). Both Mu and Beta power were extracted over C3 and C4 electrode locations. ERD was calculated by using the following formula:

$$\begin{array}{c}{\text{ERD}}_{\text{C}3|\text{C}4}=(\text{PowerC}3|\text{C}{4}_{\text{MotorActivity}}-\text{PowerC}3|\text{C}{4}_{\text{Baseline}})\hfill \\ /\text{PowerC}3|\text{C}{4}_{\text{Baseline}}\times 100\hfill \end{array}\left(2\right)$$

With positive numbers indicating ERS and negative numbers indicating ERD.

Moreover, ERDS maps were extracted as a time/frequency representation of ERD/ERS between 8 and 30 Hz (Graimann et al., 2002). ERDS maps are also known as ERSP (event-related spectral perturbation) and act as a generalization of the ERS/ERD (Makeig, 1993).

Lateralization Index

Lateralization between hemispheres is generally assessed by a lateralization index (LI), commonly used to describe the asymmetry of neural activation intensity. In this study, LI was computed on the basis of the relative power values detected over C3 and C4 electrodes (Doyle et al., 2005). In order to quantify lateralization, the spectral power at electrodes, contralateral to the movement side, was subtracted from that at ipsilateral electrodes. Moreover, we have extracted the LI for both Mu ERD and Beta ERD during training in terms of relative power, with its sign indicating contralateral ERS (negative) or ERD (positive) dominance (Doyle et al., 2005). For example, if the contralateral value is smaller than the ipsilateral value, then LI value is positive, indicating contralaterally desynchronized status in the evoked ERD during MI. Finally, the LI was computed as the average of the right and left side differences using the following formula:

$$\begin{array}{c}\text{LI}=[(\text{ERDC}{3}_{\text{Left movement}}-\text{ERDC}{4}_{\text{Left movement}}\hfill \\ +(\text{ERDC}{4}_{\text{Right movement}}-\text{ERDC}{3}_{\text{Right movement}})]/2\hfill \end{array}\left(3\right)$$

BCI Accuracy

For measuring the BCI performance during the training phase, the following formula was used for quantifying binary accuracy from the classifier:

$$\text{ACC}=(\text{TP}+\text{TN})/(\text{TP}+\text{TN}+\text{FP}+\text{FN})\left(4\right)$$

where: TP, true positive; FP, false positive; TN, true negative; FN, false negative.

fMRI Acquisition and Analysis

Task

The patient was submitted to five fMRI consecutive runs at each of the three assessment periods, while executing the following tasks: finger-tapping execution with the non-affected hand (ME-Right), finger-tapping motor imagery with both left and right hands, separately (MI-Left and MI-Right), and motor imagery with NeuRow with both left and right hands, separately (MI-NeuRow-Left and MI-NeuRow-Right). Thus, a total of 3(periods) × 5(runs) = 15 fMRI runs were performed. For all conditions, each run consisted of 7 cycles alternating a 20 s block of baseline (fixation cross) followed by a 20 s block of task; yielding a total duration of each fMRI run of 5.33 min (Pimentel et al., 2013).

In the finger-tapping execution condition (ME), the patient was instructed to execute a sequential finger-tapping task (index-middle-ring-little-index-middle-ring-little) from a first-person perspective with his non-affected arm (Figure 3a). In the finger-tapping motor imagery condition (MI), the patient had to imagine the kinesthetic experience of the previous finger-tapping task for the left and right hand separately, based on the provided stimulus/instruction (Figure 3b). Each trial started with a fixation cross, followed by a red arrow pointing to the left or right, indicating the beginning of a movement execution/imagination period, known as the standard Graz Motor Imagery protocol. In the last condition, the motor observation (MO), patient had to observe and imagine the kinesthetic experience of the rowing task from NeuRow from the first-person perspective for the left and right hand separately, based on the provided stimulus (Figure 3c). The fixation cross (baseline) from the motor imagery condition was replaced by NeuRow feedback in idle (floating without rowing); then, the corresponding movement was initiated for left or right rowing.

FIGURE 3

Figure 3. fMRI protocol. (a) Motor-Execution feedback, (b) Motor-Imagery feedback with directional arrows, (c) Motor-Observation feedback of NeuRow.

For the fMRI data acquisition, only visual and auditory feedback was delivered. The visual feedback was delivered through specialized MR-compatible fiber-optic goggles at a resolution of 640 × 480 pixels, synchronized with the console computer. The sound was delivered through MR compatible pneumatic headphones, but no haptic feedback was delivered during the fMRI session due to the lack of MR compatible equipment.

Image Acquisition

Imaging was performed on a 3T GE Signa HDxt MRI scanner (General Electrics Healthcare, Little Chalfont, United Kingdom) using 12-channel receive head coil. Functional images were acquired using a 2D multi-slice gradient-echo echo-planar imaging (EPI) sequence, with TR/TE = 2500/30 ms, flip angle = 90°c, and FOV = 224 × 224 mm², from 36 contiguous axial slices with interleaved acquisition, and 3.5 × 3.5 × 3.5 mm³ voxel size (0.75 mm slice gap), yielding whole-brain coverage. Whole-brain, structural images were acquired using a T₁-weighted 3D fast spoiled gradient-echo (FSPGR) sequence, with 1.0 × 1.0 × 0.6 mm³ voxel size.

Image Pre-processing

The following pre-processing steps were applied to the fMRI data recorded at all periods and runs prior to subsequent analyses. The first three volumes were discarded to allow the signal to reach the steady-state, and non-brain tissue was removed using FSL’s tool BET (Smith, 2002). Subsequently, slice timing and motion correction were performed using FSL’s tool MCFLIRT (Jenkinson et al., 2002), followed by high-pass temporal filtering with a cut-off period of 100 s and spatial smoothing using a Gaussian kernel with full width at half-maximum (FWHM) of 5 mm.

Because of the contamination by head motion as well as other physiological and instrumental artifacts, the pre-processed fMRI data were further submitted to an independent component analysis (ICA) cleaning procedure. Specifically, the probabilistic spatial ICA decomposition was applied, as implemented in the FSL’s tool MELODIC (Beckmann and Smith, 2004), with the default parameters, including the MELODIC’s automatic dimensionality estimation, as recommended in Salimi-Khorshidi et al. (2014). The purpose of ICA in this case was to separate neuronal from non-neuronal fMRI spatially independent components (ICs). The automatic classification of ICs was performed using FSL’s tool FIX (Salimi-Khorshidi et al., 2014). This tool extracts a large number of temporal and spatial features to be fed into a core classifier previously trained using hand-labeled components. The standard training weights were used, as the image acquisition parameters for our fMRI data. The non-neuronal related ICs were then automatically classified by FIX, and subsequently removed from the back-reconstruction step of the fMRI data, yielding cleaned fMRI data.

Statistical Analysis

For the purpose of mapping the brain areas involved in each fMRI run (left finger-tapping, left and right motor imagery, and left and right motor imagery with NeuRow), a general linear model (GLM) analysis of the pre-processed and ICA-based cleaned fMRI data was conducted. For each of the five fMRI runs, the explanatory variable of interest of the GLM was defined as a boxcar function, with 0’s during the baseline periods (fixation cross or boat floating without rowing) and 1’s during the task periods, and subsequently convolving it with a canonical double-gamma hemodynamic response function (Friston et al., 1995). A GLM containing the explanatory variable of interest, as well as the six motion parameters (rotation and translation of the head across the three main axes) estimated by MCFLIRT as confounding explanatory variables, was fit to the pre-processed data using FSL’s improved linear model (FILM). A t-test was performed on the parameter estimate for the explanatory variable of interest in each voxel and converted to Z-score. The resulting statistical parametric maps were subjected to cluster thresholding (voxel Z > 2.3, cluster p < 0.05) in order to yield the brain activation maps associated with each task (Woolrich et al., 2001).

The quantification of brain activation with each task was performed for each cerebral hemisphere separately and was based on the number of voxels exhibiting significant activation (voxels surviving the statistical thresholding; N_vox), and the maximum Z-score (Z_max). While the former approximately reflects the extent of brain activation, the latter approximately reflects its intensity.

Results

Here we present the results from the clinical scales, the MI capability assessment and the fMRI data, obtained in the three time-periods (pre; post; follow-up). In addition, we present results of the BCI performance over all sessions and the extracted EEG data, compared between the first and the last session.

Clinical Scales

In terms of motor domain as extracted by the FMA scale for the upper extremity (FMA-UE), the patient showed an improvement of 9 points at the end of the intervention (pre: 31, post: 40), followed by an improvement of 4 points (follow-up: 44) after 1 month (Table 1). This improvement is within the estimated clinically important difference (CID) scores, ranging between 4.25 and 7.25 points (Page et al., 2012). Moreover, by comparing the CID with the mean scores of prior BCI studies (M = 7.5, SD = 3.6) together with control groups (M = 4, SD = 2.2) (Mihara et al., 2013; Ramos-Murguialday et al., 2013; Ang et al., 2014, 2015; Li et al., 2014; Pichiorri et al., 2015; Frolov et al., 2016; Kim et al., 2016; Leeb et al., 2016), we are able to identify a big improvement of the patient in relation to both groups (Supplementary Figure 3).

TABLE 1

Table 1. Clinical scales: Fugl-Meyer upper limb scale (FMA), Montreal Cognitive Assessment (MoCA) and Modified Ashworth Scale (MAS).

Concerning spasticity, muscle tonus was increased but did not interfere with range of motion. SIS showed a conspicuous increase in the strength domain, however, the overall subjective assessment of the patient over his recovery remained constant (Table 2).

TABLE 2

Table 2. Stroke impact scale SIS subscales.

Motor-Imagery Capability

VMIQ Pre-Post-follow

The capability for vivid MI, was assessed through 3 sub-scales of VMIQ-2, external visual imagery (EVI), internal visual imagery (IVI) and finally kinesthetic imagery (KI) (Table 3). For EVI an increased visual capability was reported after the intervention while maintained in the follow-up (pre:19, post: 47, follow-up: 47). Further, the IVI had a small increase post-intervention but returned to the same level after 1 month (pre:47, post: 48, follow-up: 47). Regarding KI, while it remained stable at the pre-post assessment, a lower level was reported at the follow-up (pre:47, post: 47, follow-up: 39) accompanied by a slight decrease in KI vividness. This may be explained by the fact that during the one-month period after intervention, the patient did not undergo any BCI training nor exercises involving kinesthetic imagery. Nonetheless, there was still a strong use of external and internal imagery.

TABLE 3

Table 3. VMIQ-2 subscales of External Visual Imagery (EVI), Internal Visual Imagery (IVI) and Kinesthetic Imagery (KI).

Comparison With Healthy Participants

Comparing the MI capability data of the VMIQ-2 questionnaire with a group of healthy participants (N = 8) that underwent the same BCI protocol from a previous study (Vourvopoulos et al., 2016b), we can estimate a “healthy” range for motor-imagery capability of healthy population as a reference (Figure 4). Concerning the difference in EVI comparing pre-post assessments of our patient, we can observe a pronounced leap after the BCI-VR intervention, overpassing the average score of the healthy group (Figure 4). In contrast, the IVI and KI scores (that showed to be stable), are within the healthy range of the reported motor-imagery capability.

FIGURE 4

Figure 4. VMIQ-2 subscales comparison of Pre, Post and Follow-up scores with healthy data

BCI Performance

Classifier Performance

The overall classification performance during the 10 training sessions, was kept relatively low (M = 60, SD = 5.7) (Figure 5). Since we trained out classifier with features originating from both beta and mu bands, we wanted to investigate if there is a dominant frequency that could help achieving increased classification performance. A pair-samples t-test over C3 [t(9) = −0.9968, p = 0.34] and C4 [t(9) = 1.0878, p = 0.3049] during MI of the paretic arm, revealed non-significant differences between beta and mu ERD (Supplementary Figure 1).

FIGURE 5

Figure 5. (i) LDA classification performance over time within 10 sessions, (ii) distribution of performance.

Next, we compared the classification score across all sessions with the mean of two groups of healthy users. First with a group of participants (N = 8) that underwent the same BCI protocol with NeuRow (Vourvopoulos et al., 2016b) from a previous study (VR group) and secondly with a group of a study (N = 12) (Vourvopoulos et al., 2016a) that used the same feature extraction method (band power with CSP) and classification (LDA) for two-classes (left/right hand) MI data but not with VR feedback (non-VR group).

An independent sample t-test revealed significant differences between the patient LDA score (M = 60, SD = 5.7) with the VR group (M = 76, SD = 3), t(16) = −7.121. p < 0.001, but also with the non-VR group (M = 68, SD = 7.7), t(20) = −2.730, p < 0.05. Moreover, a significant difference was also found between the healthy VR and non-VR groups, t(18) = −2.720. p < 0.05 (Figure 6). Despite data showing improved performance with VR, our patient performed close to healthy in non-VR settings.

FIGURE 6

Figure 6. LDA Comparison with healthy. Statistically significant differences between Case-study, VR and non-VR groups has been observed (^∗p < 0.05).

EEG

Resting state Alpha band modulation is found to be related with cognitive and motor performance in stroke patients (Dubovik et al., 2012, 2013). We therefore analyzed the resting state Alpha rhythm pre and -post intervention (Supplementary Figure 2). Our results showed an increase in modulation of Alpha from Pre (M = 0.44, SD = 0.05) to Post (M = 0.62, SD = 0.24) although a paired-samples t-test yielded no significant differences (p = 0.20).

Furthermore, by comparing the evoked EEG activity during training with a prior study with NeuRow (Vourvopoulos et al., 2016b), we can observe a consistent trend between the first and last session. EEG power from healthy participants using the same experimental apparatus, can be used as a proxy for “healthy” EEG modulation boundaries. For all EEG bands, we found that the EEG power on the first session (Pre) is in the lower quartile (Q1) of the distribution while on the last session the EEG power always increases, closer approximating the Median of the healthy distribution inside the Interquartile Range (IQR). Current results indicate a convergence toward the healthy group EEG power (Figure 7).

FIGURE 7

Figure 7. EEG spectral power comparison Pre-Post the intervention with healthy user data.

Since both movement and imagery are associated with Mu and Beta rhythm desynchronization (McFarland et al., 2000), by measuring ERD during the patients training, we anticipated stronger ERD (in terms of higher negative percentage compared to baseline) at the end of the intervention. Nonetheless, the evoked ERD had small power in both Mu and Beta bands (Figure 8).

FIGURE 8

Figure 8. ERS/ERD during Left and Right MI for both Mu and Beta bands.

The ERD activation maps were extracted as a time/frequency representation of the first and last session during MI from the affected hand (left). Maps illustrate a clear desynchronization in the band 8–30 Hz -compared to baseline- from the contralesional electrode (C3) but not the ipsilesional side (C4). Nonetheless, in the last session, ERD is decreased but it is also more balanced (Figure 9).

FIGURE 9

Figure 9. ERS/ERD activation maps during left (lesioned) hand motor-imagery. Significant ERD is illustrated with blue.

In terms of laterality, LI shows an ipsilateral ERD dominance, evolving toward a contralateral ERD over time, balancing for both Mu and Beta bands (Figure 10). Furthermore, a paired-samples t-test revealed significant differences between the first and the last session in terms of LI ERD. Specifically, Beta band had increased ipsilateral ERD dominance in the first session (M = −2.82, SD = 2.3), balancing for both hemispheres in the last session (M = −0.6, SD = 0.3), t(199) = −16.921, p < 0.001. Similar trend for Mu band between the first (M = −1.8, SD = 3.1) and the last session (M = 0.41, SD = 0.5), t(199) = −9.832, p < 0.001.

FIGURE 10

Figure 10. Lateralization index for Mu and Beta bands across all sessions.

fMRI

The quantification of brain activation (N_vox and Z_max values) for the finger-tapping motor imagery conditions without NeuRow (MI-Left and MI-Right) and with NeuRow (MI-NeuRow-Left and MI-NeuRow Right) is depicted in Table 4; in Figure 11, the brain activation maps for these conditions are presented, separately for the left and right hand.

TABLE 4

Table 4. Quantification of brain activation in terms of number of voxels exhibiting significant activation (N_vox) maximum Z-score (Z_max) for the finger-tapping motor imagery conditions (MI-Left and MI-Right) and NeuRow condition, displaying the BCI-VR feedback (NeuRow-Left and NeuRow-Right).

FIGURE 11

Figure 11. fMRI activation maps of the motor imagery condition for the left and right hand, at the pre-intervention, post-intervention and follow-up recording sessions. The N_vox and Z_max values are associated with the hemisphere contralateral to the hand. All maps have a threshold at Z > 2.3, except for the one of the motor-imagery of the right hand at the post-intervention (Z > 4.0), because of the substantial higher Z_max.

We found that the intensity of brain activation during MI and MI-NeuRow relative to baseline (quantified by Z_max) increased substantially from the pre-intervention to the post-intervention study and was sustained at the follow-up study (except for the MI-NeuRow-Left, which decreased at the follow-up study). Importantly, by adding the NeuRow to the MI tasks, an increase in Z_max was observed at all study periods, thus supporting the relevance of NeuRow to improve the capability of the patient to perform the MI task. Moreover, an increase in the extent of brain activation (quantified by N_vox) was also observed when comparing the pre-intervention with the post-intervention and follow-up studies for all conditions. Interestingly, the lesioned (left) hemisphere was not recruited during either left or right MI conditions, nor the right MI-NeuRow condition, at the pre-intervention study (N_vox = 0); and this was reversed with the intervention (MI-Left: N_vox = 277; MI-Right: N_vox = 377; MI-NeuRow-Right: N_vox = 356) and sustained at the follow-up study (MI-Left: N_vox = 286; MI-Right: N_vox = 526; MI-NeuRow-Right: N_vox = 442). These activation maps evidence the recruitment of brain regions known to be associated with motor activation and imagery (primary motor cortex and supplementary motor area), including those in the lesioned hemisphere. In contrast with the intensity of brain activation, it was not found differences in the extent of brain activation when comparing MI and MI-NeuRow at their respective study periods.

Discussion

Our results, with an initial case study show clear improvements and recovery regarding motor function in terms of clinical scales, self-reported scales, electrophysiological data and brain imaging data.

In terms of clinical scales, FMA has shown a stable increase in motor functioning followed throughout all assessments. This so-called carryover effect is known to involve specific mechanisms of action based on movement prediction and sense of agency/body ownership. Moreover, this effect influences the ability of a patient to plan the movement and to perceive the stimulation as a part of his/her own control loop (Gandolla et al., 2016). That could indicate effective motor learning partially evoked by immersive VR through NeuRow. We therefore hypothesize that a multimodal immersive BCI-VR training could have enhanced the carryover effect of the rehabilitation process that could eventually be reflected in terms of improved motor function. This is in line with recent work in BCI’s for rehabilitation in chronic stroke patients, showing the possibility of long-lasting improvements in motor function (Ramos-Murguialday et al., 2019).

Moreover, despite the low MoCA score (as most stroke patients may show some level of cognitive impairment) – together with the low computer knowledge and only 4 years of schooling – the patient was able to learn and use motor imagery as a way to interact with the BCI-VR system.

In terms of the perceived impact of stroke through the SIS questionnaire, the results were overall ambiguous. Part of the variation could be due to internal factors of the patient such as his mood and optimism or level of frustration upon answering the questionnaire. Many items of the questionnaire require other functions such as posture, balance, proprioception and lower limb function, which were not targeted in the BCI intervention. The improvement in perceived muscle strength is of interest because of its direct correlation with upper limb motor improved, particularly in context with improved FMA scores of the upper limb.

Further, increased MI ability as reported by VMIQ-2, but also as captured by the EEG data through the Alpha and Beta bands, seem to manifest the potential of motor recovery. Hence, current methodology for motor-imagery training may provide a valuable tool to access the motor network and improve outcome after stroke. This is also in-line with prior research findings illustrating better functional outcome in the BCI group, including a significantly higher probability of achieving a clinically relevant increase in the FMA score (Pichiorri et al., 2015). Additionally, the comparison with healthy data, reveals a convergence toward the healthy motor-imagery dataset in all domains (external, interval and kinesthetic), while also maintaining a high score in follow-up.

In terms of EEG, by comparing resting state Alpha rhythm, between the first and the last session, we observed an increase in power. Moreover, using as a point of reference healthy data -undergoing the same BCI training-, we can see that after the intervention, EEG data are closer to the distribution of the healthy participants. Since MI roughly involves (to a large extent) the same cortical areas that are activated during actual motor preparation and execution (Jeannerod and Frak, 1999), this increase is likely to be indicative of motor recovery.

Furthermore, the evoked ERD during MI training did not have the anticipated power in either Mu or Beta bands. This probably explains the low LDA classification score, showing that low performance is a direct result of the evoked EEG activation and not due to classifier configuration.

Overall, BCI performance quantified in terms of LDA classification score was stable throughout all sessions. In addition, compared with two healthy groups (VR and non-VR), we can see again that VR can result into better classification scores compared with standard training (Vourvopoulos and Bermúdez, 2016a; Vourvopoulos et al., 2016b), although our patient showed a lower performance. This can highlight once more the importance of the VR feedback and the role of agency in BCI performance.

In terms of LI, previous studies have showed that movement-related neural activity is lateralized, particularly those using fMRI (Babiloni et al., 2003). Moreover, brain activation symmetry is modified after stroke due to the resulting one-sided lesion. Analysis of the lateralization and hemispheric asymmetries of neural activity might provide a valuable neurophysiological parameter in the prognosis and follow-up of patients (Cicinelli et al., 2003).

Finally, the analysis of the fMRI data showed evidence of plastic changes including recruitment in the primary motor cortex and supplementary motor area, brain regions known to be associated with motor activation and imagery, including those in the lesioned hemisphere.

Therefore, a tailored BCI-VR training paradigm could help preventing maladaptive plasticity -avoiding compensatory movements- while helping to develop normal movement patterns.

Conclusion

With this case study, we have been able to test our proposed BCI-VR paradigm, acquiring information from various sources. Clinical scales illustrated improvements in motor function, electrophysiological data showed an increase in brain activation -similar to healthy subjects and brain-imaging data have showed the effect of MI training and VR feedback, promoting plastic changes in the targeted areas of the brain. Our findings extend prior research that showed the efficacy of BCIs using MI for motor rehabilitation (Silvoni et al., 2011; Pichiorri et al., 2015; Ramos-Murguialday et al., 2019). However, the majority of previous studies have not addressed the effect of self-paced and ecologically valid scenarios through VR feedback. These results suggest that this approach could be useful with chronic stroke patients with reduced upper limb motor function. As this is a case study, additional research is needed to explore this hypothesis including combined brain data with electrophysiological information during training. This will allow us to develop a tailored BCI-VR training paradigm that could help preventing maladaptive plasticity (e.g., by avoiding compensatory movements) and help to develop normal movement patterns. Finally, this will enable us also to identify the specific benefits of brain-controlled VR training environments for neurorehabilitation.

Limitations

Although this study collected and explored 480 trials (240 per class) of post-stroke EEG signals, along with pre- and post-intervention fMRI, and clinical datasets, it is limited by its sample size. Moreover, in the absence of a control group, it is not clear to what extent conventional treatment lead to the improvements seen. Our findings, therefore, are preliminary, have limited statistical power, and should be interpreted with caution. In addition, variability in EEG band power is rather high, thus conclusion based on comparing the values for the first and the last sessions are exploratory and not confirmative.

Data Availability

The datasets generated for this study are available on request to the corresponding author.

Ethics Statement

The experimental protocol was designed in collaboration with the local healthcare system of Madeira, Portugal (SESARAM) and approved by the scientific and ethic committees of the Central Hospital of Funchal. Finally, an informed consent was obtained from the participant upon recruitment in accordance with the 1964 Declaration of Helsinki.

Author Contributions

AV and SBiB defined and designed the research study. AV participated in the development of the software. CJ and J-CF conducted the clinical assessments. AV, CJ, and J-CF collected the data. AV analyzed the EEG data. CJ analyzed the clinical scales. RA and PF analyzed the fMRI data. All authors interpreted the data. SBiB supervised the study. All authors revised and approved the final version of the manuscript.

Funding

This research was supported by the European Commission through the RehabNet project – Neuroscience Based Interactive Systems for Motor Rehabilitation – EC (303891 RehabNet FP7-PEOPLE-2011-CIG), by the Fundação para a Ciência e Tecnologia (Portuguese Foundation for Science and Technology) through SFRH/BD/97117/2013, the NeuroAugVR FCT project (PTDC/CCI-COM/31485/2017) and LARSyS (Laboratório de Robótica e Sistemas em Engenharia e Ciência) through UID/EEA/50009/2019, and by the MACBIOIDI (INTERREG program MAC/1.1.b/098). The funding bodies supported the design of the study, collection and analysis of the data, as well as the interpretation and the writing of the manuscript.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We would like to thank the MRI technician, Mr. Sidonio Fernandes, of the Hospital “Dr. Nelio Mendoça” in Funchal (Portugal) for providing access to the equipment, and assisting with the patient’s preparation and data acquisition.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnhum.2019.00244/full#supplementary-material

References

Ang, K. K., Chua, K. S. G., Phua, K. S., Wang, C., Chin, Z. Y., Kuah, C. W. K., et al. (2015). A randomized controlled trial of eeg-based motor imagery brain-computer interface robotic rehabilitation for stroke. Clin. EEG Neurosci. 46, 310–320. doi: 10.1177/1550059414522229

PubMed Abstract | CrossRef Full Text | Google Scholar

Ang, K. K., Guan, C., Phua, K. S., Wang, C., Zhou, L., Tang, K. Y., et al. (2014). Brain-computer interface-based robotic end effector system for wrist and hand rehabilitation: results of a three-armed randomized controlled trial for chronic stroke. Front. Neuroeng. 7:30. doi: 10.3389/fneng.2014.00030

PubMed Abstract | CrossRef Full Text | Google Scholar

Ansari, N. N., Naghdi, S., Arab, T. K., and Jalaie, S. (2008). The interrater and intrarater reliability of the modified ashworth scale in the assessment of muscle spasticity: limb and muscle group effect. NeuroRehabilitation 23, 231–237.

PubMed Abstract | Google Scholar

Babiloni, C., Carducci, F., Del Gratta, C., Demartin, M., Romani, G. L., Babiloni, F., et al. (2003). Hemispherical asymmetry in human SMA during voluntary simple unilateral movements. An fMRI Study. Cortex 39, 293–305. doi: 10.1016/S0010-9452(08)70110-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Beckmann, C. F., and Smith, S. M. (2004). Probabilistic independent component analysis for functional magnetic resonance imaging. IEEE Trans. Med. Imaging 23, 137–152. doi: 10.1109/TMI.2003.822821

PubMed Abstract | CrossRef Full Text | Google Scholar

Bermúdez i Badia, S., Fluet, G. G., Llorens, R., and Deutsch, J. E. (2016). “Virtual reality for sensorimotor rehabilitation post stroke: design principles and evidence,” in Neurorehabilitation Technology, eds D. J. Reinkensmeyer and V. Dietz (Cham: Springer International Publishing), 573–603. doi: 10.1007/978-3-319-28603-7_28

CrossRef Full Text | Google Scholar

Biasiucci, A., Leeb, R., Iturrate, I., Perdikis, S., Al-Khodairy, A., Corbet, T., et al. (2018). Brain-actuated functional electrical stimulation elicits lasting arm motor recovery after stroke. Nat. Commun. 9:2421. doi: 10.1038/s41467-018-04673-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Bobath, B. (1990). Adult Hemiplegia: Evaluation and Treatment. Oxford: Butterworth-Heinemann.

Google Scholar

Celnik, P., Webster, B., Glasser, D., and Cohen, L. (2008). Effects of action observation on physical training after stroke. Stroke J. Cereb. Circ. 39, 1814–1820. doi: 10.1161/STROKEAHA.107.508184

PubMed Abstract | CrossRef Full Text | Google Scholar

Cervera, M. A., Soekadar, S. R., Ushiba, J., Millán, J. D. R., Liu, M., Birbaumer, N., et al. (2018). Brain-computer interfaces for post-stroke motor rehabilitation: a meta-analysis. Ann. Clin. Transl. Neurol. 5, 651–663. doi: 10.1002/acn3.544

PubMed Abstract | CrossRef Full Text | Google Scholar

Chaudhary, U., Birbaumer, N., and Ramos-Murguialday, A. (2016). Brain–computer interfaces for communication and rehabilitation. Nat. Rev. Neurol. 12, 513–525. doi: 10.1038/nrneurol.2016.113

PubMed Abstract | CrossRef Full Text | Google Scholar

Cicinelli, P., Pasqualetti, P., Zaccagnini, M., Traversa, R., Oliveri, M., and Rossini, P. M. (2003). Interhemispheric asymmetries of motor cortex excitability in the postacute stroke stage. Stroke 34, 2653–2658. doi: 10.1161/01.STR.0000092122.96722.72

PubMed Abstract | CrossRef Full Text | Google Scholar

Delorme, A., and Makeig, S. (2004). EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods 134, 9–21.

Google Scholar

Doyle, L. M. F., Yarrow, K., and Brown, P. (2005). Lateralization of event-related beta desynchronization in the EEG during pre-cued reaction time tasks. Clin. Neurophysiol. 116, 1879–1888. doi: 10.1016/j.clinph.2005.03.017

PubMed Abstract | CrossRef Full Text | Google Scholar

Dubovik, S., Pignat, J.-M., Ptak, R., Aboulafia, T., Allet, L., Gillabert, N., et al. (2012). The behavioral significance of coherent resting-state oscillations after stroke. NeuroImage 61, 249–257. doi: 10.1016/j.neuroimage.2012.03.024

PubMed Abstract | CrossRef Full Text | Google Scholar

Dubovik, S., Ptak, R., Aboulafia, T., Magnin, C., Gillabert, N., Allet, L., et al. (2013). EEG alpha band synchrony predicts cognitive and motor performance in patients with ischemic stroke. Behav. Neurol. 26, 187–189. doi: 10.3233/BEN-2012-129007

PubMed Abstract | CrossRef Full Text | Google Scholar

Duncan, P. W., Wallace, D., Lai, S. M., Johnson, D., Embretson, S., and Laster, L. J. (1999). The stroke impact scale version 2.0: evaluation of reliability. Validity Sensit. Change Stroke 30, 2131–2140. doi: 10.1161/01.STR.30.10.2131

CrossRef Full Text | Google Scholar

Ferreira, A., Vourvopoulos, A., and Bermudez I Badia, S. (2015). “Optimizing performance of non-expert users in brain-computer interaction by means of an adaptive performance engine,” in Lecture Notes in Computer Science/Artificial Intelligence (LNCS/LNAI), eds Y. Guo, K. Friston, F. Aldo, S. Hill, H. Peng (London: Springer).

Google Scholar

Friston, K. J., Holmes, A. P., Poline, J. B., Grasby, P. J., Williams, S. C., Frackowiak, R. S., et al. (1995). Analysis of fMRI time-series revisited. NeuroImage 2, 45–53. doi: 10.1006/nimg.1995.1007

PubMed Abstract | CrossRef Full Text | Google Scholar

Frolov, A. A., Mokienko, O., Lyukmanov, R.Kh, Chernikova, L. A., Kotov, S. V., Turbina, L. G., et al. (2016). Preliminary Results of a Controlled Study of BCI–Exoskeleton Technology Efficacy in Patients with Poststroke Arm Paresis. Available at: https://cyberleninka.ru/article/n/preliminary-results-of-a-controlled-study-of-bci-exoskeleton-technology-efficacy-in-patients-with-poststroke-arm-paresis [accessed June 5, 2019].

Google Scholar

Fugl-Meyer, A. R., Jääskö, L., Leyman, I., Olsson, S., and Steglind, S. (1975). The post-stroke hemiplegic patient. 1. a method for evaluation of physical performance. Scand. J. Rehabil. Med. 7, 13–31.

PubMed Abstract | Google Scholar

Gandolla, M., Ward, N. S., Molteni, F., Guanziroli, E., Ferrigno, G., and Pedrocchi, A. (2016). The neural correlates of long-term carryover following functional electrical stimulation for stroke. Neural. Plast 2016:4192718. doi: 10.1155/2016/4192718

PubMed Abstract | CrossRef Full Text | Google Scholar

Graimann, B., Huggins, J. E., Levine, S. P., and Pfurtscheller, G. (2002). Visualization of significant ERD/ERS patterns in multichannel EEG and ECoG data. Clin. Neurophysiol. 113, 43–47. doi: 10.1016/S1388-2457(01)00697-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Holper, L., Muehlemann, T., Scholkmann, F., Eng, K., Kiper, D., and Wolf, M. (2010). Testing the potential of a virtual reality neurorehabilitation system during performance of observation, imagery and imitation of motor actions recorded by wireless functional near-infrared spectroscopy (fNIRS). J. Neuroeng. Rehabil. 7:57. doi: 10.1186/1743-0003-7-57

PubMed Abstract | CrossRef Full Text | Google Scholar

Jeannerod, M., and Frak, V. (1999). Mental imaging of motor activity in humans. Curr. Opin. Neurobiol. 9, 735–739.

Google Scholar

Jenkinson, M., Bannister, P., Brady, M., and Smith, S. (2002). Improved optimization for the robust and accurate linear registration and motion correction of brain images. NeuroImage 17, 825–841. doi: 10.1006/nimg.2002.1132

PubMed Abstract | CrossRef Full Text | Google Scholar

Kim, T., Kim, S., and Lee, B. (2016). Effects of action observational training plus brain-computer interface-based functional electrical stimulation on paretic arm motor recovery in patient with stroke: a randomized controlled trial. Occup. Ther. Int. 23, 39–47. doi: 10.1002/oti.1403

PubMed Abstract | CrossRef Full Text | Google Scholar

Klem, G. H., Lüders, H. O., Jasper, H. H., and Elger, C. (1999). The ten-twenty electrode system of the international federation. the international federation of clinical neurophysiology. Electroencephalogr. Clin. Neurophysiol. Suppl. 52, 3–6.

PubMed Abstract | Google Scholar

Kropotov, J. D. (2016). “Chapter 2.2 - Alpha Rhythms,” in Functional Neuromarkers for Psychiatry, ed. J. D. Kropotov (San Diego: Academic Press), 89–105.

Google Scholar

Leeb, R., Biasiucci, A., Schmidlin, T., Corbet, T., Vuadens, P., and Millán, J. D. R. (2016). “BCI controlled neuromuscular electrical stimulation enables sustained motor recovery in chronic stroke victims,” in Proceedings of the 6th International Brain-Computer Interface Meeting. (Asilomar, CA)

Google Scholar

Li, M., Liu, Y., Wu, Y., Liu, S., Jia, J., and Zhang, L. (2014). Neurophysiological substrates of stroke patients with motor imagery-based brain-computer interface training. Int. J. Neurosci. 124, 403–415. doi: 10.3109/00207454.2013.850082

PubMed Abstract | CrossRef Full Text | Google Scholar

Lotte, F. (2014). A Tutorial on EEG Signal Processing Techniques for Mental State Recognition in Brain-Computer Interfaces. Available at: https://hal.inria.fr/hal-01055103/document [accessed May 8, 2017].

Google Scholar

Makeig, S. (1993). Auditory event-related dynamics of the EEG spectrum and effects of exposure to tones. Electroencephalogr. Clin. Neurophysiol. 86, 283–293. doi: 10.1016/0013-4694(93)90110-H

PubMed Abstract | CrossRef Full Text | Google Scholar

Makeig, S., Bell, A. J., Jung, T.-P., and Sejnowski, T. J. (1996). “Independent component analysis of electroencephalographic data,” in Proceedings of the NIPS’95 Proceedings of the 8th International Conference on Neural Information Processing Systems, (San Diego, CA: Naval Health Research Center), 145–151.

Google Scholar

McFarland D. J Miner L. A Vaughan T. M. Wolpaw J. R. (2000). Mu and beta rhythm topographies during motor imagery and actual movements. Brain Topogr. 12, 177–186. doi: 10.1023/A:1023437823106

PubMed Abstract | CrossRef Full Text | Google Scholar

Mihara, M., Hattori, N., Hatakenaka, M., Yagura, H., Kawano, T., Hino, T., et al. (2013). Near-infrared spectroscopy-mediated neurofeedback enhances efficacy of motor imagery-based training in poststroke victims: a pilot study. Stroke 44, 1091–1098. doi: 10.1161/STROKEAHA.111.674507

PubMed Abstract | CrossRef Full Text | Google Scholar

Miller, E. L., Murray, L., Richards, L., Zorowitz, R. D., Bakas, T., Clark, P., et al. (2010). Comprehensive overview of nursing and interdisciplinary rehabilitation care of the stroke patient: a scientific statement from the American Heart Association. Stroke 41, 2402–2448. doi: 10.1161/STR.0b013e3181e7512b

PubMed Abstract | CrossRef Full Text | Google Scholar

Mizuguchi, N., and Kanosue, K. (2017). Changes in brain activity during action observation and motor imagery: their relationship with motor learning. Prog. Brain Res. 234, 189–204. doi: 10.1016/bs.pbr.2017.08.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Mozaffarian, D., Benjamin, E. J., Go, A. S., Arnett, D. K., Blaha, M. J., Cushman, M., et al. (2015). Heart disease and stroke statistics–2015 update: a report from the American Heart Association. Circulation 131, e29–e322. doi: 10.1161/CIR.0000000000000152

PubMed Abstract | CrossRef Full Text | Google Scholar

Nasreddine, Z. S., Phillips, N. A., Bédirian, V., Charbonneau, S., Whitehead, V., Collin, I., et al. (2005). The montreal cognitive assessment. MOCA: a brief screening tool for mild cognitive impairment. J. Am. Geriatr. Soc. 53, 695–699. doi: 10.1111/j.1532-5415.2005.53221.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Neuper, C., Scherer, R., Wriessnegger, S., and Pfurtscheller, G. (2009). Motor imagery and action observation: modulation of sensorimotor brain rhythms during mental control of a brain-computer interface. Clin. Neurophysiol. 120, 239–247. doi: 10.1016/j.clinph.2008.11.015

PubMed Abstract | CrossRef Full Text | Google Scholar

Page, S. J., Fulk, G. D., and Boyne, P. (2012). Clinically important differences for the upper-extremity fugl-meyer scale in people with minimal to moderate impairment due to chronic stroke. Phys. Ther. 92, 791–798. doi: 10.2522/ptj.20110009

PubMed Abstract | CrossRef Full Text | Google Scholar

Page, S. J., Gater, D. R., and Bach-Y-Rita, P. (2004). Reconsidering the motor recovery plateau in stroke rehabilitation. Arch. Phys. Med. Rehabil. 85, 1377–1381. doi: 10.1016/j.apmr.2003.12.031

PubMed Abstract | CrossRef Full Text | Google Scholar

Perrin, F., Pernier, J., Bertrand, O., and Echallier, J. F. (1989). Spherical splines for scalp potential and current density mapping. Electroencephalogr. Clin. Neurophysiol. 72, 184–187. doi: 10.1016/0013-4694(89)90180-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Pfurtscheller, G., and Aranibar, A. (1979). Evaluation of event-related desynchronization (ERD) preceding and following voluntary self-paced movement. Electroencephalogr. Clin. Neurophysiol. 46, 138–146. doi: 10.1016/0013-4694(79)90063-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Pfurtscheller, G., and Lopes da Silva, F. H. (1999). Event-related EEG/MEG synchronization and desynchronization: basic principles. Clin. Neurophysiol. Off. J. Int. Fed. Clin. Neurophysiol. 110, 1842–1857.

PubMed Abstract | Google Scholar

Pichiorri, F., Morone, G., Petti, M., Toppi, J., Pisotta, I., Molinari, M., et al. (2015). Brain-computer interface boosts motor imagery practice during stroke recovery. Ann. Neurol. 77, 851–865. doi: 10.1002/ana.24390

PubMed Abstract | CrossRef Full Text | Google Scholar

Pimentel, M. A. F., Vilela, P., Sousa, I., and Figueiredo, P. (2013). Localization of the hand motor area by arterial spin labeling and blood oxygen level-dependent functional magnetic resonance imaging. Hum. Brain Mapp. 34, 96–108. doi: 10.1002/hbm.21418

PubMed Abstract | CrossRef Full Text | Google Scholar

Pion-Tonachini, L., Makeig, S., and Kreutz-Delgado, K. (2017). Crowd labeling latent Dirichlet allocation. Knowl. Inf. Syst. 53, 749–765. doi: 10.1007/s10115-017-1053-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Prasad, G., Herman, P., Coyle, D., McDonough, S., and Crosbie, J. (2010). Applying a brain-computer interface to support motor imagery practice in people with stroke for upper limb recovery: a feasibility study. J. NeuroEng. Rehabil. 7:60. doi: 10.1186/1743-0003-7-60

PubMed Abstract | CrossRef Full Text | Google Scholar

Ramos-Murguialday, A., Broetz, D., Rea, M., Läer, L., Yilmaz, O., Brasil, F. L., et al. (2013). Brain-machine interface in chronic stroke rehabilitation: a controlled study. Ann. Neurol. 74, 100–108. doi: 10.1002/ana.23879

PubMed Abstract | CrossRef Full Text | Google Scholar

Ramos-Murguialday, A., Curado, M. R., Broetz, D., Yilmaz, Ö, Brasil, F. L., Liberati, G., et al. (2019). Brain-machine interface in chronic stroke: randomized trial long-term follow-up. Neurorehabil. Neural Repair 33, 188–198. doi: 10.1177/1545968319827573

PubMed Abstract | CrossRef Full Text | Google Scholar

Rizzolatti, G., and Craighero, L. (2004). The mirror-neuron system. Annu. Rev. Neurosci. 27, 169–192. doi: 10.1146/annurev.neuro.27.070203.144230

CrossRef Full Text | Google Scholar

Roberts, R., Callow, N., Hardy, L., Markland, D., and Bringer, J. (2008). Movement imagery ability: development and assessment of a revised version of the vividness of movement imagery questionnaire. J. Sport Exerc. Psychol. 30, 200–221. doi: 10.1123/jsep.30.2.200

PubMed Abstract | CrossRef Full Text | Google Scholar

Salimi-Khorshidi, G., Douaud, G., Beckmann, C. F., Glasser, M. F., Griffanti, L., and Smith, S. M. (2014). Automatic denoising of functional MRI data: combining independent component analysis and hierarchical fusion of classifiers. NeuroImage 90, 449–468. doi: 10.1016/j.neuroimage.2013.11.046

PubMed Abstract | CrossRef Full Text | Google Scholar

Sigrist, R., Rauter, G., Riener, R., and Wolf, P. (2013). Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review. Psychon. Bull. Rev. 20, 21–53. doi: 10.3758/s13423-012-0333-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Silvoni, S., Ramos-Murguialday, A., Cavinato, M., Volpato, C., Cisotto, G., Turolla, A., et al. (2011). Brain-computer interface in stroke: a review of progress. Clin. EEG Neurosci. 42, 245–252.

Google Scholar

Slater, M. (2017). “Implicit learning through embodiment in immersive virtual reality,” in Virtual, Augmented, and Mixed Realities in Education Smart Computing and Intelligence, eds D. Liu, C. Dede, R. Huang, and J. Richards (Singapore: Springer), 19–33. doi: 10.1007/978-981-10-5490-7_2

CrossRef Full Text | Google Scholar

Smith, S. M. (2002). Fast robust automated brain extraction. Hum. Brain Mapp. 17, 143–155. doi: 10.1002/hbm.10062

PubMed Abstract | CrossRef Full Text | Google Scholar

Sollfrank, T., Ramsay, A., Perdikis, S., Williamson, J., Murray-Smith, R., Leeb, R., et al. (2016). The effect of multimodal and enriched feedback on SMR-BCI performance. Clin. Neurophysiol. 127, 490–498. doi: 10.1016/j.clinph.2015.06.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Trompetto, C., Marinelli, L., Mori, L., Pelosin, E., Currà, A., Molfetta, L., et al. (2014). Pathophysiology of spasticity: implications for neurorehabilitation. BioMed Res. Int. 2014:8. doi: 10.1155/2014/354906

PubMed Abstract | CrossRef Full Text | Google Scholar

Várkuti, B., Guan, C., Pan, Y., Phua, K. S., Ang, K. K., Kuah, C. W. K., et al. (2013). Resting state changes in functional connectivity correlate with movement recovery for BCI and robot-assisted upper-extremity training after stroke. Neurorehabil. Neural Repair 27, 53–62. doi: 10.1177/1545968312445910

PubMed Abstract | CrossRef Full Text | Google Scholar

Vourvopoulos, A., and Bermúdez, S. (2016a). Motor priming in virtual reality can augment motor-imagery training efficacy in restorative brain-computer interaction: a within-subject analysis. J. NeuroEng. Rehabil. 13:69. doi: 10.1186/s12984-016-0173-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Vourvopoulos, A., and Bermúdez, S. (2016b). “Usability and cost-effectiveness in brain-computer interaction: is it user throughput or technology related?,” in Proceedings of the 7th Augmented Human International Conference AH ’16, (Geneva: ACM), doi: 10.1145/2875194.2875244

CrossRef Full Text | Google Scholar

Vourvopoulos, A., Bermudez i Badia, S., and Liarokapis, F. (2016a). EEG correlates of video game experience and user profile in motor-imagery-based brain–computer interaction. Vis. Comput. 33, 533–546. doi: 10.1007/s00371-016-1304-2

CrossRef Full Text | Google Scholar

Vourvopoulos, A., Ferreira, A., and Bermudez i Badia, S. (2016b). “NeuRow: an immersive vr environment for motor-imagery training with the use of brain-computer interfaces and vibrotactile feedback,” in 3rd International Conference on Physiological Computing Systems, (Portugal: Science and Technology Publications), 43–53.

Google Scholar

Vourvopoulos, A., Faria, A. L., Cameirao, M. S., and Bermudez i Badia, S. (2013). “RehabNet: A distributed architecture for motor and cognitive neuro-rehabilitation,” in Proceedings of the 2013 IEEE 15th International Conference on e-Health Networking, Applications Services (Healthcom), (Lisbon) 454–459.

Google Scholar

Welch, P. D. (1967). The use of fast Fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms. Audio Electroacous. IEEE Trans. 15, 70–73. doi: 10.1109/TAU.1967.1161901

CrossRef Full Text | Google Scholar

Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., and Vaughan, T. M. (2002). Brain-computer interfaces for communication and control. Clin. Neurophysiol. 113:767.

Google Scholar

Woolrich, M. W., Ripley, B. D., Brady, M., and Smith, S. M. (2001). Temporal autocorrelation in univariate linear modeling of FMRI data. Neuroimage 14, 1370–1386.

PubMed Abstract | Google Scholar