Haptic Adaptive Feedback to Promote Motor Learning With a Robotic Ankle Exoskeleton Integrated With a Video Game

1. Introduction

Stroke affects each year around 13.7 million people worldwide, is the second leading cause of disability and may result in a series of motor impairments including gait abnormalities (Barroso et al., 2017; World Stroke Organization, 2018). Regarding walking rehabilitation after stroke, there has been considerable controversy and debate on the effectiveness of the various approaches used (Pollock et al., 2014). In the past 20 years, other rehabilitation modalities, such as robotic therapy have been introduced to motor rehabilitation practice aiming at promoting gait recovery in patients who suffered neural-impairments (Moreno et al., 2013), including post-stroke patients. So far, results suggest that robotic therapy may be beneficial to treat acute and chronic post-stroke patients (Van der Loos et al., 2016). Nonetheless, there is no agreement on the optimal robot-assisted approaches to promote neurorecovery through plasticity mechanisms following neural injury (Kim and You, 2017; Belas dos Santos et al., 2018; Gassert, 2018; Barroso et al., 2019).

One of the most widely tested approaches is robotic guidance, which supervises trajectories during motor tasks and prevents the user from performing undesired (and possibly unsafe) deviations from prescribed trajectories. This type of robotic assistance is frequently implemented as a “tunnel” of allowed deviation around the prescribed trajectory (Ren et al., 2011; Bortole et al., 2015). Robotic guidance can be combined with virtual environments or video games. Adding video games to the therapy turns the potential motor learning into a transparent process to the user. Moreover, engagement with the training and entertainment are very important psychological aspects of games (Patton and Mussa-Ivaldi, 2004). In fact, visual feedback has been shown to improve robotic guidance therapy scenarios (Liu et al., 2006; Tamburella et al., 2019) and video games seem to be effective to improve motor function and health after stroke (Swanson and Whittinghill, 2015). Thus, different combinations of robotic guidance and video games have been proposed. A possible shortcoming of robotic guidance is that this approach might as well reduce patients' effort, and thus, the possible benefits of the therapy (Rowe et al., 2017). In this vein, Goodman et al. (2014) designed a video game that decreased the level of assistance delivered to the ankle joint by the robot if the performance (assessed as a function of the smoothness of trajectories) increased. Other strategies involve adding resistance to make the task more challenging when the performance of the user improves, which can potentially increase engagement in the task (Ren et al., 2011). Interestingly, these two opposite strategies found evidence of enhanced motor learning markers, although there is still no consensus regarding the effects of using either type of robotic guidance.

As a counterpart of robotic guidance, error-augmentation based approaches have been also proposed to enhance motor learning. Emken and Reinkensmeyer (2005) used movement-perturbation approach with a robotic device while the user was performing the target task and concluded that motor learning can be accelerated by exploiting the error-based learning mechanism. Reinkensmeyer and Patton (2009) suggested that starting with guidance force and gradually removing it and increasing error-augmentation approaches may lead to motor learning. Marchal-Crespo et al. (2014) showed that adding random disturbances while executing a simple dorsi-plantarflexion task improved motor learning and suggested that the variability introduced to the task may increase recovery due to increased effort and attention needed to perform the task. Moreover, another study showed that “challenge-based” controllers (where guidance force is given on the first stages of the recovery and error-augmentation is given later on the rehabilitation) were more beneficial for the recovery, since this represents an adaptation of the therapy to the patients' motor learning process (Marchal-Crespo et al., 2017). These functional benefits observed in these studies suggest that the nervous system learns by forming the internal model of the dynamics of the environment via error reduction (Emken and Reinkensmeyer, 2005), leading to plastic changes presumably at the cortical level (Perez et al., 2004).

Given the aforementioned literature on different approaches tested in robotic therapy, there is evidence supporting the integration of video games in challenge-based therapies, that are able to adapt the difficulty of the task to the patient's skills, always trying to keep the user motivated and engaged. This might help promoting motor learning via activity-dependent neuroplasticity (Sweatt, 2016; Gassert, 2018). In this context, the present study proposes a novel therapy protocol that combines a grounded exoskeleton perturbing the ankle joint motion with a video game based visual feedback. Ankle joint is fundamental for gait and balance as plantarflexor passive stiffness causes reduced plantarflexion torque before starting the swing phase in gait, and may as well limit dorsiflexion, compromising foot clearance in post-stroke patients (Lamontagne et al., 2002). The major novelty that this therapy protocol introduces is the autonomous modulation of the perturbations provided to the user via haptic adaptive feedback approach based on the task performance. This protocol was first tested on a validation study with healthy subjects, and later on as an usability case study with a post-stroke patient.

We hypothesized that the use of the proposed ankle rehabilitation robot would promote motor learning and increase corticospinal excitability of the dorsi-plantarflexor muscles. Although there is not a clear relationship between motor learning and corticospinal excitability (Bestmann and Krakauer, 2015), several authors have established a relation between them (Perez et al., 2004; Kida et al., 2016; Naros et al., 2016; Mawase et al., 2017; Christiansen et al., 2018; Raffin and Siebner, 2018; Mrachacz-Kersting et al., 2019). Corticospinal excitability can be assessed with Transcranial Magnetic Stimulation (TMS), by eliciting Motor Evoked Potentials (MEPs) (Rotenberg et al., 2014). Validation of our hypotheses would provide preliminary evidence of the usefulness of this novel robotic therapy to promote motor learning in the context of a pre-gait mobilization task, i.e., mobilization before undergoing gait-centered rehabilitation.

2. Materials and Methods

2.1. Participants

Ten healthy subjects (29.80 ± 6.32 years old) participated in the study. They signed an informed consent for the experiment. Experiments were conducted in accordance with the declaration of Helsinki. All experimental procedures were approved by the Bioethical subcommittee of the Ethical committee of CSIC (Spanish National Research Council), reference 008/2016.

We also performed an usability case study with one post-stroke patient (age 37). The patient suffered an haemorrhagic transformation of ischemic stroke, affecting the right middle cerebral artery, thus the most affected side of the body was the left. The experiment with the patient was performed in the facilities, and under the supervision of the professionals of Centro de Referencia Estatal de Atención Al Daño Cerebral (CEADAC). The patient was assessed by a physician using the most common scales: the Disability Rating Scale (DRS), Functional Independence Measure (FIM) and the Barthel Index (BI). For the DRS, the value was 2, corresponding with a partial level of disability. The BI score was 100, reflecting independence in the activities of daily living, while for the FIM it was 119 (85 for motor subscale and 34 for cognitive scale). The patient signed the Informed Consent, acknowledging the risks and the inclusion criteria (he was previously examined by a physician, who validated the suitability for the training). These experimental procedures were approved by the local scientific committee in CEADAC.

2.2. Experimental Platform

The Biomot ankle robot (Moltedo et al., 2016) was used in this study. Footedness preference for each subject was established according to the Waterloo footedness test (Elias et al., 1998).

This actuator is based on the MACCEPA (mechanically adjustable compliance and controllable equilibrium position actuator) concept (Bacek et al., 2015), which is driven by a joint torque control. MACCEPA concept is based on a torque-controlled rotational actuator with adjustable compliance (Figure 1). The motor is rigidly connected to the Lever Arm (LA), which is in turn connected to the Fixed Link (FL) via a spring (K). FL is attached to the wearer's foot and thus its angle represents the user's ankle angular position, and LA represents the robot position. Both LA and FL move with respect to the Output Link (OL), which is attached to the wearer's shank. Consequently, if the motor reference is set to a particular position, the wearer still has the possibility to pivot the ankle by compressing the spring. This permits to calculate the interaction torque between the wearer and the actuator by measuring the subsequent deflection of the spring (α angle = LA − FL).

FIGURE 1

Figure 1. MACCEPA actuator model schematics and actual actuator. (A) MACCEPA actuator schematics, with all its components. (B) MACCEPA attached to a cartoon foot, with the different components depicted.

MACCEPA actuator allows to provide controlled torque profiles by using a simple position controller without the need of a complex torque sensor, and with the reliability of position sensors.

2.3. Robot Control

The controller of this robotic platform comprises a zero torque controller (based on a classic Proportional/Integral/Derivative (PID) implementation) and the haptic adaptive feedback (HAF) component based on tacit adaptability—a symbiotic control strategy on exoskeletons inspired by biomimetic mechanisms, which, in turn, is based on the “tacit learning” approach for bipeds (Shimoda et al., 2015; Asín-Prieto, 2016), adapted by the performance of the user in the experimental task.

The HAF module is schematically introduced in the control architecture, depicted in Figure 2. The controller is described by Equation (1).

$\begin{array}{ll}u={\tau }_{PID}+u_{HAF}& (1)\end{array}$

where u is the output of the controller (pulse width modulation), τ_PID corresponds to the output of the torque controller (Equation 2), and u_HAF to the output of the haptic adaptive feedback module (Equation 3).

$\begin{array}{ll}{\tau }_{PID}=K_p·error+K_i·{\displaystyle {\int }_0^t}error·dt+K_d·\frac{d}{dt}error& (2)\end{array}$

$\begin{array}{ll}{u}_{HAF}=K_{HAFi}·{\displaystyle {\int }_0^t}\alpha ·dt+K_{HAFp}·\alpha & (3)\end{array}$

where K_p, K_i, and K_d are respectively the PID constants; K_HAFi and K_HAFp are respectively the integral and proportional constants of the HAF module; α angle is proportional to the interaction torque between human and robot; and error = LA_ref − LA. LA is the actual sensor information for the Lever Arm angle, whereas LA_ref is the calculated reference LA angle. This reference LA angle is calculated with the approximation of the MACCEPA actuator to a torsion spring actuator described by Equation (4).

$\begin{array}{ll}L{A}_{ref}=\frac{{\tau }_{ref}}{K_{ts}}+FL& (4)\end{array}$

where τ_ref is the reference disturbance torque to the controller, K_ts is the empirically obtained torsional stiffness constant, and FL is the Fixed Link angle, i.e., the user's ankle angle.

FIGURE 2

Figure 2. The controller of the robot comprises a zero torque Proportional/Integral/Derivative (PID) controller and the haptic adaptive feedback module (HAF constant—K_HAF—multiplied by alpha, angle proportional to the interaction between the robot and the subject), tweaking K_HAF with the performance. The subject controls the location of the character on the screen by means of the ankle joint angle. In the figure, u stands for the output of the controller, τ_ref is the disturbance torque reference, τ_PID and u_HAF are respectively the outputs from the PID torque and HAF controllers; and FL, LA, and LA_ref are respectively the angles for Fixed Link, Lever Arm, and reference for Lever Arm computed from the reference disturbance torque.

The objective of the controller is to apply higher disturbance torques when higher performance is reached (consequently adding more difficulty to the task), and vice versa (rendering the task easier with lower performances). To do this, K_HAFi and K_HAFp are empirically set to $\frac{K_{HAF}}{1000}$ and $\frac{K_{HAF}}{5}$, where K_HAF provides the modulation of the disturbance torque following this simple rule: K_HAF = 100 − performance [%], thus, the value of the constant K_HAF is updated based on task performance. Section 2.5.1 explains how this constant is modulated.

2.4. Protocol

The longitudinal intervention protocol applied on each participant is graphically described in Figure 3. The intervention lasted 4 days. Training sessions were performed in days 1–3. Four corticospinal assessments were performed in days 1, 3 (two assessments), and 4.

FIGURE 3

Figure 3. Experiment schematics. (A) Upper left figure shows the Transcranial Magnetic Stimulation (TMS) assessment setup. (B) Upper right figure shows the experimental setup together with the daily training structure: 40 training repetitions, and ten last repetitions to survey the execution after the training (at a settled disturbance torque, the maximum given by the robot: 15 N·m). And (C) Lower figure shows the longitudinal intervention structure: (1) TMS assessment (represented by the figure-of-8 coil) PRE-intervention; (2) first day training (represented by the visual paradigm); (3) second day training; (4) third day training; (5) TMS assessment POST-intervention; (6) POST30: TMS assessment 30 min after intervention; and (7) POST24h: TMS assessment 24 h after intervention. Adapted from Asín-Prieto et al. (2018), copyright 2019, Springer Nature Switzerland AG.

The training follows this daily structure: forty training repetitions (randomized trajectory profiles, as shown in Figure 4), disturbance torque modulated by the system; followed by ten evaluation repetitions [two types of disturbance torque profiles—Figure 5A, multiplied by the five possible trajectories—Figure 4] for the assessment of immediate effect. The disturbance torque provided in these assessment repetitions was set at the maximum given by the robot: 15 N·m. All repetitions had a duration of 10 s per trajectory. The resting position of the ankle was set at −2.5° (slightly plantarflexed) as the most comfortable position for the users.

FIGURE 4

Figure 4. Five possible trajectory profiles: (A) constant −2.5°; (B) straight increasing from −4 to 1°; (C) straight decreasing from −1 to −6°; (D) from −6 to −2.5 and back to −6 again; and (E) from 1 to −2.5 and back to 1. Modified from Asín-Prieto et al. (2019), copyright 2019, IEEE.

FIGURE 5

Figure 5. The behavior of the HAF module is depicted: (1) K_HAF = 100 prompts zero-torque control; (2) K_HAF = 0 normal torque control, no influence of HAF, so up to 15 N·m reference; and (3) K_HAF between 0 and 100, nearer to a zero-torque control the higher the constant K_HAF is, thus allowing to modulate the magnitude of the applied disturbance torque amplitude. Dashed line corresponds to an example of disturbance torque profile with K_HAF between 0 and 100. (A) Possible disturbance torque profiles: torque to dorsiflexion (up) and torque to plantarflexion (down). (B) Possible disturbance torque to dorsiflexion (up) direction for the patient.

The task instruction was to follow the trajectories delineated in the visual paradigm by means of the sequence of onscreen items (gas bottles) following the shortest linear path in-between. The user had to move a character (gyrocopter) with the angular position of the ankle via dorsi-plantarflexion to collect the gas bottles: dorsiflexion implied moving the avatar upwards in the screen, whereas plantarflexion implied going downwards. Meanwhile, the robot disturbed the user motion by performing plantar and dorsiflexion alternated disturbance torque profiles (see Figure 5A). These disturbance torque profiles were developed with the aim of stimulating both agonist and antagonist muscle groups, both in dorsi- and plantarflexion movements.

For the patient, we focused only on dorsiflexion disturbance torque patterns, because he was unable to avoid the full drop of the foot. Besides, the disturbance torque was modified (as seen in Figure 5B) to remove abrupt changes in the direction of the force exerted by the robot. We empirically set a maximum disturbance torque of 5 N·m.

All healthy subjects were asked to train and find a strategy to actively compensate the disturbance torque by the ankle robot, to successfully follow the trajectory on the screen, along three sessions (one every day), of 50 repetitions. For the patient, the length of the protocol was modified to 5 days (replicating the protocol used in Asín-Prieto et al., 2018).

2.5. Metrics

2.5.1. Robot-Based

We used two different metrics to quantify the performance of the user: SCORE and root mean squared error (RMSE). SCORE was calculated for each trial as the percentage of collected onscreen items, whereas RMSE was calculated by subtracting the performed trajectory from an ideal linear path between onscreen items. Note that it could be possible to collect all the onscreen items by performing a high error trajectory between them (see Figure 6 for an example). The total SCORE for each trial was shown to encourage the user to improve it along the session. The value for K_HAF was updated when the gyrocopter exceeded a (collected or uncollected) gas bottle, based on the instantaneous SCORE in the current trial, thus modulating the disturbance torque. Each trial consisted of 20 collectible bottles, thus rendering a K_HAF refresh rate of 2 Hz (20 gas bottles per 10 s). Figure 7 depicts an example of the modulation of the disturbance torque based on the SCORE metric.

FIGURE 6

Figure 6. Trajectory example, with the best trajectory between items in continuous red line, and a high error trajectory between items (with 100% SCORE as all the items are collected) in dotted blue line.

FIGURE 7

Figure 7. Example of the modulation of K_HAF. Dashed line in the upper panel depicts the actual trajectory followed by a subject, with the uncollected bottles remaining onscreen. The instantaneous SCORE is presented in the table, both in collected/total (#) and percentage (%), as well as the computed value for K_HAF. Lower panel shows in blue the reference torque (corresponding to a SCORE of 100%), and in red the actual reference disturbance torque applied to the user's ankle modulated according to the SCORE.

SCORE and RMSE were used to quantify two different sets of data: (a) assessment post-training repetitions, i.e., the 10 last repetitions of each training day (see ROBOTIC TRAINING in Figure 3), in what we called POST-train values; (b) linear fit on the sequence of the 120 training repetitions (40 training repetitions per day, concatenated for the 3 days), and selected the values of the resulting linear fitting coinciding with the first (1) and last (120) repetitions, in what we called MOD (modulated) values, where PRE-MOD and POST-MOD were the first and last values of the linear fit, respectively (see Figure 8).

FIGURE 8

Figure 8. MOD metric calculation example, for RMSE.

In addition to RMSE and SCORE after each day training (POST-train), we used two other metrics for the patient: changes in range of motion (ROM) and velocity. Before and after the training from the second to the fifth day, the patient underwent a robotic evaluation of the possible ROM. This evaluation consisted on moving up and down a ball on the screen via dorsi-plantarflexion during 30 s. The patient was asked to alternatively reach two horizontal lines (one up and one down), and the position of these lines was changed to the maximum reached in order to make the task more difficult. Although the separation between lines meant a wider ROM, the absolute position of them remained the same onscreen in order to be unnoticeable for the patient. The maximum velocity was calculated by multiplying the maximum achieved angular amplitude by the fundamental frequency (calculated with Fast Fourier Transform). We computed the change in this metric by comparing the results before and after the intervention.

2.5.2. Clinical Assessment

In the rehabilitation process there are three main phases that need to be characterized: (1) initial assessment, to identify and measure the extent of the pathology; (2) planning, to assess the problem and establish the objectives; and (3) final assessment, after the treatment. In addition to the aforementioned metrics, the clinicians at CEADAC performed a functional clinical assessment at the beginning and the end of the week for the patient, before the first training session, and after the last one. In the functional assessment protocol developed in CEADAC, among the broad set of clinical functional scales that aim to provide an objective insight in the recovery process of patients, the clinicians focus on: Timed 10 m walk, as a measure of gait speed; 6 Min Walking Test (6MWT), as a measure of resistance; Step Test, as a measure of dynamic balance; Timed Up and Go (TUG) test, that demands several potentially destabilizing maneuvers for the subject.

2.5.3. Neurophysiological Assessment

Corticospinal excitability was assessed by recording the MEPs elicited by a Magstim 200² TMS stimulator in single pulse modality in combination with a figure-of-eight double-coned coil. We followed the instructions by SENIAM (Hermens et al., 1999) to place the surface electromyography Ag/AgCl electrodes (22.225 × 34.925 mm, Vermed), recorded with a g.USBamp amplifier (g.tec), sampled at 24 KHz and highpass filtered with a 20 Hz first order Butterworth filter.

In order to map the hot spot [place where Tibialis Anterior (TAnt) MEPs peak-to-peak amplitude is higher] on the scalp, several supra-threshold pulses were delivered nearby the vertex. The hot spot, ineon, and vertex were drawn with a permanent marker on a swimming cap, in order to ensure repeatability between sessions. After locating the hot spot, the resting motor threshold (RMT), defined as the stimulation intensity that elicits MEPs of ~50 μV peak-to-peak amplitude in 5 out of 10 applied pulses (Temesi et al., 2014), was set for each participant. We recorded ipsilaterally (Kamibayashi et al., 2009) TAnt and Soleus (SO), as well as Rectus Femoris (RF, as a control muscle not involved in the robotic ankle task).

The assessment consisted in delivering 20 pulses to each of the volunteers at an intensity of 120 % of the RMT to elicit MEPs. The peak-to-peak amplitude of the MEPs was averaged. This assessment procedure was performed four times (see Figure 3): (1) before the training of the first day (PRE); immediately after the training of the third day (last training, POST); 30 min after to evaluate plastic effects (POST30); and finally 24 h after, in order to check lasting effects (POST24h).

2.5.4. Satisfaction Questionnaire

After the treatment, all subjects filled out a Likert scale (1–Very unsatisfied; 2–Unsatisfied; 3–Not satisfied nor unsatisfied; 4–Satisfied; 5–Very satisfied) questionnaire for assessing the satisfaction level with the experimental procedure.

2.6. Data Analysis

Data were analyzed with Matlab, IBM SPSS Statistics version 25, and R Studio. After examining with Shapiro-Wilk test, our data showed variables with normal distributions and variables violating the normality. Thus, for those without a normal distribution, we provide the results for non-parametric tests; and for those that present a normal distribution, we provide parametric analyses.

First of all, changes in RMSE POST-train metric, both for healthy subjects and the patient, were tested using a Friedman test of differences among repeated measures along the study, finally evaluating the size effect with Average Spearman rho ($\overline{{\rho }_s}$), and performing a Pairwise post-hoc Test for Multiple Comparisons of Rank Sums for Unreplicated Blocked Data (Conover-test) with Bonferroni correction. For the SCORE POST-train, both for healthy subjects and the patient, we performed a One-way repeated measures ANOVA, with Huynh-Feldt correction due to lack of sphericity (Mauchly's test), with partial squared omega (${\omega }_p^2$) for the size effect, and pairwise t-test post-hoc analysis, with Bonferroni correction.

Then, we tested the correlation between SCORE and RMSE with a Spearman bivariate analysis (p value of 0.05), for the evaluation ratings after each day training (POST-train), to check the relationship between metrics.

To assess changes in the PRE-MOD vs. POST-MOD of the SCORE and RMSE on the modulated repetitions, we conducted t-Student analyses, providing Cohen's d as the size effect.

Finally, changes in the corticospinal excitability were also tested using a Friedman test, and evaluating the size effect with Average Spearman rho, and performing Conover post-hoc Test with Bonferroni correction.

3. Results

Data showed normal distribution for: SCORE POST-train both for healthy subjects and the patient, and MOD for SCORE and RMSE. All the other variables presented a non-normal distribution (see Table 1 for the descriptive statistics).

TABLE 1

Table 1. Descriptive statistics for the variables analyzed for the group of healthy individuals.

3.1. Study With Healthy Individuals

There was a significant change in SCORE POST-train metric [ANOVA, F_{(1.18, 10.60)} = 6.84; p < 0.05; ${\omega }_p^2$ = 0.35; large effect size according to Field, 2018] but not in RMSE POST-train (p > 0.05). Post-hoc tests revealed that the SCORE at the third training day was significantly increased (p = 0.03) as compared to the SCORE on the first training day (see Figure 9).

FIGURE 9

Figure 9. Results for the RMSE and SCORE after each day training (POST-train) for the group of healthy individuals. Statistical significance (post-hoc comparison) depicted by the asterisk (*). (A) Mean and standard error of POST-train for RMSE per day. (B) Mean and standard error of POST-train for SCORE per day.

We found significant (p < 0.05) strong correlations (ρ > 0.70) in the evaluation ratings after each day training (POST-train); both for SCORE 1st day and RMSE 1st day (ρ = −0.89), and SCORE 3rd day and RMSE 3rd day (ρ = −0.86).

t-Student indicated that the SCORE POST-MOD of the modulated training was significantly higher than the SCORE PRE-MOD [t(9) = −4.39; p < 0.05; Cohen's d = 1.39], and that the RMSE POST-MOD was significantly lower than the RMSE PRE-MOD [t(9) = 3.05; p < 0.05; Cohen's d = 0.96]. There was a large size effect for both metrics' t-tests according to Kotrlik and Williams (2003).

TAnt MEPs peak-to-peak amplitude was significantly changed [Friedman, χ² = 9.12; p < 0.05; 3 DoF; $\overline{{\rho }_s}$ = 0.22; small effect size according to Kotrlik and Williams (2003)] across assessment sessions. Post-hoc tests revealed that TAnt MEPs peak-to-peak amplitude was significantly increased at the POST24h moment when compared to PRE (p < 0.01), POST (p = 0.03), and POST30 (p < 0.01) moments (see Figure 10). On the other hand, there was no significant change in SO nor in RF MEPs peak-to-peak amplitude across assessment moments (p > 0.05).

FIGURE 10

Figure 10. Results for the TMS assessment for the group of healthy individuals for the four evaluated moments, normalized to the mean of the evaluation before the intervention: before the full intervention (PRE), right after the full intervention (POST), 30 min after the POST (POST30), and 24 h after the end of the full intervention (POST24h); for the muscles TAnt (Tibialis Anterior), RF (Rectus Femoris) and SO (Soleus). Statistical significance (post-hoc comparison) depicted by the asterisk (*).

The satisfaction questionnaire rendered an average of 4.8 (being 5 Very satisfied), with a standard deviation of 0.42.

3.2. Usability Case Study With Post-stroke Patient

In the case study, we used the repetitions for each of the 5 training days to conduct the statistical analyses, as we had data from one individual. The results for the case study rendered significant changes in RMSE POST-train [Friedman, χ² = 9.36; p = 0.05; 4 DoF; $\overline{{\rho }_s}$ = 0.14; medium effect size according to Kotrlik and Williams (2003)]. Post-hoc tests revealed that RMSE at training day 5 was significantly decreased when compared to day 1 (p < 0.01) and day 3 (p = 0.02) (see Figure 11). On the other hand, the SCORE did not show significant differences (ANOVA, p = 0.09) across training days (see Table 2 for the descriptive statistics).

FIGURE 11

Figure 11. Results for the RMSE and SCORE POST-train, after each day training for the patient. Statistical significance (post-hoc comparison) depicted by the asterisk (*). (A) Mean and standard error of POST-train for RMSE per day. (B) Mean and standard error of POST-train for SCORE per day.

TABLE 2

Table 2. Descriptive statistics for the RMSE (°) and SCORE (%) POST-train for the patient.

Figure 12 shows the results of the evaluation of ROM and velocity before and after each training session (from the second to the fifth day). There was a decreased ROM in days 2 and 3, and increased ROM in days 4 and 5, with a positive trend across days. A different trend was obtained in velocity (there was a decrease between the second and the third day, and between the forth and the fifth, although there was a net increase in velocity). The figure also shows the difference of the maximum and minimum values of achieved ROM per day, showing that the maximum increased across days.

FIGURE 12

Figure 12. Results for the change in ROM and velocity per day (comparing before and after each day training), for the second, third, fourth, and fifth days, for the patient. (A) ROM (continuous line), maximum angles (dashed line), and minimum angles (dotted line) difference (POST-PRE) per day. (B) Velocity change per day.

Table 3 presents the improvements on the functional scales, before and after the full treatment. Modified Ashworth scale was used to assess the level of spasticity of the patient, showing no changes in muscle tone.

TABLE 3

Table 3. Clinical scales before the beginning and after the end of the full 5 days treatment.

The satisfaction questionnaire for the patient showed that he was very satisfied with the intervention.

4. Discussion

We aimed at exploring the validity of combining the robotic ankle exoskeleton with a video game designed to promote motor learning in a therapy protocol involving autonomously customized control. We have developed the video game to enhance adherence and engagement, by providing the control of the robot with the performance to modulate the task difficulty. We approached this objective by providing perturbations to the users' ankle while asking them to follow a trajectory depicted as a sequence of collectible onscreen items. The magnitude of the perturbations was modulated as function of the performance, i.e., the number of collected items, making the task more difficult if the performance increased, and vice versa.

This reward system, based on autonomously customized hardness of the task to the user, potentially promotes learning. We also computed the error as the difference between the most efficient trajectory between onscreen items and the actual performed trajectory. Moreover, we evaluated other metrics (MEPs for the corticospinal excitability for healthy individuals; clinical scales, range of motion and velocity for the patient; and satisfaction with the process of intervention for all participants) to support our novel approach to a clinical therapy.

Regarding TMS, although there are limitations and results are not consistent when extrapolating corticospinal excitability improvement to learning processes in rehabilitation (Carson et al., 2016), several recent studies point out that an increase in corticospinal excitability may be related to an improvement in motor learning (Kida et al., 2016; Naros et al., 2016; Mawase et al., 2017; Christiansen et al., 2018; Raffin and Siebner, 2018; Mrachacz-Kersting et al., 2019), and moreover, there is a relationship between the improvement in the metrics in the robotic therapy, motor learning, and corticospinal excitability enhancement in healthy subjects (Perez et al., 2004). For this reason, we decided to use TMS as a valid technique to assess the corticospinal excitability.

Regarding the validation study in healthy volunteers, we found significant improvement in the SCORE for the POST-train metrics, but not for the RMSE, although there was a strong significant negative correlation between SCORE and RMSE. would imply that similar results These results are in line to those already found by us with a single volunteer (Asín-Prieto et al., 2018), and suggested that participants would learn and master the robotic task across days.

We tested MOD metric as a customized modality for the assessment of the performance, as the controller (and thus the disturbance torque applied to the ankle) was autonomously modulated via HAF algorithm, rather than applying the maximum disturbance torque (15 N·m), as it was done for the POST-train metric. Thus, we considered the MOD metric as a more appropriate way to assess the personalized performance. In contrast to the POST-train metrics, we found significant improvements along the training in the MOD variable for both the SCORE and RMSE.

We also found a significant increase in TAnt MEPs peak-to-peak amplitude, supporting the hypothesis that an increase in performance has a relationship with corticospinal excitability. These corticospinal changes did not show muscle specificity, as our training involved only the ankle and we have not found statistical changes in RF (control muscle) neither in SO corticospinal excitability. Consequently, we can only conclude that our training lead to increased TAnt excitability. We can speculate that one of the reasons why SO had not significantly increased corticospinal excitability, may be the fact that the robot controller is in favor of gravity, and thus the force to move the robot downwards requires less muscle activation than the required TAnt activation to dorsiflex the ankle. Another possible explanation could be that TAnt, according to Brouwer and Ashby's findings (Brouwer and Ashby, 1992), presents higher corticospinal projections density than the rest of the lower-limb muscles, and thus it may be easier to assess its excitability. In this sense, corticospinal projections to TAnt in comparison to the rest of lower-limb distal muscles, are comparable to those at upper-limb level (Brouwer and Ashby, 1990), and thus we could say that our results are consistent to those in the literature for upper-limb robotic approaches (Ramos-Murguialday et al., 2014; Kraus et al., 2016). Finally, another reason for different changes in corticospinal excitability of TAnt and SO may be that TMS can possibly activate inhibitory projections (that present a lower threshold than excitatory projections) (Nielsen and Kagamihara, 1993) that are richer in plantarflexor muscles (SO) (Hudson et al., 2013). Therefore, even if corticospinal enhancement occurs for SO, this would probably not be easily observed (Fujio et al., 2019).

In the case study (with the post-stroke patient) results, we found significant improvements in the RMSE after each day training, but non-significant changes for the SCORE. The significant changes found on RMSE are consistent not only with the results of our healthy sample, but also with the results presented by others (Patton et al., 2001; Krakauer, 2006; Reinkensmeyer and Patton, 2009; Goodman et al., 2014), thus confirming our hypothesis of usability in the case study. Although we have provided some data on the performance for this patient, our main goal was to validate the usability of this robotic training framework for post-stroke rehabilitation.

When we compared the ROM and velocity before and after each day training, we found that there was a net increase both in velocity and ROM along the days, although in the velocity there was a sawtooth shape profile (third and fifth day presented a lower increase than second and fourth, respectively). Nonetheless, as higher ROMs would inevitably render lower velocities (as it depends on the maximum achieved angular amplitude and the fundamental frequency of the resulting signal, and thus a higher amplitude would decrease the speed and vice versa), the sawtooth-shaped behavior of the velocity could be explained by this phenomenon. Furthermore, the fact that the maximum dorsiflexion increased across days may indicate an improvement in dorsiflexor muscles. If these data imply a behavioral improvement in the control of the ankle, the variation occurs together with that of the clinical scales used to assess the improvement after the treatment. Consequently, we can consider that these changes in ROM and velocity tend to improve like the clinical scales.

As reported by the Likert satisfaction scale, both studies lead to full satisfaction of the participants. We found the viability of using this treatment in patients, as the patient ranged the intervention similarly to the range given by the healthy group.

Our objective was to validate our proposed therapy as a potential tool for increasing motor learning on healthy individuals. Thus, taking into consideration all these results, our hypothesis has been confirmed in the POST-train metric for the SCORE and for the TAnt excitability; and for the MOD variables. Regarding RMSE, our hypothesis has not been confirmed, probably due to the short duration of the treatment. On the other hand, for the case study with the post-stroke patient, both SCORE and RMSE have changed as hypothesized in the design of the study.

Finally, we conclude that combining a grounded exoskeleton that disturbs the ankle joint motion with a video game incorporating autonomously controlled difficulty can elicit improvements on performance, and also increased excitability of the target muscle(s). This conclusion renders our proposal as a potential rehabilitation tool. Furthermore, we have demonstrated the viability of applying this treatment approach in a usability case study with a post-stroke patient.

As future work, we aim at extending this study using more stroke patients. This rehabilitation approach may be also explored as a novel rehabilitation framework to be used in other pathologies like spinal-cord injury (Asín-Prieto et al., 2016), cerebral palsy (Lambrecht et al., 2014; Lefmann et al., 2017), or other lower limb movement disorders (Reinkensmeyer et al., 2004; Calabro et al., 2016).

4.1. Limitations

One limitation of the case study here presented is that we only enrolled a single stroke patient. Although improvements have been shown in task performance as observed in all assessed metrics, we cannot conclude that these improvements are only due to our treatment, as the patient was also enrolled on his daily therapy with physio and occupational therapists. Nonetheless, both clinical and robot-based metrics rendered a good prospective of the integrated therapy, which should be explored by us in a wider population of stroke patients, for longer therapy sessions.

To avoid very long daily sessions for the healthy controls, we discarded the option of assessing corticospinal excitability before and after each robotic-training session. Thus, we cannot isolate the daily effects of the robotic training.

Furthermore, due to the tight therapy schedule of the patient, and to avoid lengthy sessions, the professionals at CEADAC decided to remove the TMS assessment from the protocol.

Results obtained from TMS assessments partially demonstrate the effectiveness of the robotic-therapy in modulating the corticospinal excitability in the healthy group, since the MEPs were significantly increased only in TAnt but not in SOL muscle. Although TMS presents some limitations as a diagnostic tool, namely the inter-subject variability in simultaneous measurements on normal population (Choudhury et al., 2011), the intra-subject variability obtained in our study was relatively small. On the other hand, it is worth noting that our work adds more evidence to other studies showing that different tasks may lead to increased TA excitability but not to increased SOL or Gastrocnemius Medialis (MG) excitability. For instance, results presented by Fujio et al. (2019) suggest that TA excitability is susceptible to the prediction of a perturbation, whereas the SO and MG excitability presented no change for the same tasks. In any case, we plan to assess data from a control group that does not perform the robotic treatment done by the healthy subjects in this study.

We are also planning to improve the neurophysiological assessment by including other technologies, such as assessment of changes in spinal reflexes (e.g., reciprocal inhibition) (Pascual-Valdunciel et al., 2019) or paired-pulses TMS protocols. By combining these technologies, we should be able to have a better understanding on the level (spinal, supraspinal, or both) where of plastic changes occur due to the robotic therapy.

Data Availability Statement

The datasets generated for this study are available on request to the corresponding author.

Ethics Statement

The studies involving human participants were reviewed and approved by Bioethical Subcommittee of the Ethical Committee of CSIC (Spanish National Research Council). The patients/participants provided their written informed consent to participate in this study.

Author Contributions

GA-P, AM-E, FB, EU, JP, and JM: main writing process for the manuscript. GA-P, AM-E, FB, JM, JG-V, SS, and FA: contribution to study conceptualization. GA-P, AM-E, FB, and JM: study design. GA-P, AM-E, and CG-A: data acquisition. GA-P and AM-E: data analysis. GA-P, AM-E, FB, and JM: data interpretation. All authors: approval of final manuscript and agreement to be accountable for all aspects of the work while ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Funding

This research has been funded by the Commission of the European Union under the BioMot project–Smart Wearable Robots with Bioinspired Sensory-Motor Skills (Grant Agreement number IFP7-ICT-2013-10-611695), also under the ASTONISH Project–Advancing Smart Optical Imaging and Sensing for Health (Grant Agreement number H2020-EU.2.1.1.7.-ECSEL-04-2015-692470); with financial support of Spanish Ministry of Economy and Competitiveness (MINECO) under the ASSOCIATE project—A comprehensive and wearable robotics based approach to the rehabilitation and assistance to people with stroke and spinal cord injury (Grant Agreement number 799158449-58449-45-514); and with grant RYC-2014-16613, also by Spanish Ministry of Economy and Competitiveness.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We thank our colleagues from Vrije Universiteit Brussel who developed the MACCEPA actuator and provided invaluable support.

We thank the Centro de Referencia Estatal de Atención Al Daño Cerebral (CEADAC) in Madrid (Spain) for their support with the experiments. We would also like to thank all the volunteers that have participated in this study.

We were also grateful to all the reviewers and commenters that we have had along the preparation of the manuscript.

We thank also Noelia Máiquez for her contributions on the illustrations that appear in this article.

References

Asín-Prieto, G. (2016). “Tacit adaptability: EEG–informed symbiotic approach applied to rehabilitation robot control,” in Xmas Meeting (Madrid: Cajal Institute).

Google Scholar

Asín-Prieto, G., Andrés, A. I., Venkatakrishnan, A., Malik, W. Q., Dietz, V., and Rymer, W. Z. (2016). “Rehabilitation technologies for spinal injury,” in Emerging Therapies in Neurorehabilitation II, eds J. Pons, R. Raya, and J. González (Cham: Springer), 65–85.

Google Scholar

Asín-Prieto, G., Asín-Prieto, E., Martínez-Expósito, A., Pons, J. L., and Moreno, J. C. (2019). “Tacit adaptability on submaximal force control for ankle robotic training,” in 2019 Wearable Robotics Association Conference (WearRAcon) (Scottsdale, AZ: IEEE), 74–79.

Google Scholar

Asín-Prieto, G., Martínez-Expósito, A., Alnajjar, F., Shimoda, S., Pons, J. L., and Moreno, J. C. (2018). “Feasibility of submaximal force control training for robot–mediated therapy after stroke,” in International Conference on NeuroRehabilitation (Pisa: Springer), 256–260.

Google Scholar

Bacek, T., Unal, R., Moltedo, M., Junius, K., Cuypers, H., Vanderborght, B., et al. (2015). “Conceptual design of a novel variable stiffness actuator for use in lower limb exoskeletons,” in 2015 IEEE International Conference on Rehabilitation Robotics (ICORR) (Singapore: IEEE), 583–588.

Google Scholar

Barroso, F. O., Pascual-Valdunciel, A., Torricelli, D., Moreno, J. C., Del Ama-Espinosa, A., Laczko, J., et al. (2019). “Noninvasive modalities used in spinal cord injury rehabilitation,” in Spinal Cord Injury Therapy, eds A. Ibarra, E. García-Vences, and G. Guízar-Sahagún (London: IntechOpen), 95–114.

Google Scholar

Barroso, F. O., Torricelli, D., Molina-Rueda, F., Alguacil-Diego, I. M., Cano-de-la Cuerda, R., Santos, C., et al. (2017). Combining muscle synergies and biomechanical analysis to assess gait in stroke patients. J. Biomech. 63, 98–103. doi: 10.1016/j.jbiomech.2017.08.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Belas dos Santos, M., Barros de Oliveira, C., dos Santos, A., Garabello Pires, C., Dylewski, V., and Arida, R. M. (2018). A comparative study of conventional physiotherapy versus robot-assisted gait training associated to physiotherapy in individuals with ataxia after stroke. Behav. Neurol. 2018:2892065. doi: 10.1155/2018/2892065

PubMed Abstract | CrossRef Full Text | Google Scholar

Bestmann, S., and Krakauer, J. W. (2015). The uses and interpretations of the motor-evoked potential for understanding behaviour. Exp. Brain Res. 233, 679–689. doi: 10.1007/s00221-014-4183-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Bortole, M., Venkatakrishnan, A., Zhu, F., Moreno, J. C., Francisco, G. E., Pons, J. L., et al. (2015). The h2 robotic exoskeleton for gait rehabilitation after stroke: early findings from a clinical study. J. Neuroeng. Rehabil. 12:54. doi: 10.1186/s12984-015-0048-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Brouwer, B., and Ashby, P. (1990). Corticospinal projections to upper and lower limb spinal motoneurons in man. Electroencephalogr. Clin. Neurophysiol. 76, 509–519. doi: 10.1016/0013-4694(90)90002-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Brouwer, B., and Ashby, P. (1992). Corticospinal projections to lower limb motoneurons in man. Exp. Brain Res. 89, 649–654. doi: 10.1007/BF00229889

PubMed Abstract | CrossRef Full Text | Google Scholar

Calabro, R. S., Cacciola, A., Berte, F., Manuli, A., Leo, A., Bramanti, A., et al. (2016). Robotic gait rehabilitation and substitution devices in neurological disorders: where are we now? Neurol. Sci. 37, 503–514. doi: 10.1007/s10072-016-2474-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Carson, R. G., Ruddy, K. L., and McNickle, E. (2016). “What do tms-evoked motor potentials tell us about motor learning?” in Progress in Motor Control, eds J. Laczko and M. Latash (Cham: Springer), 143–157.

PubMed Abstract | Google Scholar

Choudhury, K. R., Boyle, L., Burke, M., Lombard, W., Ryan, S., and McNamara, B. (2011). Intra subject variation and correlation of motor potentials evoked by transcranial magnetic stimulation. Irish J. Med. Sci. 180, 873–880. doi: 10.1007/s11845-011-0722-4

CrossRef Full Text | Google Scholar

Christiansen, L., Madsen, M., Bojsen-Møller, E., Thomas, R., Nielsen, J. B., and Lundbye-Jensen, J. (2018). Progressive practice promotes motor learning and repeated transient increases in corticospinal excitability across multiple days. Brain Stimul. 11, 346–357. doi: 10.1016/j.brs.2017.11.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Elias, L. J., Bryden, M. P., and Bulman-Fleming, M. B. (1998). Footedness is a better predictor than is handedness of emotional lateralization. Neuropsychologia 36, 37–43. doi: 10.1016/S0028-3932(97)00107-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Emken, J. L., and Reinkensmeyer, D. J. (2005). Robot-enhanced motor learning: accelerating internal model formation during locomotion by transient dynamic amplification. IEEE Trans. Neural Syst. Rehabil. Eng. 13, 33–39. doi: 10.1109/TNSRE.2004.843173

PubMed Abstract | CrossRef Full Text | Google Scholar

Field, A. (2018). Discovering Statistics Using IBM SPSS Statistics, 5th Edn. Thousand Oaks, CA: Sage.

Google Scholar

Fujio, K., Obata, H., Kawashima, N., and Nakazawa, K. (2019). Presetting of the corticospinal excitability in the tibialis anterior muscle in relation to prediction of the magnitude and direction of postural perturbations. Front. Hum. Neurosci. 13:4. doi: 10.3389/fnhum.2019.00004

PubMed Abstract | CrossRef Full Text | Google Scholar

Gassert, R. (2018). Rehabilitation robots for the treatment of sensorimotor deficits: a neurophysiological perspective. J. Neuroeng. Rehabil. 15:46. doi: 10.1186/s12984-018-0383-x

PubMed Abstract | CrossRef Full Text | Google Scholar

Goodman, R. N., Rietschel, J. C., Roy, A., Jung, B. C., Macko, R. F., and Forrester, L. W. (2014). Increased reward in ankle robotics training enhances motor control and cortical efficiency in stroke. J. Rehabil. Res. Dev. 51:213. doi: 10.1682/JRRD.2013.02.0050

PubMed Abstract | CrossRef Full Text | Google Scholar

Hermens, H. J., Freriks, B., Merletti, R., Stegeman, D., Blok, J., Rau, G., et al. (1999). European recommendations for surface electromyography. Roessingh Res. Dev. 8, 13–54.

Google Scholar

Hudson, H. M., Griffin, D. M., Belhaj-Saïf, A., and Cheney, P. D. (2013). Cortical output to fast and slow muscles of the ankle in the rhesus macaque. Front. Neural Circuits 7:33. doi: 10.3389/fncir.2013.00033

PubMed Abstract | CrossRef Full Text | Google Scholar

Kamibayashi, K., Nakajima, T., Takahashi, M., Akai, M., and Nakazawa, K. (2009). Facilitation of corticospinal excitability in the tibialis anterior muscle during robot-assisted passive stepping in humans. Eur. J. Neurosci. 30, 100–109. doi: 10.1111/j.1460-9568.2009.06795.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Kida, H., Tsuda, Y., Ito, N., Yamamoto, Y., Owada, Y., Kamiya, Y., et al. (2016). Motor training promotes both synaptic and intrinsic plasticity of layer ii/iii pyramidal neurons in the primary motor cortex. Cereb. Cortex 26, 3494–3507. doi: 10.1093/cercor/bhw134

PubMed Abstract | CrossRef Full Text | Google Scholar

Kim, H. Y., and You, J. S. H. (2017). A review of robot-assisted gait training in stroke patients. Brain Neurorehabil. 10. doi: 10.12786/bn.2017.10.e9

CrossRef Full Text | Google Scholar

Kotrlik, J., and Williams, H. (2003). The incorporation of effect size in information technology, learning, information technology, learning, and performance research and performance research. Inform. Technol. Learn. Perform. J. 21:1.

Google Scholar

Krakauer, J. W. (2006). Motor learning: its relevance to stroke recovery and neurorehabilitation. Curr. Opin. Neurol. 19, 84–90. doi: 10.1097/01.wco.0000200544.29915.cc

PubMed Abstract | CrossRef Full Text | Google Scholar

Kraus, D., Naros, G., Bauer, R., Leão, M. T., Ziemann, U., and Gharabaghi, A. (2016). Brain-robot interface driven plasticity: distributed modulation of corticospinal excitability. Neuroimage 125, 522–532. doi: 10.1016/j.neuroimage.2015.09.074

PubMed Abstract | CrossRef Full Text | Google Scholar

Lambrecht, S., Urra, O., Grosu, S., and Nombela, S. P. (2014). “Emerging rehabilitation in cerebral palsy,” in Emerging Therapies in Neurorehabilitation, eds J. Pons and D. Torricelli (Berlin: Springer), 23–49.

Google Scholar

Lamontagne, A., Malouin, F., Richards, C., and Dumas, F. (2002). Mechanisms of disturbed motor control in ankle weakness during gait after stroke. Gait Posture 15, 244–255. doi: 10.1016/S0966-6362(01)00190-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Lefmann, S., Russo, R., and Hillier, S. (2017). The effectiveness of robotic-assisted gait training for paediatric gait disorders: systematic review. J. Neuroeng. Rehabil. 14:1. doi: 10.1186/s12984-016-0214-x

PubMed Abstract | CrossRef Full Text | Google Scholar

Liu, J., Cramer, S., and Reinkensmeyer, D. (2006). Learning to perform a new movement with robotic assistance: comparison of haptic guidance and visual demonstration. J. Neuroeng. Rehabil. 3:20. doi: 10.1186/1743-0003-3-20

PubMed Abstract | CrossRef Full Text | Google Scholar

Marchal-Crespo, L., Lopez-Oloriz, J., Jaeger, L., and Riener, R. (2014). “Optimizing learning of a locomotor task: amplifying errors as needed,” in 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (Chicago, IL: IEEE), 5304–5307.

Google Scholar

Marchal-Crespo, L., Michels, L., Jaeger, L., López-Olóriz, J., and Riener, R. (2017). Effect of error augmentation on brain activation and motor learning of a complex locomotor task. Front. Neurosci. 11:526. doi: 10.3389/fnins.2017.00526

PubMed Abstract | CrossRef Full Text | Google Scholar

Mawase, F., Uehara, S., Bastian, A. J., and Celnik, P. (2017). Motor learning enhances use-dependent plasticity. J. Neurosci. 37, 2673–2685. doi: 10.1523/JNEUROSCI.3303-16.2017

PubMed Abstract | CrossRef Full Text | Google Scholar

Moltedo, M., Bacek, T., Junius, K., Vanderborght, B., and Lefeber, D. (2016). “Mechanical design of a lightweight compliant and adaptable active ankle foot orthosis,” in 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob) (Singapore: IEEE), 1224–1229.

Google Scholar

Moreno, J. C., Barroso, F., Farina, D., Gizzi, L., Santos, C., Molinari, M., et al. (2013). Effects of robotic guidance on the coordination of locomotion. J. Neuroeng. Rehabil. 10:79. doi: 10.1186/1743-0003-10-79

PubMed Abstract | CrossRef Full Text | Google Scholar

Mrachacz-Kersting, N., Stevenson, A. J., Jørgensen, H. R., Severinsen, K. E., Aliakbaryhosseinabadi, S., Jiang, N., et al. (2019). Brain state–dependent stimulation boosts functional recovery following stroke. Ann. Neurol. 85, 84–95. doi: 10.1002/ana.25375

PubMed Abstract | CrossRef Full Text | Google Scholar

Naros, G., Geyer, M., Koch, S., Mayr, L., Ellinger, T., Grimm, F., et al. (2016). Enhanced motor learning with bilateral transcranial direct current stimulation: impact of polarity or current flow direction? Clin. Neurophysiol. 127, 2119–2126. doi: 10.1016/j.clinph.2015.12.020

CrossRef Full Text | Google Scholar

Nielsen, J., and Kagamihara, Y. (1993). The regulation of presynaptic inhibition during co-contraction of antagonistic muscles in man. J. Physiol. 464, 575–593. doi: 10.1113/jphysiol.1993.sp019652

PubMed Abstract | CrossRef Full Text | Google Scholar

Pascual-Valdunciel, A., Barroso, F. O., Muceli, S., Taylor, J., Farina, D., and Pons, J. L. (2019). “Modulation of reciprocal inhibition at the wrist as a neurophysiological correlate of tremor suppression: a pilot healthy subject study,” in 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (Berlin: IEEE), 6267–6272.

Google Scholar

Patton, J., Mussa-Ivaldi, F., and Rymer, W. (2001). “Altering movement patterns in healthy and brain-injured subjects via custom designed robotic forces,” in Proceedings of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2001, Vol. 2 (Istanbul: IEEE), 1356–1359.

Google Scholar

Patton, J. L., and Mussa-Ivaldi, F. A. (2004). Robot-assisted adaptive training: custom force fields for teaching movement patterns. IEEE Trans. Biomed. Eng. 51, 636–646. doi: 10.1109/TBME.2003.821035

PubMed Abstract | CrossRef Full Text | Google Scholar

Perez, M. A., Lungholt, B. K., Nyborg, K., and Nielsen, J. B. (2004). Motor skill training induces changes in the excitability of the leg cortical area in healthy humans. Exp. Brain Res. 159, 197–205. doi: 10.1007/s00221-004-1947-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Pollock, A., Baer, G., Campbell, P., Choo, P. L., Forster, A., Morris, J., et al. (2014). Physical rehabilitation approaches for recovery of function, balance and walking after stroke. Cochrane Database Syst. Rev. 4. doi: 10.1002/14651858.CD001920.pub3

CrossRef Full Text | Google Scholar

Raffin, E., and Siebner, H. R. (2018). Use-dependent plasticity in human primary motor hand area: synergistic interplay between training and immobilization. Cereb. Cortex 29, 356–371. doi: 10.1093/cercor/bhy226

PubMed Abstract | CrossRef Full Text | Google Scholar

Ramos-Murguialday, A., Broetz, D., Rea, M., Yilmaz, Ö., Brasil, F. L., Liberati, G., et al. (2014). Brain-machine-interface in chronic stroke rehabilitation: a controlled study. Ann. Neurol. 74, 100–108. doi: 10.1002/ana.23879

PubMed Abstract | CrossRef Full Text | Google Scholar

Reinkensmeyer, D. J., Emken, J. L., and Cramer, S. C. (2004). Robotics, motor learning, and neurologic recovery. Annu. Rev. Biomed. Eng. 6, 497–525. doi: 10.1146/annurev.bioeng.6.040803.140223

PubMed Abstract | CrossRef Full Text | Google Scholar

Reinkensmeyer, D. J., and Patton, J. L. (2009). Can robots help the learning of skilled actions? Exerc. Sport Sci. Rev. 37:43. doi: 10.1097/JES.0b013e3181912108

PubMed Abstract | CrossRef Full Text | Google Scholar

Ren, Y., Xu, T., Wang, L., Yang, C. Y., Guo, X., Harvey, R. L., et al. (2011). “Develop a wearable ankle robot for in-bed acute stroke rehabilitation,” in 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC (Boston, MA: IEEE), 7483–7486.

Google Scholar

Rotenberg, A., Horvath, J. C., and Pascual-Leone, A. (2014). Transcranial Magnetic Stimulation. New York, NY: Springer.

Google Scholar

Rowe, J. B., Chan, V., Ingemanson, M. L., Cramer, S. C., Wolbrecht, E. T., and Reinkensmeyer, D. J. (2017). Robotic assistance for training finger movement using a hebbian model: a randomized controlled trial. Neurorehabil. Neural Repair 31, 769–780. doi: 10.1177/1545968317721975

PubMed Abstract | CrossRef Full Text | Google Scholar

Shimoda, S., Costa, Á., Asín-Prieto, G., Okajima, S., Iáñez, E., Hasegawa, Y., et al. (2015). “Joint stiffness tuning of exoskeleton robot h2 by tacit learning,” in Symbiotic Interaction (Berlin: Springer), 138–144.

Google Scholar

Swanson, L. R., and Whittinghill, D. M. (2015). Intrinsic or extrinsic? using videogames to motivate stroke survivors: a systematic review. Games Health J. 4, 253–258. doi: 10.1089/g4h.2014.0074

PubMed Abstract | CrossRef Full Text | Google Scholar

Sweatt, J. D. (2016). Neural plasticity and behavior–sixty years of conceptual advances. J. Neurochem. 139, 179–199. doi: 10.1111/jnc.13580

PubMed Abstract | CrossRef Full Text | Google Scholar

Tamburella, F., Moreno, J. C., Valenzuela, D. S. H., Pisotta, I., Iosa, M., Cincotti, F., et al. (2019). Influences of the biofeedback content on robotic post-stroke gait rehabilitation: electromyographic vs joint torque biofeedback. J. Neuroeng. Rehabil. 16:95. doi: 10.1186/s12984-019-0558-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Temesi, J., Gruet, M., Rupp, T., Verges, S., and Millet, G. Y. (2014). Resting and active motor thresholds versus stimulus–response curves to determine transcranial magnetic stimulation intensity in quadriceps femoris. J. Neuroeng. Rehabil. 11:40. doi: 10.1186/1743-0003-11-40

PubMed Abstract | CrossRef Full Text | Google Scholar

Van der Loos, H. M., Reinkensmeyer, D. J., and Guglielmelli, E. (2016). “Rehabilitation and health care robotics,” in Springer Handbook of Robotics (Springer), 1685–1728.

Google Scholar

World Stroke Organization (2018). World Stroke Organization (WSO) Annual Report 2018. Available online at: https://www.world-stroke.org/images/Annual_Report_2018_online_fnal_COMPRESSED.pdf

Google Scholar