Exergames as a rehabilitation tool to enhance the upper limbs functionality and performance in chronic stroke survivors: a preliminary study

Introduction: Post-stroke hemiplegia commonly occurs in stroke survivors, negatively impacting the quality of life. Despite the benefits of initial specific post-acute treatments at the hospitals, motor functions, and physical mobility need to be constantly stimulated to avoid regression and subsequent hospitalizations for further rehabilitation treatments.

Method: This preliminary study proposes using gamified tasks in a virtual environment to stimulate and maintain upper limb mobility through a single RGB-D camera-based vision system (using Microsoft Azure Kinect DK). This solution is suitable for easy deployment and use in home environments. A cohort of 10 post-stroke subjects attended a 2-week gaming protocol consisting of Lateral Weightlifting (LWL) and Frontal Weightlifting (FWL) gamified tasks and gait as the instrumental evaluation task.

Results and discussion: Despite its short duration, there were statistically significant results (p < 0.05) between the baseline (T0) and the end of the protocol (TF) for Berg Balance Scale and Time Up-and-Go (9.8 and −12.3%, respectively). LWL and FWL showed significant results for unilateral executions: rate in FWL had an overall improvement of 38.5% (p < 0.001) and 34.9% (p < 0.01) for the paretic and non-paretic arm, respectively; similarly, rate in LWL improved by 19.9% (p < 0.05) for the paretic arm and 29.9% (p < 0.01) for non-paretic arm. Instead, bilateral executions had significant results for rate and speed: considering FWL, there was an improvement in rate with p < 0.01 (31.7% for paretic arm and 37.4% for non-paretic arm), whereas speed improved by 31.2% (p < 0.05) and 41.7% (p < 0.001) for the paretic and non-paretic arm, respectively; likewise, LWL showed improvement in rate with p < 0.001 (29.0% for paretic arm and 27.8% for non-paretic arm) and in speed with 23.6% (p < 0.05) and 23.5% (p < 0.01) for the paretic and non-paretic arms, respectively. No significant results were recorded for gait task, although an overall good improvement was detected for arm swing asymmetry (−22.6%). Hence, this study suggests the potential benefits of continuous stimulation of upper limb function through gamified exercises and performance monitoring over medium-long periods in the home environment, thus facilitating the patient's general mobility in daily activities.

1 Introduction

Stroke is a clinical syndrome characterized by acute loss of focal brain function, with symptoms lasting longer than 24 h or bearing to death, caused by reduced or interrupted blood supply to a brain area (ischemic stroke) or bleeding inside brain parenchyma (hemorrhagic stroke). Despite advances in wellness, prevention, and treatment, there is an increasing incidence of stroke events in the global population, as reported by several global reports (1, 2). In addition to well-known risk factors, aging is one of the more relevant non-modifiable conditions: reports indicate that incidence doubles with age (3). The consequences of the acute event are the leading causes of various functional deficits, both in the physical and cognitive domains, resulting in a significant long-term burden on healthcare systems (4) and poor quality of life for stroke survivors (5). Indeed, stroke survivors exhibit typical motor disabilities that limit their overall mobility, directly impacting activities of daily living and active social participation (6). Specifically, hemiparesis of the contralateral upper limb is one of the most disabling manifestations: this impairment affects more than 80% of stroke survivors, causing an acute or chronic limitation of mobility, control, and coordination in the upper limbs that hinders common daily actions (e.g., reaching and picking up objects) (7). Moreover, it has been shown that the upper limbs influence gait due to the altered coordination and limited stability, being an important aspect that prevents the achievement of a normal walking speed (8).

After the acute event, specific rehabilitation protocols are promptly activated to restore lost functions, activate compensatory strategies, and improve patients' independence in daily life. Some rehabilitative therapies focus on gait, posture, and balance to reduce the risk of falls and improve patient safety (9–11). Focusing on the upper limbs, several studies pointed out how therapies based on physical exercises play a crucial role after stroke: ad-hoc strategies are commonly established by varying the duration, workload, and frequency according to the patient's condition and implementing dedicated training sessions based on goal-, task-, or repetition-oriented approaches (6). For example, bilateral training (i.e., exercises that stress both sides concurrently) is a recent strategy to improve motor coordination that is based on well-established knowledge. Indeed, with this approach, the non-paretic arm can stimulate the motor function of the paretic arm when simultaneous movements are performed (12).

Recently, training and rehabilitation of the upper limb through technological approaches have gained increasing interest, and various solutions have been proposed to address the severity of motor impairment in post-stroke conditions. The most widely adopted technological solutions mainly involve assistive devices (13) and robots (14, 15) exploit for the most severe conditions. Several innovative methodologies for less severe conditions include virtual reality (16, 17); serious games, exergames, and gamification techniques (14, 18); and motion tracking using vision-based systems (19–23).

In this context, we present a solution for proposing and monitoring physical activities based on gamified tasks and exercises suitable for domestic use. The primary goal is to solicit upper limb mobility through gamified tasks promoting the improvement or maintenance of upper limb motor functions, including range of motion, motor control, and coordination. The gamified tasks are offered in two modes, unilateral and bilateral execution, and can be appropriately configured for game difficulty according to subjects' motor conditions. One of the platforms implemented during the REHOME project (24) was used for the study, specifically the Motor Rehabilitation and Exergames platform (MREP) (25). MREP leverages a single RGB-Depth camera (specifically, Microsoft Azure Kinect DK) and its innovative body tracking algorithm that captures body movements in real-time through a deep learning approach. Several works have recently analyzed the performance of the device compared with gold-standard motion capture systems (MOCAP), verifying its higher accuracy compared with predecessors and other optical sensors (26–28). Other studies have also analyzed the performance of the new body-tracking algorithm, verifying its accuracy, robustness, and reliability in capturing 3D movements and poses (29–31), including the analysis of the upper limb mobility (32–34). The good agreement with MOCAPs has led to using Azure Kinect in preliminary clinical studies and rehabilitation protocols (35–38). MREP offers various exercises (grouped into assessment tasks, gamified tasks, and rehabilitative exergames) to automatically assess upper and lower limb motor impairment related to neurological disorders. However, for the purposes of this preliminary study, we included only two of the available gamified tasks and one of the assessment tasks (i.e., walking) in the experimental protocol since we intended to focus only on upper limb stimulation using gamified tasks to evaluate the potential benefits on arm swing during walking on stroke survivors, as previously done on subjects with Parkinson's disease (39). The results obtained on the cohort of stroke survivors highlight the overall improvement in upper limb mobility for both the paretic and non-paretic arms. In particular, substantial improvement in speed, number of movements per minute, coordination metrics, and reduction of asymmetry in arm swing during walking was observed, thus confirming the initial hypothesis of the potential benefits of physical activities using gamified tasks. In addition, an implicit adaptation of the performance of the non-paretic arm to the paretic arm was also observed, as in (6). It is relevant to note that these findings agree with the overall clinical improvement (scales and tests) assessed at the end of the experimental protocol, despite the specific treatment response shown by each participant. Hence, this preliminary study aimed to evaluate whether a limited number of training sessions (precisely six) with exergames could, however, contribute to improving the functionality and performance of the upper limbs in post-stroke patients over a relatively short period (2 weeks): the positive and promising results obtained from the experimental study seem to confirm this trend.

2 Materials and methods

2.1 The experimental protocol

The experimental protocol was organized in assessment and training sessions. An initial clinical assessment session (T0) was fixed to allow clinicians to assess the general motor conditions of each participant before starting the experimental protocol. Traditional scales and tests commonly used in clinical practice were selected to evaluate several motor functions: the Berg Balance Scale (BBS) (40), Trunk Impairment Test scale (TIS) (41), Time Up-and-Go test (TUG) (42), and shoulder joint mobility assessment (43). All clinical tests were administered by qualified and experienced physical therapists, following the same standardized procedure and under the same environmental conditions for all the participants to avoid bias due to the subjectivity of the assessment as much as possible. The instrumental gait motor task (G) through MREP was included in the same session to collect gait patterns and information from each participant before starting the sessions of gamified training. A final clinical and instrumental assessment session (TF) was also planned at the end of the 2-week experimental protocol (after all the gamified sessions) to compare the final motor condition to the initial one. Between T0 and TF, the training sessions using the gamified tasks offered by MREP were organized over 2 weeks. In particular, three sessions per week were planned, collecting six training sessions with gamified tasks using MREP.

A group of 11 chronic stroke subjects was recruited from the Division of Neurology and Neurorehabilitation (San Giuseppe Hospital, Istituto Auxologico Italiano, Piancavallo, Verbania, Italy), after neurological examination, according to the following inclusion criteria: mild or moderate hemiparesis with disability on the upper and lower limbs (National Institutes of Health Stroke Scale—NIHSS ≤ 10, modified Rankin Scale—mRS ≤ 3). Functional status of upper and lower limbs was also assessed considering balance status (BBS and TIS), functional ambulation (TUG), and range of mobility of the paretic shoulder (standard articular goniometer). All participants were able to walk, with or without aids, at least for short periods. There were no exclusion criteria for age, sex, side, dominance, or therapy: only cognitive impairment assessed by Mini-Mental State Examination (MMSE < 26) was considered for exclusion. The study protocol was approved by the Ethics Committee of the Istituto Auxologico Italiano IRCCS (Authorization n. 2020_02_18_01): each subject was informed about the instrumentation and experimental protocol and then provided written informed consent to participate in the study. All participants performed the experimental protocol (clinical, instrumental, and training sessions) in a supervised scenario, under the same environmental conditions, and under the supervision of the clinical staff. For this period, the activities included in the experimental protocol were performed in place of traditional rehabilitation exercises to avoid confounding variables.

2.2 The vision-based systems: features and tasks

As mentioned above, the MREP (25) subsystem offers numerous tasks and exergames suitable for people with neurological disorders: among them, only a specific subgroup was considered based on the primary purposes of the study, namely, to investigate the effects of gamified tasks on upper limb mobility in post-stroke subjects. The MREP consists of a vision-based system that uses a single RGB-D camera (i.e., Azure Kinect) as a non-contact sensor to collect 3D body movements in real-time.¹ The system includes a body tracking algorithm that exploits deep learning approaches to reconstruct a 3D body skeletal model with segments and joints (44). To make interaction with the system simple and autonomous, a dedicated user interface (UI) was designed to support the user during task execution through text and audio messages. However, if necessary, the UI allows a supervisor to intervene by starting, stopping, and skipping the proposed exercises (45). A ZOTAC© ZBOX EN52060-V (16 GB RAM, NVIDIA GeForce RTX 2060 6GB, 9th generation 2.4 GHz quad-core processor) was used to run the MREP software component and manage the camera data streams. The Azure Kinect sensor was set to run at 30 fps (color and depth streams), at 1,080 p resolution (depth stream), and in Narrow Field of View (NFV) mode to detect movements from a greater distance and with a wider frontal viewing angle to prioritize tracking accuracy (27).

Regarding the experimental protocol, one instrumental evaluation task (gait) and two gamified tasks in a virtual environment were considered. In particular, the gait task was included to evaluate the potential effect of gamified tasks on arm swing ability. The gait task (G) was included to estimate subjects' ability in rhythmic arm swing and its alteration (including asymmetry) during walking, as well as some traditional spatiotemporal features of the walking pattern and trunk stability under dynamic conditions. Specifically, subjects had to complete a 6-meter-long path in front of the Azure Kinect sensor according to their best ability. Despite the short length, this path still allows the estimation of relevant gait parameters, as shown in (46).

Concerning the gamified tasks, Lateral Weightlifting (LWL) and Frontal Weightlifting (FWL) games have been included (45). These tasks propose a gamified version (i.e., in a virtual environment) of exercises commonly proposed in traditional physical rehabilitation sessions to assess, train, and improve upper limb mobility, motor control, and coordination. A virtual gymnasium scenario was designed to increase subjects' engagement during exercise execution, with an avatar in the scene (i.e., arm lifting a gym weight) that reproduces the actual arm movements. Subjects had to perform, at 2.5–3 meters from the Azure Kinect sensor, a predefined number of lateral arm adduction-abduction movements (LWL) or frontal up-down movements (FWL), stressing range of motion and speed, from which some relevant mobility parameters were estimated. To stress motor control and coordination, the game exercises include unilateral (i.e., single-arm movements) and bilateral (i.e., movements of both arms simultaneously) executions. In addition, exercises can be performed in a standing or sitting position to address subjects' instability, ensure safety, and avoid the risk of falls. Finally, the exercises can be customized by setting the number of movements or the minimum arm angle threshold according to the subject's condition. Figure 1 shows a screenshot of the MREP user interface, including the description of the principal scene subareas.

Figure 1

Figure 1. User interface for gamified tasks.

The choice to include both frontal and lateral movements is because stroke survivors commonly manifest more difficulty in lateral movements (47, 48): we, therefore, expected to detect differences in the execution, control, and coordination of movements in the two proposed directions. In addition, several studies have shown that simultaneous movements stimulate the reactivation of areas in the partially damaged hemisphere, leading to improved paretic limb functions (49).

2.3 Estimated parameters for upper limb mobility and walking ability

Starting from the collected 3D trajectories of segments and joints of the skeletal model, task-specific functional parameters were estimated for G, LWL, and FWL using MATLAB^® functions and custom-written scripts.

Whole-body model acquisition from MREP allows the G task to be analyzed from three subdomains simultaneously: traditional spatiotemporal features, parameters related to lateral trunk sway (i.e., dynamic instability), and arm swing features (including asymmetry). The same approach to data analysis and feature extraction as in Ferraris et al. and Cimolin et al. (39, 46) was taken: forward and backward arm trajectories were considered to focus on the properties of arm swing, in addition to gait parameters and body stability. As noted above, instrumental gait was proposed before (T0) and after the experimental protocol (TF) to compare performance and detect differences in the three subdomains, possibly confirmed by clinical evaluation (T0 vs. TF).

To analyze LWL and FWL, some joints of the skeletal model (mainly related to the upper body) were considered to estimate angular trajectories. The joints are intended to be connected in pairs to form relevant body segments for performance analysis. The body segments involved in the analysis are as follows: upper limb segment (wrist to clavicle joints); trunk segment (neck to pelvis joints); arm segment (clavicle to elbow joints); forearm segment (elbow to wrist joints). These segments defined two angular trajectories: the UPPER-LIMB-ANGLE (between upper limb and trunk segments) and the ELBOW-ANGLE (between arm and forearm segments). Since gamified tasks (LWL and FWL) required movements in different planes, the UPPER-LIMB-ANGLE was estimated with respect to the sagittal axis for the LWL (adduction-abduction movement) and to the transversal axis for the FWL (up-down movement). Figure 2 shows the joints involved in the analysis during the bilateral execution of the gamified tasks.

Figure 2

Figure 2. Examples of bilateral execution with skeletal model joints involved in the analysis of FWL (A) and LWL (B): pelvis (magenta), neck (cyan), clavicles (orange), elbows (blue), wrists (green).

From the UPPER-LIMB-ANGLE trajectory, other secondary parameters such as speed and rate (i.e., number of movements per minute) were estimated, in addition to angle measurements. It should be noted that the parameters were estimated for both the paretic and non-paretic sides and for both unilateral and bilateral execution. Table 1 shows the list of the functional parameters of this study.

Table 1

Table 1. List of parameters and metrics estimated for the study.

In addition to the more traditional measurements (angles, speeds, and related measures), three metrics were considered for a more in-depth view of arm mobility, especially during simultaneous movements: ARMSYM_G for the gait task, SYNC and SIMIL for the LWL and FWL tasks. ARMSYM_G is an index calculated as in (39) to assess arm swing asymmetry during gait: more severe asymmetry (considering maximum forward and backward arm angles) corresponds to more negative ARMSYM_G values. Therefore, a lower negative index value indicates an improvement in arm swing asymmetry.

In order to emphasize differences between upper limb trajectories during simultaneous movements (i.e., bilateral execution), SYNC and SIMIL metrics were included to provide a summary measure devoted explicitly to the temporal and spatial symmetry of bilateral movements, thus gaining insights into motor control and coordination.

In particular, the SYNC metric (45) refers to the temporal synchronization of simultaneous arm movements by quantifying the time lag that occurs between the upper limb trajectories above and below the preset minimum angular threshold and the consequent correspondence in bilateral movement cycles: values closer to zero are associated with good temporal synchronization during bilateral movements; values farther from zero indicate unsynchronized movements. Therefore, an improvement in temporal coordination is indicated by decreasing values of this metric.

The SIMIL metric evaluates the similarity between the 2D shapes drawn by the WRIST joint trajectories with respect to a common reference point (i.e., the NECK joint) in the two main directions of motion (according to lateral and frontal movements). To estimate the SIMIL metric, the MATLAB procrustes function (with scaling parameter disabled) was used to obtain information about the different characteristics of the paretic and non-paretic arm trajectories during bilateral movements. The procrustes function returns values close to zero for shapes with good similarity and increasing values for shapes with poor similarity. Examples of shapes drawn during bilateral execution are shown in Figure 3.

Figure 3

Figure 3. Examples of shape similarity during gamified tasks: good shape similarity in FWL (A), poor shape similarity in FWL (B), good shape similarity in LWL (C), and poor shape similarity in LWL (D).

As mentioned above, LWL and FWL tasks were proposed during the six training sessions (R1-R6) of the 2-week experimental protocol: parameters and metrics were estimated for each session. In addition, to detect performance improvements and trends, they were averaged and then compared for the first and the second week.

2.4 Statistical analysis

Statistical analysis was performed using Jamovi (version 2.2.5), an open-source modular platform for statistical computing (50), considering a 95% significance level (p < 0.05) for statistical tests. Considering the relatively small number of subjects and sessions, we tested the normality distribution of the estimated parameters through the Shapiro-Wilk test. Then, the distribution of estimated features was compared using parametric or non-parametric tests for paired samples to support our results with statistical evidence. Since all the considered parameters and clinical data showed normal distribution, data were provided as mean and standard deviation, while parametric tests (t-test) were used for statistical analysis.

3 Results

3.1 Clinical outcomes

A total of 11 post-stroke volunteers were deemed eligible and included in this single cohort study; however, one subject withdrew after the second gamified session due to personal reasons not related to difficulties with the experimental protocol, and thus was excluded from the subsequent analysis. Table 2 shows the demographic characteristics of the 10 participants who correctly completed the experimental protocol.

Table 2

Table 2. Participants' characteristics: demographic and clinical data (T0).

All the participants were able to walk (during task G) without assistive devices (such as tripods or canes), although five of the 10 participants routinely used them. Hence, a sitting position was preferred for the same subjects to ensure safety during the gamified tasks. The instrumental and training sessions were correctly completed by all subjects, except one subject who was unable to perform bilateral execution in most of the scheduled training sessions and one subject who needed to be supported by the supervisor during ambulation without assistive devices: the corresponding data were then discarded, resulting in 54 trials included for the analysis of LWL and FWL, and 18 trials included for the gait analysis.

Data analysis revealed an overall improvement in motor performance at the end of the experimental protocol (TF) for all clinical metrics considered. Specifically, TIS and BERG scores increased, as did paretic shoulder mobility angles, while TUG time decreased: all these changes denote an average improvement in patients' performance in clinical assessment of specific domains. However, only TUG and BERG show a statistically significant difference (p < 0.05), while TIS and paretic shoulder mobility are near significance (p < 0.06). The average improvement in clinical data and the percentage change at the end of the experimental protocol are shown in Table 3. These results suggest a trend of general improvement in motor condition, although each participant showed a different response to the protocol, as indicated by Table 4.

Table 3

Table 3. Average and percentage improvement of clinical assessment (TF vs. T0) over all participants.

Table 4

Table 4. Percentage change in clinical assessment (TF vs. T0) of each participant.

3.2 Gait task: main results

This subsection is devoted to showing differences in the intergroup walking ability at the end of the experimental protocol. The average estimated gait parameters for T0 and TF (Table 1) and their percentage changes are shown in Table 5.

Table 5

Table 5. Mean values (with standard deviation) and percentage changes of gait parameters (TF vs. T0) over all participants.

As reported in Table 5, almost all parameters show a relatively stable trend. Walking speed and step length show negligible intergroup deterioration, as does dynamic stability (approximately −0.05 m/s, −0.01 m, and +4 mm, respectively). In contrast, the stance phase duration slightly decreased (i.e., improved) in the gait cycle (−0.3%). Focusing on the arm swing, the maximum angle (averaged over both arms) slightly deteriorated (about −7.0 degrees). However, the most interesting result concerns the asymmetry index: ARMSYM_G shows a substantial reduction in its negative value at TF, suggesting an improvement in arm swing ability and motor coordination despite lower absolute arm angles. However, the statistical analysis found no significant difference (p > 0.05) for all estimated gait parameters, including ARMSYM_G: this could be due to each subject's different response to the protocol (as occurs for clinical assessment) or the need for a longer protocol duration to obtain statistical evidence of overall improvement in fine-grained gait parameters.

3.3 LWL and FWL tasks: main results of unilateral execution

This subsection is devoted to showing intergroup differences and trends for the LWL and FWL tasks by comparing the average parameters (Table 1) estimated for the first and second weeks of the experimental protocol. The estimated parameters for the 2 weeks and the percentage changes for the paretic and non-paretic sides are shown in Table 6.

Table 6

Table 6. Unilateral execution: trends of parameters for LWL and FWL (paretic arm and non-paretic arm) over all participants.

First, the gamified tasks highlight a significant difference between paretic and non-paretic arm performance, as expected (lower performance for the paretic arm). More specifically, while the angular parameters (i.e., UPPER-LIMB-ANGLE and ELBOW-ANGLE) show irrelevant changes in the second week (p > 0.05), an improvement in speed and rate is substantial for both tasks as confirmed by the statistical analysis for rate (p < 0.05). Other insights emerge from the analysis. Frontal movements (UPANG_FWL) seem to promote higher upper limb angles than lateral movements (UPANG_LWL), confirming the greater difficulty of post-stroke subjects in controlling lateral movements. In contrast, lateral movements seem to favor the proper extension of the upper limbs during the exercises, as indicated by elbow angles (ELANG_LWL > ELANG_FWL). As concluding remark, the frontal movements seem to promote more noticeable improvements on the paretic limb compared to lateral movements, although significant changes in speed and rate have still been observed in both.

3.4 LWL and FWL tasks: main results of bilateral execution

This subsection aims to show intergroup differences and trends for the bilateral execution of LWL and FWL tasks by comparing the average parameters (Table 1) estimated for the first and second weeks of the experimental protocol. The estimated parameters for the 2 weeks and the percentage changes for the paretic and non-paretic sides are shown in Table 7.

Table 7

Table 7. Bilateral execution: trends of parameters for LWL and FWL (paretic arm and non-paretic arm) over all participants.

As with the unilateral execution, the bilateral performance in LWL and FWL confirms the previous results, with negligible differences for angular measures (p > 0.05) but substantial improvement in speed and rate (p < 0.05). The detected improvement is a very relevant result, as it was obtained during a more complex exercise requiring more motor control and coordination. Other insights emerge from the analysis of the SYNC and SIMIL metrics (Table 8). Regarding the FWL, the time synchronization (SYNC_FWL) of both arms improves significantly in the second week (p < 0.05), while the shape similarity shows no relevant changes. In contrast, LWL shows a minimal but not significant deterioration in both metrics. It is important to note that the SYNC metric is relatively low (< 0.4) for both tasks, denoting good movement synchronization in time for the group of participants. The value is also low for the SIMIL metric in the FWL task. At the same time, it is slightly higher for the LWL task, confirming that post-stroke subjects have more difficulty in spatial coordination of lateral movements (SIMIL_LWL > SIMIL_FWL).

Table 8

Table 8. Bilateral execution: trends of mean metrics (with min-max range) for LWL and FWL over all participants.

The last result comes from comparing upper limb performance during unilateral and bilateral execution (Table 9). The analysis shows that the maximum UPPER-LIMB-ANGLE (UPANG_FWL and UPANG_LWL parameters) is lower during bilateral than unilateral execution, except for frontal movements of the paretic arm. The same is valid for rate parameters (RATE_FWL and RATE_LWL), where performance in bilateral execution is always lower. This outcome suggests an implicit adaptation of the non-paretic arm to the limited capability of the paretic one in terms of movement amplitude and velocity. However, the most significant differences are found during lateral movements regarding maximum lift angle and frequency of movements.

Table 9

Table 9. Comparison between unilateral (UNI) and bilateral (BI) execution for the upper limb angle and rate (number of movements per minute) parameters over all participants.

4 Discussion

This preliminary study assessed the exergames as an easy-to-use and engaging tool to enhance upper limb mobility in post-stroke subjects in a 2-week experimental protocol that included six training sessions. The results showed an overall improvement for several motor functions measured with scales and tests, such as shoulder joint mobility, posture (TIS scale), balance (BERG scale), and walking (TUG test). The results of the functional parameters support these achievements; in fact, we had significant improvements for both the frontal and lateral execution performed unilaterally and bilaterally with an increase of speed and rate (i.e., number of movements per minute) for both the paretic and non-paretic side, suggesting that an extended treatment could improve the upper limb mobility with positive influence also on trunk control and balance (51). Additionally, during the second week, there was an improvement in the synchronization metric for FWL, probably due to the neural plasticity process (52). However, it did not occur for LWL, probably due to the more difficult motor coordination during lateral execution. Then, the comparison of unilateral and bilateral executions showed that the bilateral execution had a smaller maximum angle for all the examined conditions apart from the paretic arm in FWL, highlighting the significant complexity characterizing the execution and control of the simultaneous movements. Finally, gait remained substantially stable, showing interesting changes only for the arm swing during walking: the reduction of both the maximum swing angle and the arm swing asymmetry suggests greater coordination in arm swing movements despite the lower amplitude. Instead, the lack of tangible improvement of the other gait parameters is justifiable by their nature: they are all related to the lower limbs, while the gamified tasks of this experimental protocol stimulate only the upper limb mobility. Hence, ad-hoc gamified tasks for the trunk, lower limbs, and balance should be created and tested as well.

Our preliminary results seem to confirm a positive trend for all participants in upper limb motor performance, even in bilateral execution, suggesting that prolonged treatment could produce many benefits for upper limb control and coordination, with consequent positive effects on overall motor condition and quality of life: by stimulating the strength, neuromuscular aspects and both paretic and non-paretic arms, the exergames improve the patients' autonomy, allowing them to maintain the functionality of those movements that confer independence, such as the personal hygiene.

The gamified tasks included frontal and lateral lifting movements, joined with unilateral and bilateral execution, to stimulate the upper limb motor function and mobility differently. This choice additionally allowed the solicitation of both motor control and coordination. Moreover, these gamified tasks also solicit the cognitive aspect: as shown in this study, exergames do not have a static difficulty. Instead, they allow the game level to be changed according to the patient's needs and characteristics: this means game (i.e., task) difficulty can be reconfigured based on the actual motor condition. If motor function improves, the game task can be configured to a superior difficulty. In contrast, difficulty can be reduced if the task is perceived as too complicated for the patient's status. Consequently, the patients become the “players” of this individualized and constantly stimulating rehabilitation therapy whose virtual environment and playful real-world scenario make Exergames suitable and ideal for the home setting: people with motor and cognitive deficits related to neurological disorders could continue the rehabilitation program at home, repeating the exercises not to forget the previously re-learned tasks. From an economic point of view, this home rehabilitation option would reduce healthcare costs and provide a helpful rehabilitation strategy that is also suitable for poor areas.

Specifically, post-stroke is one of the neurological impairments that could benefit from this type of solution. Stroke survivors promptly undergo in-hospital rehabilitation after the acute phase to begin recovery of motor functions impaired by the event as soon as possible. Despite this, most patients would need continuous and frequent maintenance activities to avoid losing the functional recovery achieved. However, this is not feasible in a hospital setting and is cost-demanding in an outpatient setting. Telemonitoring and telerehabilitation solutions could fill this gap, and exergames could prove to be efficient in ensuring continuity of treatment, facilitating the execution of specific physical exercises, stimulating the achievement of new rehabilitation goals, and ensuring greater adherence to treatment through a fun and engaging approach between maintenance rehabilitation sessions. Furthermore, the 3D body tracking system provides an easy-to-use and non-invasive way to collect patient mobility and performance data, allowing extensive monitoring, even during the off-period of annual rehabilitation cycles, which is also suitable for home settings. Hence, with a constant monitoring process, the scheduling of rehabilitation cycles may be improved, thus becoming more cost-effective and tailored to patients' needs.

This preliminary study is not without limitations. First, we have a limited number of subjects and sessions performed to draw generalized conclusions. It will be necessary to involve more subjects and to consider a longer observation period compatible with the duration of traditional rehabilitation protocols to generalize and consolidate current results. Then, a comparison with healthy subjects undergoing the same protocol based on gamified tasks is currently lacking: a future study will be organized to evaluate differences with a control group. In addition, to evaluate the effectiveness of the proposed gaming protocol, it will also be necessary to organize a dedicated and more extensive clinical study comparing post-stroke subjects undergoing only the gaming protocol and others undergoing concurrent upper limb rehabilitation treatments, although this study will necessarily require longer times to be completed. However, it will be necessary to consider a more prolonged double-blinded or cross-over study with larger sample for comparing clinical efficacy of the game-based protocol. Finally, as mentioned above, some instrumental measures showed an improvement trend in line with the clinical evaluation at the end of the gamified protocol (TF). Although clinical scales potentially vulnerable to rater subjectivity were used, the clinical evaluation was performed by qualified and experienced personnel, following standardized procedures and under the same conditions for all participants that should have mitigated the potential bias in the results due to this factor. Nevertheless, further studies will be necessary to consolidate what was observed in this study.

In conclusion, the encouraging data obtained in this study promotes the implementation of this technology, especially for monitoring and training/maintenance of motor functions in the domestic environment. Despite 2 weeks of training sessions are few in terms of rehabilitation, the problem-solving and visuospatial transformations typical of the gamified exercises have demonstrated fascinating potential. By combining the neurophysiological basis of rehabilitation in stroke patients with the potential of technological solutions, the system we studied may maintain, and perhaps improve, the gesture functionality acquired with intensive rehabilitation. Moreover, the patient's remote monitoring, the activities of daily living maintenance, and cognitive engagement may contribute to reducing the costs of the National Health Service and promoting new rehabilitation solutions in low-income countries. Notwithstanding this, it will be necessary to extend the analysis to a larger group of subjects, not necessarily post-stroke, with a more extended study time to investigate the effectiveness of the proposed solution.

As future developments, we will evaluate the possibility of automatically configuring gamified tasks through artificial intelligence (AI) models that weigh the subjects' functional capabilities and motor performance to adjust game levels appropriately: on the one hand, AI models could contribute to avoiding emotional stress (anxiety, distrust, demoralization), but on the other, they could set a proper and optimized challenging level for stimulating patients' constant improvement. In addition, we are also planning the integration of new gamified tasks dedicated to hand dexterity to comprehensively enhance and stimulate the motor functions related to the upper limbs.

Data availability statement

The datasets presented in this study can be found in online repositories. The name of the repository and accession number can be found below: Zenodo, https://zenodo.org/, DOI: 10.5281/zenodo.10207753.

Ethics statement

The studies involving humans were approved by Ethics Committee of Istituto Auxologico Italiano IRCCS (Authorization n. 2020_02_18_01). The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

LV: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Visualization, Writing – original draft, Writing – review & editing. CF: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Supervision, Visualization, Writing – original draft, Writing – review & editing. GA: Data curation, Investigation, Methodology, Software, Validation, Writing – review & editing. GP: Data curation, Project administration, Resources, Writing – review & editing. FB: Writing – original draft, Writing – review & editing. AT: Methodology, Writing – original draft, Writing – review & editing. AM: Funding acquisition, Project administration, Resources, Supervision, Writing – review & editing. LP: Formal analysis, Investigation, Methodology, Supervision, Writing – review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This work was supported by Italian Ministry of Health—Ricerca Corrente.

Acknowledgments

LV would like to thank the Ph.D. Programme in Experimental Medicine and Therapy of University of Turin. The authors would also like to thank the Department of Neuroscience “Rita Levi Montalcini” (Department of Excellence with grant from the Ministry of Education, University and Research) of University of Turin for participating in the study.

Conflict of interest

FB and AT were employed by Manima Non-Profit Organization Social Assistance and Healthcare.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

1. ^Azure Kinect DK. Available from: https://azure.microsoft.com/it-it/products/kinect-dk/ (accessed November 10, 2023).

References

1. Kim J, Thayabaranathan T, Donnan GA, Howard G, Howard VJ, Rothwell PM, et al. Global stroke statistics 2019. Int J Stroke. (2020) 15:819–38. doi: 10.1177/1747493020909545

Crossref Full Text | Google Scholar

2. Bustamante A, García-Berrocoso T, Rodriguez N, Llombart V, Ribó M, Molina C, et al. Ischemic stroke outcome: a review of the influence of post-stroke complications within the different scenarios of stroke care. Eur J Intern Med. (2016) 29:9–21. doi: 10.1016/j.ejim.2015.11.030

PubMed Abstract | Crossref Full Text | Google Scholar

3. Boehme AK, Esenwa C, Elkind MS. Stroke risk factors, genetics, and prevention. Circ Res. (2017) 120:472–95. doi: 10.1161/CIRCRESAHA.116.308398

PubMed Abstract | Crossref Full Text | Google Scholar

4. Katan M, Luft A. Global burden of stroke. Semin Neurol. (2018) 38:208–11. doi: 10.1055/s-0038-1649503

Crossref Full Text | Google Scholar

5. Ramos-Lima MJM, Brasileiro IC, Lima TL, Braga-Neto P. Quality of life after stroke: impact of clinical and sociodemographic factors. Clinics. (2018) 73:e418. doi: 10.6061/clinics/2017/e418

PubMed Abstract | Crossref Full Text | Google Scholar

6. Hatem SM, Saussez G, Della Faille M, Prist V, Zhang X, Dispa D, et al. Rehabilitation of motor function after stroke: a multiple systematic review focused on techniques to stimulate upper extremity recovery. Front Hum Neurosci. (2016) 13:10:442. doi: 10.3389/fnhum.2016.00442

PubMed Abstract | Crossref Full Text | Google Scholar

7. Bleyenheuft Y, Gordon AM. Precision grip in congenital and acquired hemiparesis: similarities in impairments and implications for neurorehabilitation Front. Hum Neurosci. (2014) 8:459. doi: 10.3389/fnhum.2014.00459

PubMed Abstract | Crossref Full Text | Google Scholar

8. Johansson GM, Frykberg GE, Grip H, Broström EW, Häger CK. Assessment of arm movements during gait in stroke – The Arm Posture Score. Gait Posture. (2014) 40:549–55. doi: 10.1016/j.gaitpost.2014.06.014

PubMed Abstract | Crossref Full Text | Google Scholar

9. Beyaert C, Vasa R, Frykberg GE. Gait post-stroke: pathophysiology and rehabilitation strategies. Neurophysiol Clin. (2015) 45:335–55. doi: 10.1016/j.neucli.2015.09.005

PubMed Abstract | Crossref Full Text | Google Scholar

10. Darekar A, McFadyen BJ, Lamontagne A, Fung J. Efficacy of virtual reality-based intervention on balance and mobility disorders post-stroke: a scoping review. J Neuroeng Rehabil. (2015) 12:46. doi: 10.1186/s12984-015-0035-3

Crossref Full Text | Google Scholar

11. Yang CL, Creath RA, Magder L, Rogers MW, McCombe Waller S. Impaired posture, movement preparation, and execution during both paretic and nonparetic reaching following stroke. J Neurophysiol. (2019) 121:1465–77. doi: 10.1152/jn.00694.2018

PubMed Abstract | Crossref Full Text | Google Scholar

12. Akremi H, Higgins J, Aissaoui R, Nadeau S. Bilateral motor coordination during upper limb symmetric pushing movements at two levels of force resistance in healthy and post-stroke individuals. Hum Mov Sci. (2022) 81:102913. doi: 10.1016/j.humov.2021.102913

PubMed Abstract | Crossref Full Text | Google Scholar

13. Barbosa IM, Alves PR, Silveira ZC. Upper limbs' assistive devices for stroke rehabilitation: a systematic review on design engineering solutions. J Braz Soc Mech Sci Eng. (2021) 43:236. doi: 10.1007/s40430-021-02919-4

Crossref Full Text | Google Scholar

14. Adomavičienė A, Daunoravičienė K, Kubilius R, Varžaitytė L, Raistenskis J. Influence of new technologies on post-stroke rehabilitation: a comparison of armeo spring to the kinect system. Medicina. (2019) 55:98. doi: 10.3390/medicina55040098

PubMed Abstract | Crossref Full Text | Google Scholar

15. Qassim HM, Wan Hasan WZ. A review on upper limb rehabilitation robots. Appl Sci. (2020) 10:6976. doi: 10.3390/app10196976

Crossref Full Text | Google Scholar

16. Lupu RG, Botezatu N, Ungureanu F, Ignat D, Moldoveanu A. Virtual reality based stroke recovery for upper limbs using leap motion. In: 2016 20th International Conference on System Theory, Control and Computing (ICSTCC). Sinaia (2016). p. 295–9.

Google Scholar

17. Kim W-S, Cho S, Ku J, Kim Y, Lee K, et al. Clinical application of virtual reality for upper limb motor rehabilitation in stroke: review of technologies and clinical evidence. J Clin Med. (2020) 9:3369. doi: 10.3390/jcm9103369

PubMed Abstract | Crossref Full Text | Google Scholar

18. Reis A, Lains J, Paredes H, Filipe V, Abrantes C, Ferreira F, et al. Developing a system for post-stroke rehabilitation: an exergames approach. In: Antona M, Stephanidis C, , editors. Universal Access in Human-Computer Interaction. Users and Context Diversity. UAHCI 2016. Lecture Notes in Computer Science. Cham: Springer (2016). 9739 p.

Google Scholar

19. Dash A, Yadav A, Chauhan A, Lahiri U. Kinect-assisted performance-sensitive upper limb exercise platform for post-stroke survivors. Front Neurosci. (2019) 13:228. doi: 10.3389/fnins.2019.00228

PubMed Abstract | Crossref Full Text | Google Scholar

20. Scano A, Molteni F, Molinari Tosatti L. Low-cost tracking systems allow fine biomechanical evaluation of upper-limb daily-life gestures in healthy people and post-stroke patients. Sensors. (2019) 19:1224. doi: 10.3390/s19051224

PubMed Abstract | Crossref Full Text | Google Scholar

21. Weiss Cohen M, Regazzoni D. Hand rehabilitation assessment system using leap motion controller. AI Soc. (2020) 35:581–94. doi: 10.1007/s00146-019-00925-8

Crossref Full Text | Google Scholar

22. Vismara L, Ferraris C, Votta V, Nerino R, Clerici C, Mauro A. Tele-monitoring and tele-rehabilitation of the hand in hemiplegic patients: a preliminary study. In: Bettelli A, Monteriù A, Gamberini L, , editors. Ambient Assisted Living. ForItAAL 2020. Lecture Notes in Electrical Engineering. Vol 884. Cham: Springer (2022).

Google Scholar

23. Fuertes Muñoz C, Mollineda RA, Gallardo Casero J, Pla F. A RGBD-based interactive system for gaming-driven rehabilitation of upper limbs. Sensors. (2019) 19:3478. doi: 10.3390/s19163478

PubMed Abstract | Crossref Full Text | Google Scholar

24. Ferraris C, Pettiti G, Amprimo G, Desideri D, Pratola R, Sacco K, et al. Telerehabilitation of cognitive, motor and sleep disorders in neurological pathologies: the REHOME project. In: 2022 IEEE Symposium on Computers and Communications (ISCC). Rhodes (2022). p. 1–6.

Google Scholar

25. Ferraris C, Ronga I, Pratola R, Coppo G, Bosso T, Falco S, et al. Usability of the REHOME solution for the telerehabilitation in neurological diseases: preliminary results on motor and cognitive platforms. Sensors. (2022) 22:9467. doi: 10.3390/s22239467

PubMed Abstract | Crossref Full Text | Google Scholar

26. Tölgyessy M, Dekan M, Chovanec L, Hubinský P. Evaluation of the Azure Kinect and its comparison to Kinect V1 and Kinect V2. Sensors. (2021) 21:413. doi: 10.3390/s21020413

PubMed Abstract | Crossref Full Text | Google Scholar

27. Yeung L-F, Yang Z, Cheng KC-C, Du D, Tong RK-Y. Effects of camera viewing angles on tracking kinematic gait patterns using Azure Kinect, Kinect v2 and Orbbec Astra Pro v2. Gait Post. (2021) 87:19–26. doi: 10.1016/j.gaitpost.2021.04.005

PubMed Abstract | Crossref Full Text | Google Scholar

28. Rosa B, Colombo Zefinetti F, Vitali A, Regazzoni D. RGB-D sensors as marker-less MOCAP systems: a comparison between Microsoft Kinect V2 and the New Microsoft Kinect Azure. In: Wright JL, Barber D, Scataglini S, Rajulu SL, , editors. Advances in Simulation and Digital Human Modeling; AHFE 2021. Lecture Notes in Networks and Systems. Cham: Springer (2021). 264 p.

Google Scholar

29. Albert JA, Owolabi V, Gebel A, Brahms CM, Granacher U, Arnrich B. Evaluation of the pose tracking performance of the Azure Kinect and Kinect v2 for gait analysis in comparison with a gold standard: a pilot study. Sensors. (2020) 20:5104. doi: 10.3390/s20185104

PubMed Abstract | Crossref Full Text | Google Scholar

30. Romeo L, Marani R, Malosio M, Perri AG, D'Orazio T. Performance analysis of body tracking with the Microsoft Azure Kinect. In: 2021 29th Mediterranean Conference on Control and Automation (MED). Bari (2021). p. 572–7.

Google Scholar

31. Tölgyessy M, Dekan M, Chovanec L. Skeleton tracking accuracy and precision evaluation of Kinect V1, Kinect V2, and the Azure Kinect. Appl Sci. (2021) 11:5756. doi: 10.3390/app11125756

Crossref Full Text | Google Scholar

32. Brambilla C, Marani R, Romeo L, Lavit Nicora M, Storm FA, Reni G, et al. Azure Kinect performance evaluation for human motion and upper limb biomechanical analysis. Heliyon. (2023) 9:e21606. doi: 10.1016/j.heliyon.2023.e21606

PubMed Abstract | Crossref Full Text | Google Scholar

33. Özsoy U, Yildirim Y, Karaşin S, Sekerci R, Süzen LB. Reliability and agreement of Azure Kinect and Kinect v2 depth sensors in the shoulder joint range of motion estimation. J Shoulder Elbow Surg. (2022) 31:2049–56. doi: 10.1016/j.jse.2022.04.007

PubMed Abstract | Crossref Full Text | Google Scholar

34. Królikowska A, Maj A, Dejnek M, Prill R, Skotowska-Machaj A, Kołcz A. Wrist motion assessment using Microsoft Azure Kinect DK: a reliability study in healthy individuals. Adv Clin Exp Med. (2023) 32:203–9. doi: 10.17219/acem/152884

PubMed Abstract | Crossref Full Text | Google Scholar

35. Bertram J, Krüger T, Röhling HM, Jelusic A, Mansow-Model S, Schniepp R, et al. Accuracy and repeatability of the Microsoft Azure Kinect for clinical measurement of motor function. PLoS ONE. (2023) 18:e0279697. doi: 10.1371/journal.pone.0279697

PubMed Abstract | Crossref Full Text | Google Scholar

36. Ma Y, Sheng B, Hart R, Zhang Y. The validity of a dual Azure Kinect-based motion capture system for gait analysis: a preliminary study. In: 2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). Auckland (2020). p. 1201–6.

Google Scholar

37. Antico M, Balletti N, Laudato G, Lazich A, Notarantonio M, Oliveto R, et al. Postural control assessment via Microsoft Azure Kinect DK: an evaluation study. Comput Methods Progr Biomed. (2021) 209:106324. doi: 10.1016/j.cmpb.2021.106324

PubMed Abstract | Crossref Full Text | Google Scholar

38. Wang G, Cheng M. Design and evaluation of an exergame system of knee with the Azure Kinect. In: Zeng J, Qin P, Jing W, Song X, Lu Z, , editors. Data Science. ICPCSEE 2021. Communications in Computer and Information Science. Singapore: Springer (2021). p. 1452.

Google Scholar

39. Ferraris C, Amprimo G, Masi G, Vismara L, Cremascoli R, Sinagra S, et al. Evaluation of arm swing features and asymmetry during gait in Parkinson's disease using the Azure Kinect Sensor. Sensors. (2022) 22:6282. doi: 10.3390/s22166282

PubMed Abstract | Crossref Full Text | Google Scholar

40. Kudlac M, Sabol J, Kaiser K, Kane C, Phillips RS. Reliability and validity of the berg balance scale in the stroke population: a systematic review. Phys Occup Ther Geriatr. (2019) 37:196–221. doi: 10.1080/02703181.2019.1631423

Crossref Full Text | Google Scholar

41. Verheyden G, Nieuwboer A, Mertin J, Preger R, Kiekens C, De Weerdt W. The Trunk Impairment Scale: a new tool to measure motor impairment of the trunk after stroke. Clin Rehabil. (2004) 18:326–34. doi: 10.1191/0269215504cr733oa

PubMed Abstract | Crossref Full Text | Google Scholar

42. Sawacha Z, Carraro E, Contessa P, Guiotto A, Masiero S, Cobelli C. Relationship between clinical and instrumental balance assessments in chronic post-stroke hemiparesis subjects. J Neuroeng Rehabil. (2013) 10:95. doi: 10.1186/1743-0003-10-95

PubMed Abstract | Crossref Full Text | Google Scholar

43. Soucie JM, Wang C, Forsyth A, Funk S, Denny M, Roach KE, et al. Range of motion measurements: reference values and a database for comparison studies. Haemophilia. (2011) 17:500–7. doi: 10.1111/j.1365-2516.2010.02399.x

PubMed Abstract | Crossref Full Text | Google Scholar

44. Liu Z. 3D Skeletal Tracking on Azure Kinect. (2019). Available from: https://www.microsoft.com/en-us/research/uploads/prod/2020/01/AKBTSDK.pdf (accessed November 10, 2023).

Google Scholar

45. Amprimo G, Masi G, Priano L, Azzaro C, Galli F, Pettiti G, et al. Assessment tasks and virtual exergames for remote monitoring of Parkinson's disease: an integrated approach based on Azure Kinect. Sensors. (2022) 22:8173. doi: 10.3390/s22218173

PubMed Abstract | Crossref Full Text | Google Scholar

46. Cimolin V, Vismara L, Ferraris C, Amprimo G, Pettiti G, Lopez R, et al. Computation of gait parameters in post stroke and Parkinson's disease: a comparative study using RGB-D sensors and optoelectronic systems. Sensors. (2022) 22:824. doi: 10.3390/s22030824

PubMed Abstract | Crossref Full Text | Google Scholar

47. Marchesi G, Ballardini G, Barone L, Giannoni P, Lentino C, De Luca A, et al. Modified functional reach test: upper-body kinematics and muscular activity in chronic stroke survivors. Sensors. (2021) 22:230. doi: 10.3390/s22010230

PubMed Abstract | Crossref Full Text | Google Scholar

48. Prat-Luri A, Moreno-Navarro P, García JA, Barbado D, Vera-Garcia FJ, Elvira JLL. Do initial trunk impairment, age, intervention onset, and training volume modulate the effectiveness of additional trunk exercise programs after stroke? A systematic review with meta-analyses. Int J Environ Res Public Health. (2020) 17:8714. doi: 10.3390/ijerph17238714

PubMed Abstract | Crossref Full Text | Google Scholar

49. Bruchez R, Gygax MJ, Roches S, Fluss J, Jacquier D, Ballabeni P, et al. Mirror therapy in children with hemiparesis: a randomized observer-blinded trial. Dev Med Child Neurol. (2016) 58:970–8. doi: 10.1111/dmcn.13117

PubMed Abstract | Crossref Full Text | Google Scholar

50. The Jamovi Project. (2021). Available from: https://www.jamovi.org/about.html (accessed November 10, 2023).

Google Scholar

51. Bigoni M, Cimolin V, Vismara L, Tarantino AG, Baudo S, Trotti C, et al. Retraining selective trunk muscle activity: a key to more successful rehabilitation outcomes for hemiparetic stroke patients. NeuroRehabilitation. (2021) 49:87–94. doi: 10.3233/NRE-210094

PubMed Abstract | Crossref Full Text | Google Scholar

52. Cauraugh JH, Summers JJ. Neural plasticity and bilateral movements: a rehabilitation approach for chronic stroke. Prog Neurobiol. (2005) 75:309–20. doi: 10.1016/j.pneurobio.2005.04.001

PubMed Abstract | Crossref Full Text | Google Scholar