Exercise repetition rate measured with simple sensors at home can be used to estimate Upper Extremity Fugl-Meyer score after stroke

1. Introduction

Recovery from stroke is a long process requiring extended periods of neurologic rehabilitation, which includes cycles of assessment, prescribed interventions, evaluation, and adjustment of interventions (1, 2). The assessment stage is crucial to determine a patient's treatment plan and to evaluate the effectiveness of that plan after its execution. There are a variety of standardized assessments for stroke rehabilitation in practice and research (3, 4) that span the domains of impairments, functional limitations, and barriers to participation a stroke survivor might face.

Any assessment used must have appropriate psychometric properties (4–6), including validity, reliability, and responsiveness. A particularly important goal is that the assessment works well when evaluators change, i.e., assessments should have good inter-rater reliability (7). Common motor impairment assessments with good psychometric properties used in stroke rehabilitation for the upper extremity include the Upper Extremity Fugl-Meyer (UEFM) Test, the Box and Block (BB) test, the Action Research Arm Test, the Nine Hole Peg Test, and the Wolf Motor Function Test (5).

A growing goal in rehabilitation research and development is to automate clinical assessments to reduce the burden on clinicians' time (8), eliminate the potential influence of evaluator subjectivity, improve the quality and access to effective remote care (9, 10) and support self-directed continuation of rehabilitation at home (1).

Home rehabilitation has multiple goals including reducing inpatient stays through early supported discharge, continuing rehabilitation at home to replace institutional rehabilitation, and providing home exercise programs to help patients maintain or augment the gains made under supervision of a health care professional (11). Studies on home rehabilitation have shown comparable outcomes for patients pursuing rehabilitation at home and patients in institutional care. In some cases, early supported discharge promoted community reintegration and reduced costs of care more than institutionalized patients (12). For people living at home continuing their care, a systematic review found significant effects in favor of home-based rehabilitation on functional independence measures, and some studies found cost benefits and increased caregiver satisfaction for individuals receiving home-based rehabilitation (13).

Exercise is important for improving functional capacity, performance of activities of daily living, and quality of life for post-stroke individuals (14) and may reduce the risk of stroke recurrence (15). Achieving the American Heart Association's recommendations of performing aerobic exercises 3 to 7 days per week and strengthening, flexibility, and neuromuscular exercise 2 to 3 days a week is difficult to achieve in institutionalized care and could be better realized through home exercise programs. However, maintaining motivation to adhere to home exercise programs is difficult for many stroke survivors (11), with rates of apathy in stroke survivors above 30% (16) and evidence that apathy has a strong effect on limiting participation in meaningful activity (17). Successfully administering assessments in the home environment could support patient motivation by tracking recovery progression and could be used by healthcare providers to adjust aspects of treatment plans, such as the specific tasks being used, without or in between in-person encounters.

The assessments previously mentioned are performance measures, in which a patient performs specific motions or activities, and the assessment is designed to analyze the body function or evaluate the execution of the activity (3, 7). These assessments are therefore prime candidates for automation, as a patient could perform the assessment activity independently, a sensorized system could record data during performance of the activity, and an algorithm could generate a score for the activity, approximating as closely as possible, the score that a trained clinical evaluator would give the patient as part of the assessment. This is preferable to creating new assessments designed specifically for technical automation due to the aforementioned psychometric requirements, which involves an extensive process of design and clinical validation (7). This work focuses on the Upper Extremity Fugl-Meyer (UEFM) (18) because it is a widely used measure in stroke rehabilitation research showing both high reliability and validity (19). The assessment was developed to consider classically defined patterns of stroke recovery where motor function first returns in proximal muscles before distal muscles, and flexor synergistic movements return before extensor synergies in the arms (19–21).

Previous efforts to objectively measure and automate the UEFM (see Table 1) have used Image Processing Systems such as the Microsoft Kinect (28), Inertial Measurement Units (IMUs) (24), and mechanical systems such as flex sensors, or a combination of these technologies (29). However, image-based systems can suffer from variability due to environmental lighting and visual clutter. IMUs and mechanical systems are frequently used in wearable configurations, often requiring precise placement of multiple sensing units, which is difficult to do independently for patients with impairments. Two strategies commonly employed are to instrument the assessment, whereby patients perform the assessment or a subset of the assessment's items and data is recorded during the performance, or estimate an assessment score from data taken during representative motions or functional tasks (7). Calculating a total, continuous value rather than estimating individual line items of an assessment could provide an advantage over conventional calculation methods which frequently rely on ordinal measures, which are less sensitive to smaller changes and potentially less precise than a continuous-valued output (30). Using these methods, prediction strength has ranged from r² of 0.21 to 0.97 (see Table 1).

TABLE 1

Table 1. Methods to predict fugl-Meyer assessment scores using sensors. Abbreviations following (7), IMU, inertial measurement units; EEG, electroencephalogram; MMS, mechanical systems; IMS, image processing systems; OMS, optoelectronic systems. N in the table is the number of participants in each study.

Using the rate of task performance to estimate impairment is not unique. For example, it is the strategy used by the Box and Blocks assessment. However, it is a strategy not often explored in attempts to sensorize and automate clinical assessments. This strategy potentially allows for a simple sensor array to be used, since it need only count reps, and a small number of test items [i.e., the task(s) whose rep rate is assessed]. These are desirable features for an assessment procedure intended for patients to autonomously execute in the home.

The goal of this work was to determine how well we could estimate the UEFM score for individuals who have experienced a stroke based on their exercise rep rate as they interacted with a sensorized home-rehabilitation system. We used the rep rates of tasks completed with the system, rather than raw sensor data, which is a method that could be easily implemented with other systems. The data we used included data from a recently published randomized controlled trial (RCT) of the sensor system (31).

2. Methods

2.1. FitMi overview

The FitMi system (Flint Rehab, LLC) consists of two wireless pucks that each contain an accelerometer, gyroscope, magnetometer, load cell, light emitting diode (LED), and a vibration motor (Figure 1). Custom software, run on a personal computer or tablet, presents a set of exercises for users to complete (Supplementary Text S1). A total of 40 exercises are available in the system, with 10 each designed for the legs, core, arms, and hands. During the exercises, a universal serial bus (USB) receiver collects sensor data from the pucks and the software shows how to move the pucks or move between each puck to start and finish a repetition of the activity, reacting to the changing position or state of the pucks indicated by the sensors. For each exercise, users are presented a target number of repetitions and a limited amount of time to complete them. A small amount of time is added for each repetition completed, encouraging users to perform repetitions at a desired rate. As users complete the target number of repetitions presented, the challenge of the experience is increased by increasing the target number of repetitions and making more difficult exercises available.

FIGURE 1

Figure 1. FitMi (produced by flint rehab, LLC) consists of two force and motion sensing pucks and a companion software application. Left: FitMi hardware. Right: an example of the FitMi software interface during an exercise. Note, FitMi can be used with a custom 10” touchscreen tablet in a kiosk mode (shown) or with an individual's existing computing hardware using a Bluetooth receiver.

2.2. Experimental protocol

Data for this study came from two experiments. The first experiment, the “In-Clinic Experiment”, had two goals: (1) Collect data from a broad population of stroke patients to evaluate usability of the system and to facilitate exploratory analysis; and (2) Screen participants for the second experiment. The second experiment was a randomized controlled trial (RCT) comparing home-based therapy with FitMi to conventional therapy for individuals in the subacute phase of stroke was performed at Rancho Los Amigos National Rehabilitation Center in Downey, CA from November of 2018 to March of 2020 (ClinicalTrials.gov #NCT03503617) (31). The in-clinic experiment included participants in the chronic phase and sub-acute phase of stroke recovery, while the subsequent RCT contained only sub-acute participants who met the following inclusion criteria: aged 18 to 85 who experienced one or more strokes between 2 weeks and 4 months prior with a baseline Upper Extremity Fugl-Meyer (UEFM) Score >5 and ≤55 out of 66. For individuals who continued on to participate in the RCT following the in-clinic experiment, the in-clinic experiment served as their baseline assessment.

Individuals who participated in the in-clinic experiment were first guided through a set of 12 exercises (A4: Reach to Target #2, A6: Wrist Supination, A7: Bicep Curls, C4: Twists, C7: Oblique Crunch, C8: Standard Crunch, H3: Gripping, H5: Key Pinch Grip, H10: Object Flipping, L1: Stomps, L5 Marching, L9: Ankle Rotation; See Supplementary Text S1) in the FitMi system by a rehabilitation therapist. A single therapist conducted the in-clinic assessment for each participant. For each exercise, the therapist ensured a standard placement of the pucks across all participants according to the instructions presented in the system. The therapist also instructed participants how to perform the exercise correctly. They then verified that the participant could perform the exercise without undesired compensation patterns (i.e., any movement patterns that could risk injury or maladaptive plasticity if performed several times in succession). If a participant was unable to complete an exercise or unable to perform the exercise without compensation, the therapist recorded that the participant performed zero repetitions of that exercise and moved on to the next exercise. Otherwise, the therapist instructed them to complete as many repetitions of the exercise as they could in 45 s and recorded the number of repetitions performed. Participants were given up to 2 min to rest between exercises.

Of the 41 participants of the in-clinic experiment, 27 participants, who met the inclusion criteria and agreed to participate, received therapy as part of the RCT. In the RCT, they were randomized into a FitMi group or a Conventional Therapy group using adaptive randomization to ensure matched levels of impairment between the groups. To accomplish this, subjects were classified by their UEFM Score into 3 levels (i.e., 5–22, 23–39, 40–55) and then randomized by alternating block allocation (32). Participants in both groups were instructed to perform self-guided therapy at home for at least three hours/week for three consecutive weeks. The FitMi group performed their therapy using the FitMi system, and the Conventional Therapy group used a paper booklet of exercises. During the at-home phase of the study, exercise instructions and recommended puck placements were provided for each exercise in written instructions and in a video that participants could view before the exercise. However, beyond these instructions, standardization of the puck placement was left to the participant. Participants’ activity in the FitMi system was recorded, including the date and time an exercise was performed, the type of exercise, the number of repetitions completed, the amount of time spent performing the exercise, and the difficulty level at which the exercise was performed. At the start of the trial, the three easiest exercises from each body region were available at the lowest difficulty level. After three weeks, each participant returned for an end-of-therapy assessment, and then again after one month for a follow-up assessment. The Conventional Therapy Group's data (n = 13) from the end-of-therapy and follow-up assessments are not used in the present study as they did not use the FitMi system during the home-therapy they performed during the RCT.

2.2.1. Clinical assessments

Therapists performed a battery of clinical assessments during the in-clinic experiment including the Upper Extremity Fugl-Meyer (UEFM) (18), Box and Blocks Test (33), the 10 Meter Walk Test (34), the Modified Ashworth Spasticity (MAS) scale (35) for the elbow, wrist, and fingers, the Visual Analog Pain (VAP) scale (36) for the upper extremity, Trunk Impairment Scale (37), Shoulder Subluxation using thumb widths (38), and Mini Mental Status (39). Several measures were taken during the end-of-therapy and follow-up assessments of the subsequent RCT. From the end-of-therapy and follow-up assessments, only the UEFM scores of the FitMi group taken during their end-of-therapy assessment are used in this study.

2.3. Statistical analysis

In the introductory session, all participants performed each exercise for 45 s, if they were able to perform the exercise. We converted the number of repetitions completed to the rate at which repetitions were completed. For analysis, the MAS was split between the different categories measured (elbow extension, elbow flexion, wrist extension, wrist flexion, finger extension, and finger flexion), and items scored with a “+” were transformed to a numerical quantity by adding 0.5 to facilitate analysis. Unless otherwise mentioned, all analyses were performed in Matlab 2020b (40).

2.3.1. Clinic data analysis

As an exploratory analysis, linear regression was used to model the relationship between each exercise performed and each assessment taken during the in-clinic experiment (n = 41). A Belsley test was used to confirm there was no collinearity between the exercises and assessments. Several nonlinear functions were fit to the rate and outcome data for the pair which presented the strongest relationship from the previous step. The goodness of the fit for each candidate function was evaluated by comparing the resulting root mean squared error (RMSE), r-squared, and appropriateness of the function for the data as determined by the study team. Finally, the best fitting model was validated using a leave one out cross validation (LOOCV) procedure.

2.3.2. Home data analysis

The subset of participants who were randomized to the FitMi treatment group of the RCT (n = 14) took the system home for 3 weeks to use without supervision. To test the suitability of the selected model for estimating clinical scores using exercise data from participants' home-activity, exercise data from these participants were used to estimate their clinical scores using the strongest model identified by the curve fitting process described above. The model was then refit to the home exercise data and participants’ UEFM scores from the end-of-therapy assessment of the RCT to improve performance and validated following LOOCV. Due to one participant not performing the “A4: Reach to Target 2” exercise at home, analyses for home data were performed without this participant (n = 13).

2.3.3. Decision tree

To incorporate multiple exercises into a single explanatory, nonlinear model, regression trees were fit to the data taken during the in-clinic experiment (n = 41) using the Decision Tree Regressor from Scikit Learn in Python. As the data set is smaller than typical for machine learning applications, and the goal of this model was explanatory rather than predictive, the entire data set was used for training models, and models were evaluated using metrics from the training data. Decision tree models can be prone to overfitting, where the generated model might describe noise of the training set more than any underlying generalizable phenomena present. To prevent overfitting, multiple models were fit with varying maximum allowable depths and minimum samples per leaf (i.e., prediction node), where increasing the depth and reducing the minimum samples per leaf results in models with increasing accuracy but also increasing potential for overfitting. Models were compared by the research team considering the complexity of the resulting model, the depth and leaf design criteria, the RMSE, and the r². The results of this process are shown in Supplementary Figure S1. The final selected model was validated following LOOCV.

3. Results

3.1. Participants

Participants were recruited from November 20, 2018 to March 12, 2020. 41 individuals participated in the in-clinic experiment, and 27 of the participants from the in-clinic experiment moved on to the clinical trial, during which 14 participants were randomized to the FitMi group (Figure 2). Participants' clinical characteristics are shown in Table 2. All participants were right hand dominant.

FIGURE 2

Figure 2. Participant flow diagram detailing screening, allocation, and assessments for individuals in the in-clinic experiment and the RCT. Follow-up data for individuals allocated to the conventional therapy group of the RCT was not used in this study.

TABLE 2

Table 2. Demographics of participants. All participants are included in the “Clinic Data” column, and the subset of participants who continued to the FitMi group are included in the “Randomized Control Trial” column. For each assessment, we list the minimum and maximum score possible in the left column, if the assessment has such limits. In the right columns, values are reported as Mean ± SD, [minimum, maximum] or Score (Number of Participants) as appropriate.

3.2. Heatmap

Figure 3 shows the results of the regression analyses exploring the relationship between each exercise performed and each outcome measured. Only regressions with an F-statistic p value ≤ 0.05 are shown, and regressions with a p value below the Bonferroni adjusted alpha value (0.00032) are indicated with an asterisk. The UEFM and Box and Blocks Test scores were strongly correlated with the rep rates from three of the exercises.

FIGURE 3

Figure 3. Heatmap showing the results of the regression analyses for each exercise and outcome pair. Regressions with an F-statistic p value > 0.05 are shown with a white box. r² values for regressions with an F-statistic p value ≤ 0.05 are shown in blue, with darker colors indicating stronger relationships. Regressions with a p value below the Bonferroni adjusted alpha value (0.00032) are indicated with an asterisk.

3.3. Curve fitting

The strongest correlation was present for the regression analysis between “A4: Reach to Target 2” and the Upper Extremity Fugl-Meyer (UEFM) assessment (adjusted r² = 0.75, p value < 0.001). To better model this relationship, we fit a second order polynomial, a power function, a logarithmic function, and an exponential function to the data (Figure 4).

FIGURE 4

Figure 4. Exercise rate and UEFM data taken from 41 participants performing the “A4: Reach to Target #2”. The exponential function provided the best fit while providing an asymptotic structure which well describes the maximum score of the UEFM.

Though the polynomial function produced better fit statistics (r² = 0.86, RMSE = 5.92), the exponential fit (r² = 0.85, RMSE = 6.00) was selected for its asymptotic structure, which is consistent with the fact that the UEFM score has a maximum. The final LOOCV resulted in an exponential model with r² = 0.83, RMSE = 6.22.

3.4. Home data

To validate the model for use in the home setting, the exponential model shown in Figure 4 was used to estimate participant's UEFM scores using the exercise rate data from participants' first performance of the “A4: Reach to Target #2” exercise at home. Though participants' rates in their first home performance were correlated with their rates performed in the clinic (r = 0.62), the resulting fit was lower quality than for the data collected in the clinic (Clinic Model with Clinic Data: r² = 0.85, RMSE = 6.00; Clinic Model with First Home Data: r² = 0.24, RMSE = 12.81). In their first at-home performance, more severely impaired participants tended to speed up and less severely impaired participants tended to slow down relative to their in-clinic performance (Figures 5A,D). This change in exercise rate was correlated with participants' initial UEFM scores (r2 = 0.38). While data from participants' last performance of the “A4: Reach to Target #2” at home exercise did not fit the clinic-based model well, they appeared to follow a more consistent pattern than the data from participants' first performance (Figure 5B). The final home performance and end-of-therapy UEFM data resulted in improved fit statistics compared to the first home performance paired with in-clinic UEFM scores, but results were still lower quality than for the data generated in the clinic (Clinic Model with Clinic Data: r² = 0.85, RMSE = 6.00; Clinic Model with Final Home Data: r² = 0.50, RMSE = 11.17). Fitting a model of the same structure to the final performance of “A4: Reach to Target #2” at home and the UEFM scores taken at end-of-therapy resulted in a model with scores closer to the clinic-generated model (Final Home Model with Final Home Data: r² = 0.80, RMSE = 8.09). Performing LOOCV on this final model produced a model with r²= 0.69, RSME = 8.70.

FIGURE 5

Figure 5. (A) Data from participants’ first performance at home of the “A4: Reach to Target #2” exercise plotted against their clinic exercise data and the model generated from the clinic data. (B) Data from participants’ last performance at home of the “A4: Reach to Target #2” exercise and their UEFM score taken at the end-of-therapy assessment plotted against their clinic exercise data and the model generated from the clinic data. In both figures (A) and (B), triangles are used to indicate exercises performed at home, circles are used to indicate exercises performed in the clinic, and dashed lines are used to connect each participant's clinic data and their respective home data. Red lines indicate that the participant's exercise rate slowed compared to their clinic performance, and blue lines indicate that the participant's exercise rate increased compared to their clinic performance. (C) A model with the same functional form as the model developed with the clinic data was fit using only the data from participants’ last performance at home of the “A4: Reach to Target #2” exercise and their UEFM scores taken at the end-of-therapy assessment. (D) Comparison of participants’ first performance at home of the “A4: Reach to Target #2” exercise plotted against their clinic exercise data evaluated using a linear regression.

3.5. Decision tree

From the iterative model generating process (Supplementary Figure S2), the decision tree with maximum allowable depth equal to 2 and minimum samples per leaf equal to 6 was chosen, Figure 6. This model presented improved training fit statistics (r² = 0.89, RMSE = 4.82) over the previous exponential model made with a single activity performed in the clinic (r² = 0.85, RMSE = 6.00). This data-driven process created a model using two of the 12 exercises: “A4: Reach to Target #2” and “H5: Key Pinch Grip”. Patients below a certain performance threshold in the reaching task were sorted to the lower range of the scale, and then again sorted to a high impairment (14 points) or medium impairment (30 points) category by a lower threshold on the same task. Patients exceeding the initial performance threshold for the reaching task, were then evaluated by their ability in a gripping task using their hand, being further sorted to a mild (43 points) or very mild impairment (53 points) category.

FIGURE 6

Figure 6. Decision tree generated using participants’ rate of activities performed in the clinic as input features to estimate their fugl-Meyer scores. For each splitting node, if a participant's rate for the specified task (A4: “Reach to Target #2 or H5: “Key Pinch Grip”) was less than or equal to the threshold rate shown, participants were sorted to the left branch, otherwise, they were sorted to the right branch.

4. Discussion

In this work, rates of specific exercise activities, captured by a sensorized home stroke rehabilitation system, were used to estimate UEFM scores. The exercises that provided the most utility (“A4: Reach to Target #2” and “H5: Key Pinch Grip”) could theoretically be measured with simple push buttons. Thus, this approach could be replicated with a simple low-cost system that would not require participants to precisely don and doff multiple sensors.

The data here were gathered using a commercially available system that has a demonstrated record of usage across thousands of users (42). This history of usage data suggests that users are able to independently understand and operate the system, which could facilitate automated execution of assessment activities using these models.

The validated models presented here (i.e., the exponential model made with clinic data (r² = 0.83 RMSE = 6.22), the exponential model made with home data (Final Home Model with Final Home Data: r² = 0.69 RMSE = 8.70), and the decision tree made with clinic data (r² = 0.72, RMSE = 7.99)) provide comparable performance to more complicated approaches (Table 1). Additionally, this work presents a model using data from individuals practicing unsupervised in the home whereas most previous uses data taken in a clinical setting.

4.1. Insights from the decision tree modeling approach

Out of the 12 exercises performed in the clinic, the exercise with the more predictive power was “A4: Reach to Target #2”. This feature appeared most strongly in the exploratory linear regression heatmap and in the decision tree modeling process. The presented decision tree model uses this reaching task and “H5: Key Pinch Grip” to predict UEFM, which mirrors clinical knowledge that recovery typically starts with proximal ability, such as a gross arm movement, and proceeds to recovery of distal function, such as finer dexterity tasks for the hand (19). As such, the decision tree model provides a data-driven approach that presents explanatory features that match clinical understanding of recovery patterns. Consistent with this idea, prior analyses of the UEFM have confirmed the individual items have a difficulty hierarchy, proceeding from proximal to distal (43, 44). Further, the idea that measurements of a smaller number of movements can be used to predict total UEFM score is consistent with studies that have created shortened versions of the UEFM assessment (45).

4.2. Difference between clinic and home performance

The models made with data taken in clinic had higher r² and lower RMSE than the model made with home data likely because there was less variance in the way participants performed their exercises in the clinic than in how they performed exercises at home. In the in-clinic phase of the study, a single therapist standardized the placement of the FitMi sensors, prevented patients from performing exercises with compensation, and set a uniform time limit for each exercise. At home, though the system provided instructions for each exercise, participants were unsupervised, so they may have performed the exercises with compensation and may have changed the placement of the pucks. The system is structured to increase the challenge of the activity as users complete target numbers of repetitions. So in contrast to the clinic scenario where patients perform as many repetitions as they can in a set time limit, the home system sets increasingly difficult target numbers of repetitions over varying durations. This feature may increase user engagement, but it may also add elements of fatigue or may encourage users to employ strategies to pace themselves. Therefore, the rates of participants interacting with the system at home may not be directly comparable to the rates performed in the clinic. Even the presence of a supervising therapist may have been an additional motivator for patients to exert themselves, that would then be absent in the home environment. Some of this variation can be seen in Figure 5A. At home, participants with higher UEFM scores decreased their exercise rate relative to their in-clinic performance, and participants with lower UEFM score increased their exercise rate relative to their in-clinic performance.

4.3. Limitations

The sample size used in this study was relatively small (n = 41 in-clinic, n = 14 at home). Our ability to model data generated at home was further limited by the sparsity and diversity of exercises that participants performed at home. Participants were not given explicit instructions on what exercises to perform at home, which resulted in varying participation across available exercises among participants. Further, in the home setting, without supervision, participants may perform exercises differently than expected. This phenomenon likely contributed to the increased variance. Conversely, for the in-clinic phase of the study, the attending therapist stopped patients if they began performing the exercises with compensation and recorded only the repetitions performed correctly, which meant some participants recorded zero repetitions for some exercises. This led to a y-intercept for the in-clinic model at an UEFM score of 14, such that the model has a floor effect for individuals with UEFM score < 14. During the recruitment, more male patients than female patients were admitted for stroke to the hospital where the trial took place, and through the inclusion screening process, only a few female participants were eligible to participate, able to be contacted, and agreed to participate in the study (n = 4). Further, the randomization process was based on UEFM score alone, which resulted in all the female participants being allocated to the control group of the RCT. The participants who used the system at home were all in the subacute phase of stroke recovery. Individuals in the chronic phase of stroke recovery may not fit this model.

4.4. Future work

As assessment is an important part of the rehabilitation process, systems designed for home-rehabilitation should aim to incorporate periodic assessments. The present study suggests that incorporating a forward-reaching exercise and measuring rep rate is a simple way to estimate UEFM score. Given sufficient fidelity, such a measurement could potentially be used to inform a healthcare provider of a patient's progress, as justification for institutional reimbursements, or serve as a motivator for individuals pursuing their rehabilitation at home unsupervised. To allow this model to be used as a clinical assessment, further research needs to be conducted to verify the test-retest reliability and the model's sensitivity to changes in exercise rate and UEFM. This will require more data taken in the desired setting (home or clinic) paired with clinical outcome measures.

In the FitMi system studied here, such an assessment could be introduced after individuals complete a set number of activities. To match the scenario created for our clinic data, an assessment should encourage participants to perform as many repetitions as they can in a set period of time and should reinforce the importance of performing the activity correctly without compensation. To limit compensation in an unsupervised home environment, IMUs could be placed on the chest and or arm (46, 47), or a motion capture system could monitor trunk motion (48–50).

Improving models such as the ones presented requires accruing larger sets of ground truth data of clinical measurements and sensorized activity. With enough resources, large trials can be conducted to recruit the participants needed, but commercial rehab system vendors and hospitals may represent an untapped dataset for natural experiments. Commercial vendors could offer video sessions with a therapist to collect clinical data, and hospitals using sensorized systems could work with researchers or developers to pair activity data with clinical measures from electronic health records.

An interesting finding was that the multivariate decision tree modeling approach did not improve model performance. This is likely due to the small sample size. Further, we did not attempt to fit a decision tree model to the home data due to the small sample size and the heterogeneity in the exercises performed. Though the decision tree algorithm could theoretically model a data set where not all participants perform the same exercises, again, a larger sample size would be required to produce reliable results. Other nonlinear modeling techniques could be applied, such as a boosted tree method, but their desirability may be limited because many such models would no longer be explanatory or interpretable.

5. Conclusion

In this work, we proposed using the rate of activities completed in a sensorized system to estimate the UEFM. The models presented use a reaching task and a gripping task, and the models captured approximately 70% of the variance in UEFM in our data. This approach could be replicated with simple push button systems that do not require participants to precisely don and doff multiple sensors and could be performed unsupervised in the home setting.

Data availability statement

The data analyzed in this study is subject to the following licenses/restrictions: The data analyzed in this study was obtained from Flint Rehab, LLC. Requests to access these datasets should be directed to Daniel K. Zondervan, dzondervan@flintrehab.com.

Ethics statement

The studies involving human participants were reviewed and approved by the Rancho Research Institute, Inc. Institutional Review Board at Rancho Los Amigos National Rehabilitation Center (IRB #263). The patients/participants provided their written informed consent to participate in this study.

Author contributions

DJR, DKZ, VAS, and CAJ: contributed to the conception and design of the study. SJS: conducted the original trial and data collection. VAS and CAJ: organized and cleaned the data and performed the statistical analysis. VAS: wrote the first draft of the manuscript, and CAJ wrote sections of the manuscript. All authors contributed to the article and approved the submitted version.

Funding

Research reported in this publication was supported by the National Institute on Aging of the National Institutes of Health under Award Number R44AG059256 and the ICT Access for Mobile Rehabilitation (mRehab) Rehabilitation Engineering Research Center, National Institute of Independent Living, Disability, and Rehabilitation Research, 90REGE0011. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Acknowledgments

We would like to thank Dr. Nicolas Schweighofer from the Division of Biokinesiology and Physical Therapy at USC for his consultation on the modeling techniques used in this manuscript. We would like to thank Nicole Bayus, the director of the Clinical Trials Division at Rancho Research Institute where the initial trial was conducted, for the guidance she provided on the data and her contributions to the original trial.

Conflict of interest

DKZ has a financial interest in Hocoma AG and Flint Rehab LLC, companies that develop and sell rehabilitation devices. Flint Rehab produces the FitMi sensor used in this study. The terms of these arrangements have been reviewed and approved by the University of California, Irvine, in accordance with its conflict-of-interest policies. DKZ has a financial interest in Flint Rehab, LLC.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fresc.2023.1181766/full#supplementary-material

Abbreviations

B, both Ischemic and hemorrhagic; BB, box and blocks; EEG, electroencephalogram; H, hemorrhagic; I, Ischemic; IMS, image processing systems; IMU, inertial measurement units; LED, light emitting diode; LOOCV, leave one out cross validation; MAS, modified ashworth scale; MMS, mechanical systems; OMS, optoelectronic systems; RCT, randomized controlled trial; RMSE, root mean squared error; UEFM, upper extremity fugl-meyer; USB, universal serial bus; VAP, visual analog pain.

References

1. Langhorne P, Bernhardt J, Kwakkel G. Stroke rehabilitation. Lancet. (2011) 377(9778):1693–702. doi: 10.1016/S0140-6736(11)60325-5

PubMed Abstract | CrossRef Full Text | Google Scholar

2. World Report on Disability 2011. WHO Guidelines Approved by the Guidelines Review Committee. Geneva: World Health Organization (2011). Available at: http://www.ncbi.nlm.nih.gov/books/NBK304079/

3. Salter K, Jutai J, Zettler L, Moses M, McClure A, Mays R, et al. Outcome measures in stroke rehabilitation. In: Evidence-based review of stroke rehabilitation. (2022). p. 9–73. Available at: http://www.ebrsr.com/evidence-review/20-outcome-measures-stroke-rehabilitation

Google Scholar

4. Santisteban L, Térémetz M, Bleton J-P, Baron J-C, Maier MA, Lindberg PG. Upper limb outcome measures used in stroke rehabilitation studies: a systematic literature review. PLoS ONE. (2016) 11:5. doi: 10.1371/journal.pone.0154792

CrossRef Full Text | Google Scholar

5. Hebert D, Patrice Lindsay M, McIntyre A, Kirton A, Rumney PG, Bagg S, et al. Canadian Stroke best practice recommendations: stroke rehabilitation practice guidelines, update 2015. Int J Stroke. (2016) 11(4):459–84. doi: 10.1177/1747493016643553

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Souza ACd, Alexandre NMC, Guirardello EdB. Psychometric properties in instruments evaluation of reliability and validity. Epidemiologia e Serviços de Saúde. (2017) 26(September):649–59. doi: 10.5123/S1679-49742017000300022

CrossRef Full Text | Google Scholar

7. Oña S, Daniel E, Baeza PS-H, Huete AJ, Balaguer C. Review of automated systems for upper limbs functional assessment in neurorehabilitation. IEEE Access. (2019) 7:32352–67. doi: 10.1109/ACCESS.2019.2901814

CrossRef Full Text | Google Scholar

8. Lee S, Lee Y-S, Kim J. Automated evaluation of upper-limb motor function impairment using fugl-Meyer assessment. IEEE Trans Neural Syst Rehabil Eng. (2018) 26(1):125–34. doi: 10.1109/TNSRE.2017.2755667

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Appleby E, Gill ST, Hayes LK, Walker TL, Walsh M, Kumar S. Effectiveness of telerehabilitation in the management of adults with stroke: a systematic review. PLoS One. (2019) 14(11):e0225150. doi: 10.1371/journal.pone.0225150

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Brennan DM, Mawson S, Brownsell S. Telerehabilitation: enabling the remote delivery of healthcare, rehabilitation, and self management. Stud Health Technol Inform. (2009) 145:231–48. doi: 10.3233/978-1-60750-018-6-231

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Mayo NE. Stroke rehabilitation at home. Stroke. (2016) 47(6):1685–91. doi: 10.1161/STROKEAHA.116.011309

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Mayo NE, Wood-Dauphinee S, Côté R, Gayton D, Carlton J, Buttery J, et al. There's No place like home. Stroke. (2000) 31(5):1016–23. doi: 10.1161/01.STR.31.5.1016

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Hillier S, Inglis-Jassiem G. Rehabilitation for community-dwelling people with stroke: home or centre based? A systematic review. Int J Stroke. (2010) 5(3):178–86. doi: 10.1111/j.1747-4949.2010.00427.x

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Billinger SA, Arena R, Bernhardt J, Eng JJ, Franklin BA, Johnson CM, et al. Physical activity and exercise recommendations for stroke survivors: a statement for healthcare professionals from the American heart association/American stroke association. Stroke. (2014) 45(8):2532–53. doi: 10.1161/STR.0000000000000022

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Hou L, Li M, Wang J, Li Y, Zheng Q, Zhang L, et al. Association between physical exercise and stroke recurrence among first-ever ischemic stroke survivors. Sci Rep. (2021) 11(1):13372. doi: 10.1038/s41598-021-92736-5

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Caeiro L, Ferro JM, Costa J. Apathy secondary to stroke: a systematic review and meta-analysis. Cerebrovasc Dis. (2013) 35(1):23–39. doi: 10.1159/000346076

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Mayo NE, Wood-Dauphinee S, Côté R, Durcan L, Carlton J. Activity, participation, and quality of life 6 months poststroke. Arch Phys Med Rehabil. (2002) 83(8):1035–42. doi: 10.1053/apmr.2002.33984

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Fugl-Meyer AR, Jääskö L, Leyman I, Olsson S, Steglind S. The post-stroke hemiplegic patient. 1. a method for evaluation of physical performance. Scand J Rehabil Med. (1975) 7(1):13–31. doi: 10.2340/1650197771331

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Gladstone DJ, Danells CJ, Black SE. The fugl-Meyer assessment of motor recovery after stroke: a critical review of its measurement properties. Neurorehabil Neural Repair. (2002) 16(3):232–40. doi: 10.1177/154596802401105171

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Sullivan KJ, Tilson JK, Cen SY, Rose DK, Hershberg J, Correa A, et al. Fugl-Meyer assessment of sensorimotor function after stroke: standardized training procedure for clinical practice and clinical trials. Stroke. (2011) 42(2):427–32. doi: 10.1161/STROKEAHA.110.592766

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Kwakkel G, Lannin NA, Borschmann K, English C, Ali M, Churilov L, et al. Standardized measurement of sensorimotor recovery in stroke trials: consensus-based core recommendations from the stroke recovery and rehabilitation roundtable. Int J Stroke. (2017) 12(5):451–61. doi: 10.1177/1747493017711813

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Pan B, Huang Z, Jin T, Wu J, Zhang Z, Shen Y. Motor function assessment of upper limb in stroke patients. J Healthc Eng. (2021) 2021(February):6621950. doi: 10.1155/2021/6621950

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Chen Z-J, He C, Gu M-H, Xu J, Huang X-L. Kinematic evaluation via inertial measurement unit associated with upper extremity motor function in subacute stroke: a cross-sectional study. J Healthc Eng. (2021) 2021:4071645. doi: 10.1155/2021/4071645

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Del Din S, Patel S, Cobelli C, Bonato P. Estimating fugl-Meyer clinical scores in stroke survivors using wearable sensors. Annual international conference of the IEEE engineering in medicine and biology society. IEEE engineering in medicine and biology society. Annual international conference. (2011): 5839–42. doi: 10.1109/IEMBS.2011.6091444

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Rech KD, Salazar AP, Marchese RR, Schifino G, Cimolin V, Pagnussat AS. Fugl-Meyer assessment scores are related with kinematic measures in people with chronic hemiparesis after stroke. J Stroke Cerebrovasc Dis. (2020) 29(1):104463. doi: 10.1016/j.jstrokecerebrovasdis.2019.104463

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Schwarz A, Veerbeek JM, Held JPO, Buurke JH, Luft AR. Measures of interjoint coordination post-stroke across different upper limb movement tasks. Front Bioeng Biotechnol. (2021) 8. doi: 10.3389/fbioe.2020.620805

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Moretti CB, Hamilton T, Edwards DJ, Peltz AR, Chang JL, Cortes M, et al. Robotic kinematic measures of the arm in chronic stroke: part 2—strong correlation with clinical outcome measures. Bioelectron Med. (2021) 7(1):21. doi: 10.1186/s42234-021-00082-8

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Kim W-S, Cho S, Baek D, Bang H, Paik N-J. Upper extremity functional evaluation by fugl-Meyer assessment scoring using depth-sensing camera in hemiplegic stroke patients. PloS One. (2016) 11(7):e0158640. doi: 10.1371/journal.pone.0158640

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Otten P, Kim J, Son SH. A framework to automate assessment of upper-limb motor function impairment: a feasibility study. Sensors. (2015) 15(8):20097–114. doi: 10.3390/s150820097

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Hobart JC, Cano SJ, Zajicek JP, Thompson AJ. Rating scales as outcome measures for clinical trials in neurology: problems, solutions, and recommendations. Lancet Neurol. (2007) 6(12):1094–105. doi: 10.1016/S1474-4422(07)70290-9

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Swanson VA, Johnson C, Zondervan DK, Bayus N, McCoy P, Joshua Ng YF, et al. Optimized home rehabilitation technology reduces upper extremity impairment compared to a conventional home exercise program: a randomized, controlled, single-blind trial in subacute stroke. Neurorehabil Neural Repair. (2023) 73(1):15459683221146996. doi: 10.1177/15459683221146995

CrossRef Full Text | Google Scholar

32. Machin D. Clinical trials: Design, conduct, and analysis. Curtis L. Meinert (1986) monographs in epidemiology and biostatistics. Volume 8. New York and Oxford: Oxford University Press (1988). 469. £55 ISBN 5035682.” Human Psychopharmacology: Clinical and Experimental 3 (2): 153–153. doi: 10.1002/hup.470030214

33. Cromwell FS, United Cerebral Palsy Associations. Occupational Therapist's Manual for basic skills assessment or primary Pre-vocational evaluation. Pasadena, Calif: Fair Oaks Print. Co (1960). https://catalog.hathitrust.org/Record/100818815

34. Wade DT, Wood VA, Heller A, Maggs J, Langton Hewer R. Walking after stroke. Measurement and recovery over the first 3 months. J Rehabil Med. (1987) 19(1):25–30. doi: 10.2340/1650197787192530

CrossRef Full Text | Google Scholar

35. Bohannon RW, Smith MB. Interrater reliability of a modified ashworth scale of muscle spasticity. Phys Ther. (1987) 67(2):206–7. doi: 10.1093/ptj/67.2.206

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Bijur PE, Silver W, Gallagher EJ. Reliability of the visual analog scale for measurement of acute pain. Acad Emerg Med. (2001) 8(12):1153–57. doi: 10.1111/j.1553-2712.2001.tb01132.x

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Verheyden G, Nieuwboer A, Mertin J, Preger R, Kiekens C, De Weerdt W. The trunk impairment scale: a new tool to measure motor impairment of the trunk after stroke. Clin Rehabil. (2004) 18(3):326–34. doi: 10.1191/0269215504cr733oa

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Paci M, Nannetti L, Taiti P, Baccini M, Pasquini J, Rinaldi L. Shoulder subluxation after stroke: relationships with pain and motor recovery. Physiother Res Int. (2007) 12(2):95–104. doi: 10.1002/pri.349

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Folstein MF, Folstein SE, McHugh PR. ‘Mini-Mental state’. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. (1975) 12(3):189–98. doi: 10.1016/0022-3956(75)90026-6

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Mathworks. MATLAB (Version 2020b) [Computer software]. (2020). Available at: https://www.mathworks.com/products/matlab.html.

41. Mathiowetz V, Volland G, Kashman N, Weber K. Adult norms for the box and block test of manual dexterity. Am J Occup Ther. (1985) 39(6):386–91. doi: 10.5014/ajot.39.6.386

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Ramos M, De Jesus E, Swanson VA, Johnson C, Anderson RK, Rabinowitz AR, et al. Using large-scale sensor data to test factors predictive of perseverance in home movement rehabilitation: optimal challenge and steady engagement. Front Neurol. (2022) 13. doi: 10.3389/fneur.2022.809343

CrossRef Full Text | Google Scholar

43. Hijikata N, Kawakami M, Ishii R, Tsuzuki K, Nakamura T, Okuyama K, et al. Item difficulty of fugl-Meyer assessment for upper extremity in persons with chronic stroke with moderate-to-severe upper limb impairment. Front Neurol. (2020) 11. doi: 10.3389/fneur.2020.577855

CrossRef Full Text | Google Scholar

44. Crow JL, Harmeling-van der Wel BC. Hierarchical properties of the motor function sections of the fugl-Meyer assessment scale for people after stroke: a retrospective study. Phys Ther. (2008) 88(12):1554–67. doi: 10.2522/ptj.20070186

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Hsieh Y-W, Hsueh I-P, Chou Y-T, Sheu C-F, Hsieh C-L, Gert K. Development and validation of a short form of the fugl-Meyer motor scale in patients with stroke. Stroke. (2007) 38(11):3052–54. doi: 10.1161/STROKEAHA.107.490730

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Wittmann F, Held JP, Lambercy O, Starkey ML, Curt A, Höver R, et al. Self-directed arm therapy at home after stroke with a sensor-based virtual reality training system. J Neuroeng Rehabil. (2016) 13(August):75. doi: 10.1186/s12984-016-0182-1

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Alankus G, Kelleher C. (2012). Reducing compensatory motions in video games for stroke rehabilitation.” In Proceedings of the SIGCHI conference on human factors in computing systems, 2049–58. Austin Texas USA: ACM. doi: 10.1145/2207676.2208354

48. Sucar LE, Luis R, Leder R, Hernández J, Sánchez I. (2010). “Gesture therapy: a vision-based system for upper extremity stroke rehabilitation.” In 2010 Annual international conference of the IEEE engineering in medicine and biology, 3690–93. doi: 10.1109/IEMBS.2010.5627458

49. Lin S, Mann J, Mansfield A, Wang RH, Harris JE, Taati B. Investigating the feasibility and acceptability of real-time visual feedback in reducing compensatory motions during self-administered stroke rehabilitation exercises: a pilot study with chronic stroke survivors. J Rehabil Assist Technol Eng. (2019) 6(January):2055668319831631. doi: 10.1177/2055668319831631

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Brokaw EB, Eckel E, Brewer BR. Usability evaluation of a kinematics focused kinect therapy program for individuals with stroke. Technol Health Care. (2015) 23(1):143–51. doi: 10.3233/THC-140880

PubMed Abstract | CrossRef Full Text | Google Scholar