Usability Evaluation of a VibroTactile Feedback System in Stroke Subjects

Background: To increase the functional capabilities of stroke subjects during activities of daily living, patients receive rehabilitative training to recover adequate motor control. With the goal to motivate self-training by use of the arm in daily life tasks, a sensor system (Arm Usage Coach, AUC) was developed that provides VibroTactile (VT) feedback if the patient does not move the affected arm above a certain threshold level. The objective of this study is to investigate the usability of this system in stroke subjects.

Method: The study was designed as a usability and user acceptance study of feedback modalities. Stroke subjects with mild to moderate arm impairments were enrolled. The subjects wore two AUC devices one on each wrist. VT feedback was given by the device on the affected arm. A semi-structured interview was performed before and after a measurement session with the AUC. In addition, the System Usability Scale (SUS) questionnaire was given.

Results: Ten ischemic chronic stroke patients (39 ± 38 months after stroke) were recruited. Four out of 10 subjects have worn the VT feedback on their dominant, affected arm. In the pre-measurement interview, eight participants indicated a preference for acoustic or visual over VT feedback. In the post evaluation interview, nine of 10 participants preferred VT over visual and acoustic feedback. On average, the AUC gave VT feedback six times during the measurement session. All participants, with the exception of one, used their dominant arm more then the non-dominant. For the SUS, eight participants responded above 80%, one between 70 and 80%, and one participant responded below 50%.

Discussion: More patients accepted and valued VT feedback after the test period, hence VT is a feasible feedback modality. The AUC can be used as a telerehabilitation device to train and maintain upper extremity use in daily life tasks.

Introduction

To gain independence and increase the quality of life, inpatient neurorehabilitation is usually necessary for hemiparetic stroke subjects (Kollen et al., 2006). The functional capabilities of these patients are assessed using standardized tests, which are intended to predict functional performance after discharge. However, the power of this prediction is poor (Bussmann et al., 2009). Therefore, daily life monitoring of movement quality and quantity would help in guidance of therapy. We previously developed a monitoring solution using a full body inertial sensor suit (Veltink et al., 2014; Klaassen et al., 2015b), with resulting metrics capable of objectifying the quality of movement of stroke subjects. Monitoring in poststroke patients demonstrated that while patients are capable of performing movements during the clinical assessments, they often do not use their affected arm in daily life (van Meulen et al., 2016). These results suggest that capability and arm training does not automatically translate into usage of the affected arm. An unobtrusive coaching system for arm usage during daily life might be able to motivate arm movement in these patients.

In addition to the INTERACTION project, a reduced sensor system was developed with the objective to coach and motivate stroke subjects in remembering to use their affected arm during daily life activities. This Arm Usage Coach (AUC) includes two inertial sensors and one VibroTactile (VT) device. The objective here is to investigate if VT feedback is accepted and the usability of the AUC in stroke subjects during simulated daily life activities. The development of the first prototype and the evaluation with healthy subjects is described in Klaassen et al. (2015a). This paper is a usability study of the first prototype with stroke patients.

Materials and Methods

Study Overview

This study was designed as a usability study, conducted at the University Hospital Zurich, to investigate the usability and the acceptance of the AUC. Stroke subjects with mild to moderate arm impairments were enrolled. A semi-structured interview was performed at enrollment, including a questionnaire, to assess the preference of different types of feedback modalities, e.g., VT, visual, and acoustic feedback among stroke subjects. Then, a measurement session was performed using the AUC to let subjects experience VT feedback, responsive to their arm activity and the overall usage of the device. Afterward another semi-structured interview was done, and the System Usability Scale (SUS) (Brook, n.d.) questionnaire was applied to evaluate the system’s usability. An overview is shown in Figure 1.

FIGURE 1

Figure 1. Flowchart of the study.

Participant Selection

Stroke subjects (above 18 years old) with a unilateral ischemic or hemorrhagic stroke and residual hemiparesis after completion of inpatient rehabilitation were enrolled between March and April 2016. Stroke subjects were required to have a mild to moderate arm impairment with a Fugl-Meyer Assessment upper extremity (FMA-UE, score range 0–66) score higher than 22 (Fugl-Meyer et al., 1975). Additional exclusion criteria were as follows: if the participant has: (1) a major untreated depression, (2) a major cognitive or communication deficits, (3) a major comprehension or memory deficits, (4) major medical comorbidity, (5) severely impaired sensation, (6) sever neglect, and (7) suffering from comprehensive aphasia. Furthermore, the aim for this usability study is to include 10 participants.

Preparation of the Study

The participants gave written informed consent in accordance with the declaration of Helsinki. The Cantonal ethics in Zurich gave approval in using the VT feedback system (nr. 06-2016). Demographic data of the participant (including age, gender, stroke event, work status, technical background, left or right handed, affected side, and arm dimensions) were documented. Furthermore, vibration sense on the affected arm was assessed using the Revised Nottingham sensory assessment (on the wrist) (Stolk-Hornsveld et al., 2006).

Preinterview

A semi-structured interview was performed with each participant before the measurement intervention. The questions, with multiple answering options, are listed in Table 1.

TABLE 1

Table 1. Questions during pre-interview.

AUC Overview

The AUC is composed of two inertial sensors (Xsens B.V.¹) (each weights 27 g), an Elitac (Elitac B.V.²) VT actuator (weighting 200 g), and a laptop (Klaassen et al., 2015a). Both sensors are wirelessly connected via an Xsens dongle, utilizing the Awinda protocol, and the Elitac system via Bluetooth. The inertial sensors are worn on each wrist of the participant. The Elitac VT actuator is placed, with Velcro on the affected arm of the participant (Figure 1). The laptop is operating a software program for providing feedback, including analysis of the sensor data, a decision feature, and feedback.

A mandatory starting pose is required from the participant, which is used as a reference pose to compute arm activity by using a metric called the difference acceleration vector (DAV) (Klaassen et al., 2015a). The length of the DAV d(t) is calculated by subtracting a reference gravitational acceleration vector g(t), obtained from the sensor data captured during the reference pose, from the current acceleration vector a(t) of the sensor during daily life movements, and taking the norm of the resulting vector.

$d\left(t\right)=\sqrt{{\left(a_x\left(t\right)-g_{x,t0}\right)}^2+{\left(a_y\left(t\right)-g_{y,t0}\right)}^2+{\left(a_z\left(t\right)-g_{z,t0}\right)}^2}$

Finally, the mean DAV is calculated over a 1-s time period of measurement data. The final decision of determining whether a certain mean DAV can be seen as arm movement activity is based on more complex algorithms, as explained in Klaassen et al. (2015a). These decision-making algorithms can be personalized by the following two input parameters of the software, namely, (1) threshold of arm activities (between 0 and 9) and (2) the ratio between the affected and non-affected arm usage (0–1, where 1 means the affected side should be used in the same amount as the non-affected side). The outcome parameters of the algorithms are amount of arm usage (when exceeding the threshold mentioned above, for the left and right arm as percentage of combined arm usage) and the amount of feedback provided over time. A default set of input parameters is used for the software for each participant (threshold = 8 m/s² and ratio = 1). The VT feedback is given at 158.3 ± 2.4 Hz and is given for only 489 ms (300 ms duration, and 189 spin-up and down time of the vibration motor to reach the vibration intensity).

Measurement Protocol

At the start of the measurement, participants were asked to don the wristbands, which include inertial sensor holders, then click the sensors into the holder and finally mount the VT actuator on the Velcro-wristband on the affected side. Then, the participants were instructed to stand in a comfortable neutral position. This will be the reference position in which arm activity is detected (Klaassen et al., 2015a).

Next, a selection of tasks, listed in Table 2, is performed by the participant twice in a specific measurement area. This measurement area consists of one room (18 m²) including a table and a chair, with a door leading to a 15 m long hallway. This set of tasks is performed twice, one time where the VT feedback device is OFF and a second time where the device is turned ON for later comparison.

TABLE 2

Table 2. Activity protocol.

Tasks 1, 2, 3, 7, 8, and 10 are based on the protocol presented by van Meulen et al. (2015). The tasks were specifically designed for measuring stroke subjects performing simulated activities of daily living. Participants were instructed to stand up from the chair, walk to a door, open the door, walk through the door, and close the door. Then, the participants were instructed to walk in the hallway for 15 m, turn around and walk 15 m back to the door, open, walk through the door, close the door, and walk to a table. On this table (height 75 cm), four blocks (10, 2.5, 5, and 7.5 cm³), a cricket ball, a sharpening stone, a drinking glass, and a marble were placed. Participants were asked to grasp each object and place them on a shelf. This combined set is part of the ARAT (Lyle, 1981) assessment while standing. After all, items were placed on the shelf, and the participants were instructed to sit down in a chair. After the measurement, arm usage and the amount of feedback that is given were presented in a visual graph on a computer screen, as shown in Figure 2.

FIGURE 2

Figure 2. Right impaired stroke subject with Arm Usage Coach prototype. (1) Inertial sensors; (2) Elitac VibroTactile actuator.

Postinterview

A semi-structured interview was done after the measurements. Two questionnaires were presented to the participants: (1) a custom-made questionnaire as listed in Table 3 and (2) the SUS (Brook, n.d.). The SUS is a well-established 10-item scale, designed to evaluate usability (effectiveness, efficiency, and satisfaction) of technical devices. Questions were scored on a five-point Likert-type scale ranging from “strongly agree” to “strongly disagree.” Combined scores were translated to a range of 0–100, with a higher score meaning better usability (Brook, n.d.). SUS scores above 90 s reflect best imaginable usability, 85 excellent, 71 good, and 50 suggest fair usability. Scores below 50 indicate that using the product or intervention in practice with will be limited due to low compliance (Bangor et al., 2008, 2009). An additional customized questionnaire (Table 4) was designed to gain more insight into the patient’s preferences in terms of feedback after using the AUC and if they would like to use the AUC at home to increase arm function in daily life.

TABLE 3

Table 3. Custom questionnaire during the post interview.

TABLE 4

Table 4. Participant’s characteristic.

Results

Participant Enrollment

Ten subjects of an ischemic stroke (39 ± 38 months after the event) were recruited in the University Hospital Zurich. Four out of 10 participants wore the AUC on the dominant, impaired arm. Six participants had arm FMA-UE score of larger or equal to 48 points; four participants showed poor to limited arm function (FMA-UE ≤ 47 points). Details of each participant are listed in Table 4. Eight participants had impairments in vibration sense (>64 Hz) on the wrist, at the radial and ulnar styloid process and between the processes. Six participants reported to have a technical occupational background. Seven participants have used self-tracking devices before, for example, a pulse watch and walking trackers. One participant used an activity tracker worn on the wrist to monitor his arm movements during daily life.

Preinterview Results

The results from the questionnaire given during the preinterview are listed in Table 5. Seven participants had experience with self-tracking devices, e.g., wrist band, walking trackers, or chest strap to measure heart rate. Seven participants mentioned that they have experience with feedback on arm movement provided by relatives, friends, therapists, or self-tracking devices. Eight participants preferred acoustic or visual over VT feedback based on their experience. Four participants mentioned that they would like to receive feedback hourly or when the arm is not moving, one participant every 15 min, and one patient once daily. All participants agreed on sharing the feedback information with a clinician.

TABLE 5

Table 5. Results interview 1.

Measurement Results with the AUC

All stroke subjects had hand/wrist function (Page et al., 2015) (FMA-UE distal > 8 points, out of 30 points) and were able to done and doff the wristbands, attach the sensors to sensor holders, and mount VT actuator on the wristband, without any additional devices. In Table 6, a summary of the measurement results are listed (over all participants), including arm usage (in percentage of time of combined left/right arm usage) for the impaired and non-impaired arm and the amount of VT feedback. In addition, an example of arm usage and VT feedback as shown by the AUC is shown in Figure 3. Each participant was able to perform the measurement session (length 15 min) and got VT feedback from the device (on average 6 ± 2 times). Overall, the non-impaired side was used in 57 ± 23% of the time during the session compared to the impaired side with 43 ± 24% of the time. Participants did not report to have any obstruction of the device during their activities. One participant did not feel the VT feedback during the simulated daily life activities, which was congruent with the perception impairment of the participant (Table 1).

TABLE 6

Table 6. Summary of the measurement results.

FIGURE 3

Figure 3. Examples of arm usage and VibroTactile (VT) feedback results. (A) Percentage of time of arm usage and (B) VT feedback over time.

Postinterview

Custom Questionnaire

The results from the questionnaire given during the postinterview are listed in Table 7. After the measurement session, nine out of 10 participants mentioned that they like VT actuation as a feedback modality. More so, seven participants liked and found the VT feedback intuitive when the affected arm was not moving. In total, nine participants would utilize the AUC on a daily basis. All participants would share data generated by the system with a clinician. All participants indicated that they would use the AUC as an addition to their routine therapy in everyday life. Furthermore, participants liked the unobtrusiveness of the VT feedback and that the surrounding environment cannot recognize the feedback. Nine out of 10 participants found the VT feedback helpful, when they do not move the impaired arm.

TABLE 7

Table 7. Results custom questionnaire during interview 2.

SUS Results

On average, patients reported a SUS score of 84 (± 20.7) out of 100 points indicating excellent usability (Figure 4) (Bangor et al., 2009). Eight participants scored above 80, one between 70 and 80, and one participant reported poor usability below 50. This individual had the worst FMA-UE score (≤ 40). Nine participants reported in the SUS that they would use the system frequently.

FIGURE 4

Figure 4. System Usability Scale results.

Discussion

The objective of this study was to investigate if VT feedback is accepted and usability of the AUC in stroke subjects during a daily life activities simulation. Based on the inclusion criteria, all included stroke subjects were able to move there affected hand and were able to mount the AUC to the wrist. In total, 70% of the participants said that they would like to obtain feedback when the impaired arm was not moving during certain activities. This indicates good acceptance of the device. In total, 9 out of 10 participants were able to feel the vibration on their impaired arm. Therefore, it appears that participants accepted VT feedback. All participants reported that they would like to use the device, that it complements their current therapy, and that they prefer to share the data with a clinician.

Patients agree to send the data from the AUC to a care professional to check on their progress and address this during therapy sessions, this could help to adapt the rehabilitation for upper extremity to the patients needs. This indicates that AUC could be used as a telemonitoring and -rehabilitation devices for upper extremity. The usability reported with the SUS was high, with 84 out of 100 points on average for all 10 participants. The usability of the AUC is therefore classified as excellent according to Bangor et al. (2009). The AUC however could, according to patients, be improved by being smaller and waterproof. It is increasingly unlikely that new usability problems will be uncovered by including more stroke subjects (Virzi, 1992).

Physical activity coaches, who mostly use accelerometers, implement different feedback strategies (in form of graphs, push-messages, VT feedback) to encourage active behavior during daily life (Cabrita et al., 2015; Achterkamp et al., 2016). In stroke, feedback is used to investigate certain interventions (e.g., constrained induced movement therapy) (Bonifer et al., 2005; Harris et al., 2009; Shi et al., 2011), to correct postures of patients during specific tasks (Ding et al., 2013), or to improve motor learning capabilities (Lieberman and Breazeal, 2007). Moreover, many studies showed that multimodal feedback strategies, proved to be effective to improve performance of patients in various tasks and scenarios (Burke et al., 2006; Prewett et al., 2006; Causo et al., 2012; Trejo-Gabriel-Galan et al., 2016). Most of these studies are performed in a lab environment, and therefore have a reduced interest in the social context of the patients. We designed the AUC based on two feedback strategies: first, knowledge of performance, implemented by VT feedback, which is given during simulated daily life tasks in this study. The second is knowledge of results, which is given through visual feedback where the arm usage is shown in a bar graph, in percentage of the left and right arm and the number of feedbacks within a given time period. This combines the real-time VT feedback with post-visual feedback. This differs by most work done in multimodal strategies, which in most studies gives a combined (near) real-time feedback. In our design, we aimed for an unobtrusive and wearable design during daily life, without the direct need of smartphone apps to make it more applicable and intuitive for stroke patients. Acoustic feedback is obtrusive in social settings and was not implemented. Furthermore, our visual feedback needs longer data processes in order to “make sense” (arm usage); therefore, there is no need for direct visual feedback. Systems to train the upper extremity function with VT feedback in stroke patients have been previously developed (Kapur et al., 2009; Acuna et al., 2010; Bark et al., 2011; Hung et al., 2015). It is known that intensive training after stroke has a positive effect in clinical outcome, but the effect of VT feedback on arm function is unclear (Hung et al., 2015). The AUC could provide the opportunity to increase the arm usage in daily life, thereby training intensity and time by providing VT feedback.

We did not observe an effect of the AUC on arm usage of the impaired side. This is due to the short observation period. Because the main objective here was to test the usability and acceptance of the system, hence a short measurement time was selected.

The combination of monitoring and training stroke patients in daily life with VT feedback is new and could be realized by using a smaller, waterproofed version of the AUC. Computational tasks should be performed on the sensor, rather than on a laptop or a smartphone.

Based on the results of this usability study, an efficacy study, with extended protocol and pre defined outcome parameter, could evaluate the impact of VT feedback on the stroke subjects arm movements in daily life activities.

Author Contributions

BK and JH drafted the manuscript and analyzed the data. JH performed patient measurements inside the hospital with remote supervision of BK. BK prepared, tested, and optimized the measurement system. B-JB helped in drafting the manuscript and assisted with data interpretation. AL, PV, and B-JB supervised the research. All the authors read, corrected/commented, and approved the final manuscript.

Conflict of Interest Statement

AL is a scientific advisor for Hocoma AG (Volketswil, Switzerland), which develops rehabilitation technology. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Funding

This work was partially supported by the INTERACTION project, which was partially funded by the European Commission under the Seventh Framework Programme (FP7-ICT-2011-7-287351).

Footnotes

1. ^http://xsens.com.
2. ^http://Elitac.nl.

References

Achterkamp, R., Dekker-Van Weering, M. G., Evering, R. M., Tabak, M., Timmerman, J. G., Hermens, H. J., et al. (2016). Strategies to improve effectiveness of physical activity coaching systems: development of personas for providing tailored feedback. Health Informatics J. doi: 10.1177/1460458216653242

PubMed Abstract | CrossRef Full Text | Google Scholar

Acuna, M., Amasay, T., and Karduna, A. R. (2010). The reliability of side to side measurements of upper extremity activity levels in healthy subjects. BMC Musculoskelet. Disord. 11:168. doi:10.1186/1471-2474-11-168

PubMed Abstract | CrossRef Full Text | Google Scholar

Bangor, A., Kortum, P., and Miller, J. (2009). Determining what individual SUS scores mean: adding an adjective rating scale. J Usability Stud. 4, 114–123.

Google Scholar

Bangor, A., Kortum, P. T., and Miller, J. T. (2008). An empirical evaluation of the system usability scale. Int. J. Hum. Comput. Interact. 24, 574–594. doi:10.1080/10447310802205776

CrossRef Full Text | Google Scholar

Bark, K., Khanna, P., Irwin, R., Kapur, P., Jax, S. A., Buxbaum, L. J., et al. (2011). “Lessons in using vibrotactile feedback to guide fast arm motions,” in 2011 IEEE World Haptics Conference (Istanbul), 355–360.

Google Scholar

Bonifer, N. M., Anderson, K. M., and Arciniegas, D. B. (2005). Constraint-induced movement therapy after stroke: efficacy for patients with minimal upper-extremity motor ability. Arch. Phys. Med. Rehabil. 86, 1867–1873. doi:10.1016/j.apmr.2005.04.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Brook, J. (n.d.). SUS: A Quick and Dirty Usability Scale. [Online]. Available at: https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html [accessed December 29, 2016].

Google Scholar

Burke, J. L., Prewett, M. S., Gray, A. A., Yang, L., Stilson, F. R., Coovert, M. D., et al. (2006). “Comparing the effects of visual-auditory and visual-tactile feedback on user performance: a meta-analysis,” in Proceedings of the 8th International Conference on Multimodal Interfaces (Banff: ACM), 108–117.

Google Scholar

Bussmann, J. B., Ebner-Priemer, U. W., and Fahrenberg, J. (2009). Ambulatory activity monitoring: progress in measurement of activity, posture, and specific motion patterns in daily life. Eur. Psychol. 14, 142–152. doi:10.1027/1016-9040.14.2.142

CrossRef Full Text | Google Scholar

Cabrita, M., Nassabi, M. H., Op Den Akker, H., Tabak, M., Hermens, H., and Vollenbroek, M. (2015). “An unobtrusive system to monitor physical functioning of the older adults: results of a pilot study,” in International Workshop on Personalization and Adaptation in Technology for Health 2015 (PATH 2015). Dublin.

Google Scholar

Causo, A., Dung, T. L., Yeo, S. H., and Chen, I. (2012). Visual cue and vibrotactile in series configuration: multimodal feedback design for Arm posture correction. J. Med. Imag. Health Inform. 2, 430–437. doi:10.1166/jmihi.2012.1118

CrossRef Full Text | Google Scholar

Ding, Z. Q., Luo, Z. Q., Causo, A., Chen, I. M., Yue, K. X., Yeo, S. H., et al. (2013). Inertia sensor-based guidance system for upperlimb posture correction. Med. Eng. Phys. 35, 269–276. doi:10.1016/j.medengphy.2011.09.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Fugl-Meyer, A. R., Jaasko, L., Leyman, I., Olsson, S., and Steglind, S. (1975). The post-stroke hemiplegic patient. 1. A method for evaluation of physical performance. Scand. J. Rehabil. Med. 7, 13–31.

PubMed Abstract | Google Scholar

Harris, J. E., Eng, J. J., Miller, W. C., and Dawson, A. S. (2009). A self-administered graded repetitive Arm supplementary program (GRASP) improves Arm function during inpatient stroke rehabilitation a multi-site randomized controlled trial. Stroke 40, 2123–2128. doi:10.1161/STROKEAHA.108.544585

PubMed Abstract | CrossRef Full Text | Google Scholar

Hung, C. T., Croft, E. A., and Machiel Van der Loos, H. F. (2015). “A wearable vibrotactile device for upper-limb bilateral motion training in stroke rehabilitation: a case study,” in Engineering in Medicine and Biology Society (EMBC), 2015 37th Annual International Conference of the IEEE (Milan: IEEE), 3480–3483.

Google Scholar

Kapur, P., Premakumar, S., Jax, S. A., Buxbaum, L. J., Dawson, A. M., and Kuchenbecker, K. J. (2009). “Vibrotactile feedback system for intuitive upper-limb rehabilitation,” in Third Joint Eurohaptics Conference and Symposium on Haptic Interfaces (Salt Lake City).

Google Scholar

Klaassen, B., Bartels, P. G. H., van Beijnum, B. J. F., and Hermens, H. (2015a). “The development and evaluation of an arm usage coach for stroke survivors,” in 2015 9th International Conference on Sensing Technology (ICST) (Auckland: IEEE), 514–519.

Google Scholar

Klaassen, B., Van Beijnum, B. J., Weusthof, M., Hofs, D., Van Meulen, F., Droog, E., et al. (2015b). “A full body sensing system for monitoring stroke patients in a home environment,” in Biomedical Engineering Systems and Technologies: 7th International Joint Conference, BIOSTEC 2014, Angers, France, March 3–6, 2014, Revised Selected Papers, eds. G. Plantier, T. Schultz, A. Fred, and H. Gamboa (Cham: Springer International Publishing), 378–393.

Google Scholar

Kollen, B., Kwakkel, G., and Lindeman, E. (2006). Functional recovery after stroke: a review of current developments in stroke rehabilitation research. Rev. Recent Clin. Trials 1, 75–80. doi:10.2174/157488706775246111

PubMed Abstract | CrossRef Full Text | Google Scholar

Lieberman, J., and Breazeal, C. (2007). TIKL: development of a wearable vibrotactile feedback suit for improved human motor learning. IEEE Trans. Robot. 23, 919–926. doi:10.1109/TRO.2007.907481

CrossRef Full Text | Google Scholar

Lyle, R. C. (1981). A performance test for assessment of upper limb function in physical rehabilitation treatment and research. Int. J. Rehabil. Res. 4, 483–492. doi:10.1097/00004356-198112000-00001

CrossRef Full Text | Google Scholar

Page, S. J., Hade, E., and Persch, A. (2015). Psychometrics of the wrist stability and hand mobility subscales of the Fugl-Meyer Assessment in moderately impaired stroke. Phys. Ther. 95, 103–108. doi:10.2522/ptj.20130235

CrossRef Full Text | Google Scholar

Prewett, M. S., Yang, L., Stilson, F. R., Gray, A. A., Coovert, M. D., Burke, J., et al. (2006). “The benefits of multimodal information: a meta-analysis comparing visual and visual-tactile feedback,” in Proceedings of the 8th International Conference on Multimodal Interfaces (Banff: ACM), 333–338.

Google Scholar

Shi, Y. X., Tian, J. H., Yang, K. H., and Zhao, Y. (2011). Modified constraint-induced movement therapy versus traditional rehabilitation in patients with upper-extremity dysfunction after stroke: a systematic review and meta-analysis. Arch. Phys. Med. Rehabil. 92, 972–982. doi:10.1016/j.apmr.2010.12.036

PubMed Abstract | CrossRef Full Text | Google Scholar

Stolk-Hornsveld, F., Crow, J., Hendriks, E., Van Der Baan, R., and Harmeling-Van Der Wel, B. (2006). The Erasmus MC modifications to the (revised) Nottingham Sensory Assessment: a reliable somatosensory assessment measure for patients with intracranial disorders. Clin. Rehabil. 20, 160–172.

Google Scholar

Trejo-Gabriel-Galan, J. M., Rogel-Melgosa, V., Gonzalez, S., Sedano, J., Villar, J. R., and Arenaza-Basterrechea, N. (2016). Rehabilitation of hemineglect of the left arm using movement detection bracelets activating a visual and acoustic alarm. J. Neuroeng. Rehabil. 13, 79. doi:10.1186/s12984-016-0191-0

PubMed Abstract | CrossRef Full Text | Google Scholar

van Meulen, F. B., Klaassen, B., Held, J., Reenalda, J., Buurke, J. H., van Beijnum, B.-J. F., et al. (2016). Objective evaluation of the quality of movement in daily life after stroke. Front. Bioeng. Biotechnol. 3:210. doi:10.3389/fbioe.2015.00210

CrossRef Full Text | Google Scholar

van Meulen, F. B., Reenalda, J., Buurke, J. H., and Veltink, P. H. (2015). Assessment of daily-life reaching performance after stroke. Ann. Biomed. Eng. 43, 478–486. doi:10.1007/s10439-014-1198-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Veltink, P. H., van Meulen, F. B., van Beijnum, B. J. F., Klaassen, B., Hermens, H., Droog, E., et al. (2014). “Ambulatory 370 gait analysis in stroke patients using ultrasound and inertial sensors,” in 13th International Symposium on 3D Analysis of Human Movement (3D-AHM 2014) (Lausanne, Switzerland), 159–162.

Google Scholar

Virzi, R. A. (1992). Refining the test phase of usability evaluation: How many subjects is enough? Hum. Fact. 34, 457–468.

Google Scholar