Accuracy, concurrent validity, and test–retest reliability of pressure-based insoles for gait measurement in chronic stroke patients

Introduction Wearables are potentially valuable tools for understanding mobility behavior in individuals with neurological disorders and how it changes depending on health status, such as after rehabilitation. However, the accurate detection of gait events, which are crucial for the evaluation of gait performance and quality, is challenging due to highly individual-specific patterns that also vary greatly in movement and speed, especially after stroke. Therefore, the purpose of this study was to assess the accuracy, concurrent validity, and test–retest reliability of a commercially available insole system in the detection of gait events and the calculation of stance duration in individuals with chronic stroke. Methods Pressure insole data were collected from 17 individuals with chronic stroke during two measurement blocks, each comprising three 10-min walking tests conducted in a clinical setting. The gait assessments were recorded with a video camera that served as a ground truth, and pressure insoles as an experimental system. We compared the number of gait events and stance durations between systems. Results and discussion Over all 3,820 gait events, 90.86% were correctly identified by the insole system. Recall values ranged from 0.994 to 1, with a precision of 1 for all measurements. The F1 score ranged from 0.997 to 1. Excellent absolute agreement (Intraclass correlation coefficient, ICC = 0.874) was observed for the calculation of the stance duration, with a slightly longer stance duration recorded by the insole system (difference of −0.01 s). Bland–Altmann analysis indicated limits of agreement of 0.33 s that were robust to changes in walking speed. This consistency makes the system well-suited for individuals post-stroke. The test–retest reliability between measurement timepoints T1 and T2 was excellent (ICC = 0.928). The mean difference in stance duration between T1 and T2 was 0.03 s. We conclude that the insole system is valid for use in a clinical setting to quantitatively assess continuous walking in individuals with stroke.


Introduction
With over 12.2 million new cases worldwide, stroke is a significant global health concern and one of the leading causes of disability in the western world (1,2).Individuals with stroke suffer from motor impairments that substantially affect their gait function (3).In comparison to their healthy counterparts, individuals with stroke frequently display reduced mobility, which negatively influences their participation in everyday activities (4,5).Therefore, rehabilitation goals commonly focus on enhancing activities such as mobility and participation in daily life (6).One of the main aims of stroke rehabilitation is to achieve each patient's potential for recovery.This includes patient-tailored counseling to maintain and increase physical activity and self-training outside the therapeutic setting (7).This patient-tailored counseling involves a continuous cycle of identification of a patient's trajectory, personalized and precise therapy, and continuous monitoring of progress during the entire patient pathway (8,9).
Wearables have the potential to serve as valuable tools across the entire patient pathway, by providing assessments of gait and mobility of individuals with stroke both in clinical and home settings.While standardized clinical assessments capture a snapshot of a patient's capacity within a controlled clinical setting, wearables present the opportunity to collect additional information about the quality and quantity of movement, as well as a patient's performance in day-to-day activities (10,11).This approach provides patients, clinicians, and researchers with the ability to continuously monitor the patient pathway, document progress, and gain insights into the capacityperformance relationship (12).It also assesses whether progress made in rehabilitation can be successfully transferred to daily life (13).A systematic review of the application of wearables in gait rehabilitation of individuals with stroke revealed that accelerometers, activity monitors, and pressure sensors are frequently employed, particularly in hospital-based and inpatient settings.Commonly evaluated gait parameters include gait speed, cadence, step count, and duration of activity (14).
Most metrics for the evaluation of gait performance and quality center around the accurate detection of two main gait events: the foot strike (FS; when the foot first touches the ground), and the foot off (FO; when the foot first leaves the ground) (15).These events, which are well-established and well-described anchors for the calculation of gait quality features, are also relevant for measures of gait quantity, such as the number of steps (16).Therefore, as a first principle, a robust, reliable, and accurate detection of gait events is paramount in a gait measuring system to make it fit for purpose.This has proven challenging over past decades of research, especially in populations with neurological conditions, in which gait patterns are extremely individualspecific and both gait patterns and movement speed are highly variable (17).Compounding this, movement patterns and speeds both change over the course of rehabilitation.
Pressure sensors integrated into insoles are adaptable to various shoe types and contain a standardized arrangement of pressure sensors to ensure both validity and comfort (18).Pressure insoles capture the force interactions between the foot surface area and the ground, presenting an intuitive, straightforward, and reliable method for gait event detection (19).Pressure-based measurement systems such as pressure mats and foot switches have been used as a silver standard for validation and algorithm development studies (20,21).Various techniques, including piezoresistive, resistive, capacitive, and piezoelectric methods, can be used as mechanisms for sensing plantar force.Different sensor technologies come with distinct sets of advantages and disadvantages.Piezoresistive and resistive methods have a broad force measurement range at a budget-friendly cost but involve the challenges of low repeatability and susceptibility to temperature and humidity.Conversely, capacitive and piezoelectric methods offer increased force sensitivity, but they are susceptible to changes in temperature and interference from electromagnetic sources (22).
Piezoresistive insoles have been validated in a population with neurological conditions aside from stroke, demonstrating their capability to accurately identify 94.5% of all steps, with recall and precision exceeding 0.98 when compared to a Vicon motion capture system (23).In healthy populations, piezoresistive pressure sensors exhibit accuracy levels ranging from an Intraclass correlation coefficient (ICC) of 0.86-0.97,depending on walking speed (24).The FeetMe ® Monitor system, validated in individuals with chronic stroke, exhibits an ICC > 0.77 and test-retest reliability of ICC > 0.9 (18).In gait rehabilitation, pressure-based insoles have already been used at different stages of the patient pathway, for example, as a supplementary tool for automatically assessing gait or for identifying activity levels and gait patterns (25-27).David et al. (28) evaluated the rehabilitation progress of 35 individuals in the acute and chronic phase of stroke using 90 s of level walking and the Timed Up and Go Test.Gait parameters, including spatiotemporal (e.g., stride time, step time), kinematic (e.g., angle at FS/FO), and gait cycle events, were assessed using the insole system eSHOE.Before and after four weeks of inpatient rehabilitation, the insoles, which are equipped with a triaxial accelerometer; a triaxial gyroscope; a triaxial magnetometer; and pressure sensors under the big toe, metatarsal heads Ⅰ and Ⅴ, and heel, were used to monitor rehabilitation progress.In a study by Munoz-Organero et al. (29), various walking patterns, including gait asymmetry, heel walking, and a low heel pressure strategy, were assessed in individuals in different rehabilitation stages after stroke during a 10-min walking trial.The insole system consists of eight force-sensitive resistors per insole, which collects data at a frequency of 100 Hz that were further integrated into a personalized self-management rehabilitation system designed for gait relearning.Similarly, a shoe-based sensor system (SmartShoe), which includes five force-sensitive resistors in each flexible insole, has demonstrated acceptable validity for the monitoring of physical activity and common activities of daily life in individuals who are at least three months post-stroke (22).
The major challenge of using wearables for gait measurement in individuals with stroke lies in the precise and consistent measurement of the gait patterns typically associated with hemiparesis.These patterns include decreased stance time on the affected side of the body, longer stance time on the unaffected side, and reduced stride frequency and step length, and all contribute to an asymmetrical walking pattern with fluctuating walking accelerations (30,31).Despite the widespread use of wearables in research, their application for gait analysis involves limitations in the validity and reliability of spatiotemporal parameters and gait events in individuals with stroke (14).
To address these limitations in validity and reliability of spatiotemporal parameters in individuals with stroke, the purpose of this study was to assess the accuracy, concurrent validity, and test-retest reliability of a commercially available insole system in the detection of gait events and the calculation of stance duration in individuals with chronic stroke.This was accomplished by comparing the data collected from the insoles with a camerabased reference system during continuous walking in a 10-min walking test (10mWT).

Participants
Individuals with stroke were recruited from the University Hospital Zurich research database.Inclusion criteria were age ≥ 18 years, chronic stage of stroke (≥ six months since the incident), and ability to walk without assistance on a level surface for at least six minutes (Functional Ambulation Category, FAC ≥ 3).The use of assistive devices was allowed.Potential participants were excluded if they had gait deficits unrelated to stroke or were unable to understand verbal instructions.Seventeen individuals with chronic stroke were recruited for this project.The study was approved by the cantonal ethics committee (BASEC-Nr.Req-2019-00565, Ethics Committee Zurich, Zurich, Switzerland).All participants provided written informed consent after being informed about the study procedure.

Experimental protocol
The participants were invited to attend a single measurement event consisting of two sessions of approximately one hour each.After obtaining informed consent, demographic information and outcome measures were collected to characterize the sample.This included age, gender, time since stroke, type of stroke, use of walking aids, 10mWT, gait speed, six-minute walk test, and Motricity Index.All participants were equipped with insoles placed within their personal footwear while performing two measurement blocks (timepoints T1 and T2), each with three repetitions of the 10mWT.A break of at least 30 min separated the two measurement blocks; during this break, the measurement system was removed from the shoe and charged.Post-break, system mounting was performed from scratch, facilitating the evaluation of test-retest reliability while minimizing the impact of day-to-day variation.All gait tests were recorded with both measurement systems, which were started and synchronized before each test.

Measurement system
A Stappone insole system (version 1.0, stapp one, Vienna, Austria) collected plantar pressure with 12 resistive textile pressure sensors in specific zones under the foot.Additionally, it comprised a triaxial accelerometer, a battery, onboard buffer storage, and a Bluetooth Low Energy 4.2 emitter for each insole.Once the data was collected, parameters were processed onboard the system using a proprietary adaptive thresholding algorithm.The sampling frequency was set to 100 Hz.Raw data for acceleration, plantar pressure, and gait event timings (FS and FO) were recorded and transferred via Bluetooth to the Stappone software (version 0.9.5.6, stapp one, Vienna, Austria) after each trial.The pressure calibration of the insoles was automatically adjusted by the software during postprocessing after each measurement was taken.The insole data were analyzed using an internal threshold-based algorithm for gait event detection.The sensor system was lightweight and could be worn for 36 h before needing a recharge.
A GoPro Hero 8 camera (GoPro, Inc., San Mateo CA, USA) served as the reference system because, unlike optoelectronic systems, it does not suffer from the uncertainty associated with human gait labeling (15).The use of manual labeling can lead to inaccuracies due to human error.To minimize these errors, the four-eyes principle was introduced.The camera was securely mounted on a rolling trolley at the height of the feet to capture all trials from a low vantage point with minimal occlusions (Figure 1).The sampling frequency was set to 120 Hz with a resolution of 1,080 pixels and no lens correction.During postprocessing, the gait events (FS and FO) were manually labeled frame by frame by one rater, and the timing was calculated as the time elapsed since the start signal.
Measurement signals were synchronized for each trial using an audio output that was generated by the insoles when the measurement was started.The GoPro was connected to the audio output using a microphone adapter, which recorded the start signal as a sound burst.During postprocessing, this burst was identified with a threshold algorithm and verified manually.

Data preprocessing
The insole system and the reference system were used to assess the stance duration for both the affected and the unaffected side.The accuracy, concurrent validity, and testretest reliability of stance duration were analyzed from the steps taken in the six 10mWTs per participant.Stance duration, defined as the time between FS and the subsequent FO of the same foot, was calculated for every step.The stance duration was calculated once based on the insole timings and once based on the video reference timings.The timings of FS and FO were calculated as the time elapsed since the starting signal of the video.To ensure correspondence between FS and FO, e.g., that both belonged to the same step on the same side, a plausibility check was performed to remove all out-of-distribution values.

Statistical analysis
The accuracy of the event detection was assessed with the F1 score to balance precision and recall.Precision was defined as the total number of true positive (TP) identifications by the insole system and the reference system divided by the sum of the TPs and the false positives (FPs; events detected by the insole system but not by the reference system).
Recall was defined as the number of TPs divided by the sum of the TPs and the false negatives (FNs; events identified by the reference system but not by the insole system).
The F1 score was used to balance precision and recall and was defined as: To evaluate the concurrent validity of the insoles, the differences between the stance durations measured by the insole system and by the reference system were analyzed.Normality of the distributions of stance duration was checked by visual inspection of QQ-plots by two investigators.For conflicting ratings, a consensus was reached through discourse.Subsequently, a paired two-sided t-test (α < 0.05) was employed to detect statistically significant differences between the systems.Furthermore, the agreement of the stance durations between the two measurement systems was characterized using Bland-Altman plots.Such plots have a strong history in the comparison of reference systems to experimental systems in the medical field (32, 33).Their utility lies in the fact that they not only compare means over the given range of the test population but also enable the identification of fixed and variable bias individually.Both features are important in judging whether a system is sufficiently precise for a given application and population.
Finally, the ICC 2,1 was used to examine the agreement between the two measurement systems.The ICC 2,1 was calculated using the following equation: where MS R = mean square for rows, MS E = mean square for error, and k = number of raters/measurements (34).According to Fleiss, an ICC ≥ 0.75 is considered "excellent," 0.40 ≤ ICC < 0.75 is considered "moderate to good," and ICC < 0.40 is considered "poor" (35).The ICC was applied, as it provides a measureindependent metric of agreement that allows for comparison between different metrics and measurement systems.
In a last comparison, we evaluated test-retest reliability for each system by comparing stance duration data from T1 with data obtained at T2.Differences were identified using a paired two-sided t-test after a visual check for normality of the distributions using QQ-plots.The ICC 2,1 (34) was again applied to permit comparability of reliability to other systems and measures.Experimental setup for the 10mWT.Participants completed three repetitions of the 10mWT while wearing the insole system within their personal footwear.As the participants walked, the GoPro camera mounted on the rolling trolley was guided by an assistant walking alongside to capture the gait events during the 10mWT`s.

Results
Seventeen individuals with chronic stroke, seven women and ten men, were recruited for the study.The average age was 65.5 years, and the mean time since stroke was 54.7 months.While seven participants were using walking aids outdoors, only five used walking aids during the trials.Descriptive information on the participants' characteristics is provided in Table 1.
Of the 17 participants, one participant was excluded from the analysis due to a measurement system failure.The data from six additional 10mWT trials were not usable due to failure of either synchronization or recording initiation.Additionally, two trials were excluded due to a scrambled gait event sequence proposed by the internal algorithm of the insole system (e.g., multiple consecutive FOs from the same side).Consequently, a total of 88 10mWT trials from 16 participants were included in this analysis.The dataset comprised 3,820 gait events, encompassing 1910 FSs and FOs, covering 995 steps on the right side and 915 steps on the left side.

Gait event detection accuracy
Of all 3,820 gait events, 90.86% were correctly identified by the insole system.A cumulative confusion matrix illustrates gait event detection for continuous walking in all participants by summation (Figure 2).The data analysis showed that 91 right FSs, 86 right FOs, 87 left FSs, and 85 left FOs were not identified as gait events by the insole system.Further investigation showed that all missing events corresponded to either the first FO or the last FS of the trial.To simulate continuous walking, these gait events were excluded from further analysis.Recall values ranged from 0.994 to 1, with a precision of 1 for all measurements.The F1 score ranged from 0.997 to 1, as shown in Figure 2.

Stance duration concurrent validity
Concurrent validity of stance duration was determined through comparison of the insole-derived timings and the video reference measurements.FP events were excluded from the analysis.Sixteen left (1.96% of a total of 815) and 17 right (1.94% of a total of 878) stance durations were excluded as outliers.No systematic reason was identified for the causation of these outliers.
The normality of the distributions of stance duration data was confirmed by both raters using the QQ-plots.The t-test performed on the stance durations calculated by the insoles and the reference system revealed no significant difference ( p = 0.06).The stance durations recorded by the insole system tended to be slightly longer than those recorded by the reference system (difference of −0.01 s).The limits of agreement showed a deviation of insole measurements of up to 0.33 s (Figure 3).The correlation analysis between the insole and the video reference system was robust, with an excellent ICC 2,1 value of 0.874.
To enhance the visual clarity of our analysis, we have provided two alternative visualizations of the data reported by the Bland-Altman plot.Specifically, we show the difference in stance duration between both systems categorized by affected vs. unaffected side (Figure 4).Furthermore, we depict differences in stance duration of three categories of ambulators (Figure 5): limited community ambulators (<0.8 m/s), community ambulators (0.8-1.4 m/s), and those able to cross the street safely (>1.4 m/s) (36).

Stance duration test-retest reliability
The normality of the distributions of stance duration data between T1 and T2 was confirmed by both raters using the QQplots.The paired, two-sided t-test indicated no significant difference (p = 0.44) between the two measurement blocks, T1 and T2.The test-retest reliability between measurement timepoints T1 and T2 was excellent, with an ICC 2,1 of 0.928.The mean difference in stance duration between T1 and T2 was 0.03 s.Bland-Altman plot for the comparison of the insole system with the reference system.The mean of the differences is indicated by the solid black line, and 95% limits of agreement are indicated by the dashed lines.The aim of this study was to assess the accuracy, concurrent validity, and -reliability of an insole system in individuals with stroke-related gait deficits during a 10mWT, measured across two sessions in a clinical setting.We targeted a wide functional range of patients to reflect the clinical reality of rehabilitation.

Gait event detection accuracy
The insole system correctly identified 90.86% of the 3,820 recorded gait events.This result aligns with a study that correctly identified 94.5% of all steps in a population with neurological conditions by employing insoles equipped with a piezoresistive device (23).Both recall and precision were >0.98.These rates are comparable to or exceed those reported in studies using accelerometers in individuals with stroke (37,38).The insole system failed to detect 9.14% of the gait events.Analysis revealed that the initial FOs at the start of the trial and the final FSs at the end of the trial were consistently missed because participants initiated and concluded trials in a standing position.Simulating continuous walking by removing the first FOs and the last FSs from the analysis resulted in a detection error rate of only 0.2%.Among these erroneously detected gait events, eight were FSs detected before the participant started walking, suggesting potential algorithm sensitivity to weight shifts without actual foot movement (39).

Stance duration concurrent validity
The validity of the stance duration measurement was excellent.In comparison to the FeetMe ® Monitor system (ICC > 0.77), which was validated in individuals with chronic stroke over three 8-min trials (18), the utilized insole system demonstrated superior performance.Our results align with the accuracy of the ZeroWire ® footswitch system, which has four piezoresistive pressure sensors on each foot and was used to record stance time in healthy participants; its accuracy ranged from an ICC of 0.86 to 0.97, depending on walking speed (24).Due to the increased variability and decreased walking speed that are often found in pathological gait, Inertial Measurement Units (IMU) often report reduced accuracy at low walking speeds of 0.4 m/s or severe deviations from normal human gait (40,41).This is reflected in the lower accuracy of IMU sensors used to detect stance duration in individuals with stroke (42).While we did not record any individuals walking at less than 0.4 m/s, the insole system performed remarkably well for 0.5 m/s-0.6 m/s speeds, which are associated with long stance durations (43).This consistency indicates that the system should function well, even for slowwalking individuals post-stroke.

Stance duration test-retest reliability
The high level of agreement between the two measurement timepoints, T1 and T2, as indicated by the ICC 2,1 of 0.92, is Boxplot representation of the difference in stance duration [ms] between the video system and the insole system for the affected and unaffected side.Boxplot representation of the difference in stance duration [ms] between the video system and the insole system for both sides together.We show how the stance duration difference compares between different levels of walking ability.consistent with findings for the FeetMe ® Monitor system (ICC > 0.9) (18).The difference between T1 and T2 can be attributed to typical variation in stance duration (44).These findings indicate that the measures are repeatable over time, even if the insole is removed and reinstalled in the shoe.It is important to note that a short pressure calibration was performed after each introduction of the system into the shoe to compensate for changes in lacing pressure.

Potential use of the insole system in clinical practice
Alterations in walking patterns or the ability to walk can serve as a valuable source of information regarding patient trajectories throughout the rehabilitation pathway and beyond.Gait strategies are influenced by various clinical factors, including cardiovascular, and musculoskeletal conditions; cardiovascular and metabolic diseases; age-related changes; and trauma (9).Wearables have found extensive application in research, and their utilization is expanding into clinical settings.They can provide information about spatiotemporal, kinematic, and kinetic parameters.Nevertheless, caution is advised in the application of these devices, as they are specifically designed to extract certain parameters in particular patient populations.Therefore, these applications need to undergo a validation process in patients with specific diseases or conditions in clinical settings, as well as in real-world conditions if they are employed in such environments (45).Most importantly, the information gathered using these sensors must align with clinical endpoints and have clinical relevance (46).This ensures that such information can be effectively used by clinicians to provide valuable additional insights into individual health conditions and to facilitate clinical decision-making.
Our findings suggest that the insole system is applicable for assessing stance duration in individuals post-stroke.However, this conclusion is limited to clinical settings and continuous, straight walking.Further research is necessary to determine whether these findings can be transferred to a real-world environment and various activities of daily life.

Conclusion
In conclusion, the insole system is accurate, valid, and reliable for detecting gait events and calculating stance duration during continuous walking of individuals with stroke, both within and across sessions.Hence, the current model of the insole system can be used in a clinical setting to quantitatively measure gait in such individuals.

TABLE 1
Characteristics of the study participants (n = 17).