Cognitive Strategy Training in Childhood-Onset Movement Disorders: Replication Across Therapists

Objective: To explore preliminary effectiveness of the Cognitive Orientation to daily Occupational Performance (CO-OP) Approach in improving outcomes in childhood-onset hyperkinetic movement disorders (HMDs) including dyskinetic cerebral palsy following deep brain stimulation (DBS) across UK clinical occupational therapists. Methods: Randomized, multiple-baseline, Single Case Experimental Design N-of-1 trial with replications across participants. Five self-selected goals were identified: three goals were worked on during CO-OP and two goals were left untreated and used to assess skills transfer. Participants were between 6 and 21 years and had received DBS surgery with baseline Manual Ability Classification System (MACS) levels I–IV. Participants were randomized to typical or extended baseline (2 vs. 6 weeks), followed by 10 weekly individual CO-OP sessions. The primary outcome was functional performance measured by the Performance Quality Rating Scale-Individualized (PQRS-I), assessed before, during, and following treatment. Outcome assessors were blinded to baseline allocation, session number, and assessment time. A non-overlapping index, Tau-U, was used to measure effect size. Results: Of the 12 participants recruited, 10 commenced and completed treatment. In total, 63% of trained goals improved with effect sizes 0.66–1.00 (“moderate” to “large” effect), seen for all children in at least one goal. Skills transfer was found in 37% of the untrained goals in six participants. Conclusions: Cognitive strategy use improved participant-selected functional goals in childhood-onset HMD, more than just practice during baseline. Preliminary effectiveness is shown when the intervention is delivered in clinical practice by different therapists in routine clinical settings.


INTRODUCTION
In children and young people with a hyperkinetic movement disorder (HMD) including dyskinetic cerebral palsy (CP) surgical treatments such as deep brain stimulation (DBS) can be effective in reducing severity of motor impairments (e.g., dystonia) (1,2). However, children can be left with persistent functional problems in outperforming everyday tasks that are important to them and their families (3).
A recent systematic review of interventions for CP indicated strong evidence that surgical interventions, such as intrathecal baclofen infusion pump or selective dorsal rhizotomy, and medical interventions, such as botulinum toxin injections, reduce tone in children with spastic CP (4). However, it is the combination of these interventions with adjunct rehabilitation such as strength training or occupational therapy that yield effective results in improving motor and/or functional outcomes. However, there is scant evidence to guide rehabilitation practice following interventions such as DBS in children and young people (4) and only a few small studies in adult-onset dystonia (5,6). Even in specialist centers, number of cases are relatively small, and the heterogeneity of these disorders in terms of etiology, motor severity, and non-motor factors make the planning and implementation of large scale randomized controlled trials (RCTs) challenging.
Nevertheless, rehabilitation approaches are available but, to date, lack robust evidence for their feasibility and efficacy. We have recently reported a proof-of-concept efficacy study (7) of the Cognitive Orientation to daily Occupational Performance (CO-OP) (8) used with children and young people with HMD and DBS in place. CO-OP is an individualized, client-centered approach that uses personalized strategies to achieve client-chosen functional goals. The results provided preliminary evidence to support the feasibility, acceptability, and potential efficacy, evaluated using single-case experimental design replications (7). In that study, treatment was delivered by a single experienced occupational therapist in a specialist pediatric movement disorder service. It is important to explore if these findings can be replicated in routine settings and implemented by local therapists.
Therapist effects, including training and experience, can be a fundamental variable in studies investigating any rehabilitation intervention effectiveness (9)(10)(11). Single-case experimental design methodology provides the opportunity, with limited numbers, to simultaneously investigate treatment efficacy and therapist effects (12,13).
This study aimed to explore whether the results from a previous single-case experimental design proof-of-concept (efficacy) study (7,14) could be replicated by other therapists (adherence to protocol) based in non-specialized services (effectiveness) (15). Further, the study sought to assess the impact of practice by using an extended baseline with repeated assessment of participant-selected goals prior to commencing treatment.

METHODS
The study evaluated the CO-OP approach, as an adjunct to DBS, delivered by non-specialist occupational therapists with a range of clinical experience, to children and young people with HMD. The full trial protocol is available (15) and was implemented with minor changes. Single-case experimental design methodology followed the Single Case Reporting Guideline in Behavioral Interventions (SCRIBE) (12).

Research Question
Does the CO-OP approach improve outcomes for children and young people with HMD on participant-selected goals when undertaken in a non-specialized clinical setting?

Design
This study used a randomized, multiple-baseline, N-of-1 design with replications across six therapists (N-of-1 with five replications) in 12 children and young people (N-of-1 trial with 11 replications) (NB: number achieved was N-of-1 with 9 replications across participants).
A consecutive series of multiple baseline N-of-1 trials was completed using concealed randomization to allocate length of baseline (2 vs. 6 weeks). The extended baseline permitted an examination of whether repeated baselining (practicing the goals without CO-OP input) and DBS only has an impact on skill improvement.

Sample Size
In an N-of-1 design with replications across participants, sample (series) size is not based on the power to test group effects using inferential statistics. Instead, each separate trial examines change over time within an individual, thus allowing us to determine whether treatment is effective for each individual using predefined quantitative and qualitative criteria. In N-of-1 studies, the number of replications (participants recruited) chosen after the first case is often based on pragmatic grounds related to the known heterogeneity of the clinical sample. The number of measurements within each time period is also based on pragmatic factors such as length of treatment and the likely variability in outcome. It is recommended that an initial N-of-1 trial plus three replications are necessary as a minimum to explore efficacy of an intervention but five replications is better (16). Similarly, it is recommended that at least three assessment points are measured as a baseline before the intervention is introduced (16) but preferably five (17). A sample size of 10 was judged to be sufficient to provide information on whether the intervention produced meaningful clinical change for individuals, to examine the direction of "average treatment effect" across individuals, and to provide estimates of the withinand between-subject variability of outcomes in this population. This sample size will also allow us to obtain information on ease of recruitment and adherence to protocol that might inform any future clinical trial. As standard in N-of-1 trial, this number of participants will allow for statistical analysis of effect size of the intervention relative to baseline.

Participants
The study included children and young people with HMD who had previously undergone DBS (or where surgery was scheduled) at the complex motor disorders service (CMDS) database at Evelina London Children's Hospital, UK, that met the inclusion criteria from information in their medical records (n = 27). The 10 patients that had participated in the previous study were not eligible. Full inclusion criteria for the study has been reported elsewhere (7): (a) diagnosis of pediatric HMD other than neurodegenerative conditions; (b) sufficient receptive and expressive communication ability to follow simple instructions and engagement with treatment; (c) age 6 to 21 years; (d) Manual Abilities Classification System (MACS) levels I-IV; (e) emerging skills in self-care; (f) ability to mobilize independently; (g) cognitive ability of 6 years of age or IQ above 70; and (h) DBS electrodes in situ and without signs of infection.
Twelve participants were recruited and admitted to the study sequentially. Further information about eligibility criteria and recruitment procedure are outlined in the study protocol (14,15). Figure 1 describes the recruitment process and randomization to two arms and reasons for exclusion. Details of the participants are provided in Table 1.

Therapists
The Evelina Children's Hospital is a tertiary treatment center providing advanced interventions for children with complex motor disorders. Children are referred from across the UK. For this trial, 19 therapists were initially contacted in centers close to the homes of the 27 potential participants, and 3 additional therapists were approached via special interest group networks. Seven agreed to take part in the study prior to enrolment of the first participant, with six eventually matched to one or more patient participants. Details of the therapists are provided in Supplementary Information 1.

Intervention Description
CO-OP intervention comprised 10 treatment sessions of up to 1 h each, delivered weekly at the participant's home. The frequency of intervention sessions was negotiated between the treating therapist and the participant and family. All therapists had attended CO-OP training workshops (2-3 days) led by CO-OP Academy certified instructors. Ongoing clinical supervision was provided by investigator HG. For more information, please see study protocols (14,15).

Primary Outcome
Performance change in self (participant)-selected goals was measured using the Performance Quality Rating Scaleindividualized (PQRS-i) (7). This is a 10-point observational scale, with 1 representing "cannot do the task at all" and 10 representing "does task very well." The PQRS-i does not penalize the child or young person for how the task is performed or whether dystonia is present or not. PQRS-i is a scale based on observation of the behavior (behavior defined as the task at hand). The child is asked to perform the chosen goals, in their natural environment (i.e., their home), using utensils and materials familiar to them. Performance was videoed by the primary investigator. Performance scores were obtained by rating video recordings of all baselines and post-intervention sessions and a sample of eighteen 5-min randomly selected video segments of the intervention sessions. The videos were randomly presented in a non-chronological order and rated by a trained independent, blinded PQRS-i rater. For more information about how the performance clips for intervention were selected, please refer to the study protocol (14).
Assessments of outcomes were completed several times at each study phase (baseline, during treatment, and end of treatment). Five task-based goals were self-identified by each participant with the aid of the Canadian Occupational Performance Measure (COPM) (18) (data not reported here). Three of the goals were addressed in the CO-OP intervention (trained goals) and two, which were not addressed in the sessions with the therapist, were used to assess generalization and transfer (untrained goals). The participants chose which goals they wished to work on in therapy.

Analysis
Outcomes and significance of change were analyzed and evaluated using a set of complementary approaches following SCRIBE guidelines. Improvement is also summarized for the group in terms of number of goals improved.

Visual Analysis
Changes in means, levels, trends, variability, latency, and consistency were evaluated using visual graph analysis (19)(20)(21). PQRS-i data were graphed with separate data points in each phase (baseline, intervention, and post-intervention).

Quantitative Statistical Analysis of Performance Change Between Baseline and Post-intervention
Serial dependency at baseline was firstly calculated using autocorrelation (AC) so that the most appropriate analysis approach could be chosen, which were (i) differences in individual means and 95% confidence interval for overall change pre-and post-ttest; (ii) regression, initially fitting a naïve linear regression model using ordinary least squares (OLS) for reference only (22,23); and (iii) effect size calculated using non-overlapping index of the Kendall's Tau for non-overlap with baseline trend control (Tau-U) (24) taking into consideration baseline AC in the calculation. Effect size is considered "large" ("very effective intervention") when Tau-U is ≥0.93, "moderate" ("effective intervention") for values 0.66 to 0.92, and "weak" when values are ≤0.65.

Clinically Significant Change Between Baseline and Post-intervention
A change of at least 2 points on the PQRS-i was used to indicate clinical significance in order to match other similar scales such as the COPM (18). Differences between mean scores from pre-and post-were calculated for each individual for each goal.
Quantitative statistical analysis of performance change between CO-OP and treatment as usual (DBS and practice of the goals): Changes in performance across the extended baseline (6 weeks with at least 18 data points) was specifically evaluated. Differences between individual goal means and 95% confidence interval for change of first session and sixth session of baseline (ttest) were calculated for those participants randomized to 6 weeks baseline length.

Analysis of Results in Relation to Therapist-Related Fidelity to Treatment
Fidelity to treatment was evaluated by reviewing randomly selected video-recorded treatment sessions by a CO-OP expert external to the study team using the CO-OP Academy fidelity checklist (25). The randomly selected treatment session per therapist was evaluated fully by the external expert. Therapist factors (training and years of experience) and the fidelity to treatment were explored for their possible impact on clinical outcome.

Adherence to CO-OP Protocol
Fidelity to treatment was evaluated by reviewing randomly selected video-recorded treatment sessions by a CO-OP expert external to the study team using the CO-OP Academy fidelity  Table 1 shows demographic characteristics for all cases recruited to this study with cases organized firstly by those allocated to 2 weeks baseline followed by the participants allocated for 6 weeks baseline. Child number reflects the order of recruitment. Diagnosis, etiology, and phenotype are provided as well as DBS duration, DBS, and MACS levels.
checklist (25). The randomly selected treatment session per therapist was evaluated fully by the external expert.

RESULTS
The participants (eight female, four male) ranged in age from 8 years and 6 months to 19 years and 10 months, with a range of primary diagnoses, etiologies, and clinical phenotypes ( Table 1). Duration since DBS ranged from 6 months to 4 years 6 months. Of the 12 participants, 1 was excluded before functional assessments and intervention commenced as scheduled surgery was canceled, and one withdrew following baseline assessment.
Of the remainder, all completed the planned 10 CO-OP sessions. For completeness, details of all 12 participants are provided in the results tables. A summary of statistical and clinical significance is represented in Table 2 and more detail is shown in Table 3 with statistical parameters and results.

Visual Analysis
Results of participant 1 are presented graphically with trained and untrained goals in Figure 2. The graph shows changes following intervention in all trained goals though mild in one of the three and significant in the other two goals. Figure 2 also shows transfer to untrained goals in one of the two goals chosen to measure this construct. The graphs for the remaining replication cases (including the case that did not proceed to intervention) are provided in Figures 3-12.

Quantitative Analysis of Performance Changes
Statistical differences between means in baseline and postintervention supported the visual inspections and indicated that all participants achieved a significant improvement on at least one trained goal, with 63% (19/30) of goals improving across participants at post-test (Tables 2, 3). Two participants improved on all of their trained goals, five improved on two goals, and three improved on a single goal. For the untrained goals, 7 of the 10 participants receiving CO-OP improved on at least one of the untrained goals, with 37% (7/19) of goals improving overall. Two participants showed deterioration in one of the two selected untrained goals. Baseline trend (i.e., auto-correlation) was found in six goals and therefore the use of Tau-U correction for baseline trend was used. Using this index, all participants achieved improvement in at least one goal with moderate or large effect size. A "large" effect (≥0.93) ("very effective intervention") was obtained in 80% of children post-intervention. In total, the effect size was "large" for 37% of trained goals overall post-treatment. For other trained goals, "moderate" effects (0.66 to 0.92) ("effective intervention") were obtained in 70% of children and for 27% of trained goals overall at post-intervention. "Weak" effects (≤0.65) were seen in 11 goals (37%) overall across eight children at postintervention. No negative effects (deterioration) were observed for trained goals. For untrained goals, "large" or "moderate" effect was seen in 37% of goals (7/19) overall in six children post-treatment, and "weak" effects in 53%. Deterioration with "moderate" negative effects were seen in two goals for two different children. Supplementary Information 2 summarizes results for effect size using non-overlapping index, Tau-U.

Clinically Significant Change
All participants showed a positive change of at least two PQRSi points (based on the difference between the phase means) on at least one goal, at the end of treatment (Tables 2, 3). Post-intervention, 6 out of the 10 participants showed clinically significant improvement on two of their trained goals and two children improved on all trained goals. For untrained goals, six children showed significant transfer on at least one goal at post-intervention.

Quantitative Statistical Analysis of Performance Change Between DBS+CO-OP and DBS+Extended Baseline/Practice of the Goals)
Differences between means in extended baseline using t-test supported the visual inspections and indicated that DBS and practice alone did not offer improvement in the majority of goals (see Supplementary Information 3). There was, however, significant improvement in 4 out of the 25 goals for four out of five participants during extended baseline/practice. Significant deterioration was also noted for two participants for a total of three goals during extended baseline/practice.

Analysis of Results in Relation to Therapist-Related Fidelity to Treatment
The six treating therapists varied in years of experience from recently qualified to 20 years (see Supplementary Information 1). Two out of the six therapists achieved <50% on fidelity checklist with the lowest scores on two of the key elements of the CO-OP approach, Guided Discovery and collaborative Dynamic Performance Analysis (DPA). The two participants treated by the therapists with the lowest fidelity score achieved improvement in goals, but the majority of overall change was measured during the baseline phase with limited improvement over the course of therapy.

DATA AVAILABILITY
Access to de-identified participant data may be requested by contacting the first author.

DISCUSSION
This is the second of two studies exploring the use of the CO-OP approach with childhood-onset HMD post-DBS. The first was set up as a proof of concept and preliminary efficacy (i.e., can the intervention be implemented?). The present study was set up as preliminary evaluation of effectiveness (i.e., can it be delivered in every day practice?) and reports the results in performance improvement when the intervention is delivered by local occupational therapists not specialized in HMD and with only basic training in the CO-OP approach based in centers across the UK. Together, the two studies provide evidence that CO-OP is a feasible and acceptable intervention for children and young people with HMD following DBS, with the potential to produce clinically meaningful improvement. The results obtained in the present study are promising on two grounds: Firstly, although treatment fidelity was variable, CO-OP can be delivered by occupational therapists independent of their years of training and with a relative low-intensity training course, which makes this approach feasible in the context of clinical practice for further formal evaluation. Secondly, replicability has been demonstrated across multiple therapists, in a substantial number of children and young people with heterogeneous presentations of HMD and DBS duration. The total number of successful replications (n = 10) reported here and in combination with those reported previously (N-of-1 plus 8 replications) exceeds the three to five replications recommended in singlecase experimental design, increasing confidence in the results. In this study, all eligible participants in the CMDS database were approached and all who consented were recruited to the study, reducing the risk of bias selection.
As in the first N-of-1 series with eight replications (7), skill improvement following CO-OP was seen in all children and young people, independently of their baseline characteristics and with a broad range of goals as outlined in Table 2. The majority (19/30) of goals addressed in therapy improved during the CO-OP intervention (63% goals improved compared to 75% of goals in the first study). This indicates slightly lower goal improvement rates than in the previous series completed by a therapist experienced in movement disorders and CO-OP. This is in line with reported literature indicating that effectiveness studies closer to the "real-world" setting often show lower improvement FIGURE 2 | Participant 1, PQRS-i scores (y-axis) for Gl-5 for each trial phase (x-axis). For participant 1, the graph indicates a significant improvement in goals 2-4 with stable baseline on goals 2 and 3 (drinking from an open cup and pouring water) and change in slope observed during the treatment session. Even though improved, variability of performance was observed at the post-intervention phase. The visual data provides evidence on means, levels, trends, variability, latency, and consistency across the different phases of the study for each participant and for each individual goal. Also shown is the OLS regression line and 95% confidence range.         rates than in efficacy studies taking place in specialist centers (26). However, the results indicate that useful results are still possible with a relative brief therapy intervention (<10 h per participant) and low training requirements, as well as for therapists with no prior experience in HMD.
Results suggest that meaningful functional improvement is possible in children with a range of etiologies and clinical presentations in childhood-onset HMD. Half of the group achieved at least two out of the three goals worked in therapy. The participants achieving only one goal included one participant with childhood stroke, a participant with CP, and one participant with a metabolic disorder.
While robust positive change was observed on at least one trained goal by all participants, some goals showed no change. Of those children who improved on only one goal, the unimproved goals worked on were doing buttons and putting shoes on for the child with stroke (participant 4), toothbrushing and drinking for child with metabolic disorder (participant 9), eating with knife and fork, and applying mascara for child with CP (participant 12). Possible explanations for the lack of improvement could relate to the difficulty of the selected goal in relation the participants' motor impairment, the fidelity of treatment, or the expertise of the therapist. Although challenging, similar goals successfully improved in other participants with the same characteristics, suggesting that the goal itself may not be the limiting factor or the clinical characteristics. Therapist effects may have been important in some of the cases that showed limited clinical gains. Fidelity to treatment was rated <50% for case 4 (38%) and case 9 (49%) who both improved on only a single goal. Therefore, wider improvement might have been hindered by the diluted delivered version of the CO-OP intervention. It may well be that outcome relates to a combination of the difficulty in the chosen goal itself and the therapists' difficulty to apply the CO-OP approach to these more challenging goals. Future trials may require greater investment in training and supervision.
Although improvement on trained goals is an important outcome, the real potential of this treatment lies in its generalizability and transferability, enabling improving performance beyond the treatment sessions, thereby broadening the reach of the CO-OP approach. In the present study skill transfer was observed in seven of the untrained goals, across five participants. This is similar to the previous single-therapist HMD single-case experimental design (7) and in adults with stroke (27). Even if evidence for the generalizability of transfer is somewhat limited, the potential is demonstrated and warrants further work to enhance this crucial outcome.
Finally, the present study permitted an assessment of the impact of repeated practice on the goals with DBS in situ prior to the systematic application of the CO-OP approach. Improvement during extended baseline (DBS and practice) ( Supplementary Information 3) was noted for 4 of 25 goals in four out of the five children randomly allocated to extended baseline. In such cases, it was hard to disentangle the effects of practice and neuromodulation with DBS. However, given that only some of the goals improved in the extended period (n = 4, 16%), it is more likely due to practice effects than DBS. In the remainder, who showed stable baseline performance, improvement could be more confidently attributed to the CO-OP approach.
As with any study, there are limitations that warrant mentioning. All of the participants in this study had DBS in situ, raising the possibility that DBS-related factors may moderate the outcome of CO-OP. The careful design of both, this and the former proof-of-concept study (7), allowed for manipulation of DBS length of neuromodulation as a variable. Those with neuromodulation in place for longer than 1 year, when most of the change has taken place, particularly in inherited genetic dystonias, showed improvement when the CO-OP intervention started and not necessarily on the baseline period. Secondly, the extended baseline used in this study allowed for close monitoring of change happening within a 6 week period with no therapy intervention, showing much less change on the goals set by the participants. Finally, and most importantly, given that neuromodulation is a global management approach, improvement would be expected to be seen across any goals (trained or untrained) the young person performs. This was not the case in either of the studies performed in the short study periods within these studies, indicating that transfer and indeed goal acquisition are most likely due to the effect of the CO-OP approach. However, goal attainments after DBS in genetic and acquired childhood onset dystonia disorders have been reported without CO-OP at 1-2 years post-DBS implant (28). Without DBS intervention, dystonia worsens (29) along with fixed deformities (30) irrespective of the cause and conventional surgical and medical approaches failed to address the needs of children with HMD (31). DBS neuromodulation is a global management approach to reducing dystonia, chorea, myoclonus, and tremor, often in children with little or no pre-existing motor repertoire in whom dystonia reduction does not equate with spontaneous acquisition of skills after DBS intervention, even with conventional practice and repetition of desired motor skill. For the networked efficacy of DBS, motor and sensory pathways must be intact (32). In addition, cognitive function appears preserved or even enhanced following DBS in isolatedgenetic (33), and in addition to cognitive stability, perceptual reasoning may be increased after DBS in acquired (34) dystonias, respectively, supporting the place of cognitive strategies to boost goal acquisition and transfer of skills to untrained goals with CO-OP approach after DBS.
It is therefore postulated that DBS accelerates or facilitates the efficacy of CO-OP by reducing dystonia and also by modifying the underlying cerebral plasticity. Since, as a guide, it may take up to 2 years for the full benefits of DBS to manifest in isolated genetic dystonias and longer in acquired dystonias, applying methods that enhance the overall speed of goal attainment through a cognitive problem-solving approach is clearly urgently required in childhood when the windows of plasticity are limited (35).
In conclusion, the presented results are promising for a number of reasons. This trial is the first attempt to systematically evaluate the potential effectiveness of a rehabilitation intervention for children and young people with HMD across therapists. Although further testing of efficacy and effectiveness through large-scale trials is required, the present study shows that CO-OP is a feasible and acceptable approach to rehabilitation following DBS for children and young people with HMD and that changes are overall significant for client-chosen goals, including in children with dyskinetic CP, for whom DBS or any other current management modality does not currently provide enough functional changes.
As described, fidelity to treatment was variable and sometimes sub-optimal, indicating the importance of training and supervision in future trials. The limited transfer achieved to untrained goals also warrants further investigation in relation to delivery of the intervention, dosage (timing, frequency, and duration), and any modifications that might be required to achieve transfer to goals not addressed in therapy for this population. Finally, the promising results from using CO-OP with HMD and DBS warrant further investigation of the CO-OP approach in dystonia, which offers a treatment option for a wider number of potential patients. Further, the results from this study and the previous study indicate that practice alone does not provide improvement in self-selected goals. This suggest that simple goal-oriented approaches recommended by experts in dystonia might not be sufficient without the more cognitive problem-solving element inherent in CO-OP. Future research comparing CO-OP with other goal-oriented approaches would be valuable.

DATA AVAILABILITY STATEMENT
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

ETHICS STATEMENT
The studies involving human participants were reviewed and approved by Oxford A Research Ethics Committee, 14/SC/1159. Written informed consent to participate in this study was provided by the participants' legal guardian/next of kin.

AUTHOR CONTRIBUTIONS
HG completed all statistical analysis. This was supervised by VC, senior trial statistician. HG: research study, conception, organization, execution, manuscript preparation, and writing of the first draft. HP, J-PL, VC, and RB: research study, conception, manuscript preparation, and review and critique. All authors contributed to the article and approved the submitted version.