Large-scale screening of clinical assessments to distinguish between states in the Integrated HD Progression Model (IHDPM)

Background Understanding the sensitivity and utility of clinical assessments across different HD stages is important for study/trial endpoint selection and clinical assessment development. The Integrated HD Progression Model (IHDPM) characterizes the complex symptom progression of HD and separates the disease into nine ordered disease states. Objective To generate a temporal map of discriminatory clinical measures across the IHDPM states. Methods We applied the IHDPM to all HD individuals in an integrated longitudinal HD dataset derived from four observational studies, obtaining disease state assignment for each study visit. Using large-scale screening, we estimated Cohen’s effect sizes to rank the discriminative power of 2,472 clinical measures for separating observations in disease state pairs. Individual trajectories through IHDPM states were examined. Discriminative analyses were limited to individuals with observations in both states of the pairs compared (N = 3,790). Results Discriminative clinical measures were heterogeneous across the HD life course. UHDRS items were frequently identified as the best state pair discriminators, with UHDRS Motor items – most notably TMS – showing the highest discriminatory power between the early-disease states and early post-transition period states. UHDRS functional items emerged as strong discriminators from the transition period and on. Cognitive assessments showed good discriminative power between all state pairs examined, excepting state 1 vs. 2. Several non-UHDRS assessments were also flagged as excellent state discriminators for specific disease phases (e.g., SF-12). For certain state pairs, single assessment items other than total/summary scores were highlighted as having excellent discriminative power. Conclusion By providing ranked quantitative scores indicating discriminatory ability of thousands of clinical measures between specific pairs of IHDPM states, our results will aid clinical trial designers select the most effective outcome measures tailored to their study cohort. Our observations may also assist in the development of end points targeting specific phases in the disease life course, through providing specific conceptual foci.


Introduction
Disease-modifying therapeutic development for Huntington's disease (HD) depends on the availability of appropriate outcome measures or clinical endpoints for use in clinical trials.Given the evolution of HD symptomatology over decades (Roos, 2010;Ross et al., 2014), the utility of specific outcome measures to discriminate between stages over the disease life-course will vary as a function of the disease stage under study.A temporal map of relevant clinical assessments at the different stages of HD will aid future clinical research and inform trial design.
To date, the endpoints in almost all HD clinical trials have been the various components of the Unified Huntington's Disease Rating Scale (UHDRS ® ′99) (Kieburtz et al., 1996), a clinical scale that consists of multiple assessments from four domains: motor function, cognitive function, behavior, and functional capacity.Commonly used clinical assessment items from the UHDRS ® ′99 include the Total Motor score (TMS), Total Functional Capacity (TFC) score, and the Symbol Digit Modalities Test (SDMT).The composite UHDRS (cUHDRS) score (Schobel et al., 2017) has also been derived from the UHDRS ® ′99 as a potential endpoint to assess disease progression.
We recently reported the development of an Integrated Huntington's Disease Progression Model (IHDPM) that comprehensively characterizes HD symptom progression (Ghosh et al., 2017;Sun et al., 2019;Mohan et al., 2022).Based on this model, HD clinical phenotypes can be segregated into nine clusters of increasing severity, defined by varying combinations of motoric, cognitive, and functional measurements.Each of the nine clusters can be considered as a disease 'state' within the clinical phenotype.This differs from a staging system, which is typically built on disease prognostic milestones that define consecutive periods distinguished qualitatively by occurrence of new events.Using the IHDPM, we aim to systematically evaluate the ability of each of the more than 2,000 individual clinical assessment items that were recorded in the integrated dataset (used to generate the IHDPM) to characterize and distinguish between different IHDPM states and, by extension, generate a temporal map of the clinical measures most likely to change between states spanning the HD life-course.

Study dataset
The creation of the integrated dataset and development of the IHDPM has been previously described in detail (Ghosh et al., 2017;Sun et al., 2019;Mohan et al., 2022).Briefly, data from four observational studies {PREDICT-HD [Neurobiological Predictors of Huntington's Disease (Predict-Hd) (NCT00051324), n.d.], REGISTRY [Registry -an Observational Study of the European Huntington's Disease Network (Ehdn) (NCT01590589), n.d.], TRACK-HD & ON (Tabrizi et al., 2011(Tabrizi et al., , 2013) ) and Enroll-HD [Enroll-Hd: A Prospective Registry Study in a Global Huntington's Disease Cohort (NCT01574053), n.d.]} were curated into an integrated database, including clinical assessments across four symptom domains (motor, function, cognition, behavior), recorded longitudinally.Records of participants who enrolled in multiple studies were linked by their ID code (HDID).These studies include both people with HD (PwHD) and controls, but only PwHD were included in the current analysis.The starting sample for analysis comprised 64,758 observations from a sample of N = 18,941 individuals (Table 1).Discriminative analyses were based on a subsample limited to individuals observed to transition between states (i.e., observations in at least two states), resulting in an analysis set of N = 3,790 unique participants (Table 1).

Characteristic
Descriptive statistics (full) Descriptive statistics (analysis)

Candidate clinical measures
The Integrated HD dataset contained variables including 'static' variables (values do not change over time, e.g., certain demographic variables and genetic information including CAG repeat length), 'interval' variables (which have a defined start and end time, e.g., existence of comorbidities and use of medications) and 'dynamic' variables (values may change at different study visits, e.g., clinical assessments).This analysis focused on the 2,472 dynamic variables -comprising individual and summary items from multiple clinical rating scales -(Supplementary Table S2) which make up the majority of variables in the Integrated HD dataset.Although the machine-learning derived IHDPM did not include behavioral measures (Mohan et al., 2022), in this analysis we also include and assess variables from the Problem Behaviors Assessment (long), as well as the UHDRS Problem Behaviors Assessment (short; PBA-s) which produces summary scores for depression, irritability/aggression, psychosis, apathy and executive function.

IHDPM system and state assignment
The IHDPM selected 44 clinical measures during the model development (Mohan et al., 2022) and, separated HD progression into nine distinct states, spanning ∼36 years (encompassing clinical motor diagnosis) (Figure 1).In case of longitudinal observations, the IHDPM model maps each visit to the most likely disease state (from one to nine), based on data from all observations up to that visit.We applied the IHDPM model to all PwHD in the Integrated HD dataset and obtained disease state assignment for all study visits.Recognizing that individuals do not necessarily transition through IHDPM states sequentially -particularly across the 'transition' states (i.e., 3, 4, 5) during which clinical motor diagnosis is expected to occurwe examined transition pathways out of individual states in those participants with sufficient longitudinal data (i.e., observations in at least two states) (Figure 2).This analysis informed the state pairwise comparisons examined.

Statistical analysis
Pairwise comparisons were conducted between clinical measure scores observed in specific pairs of IHDPM states.In order to evaluate the discriminative power of each of the clinical measures in distinguishing the IHDPM states, we used Cohen's effect size (Cohen's D).For a pair of disease states i and j, the effect size of a clinical assessment f is calculated as Integrated HD Progression Model (IHDPM) state comparisons.The pairwise comparisons examined between states are indicated.All sequential pairs were examined (e.g., S1 vs. S2).State skipping was observed in individual participant trajectories during the transition period (i.e., states 3 to 5), thus non-sequential pairwise comparisons were also examined in this phase (i.e., S3 vs. S5 and S4 vs. S6).The expected state duration time is also indicated.Participant transitions between IHDPM states.Transition pathways out of individual states were examined in those participants with sufficient longitudinal data (i.e., observations in at least two states).Matrix values and associated color-formatting (green scale) represent the frequency of individuals moving directly from the state indicated on the vertical to the state indicated on the horizontal.For most states (i.e., 1, 2, and 5-8), participants predominantly transitioned directly from one state to the immediate sequential state.State skipping was commonly observed from states 3 and 4. Sun et al. 10.3389/fnagi.2024.1320755Frontiers in Aging Neuroscience 04 frontiersin.org where x ¯i + 1 − x ¯i are the sample means of assessment f for all available observations in state i and j respectively, and S pool is the pooled standard deviation of the two groups, and is calculated as where ni and s2 denote the sample size and the sample variance of assessment f for observations assigned to state i.
For each pair of states, the Cohen's D effect size of clinical assessment items were ranked by the absolute values of their effect sizes.In this study, an assessment item f is called to be a discriminative clinical measure between states i and j if the corresponding Cohen's D is greater than 0.8 (large effect size).Among the discriminative assessments with large effect sizes, those that were unique to a specific pair of states were named the signature assessments of the two states.Not all variables were consistently available in all four studies and missing values were common in these observational studies.For each clinical variable f, the Cohen's D was calculated for pairs of states i and j only if there was a sample size of ≥30 in both states i and j.
In addition to examination of all consecutive state pairs (e.g., state 1 vs. 2), we also examined non-consecutive state pairs reflective of the most frequently observed trajectories from each state (e.g., state 3 vs.5), informed by our analysis of participant transitions between IHDPM states (see Figures 1, 2).For most states (i.e., 1, 2, and 5-8), participants predominantly transitioned directly from one state to the immediate sequential state.However, state skipping was observed.The most frequently observed transition from state 3 was to state 5, and from state 4 to state 6.As such, we performed pairwise comparisons of all assessment items between states 3 vs. 5, and 4 vs. 6.We note that other, less common, transitions were observed, for example, state 1 to state 3, and state 4 to state 7 (Figure 2); these state pairs were not analyzed.
Our analysis sample for each IHDPM state pair was limited to individuals with observations in each of the states of interest, where the individual transition trajectory was from state A to state B (i.e., the two states indicated in the pairwise comparison).This limited our discriminative analysis sample to a total of 3,790 unique participants.
To aid comparisons with other studies, we also estimated annualized effect sizes that take into account the differing lengths of time people with HD spend in each state (ranging from 3 to 8 years; see Figure 1; Mohan et al., 2022).Annualized effect sizes were estimated as Effect size of assessment between state i and state j / expected duration of state i (in years).

Results
The Integrated HD dataset provides a diversified sample of 18,941 PwHD for whom IHDPM states have been calculated.Our discriminative analysis set was limited to a subsample of those 3,790 individuals who were observed to transition between states.Table 1 summarizes the key demographic and genetic characteristics for the original and analysis samples, as well as visit counts and observation periods.Supplementary Table S1 summarizes the IHDPM state pairwise comparison results, ordered by their absolute effect size, limited to those assessment items with an absolute effect size of ≥0.8.Signature assessments -unique to a specific pair of states -are highlighted.A complete set of results, including results for all clinical measures by state pair, are provided in Supplementary Table S2.
Between the early-disease states (states 1 vs. 2, 2 vs. 3, 3 vs.4), UHDRS motor assessment measures predominantly had the highest discriminative power, with effect sizes for TMS, diagnostic confidence level (DCL), and individual chorea items ranking highest.One additional and notable source of discriminatory items for the state 1 vs. 2 comparison was the SF-12 assessment; almost all individual dimension scores (e.g., mental health, general health, social functioning) featured as discriminatory items of large effect size, and many of these were unique to this state pair.Several SF-12 dimension scores were also identified for the state 3 vs. 4 comparison.For the cognitive domain, a completion indicator variable for the UHDRS cognitive verbal fluency test (letters) was identified for the state 1 vs. 2 comparison.Stroop Color and Stroop Word Reading tests (total correct) were identified for state 2 vs. 3, alongside independence score and UHDRS Q80 -a holistic diagnostic indicator considering all disease domains.Stroop Color, Stroop Word, and the Symbol Digit Modality Test (all UHDRS cognitive components) were identified for state 3 vs. 4, alongside UHDRS Q80.The Global Clinical Impression score also featured as an item of large effect size, not only for these first three consecutive state pairs but for every state pair examined.
In the IHDPM staging system, states 3 to 5 belong to the so-called transition phase, during which most individuals experience clinical motor diagnosis (DCL = 4) (Sun et al., 2019;Mohan et al., 2022).In light of the transition pathways most commonly observed in this phase -with individuals typically transitioning directly from states 3 to 5, and from 4 to 6 -we focus on these non-sequential pairwise comparisons here.The best discriminators between states 3 and 5 were UHDRS functional measures (e.g., TFC score, independence score, and functional assessment score) alongside UHDRS motor items (e.g., TMS and DCL).Certain individual UHDRS TFC and Function items also ranked highly, including occupation, employment, domestic chores, and finances.Two items from the Caregivers Quality of Life (CQoL) assessment also feature -frustration by misconceptions of HD relatives, and discrimination toward them; these are signature assessment items unique to this state pair.A very long list of discriminators with large effect sizes were observed between states 4 and 6; functional (TFC score) and motor (TMS) measures ranked highest, but cognitive measures were also highlighted (Stroop Color, Stroop Word, SDMT), and many 'signature' measures were also observed, including laundry and automobile operation items in the UHDRS functional domain.Two PBA (long) measures -quality of work (severity; frequency) -also feature uniquely to this state pair.Between states 4 and 5 -a less common but still frequently observed state transition -the best discriminators are almost entirely functional.Total and individual item scores from the UHDRS TFC and Function/ Independence assessments heavily feature, alongside three items from the WPAI-SHP -relating to employment and the impact of HD on regular activities -which are unique to this state pair.Note that DCL is not flagged as a discriminator of large effect for this pairwise contrast, or any subsequent pairs.State 6 marks the beginning of the late-disease phase of the IHDPM staging system (Mohan et al., 2022).From state 5 to state 6, the list of discriminative assessments includes measures principally from the motor domain, with effect sizes in UHDRS TMS and total bradykinesia score ranking highest.From state 6 to state 9, the lists of discriminative measures include those from motor, cognitive, and functional domains, with an increasing emphasis on functional items discriminating between the later states.We identified the supervision of children without help item (UHDRS Function) and self-care (frequency) from the PBA (long) as signature items for state 6 vs. 7, and signature measures discriminating state 8 from state 9 included care level (functional; UHDRS TFC), completion of certain assessments (Trailmaking test, Timed Up and Go), receipt of crutches/ sticks in the last 6 months (CSRI), MMSE total score (MMSE), ocular pursuit scores; vertical and horizontal (UHDRS Motor), and the 'construction' score from the Dementia Rating Scale 2.
A large proportion of measures identified as state pair discriminators were UHDRS items.Table 2 presents the IHDPM state pairwise comparisons for core UHDRS variables across all state pairs examined.

Discussion
We conducted large-scale screening of all available dynamic clinical measures in the Integrated HD dataset and evaluated their ability to distinguish between observations of individuals in different IHDPM states spanning the life course of the disease.As expected, based on the clinical course of HD, the clinical measures that best discriminate between the IHDPM states are somewhat heterogeneous across the HD progression pathway.
A large proportion of measures identified as the best state pair discriminators were UHDRS items.In the early-disease states (states 1 vs. 2, 2 vs. 3, 3 vs. 4) and early post-transition period states (5 vs. 6, 6 vs. 7), UHDRS motor assessments -most notably TMS -showed the highest discriminative power.UHDRS functional items did not become strong discriminators between states until the transition period, specifically emerging in the state 3 to 5 comparison, and demonstrating the greatest discriminatory power for distinguishing between the final states (7 vs. 8, 8 vs. 9).Cognitive assessments (specifically, Stroop Word Reading Test, Stroop Color Naming Test, and Symbol Digit Modality Test) showed good, although typically not as large, discriminative power between almost all state pairs examined, excepting state 1 vs. 2.No PBA-s items were identified as state discriminators of large effect, in line with previous findings that show little evidence of PBA-s measure variation across the disease life course (Marder et al., 2000;Tabrizi et al., 2013).Nevertheless, it is noted that apathy (and, to a lesser extent, executive function) have moderate discriminative power beginning in the transition period, also concordant with previous reports (Tabrizi et al., 2012).
In addition to the UHDRS assessments, several other assessments were also flagged as excellent state discriminators, with variation observed over the life course.For example, the Short Form Health Survey (SF-12; a self-reported general health and well-being questionnaire with several dimensions such as general health, social functioning, and mental health) was identified as a source of several items of excellent discriminatory power between the earliest disease states (state 1 vs. 2).The Caregivers Quality of Life (CareQoL) survey was identified as a source of items for discrimination between observations in states 3 vs. 5.Three items from the Workers Productivity and Activity Impairment (WPAI-SHP) questionnaire, relating to employment and the impact of HD on regular activities, were flagged for the state 4 vs. 5 comparison, all unique to this state pair.Several unique items from other assessments were also flagged as excellent discriminators between the last two states (8 vs. 9), including items from the Mini Mental State Examination (MMSE; a clinician-administered assessment of cognitive impairment), Dementia Rating Scale 2 (DRS-2; an assessment of overall cognitive For certain state pairs, single assessment items other than total or summary scores were highlighted as having excellent discriminative power.For example, for the transitionary state comparisons (i.e., states 3 vs. 5, 4 vs. 5, and 4 vs. 6), occupation, gainful employment (accustomed work; any), finances, and domestic chores -all singular items from the UHDRS TFC and Function assessments -were measures with high discriminative power.Employment -as assessed by the WPAI-SHP -also had high discriminative power in the state 4 vs. 5 comparison, as did employment status as assessed by the CSRI.This is in line with our previous findings (Mohan et al., 2022) and the PREDICT study (Beglinger et al., 2010), which suggested that work life is the first functional ability to be affected by disease progression.Such observations may assist in the development of end points targeting specific phases in the disease life course, through providing specific conceptual foci.
Our analyses have used the IHDPM states to indicate the different stages of the HD life-course.Although the IHDPM covers a relatively larger range of HD progression than traditional HD staging systems, its observational study basis does not cover the extremely early phase of the disease between birth and state 1, nor does it cover the period just prior to death.States 1 and 9 should not be regarded as the beginning and ending of the disease and the identified discriminative and signature clinical measures may not be appropriate for future research or trials targeting disease more than 20 years before or after clinical motor diagnosis.It is also important to highlight the expected durations of the nine IHDPM states are not equal (Sun et al., 2019;Mohan et al., 2022).Future researchers should take expected state durations into consideration when using the insights from this analysis to design new studies.Finally, we acknowledge that a new state-of-the art staging system has been developed since the IHDPM was published -specifically the HD Integrated Staging System (HD-ISS) (Tabrizi et al., 2022).Work is currently underway to map the nine IHDPM states to the four HD-ISS stages (i.e., 0, 1, 2, 3).The results of the current analyses, combined with this new mapping, will highlight clinical assessments/items that could help develop tailored assessments for these HD-ISS stages, which might later be adopted as trial endpoints.
We systematically evaluated the utility of thousands of clinical measures (including single items and summary scores) from dozens of clinical assessments in distinguishing between individuals in different IHDPM states spanning ~40 years of the HD life-course, encompassing early, transitory, and late stages of the disease.We demonstrate the importance and utility of the various UHDRS assessments in distinguishing between individuals in specific neighboring pairs of IHDPM statesincluding a reiteration of the utility of the UHDRS TFC (and other functional measures) as an excellent discriminatory measure -but only from the transitional states and beyond (i.e., states 3 to 5, and subsequent pairs).We also demonstrate the utility of non-UHDRS assessments in distinguishing between IHDPM states, such as the SF-12, which critically demonstrated excellent utility in distinguishing between the earliest phases of the disease examined.These results will aid future clinical research and may inform trial design.

TABLE 2
IHDPM state pairwise comparison results for key UHDRS items.Effect sizes are presented only; complete results are presented in Supplementary TableS2.Large effect sizes (≥0.80) are indicated in bold typeface.