Derivation and validation of a composite scoring system (SAVED2) for prediction of unfavorable modified Rankin scale score following intracerebral hemorrhage

Objective To develop a composite score for predicting functional outcome post–intracerebral hemorrhage (ICeH) using proxy measures that can be assessed retrospectively. Methods Data from the observational ERICH study were used to derive a composite score (SAVED2) to predict an unfavorable 90-day modified Rankin scale (mRS) score. Independent predictors of unfavorable mRS were identified via multivariable logistic regression and assigned score weights based on effect size. Area under the curve (AUC) was used to measure the score's discriminative ability. External validation was performed in the randomized ATACH-2 trial. Results There were 2,449 patients from ERICH with valid mRS data who survived to hospital discharge. Predictors associated with unfavorable 90-day mRS score and their corresponding point values were: age ≥70 years (odds ratio [OR], 3.8; 1-point); prior stroke (OR, 2.8; 1-point); need for ventilation (OR, 2.7; 1-point); extended hospital stay (OR, 2.7; 1-point); and non-home discharge location (OR, 5.3; 2-points). Incidence of unfavorable 90-day mRS increased with higher SAVED2 scores (P < 0.001); AUC in ERICH was 0.82 (95% CI, 0.80–0.84). External validation in ATACH-2 (n = 904) found an AUC of 0.74 (95% CI, 0.70–0.77). Conclusions Using data collected at hospital discharge, the SAVED2 score predicted unfavorable mRS in patients with ICeH.


Introduction
Functional outcome following stroke is clinically meaningful and of major relevance to patients. Tools, such as the modified Rankin scale (mRS), have been developed for the accurate assessment of post-stroke functional outcome (1)(2)(3). However, functional outcome data are often unavailable or difficult to collect from real-world sources, such as electronic health records and administrative claims databases.
When long-term functional outcome data are not available, surrogate or proxy measures provide alternative methods to assess post-stroke functional outcome. Proxy measures have included discharge destination (4) and home-time calculations (5), both of which strongly correlate with functional outcome measures between 3 and 12 months post-stroke (6). For instance, in a systematic literature review, Costa et al. found that being discharged to a location other than home was associated with an unfavorable mRS score among the 2 studies that assessed this relationship and that increased home-time post-stroke was associated with improved functional outcomes (6). The majority of proxy measure evaluations have been performed in ischemic stroke populations, while proxy measures in hemorrhagic stroke have not been well characterized.
Composite risk models/scores are used to combine various known risk factors and translate them into a more easily interpretable risk assessment of an individual experiencing a particular outcome (7). Here, we sought to develop a composite scoring system to predict post-intracerebral hemorrhage (ICeH) functional outcome using various proxy measures and risk factors that can be assessed retrospectively with ease and accuracy. By developing a score to predict functional outcomes, we hope to support researchers in better characterizing outcomes post-ICeH among large populations where long-term measures of functional status may not be available.

Study population
We analyzed individual patient-level data from 2 distinct, prospective clinical studies funded by the National Institute of Neurological Disorders and Stroke (Table 1).
The Ethnic/Racial Variations of Intracerebral Hemorrhage (ERICH) study was a multicenter, prospective, case-control study designed to recruit 1,000 non-Hispanic White, 1,000 non-Hispanic Black, and 1,000 Hispanic patients with spontaneous ICeH to identify risk factors among different races and ethnicities (8). The ERICH study allowed the inclusion of critically ill patients with ICeH, including those with Glasgow Coma Scale (GCS) score <5, intraventricular bleeding, and infratentorial bleeds (8)(9)(10). ERICH also included patients who had received anticoagulation prior to ICeH, and they comprised 13.9% of the White, 7.2% of the Hispanic, and 4.7% of the Black cohorts (11).
The Antihypertensive Treatment of Acute Cerebral Hemorrhage-2 (ATACH-2) multicenter, randomized, openlabel trial evaluated the efficacy of early, intensive, antihypertensive intravenous nicardipine treatment in 1,000 patients with spontaneous ICeH (12). The study excluded patients with hemorrhage ≥60 mL, GCS score <5 at emergency department arrival, infratentorial bleeding (e.g., pons or cerebellum) and intraventricular extension, international normalized range (INR) >1.5, use of dabigatran or other oral anticoagulants, or pre-morbid disability requiring assistance in ambulation or activities of daily living (ADL). The primary outcome was death or disability (mRS score of 4-6, on a scale ranging from 0 [no symptoms] to 6 [death]) at 3 months after randomization. Details of the study design, patient populations, and methods, including those used to determine ICeH location and volume, have been previously described for both the ERICH (8) and ATACH-2 (12) studies.
Both ERICH (8) and ATACH-2 (12) reported mRS score at time points ranging from 1 month to ≥6 months post-hemorrhagic stroke. All patients in ERICH and ATACH-2 were eligible for inclusion in the current study if they survived to discharge and had a total hospital length of stay (LOS) <90 days. Patients were excluded if discharge destination data, age, mRS score, need for intubation/ventilation, hospital LOS, or prior stroke history were missing or recorded as "unknown." Attrition diagrams for ERICH and ATACH-2 are shown in Figure 1.

Outcomes
Based on proxy measures identified in a prior systematic literature review (6) and available ERICH and ATACH-2 data, potential proxy measures of functional status assessed in this study included discharge destination to home (including home healthcare or relative's/friend's home) vs. non-home locations; extended hospital LOS (defined as hospital LOS ≥8 days); and need for endotracheal intubation and ventilation. Functional outcome status was assessed using mRS at 30, 90, and 180 days after ICeH.

Statistical analysis
The association between proxy measures and unfavorable outcome classification (defined as a mRS score of 3-6) was assessed. The discriminative ability (sensitivity, specificity, positive predictive value [PPV], negative predictive value [NPV], and area under the curve [AUC]) of each proxy measure with unfavorable functional outcome was evaluated.
To derive a composite score to predict 90-day mRS score, we used proxy measures available at hospital discharge (i.e., need for intubation/ventilation, extended LOS, and discharge destination) from the ERICH study. Selected proxy measures were augmented with variables-advanced age (≥70 years) and prior stroke history-as they are included in well-known clinical risk prediction tools for patients with ICeH (i.e., the Intracranial Hemorrhage [ICH] and Functional Outcome in Patients With Primary ICeH [FUNC] scores) (13) and they are also available in realworld datasets.
We used multivariable logistic regression to identify independent predictors of unfavorable mRS score. Based on the effect size, score weights were assigned to significant proxies/covariates to create the prior Stroke history per chart history, Age ≥70 years, need for Ventilation, Extended hospital LOS ≥8 days, Discharge to locations other than home (SAVED) score. A simplified score (SAVED 2 ) was also derived by assigning points to each covariate based on the relative weight of each predictor's beta-coefficient to make interpretation easier for clinicians at the bedside. Discriminative ability of SAVED 2 was assessed using AUC. Internal validation of the SAVED 2 score was performed using data from the ERICH trial and external validation .

Derivation population (ERICH cohort)
Of the 3,000 patients with ICeH in the ERICH study, 2,449 patients were included in ≥1 of the analyses at 90, 180, or 365 days ( Figure 1A). Of the 2,175 patients who survived to discharge and who had valid mRS data at 90 days after ICeH, 2,151 patients had data available at 180 days and 2,075 patients had data available at 365 days.

Derivation of SAVED
To make a tool that clinicians can use at the patient bedside, we next derived a composite risk score to predict unfavorable outcome at 90 days for patients with ICeH who survived to discharge. Utilizing the ERICH population as the training database, we identified characteristics independently associated with unfavorable 90-day mRS score (3)(4)(5)(6) and assigned points to each covariate based on the relative weight of each predictor's beta-coefficient using multivariable logistic regression. Among the 2,175 ERICH patients with available data, predictors independently associated with unfavorable 90-day mRS scores and their corresponding point values included: prior stroke history (odds ratio [OR], 2.8; 1 point), age ≥70 years (OR, 3.8; 1 point), need for ventilation (OR, 2.7; 1 point), extended hospital LOS ≥8 days (OR, 2.7; 1 point), and discharge to a location other than home (OR, 5.3; 2 points) ( Table 4). The use of anticoagulation was also evaluated in the logistic regression model but was not found to be significantly associated with unfavorable mRS score (OR, 1.10; 95% confidence interval [CI], 0.77-1.56), potentially due to low utilization of anticoagulants (<10%) in the ERICH population.

External validation population (ATACHcohort)
Of 1,000 patients with ICeH in ATACH-2, 904 patients were included in this analysis ( Figure 1B). There were 733 patients with mRS data available at 30 days and 882 patients with mRS data available at 90 days.

External validation of SAVED
External validation in 882 eligible patients from ATACH-2 suggested that SAVED 2 maintained good discriminative ability at 90 days (AUC, 0.74; 95% CI, 0.70-0.77; Table 6) within a dataset including patients with different baseline characteristics and bleed severity from the derivation dataset. At shorter (30-day) and longer (180-day) follow-up periods of functional outcome .
/fneur. . assessment, SAVED 2 appeared to maintain its discriminative ability as evidenced by AUCs remaining in the clinically useful range, which is defined as ≥0.75 (Table 6).

Discussion
In this study, we identified proxy measures for assessing functional outcomes post-stroke and developed a novel scoring tool to predict 90-day functional outcomes based on data points that can be captured retrospectively. Our analysis confirmed that selected proxy measures, including discharge destination, extended hospital LOS, and need for mechanical ventilation, previously identified in the acute ischemic stroke literature (4, 14-16) may be useful in predicting functional outcome status in patients following ICeH. Discharge destination was highly sensitive but not as specific, whereas the need for mechanical ventilation had high specificity but lower sensitivity, suggesting that a composite measure including several of these proxy measures could create a stronger overall measure that accounts for the limited sensitivity or specificity of any given measure. We demonstrated that the use of these identified proxy measures as part of a composite risk score (SAVED 2 ) had a good-to-excellent ability to discriminate between patients likely to have an unfavorable (3-6) compared to favorable (0-2) mRS score at 90 days. These findings are important as accurate assessment of post-ICeH functional outcome is essential for both clinical and realworld evidence studies. SAVED 2 has the potential to serve as a tool to approximate functional outcome post-ICeH when standard outcome measures, such as mRS, are unavailable. The score could also be used to look at how factors, such as the hospital level, treatment patterns, and personal characteristics, relate to functional outcomes. In a clinical setting, SAVED 2 could be used to predict longer-term, 90-day outcomes at the time of hospital discharge.
While SAVED 2 appeared to have good discriminative ability at 30 days, there is additional need to confirm whether 1-month mRS score alone can be a dependable and more efficient outcome measure in clinical trials. Additional analysis of ATACH-2 trial data-including evaluation of agreement, weighted kappa, and assessment of utility-weighted mRS at 30-and 90-days after adjustment for potential confounding-could help address this question, as could developing a model to predict 90-day mRS based on 30-day mRS scores and potential covariates included in the SAVED 2 components.
This study is the first, to our knowledge, to report on the development and validation of a tool to assess functional outcomes among patients with ICeH using variables that can be assessed retrospectively. Unlike many studies that use split validation or just internal validation, we derived the SAVED 2 score using a large, diverse multiethnic cohort and validated it in a separate cohort, strengthening its generalizability to other external cohorts and populations. However, it is possible that differences in the ERICH and ATACH-2 study populations, such as the broader inclusion criteria used in ERICH to include critically ill patients, may have contributed to the slight discrepancy in AUCs observed between the 2 studies. Another limitation of our study is that the data were derived from and validated in populations with spontaneous ICeH and may not be applicable to patients with ischemic stroke or non-ICeH. Although we excluded patients with missing data on any of the key variables, data completion was high for these fields and few patients had missing data. Another limitation is that we derived SAVED 2 in the context of patients experiencing spontaneous ICeH in a pre-direct oral anticoagulant (DOAC) era, with fewer than 10% being on oral anticoagulants at the time of the bleed. Also, patients taking anticoagulants were excluded from the ATACH-2 validation cohort. It is unknown how anticoagulant use might impact results, as patients with anticoagulant-related ICeH may experience poorer prognoses compared with patients not on anticoagulants at the time of ICeH (17). Outcomes for patients with ICeH are very poor. Although it has been decreasing, the mortality burden post-ICeH remains high with a US National Inpatient Sample analysis reporting a 24% inpatient mortality rate (18). Anticoagulant use increases the risk of morbidity and mortality (17). Furthermore, patients who survive the ICeH have a high comorbidity burden. For example, in a German study in 61 patients with DOAC-related ICeH, 28 of 43 survivors (65%) had an unfavorable outcome of mRS 3 to 5, indicating moderate-to-severe disability at 3-month follow-up (19).
Functional outcomes are key for supporting ADL among patients, and we hope that this score can be used to better assess these outcomes among large population-based databases where these data were previously unavailable. To improve outcomes among patients, particularly among those taking anticoagulants, further research is needed to assess the relationship between ICeH and long-term functional outcomes. Utilizing proxy measures such as the SAVED 2 composite score may enable real-world studies of the long-term functional status associated with ICeH.

Data availability statement
The original contributions presented in the study are included in the article, further inquiries can be directed to the corresponding author.

Ethics statement
The studies involving human participants were reviewed and approved by Institutional Review Boards/Ethics Committees at the participating sites. The patients/participants provided their informed consent to participate in those studies.

Author contributions
Contributed to the design, conceptualization of the study, and interpretation of data: CC, MCo, BK, BL, MCh, and AC. Statistical analysis: CC. All authors participated in drafting and revising the manuscript for intellectual content and approved the final manuscript.

Funding
This study was funded by Alexion, AstraZeneca Rare Disease. This post-hoc analysis utilized data from ERICH (NCT01202864), which was funded by the National Institutes of Health (R01NS069763-01) and ATACH-2 (NCT01176565), funded by the National Institutes of Health (U01NS062091; U01NS061861; U01NS059041; U01NS056975) and the Intramural Research Fund for Cardiovascular Diseases of the National Cerebral and Cardiovascular Center, Japan (H23-4-3).