External Validation of the WORSEN Score for Prediction the Deterioration of Acute Ischemic Stroke in a Chinese Population

Background: Early neurological deterioration (END) has been recognized as a serious neurological complication after acute ischemic stroke. However, to date, the WORSEN score was the only one scoring system specifically developed to detect END events in acute ischemic stroke patients. The purpose of this study was to investigate the WORSEN score's utility in China, and to determine the potential predictors of END in acute stroke patients. Methods: Consecutive patients with acute ischemic stroke admitted to the Department of Neurology, Aerospace Center Hospital between March 2015 to February 2017 were recruited into the study's cohort and divided into two groups: patients with and without END. END was defined as either an increase in two or more NIHSS points, an increment of at least one point in motor power or a description of fluctuating of clinical symptoms in medical reports during the first 7 days after admission. Severe END was defined as an increase of NIHSS ≥ 4 points from baseline during the first 7 days after admission. Results: Three hundred fifty four patients with acute ischemic stroke were enrolled in the present study and 67.5% were male. END occurred in 90 of these patients and severe END occurred in 55 of these patients. Logistic regression analysis showed that an initial NIHSS score ≥8, diameter of infarction, striatocapsular infarction, and TOAST type of large arterial atherosclerosis were independent predictors for END. The area under the ROC curve (AUC) of the WORSEN score for the prediction of END was 0.80 (95%CI 0.75–0.84), with a sensitivity of 62.22%, a specificity of 88.26%, positive predictive values of 64.37% and negative predictive values of 87.27%. Meanwhile, the AUC of the WORSEN score for the prediction of severe END was 0.82 (95%CI 0.78–0.86), with a sensitivity of 70.91%, specificity of 83.95%, positive predictive values of 44.83% and negative predictive values of 94.01%. Conclusion: END is a relatively common neurological complication in patients with acute ischemic stroke. Our findings showed that the WORSEN score had a good predictive value for identifying patients with END in a Chinese population. Moving forward, multi-center studies are required for further validations


INTRODUCTION
Stroke is the second leading cause of death worldwide and is the cause of the highest disability-adjusted life-years lost in China (1). Early neurological deterioration (END), a relatively common neurological complication of stroke, occurs in 5-40% of acute ischemic stroke patients (2). Furthermore, it has been found that END is associated with poor outcomes (3,4). Therefore, early detection of patients who are at risk of developing END is an important issue in clinical practice.
Many clinical and neuroimaging predictors have been associated with END. Simonsen et al. found that high blood glucose levels, the presence of large vessel disease, and a large perfusion lesion were associated with END (5). Yi et al. showed that patients with acute minor ischemic stroke with concomitant aspirin and clopidogrel resistance are at an increased risk for END (6) and Seners et al. reported that susceptibility vessel sign extension was an independent predictor of unexplained END in thrombolysed stroke patients (7).
However, when it comes to using scoring systems for assessing END in stroke patients, the WORSEN score is the only one that has been developed ( Table 1). The WORSEN score (8), while not widely used, is based on definite clinical and imaging characteristics of stroke patients and is found to be a potentially valuable tool for detecting END in acute ischemic stroke patients. The purpose of this study was to investigate WORSEN scores' utility in our stroke population, and to determine the underlying predictor of END in acute stroke patients.

Subjects
This was a retrospective, observational study. Consecutive acute ischemic stroke patients admitted to the Department of Neurology, Aerospace Center Hospital between March 2015 and February 2017 were assessed. Patients were eligible for inclusion if they were 18 years or older, had a diagnosis of acute ischemic stroke defined in accordance with the World Health Organization criteria (9), and had symptom onset within 48 h. The diagnosis was verified by brain magnetic resonance imaging (MRI). Exclusion criteria of this present study were

END and Severe END Definition
In order to draw attention to and encourage action to be taken in clinical practices, END was defined as an increase in two or more NIHSS points, an increment of at least one point in motor power, or description of fluctuating of clinical symptoms in medical reports during the first 7 days after admission (12)(13)(14). Severe END (15) was defined as an increase of NIHSS ≥ 4 points from baseline during the first 7 days after admission. Patients were classified into 2 groups according to the presence or absence of END.

Statistical Analysis
Continuous variables were presented as the means (standard deviation, SD) or medians (interquartile range, IQR). Comparisons between the two groups were performed using Student's t test or the Mann-Whitney U test after testing for normality. The χ 2 test or Fisher's exact test was conducted to analyze categorical variables. Logistic regression analysis was performed to evaluate variables' influence on END, adjusting for baseline variables when a p < 0.1 was found in the univariate analysis. We calculated the sensitivity and specificity of different levels of WORSEN scores for prediction of END (severe END) by using receiver operating characteristic (ROC) curves. All testing was two-tailed, and p < 0.05 were considered to be statistically significant. All data were analyzed using SPSS (Version 22.0, Windows, SPSS Inc., Chicago, IL).

RESULTS
During the study period, 1,021 consecutive patients with acute ischemic stroke were potentially eligible for our study. After excluding patients that fit the exclusion criteria, 354 consecutive patients were finally included in the present study (Figure 1). The mean age was 64.2 ± 12.8 and 239 patients (67.5%) were male. END occurred in 90 of these patients (25.4%) and severe END occurred in 55 (15.5%). We identified the reasons for severe END in the 55 patients: four patients were due to hemorrhage transformation, seven patients due to malignant edema, ten patients due to recurrent stroke, seven patients due to medical complications, and 27 patients without a clear mechanism. We reported the demographics, baseline clinical characteristics and outcomes in patients with and without END ( Table 2). There were statistically significant differences between the two groups (P < 0.05) in coronary disease, stroke etiology, percentage with an initial NIHSS score≥8, white blood count (WBC), neutrophil percentage, uric acid levels, striatocapsular infarction, diameter of infarct, and WORSEN score. However, there were no significant differences in other characteristics between the two groups. As expected, patients with END had worse modified Rankin Scale(mRS) scores at discharge than patients without. Logistic regression analysis revealed that an initial NIHSS score ≥8 (odds ratio 2.64; 95%CI 1.13-6.19; P = 0.026), diameter of infarction (odds ratio 1.03; 95%CI 1.01-1.05; P = 0.011), striatocapsular infarction (odds ratio 2.18; 95%CI 1.13-4.19; P = 0.020), and TOAST type of large arterial atherosclerosis (odds ratio 2.87; 95%CI 1.13-7.30; P = 0.027) were independently associated with END ( Table 3).
The area under the ROC curve (AUC) of the WORSEN score for the prediction of END was 0.80 (95%CI 0.75-0.84). Using Youden's index, the optimal cut point for a WORSEN score was 2, with a sensitivity of 62.22%, specificity of 88.26%, positive predictive values (PPV) of 64.37% and negative predictive values (NPV) of 87.27% (Figure 2). Meanwhile, the AUC of the WORSEN score for the prediction of severe END was 0.82 (95%CI 0.78-0.86); the optimal cut point for WORSEN score remained at 2 points, with a sensitivity of 70.91%, specificity of 83.95%, positive predictive values of 44.83% and negative predictive values of 94.01% (Figure 3). Table 4 shows the sensitivity, specificity, positive predictive value, negative predictive value and Youden's index for each WORSEN score in predicting severe END.

DISCUSSION
In our study, the incidence rates of END and severe END were 25.4 and 15.5% respectively. We found that an NIHSS score≥8, diameter of infarction, striatocapsular infarction, and TOAST type of large arterial atherosclerosis were independent predictors for END. WORSEN score showed the good predictive value in  identifying patients with END after acute ischemic stroke in a Chinese population. Although END has been well-recognized as a serious problem following acute ischemic stroke, the definition of END has been inconsistent among different studies and is still under constroversy (16). The reported incidence rate of total END in acute ischemic stroke patients without thrombolysis also has much uncertainty; ranging from 13.3 to 36.8% (2). Results from our study were in line with the two prior studies (6, 17), but were slightly higher than others (18,19). This might be due to the fact that we adopted an increment in NIHSS scores≥2 points as the END definition. Siegler et al. had previously shown that the definition of END by an increase in NIHSS score of 2 or more points, as well as 4 or more points, were all associated with a poorer functional outcome  at discharge (16). Our study showed that the mean mRS (3.34 ± 1.38) in the END group was significantly higher than in the non-END group (1.29 ± 0.80), which is consistent with the above findings. In the present study, we chose an increase in NIHSS score of 2 or more points as the definition of END because we wanted to advocate for early recognition and actions. However, some studies (2,20) have pointed out that a small change in NIHSS score might be less meaningful and interrater dependent. Therefore, we also included an increase in NIHSS≥4 points for the severe END definition in our study. With regard to the risk factors associated with END, hyperglycemia on admission (5), initial severity of stroke (19,21,22), mean platelet volume (17), metabolic syndrome (4), dehydration (23), presence of albuminuria (24) and fibrinogen (25), infarct size (19,26) and proximal arterial occlusion (27) have been reported to be statistically significant in predicting END. Our findings that initial NIHSS score and infarct size are independently associated with END were consistent with previous studies, which may be explained by the fact that severe neurological impairment is a strong predictor of symptomatic intracranial hemorrhage (2) and malignant oedema (28). However, we did not find evidence that the blood sugar levels were associated with END, which is also in line with some other studies (17,25,28).
It is extremely important that clinicians are able to predicate the risk of END in patients with acute ischemic stroke in the early stages in order to choose an appropriate treatment method. Although many predictive models have been developed to project short-term or long-term functional outcomes of the disease and the risk of recurrence in patients (29)(30)(31), the WORSEN score (8) was the only one specifically developed for detection of END in acute ischemic stroke patients. Our study aimed to externally validate the WORSEN score in our population, and our findings indicated that it had a good predictive value for severe END with a sensitivity of 70.91%, specificity of 83.95%, and an the AUC of WORSEN score of 0.82 (95%CI 0.78-0.86).Fittingly, our findings were in line with prior studies.
Our study had several limitations. First, it was a single-center retrospective observational study. Patients with insufficient data for analysis were excluded and, despite the fact that included patients were consecutive, selection bias might have occurred and could have had a negative impact on our findings. Prospectively, large sample size and multi-center studies should be performed to confirm our findings.
Another limitation is in the various definitions of END. Despite the fact that we adopted the widely recommended definition of END and used logistic regression to build the predictive model, the possibility for residual confounding remains. Furthermore, although we excluded patients who accepted intravenous thrombolysis and (or) mechanical thrombectomy, we still could not eliminate the influence different treatments have on the outcomes of acute stroke patients. Finally, as the time of END has been recruited could be different for each patient, which might have also introduced bias.
In conclusion, END is a relatively common neurological complication in patients with acute ischemic stroke. Our study demonstrated that the WORSEN score had a good predictive power for identifying patients with END in a Chinese population. Moving forward, multi-center studies are required for further validate our findings.

DATA AVAILABILITY STATEMENT
The datasets for this article are not publicly available as data collection is still in progress. Requests to access the datasets should be directed to YX, yichengxu27@aliyun.com.

ETHICS STATEMENT
The studies involving human participants were reviewed and approved by the study was approved by the Institutional Ethical Committee of Aerospace Center Hospital. The ethics committee waived the requirement of written informed consent for participation.

AUTHOR CONTRIBUTIONS
YX conceived the study, participated in the design, performed statistical analyses and drafted the manuscript. YC participated in the design, collected the data, and drafted the manuscript. RC performed statistical analyses and helped to draft the manuscript.
FZ and PW participated in the statistical analyses. SY participated in the design and helped to revise the manuscript critically for important intellectual content. All authors read and approved the final manuscript.