Prognostic Accuracy of Early Warning Scores for Clinical Deterioration in Patients With COVID-19

Background: Early Warning Scores (EWS), including the National Early Warning Score 2 (NEWS2) and Modified NEWS (NEWS-C), have been recommended for triage decision in patients with COVID-19. However, the effectiveness of these EWS in COVID-19 has not been fully validated. The study aimed to investigate the predictive value of EWS to detect clinical deterioration in patients with COVID-19. Methods: Between February 7, 2020 and February 17, 2020, patients confirmed with COVID-19 were screened for this study. The outcomes were early deterioration of respiratory function (EDRF) and need for intensive respiratory support (IRS) during the treatment process. The EDRF was defined as changes in the respiratory component of the sequential organ failure assessment (SOFA) score at day 3 (ΔSOFAresp = SOFA resp at day 3–SOFAresp on admission), in which the positive value reflects clinical deterioration. The IRS was defined as the use of high flow nasal cannula oxygen therapy, noninvasive or invasive mechanical ventilation. The performances of EWS including NEWS, NEWS 2, NEWS-C, Modified Early Warning Scores (MEWS), Hamilton Early Warning Scores (HEWS), and quick sepsis-related organ failure assessment (qSOFA) for predicting EDRF and IRS were compared using the area under the receiver operating characteristic curve (AUROC). Results: A total of 116 patients were included in this study. Of them, 27 patients (23.3%) developed EDRF and 24 patients (20.7%) required IRS. Among these EWS, NEWS-C was the most accurate scoring system for predicting EDRF [AUROC 0.79 (95% CI, 0.69–0.89)] and IRS [AUROC 0.89 (95% CI, 0.82–0.96)], while NEWS 2 had the lowest accuracy in predicting EDRF [AUROC 0.59 (95% CI, 0.46–0.720)] and IRS [AUROC 0.69 (95% CI, 0.57–0.81)]. A NEWS-C ≥ 9 had a sensitivity of 59.3% and a specificity of 85.4% for predicting EDRF. For predicting IRS, a NEWS-C ≥ 9 had a sensitivity of 75% and a specificity of 88%. Conclusions: The NEWS-C was the most accurate scoring system among common EWS to identify patients with COVID-19 at risk for EDRF and need for IRS. The NEWS-C could be recommended as an early triage tool for patients with COVID-19.


INTRODUCTION
The outbreak of coronavirus disease 2019 (COVID-19) has recently become a public health emergency of international concern (1). A novel coronavirus, termed as severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), was isolated as the pathogen of COVID-19 (2). As of October 18, 2020, there have been more than 40 million confirmed COVID-19 cases and 1.1 million deaths globally from World Health Organization reports.
With a sharp increase in the number of cases and limited medical resources, healthcare systems worldwide are facing unprecedented challenges (3). Although the majority of patients with COVID-19 have mild symptoms, patients with advanced age and chronic comorbidities such as hypertension tend to have poor outcomes (4,5). Patients infected with SARS-CoV-2 tend to get worse from illness onset with a median duration of 7 days, in which severe type may deteriorate to acute respiratory distress syndrome (ARDS) or multiple organ failure (6,7). During the outbreak of the COVID-19 crisis, early and quick recognition of patients who are at high risk of clinical deterioration would therefore be significantly important (8). A severity-based approach is urgently needed for triaging high risk patients with COVID-19 (9).
The Early Warning Scores (EWS) are a variety of physiologic scoring systems widely used in the world. These systems are based on bedside indices that can be obtained easily and rapidly such as heart rate, respiratory rate, systolic blood pressure, and peripheral oxygen saturation (SpO 2 ), allowing quick and accurate identify patients at high risk of clinical deterioration. Now, various EWS were developed for early recognition of clinical deterioration. The National Early Warning Score (NEWS), the most common EWS, was initially recommended by the Royal College of Physicians (RCP) (10). It has been proved that NEWS was associated with ICU admission and death outside ICU (11,12). Its updated version, the National Early Warning Score 2 (NEWS2), with a new SpO 2 scoring scale, was published by the RCP in 2017 to improve prediction for clinical deterioration in patients with a hypercapnic respiratory failure (13). Other versions of EWS such as the Modified Early Warning Score (MEWS) (14,15) and Hamilton Early Warning Score (HEWS) (16) have also been recently developed to improve the early recognition of hospitalized patients at risk for deterioration, with a significant degree of variation in the clinical variables and the weightings assigned.
Currently, guidelines from the RCP recommends the use of the NEWS2 for initial assessment in patients with COVID-19 (17). Moreover, NEWS-C, a new version of modified NEWS, has also been recommended for triage decisions in patients with COVID-19 (18,19). However, the recommendations were only based on expert opinions and have not been fully validated in COVID-19 patients.
In this study, we aimed to compare the performance of EWS including NEWS, NEWS2, NEWS-C, HEWS, MEWS, and quick sepsis-related organ failure assessment (qSOFA) to predict early deterioration of respiratory function (EDRF) and the need for intensive respiratory support (IRS) in patients with COVID-19.

Study Design and Participants
The study was approved by the Ethics Commission of Renmin Hospital of Wuhan University (WDRY2020-K048) and was conducted in accordance with the amended Declaration of Helsinki. Written informed consent was waived by the Ethics Commission in the setting of COVID-19 crisis in Wuhan.
Patients with age ≥18 years and confirmed COVID-19 admitted between February 7, 2020 and February 17, 2020 were screened in our study. We excluded patients for pregnancy, death within 48 h of admission, or having a Do Not Resuscitate order. COVID-19 was diagnosed by the real-time RT-PCR method on nasal or pharyngeal swab specimens.

Date Collection
Baseline demographics, clinical characteristics on hospital arrival including symptoms, vital signs, and oxygen therapy, laboratory findings, treatments, and outcomes were prospectively collected by two trained reviewers. The NEWS, NEWS-C, NEWS2, HEWS, MEWS, and qSOFA were calculated based on the demographic and clinical characteristics of each patient (Supplementary Table 1).

Outcome Assessment
Respiratory function was assessed according to the respiratory component of the sequential organ failure assessment score (SOFA resp ). The EDRF was defined as a positive change in respiratory function at day 3 ( SOFA resp = SOFA resp at day 3-SOFA resp on admission). The positive value of SOFA resp reflects clinical deterioration. The IRS was defined as the use of high flow nasal cannula oxygen therapy, noninvasive or invasive mechanical ventilation. IRS was considered if the patients met the following criteria: a respiratory rate of ≥30 breaths per minute and arterial oxygen saturation (SaO 2 ) ≤ 93% or a ratio of the partial pressure of arterial oxygen (PaO 2 ) to inspired oxygen (FiO 2 ) of 300 mmHg or less while the patient was receiving oxygen therapy of ≥10 L/min for at least 15 min. The choices   of respiratory support method (high flow nasal cannula oxygen therapy, noninvasive or invasive mechanical ventilation) were at the discretion of the attending clinicians.

Statistical Analysis
Data distribution was evaluated using the Kolmogorov-Smirnov test. Continuous variables were expressed as mean ± standard deviation or median interquartile range as appropriate.
Categorical variables were expressed as frequencies and percentages. Baseline data were compared using the Student's t-test or the Mann-Whitney U test for continuous variables and the chi-square test or Fisher's exact test for categorical variables. Receiver operating characteristic (ROC) curves were constructed to assess the performance of EWS, and the optimal cut-off values were calculated by the Youden index.
The area under the receiver operating characteristic curve (AUROC) were compared by the method described by Hanley and McNeil (20). All statistical analyses were performed using the SPSS software package, version 13.0 (SPSS, Inc., Chicago, IL, USA) and MedCalc software 15.0 (MedCalc Software Ltd, Ostend, Belgium). A two-tailed P value of <0.05 was considered statistically significant.

).
A total of 24 patients (20.7%) needed IRS during the period of hospital stay. Patients with IRS also tended to be older and had a higher rate of hypertension than those without IRS (all P < 0.05). Compared with the patients without IRS, the patients requiring IRS have higher proportions of dyspnea (91.7 vs. 47.8%; P < 0.001) and a higher respiratory rate [32 (24, 25) vs. 22 (20,26); P < 0.001] but a lower baseline PaO 2 /FiO 2 value [196 (150, 260) vs. 305 (274, 329); P < 0.001]. Patients with IRS had a higher NEWS, NEWS-C, NEWS2, HEWS, MEWS, and qSOFA than non-IRS patients (all P < 0.001; Table 1). A total of 20 patients (17.2%) developed both IRS and EDRF in this cohort. The hospital mortality rate was 7.8%. The mortality in patients with EDRF or IRS was higher than those without EDRF or IRS (all P < 0.001).

Performance of EWS for Clinical Deterioration
To assess the utility of EWS to predict EDRF and need for IRS, the ROC curves were constructed and the AUROCs were calculated (Figure 2). Table 2 listed AUROC, optimal cutoff value, sensitivity, specificity, and positive and negative predictive values of EWS.

DISCUSSION
Until now, there are limited studies to evaluate the predictive value of EWS in patients with COVID-19. In the current study, we found that the NEWS-C was the most accurate scoring system among common EWS for predicting EDRF and IRS in patients with COVID-19. On the contrary, NEWS 2 had the lowest accuracy for predicting both outcomes. EWS have been developed and widely used around the world for early recognition of clinical deterioration (21). The NEWS, endorsed by RCP, is already used for predicting deterioration in many hospitals across the United Kingdom (10). It is reported that the NEWS has better performance than other EWS to identify patients at risk of ICU admission and mortality (12). Moreover, the NEWS was more accurate in predicting clinical deterioration than qSOFA in infected patients outside the ICU (24).
The NEWS 2, updated version of NEWS, was recommended by RCP in 2017. The new SpO 2 scoring scale in NEWS 2, with a lower SpO 2 threshold than NEWS, was implemented to avoid over-use of supplemental oxygen and facilitate management in hypercapnic patients (27). Recently, the NEWS 2 has been recommended for predicting clinical deterioration in patients with COVID-19 (17). In our study, NEWS 2 had a lower performance than NEWS in predicting EDRF and IRS. This is in accordance with previous study, which demonstrated that NEWS 2 did not predict clinical outcome in elderly patients with COVID-19 (28). The possible reasons were as follows: (1) the incidence of type II respiratory failure in patients with COVID-19 was low in this study; and (2) the NEWS2 modifications to NEWS may not improve discrimination of poor outcome in hospital patients including those with type II respiratory failure (26). Therefore, NEWS 2 may be inappropriate for triage decision in patients with COVID-19.  A modified NEWS, termed NEWS-C, has also been recommended for triage decision in patients with COVID-19 (18,19). The MEWS (14) and HEWS (16) have been developed to early identify clinical deterioration in generally hospitalized patients, with a significant degree of variation in the clinical variables and the weightings assigned. In this study, the NEWS-C had largest AUROC for predicting EDRF and IRS in these EWS. NEWS-C modifications to EWS added an age ≥65 years as an independent component. Several studies have showed that old aging was independently associated with mortality in patients with COVID-19 (4,23). Therefore, it may offer better predictive performance than other EWS. qSOFA, consisting of three clinical variables (mental status, respiratory rate, and blood pressure), has been proposed as a rapid screening tool for infected patients (29). The effectiveness of the qSOFA has been validated in various heterogeneous sepsis patients (30,31). Recently, several reports have demonstrated that the qSOFA can accurately assess the severity of community-acquired pneumonia (32)(33)(34). However, qSOFA had a lower performance in predicting clinical deterioration compared with other EWS in our study. This may be partially explained by the low percentage of hypotension and alter mental status in this cohort. The finding was also consistent with previous studies, in which qSOFA may not be appropriate to identify critically ill patients with COVID-19 (22,35).
This study had several limitations. First, this was a retrospective study with relatively small sample size. A larger cohort validation is still required. Second, the changes in EWS during the treatment process was not recorded in this study. Third, as the intubation rate and mortality in this population were lower than that in critically ill patients (25,35,36), caution must be taken in extrapolating the results of the study for critically ill patients. Additional assessments of organ dysfunction should be required in critically ill patients with COVID-19.

CONCLUSION
The NEWS-C was the most accurate scoring system among common EWS to identify patients with COVID-19 at risk for EDRF and need for IRS. The NEWS-C could be recommended as an early triage tool for COVID-19.

DATA AVAILABILITY STATEMENT
The original contributions presented in the study are included in the article/Supplementary Materials, further inquiries can be directed to the corresponding author/s.

ETHICS STATEMENT
The studies involving human participants were reviewed and approved by Ethics Commission of Renmin Hospital of Wuhan University. The ethics committee waived the requirement of written informed consent for participation.