Heart Failure Probability and Early Outcomes of Critically Ill Patients With COVID-19: A Prospective, Multicenter Study

Background: The relationship between cardiac functions and the fatal outcome of coronavirus disease 2019 (COVID-19) is still largely underestimated. We aim to explore the role of heart failure (HF) and NT-proBNP in the prognosis of critically ill patients with COVID-19 and construct an easy-to-use predictive model using machine learning. Methods: In this multicenter and prospective study, a total of 1,050 patients with clinical suspicion of COVID-19 were consecutively screened. Finally, 402 laboratory-confirmed critically ill patients with COVID-19 were enrolled. A “triple cut-point” strategy of NT-proBNP was applied to assess the probability of HF. The primary outcome was 30-day all-cause in-hospital death. Prognostic risk factors were analyzed using the least absolute shrinkage and selection operator (LASSO) and multivariate logistic regression, further formulating a nomogram to predict mortality. Results: Within a 30-day follow-up, 27.4% of the 402 patients died. The mortality rate of patients with HF likely was significantly higher than that of the patient with gray zone and HF unlikely (40.8% vs. 25 and 16.5%, respectively, P < 0.001). HF likely [Odds ratio (OR) 1.97, 95% CI 1.13–3.42], age (OR 1.04, 95% CI 1.02–1.06), lymphocyte (OR 0.36, 95% CI 0.19–0.68), albumin (OR 0.92, 95% CI 0.87–0.96), and total bilirubin (OR 1.02, 95% CI 1–1.04) were independently associated with the prognosis of critically ill patients with COVID-19. Moreover, a nomogram was developed by bootstrap validation, and C-index was 0.8 (95% CI 0.74–0.86). Conclusions: This study established a novel nomogram to predict the 30-day all-cause mortality of critically ill patients with COVID-19, highlighting the predominant role of the “triple cut-point” strategy of NT-proBNP, which could assist in risk stratification and improve clinical sequelae.


INTRODUCTION
An outbreak of novel infectious pneumonia, now known as coronavirus disease 2019 (COVID-19) caused by severe acute respiratory syndrome coronavirus 2, has been quickly spreading around the world since December 2019. To date, more than 168 million confirmed cases of COVID-19 have been identified worldwide, with over 3.49 million deaths. Despite the advancement of learning the etiology and clinical characteristics of COVID- 19, there have been no effective strategies to wipe out the global COVID-19 epidemic, and it is still a public health threat.
The average mortality rate was estimated globally at 3.4% by the WHO, while it is 26-52% significantly higher for patients admitted to intensive care units (ICUs) (1,2). Moreover, no medications have been proven definitely effective for curing . Thus, early evaluation and identification of individuals with high-risk mortality are of paramount importance to further guide optimal intervention strategies. Of note, critically ill patients with COVID-19 usually have multiple organ dysfunctions (4), among which cardiac involvement is prevalent, especially acute heart failure (AHF) (5). However, the role of AHF in the prognosis of COVID-19 has not been fully elucidated in prior studies, partially because comprehensive evaluations of cardiac dysfunction that utilize imaging examinations were usually unavailable in real-world practice. Hence, we proposed a "triple cut-point" strategy of N-terminal pro-brain natriuretic peptide (NT-proBNP) as a reliable and easy-touse diagnostic tool for AHF in this study (6). Moreover, although previous studies have explored the risk factors of prognosis among critically ill patients with COVID-19 (7-9), a user-friendly and clinically relevant short-time outcome prediction model for patients with COVID-19 in ICU is still lacking. Therefore, to address the gaps mentioned above, this multicenter study aims to (1) explore the potential prognostic value of the "triple cut-point" strategy of NT-proBNP and AHF in critically ill patients with COVID-19; and (2) construct and validate a simplified and effective nomogram to predict all-cause in-hospital death risk individually.

Study Population
This multicenter, prospective, and observational study consecutively enrolled 1,050 patients with clinical suspicion of COVID-19 from four ICUs in Wuhan taken over by China-Japan Friendship Hospital, Peking University People's Hospital, Peking University First Hospital, and Peking University Third Hospital from January to May 2020. Patients who met one of the following criteria would be considered to be transferred to the ICU: (1) respiratory rate >30 breath/min; (2) blood oxygen (SpO 2 ) <93%; (3) PaO 2 /FiO 2 <300 mmHg; (4) presented with respiratory failure; (5) presented with shock; or (6) other conditions that need to be monitored in the ICU. Patients were diagnosed as COVID-19 with a positive result of real-time reverse transcriptase-polymerase chain reaction assay from nasal swab specimens according to WHO guidance (10). Exclusion criteria included patients who were not diagnosed with COVID-19, younger than 18 years of age, and had incomplete data, or died within 24 h of admission to the ICU. As a result, 402 patients were included in the final analysis. The study was conducted in accordance with the Declaration of Helsinki, and the protocol was approved by the Ethics Committee of Peking University People's Hospital.

Data Collection
The clinical data from each patient were recorded by experienced physicians following ICU admission and included demographic features, preexisting comorbidities, symptoms, vital signs, and length of ICU stay. The comorbidities included hypertension, coronary heart disease (CHD), diabetes mellitus (DM), chronic kidney disease (CKD), asthma, chronic obstructive pulmonary disease, chronic bronchitis, transient ischemic attack, ischemic stroke, and hemorrhagic stroke.
All the patients, during hospitalization, were followed up for 30-days or until discharge or death. The primary outcome was 30-day all-cause death after admission.

"Triple Cut-Point" Strategy of NT-proBNP
According to the recent guideline for HF, novel NT-proBNP cut-off values have been proposed to assist with AHF diagnosis. Hence, we classified the cases into three groups using this "triple cut-point" strategy of NT-proBNP to define the probability of AHF, as shown in Table 1. In detail, HF likely was defined as plasma NT-proBNP level > 450 pg/ml in patients below 50 years, >900 pg/ml in patients between 50 and 75 years, and >1,800 pg/ml in patients over 75 years (6). HF unlikely was defined as plasma NT-proBNP level <300 pg/ml regardless of age, while the stratified approach of 300 pg/ml to 450/900/1,800 pg/ml for ages <50/50-75/>75 years were considered as "gray zone."

Statistical Analysis
Continuous variables were presented as mean ± SD if normally distributed, and median and interquartile range otherwise. The differences between the two groups were compared by the Student t-test and Mann-Whitney U test appropriately. Categorical variables were shown as n (%) and compared by χ 2 test or Fisher exact test when necessary. Kaplan-Meier survival estimates were calculated, and the logrank test was performed to compare the groups in terms of survival. The least absolute shrinkage and selection operator (LASSO) method (glmnet package), which is appropriate for regression of high-dimensional data, was used to select the most useful predictive variables from the data set. Then, the multivariate logistic regression analysis was performed to identify independent risk factors. Odds ratios (ORs) were shown with a 95% CI.
The nomogram was established based on the multivariate logistic regression analysis (rms package). A likelihood ratio test approach for model selection was performed. Nomogram performance was quantified with respect to discrimination and calibration. Discrimination (the ability of a nomogram to separate patients with all-cause in-hospital death) was quantified with the concordance index (C-index) and 95% CI. Calibration was assessed graphically by plotting the relationship between actual (observed) probabilities and predicted probabilities (calibration plot) by Hosmer goodnessof-fit test. The internal validation of performance was estimated with the bootstrapping method (500 replications). Integrated discrimination improvement (IDI) and net reclassification improvement (NRI) (survival package) were used to assess the improved ability of the "triple cut-point" strategy of NT-proBNP for the predictive value of the model.
All the tests were two-tailed, and a P < 0.05 was considered significant. The statistical analyses were performed with the SPSS version 25.0 software (SPSS Inc., Chicago, IL, United States), R programming language, and environment version 3.6.0 (http:// cran.r-project.org).

Baseline Characteristics
This study finally included 402 critically ill patients with laboratory-confirmed COVID-19 and their baseline  Table 2. Overall, the mean age of the whole cohort was 67 years, and 54.5% (n = 219) were men. At the 30-day follow-up, 110 patients had died with a 27.4% mortality risk. Compared to the survivors, the non-survivors were more likely to be older, having decreased SpO 2 and elevated heart rate (HR) (all P < 0.05). Of note, there was no significance between the two groups regarding gender and comorbidities, irrespective of hypertension, CHD, DM, and respiratory system disease. Furthermore, we compared the laboratory data between the two groups and found that the nonsurvivors had significantly increased AST, total bilirubin, blood glucose, d-dimer, and PCT as well as decreased lymphocytes, platelets, and estimated glomerular filtration rate (eGFR) (all P < 0.05).

"Triple Cut-Point" Strategy of NT-proBNP and 30-Day Mortality
According to the "triple cut-point" strategy of NT-proBNP, the patients were divided into three groups, and the overall distribution of HF unlikely, gray zone, and HF likely was 170 (42.3%), 80 (19.9%), and 152 (37.8%), respectively ( Figure 1A). Table 3, the non-survivor group has a significantly higher percentage of patients with HF likely (56.3 vs. 30.8%), and the distribution of the three groups (HF unlikely, gray zone, and HF likely) between the non-survivor and survivor groups was significantly different (P < 0.001). Otherwise, within the 30-day follow-up, we observed a mortality rate of 16.5 (28/170), 25 (20/80), and 40.8% (62/152) in group HF unlikely, gray zone, and HF likely, respectively (P < 0.001). Importantly, the mortality rate increased sharply, accompanied by the increased likelihood of AHF ( Figure 1B). The Kaplan-Meier curves of short-time survival were shown in the central illustration, illustrating a significantly shorter mean survival time for patients with HF likely (Figure 1C). The overall cumulative risk of death at 30-days was significantly higher for the HF likely group than for HF unlikely and gray zone (P < 0.001). LASSO coefficient profiles of the 28 texture features. A vertical line was drawn at the value selected using 10-fold cross-validation, where optimal l resulted in nine non-zero coefficients. LASSO, least absolute shrinkage and selection operator; hs-cTNI, higher sensitivity cardiac troponin I; "TCP" of NT-proBNP, "triple cut-point" strategy of NT-proBNP; SpO 2 , blood oxygen; HR, heart rate.

Predictors of 30-Day in-hospital Death of Critically Ill Patients With COVID-19
The least absolute shrinkage and selection operator was used to select the potential prognostic factors from numerous parameters. Finally, 28 indexes were reduced to 10 potential predictors, namely, age, lymphocyte, platelet, total bilirubin, "triple cut-point" strategy of NT-proBNP, SpO 2 , HR, albumin, hs-cTNI, and d-dimer, based on the 402 patients, and were indexes with non-zero coefficients in the LASSO regression model (Figures 2A,B). Furthermore, as shown in Table 4

Development and Validation of a Novel Nomogram for Predicting Prognosis
An optimal nomogram comprising all the above independent predictors was established to individualize the risk of 30-day inhospital death (Figure 3A). The ratios of calculated β were used to decide the proportional prognostic effect of these variables. Projections from total points on the scales below indicated the estimated probability of death. Performance accuracy was evaluated by the area under the curve (AUC) of the receiver operating characteristic (ROC) analysis. The AUC for in-hospital death was 0.781 (95% CI 0.733-0.827) (Figure 3B). The calibration curve of the nomogram for the probability of death demonstrated good agreement between prediction and observation in the primary cohort ( Figure 3C). Hosmer-Lemeshow goodness-of-fit was satisfied (P = 0.354). The C-index for the prediction nomogram was 0.798 (95% CI 0.742-0.857). The decision curve analysis (DCA) for the clinical laboratory index nomogram is presented in Figure 3D. It showed that this nomogram had more benefits than the treat-all-patients scheme or the treat-none scheme in predicting the risk of 30day in-hospital death of critically ill patients with COVID-19. Moreover, the bootstrap validation method was used to verify the predictive accuracy of the nomogram. The C-index for the nomogram of 30-day in-hospital death was 0.779 (95% CI 0.721-0.834), suggesting the accuracy of this predictive nomogram.

Incremental Predictive Value of "Triple Cut-Point" Strategy of NT-proBNP
To investigate the role of the "triple cut-point" strategy of NT-proBNP in the predictive value of the current model, NRI and IDI were calculated. Compared with the model without the "triple cut-point" strategy of NT-proBNP, the addition of the "triple cut-point" strategy of NT-proBNP resulted in a significantly improved discrimination [IDI 7.3% (95% CI 1.1-14.5%) and NRI 4.9% (95% CI 2.6-7.2%), both with P < 0.05].

DISCUSSION
In this prospective multicenter study, we recruited 402 critically ill patients with COVID-19 from four ICUs in China and established a novel nomogram to predict the 30-day all-cause mortality risk in these patients. To the best of our knowledge, there have been few risks score models for predicting the prognosis of critically ill patients with COVID-19. This study has developed a user-friendly and relatively personalized model incorporating five variables, age, "triple cut-point" strategy of NT-proBNP, albumin, lymphocyte count, and total bilirubin, to predict short-time mortality risk in critically ill Chinese patients with COVID-19, which could assist risk stratification and provide insights for timely interventions upon admission. Furthermore, it is highlighted that the "triple cut-point" strategy of NT-proBNP  The mortality risk of COVID-19 has been proven high, with 28-day mortality ranging from 26-53.8% in critically ill adult patients worldwide (1,11), indicating the imperative of proposing an easy-to-use prediction model to assist riskstratify and therapeutic optimization in clinical practice for ICU patients. Emerging evidence has tried to explore the risk factors and construct diagnostic and prognostic models in COVID-19 populations (12,13). However, previous reports regarding prognosis prediction have mainly focused on disease progression or mortality risk of the whole group without further distinguishing critically ill patients in ICU (14). The sample selection bias of the prior models could lead to poor adaptions. In line with existing data, this study reported a 27.6% mortality risk, and further constructed and validated a novel nomogram for the prediction of 30-day all-cause death. Variables referring to older age, higher level of total bilirubin, lower level of lymphocyte count and albumin, and "triple cut-point" strategy of NT-proBNP were likely to recognize individuals who are at high risk with high sensitivity and specificity. More importantly, the quantitative appraisal made it possible to estimate the likelihood of death more accurately and individually with easy and rapid access in clinical practice.
It has been widely confirmed that cardiac involvement, such as cardiac injury, arrhythmias, myocarditis, and cardiac dysfunction, was prevalent and prognostic in hospitalized patients with COVID-19 (15)(16)(17), among which HF is responsible for substantial morbidity and mortality (10). New-onset HF was observed in nearly 23% of hospitalized patients with COVID-19 and as much as one-third of those admitted to the ICU (2,9). Recent reports revealed that HF was the most frequent cause of death just after acute respiratory distress syndrome (ARDS) and sepsis, accounting for 27.4% of the proximate causes of death in patients with COVID-19 (18). However, the role of HF in the prognosis of critically ill patients with COVID-19 has not been fully elucidated, partially because of high diagnostic uncertainty. A complete diagnosis of HF usually includes symptoms, signs, biomarkers (BNP/NT-proBNP), and imaging examinations, while it is impractical and unavailable to evaluate cardiac function by echocardiography for each critically ill patient with COVID-19 in the clinical practice. Although BNP/NT-proBNP levels are easily interfered with and obscured by considerable factors, the utility of these biomarkers performed well in the emergency setting as an adjunct tool for the diagnosis and triage of dyspneic patients. As such, the guidance has recommended BNP/NT-proBNP as a diagnostic aid for HF with comparable diagnostic accuracy.
As the widely admitted biomarker in HF, NT-proBNP quantitatively reflects hemodynamic myocardial stress (19), indicating not only left ventricular (LV) systolic dysfunction but also cardiac abnormalities, such as LV diastolic dysfunction, right ventricular (RV) dysfunction, valvular dysfunction, increased pulmonary pressures, and atrial arrhythmias. Prior studies have observed ambiguous results that higher levels of BNP or NT-proBNP were found in patients with severe COVID-19 and that they were independently associated with high mortality, maybe because of single-center design, patient population selection bias, and small sample size (20)(21)(22)(23). This multicenter study demonstrated that the non-survivors had a significantly higher level of NT-proBNP than the survivors (1,564 vs. 321 ng/ml), with reasonable sample size. Consistently, a recent study described the characterization of NT-proBNP in patients with COVID-19, and 48.5% of their cohort presented NT-proBNP levels above the recommended cut-off for the identification of HF (24).
Furthermore, considering the fact that the plasma level of NT-proBNP is largely affected by age and renal functions, it seems to be not rigorous enough to use NT-proBNP as a simple continuous variable alone to predict the prognosis of patients with COVID-19. Thus, we reclassified the subjects into three groups (HF likely, gray zone, and HF unlikely) according to the recent HF guidance as to the "triple cut-point" strategy of NT-proBNP, and observed that patients with HF likely occupied 37.8% of the total cohort, of which 56.3% were non-survivors (6). Moreover, we found that patients in the HF likely group had a significantly higher risk (OR 1.97, 95% CI 1.133-3.424) for 30day all-cause death. Concerning the clinical presentations and biomarkers of HF on time would help make optimal individual treatment plans to prevent further deterioration efficiently.
It is worth noting that our prediction model did not incorporate troponin, as it was not independently associated with the outcome unexpectedly, while prior studies have suggested that troponin was a significant prognostic indicator in COVID-19 (15,25). Similarly, Dong et al. conducted a retrospective study and built a nomogram assessing the 14-day and 21-day inhospital survival of all the general patients with COVID-19. The final model was constructed based on hypertension, neutrophilto-lymphocyte ratio, and NT-proBNP (26). Elevated troponin may be a possible confounder for NT-proBNP as they were postulated to share the same pathophysiological processes and found to be both elevated in pneumonia, sepsis, ARDS, and several other non-cardiac illnesses. Hence, we speculated that an accurate classification using the "triple cut-point" strategy of NT-proBNP may decrease the confounding effect of troponin. Notably, liver injuries, such as elevated total bilirubin and decreased albumin, have also been demonstrated to be common and associated with disease severity and poor outcomes for critically ill patients in this study, in accordance with previous studies (27). Furthermore, elevated total bilirubin may also be associated with cardiac dysfunctions as a significant and independent predictor of poor cardiovascular prognosis in patients with HF (28).
Critically ill conditions with COVID-19 were usually complicated by multiple organ dysfunctions with complex pathophysiological processes involving numerous parameters, including but not limited to hypoxemia, inflammation, thromboembolism, renal failure, and cardiac damage (29). Therefore, it is of vital importance to bring all reasonable possible variables into analysis and construct a scientific prediction model relying on appropriate statistical analysis awfully. In the current study, LASSO, a machine learning algorithm, was applied to shrink the regression coefficients from amounting clinical and laboratory indicators to 10 potential predictors. Thus, this algorithm could conquer common confusing collinearity issues and yield more robust results than traditional variable screening methods.
Some cautions should be considered when interpreting our results. First, although our study is observational and the sample size is relatively small, it has a multicenter and prospective design emphasizing critically ill patients. Further investigations with a larger sample size are warranted. Second, this study did not apply other diagnostic tools to make a complete HF diagnosis. However, it is impractical and unavailable to evaluate cardiac functions by echocardiography for each critically ill patient with COVID-19. Conversely, rapid measurements of NT-proBNP have substantial medical aids to fulfill the clinical need underlying this extraordinary stressful setting, although it should never be a stand-alone test for HF diagnosis. Third, our nomogram model lacks validation in an external population. Nevertheless, internal verification indicated the predictive strength in our study.

CONCLUSIONS
In this study, we explored the independent predictors for shorttime prognosis in critically ill patients with COVID-19 in China and established a novel nomogram to predict the 30-day all-cause mortality risk for the first time, highlighting the predominant role of the "triple cut-point" strategy of NT-proBNP. This easy-touse prognostication nomogram can provide survival estimations and help identify patients with COVID-19 with a high-risk trajectory, further advancing clinical management and ultimately improving outcomes.

DATA AVAILABILITY STATEMENT
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

ETHICS STATEMENT
The studies involving human participants were reviewed and approved by the Ethics Committee of Peking University People's Hospital. The ethics committee waived the requirement for written informed consent.