A novel web-based calculator to predict 30-day all-cause in-hospital mortality for 7,202 elderly patients with heart failure in ICUs: a multicenter retrospective cohort study in the United States

Background and aims Heart failure (HF) is a significant cause of in-hospital mortality, especially for the elderly admitted to intensive care units (ICUs). This study aimed to develop a web-based calculator to predict 30-day in-hospital mortality for elderly patients with HF in the ICU and found a relationship between risk factors and the predicted probability of death. Methods and results Data (N = 4450) from the MIMIC-III/IV database were used for model training and internal testing. Data (N = 2,752) from the eICU-CRD database were used for external validation. The Brier score and area under the curve (AUC) were employed for the assessment of the proposed nomogram. Restrictive cubic splines (RCSs) found the cutoff values of variables. The smooth curve showed the relationship between the variables and the predicted probability of death. A total of 7,202 elderly patients with HF were included in the study, of which 1,212 died. Multivariate logistic regression analysis showed that 30-day mortality of HF patients in ICU was significantly associated with heart rate (HR), 24-h urine output (24h UOP), serum calcium, blood urea nitrogen (BUN), NT-proBNP, SpO2, systolic blood pressure (SBP), and temperature (P < 0.01). The AUC and Brier score of the nomogram were 0.71 (0.67, 0.75) and 0.12 (0.11, 0.15) in the testing set and 0.73 (0.70, 0.75), 0.13 (0.12, 0.15), 0.65 (0.62, 0.68), and 0.13 (0.12, 0.13) in the external validation set, respectively. The RCS plot showed that the cutoff values of variables were HR of 96 bmp, 24h UOP of 1.2 L, serum calcium of 8.7 mg/dL, BUN of 30 mg/dL, NT-pro-BNP of 5121 pg/mL, SpO2 of 93%, SBP of 137 mmHg, and a temperature of 36.4°C. Conclusion Decreased temperature, decreased SpO2, decreased 24h UOP, increased NT-proBNP, increased serum BUN, increased or decreased SBP, fast HR, and increased or decreased serum calcium increase the predicted probability of death. The web-based nomogram developed in this study showed good performance in predicting 30-day in-hospital mortality for elderly HF patients in the ICU.

Background and aims: Heart failure (HF) is a significant cause of in-hospital mortality, especially for the elderly admitted to intensive care units (ICUs).This study aimed to develop a web-based calculator to predict -day in-hospital mortality for elderly patients with HF in the ICU and found a relationship between risk factors and the predicted probability of death.

Methods and results: Data (N =
) from the MIMIC-III/IV database were used for model training and internal testing.Data (N = , ) from the eICU-CRD database were used for external validation.The Brier score and area under the curve (AUC) were employed for the assessment of the proposed nomogram.Restrictive cubic splines (RCSs) found the cuto values of variables.The smooth curve showed the relationship between the variables and the predicted probability of death.A total of , elderly patients with HF were included in the study, of which , died.Multivariate logistic regression analysis showed that -day mortality of HF patients in ICU was significantly associated with heart rate (HR), -h urine output ( h UOP), serum calcium, blood urea nitrogen (BUN), NT-proBNP, SpO , systolic blood pressure (SBP), and temperature (P < .).The AUC and Brier score of the nomogram were .
( ., . ) in the external validation set, respectively.The RCS plot showed that the cuto values of variables were HR of bmp, h UOP of .L, serum calcium of .mg/dL, BUN of mg/dL, NT-pro-BNP of pg/mL, SpO of %, SBP of mmHg, and a temperature of .• C.

Introduction
Heart failure (HF) is a typical cardiovascular syndrome of aging and a leading cause of morbidity and mortality.It is mainly caused by common cardiovascular diseases in the elderly and age-related changes in cardiovascular structure and function.The incidence of HF reaches 39.8% for the adult population aged over 80 years and 27.1% for patients who are 65-79 years old (1).HF patients in the ICU were reported to have high inhospital mortality (2), especially for the elderly (3).Previous studies have reported the risk factors for morality in HF patients (4)(5)(6).However, the evidence of risk factors for 30-day in-hospital mortality in elderly patients with HF during ICU admission is insufficient.
Recently, the nomogram was widely adopted as an accurate tool to evaluate the prognosis in HF patients (7,8).However, a convenient and effective web-based prediction model for elderly HF patients admitted to the ICU remains lacking.Therefore, this study aimed to establish a web-based nomogram to predict the 30-day in-hospital mortality for elderly HF patients during ICU admission and discover the potential risk factors, which could support clinical decision-making.

Data source
We conducted a retrospective cohort study with de-identified clinical data extracted from the Medical Information Mart for Intensive Care III/IV (MIMIC-III/IV) database and eICU-CRD (certification ID: 42039823) (9).The MIMIC-III is a publicly available single-center critical care database that contains deidentified clinical data for over 50,000 patients admitted to the Beth Israel Deaconess Medical Center.The MIMIC-IV database is a publicly available critical care database containing de-identified clinical data that contains over 70,000 ICU admissions across the United States from 2008 to 2019.The eICU-CRD database is a publicly available multi-center critical care database that was made available by Philips Healthcare in partnership with the MIT Laboratory for Computational Physiology and contains over 200,000 patients admitted to the ICU from 2014 to 2015.Additionally, the model was then externally validated in patients enrolled in the eICU-CRD database.

Study population and outcome
Patients diagnosed with HF and over the age of 65 in their last ICU stay were enrolled in the study cohort.HF was identified based on the International Classification of Diseases-Ninth Revision (ICD-9/10) diagnosis codes.Only first-time ICU admission patients were included in this study.The exclusion criteria for this study include (1) patients aged < 65 years; (2) patients without a record of N-terminal pro-brain natriuretic peptide (NT-proBNP); (3) patients without a record of an electrocardiogram; and (4) missing more than 30% of patient data.The primary outcome was 30-day in-hospital mortality.

Data extraction and preprocessing
Clinical data on the participants were collected from the three databases based on clinical guidelines and other literature published previously on relevant topics (5).Baseline characteristics with 35 variables for the included patients were recorded on the first admission to the hospital within 24 h, including vital signs, demographics, laboratory tests, and comorbidities.

Statistical analysis
Normally distributed continuous variables were expressed as the median with a standard error.Non-normally distributed continuous variables were expressed as the median with an interquartile range (IQR).Categorical data were expressed as frequency (percentage, %) and compared using the chi-square test.A Mann-Whitney U-test was applied to compare non-normally distributed variables.Missing values are imputed by the K-nearest neighbor (KNN) algorithm.
The study cohort was divided randomly into a training set with 80% of patients from MIMIC III/IV for developing the prediction model and a testing set with 20% of patients for evaluating the performance of the model.Additionally, the model was then externally validated in patients enrolled in the eICU-CRD database.
Univariate analysis was performed on the clinical data of patients who died and those who survived.The differences were statistically significant and were further included in the LASSO regression analysis.The variables selected by LASSO were included in the multivariate logistic regression analysis.The LASSO analysis of elderly patients with HF for 30-day inhospital mortality was conducted to adjust potential confounding factors by regularization.This method can improve the prediction accuracy and dimensionality reduction of high-dimensional data.Variables with non-zero coefficients in the LASSO regression with 10-fold cross-validation were selected for further analysis to ensure minimal autocorrelation.The web-based nomogram, a web calculator, can accurately estimate the risk probability of clinical outcomes based on individual characteristics (7).Then, this study developed a nomogram for the retained variables with nonzero coefficients in LASSO regression and multivariable logistic regression in the training set.The Brier score and area under the curve (AUC) were employed for the predictive performance of the dynamic nomogram.A restrictive cubic spline was applied to find the cutoff value of the variable.The smooth curve was used to show the relationship between variables and the predicted probability of death.The patients in the training set were divided into the ultrasenior group (aged over 80 years) and the elderly group (65 ≤ age ≤ 80 years) for subgroup analysis.The calibration curve was applied to analyze the agreement between the predictive results and actual observations.Clinical usefulness was evaluated by decision curve analysis (DCA) and the clinical impact curve (CIC).
All statistical analyses were performed with R 4.1.3and RStudio 2022.02.3.The R package "DMwR2" was used for KNN algorithm imputation.The LASSO and logistic regression analyses were performed by the R package "glmnet".The R package "DynNom" was used for the dynamic nomogram.The R package "riskRegression" was used for calculating Brier scores.The receiver operating characteristic curve analysis was conducted using the "ROCR" package.A P-value of < 0.05 was considered statistically significant.

Patients' characteristics
A total of 7,202 elderly patients with HF were enrolled from the three databases with ICU admissions.The screening process is shown in Figure 1.This study divided 4,450 patients from MIMIC III/IV into the training set (3,549 participants) and     The VIF values and tolerance values of the variables are shown in Table 3.

Variables and death prediction probability analysis
The RCS plot showed that the cutoff value of variables were HR of 96 bmp, SBP of 137 mmHg, temperature of 36.4 • C, SpO 2 of 93%, 24h UOP of 1.2L, NT-proBNP of 5121 pg/dL, BUN of 30 mg/dL, and serum calcium of 8.7 mg/dL (Figures 7B, 8).The smoothed curve (Figure 8) showed the values of SpO 2 (93%), serum calcium (10 mg/dL), 24h UOP (<1L), and SBP (100 to 140 mmHg) with the lowest probability of death.However, the predicted probability of death changed dramatically when the SBP was <100 mmHg or more than 160 mmHg.

Subgroup analysis
The results of the subgroup analysis (Table 4) showed that, in the subgroup of ultra-elderly patients aged > 80 years, all the selected variables except temperature were risk factors for 30-day mortality during ICU

Discussion
This study was the first attempt to develop a convenient and effective web-based nomogram model for predicting the 30-day mortality of elderly HF patients in the MIMIC-III/IV database and the eICU-CRD database.Our analysis showed that 30-day mortality during ICU admission was significantly associated with heart rate, SBP, temperature, SpO 2 , 24h UOP, NT-proBNP, BUN, and serum calcium.The nomogram model produced in this study showed excellent performance.The RCS plot and the smoothed curves increased clinical practicality with the cutoff value of variables and the value of variables with the lowest probability of death.
The incidence and prevalence of HF progressively increase in parallel with the population's age.HF remains one of the most immediate causes of death in the United States as well as in most other countries in the world (10).One study analyzed the outcomes of 1,767 HF patients from 4 countries, in which 387 patients died and 343 were hospitalized for exacerbation during the follow-up duration, and a gradual decline in survival was observed for both mortality and hospitalization over 3 years (1).A key difficulty in the management of elderly HF patients is to identify the risk factor for early mortality, as the clinical situation is often unpredictable.Previous studies have shown that sex, blood pressure, age, serum sodium, BUN, NT-proBNP levels, and EF were risk factors for mortality in HF patients (3,(11)(12)(13).Our study identified risk factors for hospital mortality for elderly HF patients during ICU admission, which is crucial for evaluating the severity of illness and initiating targeted treatment.There are some points that should be highlighted in this study.First, our nomogram model included a large number of patients from three databases and was designed specifically for elderly patients with HF during ICU admission, unlike other score models that were designed for all ICU patients.Second, with the analysis of the RCS plot and the smoothed curves, it can be utilized not only to predict the risk of mortality but also to identify the cutoff value of variables and the value of variables with the lowest probability of death, which will have a significant impact on the physician's decision-making.Third, our nomogram showed an excellent predictive probability in the external validation set, illustrating a strong degree of credibility.Additionally, we used three databases to produce an optimal prognostic prediction model for 30-day mortality and discovered general variables as risk factors, including heart rate, SBP, temperature, SpO 2 , 24h UOP, NT-proBNP, BUN, and serum calcium, which were accessible for physicians' clinical practices.Finally, LASSO regression with a satisfied penalty condition was applied to improve the prediction accuracy and was suitable for reduction in high-dimensional data, which penalized the coefficients of some useless variable features and enhanced the generalization ability of the model.
Previous studies proved that increased heart rate represents an important indicator of mortality, with each acceleration of the heart rate over 70 bpm increasing the risk not only in patients with cardiovascular disease but also in the general population (14).Heart rate was qualified as a modifiable risk factor for heart failure.In our study, we found that a heart rate of 96 bmp in elderly HF patients is the cutoff value for the risk of mortality.The better the heart rate is controlled, the lower the risk of death for the old.Williamson et al. (15) reported that adults aged 75 years or older with an SBP of <120 mmHg had significantly lower rates of fatal and non-fatal major cardiovascular events and death from any cause.Elderly patients with HF are a special group accompanied by cardiac insufficiency.Our study has obtained a more detailed SBP control interval of 100 to 140 mmHg, which is conducive to better regulation of blood pressure in clinical practice.A previous study (16) demonstrated that decreasing temperatures increased HF rehospitalization.Our study found that a decreased temperature of <36.4 • C is associated with mortality, which may be associated with systemic failure resulting in peripheral hypoperfusion.Decreased SpO 2 is often accompanied by respiratory insufficiency, which is something that many HF patients (17), especially the elderly, should focus on.Our study suggests that an increase in SpO 2 of more than 93% in these patients may have a protective effect.For elderly patients with HF, decreased 24 h UOP or increased NT-proBNP often indicate severe impairment of cardiac function, especially when combined with renal insufficiency (increased serum BUN), leading to an increased risk of death.Lutsey et al. (18) found that high serum calcium was independently associated with a greater risk of incident HF.We found that too low or too high serum calcium had a poor prognosis for elderly HF patients.
Logistic regression models showed a linear relationship between variables and the outcome.However, the relationship between the univariate analysis and outcome may be non-linear The establishment of a prediction model for patients has always been a hot topic (19,20).Therefore, in this study, the nomogram based on the eight risk factors above was applied to predicting the mortality risk of ICU-admitted older patients with HF.The application of a dynamic nomogram predicted the patient's 30-day probability of in-hospital death in elderly patients with HF.Health professionals could take targeted interventions according to the scores of different risk factors on the nomogram for each subject, improving the efficiency of interventions early.However, a non-professional statistician may find a regular nomogram inconvenient because it requires the risk calculation for each variable.Therefore, we constructed a web-based dynamic nomogram (https://huangjian.shinyapps.io/elderHFDynNomapp/) based on the nomogram model.Clinicians could access the website directly anytime and anywhere and input corresponding predictors to obtain an individual's death probability with a 95% CI.This would undoubtedly simplify the application process and facilitate decision-making, which may help tailor individualized, risk-based treatment strategies.Furthermore, it is convenient for elderly patients with HF and their caregivers to receive targeted interventions based on the results of a dynamic nomogram.This study extends this knowledge in several important ways.First, compared with the convenient logistic regression model, the nomogram produced in this study assessed risk stratification.Second, this nomogram has been generated specifically for elderly HF patients.We have provided an example of a patient in the nomogram using clinical indicators to predict the patient's 30-day probability of in-hospital death (Figure 3).This model predicted a lower probability of death for the patient at 21.3%, and we eventually confirmed that this patient was, indeed, alive when he was discharged from the hospital.The AUC value is >0.7, and the Brier score index value is <0.15, indicating that the model calibration is high and the prediction performance is good.As such, the nomogram model can accurately identify high-risk elderly HF patients and carry out early intervention to prolong survival time.
There were some limitations in this study.First, patients included in this study lacked clinical indicators and outcomes with longer follow-up duration.Multicenter clinical studies with long-term follow-up and large sample sizes are needed to validate this nomogram in future.Second, some risk factors were only analyzed for the first time within 24 h of admission, such as blood pressure, temperature, and heart rate, which were always volatile.
The analysis of factor instabilities or timing changes is required for further study.

Conclusion
This study developed a convenient and effective web-based nomogram tool to predict the 30-day mortality of elderly HF patients in the MIMIC-III/IV and the eICU-CRD databases.
Frontiers in Medicine frontiersin.orgtheir affiliated organizations, or those of the publisher, the editors and the reviewers.Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

FIGURE
FIGURE Screening process of included patients and the constructed model.HF, heart failure; ICUs, intensive care units; MIMIC-III, Medical Information Mart for Intensive Care III; ICD, International Classification of Diseases.

FIGURE
FIGUREPredictors selected by the LASSO method.(A) The coe cients of all predictors gradually return to zeros through -fold cross-validation.(B) Eleven predictors with non-zero coe cients at the rightmost dashed line.LASSO, least absolute shrinkage and selection operator.

FIGURE
FIGURENomogram to estimate the risk of mortality in elderly with heart failure.The blue histogram in (A) represents the distribution of variables, and the red dots represent specific values.(B) An interface display of the web-based nomogram.We describe the value score for each corresponding parameter and the probability of mortality for a specific instance.The numbers " " and " " in renal failure and deficiency anemias mean "No" and "Yes", respectively.Urine output, urine output in first h.BUN, serum blood urea nitrogen; INR, international normalized ratio.

FIGURE
FIGURE Clinical usefulness of nomogram in the training set and the testing set.(A) Decision curve analysis curve in the training set and the testing set.(B) Clinical impact curve in the training set.(C) Clinical impact curve in the testing set.CIC and DCA showed that the nomogram had a superior overall net benefit within the wide and practical ranges of threshold probabilities and impacted clinical outcomes.Train, train; test, test.
FIGURE ROC plot of nomogram in the training set, testing set, and external validation set.The AUC showed favorable accuracy of the nomogram model.Train, training set; test, testing set; ROC, receiver operating characteristic curves; AUROC, area under the receiver operating characteristic curves.

FIGURE
FIGUREVariable collinearity analysis and RCS plot.(A) The correlation coe cient between variables; blue represents a positive correlation, red represents a negative correlation, and the darker the color, the stronger the correlation.(B) RCS plot shows the size of the cuto value of the variable, the intersection of the red solid line, and the dotted line is the cuto value for h UOP.

FIGURE
FIGURESmooth curve.(A) For the relationship between blood oxygen saturation and predicted probability of death; the larger the value of blood oxygen saturation, the lower the predicted probability of death.(B) For the relationship between h UOP and predicted probability of death; the larger the value of h UOP, the higher the predicted probability of death small.(C) The relationship between serum calcium levels and predicted probability of death.(D) The relationship between systolic blood pressure and predicted probability of death.Red-filled shading represents confidence intervals.
TABLE Characteristics of elderly patients with heart failure from MIMIC III/IV.
TABLE The multivariate logistic regression of selected variables by LASSO regression.
TABLE Variance inflation factor and tolerance values for variables.