Development and validation of a nomogram for the early prediction of acute kidney injury in hospitalized COVID-19 patients

Introduction: Acute kidney injury (AKI) is a prevalent complication of coronavirus disease 2019 (COVID-19) and is closely linked with a poorer prognosis. The aim of this study was to develop and validate an easy-to-use and accurate early prediction model for AKI in hospitalized COVID-19 patients.

Methods: Data from 480 COVID-19-positive patients (336 in the training set and 144 in the validation set) were obtained from the public database of the Cancer Imaging Archive (TCIA). The least absolute shrinkage and selection operator (LASSO) regression method and multivariate logistic regression were used to screen potential predictive factors to construct the prediction nomogram. Receiver operating curves (ROC), calibration curves, as well as decision curve analysis (DCA) were adopted to assess the effectiveness of the nomogram. The prognostic value of the nomogram was also examined.

Results: A predictive nomogram for AKI was developed based on arterial oxygen saturation, procalcitonin, C-reactive protein, glomerular filtration rate, and the history of coronary artery disease. In the training set, the nomogram produced an AUC of 0.831 (95% confidence interval [CI]: 0.774–0.889) with a sensitivity of 85.2% and a specificity of 69.9%. In the validation set, the nomogram produced an AUC of 0.810 (95% CI: 0.737–0.871) with a sensitivity of 77.4% and a specificity of 78.8%. The calibration curve shows that the nomogram exhibited excellent calibration and fit in both the training and validation sets. DCA suggested that the nomogram has promising clinical effectiveness. In addition, the median length of stay (m-LS) for patients in the high-risk group for AKI (risk score ≥ 0.122) was 14.0 days (95% CI: 11.3–16.7 days), which was significantly longer than 8.0 days (95% CI: 7.1–8.9 days) for patients in the low-risk group (risk score <0.122) (hazard ratio (HR): 1.98, 95% CI: 1.55–2.53, p < 0.001). Moreover, the mortality rate was also significantly higher in the high-risk group than that in the low-risk group (20.6 vs. 2.9%, odd ratio (OR):8.61, 95%CI: 3.45–21.52).

Conclusions: The newly constructed nomogram model could accurately identify potential COVID-19 patients who may experience AKI during hospitalization at the very beginning of their admission and may be useful for informing clinical prognosis.

Introduction

In December 2019, Wuhan, China, reported the emergence of new coronavirus-associated pneumonia brought on by the novel SARS-CoV-2 infection (1, 2). On February 12, 2020, the World Health Organization (WHO) formally identified it as coronavirus disease 2019 (COVID-19), and on March 11, 2020, it was deemed a global pandemic. As of March 30, 2022, 227 countries and territories have been affected worldwide, with cumulatively more than 485 million cases confirmed, with over 6 million deaths (3). The main clinical feature of COVID-19 is acute respiratory symptoms (1, 2, 4, 5). Depending on the severity of the disease, patients can present with mild infections with no symptoms; moderate infections with symptoms such as fever, cough, and dyspnea; or even severe infections with acute respiratory distress syndrome (ARDS) (6, 7).

Although COVID-19 is a respiratory illness, it often results in multisystem damage that further progresses to multiple organ failure (MODS) and even, in severe cases, to patient death (4–7). The kidney is an important target organ for COVID-19 infection, and viral invasion causes acute kidney injury (AKI) through direct attack, an inflammatory storm, and inflammatory cell infiltration (8–10). The global incidence of COVID-19 in combination with AKI ranges from 0. 5 to 80%, and the incidence of AKI in the intensive care unit (ICU) ranges from 6 to 80% (11). The incidence of AKI significantly increases after COVID-19 infection (10, 12, 13). Studies have revealed that, compared to those hospitalized for non-COVID-19 reasons, COVID-19-infected hospitalized patients have an increased prevalence of AKI (31.0 vs. 18.0%) (14). A meta-analysis of 13,137 patients showed that the incidence of AKI in patients with COVID-19 was 17% (11). While, two observational studies that included 6,477 and 5,216 patients, respectively, revealed that the incidence of AKI among hospitalized COVID-19 patients was as high as 32 and 37% (15, 16). AKI increased the frequency and risk of mechanical ventilation in COVID-19 patients and lengthened their hospital stays. In addition, close to half of AKI patients did not have full recovery of renal function to baseline on discharge (16). Moreover, the incidence of AKI is linked with hospital mortality in patients with COVID-19 infection and is an independent risk factor for poor prognosis in critically ill patients (10, 12, 13, 17). A study that included 3,099 adult patients in critical condition who had COVID-19 showed that 20.6% of patients had to undergo kidney replacement therapy (KRT) for severe AKI within 14 days of entry to the intensive care unit. On day 28, the overall mortality rate for these patients was 54.9%, and up to 63.3% by the time of the last follow-up (17 days). Even among patients who were eventually discharged with a cure, there were still 33.6% of them dependent on KRT at discharge, and more than 50% of these patients still relied on KRT for the following 2 months (18). An autopsy study of patients who died from COVID-19 revealed that AKI was observed in 93.9% of patients, and 62% of patients experienced acute tubular necrosis of a different degree (14). Therefore, early clinical identification of patients who are at high risk for AKI could optimize the allocation of medical resources and enhance intervention management, thereby improving prognosis and reducing mortality.

Hence, we aimed to apply a new method to establish and validate a simple-to-use and effective early prediction model for AKI in hospitalized COVID-19 patients based on clinical characteristics, past medical history, clinical symptoms, signs, and key laboratory biochemical indicators. The model could help clinicians to screen patients with COVID-19 for the risk of AKI to identify and intervene in the early development of AKI. Furthermore, we investigated the prognostic differences between patients with high- and low-risk AKI based on predictive models.

Methods

Data collection and study design

Data from 480 COVID-19-positive patients were obtained from the public database of the Cancer Imaging Archive (TCIA) (collection of COVID-19-NY-SBU). This collection of patients was acquired at Stony Brook University with associated clinical data. AKI was defined as: (1) An increase in serum creatinine of 0.3 mg/dL within 48 h; (2) A rise in serum creatinine that is known or suspected to have happened within the previous 7 days, increasing it to 1.5 times baseline (or 50% above baseline); (3) Urine volume <0.5 ml/kg/h for 6 h. The inclusion criteria are as follows: (1) Age ≥18 years (weight ≥ 35Kg); (2) Laboratory-confirmed COVID-19 [positive polymerase chain reaction (PCR)]; (3) Expected hospital stay longer than 48 h; (4) With complete clinical information and laboratory test results. The exclusion criteria are as follows: (1) The Previous history of confirmed COVID-19. (2) The patient has received prophylactic treatment for COVID-19 within the last 30 days. (3) Patients with underlying renal disease such as chronic renal failure or post-transplantation or those on continuous renal replacement therapy, hemodialysis, or peritoneal dialysis. (4) Other diseases that may affect kidney function, such as tuberculous kidney disease, immune nephritis, and kidney tumors. (5) Presence of other serious diseases that damage life expectancy, such as acute myocardial infarction, cerebral hemorrhage, and pulmonary embolism. (6) Pregnancy and breastfeeding. The following information was collected for each patient: (1) general clinical characteristics of age, gender, and smoking history; (2) past medical history of hypertension, coronary artery disease (CAD), chronic obstructive pulmonary disease (COPD), and other lung diseases; (3) home medication history of an angiotensin-converting enzyme inhibitor (ACEI), angiotensin receptor blocker (ARB), antibiotics, and non-steroidal anti-inflammatory drugs (NSAID); (4) clinical symptoms of fever, cough, dyspnea, vomiting, diarrhea, and abdominal pain; (5) signs of oral temperature, arterial oxygen saturation (SaO2), respiratory rate, heart rate, systolic blood pressure, and mean blood pressure; (6) laboratory indicators of leukocyte count, neutrophils count, lymphocytes count, aspartate aminotransferase (AST), alanine aminotransferase (ALT), procalcitonin (PCT), C-reactive protein (CRP), sodium, potassium, chloride, lactate, blood urea nitrogen (BUN), serum creatinine (SCR), glomerular filtration rate (GFR), and glucose. All cohort patients were randomly divided into two sets at a ratio of 7:3: the training set was used to construct the prediction model, and the validation set was used to evaluate the performance of the model. The study was approved by the Ethics Committee of Yantai Yuhuangding hospital and conducted in accordance with the ethical principles of the Declaration of Helsinki. As all of the data in this work were retrieved from free online databases, informed consent was waived.

Element selection and construction of the nomogram

The least absolute shrinkage and selection operator (LASSO) regression method was utilized in the training set in order to eliminate potentially predictive elements for AKI. LASSO regression analysis was performed to gain refinement of the model by constructing a penalization function, and applicable to regression analysis of high-dimensional data with multiple covariates. In the process of parameter selection, the LASSO regression automatically shrinks the regression coefficients of 41 parameters using the penalty parameter lambda (λ). The larger the value of lambda (λ), the more the coefficients of the parameters shrink to zero. Consequently, some parameters are eliminated due to the narrowing of their coefficients to near zero, while the remaining parameters are ultimately selected. Cross-validation was adopted to validate the adjustment parameter lambda (λ) appropriateness for the LASSO regression. The lambda (λ) parameter with minimum criteria of mean-squared error was selected to screen the potential predictive elements. Factors screened in LASSO regression were subsequently analyzed in a multivariate logistic regression model to identify significant predictors of AKI in hospitalized COVID-19 patients. To avoid overfitting, elements in the multivariate logistic regression model with a p-value < 0.1 were used to construct the prediction nomogram.

Validation of the nomogram

Boost bootstrapping validation (1,000 bootstrap resamples) was used to evaluate the predictive effectiveness of the nomogram model in both the training- and validation sets. The performance metrics include the receiver operating curves (ROC), calibration curves, as well as decision curve analysis (DCA). The ROC and corresponding area under the curve (AUC) were utilized to quantify the discriminatory ability of the AKI nomogram. The AUC can be calculated by the integration of the area under the line segments, it ranges from 0.5 to 1.0, with 0.5 indicating a random and 1.0 indicating a perfectly differentiated. To evaluate the nomogram's identification and calibration, calibration curves were constructed. The Hosmer–Lemeshow test was performed to estimate the goodness-of-fit of the nomogram. To assess the nomogram model's clinical applicability and overall benefit, decision curve analysis (DCA) was utilized. DCA is an efficacious approach to the evaluation of the clinical benefits of alternative models, and when employed in nomograms, it can quantify the net benefits by performing at variable threshold probabilities. The DCA plotted the all-patient treatment scenario and the no-patient treatment scenario as two reference curves. The net benefit was calculated by deducting false-positive patients from true-positive patients, weighted by the potential damage of going untreated vs. the detrimental effects of going needless treatment. When the decision curve reveals that the nomogram is of greater benefit than the all-patient treatment scenario and the no-patient treatment scenario, it would indicate that the nomogram is clinically valid.

Nomogram-based risk-group stratification

Based on the nomogram, risk scores for AKI were calculated for each patient, and patients were then divided into high- and low-risk cohorts based on the optimal cutoff value determined by the Youden index from the ROC analysis of the training set. Differences in length of stay and last status (discharged or deceased) of patients in the high- and low-risk cohorts were compared in both training and validation sets, respectively.

Statistical analysis

The categorical variables were compared using Pearson's chi-square test and presented as percentages (%). For continuous variables, the Shapiro-Wilk test was used to test for normality, if the variables were normally distributed, the mean (standard deviation) was used for statistical description and the t-test was used for comparison between groups; otherwise, the medians [interquartile ranges (IQRs)] was used for statistical description and the Mann–Whitney U-test was used for comparison between groups. The median length of stay (m-LS) was calculated using the Kaplan-Meier approach, and the log-rank test was utilized to compare differences between high-risk and low-risk groups. The Cox proportional hazards model was used to determine the hazard ratio (HR) and its associated 95% confidence interval (CI). The mortality rate was compared by Fisher's exact or chi-squared tests, and odds ratios (ORs) with 95% CIs were calculated by logistic regression models. Statistical analysis was performed with the SPSS program (V22.0, Inc., Chicago, IL, USA) and R project (version 4.1.3, “glmnet” packages for LASSO logistic regression analysis, “forestplot” packages for plot forest, “hmisc” package for plot nomogram, “calibration curves” package for plot calibration curves, “pROC” package for plot ROC curves and calculate AUCs, and “stdca” package for DCA). A p-value < 0.05 (two-sided) was considered statistically significant.

Results

Characteristics of patients

In total, 480 patients with COVID-19-positive were included in this study; 336 were randomized into the training set, while the remaining 144 were randomized into the validation set. The flowchart of the study procedure was present in Figure 1. The baseline characteristics of the two sets of patients were essentially balanced. The incidence of AKI was 17.7% (85 of 480) in the overall population, 16.1% (54 of 336) in the training set, and 21.5% (31 of 144) in the validation set, respectively (Table 1). Supplementary Table 1 compares the baseline characteristics of patients with AKI and those without AKI. For the whole cohort, there were slightly more patients aged <60 years (256, 53.3%) than those aged ≥60 years (224, 46.7%). There were 300 male patients (62.5%), which was more than the number of female patients (180, 37.5%). The majority of patients presented with infectious and respiratory symptoms, of which 395 (82.3%) patients presented with fever, 402 (83.8%) with cough, and 370 (77.1%) with dyspnea, while there was a relatively low frequency of gastrointestinal symptoms. Of those 480 patients, 21 patients received kidney replacement therapy, and 14 received kidney transplants. For patients receiving kidney replacement therapy, the mortality rate was 52.4% (11 of 21) and the mean length of stay was 39 days (only for discharged patients). For patients receiving received a kidney transplant, the mortality rate was 0.00% with a mean length of stay was 14.6 days. The baseline information for patients, including laboratory indicators, is detailed in Table 1.

FIGURE 1

Figure 1. The flowchart of the study procedure. Abbreviations: LASSO, least absolute shrinkage and selection operator; ROC, receiver operating characteristic; DCA, decision curve analysis.

TABLE 1

Table 1. Baseline characteristics of patients in training set, validation set and all populations.

Feature selection

Feature selection was performed in the training set, and 41 parameters were included in the LASSO logistic regression analysis for predictor screening. The coefficient profile plot was produced against the log (λ) sequence (Figure 2A). Minimum criteria are used to select the tuning parameter (λ) for the LASSO regression utilizing 10-fold cross-validation. The results show that the optimal value of tuning parameter λ in the LASSO logistic regression was 0.026 when the mean-squared error reached its minimum value. Ten parameters with non-zero coefficients were screened: hypertension history, CAD, diabetes, SaO2, ALT, lactate, PCT, CRP, SCR, and GFR (Figure 2).

FIGURE 2

Figure 2. Feature selection using the least absolute shrinkage and selection operator (LASSO) Cox regression model. (A) LASSO coefficient profiles of the 41 features. (A) coefficient profile plot was produced against the log (λ) sequence. (B) Selection of tuning parameter (λ) in the LASSO regression using 10-fold cross-validation via minimum criteria. At the optimal values log (λ), where features are selected, two dotted vertical lines were drawn at the optimal scores by minimum criteria and 1-s.e. criteria.

Construction of the nomogram and performance examination

The parameters screened in the LASSO regression were utilized for the multivariate logistic regression model analysis, and the results demonstrated that only SaO2 and GFR were independent predictors of the occurrence of AKI (SaO2, OR:0.930, 95% CI: 0.881–0.982, P = 0.008; GFR, OR:0.973, 95% CI: 0.956–0.990, P = 0.002) (Figure 3). However, to avoid overfitting of the nomogram model, parameters with p < 0.1 were selected for model construction. Finally, a predictive nomogram model for the occurrence of AKI in hospitalized COVID-19 patients based on CAD, SaO2, PCT, CRP, and GFR was constructed (Figure 4, Supplementary Table 2). Based on the nomogram, the point scale scores for these five independent variables could be calculated for each patient, and their sum was the total point value. The ROC curve showed that the nomogram had favorable discrimination for AKI, with an AUC of 0.831 (95% CI: 0.774–0.889), a sensitivity of 85.2%, and a specificity of 69.9% (Figure 5A), which was significantly better than those of SCR and BUN (Supplementary Figure 1A). The calibration curves visually revealed favorable accordance between the prediction of the nomogram and the actual observations (Figure 5B). The Hosmer–Lemeshow test demonstrated a nice goodness-of-fit of the nomogram, with no significant differences observed (p = 0.247). DCA showed that the nomogram had a nice overall net benefit in the threshold probability range of 16–63%, and was superior to those of SCR and BUN (Supplementary Figure 1C), indicating that the model has promising clinical effectiveness (Figure 5C).

FIGURE 3

Figure 3. Results of multivariate logistic regression analysis in the training set. Factors with p-values < 0.1 were screened for constructing the nomogram model. Abbreviations: CAD, coronary artery disease; SaO2, artery oxygen saturation; ALT, alanine aminotransferase; PCT, procalcitonin; CRP, C-reactive protein; GFR, SCR, serum creatinine; GFR, glomerular filtration rate.

FIGURE 4

Figure 4. The nomogram was developed in the training set. It included five factors: glomerular filtration rate (GFR), artery oxygen saturation (SaO2), procalcitonin (PCT), C-reactive protein (CRP), and history of coronary artery disease (CAD). The nomogram plot provides a visual way to predict the risk of AKI for COVID-19 patients.

FIGURE 5

Figure 5. Validation of the discrimination power of the nomogram in the training and validation sets. (A,D) ROC curve analysis of the nomogram in the training and validation sets (AUC, 0.831 and 0.810, respectively); (B,E) Calibration plot of the nomogram in the training and validation sets, The black dashed diagonal line indicates the perfect prediction of the ideal model. The solid black line represents the performance of the nomogram, and the closer the fit to the diagonal line, the more accurate the prediction. The gray dashed line represents the performance of the model trained after bootstrapping validation (1,000 bootstrap resamples), which corrects the overfitting situation; (C,F) DCA analysis of the nomogram in the training and validation sets. The y-axis represents the net benefit, the x-axis represents the threshold probability. The red line represents the nomogram, and the blue and orange lines represent the all-patient treatment scenario and the no-patient treatment scenario, respectively. Abbreviations: ROC, receiver operating characteristic; AUC, area under the curve; DCA, decision curve analysis.

Validation of the nomogram

Next, we evaluated the effectiveness of the model in the validation set. Consistent with the results of the training set, the nomogram yielded a favorable AUC of the ROC curve of 0.810 (95% CI: 0.737–0.871), with a sensitivity of 77.4% and specificity of 78.8% (Figure 5D), better than those of SCR and BUN (Supplementary Figure 1B). The calibration curve and Hosmer–Lemeshow test suggested that the nomogram had good calibration and fit in the validation set (p = 0.247) (Figure 5E). Moreover, DCA visually revealed that the nomogram had an overall net benefit within a wider threshold probability in the validation set (Figure 5F, Supplementary Figure 1D). These results suggest that the nomogram functions well and has excellent predictive power for the validation set.

Nomogram-based risk stratification

Based on the nomogram constructed in the training set, the probability of occurrence of AKI was calculated for each patient. Using the optimal cutoff value of 0.122 obtained from the ROC analysis of the training set, the two sets of patients were subsequently classified into high- and low-risk groups. In the training set, there were 205 patients in the low-risk group and 131 patients in the high-risk group. The median length of stay (m-LS) for patients in the high-risk group was 14.0 days (95% CI: 11.3–16.7 days), which was significantly longer than 8.0 days (95% CI: 7.1–8.9 days) for patients in the low-risk group, which was (HR:1.98, 95%CI: 1.55–2.53, p < 0.001) (Figure 6A). Similarly, in the validation set, we also observed a significantly longer length of stay in the high-risk group than in the low-risk group (m-LS: 17.0 days vs. 8.0 days, HR: 2.10, 95% CI: 1.45–3.05, p < 0.001) (Figure 6B). In addition, we found that the mortality rate was higher in the high-risk group. In the training set, there were 27 patients in the high-risk group having a last status of death, with a mortality of 20.6%, compared to 2 (2.9%) in the low-risk group in the training set (OR: 8.61, 95% CI: 3.45–21.52, p < 0.001). The results from the validation set corroborated this finding, where the last status was deceased in 14 (23.3%) and 2 (2.4%) patients in each of the high- and low-risk groups, respectively, with statistically significant differences (OR: 12.48, 95% CI: 2.72–57.33, p = 0.001). These results indicate that the nomogram model can be applied to predict the prognosis of hospitalized COVID-19 patients (Table 2).

FIGURE 6

Figure 6. Kaplan-Meier curves for the length of stay of high-risk patients and low-risk patients based on the optimal segmentation threshold obtained from the ROC analysis in the training set (A) and the validation set (B). Abbreviations: m-LS: median length of stay.

TABLE 2

Table 2. Comparisons of last status and length of stay of patients on Nomogram-based risk stratification in training and validation set.

Discussion

In this study, we developed a predictive nomogram model for acute kidney injury in hospitalized COVID-19 patients based on SaO2, PCT, CRP, GFR, and the history of CAD. It is an easy-to-use, well-performing nomogram model with promising discrimination, predictive accuracy, and clinical practical utility. In addition, the risk score based on the nomogram was related to the length of stay and the last status of the patient. This is the first nomogram model to predict AKI in COVID-19 inpatients at an early stage. It is conducive to the early identification of high-risk patients with AKI to provide early intervention and treatment, and it can effectively and reasonably optimize the allocation and utilization of hospital beds as well as medical resources, thereby alleviating the shortage of medical resources and improving patient prognosis.

Acute kidney injury is a group of clinical syndromes characterized by a rapid decline (hours to days) in kidney function (19) and is a common complication in patients hospitalized with COVID-19 (8–10, 12, 13, 17). Patients may present with urinary abnormalities, for example, proteinuria and hematuria, elevated blood creatinine and urea nitrogen, and even positivity for SARS-CoV-2 in urine tests (1, 2, 10, 13). According to previous studies, ~40% of patients with COVID-19 had proteinuria on admission, ~10% had elevated blood creatinine during the course of the disease, ~21% had elevated blood urea nitrogen, ~43% had persistently elevated blood urea nitrogen, ~63% had proteinuria, and ~26.9% had hematuria (20, 21). The incidence of AKI varied by medical center, race, statistical size of the sample, and severity of disease in the included population. For example, According to a study of 138 Wuhan residents who were diagnosed with COVID-19, the incidence of AKI was 3.6%, while the incidence of AKI in critically ill patients was 8.3% (2). Of 1,099 patients with COVID-19, Guan et al. (4) reported an incidence of AKI of 0.5%, with 5 of 17 (2.9%) critically ill patients developing AKI. In addition, another Chinese study enrolled 710 patients with COVID-19, 52 of whom were critically ill adults, with an AKI rate of 29% (21). An Italian study showed a 15% incidence of AKI in COVID-19 patients (22), and the results of another study of 5,700 patients with COVID-19 in New York reported an AKI incidence reaching as high as 22.2% (23). Altogether, the incidence of AKI in patients with COVID-19 ranges from 0.1 to 56.9%, while it reaches 77% in patients with severe COVID-19 (2, 4, 11, 21–23). Analyzing the previous evidence, the following information can be obtained. First, kidney injury is not uncommon in patients with COVID-19 (especially those with severe disease). Second, the incidence of AKI has been inconsistent, which is mainly related to the sample size and study population, and the incidence of AKI is higher in severe and critical COVID-19 patients. Most importantly, COVID-19 complicated with AKI is an independent risk factor for poor prognosis (11, 21–23). Among patients who died from COVID-19, the incidence of AKI was as high as 37.5%, which was significantly higher than the 15% of surviving cases (24). The mortality rate of COVID-19 patients complicated with AKI was reported to be 67% (95% CI: 39.8–86.2%), and the risk of death was 13 times that of patients without AKI (OR = 13.3, 95% CI: 6.1–29.2) (25). In addition, the severity of AKI is associated with patient prognosis, and patients with the late-stage disease have a significantly higher risk of death (2, 8, 10, 12). Therefore, early assessment of AKI risk is important to guide physicians to intervene early and prevent AKI, protect renal function, and avoid progression of the patient's condition.

A nomogram may provide a quantitative and pragmatic predictive tool for risk stratification of COVID-19 patients for the development of AKI during hospitalization. In this study, GFR, SaO2, PCT, CRP, and history of CAD were selected by LASSO and multivariate regression for the construction of the predictive nomogram model of AKI. These factors have been demonstrated to correlate with the development of AKI in previous studies. An early Chinese study including 701 patients with COVID-19 showed that patients were more likely to develop AKI if their admission baseline SCR levels were higher (11.9 vs. 4%) (21). Data from a retrospective study of 306 patients from Sweden demonstrated that decreased baseline renal function increases the risk of developing AKI during hospitalization in COVID-19 patients. The risk ratios for experiencing AKI in patients with an eGFR between 30 and 59 ml/min and an eGFR <30 ml/min were 2.94 (95% CI: 1.17–7.34) and 9.93 (95% CI: 2.32–42.5), respectively (26). An international multicenter study of 939 patients identified that poor respiratory function (lower oxygen saturation and PaO2/FiO2 ratio) was a risk factor for the development of AKI (27). In addition, studies have shown that inflammatory indicators, such as C-reactive protein (27) and procalcitonin (28); underlying diseases, such as hypertension, diabetes, and chronic kidney disease; as well as coronary artery disease, are correlated with the occurrence of AKI (29, 30). There are also predictive models (scores or biomarkers) for AKI in COVID-19 patients that have been reported in previous studies. Gustavo et al. (31) investigated the capability of urinary kidney stress biomarkers (UKSB), including neutrophil gelatinase-associated lipocalin (NGAL) and tissue inhibitor of metalloproteinases-2 (TIMP-2) multiplied by insulin-like growth factor binding protein 7 (IGFBP7), for the early detection of AKI in 51 critically ill COVID-19 patients. The results showed that the AUCs of the ROC for NGAL and TIMP-2 × IGFBP7 predicting the occurrence of AKI during the entire hospitalization of patients were 0.706 (95% CI: 0.559–0.854) and 0.682 (95% CI: 0.535–0.829), respectively, with corresponding sensitivities and specificities of 54.5, 76.9, 40.0, and 88.4%. Naomi et al. (32) reported the application of serum biomarkers (SB), including serum NGAL and serum creatinine, for the prediction of AKI in 52 COVID-19 patients, with AUCs of 0.81 and 0.87, respectively. A prediction model for AKI based on proteinuria and hematuria yielded an AUC of 0.64 (95% CI: 0.62–0.67) in a large cohort study containing 5,980 COVID-19 patients; moreover, the predictive capability of the model was improved when creatinine and the presence of CKD were incorporated (33). In addition, studies have validated the predictive value of other indicators, such as D-dimer and albumin/creatinine ratio, for AKI in hospitalized COVID-19 patients (34). However, these predictive models (biomarkers) have some disadvantages. First and foremost, their performance has not been validated, which leads to a lack of confidence in their reproducibility and utility. Second, they focused only on a particular type or class of indicators, which may present only a partial characterization of the patient's disease. Third, some models were constructed based on small sample sizes, such as the UKSB model and the SB model, which may not be representative of the whole cohort population. In contrast, in the present study, we screened indicators on the basis of seven dimensions of admission information for the construction of a predictive nomogram model of AKI during hospitalization based on a training set with 366 COVID-19 patients. Importantly, we verified the effectiveness of the nomogram in both training and validation cohorts and found that the nomogram displayed promising identification, goodness-of-fit, discriminative power, and clinical effectiveness.

We also investigated the predictive value of the model for patient prognosis. The mortality rate of patients in the high-risk group was higher than that of patients in the low-risk group; furthermore, the mortality rate of high-risk patients was greater than that of low-risk patients, with a hazard ratio of 1.98 for the training set and 2.10 for the validation set. This may be because high-risk patients are susceptible to the occurrence of AKI, while previous studies have demonstrated a higher mortality rate in COVID-19 patients with AKI (35). Regardless, the nomogram model could identify patients with a potentially poor prognosis at the beginning of their admission, which is helpful for the formulation of individualized treatment strategies and the arrangement of appropriate care and treatment at an early stage.

The study has several limitations. First, it is a public database-based study, and the results may have been influenced by confounding factors beyond our control. Second, although the model performed well in the validation set, we did not evaluate its performance in an independent external validation cohort. Third, the length of stay and last status of patients are influenced by other factors, especially treatment measures; therefore, relying on the model alone to predict prognosis is not sufficient. In addition, some other indicators, such as virus load or virus-related indicators, red blood cell count, and hemoglobin concentration, may have some correlation with the occurrence of AKI; however, we were unable to investigate these further due to the limitations of data availability, which may affect the reliability and stability of our conclusions. Moreover, due to the limitations of data availability, the difference in special treatment during hospitalization, such as hemodialysis technology, between high-risk and low-risk groups was not analyzed. Hence, the predictive capability for AKI and the prognostic value of the model needs to be verified in actual clinical practice.

Conclusion

We constructed a nomogram for the early prediction of AKI in hospitalized COVID-19 patients. The model demonstrated favorable performance on the basis of the AUCs of ROC, calibration curves, and decision curve analysis. Furthermore, the nomogram exhibited promising predictive values for a prognosis for the length of stay and last status of patients. The nomogram model is helpful to reasonably and effectively optimize the allocation and utilization of medical resources at an early stage to provide appropriate care and intervention management for patients, thereby improving prognosis and reducing mortality.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by the Ethics Committee of Yantai Yuhuangding Hospital. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author contributions

Methodology: CW, XL, and DW. Software: XC, SZ, and CW. Formal analysis: CW, HS, and XC. Resources: TJ and XL. Data curation: HS and SZ. Writing—original draft preparation: CW and HS. Writing—review and editing, supervision, and project administration: CL and TJ. All authors contributed to the article and approved the submitted version.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2022.1047073/full#supplementary-material

Supplementary Figure 1. Comparison of AUCs between Nomogram with SCR and BUN in the training set (A) and validation set (B). Comparison of DCAs between Nomogram with SCR and BUN in the training set (C) and validation set (D). Abbreviations: ROC, receiver operating characteristic; AUC, area under the curve; DCA, decision curve analysis; SCR, serum creatinine; BUN, blood urea nitrogen.

Supplementary Table 1. Comparison of characteristics between patients with AKI and non-AKI.

Supplementary Table 2. The parameters of the final multifactor model for AKI.

References

1. Lu R, Zhao X, Li J, Niu P, Yang B, Wu H, et al. Genomic characterisation and epidemiology of 2019 novel coronavirus: implications for virus origins and receptor binding. Lancet. (2020) 395:565–74. doi: 10.1016/S0140-6736(20)30251-8

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Wang D, Hu B, Hu C, Zhu F, Liu X, Zhang J, et al. Clinical characteristics of 138 hospitalized patients with 2019 novel coronavirus-infected pneumonia in Wuhan, China. JAMA. (2020) 323:1061–9. doi: 10.1001/jama.2020.1585

PubMed Abstract | CrossRef Full Text | Google Scholar

3. COVID-19 coronavirus pandemic. Available online at: https://wwwworldometersinfo/coronavirus/ (accessed March 30, 2022).

4. Guan WJ Ni ZY, Hu Y, Liang WH, Ou CQ, He JX, et al. Clinical characteristics of coronavirus disease 2019 in China. N Engl J Med. (2020) 382:1708–20. doi: 10.1056/NEJMoa2002032

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Zhu N, Zhang D, Wang W, Li X, Yang B, Song J, et al. A novel coronavirus from patients with pneumonia in China, 2019. N Engl J Med. (2020) 382:727–33. doi: 10.1056/NEJMoa2001017

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Berlin DA, Gulick RM, Martinez FJ. Severe Covid-19. N Engl J Med. (2020) 383:2451–60. doi: 10.1056/NEJMcp2009575

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Gandhi RT, Lynch JB. C Del Rio. Mild or moderate Covid-19. N Engl J Med. (2020) 383:1757–66. doi: 10.1056/NEJMcp2009249

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Ronco C, Reis T. F Husain-Syed. Management of acute kidney injury in patients with COVID-19. Lancet Respirat Med. (2020) 8:738–42. doi: 10.1016/S2213-2600(20)30229-0

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Legrand M, Bell S, Forni L, Joannidis M, Koyner JL, Liu K, et al. Pathophysiology of COVID-19-associated acute kidney injury. Nat Rev Nephrol. (2021) 17:751–64. doi: 10.1038/s41581-021-00452-0

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Glowacka M, Lipka S, Mlynarska E, Franczyk B, J Rysz. Acute kidney injury in COVID-19. Int J Mol Sci. (2021) 22:8081. doi: 10.3390/ijms22158081

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Robbins-Juarez SY, Qian L, King KL, Stevens JS, Husain SA, Radhakrishnan J, et al. Outcomes for patients with COVID-19 and acute kidney injury: a systematic review and meta-analysis. Kidney Int Rep. (2020) 5:1149–60. doi: 10.1016/j.ekir.2020.06.013

PubMed Abstract | CrossRef Full Text | Google Scholar

12. RSantos PD, TLordani VA, LPeres AB, Carvalho A. Occurrence of acute kidney injury in adult patients hospitalized with COVID-19: a systematic review and meta-analysis. Nefrologia. (2021) 42:404–14. doi: 10.1016/j.nefro.2021.09.002

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Teixeira JP, Barone S, Zahedi K, M Soleimani. Kidney injury in COVID-19: epidemiology, molecular mechanisms and potential therapeutic targets. Int J Mol Sci. (2022) 23:2242. doi: 10.3390/ijms23042242

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Santoriello D, Khairallah P, Bomback AS, Xu K, Kudose S, Batal I, et al. Postmortem kidney pathology findings in patients with COVID-19. J Am Soc Nephrol. (2020) 31:2158–67. doi: 10.1681/ASN.2020050744

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Hirsch JS, Ng JH, Ross DW, Sharma P, Shah HH, Barnett RL, et al. Acute kidney injury in patients hospitalized with COVID-19. Kidney Int. (2020) 98:209–18. doi: 10.1016/j.kint.2020.05.006

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Bowe B, Cai M, Xie Y, Gibson AK, Maddukuri G, Al-Aly Z. Acute kidney injury in a national cohort of hospitalized US veterans with COVID-19. Clin J Am Soc Nephrol. (2020) 16:14–25. doi: 10.2215/CJN.09610620

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Gabarre P, Dumas G, Dupont T, Darmon M, Azoulay E, Zafrani L. Acute kidney injury in critically ill patients with COVID-19. Intensive Care Med. (2020) 46:1339–48. doi: 10.1007/s00134-020-06153-9

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Gupta S, Coca SG, Chan L, Melamed ML, Brenner SK, Hayek SS, et al. AKI treated with renal replacement therapy in critically Ill patients with COVID-19. J Am Soc Nephrol. (2021) 32:161–76. doi: 10.1681/ASN.2020060897

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Ronco C, Bellomo R, Kellum JA. Acute kidney injury. Lancet. (2019) 394:1949–64. doi: 10.1016/S0140-6736(19)32563-2

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Patel SK, Singh R, Rana J, Tiwari R, Natesan S, Harapan H, et al. The kidney and COVID-19 patients—important considerations. Travel Med Infect Dis. (2020) 37:101831. doi: 10.1016/j.tmaid.2020.101831

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Cheng Y, Luo R, Wang K, Zhang M, Wang Z, Dong L, et al. Kidney disease is associated with in-hospital death of patients with COVID-19. Kidney Int. (2020) 97:829–38. doi: 10.1016/j.kint.2020.03.005

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Cecconi M, Piovani D, Brunetta E, Aghemo A, Greco M, Ciccarelli M, et al. Early predictors of clinical deterioration in a cohort of 239 patients hospitalized for Covid-19 infection in lombardy, Italy. J Clin Med. (2020) 9:1548. doi: 10.3390/jcm9051548

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Cummings MJ, Baldwin MR, Abrams D, Jacobson SD, Meyer BJ, Balough EM, et al. Epidemiology, clinical course, and outcomes of critically ill adults with COVID-19 in New York City: a prospective cohort study. Lancet. (2020) 395:1763–70. doi: 10.1016/S0140-6736(20)31189-2

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Yang X, Yu Y, Xu J, Shu H, Ja Xia, Liu H, et al. Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China: a single-centered, retrospective, observational study. Lancet Respirat Med. (2020) 8:475–81. doi: 10.1016/S2213-2600(20)30079-5

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Oliveira CB, CLima AD, Vajgel G, Campos Coelho AV, Sandrin-Garcia P. High burden of acute kidney injury in COVID-19 pandemic: systematic review and meta-analysis. J Clin Pathol. (2021) 74:796–803. doi: 10.1136/jclinpath-2020-207023

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Xu H, Garcia-Ptacek S, Annetorp M, Bruchfeld A, Cederholm T, Johnson P, et al. Acute kidney injury and mortality risk in older adults with COVID-19. J Nephrol. (2021) 34:295–304. doi: 10.1007/s40620-021-01022-0

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Morieri ML, Ronco C, Avogaro A, Farnia F, Shestakova M, Zaytseva N, et al. In hospital risk factors for acute kidney injury and its burden in patients with SARS-CoV-2 infection: a longitudinal multinational study. Sci Rep. (2022) 12:3474. doi: 10.1038/s41598-022-07490-z

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Rahimzadeh H, Kazemian S, Rahbar M, Farrokhpour H, Montazeri M, Kafan S, et al. The risk factors and clinical outcomes associated with acute kidney injury in patients with COVID-19: data from a large cohort in iran. Kidney Blood Press Res. (2021) 46:620–8. doi: 10.1159/000517581

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Huang C, Wang Y, Li X, Ren L, Zhao J, Hu Y, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. Lancet. (2020) 395:497–506. doi: 10.1016/S0140-6736(20)30183-5

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Dereli N, Babayigit M, Menteş O, Koç F, Ari O, Dogan E, et al. Are we aware of COVID-19-related acute kidney injury in intensive care units? European review for medical and pharmacological. Sciences. (2022) 26:1753–60. doi: 10.26355/eurrev_202203_28245

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Casas-Aparicio G, Alvarado-de la Barrera C, Escamilla-Illescas D, Leon-Rodriguez I, Del Rio-Estrada PM, Calderon-Davila N, et al. Role of urinary kidney stress biomarkers for early recognition of subclinical acute kidney injury in critically Ill COVID-19 patients. Biomolecules. (2022) 12:275. doi: 10.3390/biom12020275

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Pode Shakked MHSdeO, Cheruiyot I, Benoit JL, Plebani M, Lippi G, et al. Early prediction of COVID-19-associated acute kidney injury: are serum NGAL and serum Cystatin C levels better than serum creatinine? Clin Biochem. (2022) 102:1–8. doi: 10.1016/j.clinbiochem.2022.01.006

PubMed Abstract | CrossRef Full Text | Google Scholar

33. McAdams MC Li M, Xu P, Gregg LP, Patel J, Willett DL, et al. Using dipstick urinalysis to predict the development of acute kidney injury in patients with COVID-19. BMC Nephrol. (2022) 23:50. doi: 10.1186/s12882-022-02677-y

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Yildirim C, Ozger HS, Yasar E, Tombul N, Gulbahar O, Yildiz M, et al. Early predictors of acute kidney injury in COVID-19 patients. Nephrology. (2021) 26:513–21. doi: 10.1111/nep.13856

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Ng JH, Hirsch JS, Hazzan A, Wanchoo R, Shah HH, Malieckal DA, et al. Outcomes among patients hospitalized with COVID-19 and acute kidney injury. Am J Kidney Dis. (2021) 77:204–15e1. doi: 10.1053/j.ajkd.2020.09.002

PubMed Abstract | CrossRef Full Text | Google Scholar