Predicting the gastrointestinal bleeding of HBV-related acute-on-chronic liver failure based on machine learning

Background: This study aimed to investigate the effect of gastrointestinal bleeding (GIB) on the short-term survival of hepatitis B virus-related acute-on-chronic liver failure (HBV-ACLF) patients, establish a prediction model for HBV-ACLF-related GIB via machine learning (ML) algorithms, and compare the predictive ability of various models.

Methods: A total of 583 HBV-ACLF patients from two medical centers were retrospectively enrolled, and patients from one of the centers were randomly divided into a training cohort (n = 360) and a test cohort (n = 153) at a 7:3 ratio. Patients from the other center composed the validation cohort (n = 70). Patients were divided into GIB and non-gastrointestinal bleeding (NGIB) groups according to whether they had GIB during hospitalization, and short-term survival rates were compared between the two groups. Least absolute shrinkage and selection operator (LASSO) regression was used to screen for features associated with GIB. On the basis of the screened features, we used five ML algorithms, namely, logistic regression (LR), support vector machine (SVM), decision tree (DT), random forest (RF), and K-nearest neighbors (KNN), to build a prediction model for GIB. Six metrics, namely, accuracy, area under the curve (AUC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), were used to evaluate the predictive ability of these models.

Results: In the training cohort, patients in the GIB group had significantly lower 30- and 90-day survival rates than did those in the NGIB group (48.72% versus 85.67% and 10.26% versus 64.80%, respectively), and similar results were obtained in the test cohort and the validation cohort. LASSO regression screened seven features associated with GIB, of which portal hypertension, electrolyte disturbance, and white blood cell counts were modeled features common to the five machine prediction models. The AUCs of the LR, SVM, DT, RF, and KNN models in the training cohort were 0.819, 0.924, 0.661, 1.000, and 0.865, respectively. Compared with the other four models, the LR model had the lowest PPV of 0.202 in the test cohort, the SVM model had the lowest AUC and sensitivity of 0.657 and 0.500 in the validation cohort, the DT model had the lowest sensitivity of 0.436 and 0.438 in the training and test cohorts, respectively, and the KNN model had the lowest PPV of 0.250 in the validation cohort. Notably, the RF model had the least fluctuations in accuracy, AUC, sensitivity, specificity, PPV, and NPV among the 3 cohorts, with good overall predictive ability.

Conclusion: GIB has a significant effect on short-term survival in patients with HBV-ACLF. On this basis, five ML prediction models, LR, SVM, DT, RF, and KNN, were established to have better prediction ability for GIB, among which the RF model has the most robust prediction performance, which can help clinicians intervene in advance and improve the short-term survival rate of patients.

1 Introduction

Acute-on-chronic liver failure (ACLF) is a severe clinical syndrome resulting from acute exacerbation of chronic liver disease and is characterized by a dramatic decline in liver function, multi-organ failure, and a high mortality rate (1). Hepatitis B virus (HBV) infection is the leading cause of ACLF in Asian populations, accounting for approximately 70% of the cases (2). Hepatitis B virus-related acute–chronic liver failure (HBV-ACLF) has a high mortality rate of approximately 54.3% (3). Various complications, such as gastrointestinal bleeding (GIB), hepatic encephalopathy, and hepatorenal syndrome, are some of the main reasons for the high mortality rate of HBV-ACLF. However, there is still some controversy as to whether GIB affects the survival of ACLF patients (4).

GIB is a relatively common complication of HBV-ACLF (5). The prevalence of GIB in patients with HBV-ACLF is approximately 13.86% (6). A study from China reported that upper GIB has no significant effect on short-term survival in patients with ACLF (7). However, another study found that among ACLF patients with complicated variceal bleeding, the short-term risk of death increased (8). Guo et al. compared the prognostic impact of combined bleeding in patients with HBV-ACLF and reported that the survival rate in the bleeding group (51.85%) was lower than that in the nonbleeding group (78.33%) (6). Suppose the effect of GIB on the survival of HBV-ACLF patients can be clarified. In these cases, the related risk factors for its occurrence can be identified, and GIB events can be predicted in advance; thus, intervention can effectively reduce the mortality rate of HBV-ACLF patients. Currently, there are few studies on the prediction model of GIB occurrence in HBV-ACLF patients, and only a few related studies (6, 7, 9), such as those regarding the prediction model of esophagogastric variceal bleeding in patients with chronic liver failure, the prediction model of bleeding in HBV-ACLF patients, and upper GIB in patients with ACLF, have the problems of fewer research subjects and no external data validation, which make it difficult to accurately predict the occurrence of GIB events in patients with HBV-ACLF. Therefore, a new model that can predict the occurrence of GIB in patients with HBV-ACLF is urgently needed.

Machine learning (ML) has considerable advantages in diagnosing and predicting diseases. With the rapid, recent development of artificial intelligence, its integration with clinical practice has become increasingly intense, and has also been widely used in the field of liver failure (10). For example, a support vector machine (SVM) was used to construct a 28-day prognostic model for ACLF (11), predict the prognosis of liver transplantation for ACLF via a decision tree (DT) model (12), predict the prognostic status of acetaminophen-induced acute liver failure via a random forest (RF) model (13), construct a predictive model for liver failure after hepatic cancer resection using logistic regression (LR) (14), and apply the k-nearest neighbors (KNN) algorithm to construct a biomolecular mapping model of albumin to identify acute liver failure severity (15). Although the development of ML algorithms has significantly improved the accuracy of predictive models in diagnosing liver failure and improving survival prognosis, there are differences in predictive efficacy when different ML algorithms are used for modeling (16). Therefore, choosing the appropriate ML predictive model requires further comparative analysis.

This study aimed to investigate the effect of GIB on the survival of patients with HBV-ACLF. Additionally, for the first time, ML algorithms were used to construct a prediction model for concurrent GIB in HBV-ACLF patients, and the model was validated with internal and external data. Interfering in advance for HBV-ACLF patients at high risk of GIB, reducing the mortality of ACLF patients, and providing a theoretical basis for clinical work are needed.

2 Materials and methods

2.1 Study population and design

Data on 583 HBV-ACLF patients hospitalized at the Department of Infection of the First Affiliated Hospital of Nanchang University and Jiangxi Provincial People’s Hospital from 2014 to 2021 were retrospectively collected. Among them, 513 patients from the First Affiliated Hospital of Nanchang University were randomly divided into a training cohort (n = 360) and a test cohort (n = 153) at a ratio of 7:3. Additionally, 70 patients from Jiangxi Provincial People’s Hospital were used as a validation cohort to externally validate the model. The patients were divided into a GIB group and an NGIB group according to whether they had GIB. The study flow is detailed in Figure 1. Inclusion criteria were as follows: (1) patients were collected on the basis of the diagnostic criteria of the Asian Pacific Association for the Study of the Liver (17): serum bilirubin ≥5 mg/dL, international normalized ratio (INR) ≥ 1.5 or prothrombin activity (PTA) < 40% based on chronic liver disease or cirrhosis and ascites or hepatic encephalopathy within 4 weeks; and (2) serum HBV surface antigen (HBsAg) positivity lasting more than 6 months. The exclusion criteria were as follows: (1) infection with other hepatotropic viruses (ex. Hepatitis A, C, E); (2) combined malignant tumor of the liver; (3) other severe diseases, such as respiratory/heart failure or end-stage renal disease; and (4) missing data. The Medical Research Ethics Board of the First Affiliated Hospital of Nanchang University, China, waived informed consent in the main manuscript (Ethics Board approval number: (IIT [2021]100)). All research methods were conducted in accordance with the principles of the Declaration of Helsinki.

Figure 1

Figure 1. Flowchart of the study.

2.2 Data collection

The clinical demographic records (sex, age, weight, etc.), past medical history (hypertension, diabetes, cirrhosis), laboratory tests (routine blood tests, liver and kidney function, glucose, lipids, coagulation indicators, etc.), complications before GIB (hepatic encephalopathy, hepatorenal syndrome, ascites, spontaneous bacterial peritonitis, electrolyte disturbance), radiographic examinations, treatment programs, and 30- and 90-day survival rates were collected. GIB was diagnosed on the basis of the presence of vomited blood or dark stools combined with a positive occult blood test during the patient’s hospitalization.

Artificial liver refers to the use of medical devices or technologies to simulate or replace part of the liver’s function, alleviating clinical symptoms in patients with liver failure or severe liver diseases. Common methods of artificial liver treatment include Hemodialysis, Plasma Exchange/Adsorption, and Biological Artificial Liver. Currently, artificial liver therapy is mainly used in the treatment of various types of liver failure, such as acute liver failure, ACLF, and liver dysfunction (18).

2.3 SVM

SVM is a classical algorithm for classification that is used to address binary or multiple classification problems. The essence is to find an optimal hyperplane in the feature space that allows classification to be completed while keeping the interval maximum. Small sample data are handled better, but too many variables affect its classification efficiency (19). The “e1071” R package was downloaded for constructing SVM prediction models in the R language.

2.4 DT and RF

A DT is a classification and regression method named for its rules and decisions that resemble the trunks and branches of a tree. The predictor variable in a DT acts as the root node, the prediction outcome acts as a leaf node, and the path connecting the two is the decision rule. The algorithm finds the optimal variables and combinations to classify the data correctly (20). RF is a collection of DTs that improves the robustness of the model and reduces overfitting situations, resulting in better predictive performance (21). We use the “part” and “randomForest” packages in R to construct the above two models.

2.5 KNN

KNN is a simple classification algorithm that works by calculating the distance of the variable to be classified from all neighboring training points and determining k loci, which allows the variable to be classified into a class of proximity (22). A diagnostic model for GIB was constructed using the “knn” package.

2.6 LR

Multivariate logistic regression analysis was performed via SPSS. The screened independent risk factors were used to construct an LR diagnostic model of the training cohort via a forward stepwise regression approach (23).

2.7 Statistical analysis

Statistical analysis was performed with R (version 4.13), SPSS (version 26.0), and GraphPad Prism (version 8.0). Count data are presented as frequencies, and comparisons between groups were made via the chi-square test. The measurement data were first tested for a normal distribution. Normally distributed measurement data are described by X ± S (mean standard deviation), two independent samples t tests were used for comparisons between two groups, and one-way ANOVA was used for comparisons between multiple groups. Non-normally distributed measurement data are described by M (P25–P75) (median, upper and lower quartiles), the Mann–Whitney U test was used for comparisons between two groups, and the Kruskal–Wallis H rank sum test was used for comparisons between multiple groups. The possible risk factors for GIB were subjected to LASSO regression analysis, and the filtered features were further put into a predictive model via five ML algorithms, namely, multifactorial LR, SVM, DT, RF, and KNN. The model was visualized with a nomogram. Predictive models were evaluated in terms of accuracy, area under the curve (AUC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Finally, we performed internal and external validation of the constructed model. p < 0.05 indicated a statistically significant difference.

2.8 Ethics approval and informed consent

The study was approved by the Ethics Committee of the First Affiliated Hospital of Nanchang University (IIT [2021]100). As this study was retrospective, the Ethics Committee of the First Affiliated Hospital of Nanchang University waived the informed consent for this study. All research methods were conducted in accordance with the principles of the Declaration of Helsinki.

3 Results

3.1 Clinical characteristics

Among the 583 patients with HBV-ACLF, 63 had GIB, with a prevalence of 10.8%. Among them, 513 patients at the First Affiliated Hospital of Nanchang University were randomly divided into a training cohort (n = 360) and a test cohort (n = 153) at a ratio of 7:3. The training cohort included 311 males and 49 females, with a mean age of 44.66 ± 12.06 years, and 39 patients with GIB. The test cohort included 131 males and 22 females with a mean age of 44.48 ± 11.98 years and 16 patients with GIB. Seventy patients from Jiangxi Provincial People’s Hospital served as the validation cohort, including 59 males and 11 females, with a mean age of 44.54 ± 13.27 years, and 8 patients with GIB. A comparison of the clinical characteristics of the 3 cohorts of patients revealed no statistically significant differences (Table 1), which confirmed the reliability of the results of the test and validation cohorts.

Table 1

Table 1. HBV-ACLF patient characteristics in the training, test, and validation cohorts.

3.2 Effect of GIB on short-term survival in HBV-ACLF patients

We compared the 30- and 90-day survival rates between the GIB group and the NGIB group in the 3 cohorts (Figures 2A–F). In the training cohort, the 30-day survival rate in the GIB group versus the NGIB group was 48.72% versus 85.67%, p < 0.001, and the 90-day survival rate was 10.26% versus 64.80%, p < 0.001. In the test cohort, the 30-day survival rate in the GIB group versus the NGIB group was 68.75% versus 86.86%, p = 0.031, and the 90-day survival rate was 0% versus 67.15%, p < 0.001, respectively. In the validation cohort, the 30-day survival rate in the GIB group versus the NGIB group was 25% versus 67.74%, p = 0.002, and the 90-day survival rate was 25% versus 51.61%, p = 0.014, respectively. In all 3 cohorts, patients in the GIB group had significantly lower 30- and 90-day survival rates than those in the NGIB group did.

Figure 2

Figure 2. The 30-day and 90-day survival of patients with HBV-ACLF in the GIB group versus the non-GIB group. The 30-day survival curve for the (A) training group, (B) test group, and (C) validation group. The 90-day survival curve for the (D) training group, (E) test group and (F) validation group.

3.3 Selection of clinical characteristics

A total of 32 characteristics of HBV-ACLF patients were collected, and LASSO regression analysis was performed to analyze the above characteristics of 513 patients at the First Affiliated Hospital of Nanchang University. The larger the value of λ is, the closer the feature coefficients are to 0, whereas the bias percentage tends to decrease and then increase, with an optimal λ-se of 7 (Figures 3A,B). Seven clinical features associated with GIB, including white blood cell count, cholesterol, number of artificial liver treatments, portal hypertension, spontaneous bacterial peritonitis, hepatorenal syndrome, and electrolyte disturbance, were selected on the basis of the optimal λ value.

Figure 3

Figure 3. Feature screening and LR regression modeling. (A) Relationships between the lambda values and feature coefficients; (B) relationships between the lambda values and bias percentages of the features; (C) training cohort-based nomogram; (D) training cohort ROC curve; (E) test cohort ROC curve; (F) validation cohort ROC curve; (G) training cohort fitting curve; (H) test cohort fitting curve; (I) validation cohort fitting curve; (J) training cohort calibration curve; (K) test cohort calibration curve; (L) validation cohort calibration curve.

3.4 LR-based predictive model

The seven characteristics selected by LASSO regression were included in a multivariate logistic regression analysis, which revealed that portal hypertension (OR = 2.947, 95% CI 1.270–6.839, p = 0.012), spontaneous bacterial peritonitis (OR = 5.975, 95% CI 1.708–20.904, p = 0.005), electrolyte disturbance (OR = 3.300, 95% CI 1.510–7.210, p = 0.003) and white blood cell count (OR = 1.095, 95% CI 1.013–1.184, p = 0.023) were four independent risk factors for GIB (Table 2). A prediction model was constructed on the basis of the above four characteristics: Logit(P) = ln[P/(1 − P)] = 0.993 × [Portal hypertension (0 No, 1 Yes)] + 2.025 × [Spontaneous bacterial peritonitis (0 No, 1 Yes)] + 1.205 × [Electrolyte disturbance (0 No, 1 Yes)] + 0.108 × [White blood cell count (×10⁹/L)]-5.414. To visualize the model, we simultaneously constructed a nomogram (Figure 3C). In the training cohort, the AUC value of the model was 0.819, and the sensitivity, specificity, PPV, and NPV were 0.692, 0.801, 0.300, and 0.955, respectively (Figure 3D). In the test cohort, the AUC of the model was 0.829, and the sensitivity, specificity, PPV, and NPV were 1.000, 0.540, 0.202, and 1.000, respectively (Figure 3E). In the validation cohort, the AUC of the model was 0.817, and the sensitivity, specificity, PPV, and NPV were 0.625, 0.952, 0.600, and 0.952, respectively (Figure 3F). In the training cohort, the calibration curves ideally and realistically fit better than those in the test and validation cohorts did (Figures 3G–I). All three cohort decision curves showed a better clinical net benefit of the model (Figures 3J–L).

Table 2

Table 2. Multivariate logistic regression analysis of patient characteristics associated with increased gastrointestinal bleeding risk.

3.5 SVM-based predictive model

The GIB model constructed by SVM had the highest accuracy when the number of vectors was 4 (Figure 4A). In the training cohort, the AUC value of the model was 0.924, and the sensitivity, specificity, PPV, and NPV were 0.769, 0.950, 0.652, and 0.971, respectively (Figure 4B). In the test cohort, the AUC of the model was 0.807, and the sensitivity, specificity, PPV, and NPV were 0.938, 0.606, 0.217, and 0.895, respectively (Figure 4C). In the validation cohort, the AUC of the model was 0.657, and the sensitivity, specificity, PPV, and NPV were 0.500, 0.903, 0.400, and 0.933, respectively (Figure 4D).

Figure 4

Figure 4. SVM prediction model. (A) Number of vectors and model accuracy of the SVM; (B) training cohort ROC curve; (C) test cohort ROC curve; (D) validation cohort ROC curve.

3.6 DT-based predictive model

In the DT model, we identified the 3 best split nodes (Figure 5A). The model characteristics, in descending order of importance, were portal hypertension, white blood cell count, electrolyte disturbance, number of artificial liver treatments, and spontaneous bacterial peritonitis (Figure 5B). On this basis, the top 3 most important features were selected to construct the prediction model (Figure 5C). In the training cohort, the AUC of the model was 0.661, and the sensitivity, specificity, PPV, and NPV were 0.436, 0.86, 0.274, and 0.926, respectively (Figure 5D). In the test cohort, the AUC of the model was 0.698, and the sensitivity, specificity, PPV, and NPV were 0.438, 0.978, 0.700, and 0.937, respectively (Figure 5E). In the validation cohort, the AUC value of the model was 0.777, and the sensitivity, specificity, PPV, and NPV were 0.625, 0.919, 0.500, and 0.950, respectively (Figure 5F).

Figure 5

Figure 5. DT predictive model. (A) Relationships between the number of splitting points of DTs and complexity parameters; (B) the significance of clinical features and radiomic features for the DT-based predictive model; (C) decision tree diagram; (D) training cohort ROC curve; (E) test cohort ROC curve; (F) validation cohort ROC curve.

3.7 RF-based predictive model

In the RF prediction model, the out-of-bag (OOB) error reached a minimum value of 0.122 when the number of trees in the model was 37 (Figure 6A). Furthermore, the error value was minimized to 0.328 when the split node of the tree was 7 (Figure 6B). We then analyzed the importance of the 7 features included, and the top 3 features that improved model accuracy and reduced the Gini coefficient of the model were portal hypertension, electrolyte disturbance, and hepatorenal syndrome, as well as white blood cell count, total cholesterol, and number of artificial liver treatments, respectively (Figure 6C). In the training cohort, the AUC of the model was 1.000, and the sensitivity, specificity, PPV, and NPV were 1.000, 1.000, 1.000, and 1.000, respectively (Figure 6D). In the test cohort, the AUC of the model was 0.823, and the sensitivity, specificity, PPV, and NPV were 0.625, 0.912, 0.455, and 0.954, respectively (Figure 6E). In the validation cohort, the AUC value of the model was 0.803, and the sensitivity, specificity, PPV, and NPV were 0.625, 0.903, 0.455, and 0.949, respectively (Figure 6F).

Figure 6

Figure 6. RF prediction model. (A) Relationship between the number of trees and OOB error in RF; “0” represents the GIB group, and “1” represents the NGIB group; (B) relationship between the number of split points and error in the tree; (C) importance of features in improving the model accuracy and reducing the Gini coefficient; (D) training cohort ROC curve; (E) test cohort ROC curve; (F) validation cohort ROC curve.

3.8 KNN-based predictive model

By performing hyperparametric optimization of the KNN function, we obtained the optimal kernel function as triangular as well as the optimal K value of 12 (Figure 7A). In the training cohort, the AUC of the model was 0.865, and the sensitivity, specificity, PPV and NPV were 1.000, 0.628, 0.247, and 1.000, respectively (Figure 7B). In the test cohort, the AUC of the model was 0.784, and the sensitivity, specificity, PPV, and NPV were 0.923, 0.632, 0.200, and 0.989, respectively (Figure 7C). In the validation cohort, the AUC value of the model was 0.811, and the sensitivity, specificity, PPV, and NPV were 0.875, 0.661, 0.250, and 0.976, respectively (Figure 7D).

Figure 7

Figure 7. KNN prediction model. (A) Screening for the best kernel function and k value; (B) training cohort ROC curve; (C) test cohort ROC curve; (D) validation cohort ROC curve.

3.9 Comparison of the predictive power of the 5 machine prediction models

We used six indices, namely, accuracy, AUC, sensitivity, specificity, PPV, and NPV, to evaluate the above five GIB models (Table 3). Compared with the other four machine prediction models, the RF model had better accuracy, AUC, sensitivity, specificity, PPV, and NPV in the training cohort (Figure 8A). The KNN model had a higher sensitivity of 1.000, whereas the DT model had a lower sensitivity of only 0.436. The PPVs of all three prediction models, i.e., LR, DT, and KNN, were low. In the test cohort (Figure 8B), the RF model sensitivity and PPV were significantly lower than those in the training cohort, with values of 0.625 and 0.455, respectively, which were intermediate compared with those of the other models. The three models, LR, SVM, and KNN, are much larger than the DT model in terms of sensitivity, but the trend reverses in terms of specificity and PPV. In the validation cohort (Figure 8C), the RF model had approximately the same situation as the test group for each metric. Compared with the other models, KNN had the highest sensitivity (0.875) and the lowest specificity and PPV (0.661 and 0.250, respectively).

Table 3

Table 3. A comparison of gastrointestinal bleeding models’ prediction ability.

Figure 8

Figure 8. Comparison of the prediction performance of five machine models. (A) Line graph comparing the prediction performance of models based on the training cohort; (B) line graph comparing the prediction performance of models based on the test cohort; (C) line graph comparing the prediction performance of models based on the validation cohort.

4 Discussion

GIB is a relatively common complication in patients with HBV-ACLF (24), but whether it affects patient prognosis remains controversial. Studies have reported that upper GIB does not significantly increase the risk of death in patients with ACLF (7). Another study from China explored the effects of artificial liver therapy on the prognosis and survival of patients with ACLF and reported that upper GIB was an independent risk factor for the prognosis of patients with ACLF (5). Trebicka et al. reported that the presence of ACLF was independently associated with rebleeding and mortality in patients with acute variceal bleeding (25). Shin et al. reported a 28-day mortality rate of 41% in patients with variceal bleeding combined with ACLF (26). Our study revealed that among the training cohort of 360 patients, the 30- and 90-day survival rates were significantly lower in the GIB group (48.72 and 10.26%, respectively) than in the NGIB group (85.67 and 64.80%, respectively). Relevant studies have reported that red cell distribution width, platelet count, pulse rate, and SpO2 are risk factors for early mortality in patients with acute upper gastrointestinal bleeding (27). Therefore, early monitoring of these indicators may improve the survival rate of gastrointestinal bleeding patients. ACLF and GIB interact with each other. Decreased coagulation and elevated hepatic venous pressure in patients with ACLF increase the risk of gastrointestinal hemorrhage and increase the difficulty of pharmacological and endoscopic hemostasis; conversely, ischemia and hypoxia after gastrointestinal hemorrhage exacerbate hepatic failure (4). Most studies have focused on the occurrence of ACLF in patients with chronic liver disease induced by GIB (28, 29), and few studies have investigated the prognostic impact of GIB in patients with ACLF, especially HBV-ACLF. Therefore, this study assessed the impact of GIB on the prognosis of patients with HBV-ACLF and constructed a relevant prediction model using ML algorithms to intervene in advance of bleeding events and reduce patient mortality.

The incidence of GIB in patients with HBV-ACLF was 10.8% in this study, which is similar to the 13.86% reported in the relevant literature (6). Seven clinical features associated with GIB were then identified via LASSO regression. The above features were incorporated into five ML algorithms to construct predictive models, and white blood cell count, portal hypertension, and spontaneous bacterial peritonitis were found to be three modeling features common to all five models. White blood cells are inflammatory markers, and their changes can respond to the inflammatory and immune states of the body. Tomizawa et al. studied the risk factors for the development of GIB in 1,023 patients who underwent endoscopy and reported that WBC counts were significantly greater in patients with GIB (30). Another study suggested that elevated white blood cell counts may be associated with nonvariceal upper GIB and its severity (31). White blood cells are significantly more common in patients with GIB who have nonvariceal bleeding than in those with variceal bleeding (32). Patients with HBV-ACLF have elevated white blood cell counts, which may be combined with infections and an increased inflammatory response, which puts the body in a state of stress, as well as causing poor coagulation, ultimately leading to the development of nonvariceal GIB.

Portal hypertension is common among patients with cirrhosis, in which elevated portal venous pressure leads to the obstruction of portal venous blood flow and the formation of collateral circulation in the fundic esophageal vein and other branches, and bleeding from fundic esophageal varices is a common and life-threatening complication (33). A Chinese study in which magnetic resonance imaging was used to construct a predictive model for esophageal variceal bleeding in patients with hepatitis B cirrhosis revealed that portal vein diameter was an independent risk factor for bleeding events (34). Patients with HBV-ACLF combined with decompensated cirrhosis are at high risk of bleeding from fundic esophageal varices and are prone to bleeding events in the presence of coagulation factor deficiency. Electrolyte disturbances are a regular complication among patients with ACLF. Patients have poor appetite, low intake, and excessive loss of electrolytes when ascites occurs, leading to hyponatremia, hypokalemia, etc., which affects patient prognosis (35). Several common electrolytes are involved in coagulation either as coagulation factors themselves or indirectly by regulating the coagulation process. Relevant studies have reported that sodium ions can participate in the coagulation process by binding to thrombin to modulate its conformation, increase thrombin activity, and promote the coagulation cascade reaction (36, 37). Calcium ions, as coagulation IV factors, are involved in coagulation factor activation and platelet function (38). The presence of multiple potassium channels in platelets, such as the Kv1.3, KCa3.1, GIRK and KCa1.1 channels, contributes to a variety of platelet functional responses, such as adhesion and procoagulant activity (39). When electrolyte disturbances occur in patients with HBV-ACLF, imbalances in multiple ion levels may exacerbate patients’ coagulation dysfunction and increase the probability of GIB by affecting platelet or coagulation factor function. Additionally, low total cholesterol levels can increase the fragility of red blood cells and endothelial cells, leading to thinning of the vessel walls and reduced elasticity. These changes make the blood vessels more prone to rupture and bleeding (40). The number of artificial liver treatments indirectly reflects the severity of the patient’s liver failure. Patients who require more frequent treatments generally have more severe conditions, with more pronounced systemic inflammatory responses and unstable hemodynamics, making them more susceptible to GIB (41). Spontaneous bacterial peritonitis triggers a systemic inflammatory response that leads to vasodilation of the visceral vessels, further exacerbating portal hypertension. In addition, inflammatory factors disrupt the coagulation balance and may ultimately directly damage the mucosal barrier, increasing the risk of GIB (42). In patients with HRS, inadequate effective blood volume activates the renin-angiotensin-aldosterone system, further increasing portal pressure, thereby increasing the risk of GIB (43).

With the above seven key clinical features, we constructed prediction models using each of the five ML methods: LR, SVM, DT, RF, and KNN. The model was evaluated via six metrics: accuracy, AUC, sensitivity, specificity, PPV, and NPV. Owing to the uneven sample size between the GIB and NGIB groups of the dataset, it was better to evaluate the model using the AUC as opposed to its accuracy. In the three cohorts, the LR model achieved better AUCs of 0.819, 0.829, and 0.817, respectively, but its PPVs in the modeling cohort and the training cohort were low, 0.300 and 0.202, respectively, which implies that it predicted a preponderance of positive false-positives, a common phenomenon caused by the unequal distribution of the dichotomous variables (44). We then used the model to construct a nomogram that better fit reality and clinical net benefit in the training cohort on the basis of the calibration curve and decision curve results. The RF model outperformed the other four models in the training cohort, with values of 1.000 for all the evaluation metrics. The sensitivity decreased to 0.625, and the PPV decreased to 0.455 in the test cohort, but remained significantly higher than those of the LR, SVM, and KNN models. The RF model significantly improved the PPV while maintaining a better AUC and achieved a similar performance in the validation cohort, with a better overall predictive ability than the other four models. This is related to the fact that the RF model itself is good at handling complex data, reducing overfitting, high accuracy, and robustness. This conclusion was corroborated by a predictive modeling study on omental metastasis of gastric cancer by Ahao et al. (45). The predictive power of the SVM model appeared average in comparison to the other four models, and it had the lowest AUC and sensitivity in the validation cohort at 0.657 and 0.500, respectively. The DT model performed inconsistently in terms of its predictive ability, with a lower AUC and sensitivity than the other four models in both the test and training cohorts. The predictive efficacy of the KNN model remained stable across the 3 cohorts, but the PPV was almost at the bottom, affecting the overall predictive ability of the model. The results of the lower predictive performance of the three models DT, SVM, and KNN may be related to their inherent susceptibility to overfitting, high computational complexity, and sensitivity to feature scaling relative to the RF models (44).

In this study, a prediction model for HBV-ACLF complicating GIB was constructed via five ML methods on the basis of data from two medical centers. The model was subsequently validated internally and externally, and the model similarly demonstrated a high degree of accuracy, providing a decision-making basis for preventing GIB events in ACLF patients in advance for clinical practice. There are several limitations to our study. First, both centers are in the same region, which can further collect patients nationwide and reduce the errors caused by population differences. Second, the sample size of the model constructed in this study is average, and a larger sample size is needed to verify the stability of the model. Finally, this study did not provide detailed information on the types and causes of gastrointestinal bleeding in the patients, nor did it further analyze their impact on patient survival. Future research could address these gaps.

5 Conclusion

We found that the short-term survival rate of patients with HBV-ACLF complicated by GIB was significantly lower than that of patients without GIB. On this basis, we constructed five ML models for predicting GIB and found that, compared with the LR, SVM, DT, and KNN models, the RF models have greater comprehensive predictive ability and are more instructive for clinical work.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Ethics statement

The studies involving humans were approved by the Ethics Committee of the First Affiliated Hospital of Nanchang University. The studies were conducted in accordance with the local legislation and institutional requirements. Written informed consent for participation was not required from the participants or the participants’ legal guardians/next of kin in accordance with the national legislation and institutional requirements.

Author contributions

JF: Conceptualization, Data curation, Investigation, Methodology, Project administration, Writing – original draft, Writing – review & editing. AW: Formal analysis, Methodology, Writing – review & editing. ZZ: Data curation, Investigation, Writing – review & editing. TD: Data curation, Investigation, Validation, Writing – review & editing. PS: Supervision, Writing – review & editing. WZ: Validation, Writing – review & editing. MT: Investigation, Writing – review & editing. YZ: Software, Writing – review & editing. YP: Investigation, Writing – review & editing. YW: Validation, Writing – review & editing. XW: Funding acquisition, Resources, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research and/or publication of this article. The study was funded by the Natural Science Foundation (No. 20212ACB206010) of Jiangxi Province, China and the National Natural Science Foundation of China (No. 82560133).

Acknowledgments

We thank Jiangxi Provincial People’s Hospital for providing valuable data.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The authors declare that no Gen AI was used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Jia, L, Xue, R, Zhu, Y, Zhao, J, Li, J, He, WP, et al. The efficacy and safety of methylprednisolone in hepatitis B virus-related acute-on-chronic liver failure: a prospective multi-center clinical trial. BMC Med. (2020) 18:383. doi: 10.1186/s12916-020-01814-4

PubMed Abstract | Crossref Full Text | Google Scholar

2. Sarin, SK, Kedarisetty, CK, Abbas, Z, Amarapurkar, D, Bihari, C, Chan, AC, et al. Acute-on-chronic liver failure: consensus recommendations of the Asian Pacific Association for the Study of the Liver (APASL) 2014. Hepatol Int. (2014) 8:453–71. doi: 10.1007/s12072-014-9580-2

PubMed Abstract | Crossref Full Text | Google Scholar

3. Chen, JF, Wang, KW, Zhang, SQ, Lei, ZY, Xie, JQ, Zhu, JY, et al. Dexamethasone in outcome of patients with hepatitis B virus-related acute-on-chronic liver failure. J Gastroenterol Hepatol. (2014) 29:800–6. doi: 10.1111/jgh.12454

PubMed Abstract | Crossref Full Text | Google Scholar

4. Heo, NY. Acute-on-chronic liver failure: a predictor of poor prognosis in patients with variceal bleeding or a risk factor for variceal bleeding? Clin Mol Hepatol. (2020) 26:487–8. doi: 10.3350/cmh.2020.0209

PubMed Abstract | Crossref Full Text | Google Scholar

5. Ma, WL, Ma, Y, Wang, WH, Ding, XC, Jiao, Y, Liu, SW, et al. Analysis of the prognosis and survival of patients with acute-on-chronic liver failure. Zhonghua Gan Zang Bing Za Zhi. (2023) 31:1051–5. doi: 10.3760/cma.j.cn501113-20230604-00243

PubMed Abstract | Crossref Full Text | Google Scholar

6. Guo, HY, Zhang, ZG, Zhang, YY, Mei, X, Liu, Y, Wang, JF, et al. Risks and predicting factors of bleeding complications in hepatitis B virus-related acute-on-chronic liver failure. Turk J Gastroenterol. (2020) 31:620–5. doi: 10.5152/tjg.2020.19307

PubMed Abstract | Crossref Full Text | Google Scholar

7. Zhao, H, Zhao, R, Hu, J, Zhang, X, Ma, J, Shi, Y, et al. Upper gastrointestinal hemorrhage in acute-on-chronic liver failure: prevalence, characteristics, and impact on prognosis. Expert Rev Gastroenterol Hepatol. (2019) 13:263–9. doi: 10.1080/17474124.2019.1567329

PubMed Abstract | Crossref Full Text | Google Scholar

8. Gu, W, Kimmann, M, Laleman, W, Praktiknjo, M, and Trebicka, J. To TIPS or not to TIPS in high risk of variceal rebleeding and acute-on-chronic liver failure. Semin Liver Dis. (2023) 43:189–205. doi: 10.1055/a-2107-0576

PubMed Abstract | Crossref Full Text | Google Scholar

9. Wang, X, Wang, BM, Li, G, Li, ZJ, Chen, C, and Piao, MY. A clinical prediction model and its application for bleeding in chronic liver failure patients with esophageal varices. Eur Rev Med Pharmacol Sci. (2013) 17:3046–55.

Google Scholar

10. Verma, AA, Murray, J, Greiner, R, Cohen, JP, Shojania, KG, Ghassemi, M, et al. Implementing machine learning in medicine. CMAJ. (2021) 193:E1351–7. doi: 10.1503/cmaj.202434

PubMed Abstract | Crossref Full Text | Google Scholar

11. Qiu, S, Zhao, Y, Hu, J, Zhang, Q, Wang, L, Chen, R, et al. Predicting the 28-day prognosis of acute-on-chronic liver failure patients based on machine learning. Dig Liver Dis. (2024) 56:2095–102. doi: 10.1016/j.dld.2024.06.029

PubMed Abstract | Crossref Full Text | Google Scholar

12. Ge, J, Digitale, JC, Fenton, C, McCulloch, CE, Lai, JC, Pletcher, MJ, et al. Predicting post-liver transplant outcomes in patients with acute-on-chronic liver failure using expert-augmented machine learning. Am J Transplant. (2023) 23:1908–21. doi: 10.1016/j.ajt.2023.08.022

PubMed Abstract | Crossref Full Text | Google Scholar

13. Speiser, JL, Karvellas, CJ, Wolf, BJ, Chung, D, Koch, DG, and Durkalski, VL. Predicting daily outcomes in acetaminophen-induced acute liver failure patients with machine learning techniques. Comput Methods Prog Biomed. (2019) 175:111–20. doi: 10.1016/j.cmpb.2019.04.012

PubMed Abstract | Crossref Full Text | Google Scholar

14. Tashiro, H, Onoe, T, Tanimine, N, Tazuma, S, Shibata, Y, Sudo, T, et al. Utility of machine learning in the prediction of post-hepatectomy liver failure in liver cancer. J Hepatocell Carcinoma. (2024) 11:1323–30. doi: 10.2147/JHC.S451025

PubMed Abstract | Crossref Full Text | Google Scholar

15. Sharma, N, Pandey, S, Yadav, M, Mathew, B, Bindal, V, Sharma, N, et al. Biomolecular map of albumin identifies signatures of severity and early mortality in acute liver failure. J Hepatol. (2023) 79:677–91. doi: 10.1016/j.jhep.2023.04.018

PubMed Abstract | Crossref Full Text | Google Scholar

16. Gary, PJ, Lal, A, Simonetto, DA, Gajic, O, and Gallo de Moraes, A. Acute on chronic liver failure: prognostic models and artificial intelligence applications. Hepatol Commun. (2023) 7:e0095. doi: 10.1097/HC9.0000000000000095

PubMed Abstract | Crossref Full Text | Google Scholar

17. Sarin, SK, Choudhury, A, Sharma, MK, Maiwall, R, Al Mahtab, M, Rahman, S, et al. Acute-on-chronic liver failure: consensus recommendations of the Asian Pacific association for the study of the liver (APASL): an update. Hepatol Int. (2019) 13:353–90. doi: 10.1007/s12072-019-09946-3

Crossref Full Text | Google Scholar

18. Saliba, F, Bañares, R, Larsen, FS, Wilmer, A, Parés, A, Mitzner, S, et al. Artificial liver support in patients with liver failure: a modified DELPHI consensus of international experts. Intensive Care Med. (2022) 48:1352–67. doi: 10.1007/s00134-022-06802-1

PubMed Abstract | Crossref Full Text | Google Scholar

19. Wang, H, Shao, Y, Zhou, S, Zhang, C, and Xiu, N. Support vector machine classifier via L(0/1) soft-margin loss. IEEE Trans Pattern Anal Mach Intell. (2022) 44:7253–65. doi: 10.1109/TPAMI.2021.3092177

PubMed Abstract | Crossref Full Text | Google Scholar

20. Saad, M, Zhang, Y, Jia, J, and Tian, J. Decision tree-based approach to extrapolate life cycle inventory data of manufacturing processes. J Environ Manag. (2024) 360:121152. doi: 10.1016/j.jenvman.2024.121152

PubMed Abstract | Crossref Full Text | Google Scholar

21. Hu, J, and Szymczak, S. A review on longitudinal data analysis with random forest. Brief Bioinform. (2023) 24:bbad002. doi: 10.1093/bib/bbad002

PubMed Abstract | Crossref Full Text | Google Scholar

22. Zhang, S, Li, X, Zong, M, Zhu, X, and Wang, R. Efficient kNN classification with different numbers of nearest neighbors. IEEE Trans Neural Netw Learn Syst. (2018) 29:1774–85. doi: 10.1109/TNNLS.2017.2673241

PubMed Abstract | Crossref Full Text | Google Scholar

23. Zabor, EC, Reddy, CA, Tendulkar, RD, and Patil, S. Logistic regression in clinical studies. Int J Radiat Oncol Biol Phys. (2022) 112:271–7. doi: 10.1016/j.ijrobp.2021.08.007

PubMed Abstract | Crossref Full Text | Google Scholar

24. Joshi, D, O'Grady, J, Patel, A, Shawcross, D, Connor, S, Deasy, N, et al. Cerebral oedema is rare in acute-on-chronic liver failure patients presenting with high-grade hepatic encephalopathy. Liver Int. (2014) 34:362–6. doi: 10.1111/liv.12257

PubMed Abstract | Crossref Full Text | Google Scholar

25. Trebicka, J, Gu, W, Ibáñez-Samaniego, L, Hernández-Gea, V, Pitarch, C, Garcia, E, et al. Rebleeding and mortality risk are increased by ACLF but reduced by pre-emptive TIPS. J Hepatol. (2020) 73:1082–91. doi: 10.1016/j.jhep.2020.04.024

PubMed Abstract | Crossref Full Text | Google Scholar

26. Shin, J, Yu, JH, Jin, YJ, Yim, HJ, Jung, YK, Yang, JM, et al. Acute-on-chronic liver failure as a major predictive factor for mortality in patients with variceal bleeding. Clin Mol Hepatol. (2020) 26:540–53. doi: 10.3350/cmh.2020.0034

PubMed Abstract | Crossref Full Text | Google Scholar

27. Yuan, L, and Yao, W. Development and validation of a risk prediction model for in-hospital mortality in patients with acute upper gastrointestinal bleeding. Clin Appl Thromb Hemost. (2023) 29:10760296231207806. doi: 10.1177/10760296231207806

PubMed Abstract | Crossref Full Text | Google Scholar

28. Rogalski, P, Zaborowska, M, Mazur, M, Jankowska, Ż, Piszczyk, A, Mermer, E, et al. Gastrointestinal hemorrhage as an acute-on-chronic liver failure trigger in cirrhotic patients. Adv Clin Exp Med. (2022) 31:863–71. doi: 10.17219/acem/147464

PubMed Abstract | Crossref Full Text | Google Scholar

29. Zhu, Z, and Jiang, H. Risk stratification based on acute-on-chronic liver failure in cirrhotic patients hospitalized for acute variceal bleeding. BMC Gastroenterol. (2023) 23:148. doi: 10.1186/s12876-023-02768-6

PubMed Abstract | Crossref Full Text | Google Scholar

30. Tomizawa, M, Shinozaki, F, Hasegawa, R, Shirai, Y, Motoyoshi, Y, Sugiyama, T, et al. Low hemoglobin levels are associated with upper gastrointestinal bleeding. Biomed Rep. (2016) 5:349–52. doi: 10.3892/br.2016.727

PubMed Abstract | Crossref Full Text | Google Scholar

31. Yin, L, and Yu, W. Retrospective analysis of risk factors for non-variceal upper gastrointestinal bleeding and construction of a nomogram prediction model. Am J Transl Res. (2023) 15:3385–93.

Google Scholar

32. Zhang, L, and Zhang, Y. Diagnostic values of blood count values and ratios in distinguishing between peptic ulcer bleeding and esophagogastric variceal bleeding. Clin Lab. (2020) 66. doi: 10.7754/Clin.Lab.2019.190927

Crossref Full Text | Google Scholar

33. Biecker, E. Portal hypertension and gastrointestinal bleeding: diagnosis, prevention and management. World J Gastroenterol. (2013) 19:5035–50. doi: 10.3748/wjg.v19.i31.5035

PubMed Abstract | Crossref Full Text | Google Scholar

34. Tan, BG, Tang, Z, Ou, J, Zhou, HY, Li, R, Chen, TW, et al. A novel model based on liver/spleen volumes and portal vein diameter on MRI to predict variceal bleeding in HBV cirrhosis. Eur Radiol. (2023) 33:1378–87. doi: 10.1007/s00330-022-09107-5

PubMed Abstract | Crossref Full Text | Google Scholar

35. Huang, K, Hu, JH, Wang, HF, He, WP, Chen, J, Duan, XZ, et al. Survival and prognostic factors in hepatitis B virus-related acute-on-chronic liver failure. World J Gastroenterol. (2011) 17:3448–52. doi: 10.3748/wjg.v17.i29.3448

PubMed Abstract | Crossref Full Text | Google Scholar

36. Huntington, JA. How Na+ activates thrombin--a review of the functional and structural data. Biol Chem. (2008) 389:1025–35. doi: 10.1515/BC.2008.113

PubMed Abstract | Crossref Full Text | Google Scholar

37. Pelc, LA, Koester, SK, Kukla, CR, Chen, Z, and Di Cera, E. The active site region plays a critical role in Na(+) binding to thrombin. J Biol Chem. (2022) 298:101458. doi: 10.1016/j.jbc.2021.101458

PubMed Abstract | Crossref Full Text | Google Scholar

38. Yu, J, Su, H, Wei, S, Chen, F, and Liu, C. Calcium content mediated hemostasis of calcium-modified oxidized microporous starch. J Biomater Sci Polym Ed. (2018) 29:1716–28. doi: 10.1080/09205063.2018.1481585

PubMed Abstract | Crossref Full Text | Google Scholar

39. Wright, JR, and Mahaut-Smith, MP. Why do platelets express K(+) channels? Platelets. (2021) 32:872–9. doi: 10.1080/09537104.2021.1904135

PubMed Abstract | Crossref Full Text | Google Scholar

40. Peng, J, Jin, H, Zhang, N, Zheng, S, Yu, C, Yu, J, et al. Development and evaluation of a predictive model of upper gastrointestinal bleeding in liver cirrhosis. BMC Gastroenterol. (2025) 25:142. doi: 10.1186/s12876-025-03677-6

PubMed Abstract | Crossref Full Text | Google Scholar

41. Yang, Z, Zhang, Z, Cheng, Q, Chen, G, Li, W, Ma, K, et al. Plasma perfusion combined with plasma exchange in chronic hepatitis B-related acute-on-chronic liver failure patients. Hepatol Int. (2020) 14:491–502. doi: 10.1007/s12072-020-10053-x

PubMed Abstract | Crossref Full Text | Google Scholar

42. Fernández, J, Bauer, TM, Navasa, M, and Rodés, J. Diagnosis, treatment and prevention of spontaneous bacterial peritonitis. Baillieres Best Pract Res Clin Gastroenterol. (2000) 14:975–90. doi: 10.1053/bega.2000.0142

PubMed Abstract | Crossref Full Text | Google Scholar

43. Patel, A, Zhang, C, Minacapelli, CD, Gupta, K, Catalano, C, Li, Y, et al. Outcomes, mortality, and cost burden of acute kidney injury and hepatorenal syndrome in patients with cirrhosis. J Gastrointestin Liver Dis. (2023) 32:39–50. doi: 10.15403/jgld-4618

PubMed Abstract | Crossref Full Text | Google Scholar

44. Huh, JW, Kim, HR, and Kim, YJ. Clinical prediction of pathological complete response after preoperative chemoradiotherapy for rectal cancer. Dis Colon Rectum. (2013) 56:698–703. doi: 10.1097/DCR.0b013e3182837e5b

PubMed Abstract | Crossref Full Text | Google Scholar

45. Wu, A, Luo, L, Zeng, Q, Wu, C, Shu, X, Huang, P, et al. Comparative assessment of the capability of machine learning-based radiomic models for predicting omental metastasis in locally advanced gastric cancer. Sci Rep. (2024) 14:16208. doi: 10.1038/s41598-024-66979-x

PubMed Abstract | Crossref Full Text | Google Scholar