Random forest can accurately predict the technique failure of peritoneal dialysis associated peritonitis patients

Instructions Peritoneal dialysis associated peritonitis (PDAP) is a major cause of technique failure in peritoneal dialysis (PD) patients. The purpose of this study is to construct risk prediction models by multiple machine learning (ML) algorithms and select the best one to predict technique failure in PDAP patients accurately. Methods This retrospective cohort study included maintenance PD patients in our center from January 1, 2010 to December 31, 2021. The risk prediction models for technique failure were constructed based on five ML algorithms: random forest (RF), the least absolute shrinkage and selection operator (LASSO), decision tree, k nearest neighbor (KNN), and logistic regression (LR). The internal validation was conducted in the test cohort. Results Five hundred and eight episodes of peritonitis were included in this study. The technique failure accounted for 26.38%, and the mortality rate was 4.53%. There were resignificant statistical differences between technique failure group and technique survival group in multiple baseline characteristics. The RF prediction model is the best able to predict the technique failure in PDAP patients, with the accuracy of 93.70% and area under curve (AUC) of 0.916. The sensitivity and specificity of this model was 96.67 and 86.49%, respectively. Conclusion RF prediction model could accurately predict the technique failure of PDAP patients, which demonstrated excellent predictive performance and may assist in clinical decision making.


Introduction
Peritoneal dialysis (PD) is a preferred and cost-effective renal replacement therapy for end-stage renal disease (ESRD) patients (1,2).In the context of the global prevalence of coronavirus disease 2019 (COVID-19), PD is more advantageous than hemodialysis (HD) because of its home-based use (3,4).However, peritoneal dialysis associated peritonitis (PDAP) is one of the common but severe complications in the process of PD (5), which greatly damages the peritoneal function and limits the use of PD (6).Overall peritonitis rate is 0.06-1.66episodes per patient-year (7,8), which is a significant reason for catheter dysfunction events, repeated hospitalization and technique failure (9).PDAP directly contributes to 22% catheter removal, 18% permanent transfer to HD, and 2-6% death (10), accounting the most important complication that leads to technique failure.The outcomes of peritonitis varied markedly among different centers and countries despite the guidelines of PDAP (11).Therefore, identifying the high-risk population of technique failure has important clinical value for improving the management and prognosis of PDAP patients.Machine learning (ML) algorithms are important bioinformatics research methods in the clinic, which can process a large number of clinical datasets and incorporate multiple indicators to construct prediction models to assess the risk of outcomes (12).In the PDAP population, the previous study constructed a model which showed excellent prediction performance for pathogen-specific diagnosis based on ML algorithms (13).Some studies also attempted to predict the prognosis of PDAP patients using ML algorithms.One single-center study constructed a nomogram by using multivariate COX regression, which could predict cardiovascular events in PDAP patients (14).There are also studies attempting to predict treatment failure in PDAP patients (15)(16)(17)(18).However, these studies only included a portion of clinical variables to screen predictors, which may ignore important variables.Meanwhile, these studies only used logistic regression (LR) to construct prediction models, without constructing multiple models and comparing their performances.In addition, some of the above studies did not establish independent test datasets to validate the performance of risk prediction tool (17,18).
With incorporating all possible clinical variables, we intended to construct multiple prediction models based on ML algorithms, aiming to find the best prediction model to predict the technique failure in PDAP patients, which was helpful to identify high-risk patients, optimize the management of PDAP patients, and improve their prognosis.

Study population
This retrospective study enrolled the maintenance PD patients who were hospitalized due to PDAP in our PD center of the West China Hospital, Sichuan University, from January 1, 2010 to December 31, 2021.
All PDAP patients were diagnosed according to the criteria of 2022 International Society for Peritoneal Dialysis (ISPD) guidelines (4).The details are as follows: PDAP should be diagnosed when at least two of the following are present: (1) clinical features consistent with peritonitis (abdominal pain and/or cloudy dialysis effluent); (2) dialysis effluent white cell count >100 × 10 6 /L, with >50% polymorphonuclear leukocytes; (3) positive dialysis effluent culture.
The patients included herein were chosen according to the following criteria: (1) maintenance peritoneal dialysis patients admitted to our department due to PDAP, and (2) age ≥ 18 years old.The exclusion criteria were as follows: (1) episodes with PD < 3 months, (2) episodes with recovered renal function, and (3) episodes with missing data (absence of the peritoneal dialysate white cell count or the results of pathogenic culture).
This study was approved by the Medical Ethics Committee of West China Hospital of Sichuan University, Sichuan, China.This study was registered at the Thai Clinical Trials Registry (TCTR20180313004).This study followed the Declaration of Helsinki, and written informed consents were obtained from all participants.

Data collection
The following clinical and laboratory data were collected from the medical records and laboratory information system.The demographic variables, including age, gender, height, weight, systolic blood pressure (SBP) and diastolic blood pressure (DBP) were obtained.The body mass index (BMI) was calculated based on height and weight.The patients' comorbidities were recorded, including diabetes mellitus (DM), cirrhosis, cardiovascular diseases (CVDs), connective tissue diseases, etc.The CVDs were defined as the presence of coronary heart disease, heart failure, stroke, and peripheral arterial disease (19).The Charlson Comorbidity Index Score (CCI) was calculated based on the patients' age and comorbidities (20).PD modality [continuous ambulatory peritoneal dialysis (CAPD) or automated peritoneal dialysis (APD)] and PD duration were also recorded.The laboratory variables included hemoglobin (HB), white blood cell (WBC), serum albumin (ALB), serum creatinine (SCr), blood urea nitrogen (BUN), uric acid (UA), estimated glomerular filtration rate (eGFR), triglycerides (TG), high-density lipoprotein cholesterol (HDL-C), low-density lipoprotein cholesterol (LDL-C), serum potassium (K), serum calcium (Ca), N-terminal pro-brain natriuretic peptide (NT-proBNP), high-sensitive C-reactive protein (hs-CRP), fibrinogen (FIB), ferritin, etc.And eGFR was calculated by the Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) equation.The peritoneal dialysate white cell counts on day 1, day 3 and day 5 were collected.The results of causative organisms and the regimens of initial intraperitoneal (IP) antibiotic were recorded.

Outcomes
The patients' outcomes were technique failure and technique survival.Technique failure was defined as PD catheter removal, transfer to HD (temporary or permanent) or peritonitis-related death (21).Peritonitis-related death included death within 4 weeks after the onset of PDAP, death with active peritonitis, or any death during hospitalization for a peritonitis episode.And technique survival was defined as complete resolution of peritonitis without the need for PD catheter removal (21).Complete resolution was defined as the white cell counts <100/uL in peritoneal dialysates with a relief of clinical manifestations.

Data process
Before the statistical analysis and construction of prediction models, we conducted the data preprocessing.The raw data was verified and checked independently by two authors (ZZY and XQJ).To improve the quality and reliability of the data, we have corrected the errors, omissions, duplicates, or outliers in the raw data.We transformed the data consistently and reproducibly in order to be recognized by R software.To reduce statistical workload and variables unrelated to the outcomes of PDAP patients, univariate analysis was utilized to identify all possible variables.Variables with p value < 0.1 in the univariate analysis were selected to construct the prediction models.Data with less than 15% missing values were imputed by mean or median (22).Data with more than 15% missing values were excluded from the final analysis.

Construction and validation of prediction models based on ML algorithms
Seventy-five percent of all patients were randomly allocated to the training cohort and constructed a prediction model to predict the technique failure of PDAP patients, while the remaining 25% were assigned to the test cohort for internal validation.The optimal hyperparameters were selected from 5-fold cross-validation in the training cohort.The performance of prediction models was evaluated by accuracy, sensitivity, specificity, precision, recall, F1-score and area under curve (AUC).The confusion matrix and receiver operating characteristic curves (ROC) were presented in the analysis.
In this study, five machine learning algorithms were applied to construct the prediction models of technique failure in PDAP patients, including random forest (RF), the least absolute shrinkage and selection operator (LASSO), decision tree, k nearest neighbor (KNN), and LR.

Results
The flow chart of this study is presented in Figure 1.A total of 580 PDAP episodes were enrolled in our study.However, 72 episodes were excluded: 66 episodes with PD less than 3 months, and 6 episodes missing the peritoneal dialysate white cell count.No patients had recovered renal function.

Characteristics of study participants
There were 508 eligible episodes in 413 PDAP patients in the current study.Figure 1 shows the clinical outcomes of the PDAP patients.The technique failure accounted for 26.38%, and the mortality rate was 4.53% (23 episodes).Switching from PD to HD was the leading reason (111 episodes, 21.85%) of technique failure.And technique survival was achieved in 374 (73.62%) episodes.
The baseline characteristics of all patients are shown in Table 1.The median age was 50.0 years old and the male patients accounted for 53.0%.The median PD duration was 25.2 months and CAPD was the main PD modality (97.4%).Among the eligible patients, 85 patients (16.7%) had DM, with 116 patients (22.8%) having CVDs, and the median score of CCI was 4 points.
The details of glycolipid metabolism, electrolyte, coagulation and inflammatory variables are displayed in Table 2.The PDAP related tests and initial IP antibiotic regimens are displayed in Table 3.There were 255 culture-positive episodes (50.2%) in all PDAP patients.And coagulase-negative Staphylococcus was the leading bacteria (18.3%) in the causative organisms.

Comparison between technique failure group and technique survival group
In order to explore the potential predictors, we conducted univariate analysis (Tables 1-3).Table 1 showed there were no significant statistical differences in gender, age, BMI, SBP, DBP and PD modality.However, the PD duration of the technique failure group was significantly longer than the technique survival group (37.2 months vs. 24.9months, p = 0.001).In complications, we found that higher incidence of CVDs and higher scores of CCI in the technique failure group (p = 0.016 and p = 0.022, respectively).
As shown in Table 2, there were statistical differences between the two groups in many laboratory variables such as ALB, NT-proBNP, HDL-C, FIB, ferritin, etc.
The peritoneal dialysate white cell counts in the technique failure group on day 3 (545 × 10 6 /L vs. 120 × 10 6 /L, p < 0.001) and on day 5 (270 × 10 6 /L vs. 20 × 10 6 /L, p < 0.001) were significantly higher than the technique survival group (Table 3).There was no statistical difference in the initial IP antibiotic treatment regimens between the two groups (p = 0.143), and the PDAP patients received first-generation cephalosporin plus third-generation cephalosporin frequently (Table 3).
With regard to the results of causative organisms (Table 3), there was a significantly lower rate of culture-negative episodes between the failure group and the survival group (40.3% vs. 53.7%,p = 0.009).There was also a significant statistical difference in the composition of causative organism spectrum between the two groups (p < 0.001).The main bacteria of failure group were fungi (15.7%) and coagulasenegative Staphylococcus (15.7%).While the technique survival group was mainly composed of coagulase-negative Staphylococcus (19.3%), followed by Streptococcus (5.9%).

Construction and validation of prediction models based on ML algorithms
We preprocessed the data before the construction of prediction models.We deleted some variables with high missing rates, such as IL-6, hs-CRP, and PCT.Then, we included variables with p values less than 0.05 and interpolated the missing values by the mean or median for the construction of prediction models.The patients were randomly divided into training cohort and test cohort according to the ratio of 0.75 to 0.25.There was no statistical difference in the incidence of technique failure between the training cohort and the test cohort (25.46% vs. 29.13%,p = 0.418, Figure 2), indicating that the random allocation was feasible.
We then constructed the prediction models for technique failure in PDAP patients based on LR, LASSO, RF, KNN and decision tree.Figure 3 displays the ROC curves and Table 4 shows the validation of prediction models.Our results demonstrated that the RF model had excellent performance in predicting technique failure in PDAP patients compared with other prediction models, which showed the highest AUC value (0.916).The calibration curves of prediction models were depicted in Figures 4A-E, and the LR and RF models were close to perfectly calibrated line which demonstrated favorable calibration ability.
What's more, we assessed the importance of variables based on mean decrease Gini index in the RF prediction model (Figure 5).Mean decrease Gini indicates the importance of the independent variables to the dependent variable, which refers to the total decrease of Gini from splitting on the variable averaged over all trees.And the top 5 predictors for technique failure were peritoneal dialysate white cell count on day 5, NT-proBNP, peritoneal dialysate white cell count on day 3, FIB and ferritin.Flow chart for the study participants enrollment and outcomes.

Discussion
Our study is the first to develop the prediction models by multiple ML algorithms and then select the best one for technique failure in PDAP.The RF model could accurately predict the technique failure of PDAP patients, which had the most excellent predictive performance among the five prediction models.And RF model could assess the importance of predictors associated with technique failure intuitively.Previous studies have developed prediction models for the treatment failure in PDAP patients only using LR.One multicenter and retrospective cohort study conducted in Thailand developed a risk prediction tool for peritonitis associated technique failure based on multivariate LR (15).Meng et al. established a nomogram for the prediction of peritonitis cure by using multivariate LR in PDAP patients.Although, the prediction nomogram model was intuitive and applicable to clinical practice, its C-statistic value was 0.756 (16).Another two single-center studies also only used LR analysis to established prediction models, and they lacked independent datasets to validate their prediction performance (17,18).
Unlike the previous studies only using LR to construct models, our study used 5 ML algorithms and we found that RF model was more suitable for predicting technique failure in PDAP patients than the other four prediction models.Due to the different methodological analysis of machine learning algorithms, their performance varies when dealing with the same data or problems.Random forest performs well in handling complex data or multivariate problems.
Enteric gram-negative bacteria peritonitis, n% Random forest is an ensemble learning algorithm, which is a classifier containing multiple decision trees (23).The final categories are determined according to the voting of multiple decision trees.Random forest can select samples or variables randomly to avoid overfitting.At the same time, random forest has strong adaptability to data, which can handle datasets containing a large number of variables and balance the errors generated by unbalanced datasets.What's more, random forest can still maintain high accuracy even if some features are lost (24).However, LR requires the assumption of linear relationships between variables, and variable transformations are required for nonlinear data, which is not always applicable to all variables in clinic (25).Furthermore, LR is prone to insufficient fitting and poor classification accuracy (26).
The previous studies did not incorporate some important and readily available variables, such as NT-proBNP, peritoneal dialysate white cell count on day 3 or day 5, FIB, or ferritin.We included all possible variables to avoid ignoring important variables in our prediction models.Our study demonstrated that NT-proBNP, FIB and ferritin were important predictors for technique failure in PDAP patients.When selecting variables, we conducted cross-validation to try to avoid overfitting or underfitting issues, which improved the generalization ability of prediction models.
Our study found several important predictors of technique failure.In the RF prediction model, the peritoneal dialysate white cell count on day 5 was the most important predictor, which was consistent with previous studies.A previous risk-scoring scheme demonstrated that peritoneal dialysate white cell count on day 5 was the strongest predictor, which was responsible for 88% of their predictive model (15).The retrospective study included 565 peritonitis episodes and found that the predictor of peritoneal dialysate white count >100/uL for at least 5 days showed a significant association with treatment failure [odds ratio (OR) 7.38; 95% confidence interval (CI) 3.38 to 16.1] (27).Another retrospective study of 399 peritonitis episodes, Krishnan et al. reported that the nonresolution rate was significantly higher in patients with the peritoneal dialysate cell counts exceeded 100/uL for more than 5 days (28).The peritoneal dialysate white cell count on day 3 was another important predictor of technique failure, which has been previously recognized (15,27).Comparing to the peritoneal dialysate white cell count on day 3, the cell count on day 5 might reflect the response of treatment better (15).But the peritoneal dialysate white cell count on day 3 might be more valuable to the clinicians who desired to refine the treatment strategy during the early course of PDAP (27).We recommend the peritoneal dialysate white cell count should be measured on a routine basis, especially on day 5 and day 3.
NT-proBNP was a significant predictor of technique failure which was not reported in previous studies about PDAP.There were several studies demonstrated that elevated NT-proBNP was associated with adverse outcomes in PD patients.The Adequacy of Peritoneal Dialysis in Mexico (ADEMEX) study enrolled 965 CAPD patients and found that NT-proBNP was independently highly predictive of overall survival and cardiovascular mortality (29).Two prospective cohort studies also reported that NT-proBNP is an important risk predictor of cardiovascular congestion, mortality, and adverse cardiovascular outcomes (30), and was independently predictive of an increased risk of technique failure in maintenance PD patients (31).During PDAP, the increased permeability of peritoneal membrane due to inflammation might result in decreased ultrafiltration and the subsequent volume overload and elevation of NT-proBNP.Volume overload and even congestive heart failure demand emergent hemodialysis, which may contribute to the technique failure in PDAP.This could be partially avoided by the use of icodextrin PD fluid.But icodextrin PD fluid was just available in China in recent 2 years.We still recommend it is necessary to monitor NT-proBNP regularly during PDAP and explore the relationship between NT-proBNP and fluid status in PDAP patients in the future.
With the help of prediction model, we could accurately distinguish the high-risk patients prone to technique failure in PDAP.For the high-risk patients, we suggest a closer assessment of relevant    Ranking the importance of variables based on RF model.There were some limitations in our study.Firstly, the cohort only included Chinese patients of a single PD center and the laboratory data was affected by geography and demographics.Our prediction models were internally valid, but we lacked external validation.Therefore, the generalizabilities of the prediction models were limited.Secondly, some variables such as IL-6, hs-CRP or PCT were excluded due to high missing rates.These variables may be potential predictors for technique failure, which can be explored in future.Thirdly, this was a retrospective study and might have selection bias.Several novel biomarkers showing good predictive performance of adverse outcomes in PDAP patients, which could be incorporated into the construction of prediction models in future (33,34).Multicenter and large data are required for the external validation and to evaluate the clinical utility of the prediction models.

Conclusion
In conclusion, our study constructed a reliable prediction model based on RF algorithm to predict the risk of technique failure for PDAP patients.The RF model demonstrated the best predictive performance and highest accuracy, which could accurately predict the technique failure and assist in clinical decision-making.

FIGURE 3 ROC
FIGURE 3ROC curves of prediction models based on machine learning algorithms.

FIGURE 5
FIGURE 5 Continuous variables were analyzed by Student's t test or Mann-Whitney U test, while categorical variables were analyzed by Chi-square test or Fisher's exact test.The IBM SPSS Statistics version 26.0 (IBM Corp., Armonk, NY, United States) and R software version 4.2.2 (R Foundation, Vienna, Austria) were used to perform the statistical analyses.A two-sided p value less than 0.05 was considered statistically significant.

TABLE 1
Baseline demographic characteristics.
*Comparison between technique failure group and technique survival group.The bold values were p values < 0.05.FIGURE 1

TABLE 2
Baseline characteristics of laboratory variables.Comparison between technique failure group and technique survival group.The bold values were p values < 0.05. *

TABLE 3
Baseline characteristics of peritoneal dialysate and IP antibiotic regimens.

TABLE 4
Validation of the prediction models.