Applying interpretable machine learning algorithms to predict risk factors for permanent stoma in patients after TME

Abstract

Objective:

The purpose of this study was to develop a machine learning model to identify preoperative and intraoperative high-risk factors and to predict the occurrence of permanent stoma in patients after total mesorectal excision (TME).

Methods:

A total of 1,163 patients with rectal cancer were included in the study, including 142 patients with permanent stoma. We collected 24 characteristic variables, including patient demographic characteristics, basic medical history, preoperative examination characteristics, type of surgery, and intraoperative information. Four machine learning algorithms including extreme gradient boosting (XGBoost), random forest (RF), support vector machine (SVM) and k-nearest neighbor algorithm (KNN) were applied to construct the model and evaluate the model using k-fold cross validation method, ROC curve, calibration curve, decision curve analysis (DCA) and external validation.

Results:

The XGBoost algorithm showed the best performance among the four prediction models. The ROC curve results showed that XGBoost had a high predictive accuracy with an AUC value of 0.987 in the training set and 0.963 in the validation set. The k-fold cross-validation method was used for internal validation, and the XGBoost model was stable. The calibration curves showed high predictive power of the XGBoost model. DCA curves showed higher benefit rates for patients who received interventional treatment under the XGBoost model. The AUC value for the external validation set was 0.89, indicating that the XGBoost prediction model has good extrapolation.

Conclusion:

The prediction model for permanent stoma in patients with rectal cancer derived from the XGBoost machine learning algorithm in this study has high prediction accuracy and clinical utility.

1. Introduction

Rectal cancer is a gastrointestinal tumor with an extremely high morbidity and mortality rate. The incidence of colorectal cancer is increasing year by year due to changes in people's lifestyle and dietary habits. A 2019 epidemiological survey (1) showed that the incidence of colorectal cancer ranks third among malignant tumors worldwide, after lung cancer and breast cancer. Total mesorectal excision (TME), a common surgical treatment for rectal cancer, has greatly improved the prognosis of patients with rectal cancer. The principle of surgery is to completely resect the entire mesentery of low and intermediate rectal cancer to reduce the postoperative recurrence rate of patients while increasing the rate of anal preservation and improving the survival quality of patients (2–4). However, as the level of surgery continues to evolve, clinicians are gradually discovering the limitations of TME, such as patients’ vulnerability to serious complications such as anastomotic leakage after surgery (5, 6). The quality of life for patients who develop postoperative complications is poor and there is an increased risk of secondary surgery. Therefore, temporary prophylactic stoma is often used clinically for patients with rectal cancer who have preserved anus, thus reducing the pressure at the anastomosis and reducing the risk of anastomotic leakage (7). Fortunately, some patients can undergo ostomy reversal at the appropriate time to improve quality of life. However, other patients are unable to retract for various reasons and suffer great physical and psychological damage. Some studies (8, 9) have shown that permanent stoma prolongs the life of patients but reduces their quality of life. Therefore, it is crucial to understand high risk factors for permanent stoma so that surgeons can appropriate counsel patients.

Artificial intelligence (AI) is developing rapidly in the medical field (10). Machine learning, as a major branch of AI, has the advantages of more stable model building and more accurate prediction, is favored by clinicians and is used in clinical prediction and other aspects (11, 12). In this study, we analyzed the clinical information of rectal cancer patients and applied machine learning algorithms to establish a prediction model for permanent stoma in rectal cancer patients to aid clinicians in making timely and accurate individualized treatment plans.

2. Materials and methods

2.1. Study subjects

In the study, we used data from the clinical databases of Wuxi People's Hospital affiliated to Nanjing Medical University and Wuxi Second People's Hospital. Inclusion criteria: (1) patients were diagnosed with rectal cancer by pathological examination; (2) patients were treated with TME surgery; and (3) the surgical team consisted of senior doctors who had the ability to independently perform TME and enterostomy. Exclusion criteria were as follows: (1) patients with other malignant tumors; (2) patients who had been diagnosed with distant metastasis of rectal cancer; (3) patients with other diseases of the rectum; (4) patients diagnosed with life-threatening cardiovascular diseases such as cerebral infarction; (5) patients diagnosed with important organ diseases such as liver failure or kidney failure; and (6) missing case records or missed visits. Patients were followed up for at least three years after surgery. Two surgeons performed medical history, physical examination and abdominal ultrasound, computed tomography (CT) and other imaging examinations on patients every three months. The study was approved by the Ethics Review Committee of Wuxi People's Hospital, with approval number KY22086.

2.2. Study design and data collection

Clinical data were retrospectively collected from January 2010 to January 2018 from patients with rectal cancer at Wuxi People's Hospital and Wuxi Second People's Hospital, including 25 preoperative variables (within 24 h of the day of surgery) and intraoperative variables. Preoperative variables collected included patient demographic characteristics (gender, age, smoking history, alcohol history, and body mass index), basic clinical characteristics (American Society of Anesthesiologists score, nutrition risk screening 2002 score, history of surgery, chemotherapy, and radiotherapy), basic medical history (anemia, rectal stenosis, diabetes, hypertension, hyperlipidemia, and coronary artery disease), laboratory tests (carcinoembryonic antigen, carbohydrate antigen 19-9, and albumin), and tumor characteristics (T-category, N-category, tumor recurrence, tumor size, and tumor distance from the dentate line). Intraoperative variables collected included whether a permanent stoma was performed.

2.3. Definition of permanent stoma

A permanent stoma is defined as a permanent stoma created during the patient's initial surgery or a permanent stoma created during the progression of the patient's disease. With the patient's consent, the decision to create a permanent stoma is made by the surgeon, taking into account the patient's physical condition and disease progression.

2.4. Development and evaluation of predictive models for machine learning algorithms

SPSS software and R software were applied for the construction and evaluation of clinical prediction models. (1) Univariate and multivariate regression analyses were performed. The chi-square test was applied to categorical variables to compare the differences between the two groups; a t-test was performed for continuous variables that conformed to a normal distribution; and the rank sum test was selected for continuous variables that did not conform to a normal distribution. P < 0.05 indicated that the difference was statistically significant. Logistic regression analysis of variables with significance in the univariate analysis was performed to obtain independent influences on permanent stoma in patients with rectal cancer. (2) Evaluate and build prediction models. Rectal cancer patients from Wuxi People's Hospital from January 2010 to December 2016 were selected as the internal validation set, and rectal cancer patients from Wuxi Second People's Hospital from January 2017 to January 2018 were selected as the external validation set. The internal validation set was randomly divided into training set (70%) and test set (30%). The independent impact factors derived from the regression analysis were incorporated into four machine learning algorithm prediction models: support vector machine (SVM), random forest (RF), extreme gradient boosting (XGBoost), and k-nearest neighbor algorithm (KNN). The four models were evaluated by three aspects, i.e., discrimination, calibration, and clinical usefulness, and the best model was selected for prediction analysis. The ROC curve was plotted to obtain the AUC value and determine the predictive efficiency of the model; the calibration curve was plotted to assess whether there was good agreement between the predicted and actual results of the model; and decision curve analysis (DCA) was plotted to assess the benefit to the patient after interventional treatment. Internal validation was completed using the k-fold cross-validation method. (4) External validation of the best model using an external test set, plotting ROC curves and calibration curves, and determining the generalizability and predictive efficiency of the model. (5) Model interpretation. The contribution of each feature in the sample to the prediction is obtained by SHAP analysis, i.e., the Shapley value. The SHAP summary plot, which ranks the importance of risk factors, and the SHAP force plot, which analyzes and interprets the prediction results of individual samples, are constructed based on the Shapley values.

3. Results

3.1. Basic clinical information of the patient

A total of 1,163 patients were included in the study (Figure 1), including 142 (12.21%) patients with permanent stomas.

Figure 1

3.2. Analysis of risk factors for permanent stoma in patients with rectal cancer

The results of univariate analysis showed that there was a significant difference between the permanent stoma group and the nonpermanent stoma group in terms of age, history of hypertension, history of diabetes mellitus, tumor recurrence, history of adjuvant radiotherapy, history of adjuvant chemotherapy, distance of the tumor from the dentate line, and whether there was rectal stenosis (P < 0.05). The results of multivariate analysis showed that age ≥65 years, history of hypertension, history of diabetes mellitus, history of adjuvant radiotherapy, history of adjuvant chemotherapy, tumor distance ≥5 cm from the dentate line, and rectal stenosis were independent influencing factors for permanent stoma in patients with rectal cancer (P < 0.05) (Table 1).

3.3. Model building and evaluation

The ROC curve results show that XGBoost has an AUC value as high as 0.987 in the training set; the AUC value in the validation set is 0.963, which is the best performance among the four models (Figures 2A,B, Table 2). The calibration curve results show that the calibration curves of the four models are similar to the ideal curves, and the models have high consistency between the predicted and actual results (Figure 2C). The DCA curves showed that all four models achieved a net clinical benefit relative to either the full treatment or no treatment plan (Figure 2D). The k-fold cross-validation method was used to compare the generalization ability of the four models. Taking the test set N = 256 cases (30.08%) and the remaining samples as the training set for 10-fold cross-validation, AUC = 0.9456 ± 0.0367 in the validation set of XGBoost, AUC = 0.9088 in the test set, accuracy = 0.9102 (Figures 3A–C); AUC = 0.9025 ± 0.0627 in the validation set of RF, AUC = 0.8789 in the test set, accuracy = 0.8750; AUC = 0.9274 ± 0.0440 in the validation set of SVM, AUC = 0.9006 in the test set, accuracy = 0.9414; AUC = 0.8720 ± 0.0713 in the validation set of KNN, AUC = 0.8743 in the test set, accuracy = 0.9336. After a comprehensive comparison, the XGBoost algorithm was chosen to construct the model in this study.

Figure 2

Figure 3

3.4. Model external validation

The results of the ROC curve showed that the AUC value of the external validation set was 0.89, indicating that the prediction model was highly accurate in determining the disease (Figure 3D).

3.5. Model explanation

The SHAP summary plot results showed that the risk factors for permanent stoma in patients with rectal cancer were ranked as tumor distance from the dentate line ≥5 cm, history of adjuvant chemotherapy, rectal stricture, history of diabetes mellitus, history of hypertension, history of adjuvant radiotherapy, and age ≥65 years (Figure 4). The SHAP force plot shows the predictive analysis of the study model for four patients with rectal cancer with permanent stoma. The model predicts a 0.052 probability of permanent stoma in patient I, with an increased probability of chemotherapy and rectal stenosis and a decreased probability of age <65 years; the model predicts a 0.291 probability of permanent stoma in patient II, with an increased probability of history of hypertension, chemotherapy, and rectal stenosis and a decreased probability of no history of diabetes; the model predicts a 0.964 probability of permanent stoma in patient III, and the probability was increased by history of chemotherapy, history of hypertension, rectal stenosis, and tumor <5 cm from the dentate line, and decreased by the patient's lack of diabetes; the model predicted a 0.002 probability of permanent stoma in patient IV, and decreased by the patient's lack of diabetes (Figures 5A–D).

Figure 4

Figure 5

4. Discussion

The current study evaluated risk prediction models constructed by four machine learning algorithms. Among them, the XGBoost algorithm exhibited the highest accuracy and was efficient, flexible, and universally adaptable (13). Compared with the RF algorithm, the XGBoost algorithm takes full account of the regularization problem and can effectively avoid model overfitting. The SVM algorithm and KNN algorithm have higher accuracy and can avoid the problem of overfitting well, but the stability of the two algorithms is poor when solving problems with multiple features and large samples (14). The XGBoost algorithm is more suitable for multidimensional studies and reduces the computation and training time. Compared with the SVM algorithm and KNN algorithm, the XGBoost algorithm is more advantageous. Therefore, with a comprehensive comparison of four machine learning algorithms, this study chose to use the XGBoost algorithm to construct a model to predict permanent stoma in patients with rectal cancer. Some studies (15, 16) have validated the effectiveness of machine learning algorithm applications in clinical diagnosis as well as prognosis. Moreover, machine learning techniques can also accurately predict adverse outcomes in disease progression compared to traditional diagnostic methods. Machine learning algorithms also played a great role in building the prediction model in this study. The model in this study can help clinical decision makers accurately identify high-risk patients, provide timely interventional treatment and improve patient prognosis. On the other hand, the model can help medical institutions allocate medical resources rationally, focus on the vital signs of high-risk patients, and effectively improve the survival rate of rectal cancer patients. Moreover, this study also used SHAP analysis to explain the model, and the results showed that advanced age, distance of the tumor from the dentate line, rectal stenosis, history of diabetes, history of hypertension, history of adjuvant chemotherapy, and history of adjuvant radiotherapy were risk factors for permanent stoma in patients with rectal cancer.

Patients of advanced age and those with a history of underlying medical conditions such as hypertension and diabetes mellitus are in poor physical condition. Patients have sclerotic and poorly dilated vascular walls that are friable (17–19). Additionally, their coagulation function is altered to some extent (20), which prevents rapid physiological hemostasis. All these factors lead to a weakened blood supply to the patient's gastrointestinal tract and a greater risk of postoperative anastomotic leakage. Some elderly patients do not have high quality of life requirements and have their own concerns about secondary surgery, so clinicians may prefer to consider permanent stoma for such patients. In addition, four samples were used in this study to explain how the model predicted permanent ostomy. For example, in the disease prediction analysis of samples I, II and III, a history of chemotherapy was one of the significant risk factors. Chemotherapy is often used as a treatment option for malignant tumors such as rectal cancer and liver cancer, and is highly selective to maximize inhibition of tumor growth and spread (21, 22). However, some of the chemotherapy modalities are more irritating to the abdominal cavity and aggravate the degree of abdominal adhesions. It also inhibits the normal physiological function of the bone marrow, and patients experience postoperative anemia and immune dysfunction, which affects the near and long-term outcome. On the other hand, patients can have severe abdominal inflammatory reactions as well as gastrointestinal reactions, which do not prevent the occurrence of intestinal obstruction (23). Studies by Makrin et al. (24, 25) also demonstrated that intraperitoneal chemotherapy has a detrimental effect on the recovery of the gastrointestinal tract and is most likely one of the risk factors for postoperative intestinal obstruction, further confirming the higher risk of permanent stoma in patients treated with chemotherapy. The current study also found that radiotherapy is one of the risk factors for the permanence of stoma in patients. Patients with rectal cancer choose radiotherapy to reduce the postoperative recurrence rate of the tumor and to improve survival. Zhu et al. (26, 27) discussed the effectiveness of postoperative radiotherapy for rectal cancer and demonstrated that this treatment modality can significantly reduce the local recurrence rate of rectal cancer and improve the quality of survival of patients. However, as with intraperitoneal chemotherapy, radiation therapy is more damaging to the gastrointestinal tract. On the one hand, it restricts its peristaltic function and causes stiffness of the intestinal wall, which in the long run decreases the compliance of the intestine; on the other hand, radiotherapy directly damages intestinal epithelial cells and vascular endothelial cells, and the intestinal wall gradually fibroses. Some studies (28) performed pathological biopsies of tumors in most patients treated with adjuvant radiotherapy and found reduced microvascular counts at the tumor cut edges, an increased percentage of stenotic vessels, and significant fibrosis of the surrounding intestinal wall. Additionally, a study by Kumagai et al. (29) showed that radiotherapy is highly susceptible to complications of intestinal perforation and intestinal obstruction, and these complications increase the risk of permanent stoma to some extent. The results of the current study also revealed a greater risk of perpetuation of the diseased artificial orifice in rectal stenosis. TME carries out a radical tumor resection with complete removal of the mesentery around the rectal cancer, which requires the operator to ligate at the beginning of the inferior mesenteric artery, which is highly likely to cause the intestinal canal at the anastomosis to be in a hypoxic and hypoperfused state. If patients with rectal stenosis remain in this state for a long time, the fibrous tissue of the intestinal canal will further proliferate, leading to severe postoperative bowel obstruction (27). Such patients not only feel severe pain at the anus during defecation but are also unable to undergo ostomy reversal, which further aggravates physical and psychological trauma. This suggests that clinicians should closely monitor the patient's vital status before surgery, prepare the intestine and prevent the occurrence of stenosis; select a reasonable surgical approach during surgery, fully free the colon to the splenic area to reduce anastomotic tension and ensure a good blood supply to the anastomosis; and perform regular postoperative rectal finger examination to dilate the anus and loosen the stenotic ring. Doctors can use balloon dilation to reduce the degree of stenosis when dealing with patients with mild stenosis symptoms; when dealing with patients with more severe stenosis symptoms, they should promptly inform patients of their condition, communicate well with doctors and patients, and reduce the psychological burden of patients. In addition, Mak (21) considered local recurrence of tumors as a risk factor for stoma permanence. His study showed that recurrent tumors occupy the intestinal space, making it more difficult for food to pass normally through the intestinal lumen, which can easily cause intestinal obstruction. Moreover, tumor infiltration of the intestinal canal causes stiffness of the intestinal wall, which weakens peristalsis and aggravates the degree of obstruction, so patients are highly susceptible to permanent stoma (21, 30, 31). However, in the present study, tumor recurrence was not a risk factor for permanent stoma in patients with rectal cancer. Our analysis suggests that this may be related to the small number of cases of rectal cancer recurrence in this study, and more relevant cases will be added in the future to improve the study. It also suggests that clinicians should strengthen the postoperative follow-up of rectal cancer patients, promptly review them after discovering discomfort, and intervene early to relieve symptoms.

The current study comprehensively evaluated the model in terms of discrimination, calibration, and clinical utility, but the study has some limitations. The study included multiple aspects of risk factors but did not consider aspects such as imaging. Although the machine learning algorithms were more accurate, their models were more complex and less interpretable. The entire computational and decision-making process of the model runs in a black box, which is not as intuitive and clear as the logistic regression model (32–34). On the other hand, the current study was a single-center retrospective study, which has the disadvantages of selection bias, distribution bias, and retrospective bias. It is necessary to add multicenter prospective studies to future studies to further increase the reliability of the results.

5. Conclusion

This study developed a model based on the XGBoost machine learning algorithm to predict the risk of permanent stoma in rectal cancer. The model has good prediction accuracy and clinical utility, which facilitates surgeons in diagnosing patients in a timely manner. The model predicted patients at high risk for permanent stoma and identified advanced age, distance of the tumor from the dentate line, rectal stenosis, history of diabetes, history of hypertension, history of chemotherapy, and history of radiotherapy as high risk factors.

References

• 1.

  MattiuzziCSanchis-GomarFLippiG. Concise update on colorectal cancer epidemiology. Ann Transl Med. (2019) 7(21):609. 10.21037/atm.2019.07.91

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 2.

  São JuliãoGPHabr-GamaAVailatiBBAraujoSEAFernandezLMPerezRO. New strategies in rectal cancer. Surg Clin North Am. (2017) 97(3):587–604. 10.1016/j.suc.2017.01.008

  • CrossRef
  • Google Scholar
  • View reference in article

• 3.

  HealdRJRyallRD. Recurrence and survival after total mesorectal excision for rectal cancer. Lancet. (1986) 1(8496):1479–82. 10.1016/S0140-6736(86)91510-2

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 4.

  KhalfallahMDougazWJerrayaHNouiraRBouaskerIDziriC. Prognostic factors in rectal cancer: where is the evidence?Tunis Med. (2017) 95(2):79–86.

  • Pubmed Abstract
  • Google Scholar

• 5.

  AllaixMEGiraudoGFerrareseAArezzoARebecchiFMorinoM. 10-Year oncologic outcomes after laparoscopic or open total mesorectal excision for rectal cancer. World J Surg. (2016) 40(12):3052–62. 10.1007/s00268-016-3631-x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 6.

  BedrikovetskiSDudi-VenkataNNKroonHMMooreJWHunterRASammourT. Outcomes of minimally invasive versus open proctectomy for rectal cancer: a propensity-matched analysis of bi-national colorectal cancer audit data. Dis Colon Rectum. (2020) 63(6):778–87. 10.1097/DCR.0000000000001654

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 7.

  FoppaCNgSCMontorsiMSpinelliA. Anastomotic leak in colorectal cancer patients: new insights and perspectives. Eur J Surg Oncol. (2020) 46(6):943–54. 10.1016/j.ejso.2020.02.027

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 8.

  NäsvallPDahlstrandULöwenmarkTRutegårdJGunnarssonUStrigårdK. Quality of life in patients with a permanent stoma after rectal cancer surgery. Qual Life Res. (2017) 26(1):55–64. 10.1007/s11136-016-1367-6

  • CrossRef
  • Google Scholar
  • View reference in article

• 9.

  BianchiRMamadou-PathéBvon KänelRRothRSchreinerPRosselJBet alEffect of closed and permanent stoma on disease course, psychological well-being and working capacity in Swiss IBD cohort study patients. PLoS ONE. (2022) 17(9):e0274665. 10.1371/journal.pone.0274665

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 10.

  DeoRC. Machine learning in medicine. Circulation. (2015) 132(20):1920–30. 10.1161/CIRCULATIONAHA.115.001593

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 11.

  KatiyarAMohantyAHuaJChaoSLopesRDattaAet alA Bayesian approach to determine the composition of heterogeneous cancer tissue. BMC Bioinformatics. (2018) 19(Suppl 3):90. 10.1186/s12859-018-2062-0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 12.

  CypkoMAStoehrMOeltze-JafraSDietzALemkeHU. A guide for constructing Bayesian network graphs of cancer treatment decisions. Stud Health Technol Inform. (2017) 245:1355.

  • Pubmed Abstract
  • Google Scholar
  • View reference in article

• 13.

  WangLWangXChenAJinXCheH. Prediction of type 2 diabetes risk and its effect evaluation based on the XGBoost model. Healthcare (Basel). (2020) 8(3). 10.3390/healthcare8030247

  • CrossRef
  • Google Scholar

• 14.

  HaoPYChiangJHChenYD. Possibilistic classification by support vector networks. Neural Netw. (2022) 149:40–56. 10.1016/j.neunet.2022.02.007

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 15.

  KaradaghyOAShewMNewJBurAM. Development and assessment of a machine learning model to help predict survival among patients with oral squamous cell carcinoma. JAMA Otolaryngol Head Neck Surg. (2019) 145(12):1115–20. 10.1001/jamaoto.2019.0981

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 16.

  TsengYJWangHYLinTWLuJJHsiehCHLiaoCT. Development of a machine learning model for survival risk stratification of patients with advanced oral cancer. JAMA Netw Open. (2020) 3(8):e2011768. 10.1001/jamanetworkopen.2020.11768

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 17.

  KurybaAJScottNAHillJvan der MeulenJHWalkerK. Determinants of stoma reversal in rectal cancer patients who had an anterior resection between 2009 and 2012 in the English national health service. Colorectal Dis. (2016) 18(6):O199–205. 10.1111/codi.13339

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 18.

  den DulkMSmitMPeetersKCKranenbargEMRuttenHJWiggersTet alA multivariate analysis of limiting factors for stoma reversal in patients with rectal cancer entered into the total mesorectal excision (TME) trial: a retrospective study. Lancet Oncol. (2007) 8(4):297–303. 10.1016/S1470-2045(07)70047-5

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 19.

  KairaluomaMRissanenHKulttiVMecklinJPKellokumpuI. Outcome of temporary stomas. A prospective study of temporary intestinal stomas constructed between 1989 and 1996. Dig Surg. (2002) 19(1):45–51. 10.1159/000052005

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 20.

  AfshinAForouzanfarMHReitsmaMBSurPEstepKLeeAet alHealth effects of overweight and obesity in 195 countries over 25 years. N Engl J Med. (2017) 377(1):13–27. 10.1056/NEJMoa1614362

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 21.

  MakJCKFooDCCWeiRLawWL. Sphincter-preserving surgery for low rectal cancers: incidence and risk factors for permanent stoma. World J Surg. (2017) 41(11):2912–22. 10.1007/s00268-017-4090-8

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 22.

  KimMJKimYSParkSCSohnDKKimDYChangHJet alRisk factors for permanent stoma after rectal cancer surgery with temporary ileostomy. Surgery. (2016) 159(3):721–7. 10.1016/j.surg.2015.09.011

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 23.

  SongOKimKHLeeSYKimCHKimYJKimHR. Risk factors of stoma re-creation after closure of diverting ileostomy in patients with rectal cancer who underwent low anterior resection or intersphincteric resection with loop ileostomy. Ann Surg Treat Res. (2018) 94(4):203–8. 10.4174/astr.2018.94.4.203

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 24.

  MakrinVLev-CheloucheDEven SapirEParanHRabauMGutmanM. Intraperitoneal heated chemotherapy affects healing of experimental colonic anastomosis: an animal study. J Surg Oncol. (2005) 89(1):18–22. 10.1002/jso.20161

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 25.

  MehtaPPGriffinJGantaSRangrajMSteichenF. Laparoscopic-assisted colon resections: long-term results and survival. Jsls. (2005) 9(2):184–8.

  • Pubmed Abstract
  • Google Scholar

• 26.

  LiCCLiangJAChungCYChenWTChienCR. Effectiveness of intensity-modulated radiotherapy for rectal cancer patients treated with neoadjuvant concurrent chemoradiotherapy: a population-based propensity score-matched analysis. Anticancer Res. (2019) 39(3):1479–84. 10.21873/anticanres.13265

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 27.

  ZhuHBaiBShanLWangXChenMMaoWet alPreoperative radiotherapy for patients with rectal cancer: a risk factor for non-reversal of ileostomy caused by stenosis or stiffness proximal to colorectal anastomosis. Oncotarget. (2017) 8(59):100746–53. 10.18632/oncotarget.20602

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 28.

  ZhongQHWuPHQinQYKuangYYMaTHWangHMet alPathological insights of radiotherapy-related damage to surgical margin after preoperative radiotherapy in patients with rectal cancer. Zhonghua Wai Ke Za Zhi. (2017) 55(7):507–14. 10.3760/cma.j.issn.0529-5815.2017.07.007

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 29.

  KumagaiTRahmanFSmithAM. The microbiome and radiation induced-bowel injury: evidence for potential mechanistic role in disease pathogenesis. Nutrients. (2018) 10(10). 10.3390/nu10101405

  • CrossRef
  • Google Scholar

• 30.

  LimSWKimHJKimCHHuhJWKimYJKimHR. Risk factors for permanent stoma after low anterior resection for rectal cancer. Langenbecks Arch Surg. (2013) 398(2):259–64. 10.1007/s00423-012-1038-1

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 31.

  JungingerTGönnerUTrinhTTLollertAOberholzerKBerresM. Permanent stoma after low anterior resection for rectal cancer. Dis Colon Rectum. (2010) 53(q):1632–9. 10.1007/DCR.0b013e3181ed0aae

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 32.

  RudinC. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat Mach Intell. (2019) 1(5):206–15. 10.1038/s42256-019-0048-x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 33.

  NoharaYIiharaKNakashimaN. Interpretable machine learning techniques for causal inference using balancing scores as meta-features. Annu Int Conf IEEE Eng Med Biol Soc. (2018) 2018:4042–5. 10.1109/embc.2018.8513026

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 34.

  Figueroa BarrazaJLópez DroguettEMartinsMR. Towards interpretable deep learning: a feature selection framework for prognostics and health management using deep neural networks. Sensors (Basel). (2021) 21(17). 10.3390/s21175888

  • Pubmed Abstract
  • CrossRef
  • Google Scholar