Machine Learning for the Prediction of Complications in Patients After Mitral Valve Surgery

Background: This study intended to use a machine learning model to identify critical preoperative and intraoperative variables and predict the risk of several severe complications (myocardial infarction, stroke, renal failure, and hospital mortality) after cardiac valvular surgery. Study Design and Methods: A total of 1,488 patients undergoing cardiac valvular surgery in eight large tertiary hospitals in China were examined. Fifty-four perioperative variables, such as essential demographic characteristics, concomitant disease, preoperative laboratory indicators, operation type, and intraoperative information, were collected. Machine learning models were developed and validated by 10-fold cross-validation. In each fold, Recursive Feature Elimination was used to select key variables. Ten machine learning models and logistic regression were developed. The area under the receiver operating characteristic (AUROC), accuracy (ACC), Youden index, sensitivity, specificity, F1-score, positive predictive value (PPV), and negative predictive value (NPV) were used to compare the prediction performance of different models. The SHapley Additive ex Planations package was applied to interpret the best machine learning model. Finally, a model was trained on the whole dataset with the merged key variables, and a web tool was created for clinicians to use. Results: In this study, 14 vital variables, namely, intraoperative total input, intraoperative blood loss, intraoperative colloid bolus, Classification of New York Heart Association (NYHA) heart function, preoperative hemoglobin (Hb), preoperative platelet (PLT), age, preoperative fibrinogen (FIB), intraoperative minimum red blood cell volume (Hct), body mass index (BMI), creatinine, preoperative Hct, intraoperative minimum Hb, and intraoperative autologous blood, were finally selected. The eXtreme Gradient Boosting algorithms (XGBOOST) algorithm model presented a significantly better predictive performance (AUROC: 0.90) than the other models (ACC: 81%, Youden index: 70%, sensitivity: 89%, specificity: 81%, F1-score:0.26, PPV: 15%, and NPV: 99%). Conclusion: A model for predicting several severe complications after cardiac valvular surgery was successfully developed using a machine learning algorithm based on 14 perioperative variables, which could guide clinical physicians to take appropriate preventive measures and diminish the complications for patients at high risk.


INTRODUCTION
The prevalence of unhealthy lifestyles, such as long-term highfat diet and lack of exercise, has caused the higher and higher incidence of cardiac diseases. Patients with cardiac diseases will suffer serious morbidity and mortality without reasonable interventions, which increased the number of cardiac surgery significantly. It was discovered that more than 1 million patients with heart disease need to be treated with cardiac surgery every year worldwide (1). There has also been a sharp increase in the number of patients with valvular diseases, many of which are severe and must be treated with cardiac surgery to replace insufficient valves (2,3). A large number of cardiac patients always along with various of complications after cardiac valvular surgery, these postoperative complications mainly including myocardial infarction, stroke, acute renal failure, death, and so on (4).
The high incidence of postoperative complications in cardiac surgery plays an important role in the exacerbation of hospital stay and hospitalization cost, reducing the quality of life and even elevating mortality after cardiac surgery (5). An eligible surgical treatment involves not only a smooth operation but also the early prediction of risks, provision of appropriate recommendations, and timely adoption of effective medical measures to avoid postoperative complications (6).
The most important process for a qualified medical treatment is the early prediction of postoperative complications (7). Clinicians generally give judgments whether patients have postoperative complications mainly based on the tests of clinical laboratory and examinations, or their clinical experiences, when patients have corresponding clinical indications after cardiac surgery (8). On the one hand, the tests or examinations for postoperative complications are time-sensitive; on the other hand, clinical experience is subjective, and many young clinicians do not have mature clinical experience. Based on the above situation, patients who underwent cardiac surgery always miss the optimal treatment window for postoperative complications. Thus, it is urgent to construct a risk predictive system that could implement the best outcome for patients.
Previous studies on predicting postoperative risks after cardiac surgery mainly on account of traditional stastics methods, such as linear models or logistic regression (9). However, these traditional methods usually focus on one or few clinical indicators. More and more studies have found that the preoperative and intraoperative indicators of a patient have an impact on the outcome of the patient (10). Meanwhile, many studies have proved that a prediction model based on machine learning has high accuracy in predicting clinical outcomes (11,12). Therefore, we aim to construct a model based on machine learning to predict the postoperative outcomes of patients using various preoperative and intraoperative indicators, so as to provide theoretical guidance for clinical practice.
The purpose of this study was to determine the preoperative and intraoperative risk factors associated with postoperative complications in patients undergoing cardiac valvular surgery and to develop a machine learning model to predict postoperative complications.

Study Subjects
Participants were patients aged more than 18 years but <75 who underwent cardiac valvular surgery (mitral valve replacement, mitral valvuloplasty, and tricuspid valvuloplasty) from January 2016 to December 2018 at one of the following eight tertiary hospitals: the Second Xiangya Hospital of Central South University, the Third Xiangya Hospital of Central South University, Beijing Aerospace General Hospital, Qilu Hospital of Shandong University, Fuwai Hospital National Center for Cardiovascular Diseases, Zhejiang Provincial People's Hospital, the Affiliated Hospital of Southwest Medical University, and Xiamen Cardiovascular Hospital Xiamen University. We collected 38 cases of biological valve replacement from the Third Xiangya Hospital from 2019 to 2020 for verification.
The types of surgery for cardiac valvular surgery in our study include the classical mitral valve replacement, mitral valvuloplasty, and tricuspid valvuloplasty. Since the four types of surgery account for the majority of the population, only these three procedures were included in this study.
Patients who underwent other types of surgery (coronary artery bypass grafting, CABG, atrial septal defect repair, etc.), recardiac surgery, or emergency surgery, and those whose missing rates of data were more than 80% were excluded.
Postoperative myocardial infarction, postoperative stroke, postoperative renal failure, and postoperative hospital mortality that occurred 48 h after the initial surgery were defined as relevant outcomes; Then we labeled patients who had at least one complication as "complication occurred" and patients who did not have any complication as "complication did not occur." Approval was obtained from the institutional review board of the Third Xiangya Hospital of Central South University for this study (NCT03885570). The study was reported according to the recommendations of the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement. No written consent was required in view of the purely observational nature of the study.

Statistical Analysis
Continuous variables between complication and noncomplication groups were compared by either the Student t-test or the rank-sum test as appropriate. The chi-square test or Fisher's exact test was performed to compare the differences in the categorical variables.
Then, the recursive feature elimination (RFE) algorithm was used to identify crucial variables, and we developed a machine learning model named eXtreme Gradient Boosting (XGBOOST) (13)(14)(15). In brief, RFE is a feature selection way that recursively fits a model derived from smaller feature sets until a specified termination criterion is reached. In each loop, features are graded by their importance in the trained model. By recursively eliminating one feature with the lowest importance, RFE intends to eliminate dependencies and collinearity that maybe existing in the model. Lastly, the most important features were screened out, and the XGBOOST model was developed based on the feature set. Other features were not included, because they only brought a small increment in AUROC but significantly increased the difficulty of model applications. The proposed prediction model was built in the XGBoost package in Python language, and it was carried out using the 10-fold cross-validation method, and then the AUROC was calculated.
Besides, 10 other models, CatBoost, LightGBM, MLP, SVM, LR, Random Forest, Gradiant Boosting, KNN, AdaBoost, and Naive Bayes, were developed and compared with the proposed machine learning model. These models were also developed and validated by 10-fold cross-validation, and then the AUCs were calculated. The accuracy (ACC), Youden index, sensitivity, specificity F1 score, positive predictive value (PPV), and negative predictive value (NPV) were also analyzed.
Finally, the key variables identified by REF in each fold were merged, and the 15 most important variables were selected. The XGBOOST model was trained on the whole dataset using the merged variables. After the model was established, the SHapley Additive exPlanations (SHAP) package in Python was used to explain the model by analyzing two cases. The SHAP package interpreted the output of the machine learning model using a game-theoretic approach (16). For each prediction sample, the model connected optimal credit allocation with local explanations. Besides, a web tool was created for clinicians to use our model.

Study Population
As Figure 1 demonstrates, 1,488 patients were finally included in this study, and the preoperative information of the cohort is described in Table 1. The average age of the patients was 52.59 years, men accounted for 39.05%, and the average BMI was 22.84. In the complication occurred cohort, 12.73% of the patients died in the hospital, 61.82% of the patients had a myocardial infarction after the operation, 30.91% of the patients had a stroke, and 74.55% of the patients had renal failure after the operation.

Key Variables
Fifteen variables, namely, intraoperative total input, intraoperative blood loss, intraoperative colloid bolus, NYHA, preoperative Hb, preoperative PLT, age, preoperative FIB, intraoperative minimum Hct, BMI, preoperative creatinine, preoperative Hct, intraoperative minimum Hb, intraoperative, and autologous blood were selected as crucial variables using the RFE algorithm. As expected, the patients had less intraoperative total output, hypertension, higher preoperative FIB, less intraoperative total input, higher preoperative creatinine, less intraoperative autologous blood, higher NYHA score, older age, higher intraoperative minimum HCT, lower preoperative Hb, lower preoperative PLT, lower intraoperative infusion volume, higher intraoperative minimum Hb, lower preoperative HCT, higher BMI, and lower intraoperative blood loss. After identifying the 15 variables, machine learning was used to predict several severe complications after cardiac valvular surgery. As shown in Figure 2, the AUC of the proposed model is 0.9. The proposed model significantly outperformed the conventional LR (AUC: 0.74) and seven other machine learning models. As described in Table 2

Application of the Model
The SHAP package analyzed the entire cohort, and showed the impact of each variable on predicting complications (Figure 3). The preoperative and intraoperation information of a patient was inputted into the model: age 61 years, BMI 23.44 kg/m 2 , NYHA 2, intraoperative blood loss 360 ml, intraoperative colloid infusion 3,000 ml, intraoperative total input 4,350 ml,  intraoperative autologous blood collection 120 ml, preoperative Hb 143 g/l, intraoperative minimum Hb 57 g/l, preoperative Hct 43.1%, intraoperative minimum Hct 17%, preoperative PLT 85 * 10 9 /l, preoperative creatinine 80.21, and preoperative FIB 2.82 g/l. The model analyzed that the risk of adverse events in this patient was 92.4%, indicating that the probability of severe complications for the patients was high (Figure 4A, Example 1). The preoperative and intraoperation information of another patient was inputted into the model: age 42 years, BMI 22.89 kg/m 2 , NYHA 4, intraoperative blood loss 800 ml, intraoperative colloid infusion 300 ml, intraoperative total input 2,400 ml, intraoperative autologous blood collection 0 ml, preoperative Hb 88 g/L, intraoperative minimum Hb 81 g/l, preoperative Hct 31%, intraoperative minimum Hct 81%, preoperative PLT 258 * 10 9 /l, preoperative creatinine 65.2 µmoI/l, and preoperative FIB 2.6 g/l. The predicted probability of adverse events in this patient was 5.3%, indicating that the patient had a good outcome (Figure 4B, Example 2). Furthermore, a website was established for clinicians to use the proposed model, http://www.aimedicallab.com/tool/aimlvalvecomp.html. As shown in Supplementary Figure 1, the predicted probabilities are significantly different between the positive and negative groups. If we use 50% as a cut off, our model will achieve a 100% accuracy.

DISCUSSION
As being mentioned above, the incidence of postoperative complications can be declined with an eligible medical treatment, including a smooth operation, the early prediction of postoperative risks, the provision of appropriate recommendations, and the timely adoption of effective medical measures, which has been explored by many researches (6). In our study, the incidence of mortality in patients with postoperative complications was 7%, and the incidence of mortality in all cardiac surgery patients was 0.4%. The incidence of postoperative complications in all patients was 6.7%. Based on previous studies, there is a general convergence of the incidence of postoperative complications among different hospitals. Some high-turnover institutions have low mortality rates and may be associated with higher complication rates (17). In other words, surgical patients in high-turnover facilities may experience one or more postoperative complications, but they have a low mortality rate, because these facilities have a higher rate of rescue success (18). Based on the above research studies, the ability to predict, identify, prepare, and implement the management of postoperative risks are vitally important to improve the outcomes of patients. In previous investigations, several kinds of statistic means have been discovered to figure out outcomes, analyze manifestation, and construct models for improving the outcomes of cardiac surgery (19,20). Researchers who constructed these models were primarily interested in postoperative mortality and rarely predicted other alternative outcomes, such as postoperative complications, so it is really vital to develop an effective measuring system to predict postoperative outcomes. In this study, we introduced machine learning to build the prediction model.
One of the most significant aspect is increasing the area under the receiver operating characteristic (AUROC) curves of predictive models. The AUROC of traditional predictive models is no more than 0.8 or even lower (21,22). In our research, the model based on machine learning exhibited a perfect performance. Different methods were used to prove that our prediction model has a good predictive effect on several different postoperative complications, all of the AUROCs were more than 0.8, some of which even reached 0.9. This proved that our model has a fantastic predictive effect on postoperative complications.
We also implemented two examples into our predictive model to confirm what variables were important to the predictive model, which can provide guidance for clinicians in making medical decisions, such as how to manage the cardiac surgery. In this study, we identified 14 key indicators that had a significant impact on clinical outcomes, suggesting that clinicians should take care changes in some important variables, such as NYHA, blood loss, and creatinine (23,24). This research also found that clinicians should pay close attention to changes in blood clotting function and kidney function of cardiac surgery patients. Most importantly, it can indicate to clinicians how likely a patient is to develop complications after cardiac surgery. Based on the above model, we also built an online open website. We can easily obtain the incidence of postoperative complications for a particular patient by entering several important variables in the corresponding column of this website. It is proved that the accuracy of our prediction model is very high, which can provide guidance for clinicians.
Inevitably, our research still leaves some to be desired. On the one hand, this study was a retrospective study with selection bias and confounding factors. We have enhanced the reliability of our results by incorporating multicenter data and performing robust cross-validation. At the same time, we will add prospective studies to our future studies to reduce these errors. A randomized controlled trial associated with this research should be conducted. However, the design of this type of RCT remains unclear. On the other hand, the entire process of machine learning to complete tasks operates in a black box, lacks interpretability, and is not as intuitive and clear as traditional linear models. Our results showed that the machine learning model had incomparable prediction efficiency compared with traditional linear model prediction. This study did not include patients undergoing minimally invasive mitral valve replacement, and this algorithm is not applicable to such patients. Because of the increasing use of this surgical procedure, we will include such patients in subsequent studies. Meanwhile, we have implemented a web page to promote clinical application, which is actually very meaningful and convenient.
In this study, a postoperative complication prediction model after cardiac surgery was exploited based on a machine learning algorithm, with a splendid prediction performance and convenient implementation. This model has the ability to recognize minimal risk of postoperative complications. Meanwhile, the best outcomes of patient prognosis can be achieved through an individualized assessment system. To reduce selection bias, a prospective management database for surgery patients should be built. Based on preoperative and intraoperative variables, machine learning models can be constructed and validated by the variables of surgery patients in the future. Last but not least, to measure the performance of machine learning models, a randomized controlled trial associated with this research should be conducted. It can provide suggestions for clinical work, and reduce the risk of patients and improve patient outcomes.

DATA AVAILABILITY STATEMENT
The datasets presented in this article are not readily available because. The Ethics Committee did not agree to disclose the data in a public database. Requests to access the datasets should be directed to Leping Liu, 1105380949@qq.com.

ETHICS STATEMENT
The studies involving human participants were reviewed and approved by the Institutional Review Board of the Third Xiangya Hospital of Central South University (NCT03885570). Written informed consent was not required for this study, in accordance with the local legislation and institutional requirements.

AUTHOR CONTRIBUTIONS
HJia, YW, HJi, XM, JW, YH, and XW: clinical data collection. HJia and BC: data analysis. RG, HJia, and LL: writing of the article. RG: designing of the study. QZ: construction of the model. All authors contributed to the article and approved the submitted version.