Predicting the Stone-Free Status of Percutaneous Nephrolithotomy With the Machine Learning System: Comparative Analysis With Guy’s Stone Score and the S.T.O.N.E Score System
Hong Zhao
Wanling Li2 - 1Shanghai Xuhui Central Hospital, Shanghai, China
- 2Zhongshan Hospital, Fudan University, Shanghai, China
Purpose: The aim of the study was to use machine learning methods (MLMs) to predict the stone-free status after percutaneous nephrolithotomy (PCNL). We compared the performance of this system with Guy’s stone score and the S.T.O.N.E score system.
Materials and Methods: Data from 222 patients (90 females, 41%) who underwent PCNL at our center were used. Twenty-six parameters, including individual variables, renal and stone factors, and surgical factors were used as input data for MLMs. We evaluated the efficacy of four different techniques: Lasso-logistic (LL), random forest (RF), support vector machine (SVM), and Naive Bayes. The model performance was evaluated using the area under the curve (AUC) and compared with that of Guy’s stone score and the S.T.O.N.E score system.
Results: The overall stone-free rate was 50% (111/222). To predict the stone-free status, all receiver operating characteristic curves of the four MLMs were above the curve for Guy’s stone score. The AUCs of LL, RF, SVM, and Naive Bayes were 0.879, 0.803, 0.818, and 0.803, respectively. These values were higher than the AUC of Guy’s score system, 0.800. The accuracies of the MLMs (0.803% to 0.818%) were also superior to the S.T.O.N.E score system (0.788%). Among the MLMs, Lasso-logistic showed the most favorable AUC.
Conclusion: Machine learning methods can predict the stone-free rate with AUCs not inferior to those of Guy’s stone score and the S.T.O.N.E score system.
Introduction
Since the first description of the technique in 1976 (Fernstro¨m and Johansson, 1976), percutaneous nephrolithotomy has been widespread for the treatment of renal calculi. It is the golden standard for the treatment of 2-cm kidney stones (Miernik et al., 2014). PCNL’s success rate is between 56% and 96% in various series (Matlaga et al., 2005; Akman et al., 2011; Rosette et al., 2011; Labadie et al., 2015). Many factors contribute to the success of stone clearance including the stone size, location, number, and grade of hydronephrosis, as well as surgeon’s experience. To predict the outcomes after PCNL, several scoring systems have been devised including Guy’s stone score, S.T.O.N.E nephrolithometry system, CROES nephrolithometry nomogram, and S-ReSC score (Al Adl et al., 2020). Guy’s stone score is easy to apply and has been validated in multiple studies. The S.T.O.N.E. score is based on factors determined through CT imaging, which is the currently preferred imaging modality for patients with nephrolithiasis (Noureldin et al., 2015a). The CROES nomogram was developed from data in a large multicenter database and has high statistical power. Determination of the S-ReSC score relies on stone location only, providing a simple approach to grading disease complexity (Noureldin et al., 2015b). Each system has advantages and disadvantages, but several studies suggest that their ability to predict the stone-free rate is comparable (Wu and Okeke, 2017).
Machine learning techniques have been used extensively in the field of clinical medicine, especially when used for the construction of prediction models. The outperformance of ML over conventional data analysis models has been shown in the urology-oncology literature (Hung et al., 2018; Andras et al., 2020; Rodrigo. et al., 2020; Ström et al., 2020).
In predicting post-lithotripsy outcomes with machine learning, there are only three studies published until now (De Perrot et al., 2019; Tayyebe. et al., 2019; Aminsharifi et al., 2020). Aminsharifi et al. (2020) first used the machine learning method for predicting post-PCNL outcomes compared to current scoring systems. They found machine learning-based software was superior in predicting SFS after PCNL, with an AUC of 0.915 compared to 0.615 (GSS) and 0.621 (CROES nomograms) (p < 0.01). More than 20 variables of 146 patients were inputted for the training of machine learning in their study. Alireza used a support vector machine (SVM) as the machine learning technique. We know that the machine learning algorithm includes some other methods, such as decision trees, random forests, artificial neural networks, Bayesian learning, Deep Learning, and so on. In this study, we used four machine learning methods (Lasso logistic, random forests, SVM, and Naive Bayes) to predict the SFS of PCNL with the information of 222 patients. We compared the outperformance of ML to Guy’s score and the S.T.O.N.E score system at the same time.
Patients and Methods
The study was approved by the independent ethics committee of Xu-hui Central Hospital. Between July 2017 and January 2020, 222 patients who underwent PCNL performed by one single surgeon (Dr. G.J.M.) were included in this retrospective study. All patients had computed tomography (CT) scans and IVP before surgery. Normal preoperative coagulation and negative urine cultures were verified.
All percutaneous accesses were performed under general anesthesia and in a prone position after retrograde ureteral catheterization. Access to the selected calyx was performed by Dr. G.J.M with the aid of ultrasound guidance by using an 18-gauge needle. The tract was dilated with serial dilators from 8F to 20F sheath. An 18F nephroscope (Wolf) was used to inspect the sheath, and we used a holmium laser to fragment stones with the power ranging from 60 to 90 W. Every case was demanded to place an internal ureteral stent on a suspect for the presence of mobile residual stones. A 14F nephrostomy tube was placed in the renal pelvis or the involved calyx for most patients.
Antibiotic prophylaxis was used with the second-generation cephalosporin. The medication was completed after the nephrostomy tube was removed.
Plain radiography of the kidneys, ureters, and bladders was obtained from postoperative day 1 to day 3, according to the state of the patient. The nephrostomy tube was removed when there were neither stone residues nor clinically insignificant residual fragments (diameter less than 4 mm). (Harraz et al., 2017).
All patients were asked to take out the stent for outpatient service 1 or 2 months after the surgery. If there were residual stones, they would have repeated PCNL, ureteroscopy, and shock wave lithotripsy (SWL). After that, all patients were evaluated with an ultrasound test or non-contrast CT scan after 3–6 months postoperatively. All patients accepted follow-ups for at least 1 year. PCNL was considered successful when the patient was stone-free or did not need any further intervention [clinically insignificant residual stone fragments (CIRF)] (Rassweiler et al., 2000).
Machine Learning Methods
Four types of supervised machine learning algorithms (Lasso logistic, random forests, SVM, and Naive Bayes) were applied in this study. A set of input variables comprising individual variables (age, sex, hypertension, diabetes, hyperlipidemia, urinary infection, renal insufficiency, preoperative hemoglobin, use of anticoagulants or antiplatelet medications, renal and stone factors (previous surgery, stone burden, stone location, and hydronephrosis), surgical factors (postoperative fever, septicemia, need for transfusion, length of stay, stone-free status, and ancillary procedures)) were included. The results of the stone-free status were entered as binary values: 1 (stone residues) and 0 (clinically insignificant residual stone fragments).
The machine learning models were fitted using scikit-learn 0.18 modules of Python throughout this study. Using lasso regularization and cross-validation (n fold = 10) to select the best regression, we selected lambda with 1se.lambda to screen characteristic variables. The selected variables include stone size, stone location (top/middle/bottom), and a total of four variables (Figure 1).
FIGURE 1. Selecting lambda to screen characteristic variables.
The original data set is randomly divided into the training set and the test set at 7:3 (156: 66). Lasso-logistic, SVM, and Naive Bayes considered the results of lasso regression screening as independent variables to establish a model and calculate the prediction accuracy.
The RF model is a machine learning model built on decision trees. In the decision tree, each node of the tree splits the data into two groups using a cutoff value within one of the features. The RF method can minimize the effect of the overfitting problem by creating an ensemble of randomized decision trees, each of which overfits the data and averages the results to find a better classification.
Statistical Analysis
Continuous variables were compared using the independent sample Student’s t-test. The model performance was evaluated using the area under the receiver operating characteristic (ROC) curve (AUC), which provides a measure of the discriminatory performance of the model. Sensitivity is the proportion of true positives that are classified as such; specificity measures the proportion of correctly identified true negatives; and accuracy is the proportion of correct predictions.
Results
A total of 222 patients (132 males, 59.5%) were enrolled. The mean age was 54.8 ± 13.3 years, and the mean stone burden was 563.4 ± 517.6 mm2. The mean Guy’s score was 3.2 ± 0.9, and the mean S.T.O.N.E. score was 8.9 ± 1.8. Table 1 shows the preoperative factors including individual variables and renal and stone factors. Table 2 shows the actual postoperative data for these patients. The overall SFS was 50% (111/222). Figure 2 shows the stone-free rate in each subgroup of GSS grades and the S.T.O.N.E score systems. The number of fever and infections during hospitalization was 18.9% (42) and 8.6% (19). Postoperative blood transfusion due to significant blood loss happened in nine patients (4.1%). With the follow-ups for at least 1 year, there were 12 patients (5.4%) who accepted ancillary procedures to manage residual renal stones.
TABLE 1. Preoperative factors include individual variables and renal and stone factors.
TABLE 2. Postoperative outcome variable (n = 222).
FIGURE 2. The stone-free rate in each subgroup of GSS grades and the S.T.ON.E score systems.
We have used four machine learning methods to analyze the outcomes to predict the stone-free status. Table 3 shows the AUC, sensitivity, specificity, and accuracy of each prediction method to the results of the stone-free status. When using AUC as a measure of the predictive model performance, as shown in Table 3, the AUC of Lasso logistic was 0.879. It was superior to those of RF, SVM, and Naive Bayes (0.803, 0.818, and 0.803, respectively). The AUCs of the GSS and S.T.O.N.E were 0.800 and 0.844, respectively, which were lower than the Lasso logistic. Figure 3 shows the ROC curves of the four MLMs, as well as the GSS and S.T.O.N.E score system.
TABLE 3. AUC, sensitivity, specificity, and accuracy of each prediction method for the results of the stone-free status.
FIGURE 3. The ROC curves of the four MLMs as well as the GSS and S.T.O.N.E score system.
As shown in Table 3, the accuracies of the four MLMs were also superior to those of the S.T.O.N.E score system. The sensitivities of the MLMs were 75.8–83.3%, which were higher than the S.T.O.N.E. score system. The machine learning system of LL recognized stone burden and stone location as the most highly weighted preoperative factors affecting the post-PCNL-SFR.
Discussion
The incidence and prevalence of kidney stones have increased by three times over the past 4 decades (Thongprayoon et al., 2020). The prevalence of kidney stones is estimated at about 5–10% in Europe, 4% in South America, and 1–19% in Asia currently (Sorokin et al., 2017; Liu et al., 2018). Without a doubt, kidney stones represent a considerable burden for public healthcare systems.
Thomas et al. (2011) were the first to introduce Guy’s stone score (GSS) to predict the success of the post-PCNL stone-free status (SFS). The model is reproducible, provides quick and easy office-based categorization of renal stones in four grades based on stone shape and configuration, and correlates well with the SFS; however, it fails to take into account the size and density of the stone. The S.T.O.N.E. nephrolithometry scoring system of Okhunov et al. (Zhamshid. et al., 2013) is based on non-contrast CT (NCCT) having five variables; a score of 5–6 (low complexity) has an overall SFS of 94–100%, and a score 9–13 (high complexity) has an overall SFS of 27–64%. Also, greater S.T.O.N.E. scores are associated with a greater estimated blood loss (EBL), longer operative times (LOTs), and increased length of stay (LOS) in hospital. Smith et al. (2013) developed the CROES (Clinical Research Office of the Endourological Society) nomogram to predict the SFS after PCNL based on a global database study of 5,830 patients. Six characteristics (stone burden, number, location, multiple, staghorn, and institute-level case volume) are included in this nomogram. It achieved a remarkable prediction accuracy of 76%, but it is laborsome and time-consuming.
Many studies have compared the predictive performance of these score systems in post-PCNL SFR. Most studies have examined the performance of these scoring systems to predict SFR equally but not equally to predict complications. The AUC ranges from 0.63 to 0.853 (Wu and Okeke, 2017), and the different scoring system has its drawbacks or limitations. For example, in Guy’s score system, partial staghorn stone was not clearly defined. The S.T.O.N.E. nephrolithometry scoring system relies solely on preoperative CT. The CROES nomogram requires information that might not be readily available (case volume and treatment history). So one simpler and easier application stone score system is needed nowadays. Alireza and his colleagues (Aminsharifi et al., 2017) were the first to use machine learning methods to evaluate the stone-free rate and complications after PCNL. They used ANN to predict the stone-free rate. The accuracy was 81.0–98.2%. The AUC was 0.861. In 2019, his team (Aminsharifi et al., 2020) reported they used software to predict the SFR after PCNL with the AUC of 0.915. In our study, we used four machine learning methods to predict the SFR of PCNL compared with Guy’s system and the S.T.O.N.E. nephrolithometry system. The machine learning methods (MLMs) include Lasso logistic, random forests, SVM, and Naive Bayes. The AUC of the MLMs was superior than that of Guy’s stone score system. The sensitivity and accuracy of MLMs were superior to that of the S.T.O.N.E. nephrolithometry system.
Machine learning is built on the statistical framework. Different approaches are designed to make the most accurate prediction possible. It has been proved to have a good performance to predict the SFR post-PCNL. Although we did not have an advantageous performance of AUC of 0.915 (Aminsharifi et al., 2020), in this study, we found the MLMs could predict the stone-free rate with the AUC not inferior to that of Guy’s stone score or the S.T.O.N.E score system. The machine learning algorithm mainly includes random forests, decision trees, artificial neural networks, Bayesian learning, and Deep learning. Each approach has its advantage and disadvantage. We have tried four methods to predict the stone-free rate in this study, and all of them got a fairly superior performance, as well as the clinical scoring systems being currently available. Machine learning methods are a good tool to predict the stone-free rate with AUCs after PCNL.
So far, in the field of urinary stones, there have been few studies using machine learning methods to predict operative outcomes or help make operative decisions. As one author commented (Peng et al., 2021), to improve the application of MLMs in uritholiasis, two categories should be considered: first, more people including urologists, statisticians, and computer experts need to be involved in this project; second, more data from different regions or population should be collected for future event prediction. We need to establish, manage, and share a cross-country or nationwide database, through which machine learning or AI would contribute to the field of calculi or other issues in the near future.
Data Availability Statement
The raw data supporting the conclusion of this article will be made available by the authors, without undue reservation.
Ethics Statement
Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.
Author Contributions
HZ: protocol and writing of the manuscript. WL: data analysis. JL: data collection. LL: management. JG: project development (corresponding author). HZ and WL contributed equally to this work.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors, and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary Material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmolb.2022.880291/full#supplementary-material
Abbreviations
MLMs, machine learning methods; PCNL, percutaneous nephrolithotomy; LL, Lasso logistic; RF, random forest; SVM, support vector machine; AUC, area under the curve; GSS, Guy’s score system; CROES, Clinical Research Office of the Endourological Society; SFR, stone-free rate; NCCT, non-contrast CT; EBL, estimated blood loss; LOT, longer operative time; LOS, length of stay
References
Akman, T., Binbay, M., Yuruk, E., Sari, E., Seyrek, M., Kaba, M., et al. (2011). Tubeless Procedure Is Most Important Factor in Reducing Length of Hospitalization after Percutaneous Nephrolithotomy: Results of Univariable and Multivariable Models. Urology 77 (2), 299–304. doi:10.1016/j.urology.2010.06.060
Al Adl, A. M., Mohey, A., Abdel Aal, A., Abu-Elnasr, H. A. F., El Karamany, T., Noureldin, Y. A., et al. (2020). Percutaneous Nephrolithotomy Outcomes Based on S.T.O.N.E., GUY, CROES, and S-ReSC Scoring Systems: The First Prospective Study. J. Endourology 34, 1223–1228. doi:10.1089/end.2019.0856
Aminsharifi, A., Irani, D., Pooyesh, S., Parvin, H., Dehghani, S., Yousofi, K., et al. (2017). Artificial Neural Network System to Predict the Postoperative Outcome of Percutaneous Nephrolithotomy. J. Endourology 31 (5), 461–467. doi:10.1089/end.2016.0791
Aminsharifi, A., Irani, D., Tayebi, S., Jafari Kafash, T., Shabanian, T., and Parsaei, H. (2020). Predicting the Postoperative Outcome of Percutaneous Nephrolithotomy with Machine Learning System: Software Validation and Comparative Analysis with Guy's Stone Score and the CROES Nomogram. J. Endourology 34 (6), 692–699. doi:10.1089/end.2019.0475
Andras, I., Mazzone, E., van Leeuwen, F. W. B., De Naeyer, G., van Oosterom, M. N., Beato, S., et al. (2020). Artificial Intelligence and Robotics: a Combination that Is Changing the Operating Room. World J. Urol. 38 (10), 2359–2366. doi:10.1007/s00345-019-03037-6
De Perrot, T., Hofmeister, J., Burgermeister, S., Martin, S. P., Feutry, G., Klein, J., et al. (2019). Differentiating Kidney Stones from Phleboliths in Unenhanced Low-Dose Computed Tomography Using Radiomics and Machine Learning. Eur. Radiol. 29 (9), 4776–4782. doi:10.1007/s00330-019-6004-7,
Fernstro¨m, I., and Johansson, B. (1976). Percutaneous Pyelolithotomy. A New Extraction Technique. Scand. J. Urol. Nephrol. 10, 257–259.
Harraz, A. M., Osman, Y., El-Nahas, A. R., Elsawy, A. A., Fakhreldin, I., Mahmoud, O., et al. (2017). Residual Stones after Percutaneous Nephrolithotomy: Comparison of Intraoperative Assessment and Postoperative Non-contrast Computerized Tomography. World J. Urol. 35 (8), 1241–1246. doi:10.1007/s00345-016-1990-4
Hung, A. J., Chen, J., Che, Z., Nilanon, T., Jarc, A., Titus, M., et al. (2018). Utilizing Machine Learning and Automated Performance Metrics to Evaluate Robot-Assisted Radical Prostatectomy Performance and Predict Outcomes. J. Endourology 32 (5), 438–444. doi:10.1089/end.2018.0035
Labadie, K., Okhunov, Z., Akhavein, A., Moreira, D. M., Moreno-Palacios, J., Del Junco, M., et al. (2015). Evaluation and Comparison of Urolithiasis Scoring Systems Used in Percutaneous Kidney Stone Surgery. J. Urol. 193 (1), 154–159. doi:10.1016/j.juro.2014.07.104
Liu, Y., Chen, Y., Liao, B., Luo, D., Wang, K., Li, H., et al. (2018). Epidemiology of Urolithiasis in Asia. Asian J. Urol. 5 (4), 205–214. doi:10.1016/j.ajur.2018.08.007
Matlaga, B. R., Kim, S. C., and Lingeman, J. E. (2005). Improving Outcomes of Percutaneous Nephrolithotomy: Access. EAU Update Ser. 3, 37–43. doi:10.1016/j.euus.2004.11.002
Miernik, A., Schoenthaler, M., Wilhelm, K., Wetterauer, U., Zyczkowski, M., Paradysz, A., et al. (2014). Combined Semirigid and Flexible Ureterorenoscopy via a Large Ureteral Access Sheath for Kidney Stones >2 Cm: a Bicentric Prospective Assessment. World J. Urol. 32 (3), 697–702. doi:10.1007/s00345-013-1126-z
Noureldin, Y. A., Elkoushy, M. A., and Andonian, S. (2015). Which Is Better? Guy's versus S.T.O.N.E. Nephrolithometry Scoring Systems in Predicting Stone-free Status post-percutaneous Nephrolithotomy. World J. Urol. 33, 1821–1825. doi:10.1007/s00345-015-1508-5
Noureldin, Y., Elkoushy, M., and Andonian, S. (2015). External Validation of the S.T.O.N.E. Nephrolithometry Scoring System. Cuaj 9, 190–195. doi:10.5489/cuaj.2652
Peng, L., Pan, J., Yang, D., and Zhong, W. (2021). The prospect of Machine Learning in Predicting post-lithotripsy Outcomes. World J. Urol. 39 (11), 4287–4288. doi:10.1007/s00345-020-03377-8
Rassweiler, J. J., Renner, C., and Eisenberger, F. (2000). The Management of Complex Renal Stones. BJU Int. 86, 919–928. doi:10.1046/j.1464-410x.2000.00906.x
Rodrigo., S-I., Hein, S., Reis, G., Christian., G., and Miernik, A. (2020). Current and Future Applications of Machine and Deep Learning in Urology: a Review of the Literature on Urolithiasis, Renal Cell Carcinoma, and Bladder and Prostate Cancer. World J. Urol. 38 (10), 2329–2347.
Rosette, J. d. l., Assimos, D., Desai, M., Gutierrez, J., Lingeman, J., Scarpa, R., et al. CROES PCNL Study Group (2011). The Clinical Research Office of the Endourological Society Percutaneous Nephrolithotomy Global Study: Indications, Complications, and Outcomes in 5803 Patients. J. Endourology 25 (1), 11–17. doi:10.1089/end.2010.0424
Smith, A., Averch, T. D., Shahrour, K., Opondo, D., Daels, F. P. J., Labate, G., et al. CROES PCNL Study Group (2013). A Nephrolithometric Nomogram to Predict Treatment success of Percutaneous Nephrolithotomy. J. Urol. 190 (1), 149–156. doi:10.1016/j.juro.2013.01.047
Sorokin, I., Mamoulakis, C., Miyazawa, K., Rodgers, A., Talati, J., and Lotan, Y. (2017). Epidemiology of Stone Disease across the World. World J. Urol. 35 (9), 1301–1320. doi:10.1007/s00345-017-2008-6
Ström, P., Kartasalo, K., Olsson, H., Solorzano, L., Delahunt, B., Berney, D. M., et al. (2020). Artificial Intelligence for Diagnosis and Grading of Prostate Cancer in Biopsies: a Population-Based, Diagnostic Study. Lancet Oncol. 21 (2), 222–232. doi:10.1016/s1470-2045(19)30738-7
Tayyebe., S., Parsaei, H., Aminsharifi, A., Mehdi., M. M., Jahromi Amin, T., Pouyesh, S., et al. (2019). An Artificial Intelligence-Based Clinical Decision Support System for Large Kidney Stone Treatment. Australas. Phys. Eng. Sci. Med. 42 (3), 771–779.
Thomas, K., Smith, N. C., Hegarty, N., and Glass, J. M. (2011). The Guy's Stone Score-Ggrading the Complexity of Percutaneous Nephrolithotomy Procedures. Urology 78 (2), 277–281. doi:10.1016/j.urology.2010.12.026
Thongprayoon, C., Krambeck, A. E., and Rule, A. D. (2020). Determining the True burden of Kidney Stone Disease. Nat. Rev. Nephrol. 16 (12), 736–746. doi:10.1038/s41581-020-0320-7
Wu, W. J., and Okeke, Z. (2017). Current Clinical Scoring Systems of Percutaneous Nephrolithotomy Outcomes. Nat. Rev. Urol. 14, 459–469. doi:10.1038/nrurol.2017.71
Keywords: machine learning, prediction, percutaneous nephrolithotomy, stone-free status, Guy’s stone score, S.T.O.N.E score system
Citation: Zhao H, Li W, Li J, Li L, Wang H and Guo J (2022) Predicting the Stone-Free Status of Percutaneous Nephrolithotomy With the Machine Learning System: Comparative Analysis With Guy’s Stone Score and the S.T.O.N.E Score System. Front. Pharmacol. 9:880291. doi: 10.3389/fmolb.2022.880291
Received: 21 February 2022; Accepted: 07 April 2022;
Published: 04 May 2022.
Edited by:
Xin Gao, King Abdullah University of Science and Technology, Saudi ArabiaReviewed by:
Mohammed Shahait, King Hussein Medical Center, JordanYasser A. Noureldin, Benha University, Egypt
Copyright © 2022 Zhao, Li, Li, Li, Wang and Guo. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Jianming Guo, drguojm@126.com