Classifying Ruptured Middle Cerebral Artery Aneurysms With a Machine Learning Based, Radiomics-Morphological Model: A Multicentral Study

Objective Radiomics and morphological features were associated with aneurysms rupture. However, the multicentral study of their predictive power for specific-located aneurysms rupture is rare. We aimed to determine robust radiomics features related to middle cerebral artery (MCA) aneurysms rupture and evaluate the additional value of combining morphological and radiomics features in the classification of ruptured MCA aneurysms. Methods A total of 632 patients with 668 MCA aneurysms (423 ruptured aneurysms) from five hospitals were included. Radiomics and morphological features of aneurysms were extracted on computed tomography angiography images. The model was developed using a training dataset (407 patients) and validated with the internal (152 patients) and external validation (73 patients) datasets. The support vector machine method was applied for model construction. Optimal radiomics, morphological, and clinical features were used to develop the radiomics model (R-model), morphological model (M-model), radiomics-morphological model (RM-model), clinical-morphological model (CM-model), and clinical-radiomics-morphological model (CRM-model), respectively. A comprehensive nomogram integrating clinical, morphological, and radiomics predictors was generated. Results We found seven radiomics features and four morphological predictors of MCA aneurysms rupture. The R-model obtained an area under the receiver operating curve (AUC) of 0.822 (95% CI, 0.776, 0.867), 0.817 (95% CI, 0.744, 0.890), and 0.691 (95% CI, 0.567, 0.816) in the training, temporal validation, and external validation datasets, respectively. The RM-model showed an AUC of 0.848 (95% CI, 0.810, 0.885), 0.865 (95% CI, 0.807, 0.924), and 0.721 (95% CI, 0.601, 0.841) in the three datasets. The CRM-model obtained an AUC of 0.856 (95% CI, 0.820, 0.892), 0.882 (95% CI, 0.828, 0.936), and 0.738 (95% CI, 0.618, 0.857) in the three datasets. The CRM-model and RM-model outperformed the CM-model and M-model in the internal datasets (p < 0.05), respectively. But these differences were not statistically significant in the external dataset. Decision curve analysis indicated that the CRM-model obtained the highest net benefit for most of the threshold probabilities. Conclusion Robust radiomics features were determined related to MCA aneurysm rupture. The RM-model exhibited good ability in classifying ruptured MCA aneurysms. Integrating radiomics features into conventional models might provide additional value in ruptured MCA aneurysms classification.


INTRODUCTION
Middle cerebral artery (MCA) aneurysm is the most common subtype of unruptured aneurysms (Huttunen et al., 2010;Can et al., 2015). With the improvement of imaging techniques, unruptured aneurysms have become more frequently detected (Greving et al., 2014). But therapeutic decision-making for them is controversial. On the one hand, many unruptured aneurysms stay asymptomatic and never rupture (Korja et al., 2014). The prophylactic treatment such as current endovascular and microsurgical interventions carries the risk of procedurerelated complications (Naggara et al., 2010;Zhu et al., 2020). On the other hand, once the aneurysm ruptures, the outcome is catastrophic (Vlak et al., 2011). Therefore, it is vital to screen out rupture-prone aneurysms.
Previous studies have identified that morphological features were associated with aneurysms rupture (Lindgren et al., 2016;Zhu et al., 2020). Researchers have constructed various computational methods using morphological features to evaluate the aneurysms-rupture risk Tanioka et al., 2020;Zhu et al., 2020). However, those morphological features are measured on two-dimensional images and might be affected by different readers or projections. It could impair the comparability of results.
Radiomics is an emerging technology that extracts highthroughput data from medical images (Zhou et al., 2018;Hua et al., 2020;Tomaszewski and Gillies, 2021). Recently, radiomics is frequently used in cerebrovascular disease researches . Several researchers have scoped to the whole-brain aneurysms and proved that radiomics features were related to aneurysms rupture status Ou et al., 2021). Regretfully, they did not analyze the features' robustness, which can be easily affected by a slight change in image-scanning protocols or regions of interest (ROIs) segmentation (Mackin et al., 2015;Choe et al., 2019). Moreover, the predictive ability of radiomics in those studies was not validated by any external validation dataset, which leads to the uncertainness of their results' generalizability (Collins et al., 2015;Lambin et al., 2017).
To the best of our knowledge, few studies have predicted the rupture of the location-specific aneurysm with robust radiomics features. In this study, we included a large sample of 668 MCA aneurysms. We aimed to (1) determine whether there are robust radiomics features that can classify ruptured MCA aneurysms; and (2) evaluate the additional value of combining morphological and radiomics features in classifying ruptured MCA aneurysm.

MATERIALS AND METHODS
Our study was approved by the Medical Ethics Committee of our hospital.

Study Population and Clinical Data
We performed a retrospective and multicentral study using the data from five hospitals (hospitals A, B, C, D, and E). MCA aneurysms with available computed tomography angiography (CTA) data were included. Exclusion criteria were as follows: fusiform MCA aneurysms, aneurysms combined with vascular diseases (such as Moyamoya disease and arteriovenous malformations), aneurysms with a size <3 mm, aneurysms with poor-quality images and patients underwent surgery or interventional therapy before CTA examination (see Supplementary Methods, Supplementary Digital Content 1, which illustrates details about CTA image scanning).
Patients with MCA aneurysms seen in hospital A from January 2009 to December 2019 were allocated to the training and the internal validation datasets. The training dataset encompassed the patients from the earlier period (2009)(2010)(2011)(2012)(2013)(2014)(2015)(2016)(2017). The patients from the more recent period (2018-2019) were attributed to the temporal validation dataset (internal validation dataset) (Collins et al., 2015;Al-Shahi Salman et al., 2018). For external validation, MCA aneurysms cases in four hospitals, including hospital B (from January 2018 to December 2020), hospital C (from January 2018 to December 2020), hospital D (from January 2017 to October 2019), and hospital E (from September 2019 to March 2020), were merged to one external validation dataset (Lambin et al., 2017).
Clinical data such as age, sex, history of hypertension (a diagnosis of hypertension previously made by another physician or use of antihypertensive drugs), cigarette smoking (previous smoker or current smoker), and aneurysm side were collected. Rupture status of aneurysms was evaluated using the following criterion: (1) for patients with subarachnoid hemorrhage (SAH), aneurysms adjacent to the cisternal clots were judged ruptured, and those aneurysms not adjacent to the cisternal clots were judged on digital subtraction angiography (DSA). (2) Asymptomatic patients without SAH were identified to be unruptured (Shi et al., 2021).

Morphological Predictors Discovery
Morphological features such as aneurysm location (divided into M1, the proximal segment of the middle cerebral artery; Mbif, main middle cerebral artery bifurcation; Mdist, distal middle cerebral artery), aneurysm size, vessel size, aneurysm height, perpendicular height, aspect ratio (AR), size ratio (SR), aneurysm angle, flow angle, vessel angle, daughter dome, and irregular shape were measured as described in previous studies (Can et al., 2015;Chen et al., 2020;Zhu et al., 2020; Figure 1C; see Supplementary Methods, Supplementary Digital Content 2, which illustrates detailed definitions of morphological features).
We implemented univariate analysis to find morphological factors that were associated with MCA aneurysm rupture. After that, the multivariable logistic regression was performed to identify independent morphological predictors of MCA aneurysm rupture.

Optimal Radiomics Signature Detection
The workflow process of radiomics analysis is shown in Figures 1A,B. ROIs of aneurysms were manually segmented by a neuroradiologist on each slice of CTA images ( Figure 1A). Then, 50 aneurysms were randomly selected to be re-segmented by another neuroradiologist. We calculated the inter-class correlation coefficient (ICC) to evaluate the inter-observer reproducibility.
As the images were acquired from different CT scanners with different parameters, we performed data preprocessing before radiomics feature extraction (Lambin et al., 2017;Morin et al., 2018;Chen et al., 2021). Image resampling and graylevel discretization were used to reduce the variability of radiomics features (Shafiq-Ul-Hassan et al., 2018). A total of 1316 radiomics features were extracted from each ROI (see Supplementary Digital Content 3, which illustrates the possible pathophysiologic meaning of the features). All radiomics features were standardized by z-score to eliminate unit limits of each feature (Yang et al., 2019; see Supplementary Methods, Supplementary Digital Content 4, for further details about radiomics analysis). Figure 1B indicates the feature selection procedure. Firstly, features with poor reproducibility (an ICC of <0.7) were excluded. After that, we used two different kinds of schemes to select informative radiomics features: (1) the minimum redundancy maximum relevance (mRMR) method was performed to rank the top 50 rupture-associated features while minimizing intra-feature correlation (Ding and Peng, 2005;Castiglioni et al., 2019), and then we used the least absolute shrinkage and selection operator (LASSO) method to select optimal features from those 50 features (Sauerbrei et al., 2007) ("mRMR-LASSO method"); (2) only LASSO method was used to selected optimal features ("LASSO method") (see Supplementary Methods, Supplementary Digital Content 5, for further detailed information of mRMR and LASSO). We applied the logistics regression model to build radiomics signatures. Discrimination ability of the "mRMR-LASSO model" and "LASSO model" were compared. The features with better performance were used for subsequent analysis. Moreover, we calculated the Rad score through a linear combination of selected features by multiplying with their LASSO coefficients (Huang et al., 2016).

Machine Learning Models Development and Validation
Support vector machine (SVM) is a supervised machine learning method that classifies data points by maximizing the distance between classes in a high-dimensional space (Orru et al., 2012). We applied SVM with a 10-fold cross-validation to construct models. As shown in Figure 1D, optimal radiomics features were introduced into the radiomics model (R-model). The morphological predictors of aneurysm rupture were introduced into the morphological model (M-model). Optimal radiomics features and morphological features were put together to generate the radiomics-morphological model (RM-model). Models were trained using the training dataset and validated in the temporal and external validation datasets. Variance inflation factor (VIF) was used to detect multicollinearity of the enrolled features, and a VIF of ≥5 was considered as multicollinearity (Akinwande et al., 2015).
The model performance was evaluated using the receiver operating characteristic (ROC) curve. The DeLong test was used for comparisons of an area under the receiver operating curves (AUCs) of different models (DeLong et al., 1988). The decision curve analysis (DCA) was applied to assess the clinical utility of the models. Besides, net reclassification improvement (NRI) (Pencina et al., 2011) was calculated to evaluate the improvement in the discrimination ability of different models.

Nomogram Construction and Evaluation
To provide an easy and visualized rupture risk-scoring system, we constructed a comprehensive nomogram. The top five rupture-associated factors among the Rad score, clinical and morphological features were selected using the mRMR method. These five factors were used to generate the nomogram. The discrimination of the nomogram was assessed with ROC curves. The agreement between predicted rupture and observed rupture was evaluated using the calibration curve and the Hosmer-Lemeshow test (Kramer and Zimmerman, 2007). The discrimination and calibration of the nomogram were appraised in the training and validation datasets.

Statistical Analysis
Categorical variables are presented as counts (with percentages), while continuous variables are presented as medians [interquartile range, (IQR)]. We used Student t-tests or Mann-Whitney U tests to evaluate the differences in continuous variables. Differences in categorical variables were assessed using the χ 2 test or Fisher exact test (two-tailed). A p-value of <0.05 indicates a statistical difference. Statistical analysis and model construction were conducted using SPSS (version 24.0) and R (version 3.6.1).

Clinical and Morphological Characteristics
A total of 632 patients with 668 MCA aneurysms (423 ruptured aneurysms) from five hospitals were included in our study. Multiple aneurysms were presented in 132 (20.9%) patients. Thirty-two of the 132 (24.2%) patients have bilateral MCA aneurysms. There were 407, 152, and 73 patients with 438, 155, and 75 aneurysms in the training, temporal validation, and external validation datasets. Table 1 shows the clinical and morphological characteristics of the training dataset (see Supplementary Digital Content 6, which shows characteristics of validation datasets). In univariate analysis, patients with ruptured aneurysms were younger and were less common to have a history of hypertension (p < 0.05).
For morphological features, the mean ICC value of the nine morphological features was 0.924. The kappa value of the irregular shape and daughter dome is 0.603 and 0.838, respectively (p-value for all <0.001). Vessel size, aneurysm size, neck size, AR, SR, aneurysm height, perpendicular height, aneurysm angle, irregular shape, and daughter dome were associated with aneurysm rupture (p < 0.05). The result of multivariate analysis (

Optimal Radiomics Features Detection
The mean ICC value of the overall 1316 radiomics features was 0.751. Eight hundred and eighty-one radiomics features showed a high interobserver agreement (an ICC value of ≥0.7). As shown in Table 3, the "mRMR-LASSO model" presented an AUC of 0.767 and 0.828 in the training and temporal validation dataset, respectively. The "mRMR-LASSO model" presented a higher discrimination ability than the "LASSO model." Therefore, those features selected using the "mRMR-LASSO method" were used for R-model construction. The Rad score was calculated (see Supplementary Results, Supplementary Digital Content 7, which indicates the Rad score calculation formula).

Machine Learning Models Construction and Evaluation
The performance of the models was shown in Table 4 and Figure 2. The R-model obtained an AUC of 0.822 (95% CI, 0.776, 0.867) in the training dataset. In the temporal and external validation dataset, the R-model presented an AUC of 0.817 (95% CI, 0.744, 0.890) and 0.691 (95% CI, 0.567, 0.816), respectively. Four morphological predictors, including SR, neck   Table 5.
The optimal radiomics features and morphological predictors were enrolled in the RM-model. The RM-model exhibited good    (Figure 2 and Table 4). Multicollinearity was not observed between those selected radiomics features and morphological predictors (VIF for all <2). significant in the temporal validation dataset (p < 0.005) while it was not statistically significant in the external validation dataset (p = 0.224).
We further added clinical features (hypertension, smoking, age, and sex) into the M-model and RM-model to construct the clinical-morphological model (CM-model) and clinicalradiomics-morphological model (CRM-model). As it was shown in Table 4 and Figure 2, the CRM-model obtained an AUC (95% CI) of 0.856 (0.820, 0.892), 0.882 (0.828, 0.936), and 0.738 (0.618, 0.857) in the three datasets, respectively. The CRMmodel outperformed the CM-model in the training and temporal validation datasets (p < 0.05). However, the difference was not observed in the external validation dataset. We further calculated NRI to evaluate the improvement of discrimination by adding radiomics features to CM-model. We found that adding radiomics features to CM-mode improved the net reclassification indices in the three datasets (additive NRI, 52.40%, 89.66%, and 24.62%, respectively). This indicated that compared to the CMmodel, the CRM-model correctly reclassified 52.40%, 89.66%, and 24.62% cases in the three datasets, respectively.
With respect to clinical utility, the DCA ( Figure 3B) indicated that the RM-model had a higher overall net benefit in distinguishing ruptured aneurysms than the single R-model and M-model for most of the threshold probabilities. The CRMmodel obtained the highest net benefit for most of the threshold probabilities. Combining radiomics features to conventional models resulted in an extra net-benefit compared with the M-model and CM-model.

DISCUSSION
This study developed and validated ruptured MCA aneurysms classification models based on radiomics and morphological features in a large and multicentral dataset (a total of 668 MCA aneurysms were enrolled). It was proved that robust radiomics features extracted from CTA images could classify ruptured MCA aneurysms. The RM-model could identify more than 74% ruptured MCA aneurysms with a specificity of 72-81%. Additionally, we provided an easy and visualized rupture risk-scoring system for MCA aneurysm patients through the nomogram.
In this study, a total of 1316 radiomics features were extracted for each aneurysm. Among those numerous radiomics features, the features indicating image heterogeneity (e.g., Dependence Non-Uniformity Normalized, Cluster Shade, and Variance) were screened out as the optimal signature for ruptured aneurysms classification. This is in agreement with the study of Ou et al. (2021). The possible explanation for this is that CTA image heterogeneity is caused by the contrast maldistribution in the vessel lumen. The uneven distribution of contrast indicates turbulent flow (George et al., 2016;Aghayev et al., 2018;Ou et al., 2021), which was one of the risk factors for aneurysms rupture (Lv et al., 2020). Therefore, we speculate that CTAderived radiomics features might imply the hemodynamics condition of aneurysms.
Unlike former research Ou et al., 2021), shape radiomics features were not selected for model construction. We exclude some shape features before the feature selection due to their poor interobserver agreement (e.g., the shape radiomics features Sphericity and Flatness obtained an ICC value of 0.536 and 0.590, respectively). Only highly stable features were used in the model construction procedure. The temporal and external validation datasets further verified the robustness and generalizability of the results.
It is generally acknowledged that vessel wall degradation and abnormal morphological condition are related to aneurysms' rupture. We found that daughter dome, multiplicity, neck size, and SR were independent predictors for MCA aneurysm rupture, which have been reported by other researchers (Can et al., 2015;Zhang et al., 2019;Ou et al., 2020;Tanioka et al., 2020;Zhu et al., 2020).
Both radiomics and morphological features originate from CTA images, but their biological meanings and analysis procedures are different. Radiomics features describe the shape and texture characteristics of aneurysms from the micro point of view (Lambin et al., 2017;Xu et al., 2019), while the morphological features measure the macroscopic observation of the aneurysms. Radiomics features are threedimensional derived, high-through biomarkers but lacking clinical interpretability. Meanwhile, those morphological features are two-dimensional measured features with relatively less information, but they are still fundamental tools for aneurysm evaluation. Though the statistically significant improvement in AUC of adding radiomics was not observed in the external dataset, considering its objectivity and its extra net-benefit in DCA, radiomics could be a possible choice in clinical practice, enabling better patient management.
Aneurysms rupture leads to catastrophic consequences. Preventive treatment for high rupture risk patients is necessary. However, preventive treatments for low rupture risk patients may cause more harm than good due to operationrelated complications (Naggara et al., 2010). Therefore, except for stratifying rupture-prone aneurysms, accurate identification of aneurysms at a relatively low rupture risk is also essential. Compared to the CM-model (which is routinely used in clinical practice), the CRM-model obtained a higher specificity, positive predictive value (PPV), and the CRM-model correctly reclassified 52.40%, 89.66%, and 24.62% patients in the three datasets, respectively. This indicated that adding radiomics features to conventional models might not only classify ruptured aneurysms but also help in recognizing unruptured aneurysms, which may reduce unnecessary treatment for unruptured aneurysms patients.
There are some limitations in this study. First, this is a retrospective, cross-sectional study without the longitudinal follow-up of aneurysms, which might inherently cause biases. Second, the aneurysms' morphological changes after rupture were not considered because it is hard to collect the morphological change before and after the aneurysm rupture due to ethical issues. Third, most patients with SAH history were excluded because of their surgery experience. This could cause potential selection bias. Forth, images from external validation datasets contain different scanning protocols, which might affect the validation results. We performed image resampling and gray-level discretization to reduce the variability of images. Moreover, samples from the other four hospitals were merged into one external validation dataset. The good results further indicated the robustness of the models.

CONCLUSION
In conclusion, we analyzed the MCA aneurysms rupture by clinical, radiomics, and morphological features using multicentral data. We answered two critical questions: (1) Robust radiomics features could classify ruptured MCA aneurysms.
(2) The integration of radiomics into conventional clinical and morphological models might provide additional benefit in ruptured MCA aneurysms classification. An easy and visualized rupture risk-scoring nomogram was generated. This may aid in the rupture-risk assessment of MCA aneurysms.

DATA AVAILABILITY STATEMENT
The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author/s.

ETHICS STATEMENT
The studies involving human participants were reviewed and approved by the Medical Ethics Committee of the First Affiliated Hospital of Wenzhou Medical University. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.