Rebleeding of Ruptured Intracranial Aneurysm After Admission: A Multidimensional Nomogram Model to Risk Assessment

Objective Rebleeding is recognized as the main cause of mortality after intracranial aneurysm rupture. Though timely intervention can prevent poor prognosis, there is no agreement on the surgical priority and choosing medical treatment for a short period after rupture. The aim of this study was to investigate the risk factors related to the rebleeding after admission and establish predicting models for better clinical decision-making. Methods The patients with ruptured intracranial aneurysms (RIAs) between January 2018 and September 2020 were reviewed. All patients fell to the primary and the validation cohort by January 2020. The hemodynamic parameters were determined through the computational fluid dynamics simulation. Cox regression analysis was conducted to identify the risk factors of rebleeding. Based on the independent risk factors, nomogram models were built, and their predicting accuracy was assessed by using the area under the curves (AUCs). Result A total of 577 patients with RIAs were enrolled in this present study, 86 patients of them were identified as undergoing rebleeding after admission. Thirteen parameters were identified as significantly different between stable and rebleeding aneurysms in the primary cohort. Cox regression analysis demonstrated that six parameters, including hypertension [hazard ratio (HR), 2.54; P = 0.044], bifurcation site (HR, 1.95; P = 0.013), irregular shape (HR, 4.22; P = 0.002), aspect ratio (HR, 12.91; P < 0.001), normalized wall shear stress average (HR, 0.16; P = 0.002), and oscillatory stress index (HR, 1.14; P < 0.001) were independent risk factors related to the rebleeding after admission. Two nomograms were established, the nomogram including clinical, morphological, and hemodynamic features (CMH nomogram) had the highest predicting accuracy (AUC, 0.92), followed by the nomogram including clinical and morphological features (CM nomogram; AUC, 0.83), ELAPSS score (AUC, 0.61), and PHASES score (AUC, 0.54). The calibration curve for the probability of rebleeding showed good agreement between prediction by nomograms and actual observation. In the validation cohort, the discrimination of the CMH nomogram was superior to the other models (AUC, 0.93 vs. 0.86, 0.71 and 0.48). Conclusion We presented two nomogram models, named CMH nomogram and CM nomogram, which could assist in identifying the RIAs with high risk of rebleeding.


INTRODUCTION
Intracranial aneurysms (IAs), a common cerebrovascular disease in the aging population, refer to the main cause of subarachnoid hemorrhage. Rebleeding is recognized as a catastrophic event with high mortality after aneurysmal subarachnoid hemorrhage (aSAH) (Rosenørn et al., 1987;Jaechan et al., 2015;Kienzler et al., 2016). Though timely surgical intervention can effectively protect aSAH patients from poor outcome (Ko et al., 2011;Cordonnier et al., 2018;Darkwah Oppong et al., 2018), for several reasons, a notable number of patients cannot receive treatment as soon as they are sent to a hospital. For the reason that the most rebleeding occurs within 6 h after the initial hemorrhage (Rosenørn et al., 1987;Hijdra et al., 1988;Jaechan et al., 2015), patients to be prioritized should be determined.
The key to making medical decisions for this condition is to identify the rebleeding risk of ruptured intracranial aneurysms (RIAs). However, a predicting model has not been built, or reliable factors have not been set to discriminate the RIAs at high risk of rebleeding. There are several aspects involved in the mechanism of IAs rupture, which primarily include the structure damage of the aneurysm wall (Frösen et al., 2012) and the hemodynamic condition of IAs (Meng et al., 2012Dolan et al., 2013). Though some comorbidities could elevate the risk of rebleeding (e.g., hypertension), rebleeding still occurred in approximately 22% of patients after they had received the effective management (Boogaarts et al., 2015). It is noteworthy that some existing studies reported that morphological characteristics could predict the risk of rebleeding after aSAH (Starke et al., 2011;Boogaarts et al., 2015). However, a meta-analysis revealed the low quality of current evidence and low predicting accuracy of reported parameters (Boogaarts et al., 2015), which may be insufficient in clinical risk assessment. As indicated from our preliminary study, the hemodynamic characteristics could help discriminate the RIAs at high risk of rebleeding . Based on the mentioned facts, this study assumed that building a multidimensional predicting model can effectively discriminate the RIAs at high risk of rebleeding.
The present study aimed to build a risk assessment model by exploiting multidimensional characteristics of RIAs. The clinical, morphological, and hemodynamic characteristics of a group of RIAs in a neurosurgical center were retrospectively reviewed. This study considered that this current can present more insights into the factors of rebleeding and contribute to better medical decisions.

Patient Selected and Study Design
The patients with RIAs from January 2018 to September 2020 were retrospectively reviewed. Patients were enrolled by complying with the following standards: (1) an angiogram (CT angiogram, CTA) was performed after IA rupture; (3) the patients were sent to our institution within 12 h as soon as aSAH was identified (by symptoms, e.g., acute headache and sudden coma); and (4) clinical records were complete, or clinical history can be traced.
This study excluded the patients (1) having other intracranial tumors, angiostenosis and angio-malformation (e.g., arteriovenous malformation and cavernous malformation); (2) having a family history of IAs or connective tissue disease; (3) having multiple IAs, causing the source of the bleeding or rebleeding difficult to identify; (4) with dissecting or thrombus IAs; and (5) receiving special treatment for RIAs in other medical institutions before admission.
Rebleeding was the primary endpoint in this study and was diagnosed based on radiological findings: the magnitude of subarachnoid, intracerebral, or intraventricular blood significantly increased on CT after the admission, and the magnitude of bleeding did not increase and remained stable at/before admission.
The IAs which underwent rebleeding after the admission were identified as the rebleeding aneurysms, whereas the IAs without rebleeding before surgical intervention were found as the stable aneurysms. Rebleeding events were identified by two experienced neurosurgeons (PJ and JW, who were blind to clinical information and had worked as cerebral vascular neurosurgeons for more than 5 years) in accordance with the bleeding presentation on medical record and CT after the admission. Furthermore, the discrepancies were solved by consulting a senior neurosurgeon (SW, who had worked as a cerebral vascular neurosurgeon for more than 15 years).
Patients enrolled from August 2018 to December 2019 formed the primary cohort (411 patients with 411 IAs), which was adopted to develop the predictive model; while patients enrolled from January 2020 to September 2020 were classified as the validation cohort (127 patients with 127 IAs). The ratio of numbers of unruptured IAs in the primary cohort and the validation cohort reached approximately 3:1.

Perioperative Management
After admission, acute lowering of systolic pressure to 120-140 mmHg was the target (Connolly et al., 2012; American Society of Anesthesiologists Task Force on Perioperative Blood Management., 2015); all patients with Hunt-Hess I-II received surgical intervention within 72 h. However, once neurological condition progressively deteriorated, an emergency intervention would be performed.
Specific to patients with Hunt-Hess grade III-V at the admission, immediate surgical intervention was not recommended (Connolly et al., 2012). The patients who had not received immediate intervention would receive standard care following the guidelines (Connolly et al., 2012). After the admission, a CT would be performed per day, or when the patient had a sudden disorder of consciousness, or gradually worsening neurological states or convulsion after the admission. Surgical intervention was only considered when the patient's neurological status progressively deteriorated, or a rebleeding or a cerebral hernia was found in the radiological examination.

Clinical Information and Morphology Assessment
Clinical information was collected from electronical medical records about age, gender, comorbidities (e.g., hypertension, dyslipidemia, diabetes mellitus, coronary heart disease, and ischemic stroke), Hunt-Hess grade at the admission, time from the admission to the rebleeding (the time from admission to neurological symptoms) or intervention, blood pressure at the admission, and blood pressure before rebleeding/intervention. Moreover, the time interval from admission to rebleeding or intervention was recorded. Furthermore, Modified Fisher scale (mFS) and IA site were collected from the radiological data.
The morphology assessment was performed according to our previous studies Yang et al., 2021). The reconstruction of vascular model and measurement of morphological parameters were conducted by two neurosurgeons (QL and YY, who were blind to clinical information and had worked as cerebral vascular neurosurgeons for more than 3 years). The Digital Imaging and Communications in Medicine data were introduced into Mimics 17.0 (Mimics Research 17.0, Materialize, Belgium) and then reconstructed for subsequent studies. The pathological protruding region was recognized as the IA sac and was separated from the parent artery for further analysis. Two neurosurgeons separated the IA sacs independently, the discrepancy was solved by consulting a senior FIGURE 1 | The study flow chart. In this study, 411 appropriate patients with ruptured IAs were reviewed. A total of 70 patients were identified as rebleeding after the admission. Patients enrolled from August 2018 to December 2019 formed the primary cohort (411 patients with 411 IAs), which was adopted to develop the predictive model; while patients enrolled from January 2020 to September 2020 were classified as the validation cohort (127 patients with 127 IAs). IAs, intracranial aneurysms; DICOM, Digital Imaging and Communications in Medicine data. neurosurgeon (HWH, who had worked in neurointervention for more than 15 years).
This study measured aneurysm size (S), diameter of dome (D), perpendicular height (H), diameter of parent artery, vessel angle (VA), aneurysm inclination angle (AA), volume, and surface area according to our previous study (Jiang et al., 2018a). The same neurosurgeons measured these morphological parameters independently, the discrepancy was solved by consulting a senior neurosurgeon (SW). The averages of measurements from each neurosurgeon were taken to be analyzed in depth. The morphological parameters involved here are also listed in Supplementary Table 1. Aspect ratio (AR), size ratio (SR), undulation index (UI), and non-sphericity index (NSI) were calculated according to previous study (Dhar et al., 2008). An irregular shape was defined as small bleb(s) or secondary aneurysm(s) protruding from the IA fundus or bi-/multilobular IA fundus.

Computational Fluid Dynamics Simulating and Hemodynamic Assessment
The hemodynamic analysis protocol was referred to our previously conducted studies Yang et al., 2021). For no saccular IAs sited in A3-A5 (anterior cerebral artery), M3-M5 (middle cerebral artery), P3-P4 (posterior cerebral artery), and vertebral artery in this study, we kept the vascular from internal carotid artery to M2 and A2 for IAs sited in anterior circulation and the vascular from basilar artery to P2 for IAs sited in posterior circulation. Meshing was performed using STAR-CCM (STAR-CCM + 12, Siemens, German), which automatically created 4 to 5 million unites of finite tetrahedral, prism elements and optimal boundary layers. The simulations were performed using STAR-CCM fluid workstation (STAR-CCM + 12, Siemens, Germany). The Navier-Stokers equation was employed as the solver in pulsatile blood. To conduct in-depth analyses, the pulsatile waveform of the internal carotid artery (waveform at cervical segmentation, for IAs sited in anterior circulation) and basilar artery (for IAs sited in posterior circulation) from a representative patient (Supplementary Figure 1) were adopted. The pulsatile waveform was obtained using Origin 2018b (OriginLab Corporation, Massachusetts, United States) and was exported to a commaseparated value file for further analysis. Blood was assumed as the incompressible Newtonian fluid. We set the blood as density ρ = 1056 kg/m 3 and viscosity µ = 0.0035 Poise. Pulsatile curve was set as the velocity inlet boundary condition, and velocity boundary condition (the mass flow rate was obtained from a population-based study (Tegeler et al., 2013)) was set at the outlet. Under the residuals < 10 −5 , the results would be considered converged (Tian et al., 2016). A time step of 0.0001s was used. A cardiac cycle was divided into 800 steps (total 0.8 s per cycle). Four pulsatile cycles were simulated. The last cycle was yielded for subsequent studies.
Based on separated IA sacs, we extracted the time-averaged WSS and pressure over a cycle. The oscillatory shear index (OSI) and relative resident time (RRT) were calculated. In addition, the spatially average WSS, pressure, RRT and OSI over the aneurysm surface were obtained. Moreover, the hemodynamic parameters involved here are also listed in Supplementary Table 1. Specific to each model, WSS maximum (WSSM), WSS average (WSSA), WSS gradient (WSSG), and pressure average (PA), were obtained from the IA region, and parent pressure average, parent WSS average were determined according to the parent artery region. Low shear area was defined as the area with WSS < 10% of WSS of parent artery according to previous study (Xiang et al., 2011), and the percentage of low WSS area in IA dome (i.e., low shear area ratio, LSAR) was determined. The normalization of pressure and WSS was performed based on the hemodynamic status of parent artery. Furthermore, the normalized WSS average (NWSSA), normalized pressure average (NPA), and normalized WSS maximum (NWSSM) were calculated, respectively.

Statistical Analysis
Measurement variables were compared using chi-square test or Fisher's exact test. Continuous variables were compared using the  independent samples' t-test. PHASES score and ELAPSS score were calculated by complying with previous protocols (Greving et al., 2014;Backes et al., 2017). The parameters with significance in univariable analysis were inputted into Cox regression model to identify the independent risk factors. The result was expressed as hazard ratio (HR) and 95% confidence interval (CI). The performance of the nomograms to predict the rebleeding was measured with AUCs in receiver operating characteristic curve (ROC) analyses. An AUC > 0.7 was considered a clinical utility. The cutoff value was calculated using the Youden index. To further assess the predictive accuracy of nomograms, the risk, assessed by using nomogram models, was adopted to categorize patients as the high-risk group and the low-risk group at the risk as 50% (with the highest Youden index). The survival analysis was conducted by using Kaplan-Meier model. The statistical analyses were conducted by employing SPSS 24.0 (SPSS, Chicago, IL, United States), with two-sided P < 0.05 showing statistical significance. Subsequently, nomograms to predict the rebleeding were formulated. All parameters of interest as described above fell to three categories, i.e., clinical, morphological and hemodynamic features (Supplementary Table 2). Based on results of the multivariate Cox regression analyses, this study developed two nomogram models incorporating factors independently associated with the primary endpoint, i.e., the clinical + morphological model (CM model) and clinical + morphological + hemodynamic model (CMH model). In addition, the calibration curves were plotted to assess the calibration of the nomograms. The nomograms were subjected to bootstrapping validation (1000 bootstrap resamples). Furthermore, the nomograms were developed with the package of "rms" in R version 3.6.2.

Demographic, Clinical, Radiological, and Hemodynamic Difference in the Primary Cohort
On the whole, 411 appropriate patients with ruptured IAs were reviewed (Figure 1). 70 patients were identified as rebleeding after the admission ranged in age from 49 to 68 years (mean: 58.4 ± 6.0 years). The rebleeding rate was 17.0%. The demographic and clinical information was given in Table 1. Of all patients in the primary cohort, the percentage of male patients was 39.2% (161/411). 34.1% (150/411) patients had hypertension. Several representative cases were presented in Figure 2.

Nomogram Models to Predict the Rebleeding After the Admission
Based on the result of multivariate Cox regression analysis and parameter category, two nomograms, i.e., clinical + morphological model (CM model, Figure 4A) and clinical + morphological + hemodynamic model (CMH model, Figure 4B) were built. With the risk as 50%, all patients were categorized as the high-risk group and the low-risk group. The survival curves are presented as Figures 4C,D. Here, the patients in the high-risk group, recognized by the CM model and CMH model respectively, had higher risk of rebleeding after the admission (both P < 0.001). The calibration plots display a substantial agreement between the prediction by each nomogram and the actual observation, in the risk of rebleeding before the intervention (Figures 5A,B). The ROC analyses based on the primary cohort showed a good predicting accuracy of two nomogram models (AUC = 0.83 and 0.92, respectively), whereas a poor predicting accuracy of PHASES and ELAPSS score was found (AUC = 0.54 and 0.61, respectively). The CMH model had a higher predicting accuracy compared with the CM model (P < 0.05). The results of ROC analyses are summarized as Figures 5C,C' and given in Table 4.

Validation of Predicting Accuracy of Nomogram Models to the Rebleeding After the Admission
The information of the validation cohort was given in Table 5. No significant difference was identified between the primary cohort and the validation cohort (Supplementary Table 3). As indicated from the ROC analyses (Figures 5D,D'), the CMH model exhibited the highest predicting accuracy (AUC = 0.93), followed by CM model (AUC = 0.86); however, the PHASES and ELAPSS score performed poorly (AUC = 0.53 and 0.51, respectively) in predicting the rebleeding after the admission. The result of ROC analyses based on the validation cohort is also listed in Table 4.

DISCUSSION
Rebleeding refers to a main cause of morbidity for aSAH patients.
Existing study reported that the morphology and hemodynamics of IAs would change after aSAH, which makes IAs prone to be stable (Skodvin et al., 2017). However, some RIAs may not reach a stable condition and had a high risk of rebleeding. In this study, we confirmed the predictive value of hemodynamic parameters for rebleeding after the admission and build a predicting model to discriminate the RIAs at high risk of rebleeding. This study demonstrated the relationship between the history of hypertension and rebleeding after the admission. Systematic artery hypertension was recognized as the major cause of cardiovascular disease. Previous cohort and animal studies confirmed that hypertension could increase the risk of IA natural rupture (Lindgren et al., 2014;Tada et al., 2014); thus, the hypertension was considered in subsequent predicting models (Greving et al., 2014;Backes et al., 2017). Here, the hypertension was also found as the independent risk factor for the rebleeding after the admission, demonstrating that the risk of rebleeding of RIA patients with hypertension was approximately 2.7 times that of patients without hypertension. Though the blood pressure was well under control after the admission, the systematic artery hypertension had caused damage to vessels throughout the body before the IAs rupture. Accordingly, this study considered that the risk of rebleeding was higher in patients with hypertension as compared to patients without hypertension.
In this study, the RIAs sited in bifurcation, with irregular shape, and larger AR were found with a high risk of rebleeding after the admission. The IAs sited in bifurcation were more possible to suffer from the impact of blood flow; the dynamic change from direct impact area to surrounding area could cause physical injury to the endothelia of vessels, and thus to the aneurysm wall (Metaxa et al., 2010). In addition to the bifurcation site, irregular shape is also a sign of high risk of rupture. Irregular shape generally suggested a more significantly fragile area in IAs, which could present as bled or second aneurysm protruding from the primary IAs, as compared with the surrounding area in the aneurysm dome. As revealed from existing studies, the bled or second aneurysm in an irregular aneurysm was generally thin and blood-blister like, and the rupture areas were commonly associated with these bled or second aneurysms (Kawaguchi et al., 2012;Jiang et al., 2020). Notably, the hemodynamic condition of the bled aneurysm is generally low WSS and high OSI (Kawaguchi et al., 2012). Thus, it is easy to understand that the RIAs sited in bifurcation with an irregular shape has a higher risk of rebleeding after the admission. The IAs with large AR generally have large size and relatively narrow neck, often with unstable hemodynamic condition and severe damage in the aneurysm wall (Qiu et al., 2017); therefore, this parameter was confirmed as a predictor for IA natural rupture (Backes et al., 2017). Our preliminary study also reported that the AR was a predictor for rebleeding of RIAs. This study demonstrated that the risk of rebleeding increased by 12.9 per 1 of AR, suggesting that AR is FIGURE 5 | The predictive accuracy of models for rebleeding. (A,B) The calibration plots display a substantial agreement between the prediction by each nomogram and the actual observation, in the risk of rebleeding before the surgery. (C,C',D,D') The ROC analyses based on the primary cohort and validation cohort showed a good predicting accuracy of two nomogram models, whereas a poor predicting accuracy of PHASES and ELAPSS score was found. * P < 0.05; * * P < 0.01. a vital parameter to identify the RIAs at high risk of rebleeding after the admission.
This study also confirmed the role of hemodynamic parameters, mainly the WSSA and OSI, in predicting the risk of rebleeding after the admission. The hemodynamic condition could induce inflammation filtration and vessel remodeling in the vascular wall (Albarran-Juarez et al., 2018;Lu et al., 2019;Zhang et al., 2020), which would cause atherosclerosis, IA growth/rupture, etc. Recent studies reported that oscillator flow could induce the inflammation in the vascular wall and inhibit the expression of vascular protective factors (Albarran-Juarez et al., 2018) (e.g., endothelial nitric oxide synthase). Moreover, the low WSS could activate the inflammatory pathway ) (e.g., ROS pathway, pyroptosis pathway and nuclear factor kappa B pathway) to induce inflammation, which could promote the damage of vessels. This study demonstrated that the RIAs with low WSSA and high OSI had a high risk of rebleeding after the admission. Notably, as suggested by existing pathological research, the inflammation in the aneurysm wall is the main cause of IA rupture and growth (Frösen et al., 2004(Frösen et al., , 2012Jiang et al., 2018b;Tulamo et al., 2018). According to Frosen et al., the pathological characteristics of the aneurysm wall fell to four levels by largely complying with the inflammation; the severer the level, the higher the risk of aneurysm rupture will be (Frösen et al., 2004). Since WSS and OSI could induce the inflammation in the vascular wall, the low WSS and the high OSI in RIAs suggested that the damage for the aneurysm wall was continuous after initial hemorrhage; besides, for rebleeding aneurysms, the bleeding stopped before IA reaching a stable status. The mentioned result also demonstrated the clinical utility of computational fluid dynamics. Based on the independent risk factors, two nomogram models were set to identify the RIAs at high risk of rebleeding. The CM model included the clinical and morphological characteristics (e.g., hypertension, bifurcation site, irregular shape, and AR). The hemodynamic characteristics were further integrated with the  other characteristics to build the CMH model. The predicting value of the CM model and CMH model was confirmed as good for clinical utility. Using the risk as 50% assessed by each nomogram model, we confirmed that the rupture risk was higher and interval from initial hemorrhage to rebleeding was shorter in the high-risk group as compared with the lowrisk group. As indicated from the further comparison, these two nomogram models exhibited higher predicting accuracy as compared with PHASES and ELAPSS models. Interestingly, the CMH model had higher predicting accuracy as compared with the CM model. For this phenomenon, this study considered that the stability of IAs mainly involved two aspects, i.e., internal hemodynamic condition and pathological characteristics of the aneurysm wall. Taking multidimensional risk factors into consideration could help to comprehensively understand the stability of RIAs, demonstrating that the CMH model would have a higher predicting accuracy as compared with the CM model. However, the CM model could be more easily handled in clinical work as compared with the CMH model, especially in emergency conditions. Though the hemodynamic analysis is limited in clinical work for its technical barrier and time-consumption now, the tool, i.e., aView, has been reported in previous study (Xiang et al., 2017); therefore, the clinical practical hemodynamic analysis tool would arise to assist in quickly identifying the hemodynamic characteristics of RIAs in the future. In the developing nations, because of large populations but limited medical resources (Bian et al., 2012), the sequence of treatment is essential to make a treatment strategy for IAs. This fact reveals that a patient has to wait a long time for appropriate treatment immediately to get medical intervention after IA rupture. Though the bleeding stopped in some RIAs, RIAs may not reach a real stable condition. This study built two models with good accuracy to identify the high-risk RIAs. To avoid rebleeding, an immediate surgical intervention was recommended for the RIAs at high risk of rebleeding, and a priority should be given to IAs with higher scores.
There are several limitations here. First, the inlet boundary condition was from a representative patient, which can affect the result of computational fluid dynamics since this method is sensitive to velocity and waveform . However, this study used normalized parameters that can reduce the effects exerted by this problem. Second, the morphology can significantly impact the hemodynamics. Due to the change of morphology and hemodynamics after indiscoverable IA rupture (Skodvin et al., 2017) and the effect of hemorrhage on the quality of radiological images, our conclusion may be limited. Third, this study was a single center and retrospective study, which may limit our conclusion. Fourth, this study only considered the utility of PHASES and ELAPSS score in predicting the risk of rebleeding. There were several other models which could help in discriminating the high-risk IAs, e.g., Detmer's model (Detmer et al., 2018). However, the PHASES and ELAPSS score were representative models to discriminate the high-risk IAs; thus, we compared the predictive accuracy between our nomograms and these models. Indications remain the focus in IA treatment, and our models have their clinical utility to help clinical work identify the optimal sequence of treatment, though some limitations remain.

CONCLUSION
Hemodynamic parameters could serve as the predictors for rebleeding after admission. Two nomogram models were presented, i.e., CMH nomogram and CM nomogram, helping to identify the RIAs at high risk of rebleeding. For RIAs at high risk of rebleeding, intervention should be prioritized, and medical treatment is not recommended after rupture.

DATA AVAILABILITY STATEMENT
The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding authors.

ETHICS STATEMENT
This study was approved by the Institutional Review Board of Beijing Tiantan Hospital. Written informed consents were obtained and the privacy of patients was effectively protected.

AUTHOR CONTRIBUTIONS
QL and PJ: conception and design. QL, YY, JY, PJ, ML, SY, and NW: acquisition of data. QL and YY: analysis and interpretation of data. QL: drafting the article. PJ and SW: critically revising the article. SW: approving the final version of the manuscript on behalf of other authors and study supervision. All authors: reviewing the submitted version of manuscript.