Radiomics Nomograms Based on Multi-Parametric MRI for Preoperative Differential Diagnosis of Malignant and Benign Sinonasal Tumors: A Two-Centre Study

Objectives To investigate the efficacy of multi-parametric MRI-based radiomics nomograms for preoperative distinction between benign and malignant sinonasal tumors. Methods Data of 244 patients with sinonasal tumor (training set, n=192; test set, n=52) who had undergone pre-contrast MRI, and 101 patients who underwent post-contrast MRI (training set, n=74; test set, n=27) were retrospectively analyzed. Independent predictors of malignancy were identified and their performance were evaluated. Seven radiomics signatures (RSs) using maximum relevance minimum redundancy (mRMR), and the least absolute shrinkage selection operator (LASSO) algorithm were established. The radiomics nomograms, comprising the clinical model and the RS algorithms were built: one based on pre-contrast MRI (RNWOC); the other based on pre-contrast and post-contrast MRI (RNWC). The performances of the models were evaluated with area under the curve (AUC), calibration, and decision curve analysis (DCA) respectively. Results The efficacy of the clinical model (AUC=0.81) of RNWC was higher than that of the model (AUC=0.76) of RNWOC in the test set. There was no significant difference in the AUC of radiomic algorithms in the test set. The RS-T1T2 (AUC=0.74) and RS-T1T2T1C (RSWC, AUC=0.81) achieved a good distinction efficacy in the test set. The RNWC and the RNWOC showed excellent distinction (AUC=0.89 and 0.82 respectively) in the test set. The DCA of the nomograms showed better clinical usefulness than the clinical models and radiomics signatures. Conclusions The radiomics nomograms combining the clinical model and RS can be accurately, safely and efficiently used to distinguish between benign and malignant sinonasal tumors.


INTRODUCTION
Benign and malignant tumors are widely distributed in the sinonasal area (1)(2)(3). Patients with malignant sinonasal tumors often require surgery, radiotherapy or chemotherapy, and usually have a poor prognosis (4), while patients with benign tumors usually require only clinical follow-up or direct total surgical resection (5,6). Thus, it is important for the radiologist to differentiate between the types (7)(8)(9). In most cases, histopathological analysis is a safe and important diagnostic tool for the evaluation of sinonasal tumors (10). However, its diagnostic sensitivity is low because of the surrounding inflammatory tissue, which usually accompanies the biopsy specimen. CT and MRI imaging play a crucial role in the differentiation of sinonasal tumors (11)(12)(13). However, the image morphologies of malignant and benign sinonasal neoplasms often overlap, and are nonspecific (12,14,15). Hence, a non-invasive, accurate, reliable, and convenient method for distinguishing between benign and malignant sinonasal tumors is necessary.
Radiomics is an emerging method for medical image analysis (16). Radiomics is defined as a favorable biomarker which extracts and analyzes a large number of advanced quantitative features from medical imaging (17,18). This method can identify the heterogeneity and microenvironment of various tumors. Recently, radiomics has been applied broadly to tumor qualitative analysis, evaluation of efficacy, genetic analysis, tumor staging, and prediction of prognosis (19)(20)(21). Additionally, radiomics is fast, economical, and reproducible. Therefore, MRI-based radiomics could be effective to distinguish malignant from benign sinonasal tumors.
However, to the best of our knowledge, radiomics has not been widely used to differentiated malignant from benign sinonasal tumors, and needs further research. The aim of this study was to evaluate the performance of the MRI-based radiomics nomograms in discriminating between benign and malignant sinonasal tumors.

Patients
We searched the medical data of patients with sinonasal tumors admitted at two institutions between March 2006 and April 2020. And the retrospective study was ethically approved by both hospital's institutional review board and the need for written informed consent was waived off. The inclusion criteria were as follows: (a) patients with histopathologically confirmed sinonasal tumors; and (b) patients underwent MRI examination less than 10 days before surgery. And the exclusion criteria were as follows: (a) patients without complete medical data; and (b) Poor quality MRI images such as a signal-to-noise ratio (s/n) ≤1.0. In all, 244 patients with sinonasal tumor were retrospectively included in the research. 192 sinonasal tumors patients from our hospital constituted the training set, while 52 patients from another hospital constituted the test set according to the TRIPOD statement (22). According to the histopathological results and the latest WHO classification (23), all tumors were classified as benign or malignant tumor.

MRI Image Processing and Radiomics Feature Extraction
By using ITK-SNAP software (http://www.itksnap.org) (24), three-dimensional region of interest (3D-ROI) was conducted on the preoperative T1WIs, FS-FSE-T2WIs, and CE-T1WIs. According to the contour of the tumor from each transverse layer, the ROI was outlined on preoperative T1WI, FS-FSE-T2WI, and CE-T1WIs sequences, and turned into a 3D-ROI automatically. The 3D-ROI segmentation included the entire primary tumor while avoided obvious peripheral inflammation, cystic or necrotic regions and macrovessels.
The intra and inter-observer performance during radiomics feature extraction was evaluated by calculating the intra-/interobserver correlation coefficients (ICCs). Two radiologists with five years of working experience drew the 3D-ROIs, who were blinded to the clinical and pathologic information. And the same radiologists performed the second ROI manual segmentation after 1 month. Sinonasal tumors with both intra and interobserver ICCs ≥0.80 were included in subsequent analysis. ICCs above 0.75 indicated good performance.
Feature extraction from medical images was executed on the preoperative T1WIs, FS-FSE-T2WIs, and CE-T1WIs by using the open-source Pyradiomics package (https://github.com/Radiomicss/ pyradiomicss) (25). Many MRI texture features and the heterogeneity within the ROIs can be processed satisfactorily by the software. The MRI texture recognition is improved by using a series of preprocessing methods. First, all MRI images were resampled to voxel size of 1×1×1 mm³. Next, the MRI image intensity was normalized into standardized intensity with a mean value of zero and a standard deviation value of one. Ultimately, quantitative texture features were extracted with 1,224 totally. The seven categories of feature extraction were as follows: shape; grey level co-occurrence matrix; grey level run length matrix; grey level size zone matrix; grey level dependence matrix; first-order statistics and neighborhood grey tone difference matrix.

ComBat Compensation Method
The maximum likelihood method was used to estimate additive and multiplicative batch effects based on a given feature distribution. Recent research showed its potential in improving the repeatability between different centers (26,27). This study corrected the MRIner models by applying the R ComBat script (https://github.com/jfortin1/ComBatHarmonization) (28).

Analysis of Morphological Features of MRI Image
All MRI images were assessed by two radiologists with ten years of working experience who distinguished malignant from benign tumors based on visual assessment, and were blinded to the clinical and pathologic information. The criteria of morphological MRI feature to evaluated sinonasal tumors are as follows: (1) heterogeneity; (2) T1 hyperintensity signal matrix; (3) T2 hypointensity signal matrix; (4) margin (well or ill-defined); (5) size (major axis <5 cm or ≥5 cm); (6) necrosis matrix; (7) myxoid matrix; (8) septations; (9) degree of enhancement (mild, moderate and marked); (10) pattern of enhancement (non-homogeneous) and (11) bone involvement (including osteosclerosis, bone destruction, or both). Presence of necrosis or myxoid matrices, and septations were defined as being beyond 10% of the entire tumor. The above MRI features were selected based on a previous research (29). MRI morphological features1−3 and 6−11 were categorized as positive or negative.

Construction of Radiomics Signature
R software (version 3.5.1) was applied to select MRI feature and construct radiomics model. First, the maximum relevance minimum redundancy (mRMR) algorithm was used to obtain the top 20 relevant features and eliminate the redundant and irrelevant feature for distinction between malignant and benign tumors. Next, the most predictive features were selected by the least absolute shrinkage and selection operator (LASSO) regression analysis. MRI features with non-zero coefficient variables were obtained and partial candidate feature coefficients were compressed to zero. Subsequently, a total of the seven radiomics signatures were constructed through the selected features of single sequences, combined sequences, and multi-parametric sequences as shown in Figure 2. The radiomics scores (rad-scores) of each patient were calculated. The performance of the radiomics signatures in distinguishing malignant from benign tumors were evaluated by the areas under the receiver operating characteristic (ROC) curve (AUC) of the test sets.

Development of Clinical Model and Radiomics Nomogram
By using Wilcoxon's test (continuous variables) and Chi-squaretest (nominal variables), the clinical data and MRI morphological features were analyzed. Univariate logistic analysis was used to analyzed the features with p <0.1, and a multivariate logistic regression analysis was used to select the independent predictive factors for sinonasal malignancy by analyzing features with p <0.05. Factors with p <0.1 were considered significant and were used to create the clinical models. Finally, to establish reliable radiomics nomograms, the clinical models and the best RS were evaluated by multivariate logistic regression. Two different radiomics nomograms for this study were created (one based on pre-contrast MRI with CE T1WI, and another based on pre-contrast MRI).

Performance Assessment of Different Models
To assess the goodness of fit of the radiomics nomograms, the Hosmer-Lemeshow (30) test was executed. The ability of the models to identify malignant and benign tumors was evaluated by comparing the following indicators in the training and test sets: AUC, accuracy, sensitivity, and specificity. The Delong test was used to analyze the AUC between each two of all the models. The reliability and effectiveness of the nomograms was evaluated by using a decision curve analysis (DCA). The performance of DCA was obtained by analyzing the net benefits of a series of threshold probabilities in the entire retrospective cohort (31).

Clinical Information and MRI Features of Patients
The diagnostic results and classification of the 244 patients are shown in Supplementary S1.
The clinical data and MRI morphology of patients with precontrast MRI are shown in Table 1. The results of p <0.10 were considered significant and were included in the subsequent study shown in Table 3. The independent predictors identified for sinonasal tumor malignancy were necrosis, margin, septations, and bone involvement (all p <0.05) based on the results of the univariate analyses. A clinical model involving bone involvement, margin, and septations (all p <0.10) in the test set (AUC=0.76) were established based on the results of the multivariate logistic regression analysis.
The clinical and MRI morphology of patients who underwent pre-contrast and post-contrast MRI are shown in Table 2. The results of p <0.10 were considered significant and were included in the subsequent study shown in Table 3. The malignant predictive factors were bone involvement, heterogeneity, margin, and marked enhancement on MRI (all p <0.05) based on the results of univariate analyses. Based on the results of the multivariate logistic regression analysis, the parameters margin, bone involvement, and marked enhancement (all p <0.10) were included to create a clinical model, with an AUC of 0.81 in the test set. The performance of the clinical model based on precontrast and post-contrast MRI was higher than that of the clinical model based only on pre-contrast MRI.

Performance of the MRI-Radiomics Signatures
AUCs of the seven algorithms are shown in Figure 3. In the training set, the following four algorithms had significant differences in AUC (RS-T1 and RS-T1C, p=0.0002; RS-T2 and RS-T1C, p=0.0038; RS-T1T1C and RS-T1T2, p=0.0087; RS-T2T1C and RS-T1T2, p=0.0304). There was no statistically significant difference between the AUC values of the remaining algorithms. There was no significant difference in the AUC of algorithms in the test set.

Construction of the RNWOC and the RNWC and Performance of Different Models
The RNWC and the RNWOC were subsequently constructed as shown in Figure 4A. RNWC was the combination of the RSWC algorithm and significant clinical factors based on pre-contrast and post-contrast MRI, while the RNWOC integrated the remarkable clinical factors based on pre-contrast MRI and the RS-T1T2 algorithm. The performance of the RNWC and the RNWOC is shown in Tables 4 and 5, respectively. The discrimination performance of the RNWC and the RNWOC was excellent in the test (AUC=0.89 and 0.82, respectively) sets. The calibration curves of the two nomograms indicated that the models were appropriate in both sets as shown in Figure 4. The DCA of the two nomograms are shown in Figure 5. The two

DISCUSSION
The aim of this study was to evaluate the performance of radiomics nomograms for preoperative differentiation between benign and malignant sinonasal tumors. We established two radiomics nomograms for preoperative identification of malignant sinonasal tumors. One was the RNWOC comprising the significant clinical factors based on pre-contrast MRI and RS-T1T2 algorithm. The other was RNWC, integrating the RSWC algorithm and significant clinical factors based on pre-contrast and post-contrast MRI that had favorable values for predicting malignant tumors. Compared to the RNWC, the ability of the RNWOC to identify malignant from benign sinonasal tumors had better generalization based more cases. However, the RNWC achieved relatively better efficacy than the RNWOC. Thus, the two radiomics nomograms can assist in clinical decision-making by preoperative identification of malignant sinonasal tumors. There are different treatment regimens for benign and malignant sinonasal tumors (32,33). Thus, distinction between benign and malignant tumors is essential for therapeutic decisions. In the present study, the morphological features based on the precontrast MRI (heterogeneity, T1 high signal, T2 low signal, size, margin, septations, myxoid, necrosis, and bone involvement) and the traditional clinical data (age and gender) were analyzed. In addition, the pattern of enhancement and degree of enhancement seen on the post-contrast MRI, were also analyzed. The results showed that margin, septations, bone involvement (on pre-contrast MRI) and marked enhancement, margin, bone involvement (precontrast and post-contrast MRI) are significant risk factors to predict malignant sinonasal tumors. However, the significant risk factors based on pre-contrast MRI achieved an AUC of 0.76 in the test set, whereas those based on pre-contrast and post-contrast MRI achieved an AUC of 0.81 in the test set. The clinical model based on pre-contrast and post-contrast MRI was relatively poor at effectively     identifying benign and malignant tumors with 67% accuracy in the test set, while the clinical model based on pre-contrast MRI had an accuracy of 78%. The discriminating ability between these two models is different and unstable. The results might imply that contribution of the traditional clinical data and MRI morphology in the differential diagnosis of malignant and benign sinonasal tumors is limited. In recent times, radiomics had surpassed the traditional visual assessment of CT and MRI images, and has become an effective and reliable image processing method that evaluates both biological and histological characteristics of lesions. Previous studies showed that radiomics had a strong ability to identify malignant from benign tumors. Zhang et al. (34) reported that the model based on radiomics features from TWI, DKI, and quantitative DCE pharmacokinetic parameter maps was a good tool to differentiate malignant and benign breast lesions, with an AUC of 0.92 in the test set. Wang et al. (35) reported that the radiomics nomogram can be used to classify between malignant and benign soft-tissue masses in the extremities with an AUC of 0.94 in the test set. This study analyzed all 1,224 MRI features to establish the radiomics using the mRMR and LASSO algorithm. The mRMR was reliable for the selection of features with more credible coefficients and fewer redundancies (36). LASSO can obtain the reliable features selected in both, ridge regression and subset, and is excellent for feature screening (37). Subsequently, seven radiomics algorithms were built to differentiate between benign and malignant sinonasal tumors. In these algorithms based on the single parametric MRI, we found that RS-T1C had the strongest ability to distinguish benign and malignant tumors. In algorithms based on multi-parametric MRI, RSWC had the best test efficacy. In addition, the accuracy of RSWC was higher than that of RS-T1C in differentiating between benign and malignant sinonasal masses in test set. The results demonstrated that multi-parameter MRI had a higher potential to reflect all information about the tumors, compared to the singleparameter MRI.
The MRI plays an important role in distinguishing between benign and malignant tumors in the sinonasal area. The RNWOC based on pre-contrast MRI images and RNWC based on pre-contrast and post-contrast MRI images were constructed. Depending on the perfusion and permeability of tumor blood FIGURE 5 | DCA of the radiomics nomograms. In the RNWC, the decision curves indicated that the radiomics nomograms were more beneficial than the clinical and RS model when the threshold probability is between 0.1 and 0.9. In the RNWOC, the threshold probability was between 0.2 and 1.0. vessels, the MRI enhanced images had a higher potential to reflect all tumor information compared to the pre-contrast MRI images. Thus, the performance of RNWC in differentiating between malignant and benign tumors in the sinonasal area was better than that of RNWOC. However, there are some limitations of post-contrast MRI that restricts its widespread use in clinical practice. Post-contrast MRI is an invasive procedure. Moreover, contrast agents might cause adverse reactions in some patients, aggravate the metabolic burden of liver and kidney, and be deposited in the brain leading to damage. Hence, the RNWOC based on pre-contrast MRI is widely used in clinical practice. To identify malignant from benign sinonasal tumors more accurately, the RNWC were also available. Further research is necessary to understand which strategy should be implemented.
By fully analyzing and using the clinical and imaging information, tumors in the sinonasal area can be diagnosed and treated more effectively. The MRI morphology and radiomics nomograms are two completely different imaging analysis and processing methods, but both are derived from imaging images. There is no standardization for the evaluation of MRI images, and the interpretation is subjective, based on the experience of the radiologist. However, in routine clinical practice, the identification malignant from benign sinonasal tumors by the radiologists is convenient and cost effective. Nevertheless, radiomics has significant advantages over the traditional image interpretation methods. The results of radiomics nomograms are quantifiable and independent of subjective factors. Moreover, radiomics nomograms can detect microscopic features such as tumor heterogeneity, which might be missed by the human eye. These advantages are significant in differentiating between benign and malignant sinonasal tumors.
A lot of research has been performed to distinguish benign from malignant sinonasal tumors. Wang et al. (38) reported that semiquantitative DCE-MRI parameter was an effective method to identify malignant and benign neoplasms with an AUC of 0.693 and accuracy of 70.4%. Xiao et al. (39) found that the apparent diffusion coefficient (ADC) had a good performance in differentiating between benign and malignant tumors with AUC of 0.754 and accuracy of 68.6%. Zhang et al. (40) analyzed clinical parameters and MRI-based radiomics features of 197 sinonasal tumor patients, and found that the radiomic nomogram with an AUC of 0.91 can effectively predict malignant sinonasal tumors. The research results of this study are basically consistent with previously published relevant research results. Being different from previous studies, more cases, more quantitative radiomic features and postcontrast MR sequences are involved in our research.

LIMITATIONS
There were several limitations in this study. First, the potential selection bias was inevitable since it was retrospective study. Second, the segmentation of ROIs in this study was by manual delineation, and thus could be vulnerable to potential human error. Third, this sample was relatively small, though we included cases from two medical institutions. Future studies with larger sample size are necessary. Fourth, this study has the potential to help in discriminating between benign and malignant sinonasal tumors. However, the image morphologies of malignant and benign sinonasal neoplasms often overlap. Radiomics would help in identifying malignant sinonasal tumor in this condition. And the final diagnosis is still requiring histopathology for confirmation. Further, the combat compensation method was used to preserve the distinguishing characteristics of texture patterns while eliminating influences of the scanner and protocol in this study. This strategy is appropriate to facilitate multicenter radiomic analyses. We will collect more MRI sequences, such as DCE-MRI or ADC, to verify the effectiveness of the method in a future study.

CONCLUSION
In conclusion, we constructed and evaluated two radiomics nomograms in this study, which can effectively distinguish malignant from benign tumors in the sinonasal area. The RNWC had the best performance in differentiating between benign and malignant tumors. The RNWOC is widely used, simple and safe, and has high stability and reliability. We recommend these two models to be used in clinical practice.

DATA AVAILABILITY STATEMENT
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

ETHICS STATEMENT
The studies involving human participants were reviewed and approved by The Affiliated Hospital of Qingdao University and Beijing Friendship Hospital, Capital Medical University. The ethics committee waived the requirement of written informed consent for participation.

AUTHOR CONTRIBUTIONS
S-CB and HZ have contributed equally to this work. D-PH and H-XW conceptualized the study. Y-QG, H-XW, and D-PH contributed to the methodology and validated the study. PZ, S-CB and HZ contributed to the formal analysis and investigation. S-CB and HZ wrote and prepared the original draft. All authors wrote and prepared the original draft. D-PH and Z-CW supervised the study. All authors contributed to the article and approved the submitted version.