Development and Validation a Nomogram Incorporating CT Radiomics Signatures and Radiological Features for Differentiating Invasive Adenocarcinoma From Adenocarcinoma In Situ and Minimally Invasive Adenocarcinoma Presenting as Ground-Glass Nodules Measuring 5-10mm in Diameter

Purpose To develop and validate a nomogram for differentiating invasive adenocarcinoma (IAC) from adenocarcinoma in situ (AIS) and minimally invasive adenocarcinoma (MIA) presenting as ground-glass nodules (GGNs) measuring 5-10mm in diameter. Materials and Methods This retrospective study included 446 patients with 478 GGNs histopathologically confirmed AIS, MIA or IAC. These patients were assigned to a primary cohort, an internal validation cohort and an external validation cohort. The segmentation of these GGNs on thin-slice computed tomography (CT) were performed semi-automatically with in-house software. Radiomics features were then extracted from unenhanced CT images with PyRadiomics. Radiological features of these GGNs were also collected. Radiomics features were investigated for usefulness in building radiomics signatures by spearman correlation analysis, minimum redundancy maximum relevance (mRMR) feature ranking method and least absolute shrinkage and selection operator (LASSO) classifier. Multivariable logistic regression analysis was used to develop a nomogram incorporating the radiomics signature and radiological features. The performance of the nomogram was assessed with discrimination, calibration, clinical usefulness and evaluated on the validation cohorts. Results Five radiomics features remained after features selection. The model incorporating radiomics signatures and four radiological features (bubble-like appearance, tumor-lung interface, mean CT value, average diameter) showed good calibration and good discrimination with AUC of 0.831(95%CI, 0.772~0.890). Application of the nomogram in the internal validation cohort with AUC of 0.792 (95%CI, 0.712~0.871) and in the external validation cohort with AUC of 0.833 (95%CI, 0.729-0.938) also indicated good calibration and good discrimination. The decision curve analysis demonstrated that the nomogram was clinically useful. Conclusion This study presents a nomogram incorporating the radiomics signatures and radiological features, which can be used to predict the risk of IAC in patients with GGNs measuring 5-10mm in diameter individually.


INTRODUCTION
Lung cancer is one of the most commonly diagnosed human malignancy and the leading cause of cancer-related death worldwide (1). Adenocarcinoma is the most common histologic type of lung cancer and its incidence has increased over the past few decades, accounting for more than 40% of the total nowadays (2). It was classified into atypical adenomatous hyperplasia (AAH), adenocarcinoma in situ (AIS), minimally invasive adenocarcinoma (MIA) and invasive adenocarcinoma (IAC) in the 2015 World Health Organization (WHO) classification of lung tumors (3). Patients with IAC have a higher risk of recurrence and are usually treated with lobectomy (4). Nowadays, segmentectomy is suggested for selected patients with clinical N0 IAC of no more than 2cm in diameter (4). Patients with AIS or MIA(AIS/MIA) are managed with active surveillance or sublobar resection because of excellent prognosis (5). The subtypes of adenocarcinoma are currently determined mainly by biopsy or postoperative pathological sections in clinical practice, which are invasive and risky. Discriminating IAC from AIS/MIA before surgery could help clinicians to assess prognosis in order to improve clinical decision making and avoid over-or undertreatment, without the need for invasive procedures.
Adenocarcinoma frequently presents as pulmonary nodules including ground-glass nodules (GGNs) and solid nodules on computed tomography (CT). Radiological features such as air bronchogram, margin, pleural indentation have been found to be related with the malignancy or tumor histology of GGNs (6)(7)(8). These features are subjective, qualitative, and sometimes are not easily to be determined in small nodules with a diameter less than 10mm. Nodules with diameter less than 5mm are usually benign (9), however, some nodules less than 10mm have been pathologically confirmed as IAC (10).
Radiomics refers to high-throughput extraction of large amounts of image features from radiographic images (11,12). Radiomics features can be calculated by computational methodologies to quantify the characteristics of tumor tissues and provide a detailed and comprehensive characterization of the tumor phenotype (13). Compared with conventional biomarkers, radiomics-based features are three-dimensional and the process of image acquisition is easy to perform, noninvasive and cost-effective. Several studies have shown the value of radiomics-based features in differentiating tumor subtypes by using different medical imaging modalities such as CT (14), magnetic resonance imaging (MRI) (15,16) and positron emission tomography (PET) (17). Radiomics biomarkers have also been shown to be associated with several clinical events or endpoints, including tumor diagnosis (benign/malignant) (18), tumor subtyping (19), treatment response (20), patient survival (21), tumor recurrence and distant metastasis (22), tumor gene expression (23).
The purpose of this study was to investigate the ability of CT radiomics features combined with CT radiological features to differentiate IAC from AIS/MIA, and develop a nomogram incorporating CT radiomics signatures and radiological features to provide an individual, preoperative assessment of the risk of IAC in patients with GGNs measuring 5-10 mm.

Patient Cohort
Surgical datasets of three hospitals were reviewed. Patients were selected if they presented as lung nodules on chest CT scans and were diagnosed as pulmonary adenocarcinomas on the basis of pathologic analysis of surgical specimens. The nodules with the histopathological results AIS, MIA or IAC and the average diameter of nodule between 5mm and 10mm in CT scans were included. The exclusion criteria were as follows: 1) no routine CT examination had been performed in the month before surgery; 2) a series of consecutive CT images with a thickness of more than 1 mm; 3) CT images with severe respiratory motion artifacts; 4) the average diameter of nodule was smaller than 5mm or larger than 10mm; 5) the nodule presenting as a solid nodule. Some patients may have more than one nodule. These nodules were analyzed independently because they may be of different types.
A total of 354 eligible patients from hospital 1 and 2 were included, 219 patients with 230 GGNs between September 2015 and December 2017 in the primary cohort and 135 patients with 154 GGNs between January 2018 and July 2019 in the internal validation cohort. A total of 92 patients with 94 GGNs in hospital 3 between October 2016 and October 2020 were included in the external validation cohort. The flowchart of patient selection is listed in Figure 1. The study was approved by the institutional review boards of participating hospitals.

Radiological Features Extraction and Radiomics Features Extraction
Several radiological features were recorded. Two quantitative features were average diameter (defined as the mean of the longest diameter of the nodule and its perpendicular diameter at the same maximum axial slice) and mean CT value (measured for a sufficiently large round or oval regions of interest within the nodule on the maximum axial section). Qualitative features included nodule location (right upper lobe, right middle lobe, right lower lobe, left upper lobe, left lower lobe), type of nodule (pure GGN or mixed GGN), bubble-like appearance, pleural indentation, air bronchogram, pulmonary blood vessel change, margin defined as lobulation, spiculation or tumor-lung interface. Bubble-like appearance was defined as air-attenuated, vesicle-like lucency within the nodule. Air bronchogram sign was defined as the presence of ladiolucent bronchi within lesions. Vessels convergence or vessels dilatation within GGNs indicated pulmonary blood vessel change. Lobulation was defined when a portion of the nodule's surface showed a wavy or scalloped configuration. Spiculation was defined as the presence of strands extending from the margin of the nodule into the lung parenchyma without reaching the pleural surface. Tumor-lung interface was recorded as clear if the nodules were well-defined.
The measurements were performed by two radiologists with more than 5 years of experience in chest radiology. The two radiologists measured each imaging feature independently, and the difference was reevaluated by the third radiologist with more than 20 years of experience in chest radiology. Any disagreements were resolved by consensus.
Nodule segmentation was performed semi-automatically with in-house software (24) and manually reviewed slice by slice by a radiologist with 6 years of experience in chest CT imaging and confirmed by another radiologist with 20 years of experience. A slice example of the nodule segmentation was provided in supplementary figure 1. After nodule segmentation, radiomics features were extracted from each nodule with open source PyRadiomics software (https://pyradiomics.readthedocs.io/en/ latest/index.html) using the default settings. The software automatically calculated radiomics features for each included nodule.

Radiomics Feature Selection
A total of 1525 features were extracted from CT images (the detailed radiomics features list was described in supplementary material). First, the variance of features close to 0 were removed. Pair-wise Spearman correlation analysis was performed to identify the redundant features. Features with the mean absolute correlation higher than 0.9 was considered redundant and eliminated. Then, a multivariable ranking method (minimum redundancy maximum relevance [mRMR]) was used to identify the most important features based on a heuristic scoring criterion, and only the top-ranked features were kept. Next, the top-ranking radiomics features were input into the least absolute shrinkage and selection operator (LASSO), which is suitable for regression of high-dimensional data, to obtain the optimal subset of radiomics features to build the radiomics signature for the evaluation of IAC and AIS/MIA. The receiver operating characteristic curve (AUC) was plotted versus log(l) in order to identify the optimal value of log(l). The optimal value was identified by the minimum criterion. The radiomics score (rad-score) of each GGN was calculated via a linear combination of selected features that were weighted by their respective coefficients.

Model Building and Performance Assessment
The significance of associations with IAC and AIS/MIA was evaluated using the Fisher exact test for qualitative features and Mann-Whitney U test for mean CT value and average diameter. Two-sided p<0.1 was considered to indicate significant difference for qualitative features and p<0.05 for quantitative features.
The significantly different radiological features between the IAC group and the AIS/MIA group in the primary cohort combined with rad-score were included in the subsequent multivariable logistic regression analysis. Forward and backward step-wise selection was applied using the likelihood ratio test. We determined the optimal combinations of the features using the AKaike information criterion (AIC) (25). A nomogram was then constructed based on the multivariable logistic model. The discrimination of the nomogram was assessed with the AUC and validated in two validation cohorts. The calibration curves were used to assess the calibration of the nomogram. The goodness-of-fit of the nomogram was assessed with the Hosmer-Lemeshow test.

Clinical Usefulness of Nomogram
To evaluate the potential clinical diagnostic effects of the nomogram model, a decision curve analysis was performed, which quantified the net benefits of using such a model at different threshold probabilities.

Statistical Analysis
Statistical analyses were conducted with R software (version 3.6.3). The spearman correlation analysis was performed using the "caret" package. LASSO logistic regression was performed using the "glmnet" package. Logistic regression, nomogram construction and calibration plots were performed using the "rms" package. The decision curve was plotted using the "rmda" package. The Hosmer-Lemeshow test was done with the "vcdExtra" package. The ROCs were plotted and the DeLong test was used for pairwise comparisons between models using the "pROC" package. A twosided p value <0.05 was considered significant.

Patients' Characteristics
Patients' basic characteristics and nodule information in the primary and the validation cohorts are listed in Table 1. There were no statistically significant differences in gender distribution and age group between the primary cohort and the internal validation cohort, or between the primary cohort and the external validation cohort. Spiculation, lobulation, air bronchogram, and pulmonary blood vessel change didn't show statistically significant difference between the IAC group and the AIS/MIA group either in the primary cohort or two validation cohorts. Mixed GGN and bubblelike appearance were significantly more common in the IAC group both in the primary cohort and two validation cohorts. Average diameter and mean CT value were significantly higher in the IAC group both in the primary cohort and two validation cohorts.

Radiomics Features Selection and Radiomics Model Building
A total of 97 features with variance close to 0 were removed. Subsequently, after pair-wise spearman analysis, 246 features with the mean absolute correlation less than 0.9 remained. These features were ranked by mRMR, and then the top 100 features were selected. The LASSO classifier was trained on the primary cohort using the top 100 features. Five features with nonzero coefficients in the LASSO logistic model were selected (Figure 2). The rad-score was calculated for each patient based on the formula presented in the supplementary material.

Nomogram Model Building, Assessment, and Validation
The radiological features that showed significant difference in univariate analyses in the primary cohort were included in multivariable logistic regression analysis. The predictors associated with IAC were bubble-like appearance, tumor-lung interface, mean CT value and average diameter. A nomogram model that incorporated these predictors and radscore was developed ( Table 2) and presented as the nomogram (Figure 3). The nomogram model yielded an AUC of 0.831 (95%CI, 0.772-0.890) in the primary cohort ( Figure 4A The calibration curves of the nomogram are shown in Figure 5. The Hosmer-Lemeshow test yielded a nonsignificant p value in the primary cohort, 0.225 in the internal validation cohort and 0.115 in the external validation cohort, which indicated good calibration power.

Clinical Usefulness of the Nomogram
The decision curve analysis showed that the nomogram had a higher overall net benefit, which indicated that the nomogram was clinically useful (Figure 6A). With a threshold probability of 10%, use of the nomogram could provide an added net benefit compared to the "treat-all" or "treat-none" strategy. Moreover, the similar findings were also observed in the internal validation cohort ( Figure 6B) and the external validation cohort ( Figure 6C).

DISCUSSION
We developed and validated a nomogram incorporating radiomics signature and radiological features for individualized preoperative predicting the risk of IAC in patients with GGNs  measuring 5-10mm. The results showed that the discrimination and calibration of the nomogram model was favorable. This study provided a non-invasive preoperative prediction tool to identify patients with GGNs in a high risk of IAC. The nomogram model finally incorporated the rad-score, based on five radiomics features, and four radiological features, according to AIC. Bubble-like appearance was more common in the IAC group than that in the AIS/MIA group, which was also found in the former study (26). Nodule diameter has always been considered as an important indicator in nodule management. Our model also identified the average diameter as an independent predictor for IAC prediction. In our study, clear tumor-lung interface was more common in the AIS/MIA group than that in the IAC group both in the primary cohort and the internal validation cohort, which was contrary to two previous studies (27,28). The study by Wu et al. (27) included atypical adenomatous hyperplasia as preinvasive lesion. If atypical adenomatous hyperplasia was excluded, there was no significant difference in terms of tumor-lung interface between the IAC group and the AIS/MIA group, which was consistent with that in our external validation cohort. The study by Jin et al. (28) included nodules with diameter less than 30mm, while the diameter of nodules in our study was between 5-10mm. In addition, both studies (27,28) included pure GGN only. Further studies are needed to confirm the relationship between tumor-lung interface and the invasiveness of lung adenocarcinoma. Mean CT value was higher in the IAC group in our study, which was consistent with the study by She et al. (29). Increased mean CT value reflected the increased heterogeneity of GGN (30). Zhao et al. (31) constructed a model included radiomics signature and mean CT value to predict the invasiveness of nodules. Another study (32) demonstrated that the AUC of a model constructed to distinguish between invasive and non-invasive lesions including only mean CT value reached 0.808.
The present radiomics signatures consisted of five radiomics features. Root mean squared (RMS) is first-order histogram feature. It also remained in radiomics model in the study by Weng et al. (33) and in the study by She et al. (29). RMS is related to the characteristics of the intensity distribution in the pulmonary nodules. Both dependence entropy and large dependence high gray level emphasis are gray level dependence matrix features which indicate the relationship between the gray-level intensity of CT voxels and the invasiveness of GGNs. The higher value indicated more heterogeneity in the texture patterns. Wavelet.LHL_ gldm_DependenceEntropy and gradient_ glszm_ZoneEntropy are radiomics features undergoing image transformation with a filter. Both are calculated from gray-level intensity features. The higher values of these features in the IAC group meant that IAC was more   (34) had a higher AUC of 0.98. One reason might be the study included only part-solid nodules. The radiomics model combined ground-glass and solid features. In addition, the larger diameter of the pulmonary nodules in that study might be another reason. Even so, there are some limitations in this study. First, the ratio of the IAC group and the AIS/MIA group was consistent with the actual clinical scenario. We didn't selectively collect the samples to balance the two groups, so the imbalanced sample ratio of IAC and AIS/MIA may have had an impact on the nomogram model. Second, the CT images in this study came from four different CT scanner, which may cause potential variability because of different parameters. Third, the reconstruction matrix of 512*512 for small GGNs may limit the diagnosis ability of radiomics. Scanning and reconstruction of local regions of the target images can reduce the size of pixels and increase the information of segmented areas of small GGNs, thus improving the diagnosis ability. Higher pixel matrix, such as 1024*1024 or 2048*2048 could break the limitation of CT image reconstruction matrix and improve the diagnosis ability. Last, the data collection is retrospective, a larger prospective longitudinal cohort is needed to confirm the performance of our nomogram model.
In summary, this study presents a nomogram incorporating radiomics features and radiological features of CT images to predict the risk of IAC in patients with GGNs measuring 5-10mm in diameter. The nomogram can serve as a potential tool to guide individual diagnosis and help clinician choose the optimal intervention.

DATA AVAILABILITY STATEMENT
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

ETHICS STATEMENT
The studies involving human participants were reviewed and approved by The Institutional Review Board of Shanghai Public Health Clinical Center, Fudan University and Affiliated Hospital of Nantong University. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

AUTHOR CONTRIBUTIONS
LS: the acquisition of data, analysis of data, and drafting the article. WS: the acquisition of data and the interpretation of data. XP: the analysis of data. YZ: the acquisition of data. LZ: the acquisition of data. YW: the acquisition of data. MF: the acquisition of data. JZ: the acquisition of data. FS: the conception and design of the study, revising the article, and final approval of the version to be The black line represents the assumption that no patients have IAC. The gray line represents the assumption that all patients have IAC. The red line represents the net benefit of using the nomogram model to predict IAC. The decision curve demonstrates that if the threshold probability is >10%, using the nomogram for IAC prediction adds more benefit than predicting either all or no patients. IAC: invasive adenocarcinoma.
submitted. LL: the conception of the study, revising the article, and final approval of the version to be submitted. All authors contributed to the article and approved the submitted version.