Radiomics Based on Digital Mammography Helps to Identify Mammographic Masses Suspicious for Cancer

Abstract

Objectives:

This study aims to build radiomics model of Breast Imaging Reporting and Data System (BI-RADS) category 4 and 5 mammographic masses extracted from digital mammography (DM) for mammographic masses characterization by using a sensitivity threshold similar to that of biopsy.

Materials and Methods:

This retrospective study included 288 female patients (age, 52.41 ± 10.31) who had BI-RADS category 4 or 5 mammographic masses with an indication for biopsy. The patients were divided into two temporal set (training set, 82 malignancies and 110 benign lesions; independent test set, 48 malignancies and 48 benign lesions). A total of 188 radiomics features were extracted from mammographic masses on the combination of craniocaudal (CC) position images and mediolateral oblique (MLO) position images. For the training set, Pearson’s correlation and the least absolute shrinkage and selection operator (LASSO) were used to select non-redundant radiomics features and useful radiomics features, respectively, and support vector machine (SVM) was applied to construct a radiomics model. The receiver operating characteristic curve (ROC) analysis was used to evaluate the classification performance of the radiomics model and to determine a threshold value with a sensitivity higher than 98% to predict the mammographic masses malignancy. For independent test set, identical threshold value was used to validate the classification performance of the radiomics model. The stability of the radiomics model was evaluated by using a fivefold cross-validation method, and two breast radiologists assessed the diagnostic agreement of the radiomics model.

Results:

In the training set, the radiomics model obtained an area under the receiver operating characteristic curve (AUC) of 0.934 [95% confidence intervals (95% CI), 0.898–0.971], a sensitivity of 98.8% (81/82), a threshold of 0.22, and a specificity of 60% (66/110). In the test set, the radiomics model obtained an AUC of 0.901 (95% CI, 0.835–0.961), a sensitivity of 95.8% (46/48), and a specificity of 66.7% (32/48). The radiomics model had relatively stable sensitivities in fivefold cross-validation (training set, 97.39% ± 3.9%; test set, 98.7% ± 4%).

Conclusion:

The radiomics method based on DM may help reduce the temporarily unnecessary invasive biopsies for benign mammographic masses over-classified in BI-RADS category 4 and 5 while providing similar diagnostic performance for malignant mammographic masses as biopsies.

Introduction

In 2020, female breast cancer (BC) became the most common type of cancer with an estimated 2.3 million new cases (11.7%), followed by lung cancer (11.4%) (1). Treatment of BC relies on conducting an accurate diagnosis, including histological, molecular, and clinical phenotypes. Non-invasive imaging techniques such as mammography, ultrasound, and magnetic resonance (MR) are available for qualitative and quantitative analysis of BC in clinical practice. The American College of Radiology Breast Imaging Reporting and Data System (BI-RADS) is a standardized assessment structure that enables radiologists to clearly and concisely communicate results of breast imaging to referring physicians (2). In the fifth edition of the BI-RADS atlas (3), category 4 and 5 breast lesions are defined as suspicious cancerous lesions, and a biopsy is recommended for further diagnosis. Recent studies have shown that a large number of benign lesions are present in category 4 and 5 breast lesions, particularly in the mammography reporting system, exposing these patients to invasive biopsies (4–6). Depending on the technique, the sensitivity values of biopsy results ranged from 87% to >97% (7–9).

Radiomics is a high-throughput image mining technique that aims to enhance the predictive power of medical images by quantifying the morphology, intensity distribution, and texture patterns of lesions. Recent investigators have examined the role of mammography, ultrasound, and MR radiomics in the prediction of molecular subtypes (10–12), lymph node metastasis (13–15), response to neoadjuvant chemotherapy (16–18), recurrence risk (19, 20), and disease-free survival (21, 22) of BC. However, with the initiative of precision medicine (23–25), the reduction in overdiagnosis and overtreatment of breast lesions through non-invasive radiomics method is also a topic worth investigating.

There are four main findings of breast lesions on diagnostic mammography images: masses, calcifications, architectural distortion, and asymmetries. One large sample study (26) showed that BC most often presented as mass at 56%, followed by calcifications at 29%, asymmetry at 12%, and architectural deformities at 4%, and another small sample study (27) suggest that approximately 50% of breast lesions presenting as a mass were ultimately confirmed benign lesions. In addition, mass may be the only finding or one of the combined findings of breast lesions in mammography (we defined these masses as mammographic masses). Although experienced radiologists have a high diagnostic accuracy in identifying benign and malignant mammographic masses, less experienced radiologists sometimes make excess errors (28) such as benign mammographic masses are over-classified as BIRADS category 4 or even 5.

A previous study has shown that combining both craniocaudal (CC) and mediolateral oblique (MLO) positions radiomics data had good classification performance between HER2-enriched BC and non-HER2-enriched BC (11). Here, we combined both CC and MLO positions radiomics data aimed to explore a model with a sensitivity more than 98% for the characterization of BI-RADS category 4 and 5 mammographic masses, thereby achieving a reduction in biopsies of benign lesions at a very low rate of missed malignant lesions.

Materials and Methods

Patients

This retrospective study was granted approval by the local institutional board, and written informed consent was waived. A total of 288 patients’ clinical and mammographic images data were included in this study from December 2018 to February 2021. The inclusion criteria were as follows: (a) patients who had suspected breast tumor accepted mammography and (b) patients with mass as defined by BI-RADS mammography lexicon (occupancy structures with protrude outward in contour on both CC and MLO position images) and classified in category 4 or 5. The exclusion criteria were as follows: (a) patients without a clear benign or malignant pathological result; (b) patients who had multifocal or bilateral mammographic masses; (c) patients accepted biopsy before mammography examination; (d) patients underwent any treatment before mammography screening, including surgery, chemotherapy or radiotherapy, and anti-HER2 therapy.

Imaging and Saving Acquisition

All patients were examined with a GE Senographe Essential (GE Medical Systems, Waukesha, WI), and all mammographic images were saved at 12-bit quantization level and 100-μm pixel size. The mammographic images were not further processed or normalized (29, 30).

Radiomics Analysis of Mammographic Masses

Both CC and MLO position images of all patients were used to conduct mammographic mass masking, and the CC radiomics features and MLO radiomics features were extracted as separate features. Two breast radiologists (radiologist 1, 4 years’ experience; radiologist 2, 10 years’ experience) who were blinded to the pathological results manually masked the masses in 3Dslicer 4.10.2 (www.slicer.org) (Figure 1). A total of 188 radiomics features were extracted from mammographic masses by using the Pyradiomics python package, including 4 shape features, 34 density features, and 150 texture features. Shape features were used to quantify the size and regularity of the mammographic masses, including maximum two-dimensional (2D) diameter and perimeter to surface ratio, with a lower value of perimeter-to-surface ratio indicating a more regular mammographic mass. First-order features refer to radiologically relevant information about the density of mammographic masses such as mean value and kurtosis. Texture features were calculated based on gray-level co-occurrence matrix, gray-level dependence matrix, gray-level run-length matrix, gray-level size zone matrix, and neighborhood gray-tone difference matrix, which were used to quantify the randomness, correlations, variation, homogeneity, and heterogeneity of mammographic masses. The detailed formulae for the calculation of the radiomics features can be found here (31), and the data from radiologist 1 were used to build a radiomics model.

Figure 1

Patients Grouping and Feature Selection

The patients were divided into two temporal sets based on the order in which they accepted their mammography examinations. The training set consisted of the first two-thirds of patients, and the independent test set comprised the last one-third of patients. In order to avoid some potential bias such as model over-fitting, we applied Pearson’s correlation and the least absolute shrinkage and selection operator (LASSO) regression to screen out non-redundant and useful radiomics features in the training set, respectively. For the Pearson’s correlation method, each radiomics feature generated 187 correlation coefficients and 1 corresponding mean absolute correlation coefficient. If two radiomics features had a coefficient exceeding 0.8, the radiomics feature with the larger mean absolute correlation coefficient was deleted. This was implemented in R software version 4.0.1 with package “caret.” For the LASSO regression (alpha=1, no elastic net), a 10-fold cross-validation method with 1 standard error of the minimum mean-square error criteria was used to select radiomics features (32), and corresponding λ values were also be calculated. In this study, the radiomics features with non-zero coefficient at the suitable value of parameter λ were determined as useful radiomics features. This was implemented in R software version 4.0.1 with “glmnet” package.

Radiomics Model Construction and Testing

The malignant mammographic masses were coded as 1, and the benign mammographic masses were coded as 0. The support vector machines (SVMs) with linear kernel (output predicted probability and other parameters are default parameters) were used to construct a radiomics model in this study because of popularity and efficiency in BC (33). This was done in R software version 4.0.1 with “e1071” package. The useful radiomics features were used to construct a radiomics model in the training set for distinguishing between benign and malignant mammographic masses. The receiver operating characteristic curve (ROC) analysis was used to evaluate the classification performance of the radiomics model, including the area under the receiver operating characteristic curve (AUC), the probability threshold value (cut-point) of higher than 98% sensitivity, and corresponding specificity (5). The independent test set was used to test the results of the training set by using the “predict” function, and the probability threshold value was also tested. The ROC analysis was implemented in R software version 4.0.1 with package “pROC.” The workflow of this study is reported in Figure 2.

Figure 2

Relative Importance of Useful Radiomics Features

The magnitude of the coefficients in the LASSO algorithm were used to measure the relative weight of useful features as previously described (34, 35). Furthermore, the useful features were grouped by category, such as shape feature group, first-order feature group, and texture feature group, which were added sequentially to the final SVM model, and the AUC of each addition was calculated to assess whether all three groups of useful features contributed to the model.

Radiomics Model Stability

In this study, a fivefold cross-validation method was used to evaluate the stability of radiomics model. In order not to break the concealment of the test set data, a fivefold cross-validation method was performed separately in the training and test sets. Specifically, the training and test sets were divided into 10 subsets each by fivefold cross-validation method. The “predict” function of R was used to test the diagnostic performance of the radiomics model in these subsets, and the mean and standard deviation of AUCs, sensitivities, and specificities were calculated to measure the stability of the radiomics model. The fivefold cross-validation method was performed in R software version 4.0.1 with package “caret.”

Reproducibility Assessment

The intra-class correlation coefficients (ICCs) of interobserver (radiologist 1 vs. radiologist 2) were calculated to evaluate the reproducibility of the radiomics features extraction. All data from radiologists 1 and 2 were used as separate test sets so that a kappa value for both radiologists could be calculated to assess the diagnostic reproducibility of the radiomics model.

Statistical Analysis

All statistical analyses were performed in R software version 4.0.1. All ROC analysis were based on package “pROC,” and the differences of AUC were calculated on Delong’s test. All confidence intervals (CI) were derived from 1,000 bootstrap replicates. All statistical tests were two-sided, and the Bonferroni’s method was used to adjust for multiple comparisons.

Results

Clinical Data of Patients

This study included 288 female patients (age, 52.41± 10.31) with solitary BI-RADS category 4 or 5 mammographic masses, including 130 cases of malignant mammographic masses and 158 cases of benign mammographic masses. The malignant mammographic masses include invasive ductal carcinoma (n=51), invasive lobular carcinoma (n=37), mucinous carcinoma (n=22), and ductal carcinoma in situ (n=20), while benign mammographic masses include fibroadenomas (n=87), adenosis (n=38), and hyperplasia (n=33).

Patients Grouping and Feature Selection

The training set included 82 malignant and 110 benign mammographic masses; the independent test set consisted of 48 malignant and 48 benign mammographic masses. Baseline characteristics of study population in training and test sets are reported in Table 1, including age, mass size, mass shape, mass margin, breast density, and BI-RADS category. The Pearson’s correlation method screened out 32 non-redundant radiomics features (Supplementary Figure S1), and the LASSO method further selected 14 useful radiomics features (Figures 3A, B).

Figure 3

Radiomics Model Construction and Testing

In the training set, the radiomics model obtained an AUC of 0.934 (95% CI, 0.898–0.971), a threshold of 0.22, a sensitivity of 98.8% [81/82], and a specificity of 60% [66/110]. In the test set, the radiomics model obtained an AUC of 0.901 (95% CI, 0.835–0.961), a sensitivity of 95.8% [46/48], and a specificity of 66.7% [32/48] (Figure 4A).

Figure 4

Relative Importance of Useful Radiomics Features

For the useful single radiomics feature, perimeter-to-surface ratio from shape features had the largest absolute weight value of 1.234, followed by coarseness from texture features of 0.4 and mean from first-order features of 0.389 (Figure 3C, details in Table 2). For useful single-category radiomics features, texture feature group obtained the largest absolute weight value of 1.435, followed by shape feature group of 1.234, and lastly by first-order feature group of 0.96 (Figure 3C, details in Table 2). When shape feature group, first-order feature group, and texture feature group were added sequentially to the final SVM model, ROC analysis showed a significant increase in AUC for each addition (shape feature group, AUC=0.613; shape feature group plus first-order feature group, AUC=0.835; first-order feature group plus and first-order feature group plus texture feature group, AUC=0.934; p<0.001 for each addition) (Figure 3D, details in Table 3).

Radiomics Model Stability

In the training set, the average AUC was 0.9 ± 0.038, average sensitivity was 97.39% ± 3.9%, and average specificity was 50% ± 12.5%. In the test set, the average AUC was 0.915 ± 0.062, average sensitivity was 98.7% ± 4%, and average specificity was 36.7% ± 8.6% (Figure 4B, details in Table 4 and Supplementary Figure S2).

Reproducibility Assessment

Of the 188 extracted radiomics feature, more than 95% [180/188] radiomics features obtained good reproducibility (ICC>0.75) between radiologist 1 and radiologist 2. The radiomics model obtained good agreement between radiologist 1 and radiologist 2 (Cohen’s kappa=0.748; 95% CI, 0.67–0.825; p<0.001).

Discussion

In this study, we built a radiomics model with similar sensitivity to biopsy for predicting malignancy of BI-RADS category 4 and 5 mammographic masses by using the combination of CC and MLO position images from DM. In both training and test sets, the radiomics model obtained specificity by over 60% while preserving sensitivity more than 95.8%. Both AUC and sensitivity were relatively stable, while the specificity was not so stable. These experimental results suggest that the non-invasive imaging radiomics method could achieve similar sensitivity to biopsy while avoiding some benign mammographic masses to undergo unnecessary invasive biopsy.

We are aware of several papers that use mammographic radiomics data to differentiate between benign and malignant breast lesions (30, 36–38), and we are aware that several of these papers incorporated morphological features (36, 37), and several that did not (30, 38). However, most breast lesions that present alone as calcification, architectural distortion, or asymmetries tend to lack distinct grayscale contrast boundaries, which compromises the masking of tumors. Besides, non-invasive imaging techniques with similar biopsy sensitivity are important for differentiating between benign and malignant BI-RADS category 4 and 5 breast lesions for which invasive biopsy was already indicated. To the best of our knowledge, this is the first study to use mammographic radiomics data to predict the malignancy of BI-RADS category 4 and 5 mammographic masses with a sensitivity similar to that of a biopsy, and we believe that the radiomics model is more limited but more relevant.

When using radiomics data to predict the malignancy of breast lesions, commonly used mammography images include CC position alone (37, 39), mixed CC and MLO position (30, 36), and combination of CC position and MLO position (40, 41). Gupta et al. demonstrated that the corresponding first-order and texture features of mammographic masses between CC position and MLO position were not strongly correlated (42), suggesting that the inclusion of first-order and texture features from multiple mammographic positions may impact the accuracy of diagnosis of mammographic masses. Ma et al. has shown that by combining both CC and MLO position, radiomics data had higher classification performance between HER2-enriched BC and non-HER2-enriched BC than using CC position alone and MLO position alone (11). Hence, we applied both CC and MLO position radiomics data to predict the malignancy of mammographic masses suspicious for cancer, and the results showed that this method had good classification performance.

Recent studies have shown that the use of a single random training–test set split may lead to unreliable results in small sample radiomics machine learning studies (43). In our study, we divided the training and test sets based on the chronological order in which patients underwent mammography examinations, and we performed random fivefold cross-validation in training and test sets, respectively, to assess the stability of the radiomics model. Although the specificity gives a large error (12.5%, 8.6%), the AUC (0.038, 0.062) and sensitivity (3.9%, 4%) were relatively stable. This is consistent with previous findings that cross-validation may lead to large error bars for small sample sizes (44).

Of the 188 mammographic radiomics features, the feature selection method screened out 14 useful features in the training set (n=192, ratio 14:1). As suggested by Gillies et al., each radiomics feature requires at least 10 samples to support in a classifier (45). Previous publication has shown that maximum 2D diameter is a useful feature for predicting BC (37). In this study, we found a significant positive correlation between maximum 2D diameter and gray-level size zone matrix-based gray level non-uniformity in the CC position images (r=0.82, p<0.001) and MLO position images (r=0.924, p<0.001) of the training set (Figure 5). Thus, the maximum 2D diameter feature was removed when filtering features in the Pearson’s correlation analysis. It is worth noting that although the size feature was not included in our radiomics model, we do not consider it unimportant in predicting malignancy of mammographic masses suspicious for cancer.

Figure 5

Of the 14 useful mammographic radiomics features, perimeter-to-surface ratio had the greatest weight value of 1.234, suggesting that morphological features remain important in identifying benign lesions or malignancies for mammographic masses suspicious for cancer. The mean density obtained moderate weight values of 0.389 and 0.079, which was higher in the malignant group than in the benign group, indicating that the malignant mammographic masses tend to be higher density than benign mammographic masses. In addition, the malignant group had higher value for inverse difference normalized, zone variance, gray level non-uniformity, and busyness, and lower value for uniformity, skewness, coarseness, small-dependence high gray-level emphasis, and large-dependence low-gray level emphasis, indicating more heterogeneity of malignant mammographic masses in the density and texture patterns (31) (Figure 6). This experimental result is consistent with the pathological fact that malignant tumors tend to be heterogeneous, while benign lesions tend to be homogeneous (46).

Figure 6

Admittedly, our study has several limitations. First, mammographic radiomics data for this study were collected from a single center with a limited number of participants, and further multicenter testing is needed. Second, the mammographic masses were masked by manual method; however, good inter-observer reproducibility was obtained in feature extraction and model diagnosis. Some publications indicate that semi-automatic segmentation method had higher inter-observer reproducibility (47, 48). Further work is needed to construct a semi-automatic segmentation method for mammographic masses. Finally, the radiomics model was constructed by 2D images, which may lose some important information of mammographic masses. However, some publications showed that 2D radiomics features had higher classification performance than 3D radiomics features in lung cancer (47, 48).

In conclusion, a mammographic radiomics model combining both CC and MLO position images had excellent sensitivity and moderate specificity in differentiating malignancies and benign lesions for BI-RADS category 4 and 5 mammographic masses. It may help reduce the temporarily unnecessary invasive biopsies for benign mammographic masses while providing similar diagnostic performance for malignant mammographic masses as biopsies.

Funding

This work was supported by the Scientific Research Foundation for Advanced Talents, Xiang’an Hospital of Xiamen University (no. PM201809170011).

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

• 1

  SungHFerlayJSiegelRLLaversanneMSoerjomataramIJemalAet al. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J Clin (2021) 71:209–49. doi: 10.3322/caac.21660

  • CrossRef
  • Google Scholar

• 2

  RaoAAFeneisJLalondeC. Ojeda-Fournier H. A Pictorial Review of Changes in the BI-RADS Fifth Edition. Radiographics (2016) 36:623–39. doi: 10.1148/rg.2016150178

  • CrossRef
  • Google Scholar

• 3

  BalleyguierCArfi-RoucheJBoyerBGauthierEHelinVLoshkajianAet al. A New Automated Method to Evaluate 2D Mammographic Breast Density According to BI-RADS® Atlas Fifth Edition Recommendations. Eur Radiol (2019) 29:3830–8. doi: 10.1007/s00330-019-06016-y

  • CrossRef
  • Google Scholar

• 4

  StrobelKSchradingSHansenNLBarabaschAKuhlCK. Assessment of BI-RADS Category 4 Lesions Detected With Screening Mammography and Screening US: Utility of MR Imaging. Radiology (2015) 274:343–51. doi: 10.1148/radiol.14140645

  • CrossRef
  • Google Scholar

• 5

  BickelhauptSJaegerPFLaunFBLedererWDanielHKuderTAet al. Radiomics Based on Adapted Diffusion Kurtosis Imaging Helps to Clarify Most Mammographic Findings Suspicious for Cancer. Radiology (2018) 287:761–70. doi: 10.1148/radiol.2017170273

  • CrossRef
  • Google Scholar

• 6

  AlìMD’AmicoNCInterlenghiMManiglioMFazziniDSchiaffinoSet al. A Decision Support System Based on BI-RADS and Radiomic Classifiers to Reduce False Positive Breast Calcifications at Digital Breast Tomosynthesis: A Preliminary Study. Appl Sci (2021) 11:2503. doi: 10.3390/app11062503

  • CrossRef
  • Google Scholar

• 7

  CiattoSHoussamiNAmbrogettiDBianchiSBonardiRBrancatoBet al. Accuracy and Underestimation of Malignancy of Breast Core Needle Biopsy: The Florence Experience of Over 4000 Consecutive Biopsies. Breast Cancer Res Treat (2007) 101:291–7. doi: 10.1007/s10549-006-9289-6

  • CrossRef
  • Google Scholar

• 8

  WangMHeXChangYSunGThabaneL. A Sensitivity and Specificity Comparison of Fine Needle Aspiration Cytology and Core Needle Biopsy in Evaluation of Suspicious Breast Lesions: A Systematic Review and Meta-Analysis. Breast (2017) 31:157–66. doi: 10.1016/j.breast.2016.11.009

  • CrossRef
  • Google Scholar

• 9

  YoukJHKimEKKimMJLeeJYOhKK. Missed Breast Cancers at US-Guided Core Needle Biopsy: How to Reduce Them. Radiographics (2007) 27:79–94. doi: 10.1148/rg.271065029

  • CrossRef
  • Google Scholar

• 10

  LeithnerDHorvatJVMarinoMABernard-DavilaBJochelsonMSOchoa-AlbizteguiREet al. Radiomic Signatures With Contrast-Enhanced Magnetic Resonance Imaging for the Assessment of Breast Cancer Receptor Status and Molecular Subtypes: Initial Results. Breast Cancer Res (2019) 21:106. doi: 10.1186/s13058-019-1187-z

  • CrossRef
  • Google Scholar

• 11

  MaWZhaoYJiYGuoXJianXLiuPet al. Breast Cancer Molecular Subtype Prediction by Mammographic Radiomic Features. Acad Radiol (2019) 26:196–201. doi: 10.1016/j.acra.2018.01.023

  • CrossRef
  • Google Scholar

• 12

  GuoYHuYQiaoMWangYYuJLiJet al. Radiomics Analysis on Ultrasound for Prediction of Biologic Behavior in Breast Invasive Ductal Carcinoma. Clin Breast Cancer (2018) 18:e335–e44. doi: 10.1016/j.clbc.2017.08.002

  • CrossRef
  • Google Scholar

• 13

  HanLZhuYLiuZYuTHeCJiangWet al. Radiomic Nomogram for Prediction of Axillary Lymph Node Metastasis in Breast Cancer. Eur Radiol (2019) 29:3820–9. doi: 10.1007/s00330-018-5981-2

  • CrossRef
  • Google Scholar

• 14

  SunQLinXZhaoYLiLYanKLiangDet al. Deep Learning vs. Radiomics for Predicting Axillary Lymph Node Metastasis of Breast Cancer Using Ultrasound Images: Don't Forget the Peritumoral Region. Front Oncol (2020) 10:53. doi: 10.3389/fonc.2020.00053

  • CrossRef
  • Google Scholar

• 15

  YangJWangTYangLWangYLiHZhouXet al. Preoperative Prediction of Axillary Lymph Node Metastasis in Breast Cancer Using Mammography-Based Radiomics Method. Sci Rep (2019) 9:4429. doi: 10.1038/s41598-019-40831-z

  • CrossRef
  • Google Scholar

• 16

  WangZLinFMaHShiYDongJYangPet al. Contrast-Enhanced Spectral Mammography-Based Radiomics Nomogram for the Prediction of Neoadjuvant Chemotherapy-Insensitive Breast Cancers. Front Oncol (2021) 11:605230. doi: 10.3389/fonc.2021.605230

  • CrossRef
  • Google Scholar

• 17

  LiuZLiZQuJZhangRZhouXLiLet al. Radiomics of Multiparametric MRI for Pretreatment Prediction of Pathologic Complete Response to Neoadjuvant Chemotherapy in Breast Cancer: A Multicenter Study. Clin Cancer Res (2019) 25:3538–47. doi: 10.1158/1078-0432.Ccr-18-3190

  • CrossRef
  • Google Scholar

• 18

  DiCenzoDQuiaoitKFatimaKBhardwajDSannachiLGangehMet al. Quantitative Ultrasound Radiomics in Predicting Response to Neoadjuvant Chemotherapy in Patients With Locally Advanced Breast Cancer: Results From Multi-Institutional Study. Cancer Med (2020) 9:5798–806. doi: 10.1002/cam4.3255

  • CrossRef
  • Google Scholar

• 19

  YuFHangJDengJYangBWangJYeXet al. Radiomics Features on Ultrasound Imaging for the Prediction of Disease-Free Survival in Triple Negative Breast Cancer: A Multi-Institutional Study. Br J Radiol (2021) 94:20210188. doi: 10.1259/bjr.20210188

  • CrossRef
  • Google Scholar

• 20

  LiHZhuYBurnsideESDrukkerKHoadleyKAFanCet al. MR Imaging Radiomics Signatures for Predicting the Risk of Breast Cancer Recurrence as Given by Research Versions of MammaPrint, Oncotype DX, and PAM50 Gene Assays. Radiology (2016) 281:382–91. doi: 10.1148/radiol.2016152110

  • CrossRef
  • Google Scholar

• 21

  ParkHLimYKoESChoHHLeeJEHanBKet al. Radiomics Signature on Magnetic Resonance Imaging: Association With Disease-Free Survival in Patients With Invasive Breast Cancer. Clin Cancer Res (2018) 24:4705–14. doi: 10.1158/1078-0432.Ccr-17-3783

  • CrossRef
  • Google Scholar

• 22

  XiongLChenHTangXChenBJiangXLiuLet al. Ultrasound-Based Radiomics Analysis for Predicting Disease-Free Survival of Invasive Breast Cancer. Front Oncol (2021) 11:621993. doi: 10.3389/fonc.2021.621993

  • CrossRef
  • Google Scholar

• 23

  CollinsFSVarmusH. A New Initiative on Precision Medicine. N Engl J Med (2015) 372:793–5. doi: 10.1056/NEJMp1500523

  • CrossRef
  • Google Scholar

• 24

  HodsonR. Precision Medicine. Nature (2016) 537:S49. doi: 10.1038/537S49a

  • CrossRef
  • Google Scholar

• 25

  JamesonJLLongoDL. Precision Medicine–Personalized, Problematic, and Promising. N Engl J Med (2015) 372:2229–34. doi: 10.1056/NEJMsb1503104

  • CrossRef
  • Google Scholar

• 26

  VenkatesanAChuPKerlikowskeKSicklesEASmith-BindmanR. Positive Predictive Value of Specific Mammographic Findings According to Reader and Patient Variables. Radiology (2009) 250:648–57. doi: 10.1148/radiol.2503080541

  • CrossRef
  • Google Scholar

• 27

  VarelaCKarssemeijerNHendriksJHHollandR. Use of Prior Mammograms in the Classification of Benign and Malignant Masses. Eur J Radiol (2005) 56:248–55. doi: 10.1016/j.ejrad.2005.04.007

  • CrossRef
  • Google Scholar

• 28

  BoyerBCanaleSArfi-RoucheJMonzaniQKhaledWBalleyguierC. Variability and Errors When Applying the BIRADS Mammography Classification. Eur J Radiol (2013) 82:388–97. doi: 10.1016/j.ejrad.2012.02.005

  • CrossRef
  • Google Scholar

• 29

  WangLYangWXieXLiuWWangHShenJet al. Application of Digital Mammography-Based Radiomics in the Differentiation of Benign and Malignant Round-Like Breast Tumors and the Prediction of Molecular Subtypes. Gland Surg (2020) 9:2005–16. doi: 10.21037/gs-20-473

  • CrossRef
  • Google Scholar

• 30

  LiZYuLWangXYuHGaoYRenYet al. Diagnostic Performance of Mammographic Texture Analysis in the Differential Diagnosis of Benign and Malignant Breast Tumors. Clin Breast Cancer (2018) 18:e621-e7. doi: 10.1016/j.clbc.2017.11.004

  • CrossRef
  • Google Scholar

• 31

  van GriethuysenJJMFedorovAParmarCHosnyAAucoinNNarayanVet al. Computational Radiomics System to Decode the Radiographic Phenotype. Cancer Res (2017) 77:e104–e7. doi: 10.1158/0008-5472.Can-17-0339

  • CrossRef
  • Google Scholar

• 32

  TibshiraniR. Regression Shrinkage and Selection via the Lasso. J R Stat Soc Ser B-Methodol (1996) 58:267–88. doi: 10.1111/j.2517-6161.1996.tb02080.x

  • CrossRef
  • Google Scholar

• 33

  OzerMESaricaPOArgaKY. New Machine Learning Applications to Accelerate Personalized Medicine in Breast Cancer: Rise of the Support Vector Machines. Omics (2020) 24:241–6. doi: 10.1089/omi.2020.0001

  • CrossRef
  • Google Scholar

• 34

  LiJWCaoYCZhaoZJShiZTDuanXQChangCet al. Prediction for Pathological and Immunohistochemical Characteristics of Triple-Negative Invasive Breast Carcinomas: The Performance Comparison Between Quantitative and Qualitative Sonographic Feature Analysis. Eur Radiol (2022) 32(3):1590–600. doi: 10.1007/s00330-021-08224-x

  • CrossRef
  • Google Scholar

• 35

  LiZChenFZhangSMaXXiaYShenFet al. The Feasibility of MRI-Based Radiomics Model in Presurgical Evaluation of Tumor Budding in Locally Advanced Rectal Cancer. Abdom Radiol (NY) (2022) 47:56–65. doi: 10.1007/s00261-021-03311-5

  • CrossRef
  • Google Scholar

• 36

  MaoNYinPWangQLiuMDongJZhangXet al. Added Value of Radiomics on Mammography for Breast Cancer Diagnosis: A Feasibility Study. J Am Coll Radiol (2019) 16:485–91. doi: 10.1016/j.jacr.2018.09.041

  • CrossRef
  • Google Scholar

• 37

  LiHMendelKRLanLShethDGigerML. Digital Mammography in Breast Cancer: Additive Value of Radiomics of Breast Parenchyma. Radiology (2019) 291:15–20. doi: 10.1148/radiol.2019181113

  • CrossRef
  • Google Scholar

• 38

  KarahaliouASkiadopoulosSBoniatisISakellaropoulosPLikakiEPanayiotakisGet al. Texture Analysis of Tissue Surrounding Microcalcifications on Mammograms for Breast Cancer Diagnosis. Br J Radiol (2007) 80:648–56. doi: 10.1259/bjr/30415751

  • CrossRef
  • Google Scholar

• 39

  AcciavattiRJCohenEAMaghsoudiOHGastouniotiAPantaloneLHsiehMKet al. Incorporating Robustness to Imaging Physics Into Radiomic Feature Selection for Breast Cancer Risk Estimation. Cancers (Basel) (2021) 13(21):5497. doi: 10.3390/cancers13215497

  • CrossRef
  • Google Scholar

• 40

  LeiCWeiWLiuZXiongQYangCYangMet al. Mammography-Based Radiomic Analysis for Predicting Benign BI-RADS Category 4 Calcifications. Eur J Radiol (2019) 121:108711. doi: 10.1016/j.ejrad.2019.108711

  • CrossRef
  • Google Scholar

• 41

  WangSSunYLiRMaoNLiQJiangTet al. Diagnostic Performance of Perilesional Radiomics Analysis of Contrast-Enhanced Mammography for the Differentiation of Benign and Malignant Breast Lesions. Eur Radiol (2022) 32:639–49. doi: 10.1007/s00330-021-08134-y

  • CrossRef
  • Google Scholar

• 42

  GuptaSMarkeyMK. Correspondence in Texture Features Between Two Mammographic Views. Med Phys (2005) 32:1598–606. doi: 10.1118/1.1915013

  • CrossRef
  • Google Scholar

• 43

  AnCParkYWAhnSSHanKKimHLeeSK. Radiomics Machine Learning Study With a Small Sample Size: Single Random Training-Test Set Split may Lead to Unreliable Results. PloS One (2021) 16:e0256152. doi: 10.1371/journal.pone.0256152

  • CrossRef
  • Google Scholar

• 44

  VaroquauxG. Cross-Validation Failure: Small Sample Sizes Lead to Large Error Bars. Neuroimage (2018) 180:68–77. doi: 10.1016/j.neuroimage.2017.06.061

  • CrossRef
  • Google Scholar

• 45

  GilliesRJKinahanPEHricakH. Radiomics: Images Are More Than Pictures, They Are Data. Radiology (2016) 278:563–77. doi: 10.1148/radiol.2015151169

  • CrossRef
  • Google Scholar

• 46

  WangXWanQChenHLiYLiX. Classification of Pulmonary Lesion Based on Multiparametric MRI: Utility of Radiomics and Comparison of Machine Learning Methods. Eur Radiol (2020) 30:4595–605. doi: 10.1007/s00330-020-06768-y

  • CrossRef
  • Google Scholar

• 47

  ShenCLiuZGuanMSongJLianYWangSet al. 2D and 3D CT Radiomics Features Prognostic Performance Comparison in Non-Small Cell Lung Cancer. Transl Oncol (2017) 10:886–94. doi: 10.1016/j.tranon.2017.08.007

  • CrossRef
  • Google Scholar

• 48

  LacroixMFrouinFDirandASNiocheCOrlhacFBernaudinJFet al. Correction for Magnetic Field Inhomogeneities and Normalization of Voxel Values Are Needed to Better Reveal the Potential of MR Radiomic Features in Lung Cancer. Front Oncol (2020) 10:43. doi: 10.3389/fonc.2020.00043

  • CrossRef
  • Google Scholar