ORIGINAL RESEARCH article

Front. Oncol., 05 January 2026

Sec. Cancer Imaging and Image-directed Interventions

Volume 15 - 2025 | https://doi.org/10.3389/fonc.2025.1707586

Ultrasound radiomics-based machine learning models for risk stratification of follicular thyroid tumors

  • 1. Department of ultrasound, The First Affiliated Hospital With Nanjing Medical University, Nanjing, China

  • 2. Department of nuclear medicine, The Shanghai Tenth Clinical Medical College with Nanjing Medical University, Shanghai, China

Abstract

Background:

Follicular thyroid carcinoma (FTC) is the second most common malignant thyroid tumor. Preoperative differentiation among follicular thyroid adenoma (FA), follicular tumor of uncertain malignant potential (FT-UMP), and FTC remains challenging using conventional ultrasound and fine-needle aspiration. This study aims to develop a machine learning model utilizing ultrasound radiomic features to improve risk stratification of follicular thyroid tumors.

Methods:

A total of 277 patients with histopathologically confirmed follicular tumors (163 FA, 63 FT-UMP, 51 FTC) were included. Clinical and ultrasound features, along with radiomic features from intratumoral and peritumoral regions, were extracted from preoperative ultrasound images. Three machine learning models—logistic regression (LR), support vector machine (SVM), and random forest (RF)—were trained to construct four models: clinical-ultrasound (U), clinical-ultrasound with intratumoral radiomics (UI), clinical-ultrasound with peritumoral radiomics (UP), and clinical-ultrasound with combined intratumoral and peritumoral radiomics (UIP).

Results:

The RF-based clinical-ultrasound model demonstrated the highest accuracy (test: 0.643) but exhibited significant overfitting in radiomics-based models. The SVM model showed moderate performance. The LR model in the UP and UIP models delivered stable performance, achieving the highest test accuracy of 0.643. Specifically, the UP model showed improved micro-AUC, specificity, negative predictive value (NPV), and F1 score. The LR model exhibited high sensitivity but low specificity for benign nodules, and high specificity but low sensitivity for malignant nodules. All models performed poorly in identifying FT-UMP nodules.

Conclusion:

Integrating peritumoral radiomic features with clinical-ultrasound features using logistic regression enhances the differentiation between benign and malignant follicular thyroid tumors.

1 Introduction

Follicular thyroid carcinoma (FTC) is the second most common thyroid malignant tumor, accounting for about 17% of all thyroid cancers (). About 10–15% of FTC patients develop distant metastases to the lungs and bones, leading to poorer survival outcomes compared to papillary thyroid carcinoma (PTC) (–). Thus, early diagnosis of FTC is crucial for treatment and prognosis.

Ultrasound is the primary imaging modality for evaluating thyroid nodules, enabling classification based sonographic features and assessment of malignant potential (). The currently widely used TI-RADS classification systems are built on this, while they are mainly established based on the ultrasound features for PTC, and have certain limitations in the diagnosis of follicular tumors (, ). In ultrasonographic images, the FTC has some similarities with benign (Follicular Thyroid Adenoma,FA) and low-risk (Follicular Tumor of Uncertain Malignant Potential, FT-UMP) nodules (). Even fine-needle aspiration (FNA) cannot reliably differentiate among these entities (). Only complete removal of the lesion and exploration of its capsular infiltration can clearly determine the specific type of the nodules (, ). In addition, the clinical management of low-risk nodules is also controversial.2025 ATA Management Guidelines for Adult Patients with Differentiated Thyroid Cancer state that further treatment with completion thyroidectomy/lymphadenectomy and/or radioactive iodine therapy for low-risk nodules like FT-UMP is not advised routinely. The optimal approach to monitoring of these tumors should be determined according to the surgical approaches, laboratory findings, and the patient’s wishes (). Therefore, extracting more diagnostic information from ultrasound images to find a more accurate method for differentiating follicular tumors, so as to reduce unnecessary surgeries/therapies, is currently the focus of research. Previous studies (–) have mostly focused on differentiating FTA and FTC, with limited attention to low-risk follicular tumors. It is worth noting that FT-UMPs are defined as well-differentiated thyroid tumors with follicular architecture that are encapsulated or unencapsulated but well-circumscribed, in which invasion remains questionable after thorough sampling and exhaustive examination (, ). That means FT-UMP is generally an indolent disease, but some patients still show distant recurrence. Due to its inherent uncertainties, there are certain challenges in determining the clinical follow-up period or making surgical decisions ().

The aim of this study is to develop a hierarchical model based on ultrasound images using machine learning methods, in order to assess the risk stratification of follicular tumors and provide a reference for clinical decision-making.

2 Method

2.1 Patients

From January 2020 to September 2023, 277 patients with histopathologically confirmed follicular thyroid tumors (163 FTA, 63 FT-UMP, 51 FTC) were included. Preoperative neck ultrasound was performed by an operator with over 10 years of experience using EPIQ5 (Philips Healthcare) equipped with a 6–18 MHz linear transducer. The following sonographic features were recorded: nodule number (single, multiple); location (upper, middle, lower, isthmus); maximum diameter (≤1 cm, 1–4 cm, >4 cm); echogenicity (hyperechoic, isoechoic, hypoechoic); aspect ratio (<1, ≥1); composition (solid, predominantly solid, predominantly cystic); calcifications (absent,microcalcification,macrocalcification,rim calcification);shape (round-to-oval, irregular); margin (smooth, spiculated, unclear); halo (absent, regular complete, irregular interrupted); blood flow (grade 0: no signal; grade 1: partial clear signals; grade 2: moderate signals; grade 3: abundant signals); elasticity score (1–5).Ultrasound features were independently evaluated by 2 attending physicians with more than 5 years of experience in the thyroid imaging. They had no prior knowledge of the pathological findings. Features in which there was disagreement were recorded after agreement between the two physicians. Features with intraclass correlation coefficients (ICCs) greater than the reproducibility threshold of 0.9, both in intraobserver and interobserver assessments, were screened out for subsequent analysis.

2.2 Ultrasound radiomic analysis

2.2.1 Clinical and ultrasound features

Univariate and multivariate logistic analysis(ordered multinomial logistic regression) was conducted on clinical and ultrasound features. The response variable is a ordinal variable with three classes. Features with a statistical value of P < 0.05 were included in the subsequent analysis.

2.2.2 Radiomic features

ITK-SNAP 3.6.0 was used to manually delineate intratumoral regions of interest (ROIs) on DICOM images of the maximum cross section on grayscale US images, which was regarded as the intratumoral ROI. Peritumoral ROIs were generated by expanding the intratumoral boundary outward by 2 voxels. To appraise the feature reliability, reader 1 and reader 2 (with 5 years of experience in thyroid imaging) independently outlined ROIs on images from 30 randomly selected patients. Reader 2 repeated the same process within 1 week. The intra and interobserver reproducibility were assessed by the ICC, and an ICC greater than 0.9 indicated good consistency. Radiomics features were automatically extracted using an open-source Python package, PyRadiomics (https://pyradiomics.readthedocs.io/).

Feature values were normalized via Z-score transformation (Equation 3) (). The normalization method is as follows:

column: represents a certain column (feature) in the dataset; mean: the mean of this column; std: the standard deviation of this column.

One-way ANOVA was conducted on all the normalized ultrasound radiomics features of FTA, FT-UMP and FTC groups, and only those with P < 0.05 were retained. The Pearson correlation coefficient was used to calculate the correlation between features. To maximize the discriminative power of radiomics features, we employed a correlation-based greedy recursive backward elimination strategy for feature selection. This method iteratively identifies and removes the most redundant features based on internal correlation structure. In each iteration, we computed absolute pairwise correlation coefficients among all remaining features and calculated the average correlation for each feature. The feature showing the highest average correlation, indicating the least unique information was eliminated. This process repeated recursively until the maximum pairwise correlation among remaining features fell below a predefined threshold(0.9). As a model-agnostic preprocessing approach, this method effectively mitigates multicollinearity while preserving the most representative features, thereby enhancing model robustness and interpretability. By explicitly addressing feature redundancy, our approach provides a computationally efficient framework for feature subset selection.

The selected features was randomly divided into a training cohort and a test cohort at a ratio of 8:2. Then, we input the features into a polynomial logistic regression model with L1 penalty term of the Least Absolute Shrinkage and Selection Operator (LASSO) algorithm to directly solve the three-classification problem. The penalty coefficient λ of LASSO was optimized for selection by 5-fold cross validation. The final selected results were included in the subsequent analysis.

2.3 Model construction

These features (clinical and ultrasound and radiomic features) were then input into machine learning models (Logistic Regression, LR;Support Vector Machine, SVM;Random Forest, RF) to construct intratumoral and intratumoral-peritumoral dual-region radiomics ultrasound radiomics models. Stratified five-fold cross-validation was used to find the optimal parameters of the models and determine the final models.

We also included a SHapley Additive exPlanations(SHAP) ()method to enhance the transparency and explainability of the best-performing model by prioritizing the importance of features, in terms of assessing the contribution to model performance.

The workflow of this study is presented in Figure 1.

Figure 1

This study was approved by the Ethics Committee on October 25.2023.with the approval no.2023-SR-691.This study complied with the Declaration of Helsinki.

2.4 Statistical analysis

Statistical analyses were performed using IBM SPSS 26.0 (IBM, New York, USA). The normality of the data was verified via the Kolmogorov-Smirnov test. Continuous data conforming to a normal distribution are expressed as the mean ± standard deviation (SD) and were compared using one-way ANOVA; non-normal data are expressed as the median [interquartile range (IQR)] and were compared using the Kruskal-Wallis H test. Categorical data are expressed as numbers (%) and were compared using a chi-square test or Fisher’s exact test, as appropriate. A P < 0.05 represented a statistically significant difference.

3 Result

The clinical and ultrasound features in the training cohort and the test cohort (shown in Table 1) were included in the univariate and multivariate logistic regression analysis. The results (Table 2, Figure 2) showed that shape, margin, shadow and elasticity score(P<0.05) were independent risk factors. Based on these factors, machine learning models were constructed.

Table 1

FeaturesTraining cohortTest cohort
BenignLow-riskMalignantP valueBenignLow-riskMalignantP value
Age M (Q1, Q3)45.000 (34.000, 56.000)51.000 (39.000, 59.250)42.000 (28.000, 57.000)0.09648.000 (40.000, 57.000)52.000 (39.000, 55.000)36.000 (31.250, 46.250)0.149
Gender0.2650.651
male39(30.000)9(18.000)11(26.829)9(27.273)3(23.077)4(40.000)
female91(70.000)41(82.000)30(73.171)24(72.727)10(76.923)6(60.000)
Nodule Number0.2270.519
single110(84.615)46(92.000)38(92.683)30(90.909)13(100.000)9(90.000)
multiple20(15.385)4(8.000)3(7.317)3(9.091)–1(10.000)
Location0.1480.437
upper19(14.615)9(18.000)5(12.195)7(21.212)1(7.692)2(20.000)
middle39(30.000)23(46.000)19(46.341)14(42.424)10(76.923)5(50.000)
lower65(50.000)17(34.000)17(41.463)9(27.273)2(15.385)3(30.000)
iethmus7(5.385)1(2.000)–3(9.091)––
Nodule max diameter0.0050.626
≤1 cm9(6.923)5(10.000)–3(9.091)––
1-4cm74(56.923)31(62.000)15(36.585)18(54.545)8(61.538)5(50.000)
>4 cm47(36.154)14(28.000)26(63.415)12(36.364)5(38.462)5(50.000)
Echogenicity0.0070.267
hypoechoic21(16.154)12(24.000)18(43.902)8(24.242)5(38.462)5(50.000)
isoechoic107(82.308)37(74.000)22(53.659)25(75.758)8(61.538)5(50.000)
hyperechoic2(1.538)1(2.000)1(2.439)–––
Aspect Ratio0.1731
<1129(99.231)48(96.000)41(100.000)33(100.000)13(100.000)10(100.000)
≥11(0.769)2(4.000)––––
Composition0.0370.194
solid53(40.769)25(50.000)27(65.854)16(48.485)5(38.462)8(80.000)
predominantly solid65(50.000)20(40.000)14(34.146)12(36.364)4(30.769)2(20.000)
predominantly cystic12(9.231)5(10.000)–5(15.152)4(30.769)–
Calcifications<0.0010.089
absent91(70.000)35(70.000)14(34.146)25(75.758)9(69.231)6(60.000)
rim calcification8(6.154)2(4.000)2(4.878)2(6.061)––
microcalcification25(19.231)5(10.000)7(17.073)4(12.121)––
macrocalcification6(4.615)8(16.000)18(43.902)2(6.061)4(30.769)4(40.000)
Shape0.0520.097
round to oval121(93.077)46(92.000)33(80.488)29(87.879)13(100.000)7(70.000)
irregular9(6.923)4(8.000)8(19.512)4(12.121)–3(30.000)
Margin<0.0010.154
smooth124(95.385)47(94.000)34(82.927)30(90.909)13(100.000)8(80.000)
spiculated5(3.846)3(6.000)–2(6.061)––
unclear1(0.769)–7(17.073)1(3.030)–2(20.000)
Shadow0.0020.464
absent93(71.538)28(56.000)23(56.098)20(60.606)7(53.846)5(50.000)
regular complete halo33(25.385)19(38.000)10(24.390)12(36.364)4(30.769)3(30.000)
irregular interrupted halo4(3.077)3(6.000)8(19.512)1(3.030)2(15.385)2(20.000)
Blood Flow0.0280.670
08(6.154)1(2.000)–3(9.091)––
16(4.615)3(6.000)––––
226(20.000)12(24.000)2(4.878)4(12.121)2(15.385)1(10.000)
390(69.231)34(68.000)39(95.122)26(78.788)11(84.615)9(90.000)
Elasticity score0.0110.083
11(0.769)–––––
275(57.692)28(56.000)12(29.268)19(57.576)8(61.538)2(20.000)
351(39.231)19(38.000)29(70.732)14(42.424)5(38.462)8(80.000)
43(2.308)3(6.000)––––
5–––

The clinical and ultrasound features in the training cohort and the test cohort.

Values are presented as count (%).

Differences were considered statistically significant at P<0.05.

Table 2

VariablesUnivariateMultivariate
βS.EtPOR (95%CI)βS.EtPOR (95%CI)
Gender
male1.00 (Reference)
female0.20.270.720.4731.22 (0.71 ~ 2.08)
Nodule Number
single1.00 (Reference)
multiple-0.730.43-1.720.0860.48 (0.21 ~ 1.11)
Location
upper1.00 (Reference)
middle0.450.351.280.21.57 (0.79 ~ 3.11)
lower-0.150.36-0.420.6730.86 (0.42 ~ 1.74)
iethmus-1.91.09-1.740.0810.15 (0.02 ~ 1.27)
Nodule max diameter
≤1 cm1.00 (Reference)
1-4cm0.530.540.970.3311.69 (0.59 ~ 4.88)
>4 cm0.990.551.80.0722.69 (0.92 ~ 7.88)
Echogenicity
hypoechoic1.00 (Reference)1.00 (Reference)
isoechoic-1.020.27-3.78<.0010.36 (0.21 ~ 0.61)-1.380.34-4.06<.0010.25 (0.13 ~ 0.49)
hyperechoic-0.370.97-0.380.7060.69 (0.10 ~ 4.67)-0.851.02-0.830.4040.43 (0.06 ~ 3.16)
Aspect Ratio
<11.00 (Reference)
≥10.370.950.390.6941.45 (0.22 ~ 9.39)
Composition
solid1.00 (Reference)1.00 (Reference)
predominantly solid-0.660.25-2.60.0090.52 (0.31 ~ 0.85)-0.210.31-0.670.5050.81 (0.44 ~ 1.49)
predominantly cystic-0.830.43-1.930.0530.44 (0.19 ~ 1.01)-1.140.58-1.980.0480.32 (0.10 ~ 0.99)
Calcifications
absent1.00 (Reference)1.00 (Reference)
rim calcification-0.230.61-0.380.7040.79 (0.24 ~ 2.61)0.690.70.980.3262.00 (0.50 ~ 7.94)
microcalcification-0.160.38-0.420.6710.85 (0.41 ~ 1.78)-0.770.51-1.520.1270.46 (0.17 ~ 1.25)
macrocalcification2.040.345.96<.0017.68 (3.93 ~ 15.01)2.780.456.17<.00116.07 (6.65 ~ 38.81)
Shape
round to oval1.00 (Reference)1.00 (Reference)
irregular0.840.392.140.0322.31 (1.07 ~ 4.96)-1.810.71-2.540.0110.16 (0.04 ~ 0.66)
Margin
smooth1.00 (Reference)1.00 (Reference)
spiculated-0.580.68-0.850.3930.56 (0.15 ~ 2.12)-15.830.49-32.51<.0010.00 (0.00 ~ 0.00)
unclear2.970.83.69<.00119.51 (4.03 ~ 94.47)5.631.154.91<.001279.02 (29.48 ~ 2640.70)
Shadow
absent1.00 (Reference)1.00 (Reference)
regular complete halo0.280.261.080.2811.33 (0.79 ~ 2.22)0.810.342.410.0162.26 (1.16 ~ 4.37)
irregular interrupted halo1.720.463.78<.0015.60 (2.29 ~ 13.69)2.120.633.35<.0018.36 (2.42 ~ 28.97)
Blood Flow
01.00 (Reference)1.00 (Reference)
11.551.241.250.2114.73 (0.41 ~ 54.14)-0.681.49-0.460.6470.51 (0.03 ~ 9.30)
21.741.081.60.1095.68 (0.68 ~ 47.38)1.571.121.40.1624.80 (0.53 ~ 43.30)
32.251.052.140.0329.53 (1.21 ~ 74.86)1.261.081.160.2463.52 (0.42 ~ 29.43)
elasticity score
11.00 (Reference)1.00 (Reference)
213.950.2361.69<.0011139545.58 (731645.99 ~ 1774853.06)18.260.445.42<.00185332689.84 (38803681.67 ~ 187654048.30)
314.70.2364.47<.0012424724.69 (1550837.14 ~ 3791042.71)18.420.4244.35<.001100165968.84 (44376073.94 ~ 226095290.18)
414.280.5526.03<.0011592077.02 (543234.89 ~ 4665954.38)36.160.4974.28<.0015036124771593869.00 (1939834166018742.75 ~ 13074598416376352.00)
Age00.010.470.6411.00 (0.99 ~ 1.02)

Univariate and multivariate logistic regression analysis results of clinical and ultrasound features.

OR, Odds Ratio, CI, Confidence Interval.

Differences were considered statistically significant at P<0.05.

Bold values indicate values with P < 0.05.

Figure 2

1561 radiomic features were extracted for the tumor interior and the tumor periphery, including 14 shape-based features; 306 first-order statistics; and 1241 second-order features. The extracted features were screened out based on the independent sample t-test (P < 0.05) for 129,57and 198 intratumoral, peritumoral and intratumoral-peritumoral dual-region features respectively. Then, 35,24 and 59 selected features were filtered based on the correlation coefficient. Finally, using the LASSO regression algorithm, the non-zero coefficients of the features were reduced to 14,15 and 24 predictive features respectively. Coefficients and mean standard error of 5 folds validation is show in Figure 3. Rad score is show as follow, Figure 4 shows the coefficients value in the final selected non-zero features. Then these features were input into machine learning models to construct ultrasound imaging radiomics models for intratumoral, peritumoral and intratumoral-peritumoral dual-region.

Figure 3

Figure 4

3.1 Clinical-ultrasound features model

As shown in Table 3 & Table 4, the RF model achieved the highest accuracy (0.647 train,0.643 test)and micro-AUC(0.805 train,0.772 test),and also demonstrates the most stable ROC performance across different nodule classifications (Figure 5). SVM model showed moderate performance, with a slight decrease in test performance (accuracy: 0.593 train vs. 0.571 test),which was the worst performance among the three models. The accuracy of the LR model was moderate, but its micro-AUC was the lowest among the three models (0.661train,0.669test).

Table 3

CohortModelAccuracyCohortModelAccuracy
TrainU-LR0.588TestU-LR0.589
U-SVM0.593U-SVM0.571
U-RF0.647U-RF0.643
UI-LR0.588UI-LR0.589
UI-SVM0.588UI-SVM0.589
UI-RF1UI-RF0.589
UP-LR0.62UP-LR0.643
UP-SVM0.606UP-SVM0.625
UP-RF1UP-RF0.625
UIP-LR0.615UIP-LR0.643
UIP-SVM0.606UIP-SVM0.625
UIP-RF1UIP-RF0.661

The accuracy for each tumor type.

Table 4

GroupCohortModelLabelAccAUC95% CISensitivitySpecificityPPVNPVPrecisionRecallF1Threshold
U-modelTrainLRMicro-AUC0.7240.6610.614-0.7070.5840.7940.5860.7920.5860.5840.5850.451
Benign0.5880.3790.304-0.454100.58800.58810.7410.451
Low-risk0.7740.4890.400-0.5780100.7740000.246
Malignant0.8140.3630.270-0.4570100.8140000.19
SVMMicro-AUC0.7320.7120.669-0.7560.5610.8170.6050.7880.6050.5610.5820.417
Benign0.6150.5310.455-0.6080.9920.0770.6060.8750.6060.9920.7520.606
Low-risk0.7740.4970.410-0.5850100.7740000.226
Malignant0.8420.4880.376-0.6010.1710.9940.8750.840.8750.1710.2860.417
RFMicro-AUC0.7710.8050.770-0.8400.5840.8640.6830.8060.6830.5840.6290.34
Benign0.6610.690.620-0.7600.9770.2090.6380.8640.6380.9770.7720.606
Low-risk0.7830.6390.555-0.7230.04110.78110.040.0770.235
Malignant0.8510.7790.697-0.8610.3410.9670.70.8660.70.3410.4590.267
TestLRMicro-AUC0.7260.6690.577-0.7620.5890.7950.5890.7950.5890.5890.5890.516
Benign0.5890.3960.244-0.548100.58900.58910.7420.516
Low-risk0.7680.5690.390-0.7480100.7680000.211
Malignant0.8210.2970.117-0.4760100.8210000.19
SVMMicro-AUC0.720.7080.622-0.7940.5360.8120.5880.7780.5880.5360.5610.417
Benign0.6070.5330.380-0.6860.970.0870.6040.6670.6040.970.7440.606
Low-risk0.7680.3740.228-0.5200100.7680000.225
Malignant0.8390.5250.283-0.7670.20.9780.6670.8490.6670.20.3080.417
RFMicro-AUC0.7320.7720.698-0.8470.50.8480.6220.7720.6220.50.5540.43
Benign0.6790.5820.424-0.7390.970.2610.6530.8570.6530.970.780.43
Low-risk0.7680.5440.361-0.7270100.7680000.171
Malignant0.8390.7470.557-0.9360.40.9350.5710.8780.5710.40.4710.267
UI-modelTrainLRMicro-AUC0.7060.7110.668-0.7530.3570.880.5980.7330.5980.3570.4480.298
Benign0.5880.5430.466-0.619100.58800.58810.7410.662
Low-risk0.7740.540.454-0.6270100.7740000.196
Malignant0.8140.5280.439-0.6160100.8140000.17
SVMMicro-AUC0.7250.7170.676-0.7590.5880.7940.5880.7940.5880.5880.5880.234
Benign0.5880.520.443-0.598100.58800.58810.7410.589
Low-risk0.7740.550.459-0.6420100.7740000.229
Malignant0.8140.6220.536-0.7070100.8140000.186
RFMicro-AUC111.000-1.00011111110.515
Benign111.000-1.00011111110.698
Low-risk111.000-1.00011111110.515
Malignant111.000-1.00011111110.538
TestLRMicro-AUC0.7440.7330.650-0.8160.3750.9290.7240.7480.7240.3750.4940.266
Benign0.5890.6030.451-0.755100.58900.58910.7420.5
Low-risk0.7680.6060.427-0.7860100.7680000.266
Malignant0.8210.530.345-0.7160100.8210000.188
SVMMicro-AUC0.7260.6810.592-0.7710.5890.7950.5890.7950.5890.5890.5890.235
Benign0.5890.4680.314-0.621100.58900.58910.7420.586
Low-risk0.7680.3310.167-0.4950100.7680000.235
Malignant0.8210.4330.177-0.6890100.8210000.188
RFMicro-AUC0.7080.7190.636-0.8030.4640.830.5780.7560.5780.4640.5150.36
Benign0.6250.5480.390-0.7060.9090.2170.6250.6250.6250.9090.7410.418
Low-risk0.6960.4360.262-0.6090.0770.8840.1670.760.1670.0770.1050.098
Malignant0.8570.7430.569-0.9180.2110.85210.20.3330.142
UP-modelTrainLRMicro-AUC0.7440.7780.740-0.8150.4750.8780.660.770.660.4750.5530.372
Benign0.6520.6630.591-0.7360.9380.2420.6390.7330.6390.9380.760.62
Low-risk0.7780.6330.550-0.7170.120.9710.5450.790.5450.120.1970.213
Malignant0.810.710.620-0.8000.220.9440.4740.8420.4740.220.30.229
SVMMicro-AUC0.7350.7510.711-0.7900.5930.8050.6040.7980.6040.5930.5980.321
Benign0.620.6470.572-0.7210.9850.0990.610.8180.610.9850.7530.6
Low-risk0.7740.5410.449-0.6330100.7740000.232
Malignant0.8190.7330.649-0.8170.1460.9720.5450.8330.5450.1460.2310.169
RFMicro-AUC111.000-1.00011111110.529
Benign111.000-1.00011111110.674
Low-risk111.000-1.00011111110.543
Malignant111.000-1.00011111110.529
TestLRMicro-AUC0.7740.7740.695-0.8540.4820.920.750.780.750.4820.5870.35
Benign0.7140.7050.550-0.86010.3040.67310.67310.8050.434
Low-risk0.7320.4540.272-0.63700.95300.7590000.353
Malignant0.8390.80.623-0.9770.30.9570.60.8630.60.30.40.252
SVMMicro-AUC0.7560.7620.685-0.8380.6070.830.6420.8090.6420.6070.6240.443
Benign0.6960.6860.533-0.84010.2610.6610.6610.7950.595
Low-risk0.7680.3940.197-0.5900100.7680000.236
Malignant0.8210.7960.662-0.9290.30.9350.50.860.50.30.3750.163
RFMicro-AUC0.7740.7950.726-0.8650.5540.8840.7050.7980.7050.5540.620.387
Benign0.6250.6610.514-0.8090.9390.1740.620.6670.620.9390.7470.52
Low-risk0.7680.5850.434-0.7360100.7680000.173
Malignant0.8570.8320.678-0.9850.40.9570.6670.880.6670.40.50.284
UIP-modelTrainLRMicro-AUC0.7540.7840.748-0.8210.480.8910.6880.7740.6880.480.5650.34
Benign0.6470.6770.605-0.7480.9380.2310.6350.7240.6350.9380.7580.505
Low-risk0.7830.6590.577-0.7420.120.9770.60.7910.60.120.20.205
Malignant0.8010.720.632-0.8070.1950.9390.4210.8370.4210.1950.2670.183
SVMMicro-AUC0.7390.7480.708-0.7880.5930.8120.6120.80.6120.5930.6020.289
Benign0.6110.6370.563-0.7110.9920.0660.6030.8570.6030.9920.750.596
Low-risk0.7740.5330.440-0.6260100.7740000.234
Malignant0.8190.7270.645-0.8090.0980.9830.5710.8270.5710.0980.1670.17
RFMicro-AUC111.000-1.00011111110.535
Benign111.000-1.00011111110.632
Low-risk111.000-1.00011111110.573
Malignant111.000-1.00011111110.535
TestLRMicro-AUC0.7320.7820.707-0.8560.4290.8840.6490.7560.6490.4290.5160.407
Benign0.7140.7010.551-0.8510.970.3480.6810.8890.6810.970.80.416
Low-risk0.750.5380.359-0.71800.97700.7640000.322
Malignant0.8210.780.631-0.9300.40.9130.50.8750.50.40.4440.202
SVMMicro-AUC0.7620.7440.658-0.8290.6250.830.6480.8160.6480.6250.6360.505
Benign0.6790.7270.589-0.86510.2170.64710.64710.7860.599
Low-risk0.7680.4420.229-0.6550100.7680000.239
Malignant0.8040.7570.610-0.9030.20.9350.40.8430.40.20.2670.166
RFMicro-AUC0.7740.8150.746-0.8830.4640.9290.7650.7760.7650.4640.5780.336
Benign0.6610.7330.593-0.8740.8790.3480.6590.6670.6590.8790.7530.48
Low-risk0.7320.6230.459-0.7880.1540.9070.3330.780.3330.1540.2110.262
Malignant0.9290.8330.651-1.0000.6110.9210.60.750.336

Results of clinical-ultrasound, intratumoral and peritumoral radiomics features models.

Figure 5

All models showed high sensitivity(>0.9) but low specificity(<0.3) for benign nodules, and high specificity(>0.9) but low sensitivity(<0.4) for malignant nodules, while performance was poor in low-risk group with near-zero sensitivity. Among them, RF achieved the highest F1-scores for malignant nodules (0.459 train, 0.471 test).

3.2 Ultrasound radiomics-based models

3.2.1 Clinical-ultrasound and intratumoral radiomics features model

As shown in Table 3 & Table 4, the RF model achieved a perfect score on all metrics (Accuracy=1.0, AUC = 1.0, Sensitivity=1.0, Specificity=1.0)in training cohorts but a significant performance drop on the test cohort, confirming the overfitting of the data.

The accuracy of the LR model in the UI model was consistent with that in the U model, and also with that of the SVM model in the UI model(0.589 test).However, LR’s test micro-AUC was higher than SVM model(0.733 vs 0.681)).The ROC of LR model was also higher than that of SVM model in different nodule classifications (Figure 5).

All models were highly biased in assessing benign, low-risk, and malignant nodules, with sensitivity and specificity data close to 0 or 1.

3.2.2 Clinical-ultrasound and peritumoral radiomics features model

As shown in Tables 3, 4, the RF model showed a clear overfitting performance again.

LR model showed slightly higher accuracy than SVM model(0.643 test vs 0.625 test), and both higher than that in UI model. The same is true in terms of ROC (Figure 5).

Notably, in the malignant nodule group, although the sensitivity(=0.3) was still low, the AUC and F1 of the models were significantly increased compared with U models in test cohort(UP model(LR AUC 0.8,F1 0.4;SVM AUC 0.796,F1 0.375) vs U model(LR AUC 0.297,F1 0.0;SVM AUC 0.525,F1 0.0) vs UI model(LR AUC 0.53,F1 0.0;SVM AUC 0.433,F1 0.0)).

3.2.3 Clinical-ultrasound and intratumoral-peritumoral dual-region radiomics features model

The accuracy of LR model and SVM model in UIP model was consistent with that in UP model(test 0.643,0.625), and the accuracy and ROC of LR model was higher than that of SVM model.

Despite its overfitting, the RF model’s test performance for the malignant group was the highest of all models(accuracy 0.929, AUC 0.833,sensitivity 0.60,NPV 0.92,F1 0.75), as shown in Tables 3, 4, Figure 5.

Figure 6 presents the confusion matrices for the all models, showing the classification performance for the three tumor classifications.

Figure 6

3.3 Comparison of diagnostic efficacy between UP-LR model and TI-RADS

ACR-TIRADS was used to classify all included nodules as benign, low-risk, and malignant. We performed ROC curve analysis in three separate rounds(benign vs low-risk and malignant; low-risk vs benign and malignant; malignant vs benign and low-risk). In each round, the target TI-RADS category and its corresponding pathological result were coded as 0, while all other categories and pathological types were combined and coded as 1. ROC curve analysis was then employed to calculate the AUC and its confidence interval for that specific category and then compared with that of the best-performing model (UP-LR model),as showed in Table 5. In the diagnosis of benign nodules, the AUC of the UP-LR model was 0.705(95% CI 0.550–0.860); the AUC of TI-RADS was 0.525 (95% CI 0.525–0.659); In the diagnosis of low-risk nodules, the AUC of the UP-LR model was 0.454 (95% CI 0.272–0.637); the AUC of TI-RADS was 0.461 (95% CI 0.38–0.543);In the diagnosis of malignant nodules, the AUC of the UP-LR model was 0.8 (95% CI 0.623–0.977); the AUC of TI-RADS was 0.525 (95% CI 0.435–0.615).

Table 5

Area under the curve
GroupLabelAreaAsymptotic 95% confidence interval
Lower boundUpper bound
TI-RADSbegnin0.5920.5250.659
low-risk0.4610.380.543
malignent0.5250.4350.615
UP modelBenign0.7050.550.86
Low-risk0.4540.2720.637
Malignant0.80.6230.977

Comparison of ROC analysis between UP-LR model and TI-RADS.

a Under the nonparametric assumption.

b Null hypothesis: true area = 0.5.

3.4 SHapley Additive exPlanations of the best-performing model

As shown in Figure 7, for the three-category classification of thyroid nodules, the top-ranked features are almost exclusively peritumoral radiomics features, particularly wavelet-transformed first-order statistics (such as interquartile range and percentiles) and texture features (e.g., ngtdm_Strength). Traditional ultrasound features generally rank lower than the radiomics features. This highlights the importance and necessity of incorporating peritumoral regions into the analysis. From benign to low-risk to malignant nodules, as the invasiveness of the nodules increases, the values of these features tend to be higher and are more frequently associated with positive SHAP values.

Figure 7

4 Discussion

Preoperative diagnosis of thyroid follicular tumors is a research hotspot and always a challenge in clinical practice. However, as we understand, there are few articles focusing on the differentiation of tumors classified as FA, FT-UMP, and FTC. The challenge lies in determining the presence of invasive growth, particularly capsular invasion and vascular invasion (, ). Most radiomics studies focus on the features within the tumor and ignore the key biological features available in the surrounding region (, ).Therefore, based on ultrasound images, we have developed intratumoral and peritumoral radiomic models to explore better methods for distinguishing tumor classifications. Combining intratumoral and peritumoral radiomics, which captures the interplay between tumor biology and the microenvironment, may surpass methods involving only intratumoral features and could lead to novel applications in thyroid tumor radiomics (, )Three machine learning models, LR,SVM and RF, were used to establish U,UI,UP and UIP models respectively by using clinical and ultrasound features, intratumoral radiomics and peritumoral radiomics. From the results, it can be seen that model LR in model UIP has better stability and generalization ability.

While the overall accuracy is low, the UP-LR model shows promising discrimination among benign, low-risk, and malignant cases(all AUC >0.70),with the highest performance observed in the malignant nodules.

Compared with the commonly used TI-RADS model, the diagnostic efficiency of this model for benign and malignant nodules is significantly higher. In diagnosing benign nodules, the model achieves a sensitivity of 1.00, ensuring that all benign nodules were identified, but its low specificity led to the possibility of misdiagnosis. This means that if the model classifies a nodule as non-benign, the result is highly reliable (NPV = 1.0). For diagnosing malignant and low-risk nodules, the model exhibited extremely high specificity(both >0.95), especially for malignant nodules (PPV = 0.6, specificity = 0.957),which means if the model classifies a nodule as malignant or low-risk, the result is highly reliable and further diagnostic confirmation is recommended. Notably, the AUC was 0.8 for malignant nodules and only 0.454 for low-risk nodules. It shows that the model performed poorly in identifying low-risk nodules. This may be related to the insufficient number of low-risk samples in the dataset and the significant overlap of features between low-risk and benign and malignant nodules. Indeed, incorporating the more challenging FT-UMP categories into the three-classification problem may indeed lead to a decrease in the overall performance of the model (). FT-UMP may partially overlap with FTA and FTC in imaging, increasing the difficulty of classification. FT-UMP is diagnostically challenging, even for pathologists, which is also reflected in our models. Specifically, we suggest that this model can be used as a complementary tool after inconclusive FNA results (Bethesda III/IV) to help prioritize high-risk patients for surgery and reduce unnecessary interventions in low-risk cases.

The RF model demonstrated the best performance in the U model, with high accuracy and AUC. However, in the UI, UP, and UIP models, it exhibited significant overfitting. Although this model showed the highest diagnostic value for malignant nodules (AUC 0.833), this resulted in considerable instability when applied to unknown data. Therefore, it is not recommended as a primary screening tool but could be considered as a secondary confirmation tool for high-risk nodules. In contrast, LR and SVM models exhibited better stability, though SVM’s overall efficacy was lower than that of LR. Considering the performance of all models, the LR model achieved higher AUC and accuracy in the test cohort, particularly in the UP model, which outperformed the UIP model in accuracy, specificity, PPV, sensitivity, and F1 score. We observed that incorporating intratumoral radiomic features did not improve model performance; instead, it led to a decline in certain key metrics. This suggests that intratumoral radiomic features may be redundant with peritumoral or clinical and ultrasound features, indicating that peritumoral radiomic features provide more critical discriminative information, which is similar with some previous studies (, ).

As previously described, follicular tumors share a similar underlying structure. Histologically, they form follicular structures resembling the normal thyroid gland, albeit with varying degrees of differentiation and spatial arrangement. The key diagnostic criteria revolve around the presence of invasive growth—specifically, its type and extent. A diagnosis of FTC requires evidence of definite vascular invasion or extensive capsular invasion. In contrast, FT-UMP represents a borderline category where the pathologist observes suspicious or focal signs of invasion under the microscope, which are insufficient to meet the strict diagnostic thresholds for FTC (). This pathological continuum explains the considerable challenge in distinguishing these entities in our radiomics study, as their essential difference lies in microscopic invasive behavior rather than macroscopic imaging findings. We aimed to capture more discriminative information regarding this behavior through the selection of peritumoral radiomics features.

When capsular or vascular invasion occurs in a thyroid nodule, remodeling of the extracellular matrix, increased local echogenicity heterogeneity, and ill-defined margins may contribute to elevated tissue heterogeneity in the peritumoral region. These changes are reflected in radiomic features such as InterquartileRange and Variance. The formation of neovascularization and tumor thrombi may correlate with high-intensity image traits, including the 90th Percentile, Maximum, and various HighGrayLevelEmphasis features. Meanwhile, attributes such as Strength may also indicate underlying tissue destruction and remodeling processes (, ).

These observations align with the results of our SHAP analysis. In the three-class classification of thyroid nodules, all peritumoral radiomics features were ranked as top contributors, outperforming traditional ultrasound characteristics. This reinforces the proposition that peritumoral radiomics features serve as the most powerful indicators in our model for stratifying tumor risk, capturing information beyond conventional visual assessment criteria. It further validates the value and necessity of our study design incorporating the peritumoral region. With increasing nodule aggressiveness, the values of these imaging features tend to rise and more frequently assume positive values—a pattern fully consistent with clinical understanding and supportive of the rationality of the model’s decision logic. Thereby, the model’s applicability and reliability in clinical practice are considerably enhanced.

Our study also has certain limitations. First, we agree that the poorer performance of the low-risk category (FT-UMP) is a major limitation. We will recommend that future studies integrate molecular markers, such as mutational analysis, with radiomics to improve the classification of critical categories such as FT-UMP. Second, the sample size of our study was limited and all were from a single center, which showed overfitting results. Therefore, we need to expand the sample size or conduct multicenter studies for external validation to further refine our models. In this study, only gray-scale ultrasound images were used to extract radiomics features. We found that the existing multimodal ultrasound images (contrast-enhanced ultrasound, elastic ultrasound) were also used to obtain radiomics features to understand the lesions more comprehensively (–). To address this issue, further prospective studies could be conducted to improve the prediction accuracy in our future work.

5 Conclusion

The logistic regression model incorporating clinical-ultrasound features along with peritumoral radiomic features achieved stable performance in distinguishing between benign and malignant follicular thyroid tumors. It offers a valuable reference for clinical decision-making in the management of follicular thyroid tumors.

Statements

Data availability statement

The original contributions presented in the study are included in the article/supplementary material. Further inquiries can be directed to the corresponding author.

Ethics statement

The studies involving humans were approved by the Ethics Committee of the First Affiliated Hospital of Nanjing Medical University (Jiangsu Provincial People’s Hospital)on October 25.2023 with the approval no.2023-SR-691. This study complied with the Declaration of Helsinki. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author contributions

YY: Conceptualization, Writing – original draft. XW: Data curation, Writing – original draft. HD: Formal Analysis, Writing – original draft. KC: Investigation, Resources, Writing – review & editing. FY: Methodology, Writing – review & editing.

Funding

The author(s) declared that financial support was not received for this work and/or its publication.

Conflict of interest

The authors declared that this work was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declared that generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Summary

Keywords

follicular thyroid tumors, machine learning, peritumoral radiomics, risk stratification, ultrasound

Citation

Yuan Y, Wang X, Deng H, Cao K and Yu F (2026) Ultrasound radiomics-based machine learning models for risk stratification of follicular thyroid tumors. Front. Oncol. 15:1707586. doi: 10.3389/fonc.2025.1707586

Received

17 September 2025

Revised

22 November 2025

Accepted

26 November 2025

Published

05 January 2026

Volume

15 - 2025

Edited by

Hsiang-Chen Wang, National Chung Cheng University, Taiwan

Reviewed by

Ziye Yan, Zhejiang Normal University, China

Jiahui Chen, Thyroid Head and Neck Cancer Foundation, United States

Andrew M. Thomas, The University of Iowa, United States

Updates

Copyright

*Correspondence: Fei Yu,

Disclaimer

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Outline

Figures

Cite article

Copy to clipboard


Export citation file


Share article

Article metrics