Michele Avanzo1*
Paolo Soda2
Marco Bertolini3
Andrea Bettinelli4
Tiziana Rancati5
Joseph Stancanello6
Osvaldo Rampado7
Giovanni Pirrone1Annalisa Drigo1
- 1Medical Physics Department, Centro di Riferimento Oncologico di Aviano (CRO) IRCCS, Aviano, Italy
- 2Department of Engineering, University Campus Biomedico of Rome, Rome, Italy
- 3Medical Physics Department, Azienda Unità Sanitaria Locale - IRCCS di Reggio Emilia: Reggio Emilia, Emilia-Romagna, Italy
- 4Medical Physics Department, Veneto Institute of Oncology IOV – IRCCS, Padua, Italy
- 5Data Science Unit, Department of Epidemiology and Data Science, Fondazione IRCCS Istituto Nazionale dei Tumori, Milano, Italy
- 6Elekta SA, Boulogne-Billancourt, France
- 7Medical Physics Unit, A.O.U. Città Della Salute e Della Scienza di Torino, Torino, Italy
Introduction: Radiomics aims to develop image-based biomarkers by combining quantitative analysis of medical images with artificial intelligence (AI) through a robust, reproducible pipeline. Scientific societies, task groups, and consortia have published several guidelines to help researchers design robust radiomics studies. This review summarizes existing guidelines, recommendations, and regulations for designing radiomics studies that can lead to clinically adoptable biomarkers.
Methods: Relevant articles were identified through a PubMed systematic review using “radiomics” and “guideline” as keywords. Of 314 retrieved papers, after screening 99 articles were deemed relevant for extracting recommendations on developing image-based biomarkers. Additional guidelines were searched by the authors.
Results: We can synthesize the systematic review in the following high consensus recommendations divided into five major areas: a) Study Design: Carefully define the study rationale, objectives, and outcomes, ensuring the dataset is of adequate size and quality; b) Data Workflow: Use standardized protocols for image acquisition, reconstruction, preprocessing, and feature extraction—following IBSI guidelines where applicable; c) Model Development and Validation: Follow best practices for model development, including prevention of data leakage, dimensionality reduction, strategies to enhance model interpretability, and establish biological plausibility; d) Transparency and Reproducibility: Publish results with sufficient methodological details to ensure rigor and generalizability and promote open science by sharing codes and data; e) Quality and compliance: Evaluate study compliance with relevant guidelines and regulations using appropriate quality metrics.
Conclusion: Radiomics promises to offer clinically useful imaging biomarkers and can represent a significant step in personalized medicine. In the present systematic review we identified five key guidelines and regulations developed in recent years, specifically for radiomics or AI, that can guide the research community in designing and conducting radiomic studies that result in an imaging biomarker suitable for clinical practice.
Keypoints
• Study design: Carefully define the study rationale, objectives, and outcomes, ensuring the dataset is of adequate size and quality.
• Data Workflow: Use standardized protocols for image acquisition, reconstruction, preprocessing, and feature extraction—following IBSI guidelines where applicable.
• Model development and validation: Follow best practices for model development, including prevention of data leakage, dimensionality reduction, strategies to enhance model interpretability, and establish biological plausibility.
• Transparency and Reproducibility: Publish results with sufficient methodological details to ensure rigor and generalizability and promote open science by sharing codes and data.
• Quality and Compliance: Evaluate study compliance with relevant guidelines and regulations using appropriate quality metrics.
1 Introduction
Radiomics applies quantitative analysis to medical images using high throughput calculations to extract mathematical descriptors called radiomic features (1–3), assuming that images encode biological processes which can be captured through voxel relationships, intensity distributions, and texture (1, 2). Thus, images are converted into data that can be mined using artificial intelligence (AI) techniques to study correlations with diagnostic or clinical endpoints. The aim is to identify features, or a radiomic signature (a combination of features), that could be used to diagnose or predict patient outcomes (4) or to tailor precise medical applications to individual patients (5), thereby helping shift from population-based approaches to truly individualized medicine.
Driven by the potential to enhance diagnostic accuracy and support clinical decision-making, interest in radiomics and in AI applied to medical imaging—fields that are closely interconnected—has been steadily increasing (1–3, 6, 7). However, radiomics has faced challenges in building clinically meaningful imaging biomarkers during its progress, stemming from the time-consuming data collection which involves manual segmentation (8), the multiple sources of variability impacting generalizability and reproducibility, and the large number of feature extracted, often redundant or irrelevant for the task at hand, bringing problems of overfitting. As a consequence of these challenges, no published study has yet prospectively implemented radiomic models as a routine clinical decision-support tool (9).
To address the need for more generalizable results, it is necessary to standardize the multistep process of radiomic analysis, encompassing image acquisition, feature calculation, and machine learning (ML) (1, 10). Equally crucial is defining the safety and efficacy boundaries of any proposed radiomic approach (11). As a result, major efforts have focused on creating recommendations for robust analysis, validation, and data sharing. The purpose of this review was to summarize the existing indications, guidelines, and regulations that radiomics researchers—particularly within the medical physics community—must follow to develop clinically usable imaging biomarkers.
2 Systematic review
A systematic review was performed searching Pubmed for studies providing radiomics guidelines using the keywords “radiomics” and “guidelines”. The results were screened using the Rayyan web-based tool (12), yielding 312 papers. After removing duplicates and studies that provided no recommendations, 99 papers were deemed relevant (Figure 1). The recommendations extracted are summarized in Table 1, where they are grouped in five major workflow areas-from defining the research question to evaluating the clinical utility of the radiomic tool.
Figure 1. Flow diagram visually summarizing the screening process according to PRISMA statement (13).
Table 1. Guidelines on radiomics summarized from the systematic review.
2.1 Designing an image biomarker study
Seminal radiomics papers hypothesized that distinctive image-derived features could help predict prognosis and therapeutic response across cancer types (125, 126), thereby identifying imaging biomarkers. According to the FDA and NIH Biomarker Working Group (29), a biomarker is “an indicator of normal biological processes, pathogenic processes, or responses to an exposure or intervention”. This definition includes radiographic characteristics, thus explicitly including features that can be extracted from medical images (127). These can be semantic or agnostic. Semantic features are qualitative descriptors assessed visually by radiologists, such as size, shape, location, vascularity, spiculation, or necrosis. Radiomic features are agnostic features, meaning that they are mathematically extracted imaging descriptors (128). Imaging biomarkers are non-invasive and allow comprehensive evaluation of the 3D tumor landscape through extraction of relevant imaging data. Examples of quantitative imaging biomarkers already used in the clinic include standardized uptake value (SUV) from PET (57) for tumor glycolytic activity, longest tumor diameter for assessing response in RECIST (119), and splenic volume from CT or ultrasound (29, 129–131).
Cancer Research UK (CRUK) and the European Organisation for Research and Treatment of Cancer (EORTC) produced 14 key recommendations for achieving the clinical translation of imaging biomarkers into clinical practice (29). These guidelines, along with those identified in our systematic review, begin with defining the research question and rationale for using radiomics (20).
The selection of the primary “endpoint” or “outcome measure” has considerable influence on the reliability and interpretability of radiomics studies. To maximize informativeness, primary endpoints should be well defined, reliable, measurable, and interpretable (14, 20, 21, 23–26, 28), and ideally based on gold-standard methods (27). Also, the possible role of the radiomics model in clinical decision-making should be defined at this stage (20, 23, 24).
When designing a radiomic study, the type of biomarker under investigation determines study design (6). A predictive biomarker stratifies patients more likely to respond to a therapeutic agent, e.g., achieving local control (103, 132). A prognostic biomarker is used to identify the likelihood of a clinical event, which could be recurrence, progression, survival or a side effect. Examples include the assessment of the probability of survival in patients with a specific cancer type (133). A diagnostic biomarker can detect a disease or a specific subtype e.g., the presence of a gene mutation (134). Delta-radiomics, where radiomic features monitor tumor changes during therapy (135), is an example of monitoring biomarker.
Biomarker assessment should be accurate, reproducible, and feasible over time (136). When biomarkers also reflect the effect of disease-specific treatments they are termed 'surrogate endpoints' (137), characterized by a biological link to the true endpoint, proven prognostic value, and evidence that treatment effects on the surrogate reflect clinical outcomes (138).
At this stage, inclusion and exclusion criteria, as well as the selection process (random or consecutive), should be defined (14, 18, 20, 22, 32), ensuring that study participants are representative of the intended patient population and that there is appropriate diversity and balance of characteristics (18–20, 30, 48).
A size-estimation analysis should also be carried out to determine the sample size needed to answer the research question (20). Radiomic studies are often monocentric and retrospective, partly due to challenges in collecting and securely sharing prospective patient data (139). In retrospective radiomic studies, imaging protocols (acquisition and reconstruction) are often uncontrolled or non-standardized (8). Such studies are also prone to bias, as they depend on existing medical records and offer limited control over participant selection (140).
2.2 Data collection
The radiomics workflow begins with image acquisition (1, 8, 128). Images should be acquired at the appropriate time relative to the endpoint and in accordance with standardized, well-documented protocols or consensus guidelines for acquisition and reconstruction (20, 21, 24, 26, 38), such as those from the Quantitative Imaging Biomarkers Alliance (QIBA) for MRI (56) or European Association of Nuclear Medicine (EANM) for nuclear medicine (57). Image acquisition guidelines help remove confounding factors (16, 21, 32, 40, 44, 45, 59–61) that may obscure correlations between tumor biology and imaging data (20, 53, 58). However, less rigid protocols may sometimes produce radiomic models that better reflect real-world scenarios (15, 48, 62). Imaging protocol details should be disclosed, including scanner vendor and acquisition and reconstruction parameters (32, 63), such as filters, field of view, acquisition matrix dimensions and resolution, CT x-ray energy and exposure, MRI sequence, PET acquisition time, and administered activity (15, 20, 30, 33, 34, 42, 43, 64–69).
Segmentation also critically affects the radiomics workflow. In fact, if performed manually, it affects radiomic features by inter-observer variability of contours (8, 141). Unfortunately, there is no consensus on how to delineate tumor regions specifically for radiomic analysis (123). In order to reduce the variability in radiomic feature values because of the Volume of Interest (VOI) definition, it is recommended to define a common rule among the patient dataset for segmenting lesions (142). Semi-automated or fully automated segmentation, also made possible by deep neural networks, can reduce inter-user variability of contours and improve the stability of radiomic features (8, 143, 144). Ground truth description and the definition of the annotation approach are of particular importance, specifying the number and expertise of experts involved, and the methodology adopted to get the final consensus. In the case of multiple experts involved, to reduce annotation biases, the intra- and inter-observer variability (145), using the intraclass correlation coefficient (146) or the Cohen's kappa (147), is encouraged.
Images should be exported as Digital Imaging and Communications in Medicine (DICOM) files in a lossless format (15), including regions of interest (ROIs) in DICOM RT structure format (72). The image de-identification/anonymization protocol should be disclosed (73). After collecting imaging and clinical data, these should be reviewed for errors and missing entries (48) and stored in a curated repository for analysis by a competent data curator (15, 48). Clinical data should be collected using a standardized methodology (e.g., cancer stage, treatment details, and adequate follow-up) (30, 70). The diagnostic protocol should be also fully described, including the imaging modality used, the timing of acquisitions if multiple time points are involved and the data de-identification/anonymization process (38, 47, 71).
2.3 Preprocessing and feature extraction
An international consortium, the Image Biomarker Standardization Initiative (IBSI) (148), standardized the overall workflow of radiomic feature extraction (depicted in Figure 2), providing guidelines for each of its steps (Table 2), standard nomenclature and mathematical formulations of features (149), and standardized protocols for filtering in radiomics (150).
Figure 2. Radiomic workflow as standardized by IBSI. *optional steps. MORPH: morphology, LI: local intensity, IS: intensity-based statistics, IH: intensity histogram, IVH: intensity-volume histogram, GLCM: grey level co-occurrence matrix, GLRLM: grey level run length matrix, GLSZM: grey level size zone matrix, GLDZM: grey level distance zone matrix, NGTDM: neighborhood grey tone difference matrix, NGLDM: neighboring grey level dependence matrix.
Table 2. IBSI recommendations regarding both pre-processing and feature calculation.
Preprocessing aims to reduce image noise, enhance image quality, and harmonize images before feature computation. It can mitigate protocol-related differences, for example by interpolating datasets to a common grid size (151), and includes converting gray levels to meaningful physical units such as SUV in PET or Hounsfield Units in CT (103). Filtering—typically via convolution kernels—can highlight image characteristics such as edges and textures; common filters include wavelets, Gaussian smoothing, and Laplacian edge-enhancing filters, often applied in combination (152, 153). Gray-level discretization into fixed numbers of bins or fixed bin widths is essential for texture analysis, as most matrices require integer values (154), while voxel resampling to a common isotropic size is necessary for certain feature calculations (155). Preprocessing steps should comply with IBSI guidelines (18, 78) and be fully documented, including choices such as intensity discretization strategies [e.g., fixed bin number or fixed bin width such as 25 HU/bin (33, 78)] and histogram equalization (33, 42, 43, 65, 80). When applicable, the rationale for the chosen discretization should be provided—for instance, using 64 bins within a standardized SUV range of 0–20 for PET imaging (24, 30, 85).
Feature extraction is the crucial step in which quantitative features are computed from the ROI, capturing its shape (e.g., sphericity, compactness), statistical (e.g., intensity range, contrast) and textural (e.g., heterogeneity, homogeneity) properties. In order to provide standardized results, a radiomics study should adhere to the IBSI methodology and nomenclature regarding the feature extraction pipeline (20, 24, 25, 30, 33, 42, 43, 49, 65, 69, 86), and the adopted workflow for feature extraction should be well described (43, 65).
Many radiomic software tools are available for commonly employed programming languages, including Python, R, and MATLAB, offering a range of functionalities, from image pre-processing to image segmentation and statistical analysis. Some commonly used and freely available packages include MIRP (156), S-IBEX (157), LIFEx (158), SERA (258), RaCaT (159), ROdiomiX (160), MITK Phenotyping (161), MODDICOM (162), PyRadiomics (163), RadiomiCRO (164).
Radiomic software should comply with the IBSI (18, 32, 68, 69, 85, 87–91). To comply with the IBSI, a software should provide the 169 IBSI-standardized features grouped in 11 feature families, two re-segmentation algorithms (range re-segmentation and outlier filtering), two discretization methods: fixed bin number (FBN) and fixed bin size (FBS) and nine different aggregation methods (e.g., 2D:avg, 2D:mrg, 3D:avg). The software type and version of code used for computation of features should be described (33, 43, 65, 80, 85).
Harmonization is the reduction of variability among radiomic features due to differences in imaging devices, acquisition parameters, reconstruction methods or other factors that may differ among centers (29). Mali et al. (165) identified two domains for harmonization, the image and feature domains. The image domain operates on differences generated by technical parameters used in image acquisition. Image harmonization can rely on digital phantoms (99, 154, 166–182). In feature domain harmonization, a feature normalization is performed resulting in a common range of features. Approaches to mitigate feature variance include residual harmonization (183, 184) and ComBat, originally developed in genomics (185) to suppress non-biologic sources of variance in multicenter studies, and later widely applied to radiomics (4, 184, 186–189), for its low computational burden and open-source availability (187). ComBat has certain limitations, primarily due to its assumption that data from each center follow a normal distribution—an assumption that may not always hold true. Moreover, it does not address collinearity, i.e., feature variations that are simultaneously influenced by both the investigated effect and technical parameters. The modified ComBat (M-ComBat) assumes one center as a reference for the others, allowing new centers to be added incrementally without repeated recalibration. Bootstrap ComBat (B-ComBat and BM-ComBat) employs Monte Carlo resampling to obtain multiple estimates, thereby improving robustness (190). Alternatively, Bertolini et al. proposed to use the grey level in an organ, e.g., healthy lung tissue in CT, as a reference level to compare and adjust values among centres (191).
2.4 Building a radiomic model
Like other “omics” fields, radiomics involves many variables and complex, non-linear relationships, making machine learning (ML) the predominant technique for analysis (3, 6, 192). ML requires a training phase, where it analyses a set of data whose endpoint is known, builds a model for classification or prediction and measures its performance (193). Pioneering works on radiomics focused mostly on assigning a categorical output variable, such as whether or not a disease has recurred, or whether the patient is alive beyond a certain time threshold (164). Some endpoints can also be modeled as regression tasks, such as predicting overall survival (194, 195) using survival times and relative risks (196, 197).
It is crucial to describe in detail the structure of the model, from preprocessing to the ML pipeline, including software programs and libraries used. It is also essential to specify the technical, clinical or ethical guidelines that were adopted. The most widely used ML algorithms include decision trees (198), ensembles of decision trees (199), support vector machines (200), and neural networks (201). Choosing a suitable ML model may involve comparing multiple approaches (18, 21, 67, 94, 104) or favoring more interpretable models such as decision trees or logistic regression, especially with small datasets (102). The final model should be selected using an established criterion, such as the Akaike Information Criterion (AIC) or similar methods (14). The type of model used, along with its architecture, software, and version, should be clearly specified (14, 19, 20, 22, 54).
Radiomic studies often use feature selection algorithms to remove non-predictive or redundant variables and identify the optimal subset from the large number of extracted features (202, 203). Broadly, feature selection methods fall into three categories: (i) Embedded methods, where the learning algorithm itself determines the optimal subset of features; (ii) Filter methods, which select or discard features prior to learning based on their relevance; and (iii) Wrapper methods, which assess model performance across different feature subsets to identify the most suitable set (14, 16, 20, 21, 54, 71, 90, 101). The final number of features can be set using rules of thumb, e.g., ≤10 features per instance (14), ≤50 weights in deep learning (102) or by selecting the number that yields optimal performance (103).
The tuning of the parameters in the ML pipeline should be rigorously carried out on the training set to avoid bias. The No Free Lunch Theorem states that no algorithm can be optimally tuned for all problems (204). Moreover, tuned parameters often perform no better than default values reported in the literature (205). The tuning strategy needs to be fully described, along with the number of iterations and range of hyperparameters (14, 30).
Choosing appropriate performance metrics is crucial for reliable model development and evaluation. For classification tasks, performance is commonly measured using metrics derived from the confusion matrix, such as accuracy, specificity, sensitivity, precision and recall. However, class imbalance—where one class has far fewer instances—is frequent in radiomics and can reduce performance on minority classes (206). Metrics such as the F1-score, balanced accuracy, and the geometric mean of accuracy account for imbalance (207). For models that output posterior probabilities, performance can be assessed with ROC curves and summarized by the area under the curve (AUROC). AUROC reflects the probability that a positive instance is ranked above a negative one, equivalent to the Wilcoxon rank-sum test (208). Despite the usefulness of the AUROC, ROC curves should still be reported, since models with the same AUC can perform differently in specific regions of the ROC space (208). The area under the Precision-Recall curve allows assessment of model performance on the minority class, independent of the majority class (209, 210).
The metrics for regression, e.g., predicting the time of the occurrence of metastasis, include the Mean Absolute Error (MAE), the Mean Squared Error (MSE), the Relative Absolute Error (RAE), the Root Mean Square Error (RMSE), and the Relative Squared Error (RSE): for all of them, the more their values are close to zero, the better the performance. R2 is used only for those methods that employ linear regression while adjusted R2 also accounts for the goodness of fit, sample size, and the number of predictors used (211). To overcome the issue of class imbalance, re-sampling can be applied, of which over-sampling has been proven more effective in ML (212, 213). Commonly used methods for oversampling minority class include the Synthetic Minority Over-sampling Technique (SMOTE) (14, 30, 39). Finally, robust statistical assessment of results is essential. There are many statistical tests to be considered and one of the first choices is whether to use pairwise or multiple tests (214, 215).
Because ML models often function as “black boxes,” interpreting their outputs is essential to avoid conflating correlation with causation, reinforcing bias, or emphasizing irrelevant data (30, 68). Beyond favoring inherently interpretable models (61), interpretability can be enhanced through class activation maps, feature importance analyses, e.g., by Shapley additive explanations (14, 22, 102, 120), and by examining misclassified cases (47).
2.5 Dissemination of results
When publishing a study on radiomics, the primary goal should be to enable other researchers to reproduce its results. The lack of reproducibility, mainly due to insufficient description of data source and methods in the manuscripts (11), is the Achilles' heel of radiomics (20). Koçak et al. identified the lack of consensus in many steps of the radiomic pipeline, stressing the importance of transparently reporting key information (216), and later produced an extensive list of 22 best practices for the steps of pre-modeling, modeling, and post-modeling (217). Several proposals have addressed the necessary quality checklists to ensure rigor, quality, and generalizability of radiomic studies and assess their quality, starting with the radiomic quality score (RQS) (21), the METRICS quality checklist (32) and the PET radiomics checklist (123). The Radiomic Analysis and Data Modeling (RANDAM) checklist (20) contains five main components to be used in a radiomic study, ensuring that a model is exactly reusable by other researchers. The Standards for Reporting Diagnostic Accuracy (STARD) initiative introduced a flow diagram describing key aspects of diagnostic studies, including patient recruitment, test execution order, and the number of patients undergoing the index test and/or reference standard (112). STARD was recently expanded with the STARD-AI statement (81), specifically tailored to AI-based studies. The CLAIM checklist (22), deriving from STARD, provides additional best-practice recommendations for researchers. Study data quality and diagnostic accuracy can be assessed—particularly regarding patient selection, applicability, and endpoint measurement—using tools such as QUADAS-2 (18, 35, 58, 61, 87, 98, 101, 109, 114, 118, 121) or QUIPS (25).
Authors of radiomic studies are strongly encouraged to deposit the data and/or software used for modeling and analysis in publicly accessible repositories and to provide direct links to these resources. Open data consist of datasets that can be freely used, modified, and shared for any purpose, including external validation. Several online repositories host imaging datasets, most notably The Cancer Imaging Archive (TCIA) maintained by the National Cancer Institute (218), giving the possibility of training or validating new models (219). Open-source refers to software whose source code is freely available to access, modify, and share, ensuring transparency in the methods used to obtain scientific results and enabling others to extend or adapt the code for related research. In addition to open-source tools for radiomic feature extraction, widely used free libraries such as scikit-learn (http://www.scikit-learn.org) can be run within Python (http://www.python.org), a high-level and easy-to-learn programming language. An example of radiomic feature extraction using open-source software on a patient from The Cancer Imaging Archive (TCIA) is shown in Figure 3.
Figure 3. (a–c) Example CT scan of a head and neck cancer patient from TCIA, shown with the contoured gross tumor volume (GTV). Side-by-side images display the first-order radiomic feature “entropy” calculated voxel-wise within the GTV using the open-source PyRadiomics package, in axial (a), coronal (b), and sagittal (c) views. Feature extraction was performed with PyRadiomics running in the free Google Colab environment (colab.research.google.com).
3 Validating a radiomics biomarker for clinical use
In this section we will summarize the steps required for the ultimate goal of radiomic biomarker, adoption in clinical routine. Technical validation of a biomarker includes measurements of reproducibility and robustness to ensure that the biomarker can be measured in any geographical location by any operator or equipment, producing comparable results. Reproducibility and robustness should not be used as the only figure of merit to evaluate features. They can still be compared against sensitivity, the ability of a feature to change when the grey-level distribution undergoes a change potentially occurring in a clinical context (146).
Clinical validation is the process of demonstrating the clinical utility of a feature or multiple features embedded in a statistical or ML model, e.g., by association with an endpoint or characteristics of a disease (97). Translation into clinical practice implies the commercialization of the output of the investigational study by a manufacturer, which has to take responsibility and liability for the safety and efficacy of the product. Regulators are tasked to guarantee the product's development and post-marketing of the product according to applicable laws, directives, and derived standards (220).
3.1 Repeatability and robustness
Repeatability refers to the precision of measurements performed multiple times in the same subject (in vitro or in vivo) under identical conditions using the same equipment, software, and operators over a short timeframe (221). Robustness is the ability of features to reflect the biological or clinical characteristics of the tumor or organ without being affected by technical factors, such as different scanners, imaging protocols, feature extraction methods, or operator-dependent procedures like manual segmentation (99, 145), ensuring that a robust feature can distinguish between patient groups even under varying conditions (137). The QIBA initiative suggested methods for measuring in-phantom robustness to change in imaging protocols (222).
Robustness to the image acquisition process has been extensively studied (155, 223–228), examining the effects of different scanning protocols or machines using phantoms specifically designed for CT (167, 171, 229), PET and MRI (8). These studies demonstrated radiomic features dependency on both the acquisition (99, 154, 179–182) and pre-processing parameters (166–178). For improving robustness, IBSI concluded that the same pre-processing parameters should be selected in a multicentric data collection (148). Images should be also free from artifacts in the region of interest, such as those caused by metal implants in CT (133) or motion (15).
Benchmarking evaluates the agreement among radiomics software tools. Compliance with the IBSI nomenclature and feature definitions is assessed by computing over 2,100 feature values using six specific extraction parameter configurations on two IBSI phantoms, and comparing the results with the reference values provided by the initiative. This process has led to the standardization of multiple open- and closed-source radiomic software and the publication of studies assessing feature robustness across different tools (91, 230–232). For example, Bettinelli et al. (91) evaluated seven software programs using the custom ImSURE digital phantoms (89), demonstrating that certain features are influenced by software-specific implementation choices in interpolation, discretization, and aggregation methods.
3.2 Clinical validation
The goal of any predictive model is to provide reliable outcome predictions for new patients. Since performance in prospective patients is unknown, it must be estimated from available data and cohorts. Validation measures how predictions perform on out-of-sample data. Radiomic models follow the same validation principles as other predictive models, which can generally be divided into three classes of procedures (115, 233).
Apparent validation assesses performance using the same dataset as model development. It generally results in optimistic biased estimates but may produce unsatisfactory results if the model is applied to new data. It should be limited to proof-of-principle studies on small data sets.
Internal validation, in which training and validation data are drawn from the same dataset through random or non-random splitting, estimates model accuracy for subjects similar to those in the development sample — that is, a dataset with the same feature distribution. It helps correct for overfitting and provides an estimate of optimism when the case mix is unchanged. Here, case mix refers to the distribution of features, both included and not included in the model (234). Internal validation approaches include split-sample (hold-out) methods (115, 233), where data are partitioned into two groups: one for model training (e.g., 50%) and the other for evaluation (e.g., 50%). Random splits can skew predictor and outcome distributions, especially with rare outcomes, and reduce training or validation data, leading to unstable models (115). Non-random splits use fixed criteria, such as enrollment date, center, clinician, or imaging device, which may produce differing case mixes between development and validation sets, providing an initial estimate of model generalizability.
K-fold cross-validation extends the random split-sample approach by dividing the dataset into k groups. The model is trained on k–1 groups and tested on the remaining group, repeating the process until every subject is used for both training and evaluation. Performance is the average across all k iterations (208).
Bootstrap validation draws random samples with replacement from the original dataset, so some subjects appear multiple times while others are excluded (235). Typically, a large number of bootstrap samples (Z ≥ 1,000) are generated, producing a distribution of performance estimates that allows reporting not only the mean but also measures such as standard error, standard deviation, and quartiles, capturing model uncertainty. Bootstrap validation is particularly effective in high-dimensional settings, where the number of predictors exceeds the sample size, as often occurs in radiomics (236).
External validation tests models on independent datasets from different institutions or populations to ensure generalizability (237) and is feasible only if the training and validation datasets are compatible. It typically involves: (i) validating predictors in the new dataset by comparing risk factor effects, (ii) assessing calibration, i.e., the agreement between observed and predicted outcomes (usually via a calibration plot), and (iii) evaluating discrimination using an appropriate metric. External validation may fail if the case mix differs from the development data, for example, if the predictive effects of features vary across cohorts. Overfitting in the development dataset, especially when many predictors are used with a small sample, can also reduce generalizability. In such cases, the model can be updated or adjusted using the validation dataset, which is encouraged when the validation set is large (234).
Clinical validation of a radiomic tool includes discussing its potential applications in clinical practice (17, 21, 22, 24, 33, 44, 111). Before implementation, the patient benefit must be demonstrated by comparing the tool against a defined reference or gold standard (14, 20–22, 30–33), with justification for the choice (14, 22) and discussion of its limitations (14). The added value of radiomics should be evaluated not only statistically—for example, the significance of an AUC increase (33, 43) —but also in terms of cost-effectiveness (17), such as quality-adjusted life years (QALY) (21, 35, 42, 46, 98). Finally, the tool's performance should be assessed within the context of its intended clinical use (24).
3.3 Biological validation
Biological validation refers to establishing a link between imaging biomarkers and tumor biology (137). Although not mandatory for clinical application (97), it is highly beneficial to show that an intervention's effect on a surrogate endpoint (e.g., reduced GLCM contrast after radiotherapy, reflecting decreased heterogeneity) reliably predicts its effect on the true clinical endpoint (e.g., tumor control after radiotherapy). This process involves correlating radiomic with non-radiomic features (21, 33, 54), confirming biological plausibility (20), and linking macroscopic tumor phenotypes captured by radiomics to microscopic tumor biology (46, 60, 93). Biological validation reduces the risk that radiomic features are selected by chance or reflect only the development dataset (97).
Evidence has been found linking radiomic information to tumor biology, such as its association with EGFR mutation status (134). Sun et al. (238) validated a radiomic signature from contrast-enhanced CT for tumor-infiltrating CD8 cells, related to the tumor-immune phenotype, using a gene expression signature. Preclinical studies may explore more closely the association of radiomic features with specific cell morphology and molecular pathways (97, 239) using diverse modalities such as Cone Beam CT (240), PET and MRI (241), Photon counting CT (242). Gene expression changes were linked to radiomic signatures by analyzing tumor models in wild-type vs. knockdown mice and correlating the results with patient data (243). Despite promising findings, linking biology and radiomics remains difficult (244) because of genetic complexity, indirect links to phenotype, and numerous correlated parameters requiring multiple comparisons (2, 97).
3.4 Regulatory framework
In the European Union (EU), The European Medical Devices Regulation (EU) 2017/745 (EU MDR) and in vitro Diagnostic Medical Devices Regulation (EU) 2017/746 (EU IVDR) in combination with the General Data Protection Regulation (EU) 2016/679 (GDPR) contain requirements for AI (245). The EU proposed a regulation called “The Artificial Intelligence Act (AI Act)” (246) aiming at introducing a common regulatory and legal framework for AI. Healthcare AI applications, as per the AI Act, are high-risk systems in that they may pose significant threats to health.
3.4.1 USA
The US Food and Drug Administration (FDA) 510(k) is a premarket submission to demonstrate safety of a device, where in-scope and out-scope applications of the product must be clearly highlighted as well as the possible behavior of the product in the so-called edge cases, i.e., cases lying on the limits of the applicability boundaries of that given product (123). The FDA together with Health Canada, and the United Kingdom's Medicines and Healthcare Products Regulatory Agency (MHRA) issued the Good Machine Learning Practice (GMLP) for medical device development: guiding principles” where they jointly identified 10 guiding principles (247).
3.4.2 Japan
The Japanese Pharmaceuticals and Medical Devices Agency 's Scientific Committee published “Issues and recommendations on AI-based medical diagnosis systems and medical devices” (248). Then the Japanese cabinet approved the Regulatory Reform Implementation Plan in June 2021, which summarized various policies to address most of the issues to be tackled in the future.
3.4.3 China
In July 2021 the National Medical Products Administration (NMPA, formerly the China Food and Drug Administration, or CFDA) issued the “Guideline on Artificial Intelligence Medical Devices” (249), a guideline paying attention to the AI medical device life-cycle process. Unapproved or low-maturity AI-based medical software is classified as class II or III, dipending on whether it involves assisted decision-making. High-maturity AI-based medical software devices must follow the Medical Device Classification Catalog.
4 Future directions
Several barriers slow the translation of radiomics into practice. These include the absence of standardized imaging acquisition and preprocessing protocols (244), the frequent use of small or single-center datasets that limit generalizability (250), and the broader need for large, diverse, and well-annotated cohorts to enable robust model development and validation. Additional challenges involve demonstrating clinical utility in real-world settings and integrating radiomics outputs seamlessly into existing workflows (244).
A key obstacle remains the limited availability of large annotated datasets—restricted by privacy concerns, proprietary limitations, and heterogeneous data formats. This highlights the importance of multi-institutional collaboration and the creation of centralized imaging repositories to ensure access to diverse, high-quality data (251).
Challenges arising during model development can also be mitigated through methodological best practices. Data imbalance may be addressed via oversampling algorithms, while dimensionality reduction and regularization can help prevent overfitting. It is essential to explicitly report measures taken to prevent information leakage. Data leakage is the situation in which information available during evaluation is unintentionally included in the training process, causing the training and evaluation data to no longer be truly independent (252). Because data leakage occurs during training, it can artificially boost the model's reported performance, making the evaluation metrics misleading compared to how the model will truly perform during inference Leakage controls include such as defining an independent test set before any normalization, feature selection, hyperparameter tuning, or model training (16, 18, 19, 43). The test set should be used only once for the final model evaluation to avoid biased or overly optimistic results (14).
Model interpretability, a major prerequisite for clinical adoption, can be strengthened through methods such as activation maps, feature-importance analyses (14, 22), Shapley additive explanations (102, 120), and systematic failure analysis of misclassified cases (47). Transparency and reproducibility can be further enhanced by sharing code, raw data, and complete methodological descriptions (251). As discussed in Sections 1.3 and 1.5, the availability of open-source libraries has already played an important role in facilitating radiomics research.
External testing by independent teams in radiomics helps verify that a model's performance is not limited to one dataset or experimental setup, demonstrating its potential for wider clinical use. However, such testing is still conducted by only a small number of research groups (253)
Another major challenge for clinical implementation is establishing biological plausibility. This involves demonstrating robust and reproducible relationships between radiomic phenotypes and the underlying microscopic biology (33, 42, 43, 46, 60, 93), While preclinical studies can investigate these links more directly, clinical studies may assess associations between radiomic features and histopathology, gene mutations or expression profiles, and metabolic characteristics (254).
Guidelines in radiomics offer structured pathways to overcome these barriers. However, their adoption remains limited. A meta-research analysis of 117 radiomics studies found that only 7 (6%) reported adherence to at least one quality or reporting framework (14). Similarly, another study reported that only 2 of 33 papers (6%) included a self-reported quality score (73). Comparable findings have been echoed by other authors (122, 255).
Based on the findings of our systematic review, we can synthesize the high consensus recommendations divided into 5 major areas: (a) Study Design: Carefully define the study rationale, objectives, and outcomes, ensuring the dataset is of adequate size and quality, (b) Data Workflow: Use standardized protocols for image acquisition, reconstruction, preprocessing, and feature extraction—following IBSI guidelines where applicable, (c) Model Development and Validation: Follow best practices for model development, including prevention of data leakage, dimensionality reduction, strategies to enhance model interpretability, and establish biological plausibility, (d) Transparency and Reproducibility: Publish results with sufficient methodological details to ensure rigor and generalizability and promote open science by sharing codes and data, (e) Quality and Compliance: Evaluate study compliance with relevant guidelines and regulations using appropriate quality metrics.
There remain several areas where further standardization through guidelines and recommendations is needed. First, this review focuses exclusively on handcrafted features, even though deep learning–based features are receiving increasing attention for their ability to capture more abstract and high-level image representations (256). Second, in studies involving rare diseases, datasets are often small and highly imbalanced. Existing guidelines should therefore be expanded to address strategies for managing class imbalance and to provide recommendations on appropriate algorithms, model architectures, and parameter settings for such scenarios. Furthermore, federated learning has recently emerged as a promising approach to reduce the reliance on large, centrally collected datasets (257). However, this technique also requires the development of dedicated standards and guidelines to ensure consistency, reproducibility, and data security across institutions.
5 Conclusions
Radiomics holds the promise of delivering clinically meaningful imaging biomarkers and represents an important step toward truly personalized medicine. To achieve this goal, researchers must first clearly define the study objective—particularly the specific type of imaging biomarker under investigation—and then follow a rigorously standardized workflow aligned with established guidelines. In this systematic review, we identified five major guidelines and regulatory frameworks developed in recent years for radiomics and AI, which collectively provide the research community with essential direction for designing, conducting, and validating radiomic studies capable of producing biomarkers suitable for clinical implementation.
Author contributions
MA: Data curation, Methodology, Writing – original draft, Writing – review & editing. PS: Supervision, Writing – original draft, Writing – review & editing. MB: Investigation, Methodology, Writing – original draft, Writing – review & editing. AB: Investigation, Methodology, Writing – original draft, Writing – review & editing. TR: Investigation, Methodology, Writing – original draft, Writing – review & editing. JS: Investigation, Methodology, Supervision, Writing – original draft, Writing – review & editing. OR: Investigation, Writing – original draft, Writing – review & editing. GP: Writing – original draft, Writing – review & editing. AD: Writing – original draft, Writing – review & editing.
Funding
The author(s) declared that financial support was received for this work and/or its publication. This work was supported by the Italian Ministry of Health (Ricerca Corrente).
Conflict of interest
The author(s) declared that this work was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The authors MA, PS, TR declared that they were an editorial board member of Frontiers at the time of submission. This had no impact on the peer review process and the final decision.
Generative AI statement
The author(s) declared that generative AI was not used in the creation of this manuscript.
Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1. Avanzo M, Stancanello J, El Naqa I. Beyond imaging: the promise of radiomics. Phys Med. (2017) 38:122–39. doi: 10.1016/j.ejmp.2017.05.071
2. Tomaszewski MR, Gillies RJ. The biological meaning of radiomic features. Radiology. (2021) 298(3):505–16. doi: 10.1148/radiol.2021202553
3. Avanzo M, Wei L, Stancanello J, Vallieres M, Rao A, Morin O, et al. Machine and deep learning methods for radiomics. Med Phys. (2020) 47(5):e185–202. doi: 10.1002/mp.13678
4. Orlhac F, Eertink JJ, Cottereau AS, Zijlstra JM, Thieblemont C, Meignan M, et al. A guide to ComBat harmonization of imaging biomarkers in multicenter studies. J Nucl Med. (2022) 63(2):172–9. doi: 10.2967/jnumed.121.262464
5. Vallieres M, Kay-Rivest E, Perrin L, Liem X, Furstoss C, Aerts HJWL, et al. Radiomics strategies for risk assessment of tumour failure in head-and-neck cancer. Sci Rep. (2017) 7(1):10117. doi: 10.1038/s41598-017-10371-5
6. Avanzo M, Porzio M, Lorenzon L, Milan L, Sghedoni R, Russo G, et al. Artificial intelligence applications in medical imaging: a review of the medical physics research in Italy. Phys Med. (2021) 83:221–41. doi: 10.1016/j.ejmp.2021.04.010
7. Maniaci A, Lavalle S, Gagliano C, Lentini M, Masiello E, Parisi F, et al. The integration of radiomics and artificial intelligence in modern medicine. Life. (2024) 14(10):1248. doi: 10.3390/life14101248
8. van Timmeren JE, Cester D, Tanadini-Lang S, Alkadhi H, Baessler B. Radiomics in medical imaging—“how-to” guide and critical reflection. Insights Imaging. (2020) 11(1):91. doi: 10.1186/s13244-020-00887-2
9. Horvat N, Papanikolaou N, Koh DM. Radiomics beyond the hype: a critical evaluation toward oncologic clinical use. Radiol Artif Intell. (2024) 6(4):e230437. doi: 10.1148/ryai.230437
10. Castiglioni I, Gallivanone F, Soda P, Avanzo M, Stancanello J, Aiello M, et al. AI-based applications in hybrid imaging: how to build smart and truly multi-parametric decision models for radiomics. Eur J Nucl Med Mol Imaging. (2019) 46(13):2673–99. doi: 10.1007/s00259-019-04414-4
11. Roberts M, Driggs D, Thorpe M, Gilbey J, Yeung M, Ursprung S, et al. Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans. Nat Mach Intell. (2021) 3(3):199–217. doi: 10.1038/s42256-021-00307-0
12. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan—a web and mobile app for systematic reviews. Syst Rev. (2016) 5(1):210. doi: 10.1186/s13643-016-0384-4
13. Page MJ, Moher D, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. PRISMA 2020 Explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. Br Med J. (2021) 372:n160. doi: 10.1136/bmj.n160
14. Kocak B, Baessler B, Bakas S, Cuocolo R, Fedorov A, Maier Hein L, et al. CheckList for EvaluAtion of Radiomics research (CLEAR): a step-by-step reporting guideline for authors and reviewers endorsed by ESR and EuSoMII. Insights Imaging. (2023) 14(1):75. doi: 10.1186/s13244-023-01415-8
15. Shur JD, Doran SJ, Kumar S, Ap Dafydd D, Downey K, O’Connor JPB, et al. Radiomics in oncology: a practical guide. Radiographics. (2021) 41(6):1717–32. doi: 10.1148/rg.2021210037
16. Papanikolaou N, Matos C, Koh DM. How to develop a meaningful radiomic signature for clinical use in oncologic patients. Cancer Imaging. (2020) 20(1):33. doi: 10.1186/s40644-020-00311-4
17. Park JE, Kim D, Kim HS, Park SY, Kim JY, Cho SJ, et al. Quality of science and reporting of radiomics in oncologic studies: room for improvement according to radiomics quality score and TRIPOD statement. Eur Radiol. (2020) 30(1):523–36. doi: 10.1007/s00330-019-06360-z.31350588
18. Hatt M, Krizsan AK, Rahmim A, Bradshaw TJ, Costa PF, Forgacs A, et al. Joint EANM/SNMMI guideline on radiomics in nuclear medicine: jointly supported by the EANM physics committee and the SNMMI physics, instrumentation and data sciences council. Eur J Nucl Med Mol Imaging. (2023) 50(2):352–75. doi: 10.1007/s00259-022-06001-6
19. Rogasch JMM, Shi K, Kersting D, Seifert R. Methodological evaluation of original articles on radiomics and machine learning for outcome prediction based on positron emission tomography (PET). Nuklearmedizin. (2023) 62(6):361–9. doi: 10.1055/a-2198-0545
20. Wang X, Pennello G, de Souza NM, Huang EP, Buckler AJ, Barnhart HX, et al. Multiparametric data-driven imaging markers: guidelines for development, application and reporting of model outputs in radiomics. Acad Radiol. (2023) 30(2):215–29. doi: 10.1016/j.acra.2022.10.001
21. Lambin P, Leijenaar RTH, Deist TM, Peerlings J, de Jong EEC, van Timmeren J, et al. Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol. (2017) 14(12):749–62. doi: 10.1038/nrclinonc.2017.141
22. Mongan J, Moy L, Kahn CE. Checklist for artificial intelligence in medical imaging (CLAIM): a guide for authors and reviewers. Radiol Artif Intell. (2020) 2(2):e200029. doi: 10.1148/ryai.2020200029
23. Davey MS, Davey MG, Ryan ÉJ, Hogan AM, Kerin MJ, Joyce M. The use of radiomic analysis of magnetic resonance imaging in predicting distant metastases of rectal carcinoma following surgical resection: a systematic review and meta-analysis. Colorectal Dis. (2021) 23(12):3065–72. doi: 10.1111/codi.15919
24. Huang EP, O’Connor JPB, McShane LM, Giger ML, Lambin P, Kinahan PE, et al. Criteria for the translation of radiomics into clinically useful tests. Nat Rev Clin Oncol. (2023) 20(2):69–82. doi: 10.1038/s41571-022-00707-0
25. Chiu HY, Wang TW, Hsu MS, Chao HS, Liao CY, Lu CF, et al. Progress in serial imaging for prognostic stratification of lung cancer patients receiving immunotherapy: a systematic review and meta-analysis. Cancers (Basel). (2024) 16(3):615. doi: 10.3390/cancers16030615
26. Won SY, Park YW, Ahn SS, Moon JH, Kim EH, Kang SG, et al. Quality assessment of meningioma radiomics studies: bridging the gap between exploratory research and clinical applications. Eur J Radiol. (2021) 138:109673. doi: 10.1016/j.ejrad.2021.109673
27. Oh KE, Vasandani N, Anwar A. Radiomics to differentiate malignant and benign breast lesions: a systematic review and diagnostic test accuracy meta-analysis. Cureus. (2023) 15(11):e49015. doi: 10.7759/cureus.49015
28. Fleming TR, Powers JH. Biomarkers and surrogate endpoints in clinical trials. Stat Med. (2012) 31(25):2973–84. doi: 10.1002/sim.5403
29. O’Connor JPB, Aboagye EO, Adams JE, Aerts HJWL, Barrington SF, Beer AJ, et al. Imaging biomarker roadmap for cancer studies. Nat Rev Clin Oncol. (2017) 14(3):169–86. doi: 10.1038/nrclinonc.2016.162
30. Sollini M, Cozzi L, Ninatti G, Antunovic L, Cavinato L, Chiti A, et al. PET/CT radiomics in breast cancer: mind the step. Methods. (2021) 188:122–32. doi: 10.1016/j.ymeth.2020.01.007
31. Zheng X, He B, Hu Y, Ren M, Chen Z, Zhang Z, et al. Diagnostic accuracy of deep learning and radiomics in lung cancer staging: a systematic review and meta-analysis. Front Public Health. (2022) 10:938113. doi: 10.3389/fpubh.2022.938113
32. Kocak B, Akinci D’Antonoli T, Mercaldo N, Alberich-Bayarri A, Baessler B, Ambrosini I, et al. METhodological RadiomICs score (METRICS): a quality scoring tool for radiomics research endorsed by EuSoMII. Insights Imaging. 2024;15(1):8. doi: 10.1186/s13244-023-01572-w
33. Ibrahim A, Vallières M, Woodruff H, Primakov S, Beheshti M, Keek S, et al. Radiomics analysis for clinical decision support in nuclear medicine. Semin Nucl Med. (2019) 49(5):438–49. doi: 10.1053/j.semnuclmed.2019.06.005
34. Morland D, Triumbari EKA, Boldrini L, Gatta R, Pizzuto D, Annunziata S. Radiomics in oncological PET imaging: a systematic review—part 2, infradiaphragmatic cancers, blood malignancies, melanoma and musculoskeletal cancers. Diagnostics. (2022) 12(6):1330. doi: 10.3390/diagnostics12061330
35. Yang C, Jiang Z, Cheng T, Zhou R, Wang G, Jing D, et al. Radiomics for predicting response of neoadjuvant chemotherapy in nasopharyngeal carcinoma: a systematic review and meta-analysis. Front Oncol. (2022) 4:12:893103. doi: 10.3389/fonc.2022.89310
36. Zhang S, Liu R, Wang Y, Zhang Y, Li M, Wang Y, et al. Ultrasound-Base radiomics for discerning lymph node metastasis in thyroid cancer: a systematic review and meta-analysis. Acad Radiol. (2024) 31(8):3118–30. doi: 10.1016/j.acra.2024.03.012
37. Koechli C, Zwahlen DR, Schucht P, Windisch P. Radiomics and machine learning for predicting the consistency of benign tumors of the central nervous system: a systematic review. Eur J Radiol. (2023) 164:110866. doi: 10.1016/j.ejrad.2023.110866
38. Zhang F, Wu G, Chen N, Li R. The predictive value of radiomics-based machine learning for peritoneal metastasis in gastric cancer patients: a systematic review and meta-analysis. Front Oncol. (2023) 13:1196053. doi: 10.3389/fonc.2023.1196053
39. Jha AK, Mithun S, Sherkhane UB, Jaiswar V, Osong B, Purandare N, et al. Systematic review and meta-analysis of prediction models used in cervical cancer. Artif Intell Med. (2023) 139:102549. doi: 10.1016/j.artmed.2023.102549
40. Shankar LK, Schöder H, Sharon E, Wolchok J, Knopp MV, Wahl RL, et al. Harnessing imaging tools to guide immunotherapy trials: summary from the national cancer institute cancer imaging steering committee workshop. Lancet Oncol. (2023) 24(3):e133–43. doi: 10.1016/S1470-2045(22)00742-2
41. Zhou L, Chen Y, Li Y, Wu C, Xue C, Wang X. Diagnostic value of radiomics in predicting ki-67 and cytokeratin 19 expression in hepatocellular carcinoma: a systematic review and meta-analysis. Front Oncol. (2024) 13:1323534. doi: 10.3389/fonc.2023.1323534
42. Park CJ, Park YW, Ahn SS, Kim D, Kim EH, Kang SG, et al. Quality of radiomics research on brain metastasis: a roadmap to promote clinical translation. Korean J Radiol. (2022) 23(1):77–88. doi: 10.3348/kjr.2021.0421
43. Vallieres M, Zwanenburg A, Badic B, Cheze Le Rest C, Visvikis D, Hatt M. Responsible radiomics research for faster clinical translation. J Nucl Med. (2018) 59(2):189–93. doi: 10.2967/jnumed.117.200501
44. Shi Z, Zhang Z, Liu Z, Zhao L, Ye Z, Dekker A, et al. Methodological quality of machine learning-based quantitative imaging analysis studies in esophageal cancer: a systematic review of clinical outcome prediction after concurrent chemoradiotherapy. Eur J Nucl Med Mol Imaging. (2022) 49(8):2462–81. doi: 10.1007/s00259-021-05658-9
45. Aldawsari AM, Al-Qaisieh B, Broadbent DA, Bird D, Murray L, Speight R. The role and potential of using quantitative MRI biomarkers for imaging guidance in brain cancer radiotherapy treatment planning: a systematic review. Phys Imaging Radiat Oncol. (2023) 27:100476. doi: 10.1016/j.phro.2023.100476
46. Wesdorp NJ, Hellingman T, Jansma EP, van Waesberghe JHTM, Boellaard R, Punt CJA, et al. Advanced analytics and artificial intelligence in gastrointestinal cancer: a systematic review of radiomics predicting response to treatment. Eur J Nucl Med Mol Imaging. (2021) 48(6):1785–94. doi: 10.1007/s00259-020-05142-w
47. Alipour E, Pooyan A, Shomal Zadeh F, Darbandi AD, Bonaffini PA, Chalian M. Current Status and future of artificial intelligence in MM imaging: a systematic review. Diagnostics (Basel). (2023) 13(21):3372. doi: 10.3390/diagnostics13213372
48. Maragna R, Giacari CM, Guglielmo M, Baggiano A, Fusini L, Guaricci AI, et al. Artificial intelligence based multimodality imaging: a new frontier in coronary artery disease management. Front Cardiovasc Med. (2021) 8:736223. doi: 10.3389/fcvm.2021.736223
49. Chang S, Han K, Suh YJ, Choi BW. Quality of science and reporting for radiomics in cardiac magnetic resonance imaging studies: a systematic review. Eur Radiol. (2022) 32(7):4361–73. doi: 10.1007/s00330-022-08587-9
50. Petersen GIC, Shatalov J, Verma T, Brim WR, Subramanian H, Brackett A, et al. Machine learning in differentiating gliomas from primary CNS lymphomas: a systematic review, reporting quality, and risk of bias assessment. Am J Neuroradiol. (2022) 43(4):526–33. doi: 10.3174/ajnr.A7473
51. Araújo ALD, Moraes MC, Pérez-de-Oliveira ME, da Silva VM, Saldivia-Siracusa C, Pedroso CM, et al. Machine learning for the prediction of toxicities from head and neck cancer treatment: a systematic review with meta-analysis. Oral Oncol. (2023) 140:106386. doi: 10.1016/j.oraloncology.2023.106386
52. Wolff RF, Moons KGM, Riley RD, Whiting PF, Westwood M, Collins GS, et al. PROBAST: a tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med. (2019) 170(1):51–8. doi: 10.7326/M18-1376
53. Avery EW, Behland J, Mak A, Haider SP, Zeevi T, Sanelli PC, et al. Dataset on acute stroke risk stratification from CT angiographic radiomics. Data Brief. (2022) 44:108542. doi: 10.1016/j.dib.2022.108542
54. Wang W, Peng Y, Feng X, Zhao Y, Seeruttun SR, Zhang J, et al. Development and validation of a computed tomography–based radiomics signature to predict response to neoadjuvant chemotherapy for locally advanced gastric cancer. JAMA Netw Open. (2021) 4(8):e2121143. doi: 10.1001/jamanetworkopen.2021.21143
55. Luo X, Zheng R, Zhang J, He J, Luo W, Jiang Z, Li Q. CT-based radiomics for predicting Ki-67 expression in lung cancer: a systematic review and meta-analysis. Front Oncol. (2024) 14:1329801. doi: 10.3389/fonc.2024.1329801
56. O’Sullivan S, McDermott R, Keys M, O’Sullivan M, Armstrong J, Faul C. Imaging response assessment following stereotactic body radiotherapy for solid tumour metastases of the spine: current challenges and future directions. J Med Imaging Radiat Oncol. (2020) 64(3):385–97. doi: 10.1111/1754-9485.13032
57. Boellaard R. Standards for PET image acquisition and quantitative data analysis. J Nucl Med. (2009) 50(Suppl 1):11S–20. doi: 10.2967/jnumed.108.057182
58. Guha A, Goda JS, Dasgupta A, Mahajan A, Halder S, Gawde J, et al. Classifying primary central nervous system lymphoma from glioblastoma using deep learning and radiomics based machine learning approach - a systematic review and meta-analysis. Front Oncol. (2022) 12:884173. doi: 10.3389/fonc.2022.884173
59. Ma JW, Li M. Molecular typing of lung adenocarcinoma with computed tomography and CT image-based radiomics: a narrative review of research progress and prospects. Transl Cancer Res. (2021) 10(9):4217–31. doi: 10.21037/tcr-21-1037
60. Rangwani S, Ardeshna DR, Rodgers B, Melnychuk J, Turner R, Culp S, et al. Application of artificial intelligence in the management of pancreatic cystic lesions. Biomimetics (Basel). (2022) 7(2):79. doi: 10.3390/biomimetics7020079
61. Nguyen HS, Ho DKN, Nguyen NN, Tran HM, Tam KW, Le NQK. Predicting EGFR mutation Status in non–small cell lung cancer using artificial intelligence: a systematic review and meta-analysis. Acad Radiol. (2024) 31(2):660–83. doi: 10.1016/j.acra.2023.03.040
62. Balana C, Castañer S, Carrato C, Moran T, Lopez-Paradís A, Domenech M, et al. Preoperative diagnosis and molecular characterization of gliomas with liquid biopsy and radiogenomics. Front Neurol. (2022) 13:865171. doi: 10.3389/fneur.2022.865171
63. van Velden FH, Kramer GM, Frings V, Nissen IA, Mulder ER, de Langen AJ, et al. Repeatability of radiomic features in non-small-cell lung cancer [(18)F]FDG-PET/CT studies: impact of reconstruction and delineation. Mol Imaging Biol. (2016) 18(5):788–95. doi: 10.1007/s11307-016-0940-2
64. Parker W, Jaremko JL, Cicero M, Azar M, El-Emam K, Gray BG, et al. Canadian association of radiologists white paper on de-identification of medical imaging: part 1, general principles. Can Assoc Radiol J. (2021) 72(1):13–24. doi: 10.1177/0846537120967349
65. Zwanenburg A, Leger S, Vallieres M, Lock S. Image biomarker standardisation initiative. arXiv e-prints. arXiv [Preprint]. (2016). ar:1612.07003. doi: 10.48550/arXiv.1612.07003
66. Dalal V, Carmicheal J, Dhaliwal A, Jain M, Kaur S, Batra SK. Radiomics in stratification of pancreatic cystic lesions: machine learning in action. Cancer Lett. (2020) 469:228–37. doi: 10.1016/j.canlet.2019.10.023
67. Acharya UR, Hagiwara Y, Sudarshan VK, Chan WY, Ng KH. Towards precision medicine: from quantitative imaging to radiomics. J Zhejiang Univ Sci B. (2018) 19(1):6–24. doi: 10.1631/jzus.B1700260
68. Brunasso L, Ferini G, Bonosi L, Costanzo R, Musso S, Benigno UE, et al. A spotlight on the role of radiomics and machine-learning applications in the management of intracranial meningiomas: a new perspective in neuro-oncology: a review. Life. (2022) 12(4):586. doi: 10.3390/life12040586
69. Beaudoin AM, Ho JK, Lam A, Thijs V. Radiomics studies on ischemic stroke and carotid atherosclerotic disease: a reporting quality assessment. Can Assoc Radiol J. (2024) 75(3):549–57. doi: 10.1177/08465371241234545
70. Chetan MR, Gleeson FV. Radiomics in predicting treatment response in non-small-cell lung cancer: current status, challenges and future perspectives. Eur Radiol. (2021) 31(2):1049–58. doi: 10.1007/s00330-020-07141-9
71. Tan D, Mohamad Salleh SA, Manan HA, Yahya N. Delta-radiomics-based models for toxicity prediction in radiotherapy: a systematic review and meta-analysis. J Med Imaging Radiat Oncol. (2023) 67(5):564–79. doi: 10.1111/1754-9485.13546
72. Ger RB, Cardenas CE, Anderson BM, Yang J, Mackin DS, Zhang L, et al. Guidelines and experience using Imaging Biomarker Explorer (IBEX) for radiomics. J Vis Exp. (2018) 131:57132. doi: 10.3791/57132
73. Parker W, Jaremko JL, Cicero M, Azar M, El-Emam K, Gray BG, et al. Canadian association of radiologists white paper on de-identification of medical imaging: part 2, practical considerations. Can Assoc Radiol J. (2021) 72(1):25–34. doi: 10.1177/0846537120967345
74. Guglielmo P, Marturano F, Bettinelli A, Sepulcri M, Pasello G, Gregianin M, et al. Additional value of PET and CT image-based features in the detection of occult lymph node metastases in lung cancer: a systematic review of the literature. Diagnostics (Basel). (2023) 13(13):2153. doi: 10.3390/diagnostics13132153
75. Won SY, Lee N, Park YW, Ahn SS, Ku CR, Kim EH, et al. Quality reporting of radiomics analysis in pituitary adenomas: promoting clinical translation. Br J Radiol. (2022) 95(1139):20220401. doi: 10.1259/bjr.20220401
76. Ger RB, Craft DF, Mackin DS, Zhou S, Layman RR, Jones AK, et al. Practical guidelines for handling head and neck computed tomography artifacts for quantitative image analysis. Comput Med Imaging Graph. (2018) 69:134–9. doi: 10.1016/j.compmedimag.2018.09.002
77. Foltyn-Dumitru M, Schell M, Rastogi A, Sahm F, Kessler T, Wick W, et al. Impact of signal intensity normalization of MRI on the generalizability of radiomic-based prediction of molecular glioma subtypes. Eur Radiol. (2024) 34(4):2782–90. doi: 10.1007/s00330-023-10034-2
78. Carré A, Klausner G, Edjlali M, Lerousseau M, Briend-Diop J, Sun R, et al. Standardization of brain MR images across machines and protocols: bridging the gap for MRI-based radiomics. Sci Rep. (2020) 10(1):12340. doi: 10.1038/s41598-020-69298-z
79. Mu W, Jiang L, Shi Y, Tunali I, Gray JE, Katsoulakis E, et al. Non-invasive measurement of PD-L1 status and prediction of immunotherapy response using deep learning of PET/CT images. J Immunother Cancer. (2021) 9(6):e002118. doi: 10.1136/jitc-2020-002118
80. Kocak B, Yuzkan S, Mutlu S, Bulut E, Kavukoglu I. Publications poorly report the essential RadiOmics ParametERs (PROPER): a meta-research on quality of reporting. Eur J Radiol. (2023) 167:111088. doi: 10.1016/j.ejrad.2023.111088.37713968
81. Sounderajah V, Guni A, Liu X, Collins GS, Karthikesalingam A, Markar SR, et al. The STARD-AI reporting guideline for diagnostic accuracy studies using artificial intelligence. Nat Med. (2025) 31(10):3283–9. doi: 10.1038/s41591-025-03953-8
82. Jethanandani A, Lin TA, Volpe S, Elhalawani H, Mohamed ASR, Yang P, et al. Exploring applications of radiomics in magnetic resonance imaging of head and neck cancer: a systematic review. Front Oncol. (2018) 8:131. doi: 10.3389/fonc.2018.00131
83. Xue C, Yuan J, Lo GG, Chang ATY, Poon DMC, Wong OL, et al. Radiomics feature reliability assessed by intraclass correlation coefficient: a systematic review. Quant Imaging Med Surg. (2021) 11(10):4431–60. doi: 10.21037/qims-21-86
84. Al-Ghoula F, Patel K, Falde S, Rajagopalan S, Bartholmai B, Maldonado F, et al. Assessment of interobserver concordance in radiomic tools for lung nodule classification, with a focus on BRODERS and SILA. Sci Rep. (2023) 13(1):21725. doi: 10.1038/s41598-023-48567-7
85. Traverso A, Wee L, Dekker A, Gillies R. Repeatability and reproducibility of radiomic features: a systematic review. Int J Radiat Oncol Biol Phys. (2018) 102(4):1143–58. doi: 10.1016/j.ijrobp.2018.05.053
86. Wesdorp NJ, van Goor VJ, Kemna R, Jansma EP, van Waesberghe JHTM, Swijnenburg RJ, et al. Advanced image analytics predicting clinical outcomes in patients with colorectal liver metastases: a systematic review of the literature. Surg Oncol. (2021) 38:101578. doi: 10.1016/j.suronc.2021.101578
87. Yang Z, Gong J, Li J, Sun H, Pan Y, Zhao L. The gap before real clinical application of imaging-based machine-learning and radiomic models for chemoradiation outcome prediction in esophageal cancer: a systematic review and meta-analysis. Int J Surg. (2023) 109(8):2451–66. doi: 10.1097/JS9.0000000000000441
88. Lei M, Varghese B, Hwang D, Cen S, Lei X, Desai B, et al. Benchmarking various radiomic toolkit features while applying the image biomarker standardization initiative toward clinical translation of radiomic analysis. J Digit Imaging. (2021) 34(5):1156–70. doi: 10.1007/s10278-021-00506-6
89. Bettinelli A, Marturano F, Sarnelli A, Bertoldo A, Paiusco M. The ImSURE phantoms: a digital dataset for radiomic software benchmarking and investigation. Sci Data. (2022) 9:695. doi: 10.1038/s41597-022-01715-6
90. Bagher-Ebadian H, Siddiqui F, Ghanem AI, Zhu S, Lu M, Movsas B, et al. Radiomics outperforms clinical factors in characterizing human papilloma virus (HPV) for patients with oropharyngeal squamous cell carcinomas. Biomed Phys Eng Express. (2022) 8(4):045010. doi: 10.1088/2057-1976/ac39ab
91. Bettinelli A, Marturano F, Avanzo M, Loi E, Menghi E, Mezzenga E, et al. A novel benchmarking approach to assess the agreement among radiomic tools. Radiology. (2022) 303(3):533–41. doi: 10.1148/radiol.211604
92. Shi Z, Traverso A, van Soest J, Dekker A, Wee L. Technical note: ontology-guided radiomics analysis workflow (O-RAW). Med Phys. (2019) 46(12):5677–84. doi: 10.1002/mp.13844
93. Kothari G, Korte J, Lehrer EJ, Zaorsky NG, Lazarakis S, Kron T, et al. A systematic review and meta-analysis of the prognostic value of radiomics based models in non-small cell lung cancer treated with curative radiotherapy. Radiother Oncol. (2021) 155:188–203. doi: 10.1016/j.radonc.2020.10.023
94. Fanciullo C, Gitto S, Carlicchi E, Albano D, Messina C, Sconfienza LM. Radiomics of musculoskeletal sarcomas: a narrative review. J Imaging. (2022) 8(2):45. doi: 10.3390/jimaging8020045
95. Prenosil GA, Weitzel T, Fürstner M, Hentschel M, Krause T, Cumming P, et al. Towards guidelines to harmonize textural features in PET: haralick textural features vary with image noise, but exposure-invariant domains enable comparable PET radiomics. PLoS One. (2020) 15(3):e0229560. doi: 10.1371/journal.pone.0229560
96. Pasini G, Stefano A, Russo G, Comelli A, Marinozzi F, Bini F. Phenotyping the histopathological subtypes of non-small-cell lung carcinoma: how beneficial is radiomics? Diagnostics. (2023) 13(6):1167. doi: 10.3390/diagnostics13061167
97. Fournier L, Costaridou L, Bidaut L, Michoux N, Lecouvet FE, de Geus-Oei LF, et al. Incorporating radiomics into clinical trials: expert consensus endorsed by the European society of radiology on considerations for data-driven compared to biologically driven quantitative biomarkers. Eur Radiol. (2021) 31(8):6001–12. doi: 10.1007/s00330-020-07598-8
98. Zhu F, Yang C, Zou J, Ma W, Wei Y, Zhao Z. The classification of benign and malignant lung nodules based on CT radiomics: a systematic review, quality score assessment, and meta-analysis. Acta Radiol. (2023) 64(12):3074–84. doi: 10.1177/02841851231205737
99. Escudero Sanchez L, Rundo L, Gill AB, Hoare M, Mendes Serrao E, Sala E. Robustness of radiomic features in CT images with different slice thickness, comparing liver tumour and muscle. Sci Rep. (2021) 11(1):8262. doi: 10.1038/s41598-021-87598-w
100. Loi S, Mori M, Palumbo D, Crippa S, Palazzo G, Spezi E, et al. Limited impact of discretization/interpolation parameters on the predictive power of CT radiomic features in a surgical cohort of pancreatic cancer patients. Radiol Med. (2023) 128(7):799–807. doi: 10.1007/s11547-023-01649-y
101. Kao YS, Lin KT. A meta-analysis of the diagnostic test accuracy of CT-based radiomics for the prediction of COVID-19 severity. Radiol Med. (2022) 127(7):754–62. doi: 10.1007/s11547-022-01510-8
102. Prinzi F, Currieri T, Gaglio S, Vitabile S. Shallow and deep learning classifiers in medical image analysis. Eur Radiol Exper. (2024) 8(1):26. doi: 10.1186/s41747-024-00428-2
103. Avanzo M, Gagliardi V, Stancanello J, Blanck O, Pirrone G, El Naqa I, et al. Combining computed tomography and biologically effective dose in radiomics and deep learning improves prediction of tumor response to robotic lung stereotactic body radiation therapy. Med Phys. (2021) 48(10):6257–69. doi: 10.1002/mp.15178
104. DeJohn CR, Grant SR, Seshadri M. Application of machine learning methods to improve the performance of ultrasound in head and neck oncology: a literature review. Cancers (Basel). (2022) 14(3):665. doi: 10.3390/cancers14030665
105. Wang S, Xiao F, Sun W, Yang C, Ma C, Huang Y, et al. Radiomics analysis based on magnetic resonance imaging for preoperative overall survival prediction in isocitrate dehydrogenase wild-type glioblastoma. Front Neurosci. (2021) 15:791776. doi: 10.3389/fnins.2021.791776
106. He J, Liu Y, Li J, Liu S. Accuracy of radiomics in the diagnosis and preoperative high-risk assessment of endometrial cancer: a systematic review and meta-analysis. Front Oncol. (2024) 14:1334546. doi: 10.3389/fonc.2024.1334546
107. Dragoș HM, Stan A, Pintican R, Feier D, Lebovici A, Ștefan PP, et al. MRI Radiomics and predictive models in assessing ischemic stroke outcome—a systematic review. Diagnostics. (2023) 13(5):857. doi: 10.3390/diagnostics13050857
108. Wu Q, Chang Y, Yang C, Liu H, Chen F, Dong H, et al. Adjuvant chemotherapy or no adjuvant chemotherapy? A prediction model for the risk stratification of recurrence or metastasis of nasopharyngeal carcinoma combining MRI radiomics with clinical factors. PLoS One. (2023) 18(9):e0287031. doi: 10.1371/journal.pone.0287031
109. Bedrikovetski S, Dudi-Venkata NN, Kroon HM, Seow W, Vather R, Carneiro G, et al. Artificial intelligence for pre-operative lymph node staging in colorectal cancer: a systematic review and meta-analysis. BMC Cancer. (2021) 21:1058. doi: 10.1186/s12885-021-08773-w
110. Berger T, Noble DJ, Yang Z, Shelley LEA, McMullan T, Bates A, et al. Assessing the generalisability of radiomics features previously identified as predictive of radiation-induced sticky saliva and xerostomia. Phys Imaging Radiat Oncol. (2023) 25:100404. doi: 10.1016/j.phro.2022.12.001
111. Warman A, Kalluri AL, Azad TD. Machine learning predictive models in neurosurgery: an appraisal based on the TRIPOD guidelines. Systematic review. Neurosurg Focus. (2023) 54(6):E8. doi: 10.3171/2023.3.FOCUS2386
112. Stahl AC, Tietz AS, Dewey M, Kendziora B. Has the quality of reporting improved since it became mandatory to use the Standards for reporting diagnostic accuracy? Insights Imaging. (2023) 14(1):85. doi: 10.1186/s13244-023-01432-7
113. Kocak B, Akinci D'Antonoli T, Ates Kus E, Keles A, Kala A, Kose F, et al. Self-reported checklists and quality scoring tools in radiomics: a meta-research. Eur Radiol. (2024) 34(8):5028–40. doi: 10.1007/s00330-023-10487-5
114. Doniselli FM, Pascuzzo R, Mazzi F, Padelli F, Moscatelli M, Akinci D’Antonoli T, et al. Quality assessment of the MRI-radiomics studies for MGMT promoter methylation prediction in glioma: a systematic review and meta-analysis. Eur Radiol. (2024) 34(9):5802–15. doi: 10.1007/s00330-024-10594-x
115. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMC Med. (2015) 13:1-014-0241-z. doi: 10.1186/s12916-014-0241-z
116. Wilkinson MD, Dumontier M, Aalbersberg IJ, Appleton G, Axton M, Baak A, et al. The FAIR guiding principles for scientific data management and stewardship. Sci Data. (2016) 3:160018. doi: 10.1038/sdata.2016.18
117. Feng Y, Gong J, Hu T, Liu Z, Sun Y, Tong T. Radiomics for predicting survival in patients with locally advanced rectal cancer: a systematic review and meta-analysis. Quant Imaging Med Surg. (2023) 13(12):8395–412. doi: 10.21037/qims-23-692
118. Wang Q, Xu J, Wang A, Chen Y, Wang T, Chen D, et al. Systematic review of machine learning-based radiomics approach for predicting microsatellite instability status in colorectal cancer. Radiol Med. (2023) 128(2):136–48. doi: 10.1007/s11547-023-01593-x
119. Spadarella G, Stanzione A, Akinci D’Antonoli T, Andreychenko A, Fanni SC, Ugga L, et al. Systematic review of the radiomics quality score applications: an EuSoMII radiomics auditing group initiative. Eur Radiol. (2023) 33(3):1884–94. doi: 10.1007/s00330-022-09187-3
120. Lee S, Han K, Suh YJ. Quality assessment of radiomics research in cardiac CT: a systematic review. Eur Radiol. (2022) 32(5):3458–68. doi: 10.1007/s00330-021-08429-0
121. Bhandari AP, Liong R, Koppen J, Murthy SV, Lasocki A. Noninvasive determination of IDH and 1p19q Status of lower-grade gliomas using MRI radiomics: a systematic review. AJNR Am J Neuroradiol. (2021) 42(1):94–101. doi: 10.3174/ajnr.A6875
122. Gupta R, Bilgin C, Jabal MS, Kandemirli S, Ghozy S, Kobeissi H, et al. Quality assessment of radiomics studies on functional outcomes after acute ischemic stroke-A systematic review. World Neurosurg. (2024) 183:164–71. doi: 10.1016/j.wneu.2023.11.154
123. Orlhac F, Nioche C, Klyuzhin I, Rahmim A, Buvat I. Radiomics in PET imaging: a practical guide for newcomers. PET Clin. (2021) 16(4):597–612. doi: 10.1016/j.cpet.2021.06.007
124. Pfaehler E, Zhovannik I, Wei L, Boellaard R, Dekker A, Monshouwer R, et al. A systematic review and quality of reporting checklist for repeatability and reproducibility of radiomic features. Phys Imaging Radiat Oncol. (2021) 20:69–75. doi: 10.1016/j.phro.2021.10.007
125. Gillies RJ, Kinahan PE, Hricak H. Radiomics: images are more than pictures, they are data. Radiology. (2016) 278(2):563–77. doi: 10.1148/radiol.2015151169
126. Aerts HJ, Velazquez ER, Leijenaar RT, Parmar C, Grossmann P, Carvalho S, et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun. (2014) 5:4006. doi: 10.1038/ncomms5006
127. FDA-NIH Biomarker Working Group. BEST (Biomarkers, EndpointS, and Other Tools) Resource. Silver Spring (MD): Food and Drug Administration (US) (2016). Available online at: http://www.ncbi.nlm.nih.gov/books/NBK326791/ (Accessed February 08, 2023).
128. Park JE, Kim HS. Radiomics as a quantitative imaging biomarker: practical considerations and the current standpoint in neuro-oncologic studies. Nucl Med Mol Imaging. (2018) 52(2):99–108. doi: 10.1007/s13139-017-0512-7
129. Chauvie S, Mazzoni LN, O’Doherty J. A review on the use of imaging biomarkers in oncology clinical trials: quality assurance strategies for technical validation. Tomography. (2023) 9(5):1876–902. doi: 10.3390/tomography9050149
130. Kinahan PE, Perlman ES, Sunderland JJ, Subramaniam R, Wollenweber SD, Turkington TG, et al. The QIBA profile for FDG PET/CT as an imaging biomarker measuring response to cancer therapy. Radiology. (2020) 294(3):647–57. doi: 10.1148/radiol.2019191882
131. Cai WL, Hong GB. Quantitative image analysis for evaluation of tumor response in clinical oncology. Chronic Dis Translat Med. (2018) 4(1):18–28. doi: 10.1016/j.cdtm.2018.01.002
132. Califf RM. Biomarker definitions and their applications. Exp Biol Med (Maywood). (2018) 243(3):213–21. doi: 10.1177/1535370217750088
133. Leijenaar RT, Carvalho S, Hoebers FJ, Aerts HJ, van Elmpt WJ, Huang SH, et al. External validation of a prognostic CT-based radiomic signature in oropharyngeal squamous cell carcinoma. Acta Oncol. (2015) 54(9):1423–9. doi: 10.3109/0284186X.2015.1061214
134. Li S, Ding C, Zhang H, Song J, Wu L. Radiomics for the prediction of EGFR mutation subtypes in non-small cell lung cancer. Med Phys. (2019) 46(10):4545–52. doi: 10.1002/mp.13747
135. Simpson G, Jin W, Spieler B, Portelance L, Mellon E, Kwon D, et al. Predictive Value of delta-radiomics texture features in 0.35 tesla magnetic resonance setup images acquired during stereotactic ablative radiotherapy of pancreatic cancer. Front Oncol. (2022) 2:807725. doi: 10.3389/fonc.2022.807725
136. Smith JJ, Sorensen AG, Thrall JH. Biomarkers in imaging: realizing radiology’s future. Radiology. (2003) 227(3):633–8. doi: 10.1148/radiol.2273020518
137. Schuster DP. The opportunities and challenges of developing imaging biomarkers to study lung function and disease. Am J Respir Crit Care Med. (2007) 176(3):224–30. doi: 10.1164/rccm.200703-462PP
138. Medeiros FA. Biomarkers and surrogate endpoints in glaucoma clinical trials. Br J Ophthalmol. (2015) 99(5):599–603. doi: 10.1136/bjophthalmol-2014-305550
139. Bleker J, Kwee TC, Yakar D. Quality of multicenter studies using MRI radiomics for diagnosing clinically significant prostate cancer: a systematic review. Life (Basel). (2022) 12(7):946. doi: 10.3390/life12070946
140. Talari K, Goyal M. Retrospective studies—utility and caveats. J R Coll Physicians Edinb. (2020) 50(4):398–402. doi: 10.4997/jrcpe.2020.409
141. Pavic M, Bogowicz M, Würms X, Glatz S, Finazzi T, Riesterer O, et al. Influence of inter-observer delineation variability on radiomics stability in different tumor sites. Acta Oncol. (2018) 57(8):1070–4. doi: 10.1080/0284186X.2018.1445283
142. Fontaine P, Andrearczyk V, Oreiller V, Abler D, Castelli J, Acosta O, et al. Cleaning radiotherapy contours for radiomics studies, is it worth it? A head and neck cancer study. Clin Transl Radiat Oncol. (2022) 33:153–8. doi: 10.1016/j.ctro.2022.01.003
143. Ait Skourt BEl Hassani AMajda A. Lung CT image segmentation using deep neural networks. Procedia Comput Sci. (2018) 127:109–13. doi: 10.1016/j.procs.2018.01.104
144. Zhong Z, Kim Y, Zhou L, Plichta K, Allen B, Buatti J, et al. 3D Fully convolutional networks for co-segmentation of tumors on PET-CT images. 2018 IEEE 15th Int Sympos Biomed Imaging (ISBI 2018). (2018) (Generic):228–31. doi: 10.1109/ISBI.2018.8363561
145. Granzier RWY, Verbakel NMH, Ibrahim A, van Timmeren JE, van Nijnatten TJA, Leijenaar RTH, et al. MRI-based radiomics in breast cancer: feature robustness with respect to inter-observer segmentation variability. Sci Rep. (2020) 10(1):14163. doi: 10.1038/s41598-020-70940-z
146. Placidi L, Gioscio E, Garibaldi C, Rancati T, Fanizzi A, Maestri D, et al. A multicentre evaluation of dosiomics features reproducibility, stability and sensitivity. Cancers (Basel). (2021) 13(15):3835. doi: 10.3390/cancers13153835
147. McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb). (2012) 22(3):276–82. doi: 10.11613/BM.2012.031
148. Zwanenburg A, Vallières M, Abdalah MA, Aerts HJWL, Andrearczyk V, Apte A, et al. The image biomarker standardization initiative: standardized quantitative radiomics for high-throughput image-based phenotyping. Radiology. (2020) 295(2):328–38. doi: 10.1148/radiol.2020191145
149. Image Biomarker Standardisation Initiative: Reference Manual (v11). (2019). Available online at: https://arxiv.org/pdf/1612.07003v11.pdf (Accessed September 30, 2022).
150. Depeursinge A, Andrearczyk V, Whybra P, Van Griethuysen J, Müller H, Schaer R, et al. Standardised convolutional filtering for Radiomics Image Biomarker Standardisation Initiative (IBSI): Reference Manual. (2022). Available online at: https://arxiv.org/pdf/2006.05470.pdf (Accessed November 14, 2022).
151. Hatt M, Rest CCL, Tixier F, Badic B, Schick U, Visvikis D. Radiomics: data are also images. J Nucl Med. (2019) 60(Suppl 2):38S–44. doi: 10.2967/jnumed.118.220582
152. Bagher-Ebadian H, Siddiqui F, Liu C, Movsas B, Chetty IJ. On the impact of smoothing and noise on robustness of CT and CBCT radiomics features for patients with head and neck cancers. Med Phys. (2017) 44(5):1755–70. doi: 10.1002/mp.12188
153. Demircioğlu A. The effect of preprocessing filters on predictive performance in radiomics. Eur Radiol Exper. (2022) 6(1):40. doi: 10.1186/s41747-022-00294-w
154. Branchini M, Zorz A, Zucchetta P, Bettinelli A, De Monte F, Cecchin D, et al. Impact of acquisition count statistics reduction and SUV discretization on PET radiomic features in pediatric 18F-FDG-PET/MRI examinations. Phys Med. (2019) 59:117–26. doi: 10.1016/j.ejmp.2019.03.005
155. Shafiq-ul-Hassan M, Zhang GG, Latifi K, Ullah G, Hunt DC, Balagurunathan Y, et al. Intrinsic dependencies of CT radiomic features on voxel size and number of gray levels. Med Phys. (2017) 44(3):1050–62. doi: 10.1002/mp.12123
156. Zwanenburg A, Löck S. MIRP: a python package for standardised radiomics. J Open Source Softw. (2024) 9:6413. doi: 10.21105/joss.06413
157. Bettinelli A, Branchini M, De Monte F, Scaggion A, Paiusco M. Technical note: an IBEX adaption toward image biomarker standardization. Med Phys. (2020) 47(3):1167–73. doi: 10.1002/mp.13956
158. Nioche C, Orlhac F, Boughdad S, Reuze S, Goya-Outi J, Robert C, et al. Lifex: a freeware for radiomic feature calculation in multimodality imaging to accelerate advances in the characterization of tumor heterogeneity. Cancer Res. (2018) 78(16):4786–9. doi: 10.1158/0008-5472.CAN-18-0125
159. Pfaehler E, Zwanenburg A, de Jong JR, Boellaard R. RACAT: an open source and easy to use radiomics calculator tool. PLoS One. (2019) 14(2):1–26. doi: 10.1371/journal.pone.0212223
160. Bagher-Ebadian H, Chetty IJ. Technical note: rOdiomiX: a validated software for radiomics analysis of medical images in radiation oncology. Med Phys. (2021) 48(1):354–65. doi: 10.1002/mp.14590
161. Götz M, Nolden M, Maier-Hein K. MITK Phenotyping: an open-source toolchain for image-based personalized medicine with radiomics. Radiother Oncol. (2019) 131:108–11. doi: 10.1016/j.radonc.2018.11.021
162. Dinapoli N, Alitto AR, Vallati M, Gatta R, Autorino R, Boldrini L, et al. Moddicom: a complete and easily accessible library for prognostic evaluations relying on image features.. Proc Ann Int Conf IEEE Engin Med Biol Soc EMBS. (2015):771–4. doi: 10.1109/EMBC.2015.7318476
163. PyRadiomics from the Computational Imaging & Bioinformatics Lab - Harvard Medical School. (Online). (2017).
164. Avanzo M, Pirrone G, Vinante L, Caroli A, Stancanello J, Drigo A, et al. Electron density and biologically effective dose (BED) radiomics-based machine learning models to predict late radiation-induced subcutaneous fibrosis. Front Oncol. (2020) 10:490. doi: 10.3389/fonc.2020.00490
165. Mali SA, Ibrahim A, Woodruff HC, Andrearczyk V, Müller H, Primakov S, et al. Making radiomics more reproducible across scanner and imaging protocol variations: a review of harmonization methods. JPM. (2021) 11(9):842. doi: 10.3390/jpm11090842
166. Zhao B, Tan Y, Tsai WY, Schwartz LH, Lu L. Exploring variability in CT characterization of tumors: a preliminary phantom study. Transl Oncol. (2014) 7(1):88–93. doi: 10.1593/tlo.13865
167. Mackin D, Fave X, Zhang L, Fried D, Yang J, Taylor B, et al. Measuring computed tomography scanner variability of radiomics features. Invest Radiol. (2015) 50(11):757–65. doi: 10.1097/RLI.0000000000000180
168. Caramella C, Allorant A, Orlhac F, Bidault F, Asselain B, Ammari S, et al. Can we trust the calculation of texture indices of CT images? A phantom study. Med Phys. (2018) 45(4):1529–36. doi: 10.1002/mp.12809
169. Berenguer R, Pastor-Juan MDR, Canales-Vázquez J, Castro-García M, Villas MV, Mansilla Legorburo F, et al. Radiomics of CT features may be nonreproducible and redundant: influence of CT acquisition parameters. Radiology. (2018) 288(2):407–15. doi: 10.1148/radiol.2018172361
170. Edalat-Javid M, Shiri I, Hajianfar G, Abdollahi H, Arabi H, Oveisi N, et al. Cardiac SPECT radiomic features repeatability and reproducibility: a multi-scanner phantom study. J Nucl Cardiol. (2021) 28(6):2730–44. doi: 10.1007/s12350-020-02109-0
171. Mackin D, Ger R, Dodge C, Fave X, Chi PC, Zhang L, et al. Effect of tube current on computed tomography radiomic features. Sci Rep. (2018) 8(1):2354. doi: 10.1038/s41598-018-20713-6
172. Yang F, Dogan N, Stoyanova R, Ford JC. Evaluation of radiomic texture feature error due to MRI acquisition and reconstruction: a simulation study utilizing ground truth. Phys Med. (2018) 50:26–36. doi: 10.1016/j.ejmp.2018.05.017
173. Mackin D, Fave X, Zhang L, Yang J, Jones AK, Ng CS, et al. Harmonizing the pixel size in retrospective computed tomography radiomics studies. PLoS One. (2017) 12(9):e0178524. doi: 10.1371/journal.pone.0178524
174. Midya A, Chakraborty J, Gönen M, Do RKG, Simpson AL. Influence of CT acquisition and reconstruction parameters on radiomic feature reproducibility. J Med Imaging (Bellingham). (2018) 5(1):011020. doi: 10.1117/1.JMI.5.1.011020
175. Ligero M, Jordi-Ollero O, Bernatowicz K, Garcia-Ruiz A, Delgado-Muñoz E, Leiva D, et al. Minimizing acquisition-related radiomics variability by image resampling and batch effect correction to allow for large-scale data analysis. Eur Radiol. (2021) 31(3):1460–70. doi: 10.1007/s00330-020-07174-0
176. Shur J, Blackledge M, D’Arcy J, Collins DJ, Bali M, O’Leach M, et al. MRI Texture feature repeatability and image acquisition factor robustness, a phantom study and in silico study. Eur Radiol Exp. (2021) 5(1):2. doi: 10.1186/s41747-020-00199-6
177. Bologna M, Tenconi C, Corino VDA, Annunziata G, Orlandi E, Calareso G, et al. Repeatability and reproducibility of MRI-radiomic features: a phantom experiment on a 1.5T scanner. Med Phys. (2023) 50(2):750–62. doi: 10.1002/mp.16054
178. Bologna M, Corino V, Mainardi L. Technical note: virtual phantom analyses for preprocessing evaluation and detection of a robust feature set for MRI-radiomics of the brain. Med Phys. (2019) 46(11):5116–23. doi: 10.1002/mp.13834
179. Erdal BS, Demirer M, Little KJ, Amadi CC, Ibrahim GFM, O’Donnell TP, et al. Are quantitative features of lung nodules reproducible at different CT acquisition and reconstruction parameters? PLoS One. (2020) 15(10):e0240184. doi: 10.1371/journal.pone.0240184
180. Khorrami M, Bera K, Thawani R, Rajiah P, Gupta A, Fu P, et al. Distinguishing granulomas from adenocarcinomas by integrating stable and discriminating radiomic features on non-contrast computed tomography scans. Eur J Cancer. (2021) 148:146–58. doi: 10.1016/j.ejca.2021.02.008
181. Crombé A, Saut O, Guigui J, Italiano A, Buy X, Kind M. Influence of temporal parameters of DCE-MRI on the quantification of heterogeneity in tumor vascularization. J Magn Reson Imaging. (2019) 50(6):1773–88. doi: 10.1002/jmri.26753
182. Rinaldi L, De Angelis SP, Raimondi S, Rizzo S, Fanciullo C, Rampinelli C, et al. Reproducibility of radiomic features in CT images of NSCLC patients: an integrative analysis on the impact of acquisition and reconstruction parameters. Eur Radiol Exp. (2022) 6(1):2. doi: 10.1186/s41747-021-00258-6
183. Fortin JP, Parker D, Tunç B, Watanabe T, Elliott MA, Ruparel K, et al. Harmonization of multi-site diffusion tensor imaging data. Neuro Image. (2017) 161:149–70. doi: 10.1016/j.neuroimage.2017.08.047
184. Fortin JP, Cullen N, Sheline YI, Taylor WD, Aselcioglu I, Cook PA, et al. Harmonization of cortical thickness measurements across scanners and sites. Neuro Image. (2018) 167:104–20. doi: 10.1016/j.neuroimage.2017.11.024
185. Johnson WE, Li C, Rabinovic A. Adjusting batch effects in microarray expression data using empirical Bayes methods. Biostatistics. (2007) 8(1):118–27. doi: 10.1093/biostatistics/kxj037
186. Orlhac F, Frouin F, Nioche C, Ayache N, Buvat I. Validation of A method to compensate multicenter effects affecting CT radiomics. Radiology. (2019) 291(1):53–9. doi: 10.1148/radiol.2019182023
187. Mahon RN, Ghita M, Hugo GD, Weiss E. Combat harmonization for radiomic features in independent phantom and lung cancer patient computed tomography datasets. Phys Med Biol. (2020) 65(1):015010. doi: 10.1088/1361-6560/ab6177
188. Orlhac F, Lecler A, Savatovski J, Goya-Outi J, Nioche C, Charbonneau F, et al. How can we combat multicenter variability in MR radiomics? Validation of a correction procedure. Eur Radiol. (2021) 31(4):2272–80. doi: 10.1007/s00330-020-07284-9
189. Orlhac F, Boughdad S, Philippe C, Stalla-Bourdillon H, Nioche C, Champion L, et al. A postreconstruction harmonization method for multicenter radiomic studies in PET. J Nucl Med. (2018) 59(8):1321–8. doi: 10.2967/jnumed.117.199935
190. Da-Ano R, Masson I, Lucia F, Doré M, Robin P, Alfieri J, et al. Performance comparison of modified ComBat for harmonization of radiomic features for multicenter studies. Sci Rep. (2020) 10(1):10248. doi: 10.1038/s41598-020-66110-w
191. Bertolini M, Trojani V, Botti A, Cucurachi N, Galaverni M, Cozzi S, et al. Novel harmonization method for multi-centric radiomic studies in non-small cell lung cancer. Current Oncology. (2022) 29(8):5179–94. doi: 10.3390/curroncol29080410
192. Rogers W, Thulasi Seetha S, Refaee TAG, Lieverse RIY, Granzier RWY, Ibrahim A, et al. Radiomics: from qualitative to quantitative imaging. Br J Radiol. (2020) 93(1108):20190948. doi: 10.1259/bjr.20190948
193. Sorantin E, Grasser MG, Hemmelmayr A, Tschauner S, Hrzic F, Weiss V, et al. The augmented radiologist: artificial intelligence in the practice of radiology. Pediatr Radiol. (2022) 52(11):2074–86. doi: 10.1007/s00247-021-05177-7
194. He Q, Li X, Kim DWN, Jia X, Gu X, Zhen X, et al. Feasibility study of a multi-criteria decision-making based hierarchical model for multi-modality feature and multi-classifier fusion: applications in medical prognosis prediction. Information Fusion. (2020) 55:207–19. doi: 10.1016/j.inffus.2019.09.001
195. Caruso CM, Guarrasi V, Cordelli E, Sicilia R, Gentile S, Messina L, et al. A multimodal ensemble driven by multiobjective optimisation to predict overall survival in non-small-cell lung cancer. J Imaging. (2022) 8(11):298. doi: 10.3390/jimaging8110298
196. Caruso CM, Guarrasi V, Ramella S, Soda P. A deep learning approach for overall survival prediction in lung cancer with missing values. Comput Methods Programs Biomed. (2024) 254:108308. doi: 10.1016/j.cmpb.2024.108308
197. Lee C, Zame W, Yoon J, van der Schaar M. Deephit: a deep learning approach to survival analysis with competing risks. Proc AAAI Conf Artif Intell. (2018) 32(1). doi: 10.1609/aaai.v32i1.11842
198. Morgan JN, Sonquist JA. Problems in the analysis of survey data, and a proposal. J Am Stat Assoc. (1963) 58(302):415–34. doi: 10.1080/01621459.1963.10500855
199. Galar M, Fernandez A, Barrenechea E, Bustince H, Herrera F. A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches. IEEE Transact Syst Man Cybernetics, Part C (Applications and Reviews). (2012) 42(4):463–84. doi: 10.1109/TSMCC.2011.2161285
200. Boser BE, Guyon IM, Vapnik VN. A training algorithm for optimal margin classifiers. In: Proceedings of the Fifth Annual Workshop on Computational Learning Theory. New York, NY, USA: Association for Computing Machinery (1992). p. 144–52. (COLT ‘92). doi: 10.1145/130385.130401
201. Baxt WG. Use of an artificial neural network for the diagnosis of myocardial infarction. Ann Intern Med. (1991) 115(11):843–8. doi: 10.7326/0003-4819-115-11-843
202. Blum AL, Langley P. Selection of relevant features and examples in machine learning. Artif Intell. (1997) 97(1):245–71. doi: 10.1016/S0004-3702(97)00063-5
203. Hall M. Correlation-Based feature selection for machine learning. Department of Computer science. University of Waikato, Hamilton, New Zealand (2000) 19.
204. Wolpert DH, Macready WG. No free lunch theorems for optimization. IEEE Transact Evol Comput. (1997) 1(1):67–82. doi: 10.1109/4235.585893
205. Arcuri A, Fraser G. Parameter tuning or default values? An empirical investigation in search-based software engineering. Empir Software Eng. (2013) 18(3):594–623. doi: 10.1007/s10664-013-9249-9
206. Maddalo M, Fanizzi A, Lambri N, Loi E, Branchini M, Lorenzon L, et al. Robust machine learning challenge: an AIFM multicentric competition to spread knowledge, identify common pitfalls and recommend best practice. Phys Med. (2024) 127:104834. doi: 10.1016/j.ejmp.2024.104834
207. Zhang C, Soda P, Bi J, Fan G, Almpanidis G, García S, et al. An empirical study on the joint impact of feature selection and data resampling on imbalance classification. Appl Intell. (2022) 53(5):5449–61. doi: 10.1007/s10489-022-03772-1
208. Fawcett T. ROC Graphs: notes and practical considerations for researchers. ReCALL. (2004) 31:1–38.
209. Fernández A, García S, Galar M, Prati RC, Krawczyk B, Herrera F. Learning from Imbalanced Data Sets. Cham: Springer International Publishing (2018). Available online at: http://link.springer.com/10.1007/978-3-319-98074-4 (Accessed August 10, 2023).
210. Japkowicz N. Assessment metrics for imbalanced learning. In: Imbalanced Learning. Hoboken, NJ: John Wiley & Sons, Ltd (2013). p. 187–206. Available online at: https://onlinelibrary.wiley.com/doi/abs/10.1002/9781118646106.ch8
211. Avanzo M, Pirrone G, Mileto M, Massarut S, Stancanello J, Baradaran-Ghahfarokhi M, et al. Prediction of skin dose in low-kV intraoperative radiotherapy using machine learning models trained on results of in vivo dosimetry. Med Phys. (2019) 46(3):1447–54. doi: 10.1002/mp.13379
212. Buda M, Maki A, Mazurowski MA. A systematic study of the class imbalance problem in convolutional neural networks. arXiv e-prints. 2017;(Generic):arXiv:1710.05381.
213. Lemaitre G, Nogueira F, Aridas CK. Imbalanced-learn: a python toolbox to tackle the curse of imbalanced datasets in machine learning. arXiv e-prints. 2016;(Generic):arXiv:1609.06570.
214. Sheskin D. Handbook of Parametric and Nonparametric Statistical Procedures. New York, NY: Chapman & Hall/CRC (2007). doi: 10.5555/1529939
215. Luengo J, García S, Herrera F. A study on the use of statistical tests for experimentation with neural networks: analysis of parametric test conditions and non-parametric tests. Expert Syst Appl. (2009) 36:7798–808. doi: 10.1016/j.eswa.2008.11.041
216. Koçak B, Durmaz EŞ, Ateş E, Kılıçkesmez Ö. Radiomics with artificial intelligence: a practical guide for beginners. Diagn Interv Radiol. (2019) 25(6):485–95. doi: 10.5152/dir.2019.19321
217. Koçak B. Key concepts, common pitfalls, and best practices in artificial intelligence and machine learning: focus on radiomics. Diagn Interv Radiol. (2022) 28(5):450–62. doi: 10.5152/dir.2022.211297
218. Prior FW, Clark K, Commean P, Freymann J, Jaffe C, Kirby J, et al. TCIA: an information resource to enable open science. Conf Proc IEEE Eng Med Biol Soc. (2013) 2013:1282–5. doi: 10.1109/EMBC.2013.6609742
219. Regulations.gov. (2019). Available online at: https://www.regulations.gov/document/FDA-2019-N-1185-0001 (Accessed November 28, 2021).
220. Zanca F, Brusasco C, Pesapane F, Kwade Z, Beckers R, Avanzo M. Regulatory aspects of the use of artificial intelligence medical software. Semin Radiat Oncol. (2022) 32(4):432–41. doi: 10.1016/j.semradonc.2022.06.012
221. Leijenaar RT, Carvalho S, Velazquez ER, van Elmpt WJ, Parmar C, Hoekstra OS, et al. Stability of FDG-PET radiomics features: an integrated analysis of test-retest and inter-observer variability. Acta Oncol. (2013) 52(7):1391–7. doi: 10.3109/0284186X.2013.812798
222. Shukla-Dave A, Obuchowski NA, Chenevert TL, Jambawalikar S, Schwartz LH, Malyarenko D, et al. Quantitative imaging biomarkers alliance (QIBA) recommendations for improved precision of DWI and DCE-MRI derived biomarkers in multicenter oncology trials. J Magn Reson Imaging. (2019) 49(7):e101–21. doi: 10.1002/jmri.26518
223. Jha AK, Bradshaw TJ, Buvat I, Hatt M, Kc P, Liu C, et al. Nuclear medicine and artificial intelligence: best practices for evaluation (the RELAINCE guidelines). J Nucl Med. (2022) 63(9):1288–99. doi: 10.2967/jnumed.121.263239
224. Zhao B, Tan Y, Tsai WY, Qi J, Xie C, Lu L, et al. Reproducibility of radiomics for deciphering tumor phenotype with imaging. Sci Rep. (2016) 6(1):23428. doi: 10.1038/srep23428
225. He L, Huang Y, Ma Z, Liang C, Liang C, Liu Z. Effects of contrast-enhancement, reconstruction slice thickness and convolution kernel on the diagnostic performance of radiomics signature in solitary pulmonary nodule. Sci Rep. (2016) 6(1):34921. doi: 10.1038/srep34921
226. Bogowicz M, Riesterer O, Bundschuh RA, Veit-Haibach P, Hüllner M, Studer G, et al. Stability of radiomic features in CT perfusion maps. Phys Med Biol. (2016) 61(24):8736–49. doi: 10.1088/1361-6560/61/24/8736
227. Duda D, Kretowski M, Bezy-Wendling J. Effect of slice thickness on texture-based classification of liver dynamic CT scans. In: Saeed K, Chaki R, Cortesi A, Wierzchoń S, curatori. Computer Information Systems and Industrial Management. Berlin, Heidelberg: Springer Berlin Heidelberg; 2013. p. 96–107. (Lecture Notes in Computer Science; vol. 8104). Available online at: http://link.springer.com/10.1007/978-3-642-40925-7_10 (Accessed May 22, 2023).
228. Fave X, Zhang L, Yang J, Mackin D, Balter P, Gomez D, et al. Impact of image preprocessing on the volume dependence and prognostic potential of radiomics features in non-small cell lung cancer. Transl Cancer Res. (2016) 5(4):349–63. doi: 10.21037/tcr.2016.07.11
229. Rinaldi L, Pezzotta F, Santaniello T, De Marco P, Bianchini L, Origgi D, et al. HeLLePhant: a phantom mimicking non-small cell lung cancer for texture analysis in CT images. Phys Med. (2022) 97:13–24. doi: 10.1016/j.ejmp.2022.03.010
230. Foy JJ, Robinson KR, Li H, Giger ML, Al-Hallaq H, Armato SG. Variation in algorithm implementation across radiomics software. J Med Imaging. (2018) 5(04):1. doi: 10.1117/1.JMI.5.4.044505
231. Fornacon-Wood I, Mistry H, Ackermann CJ, Blackhall F, McPartlin A, Faivre-Finn C, et al. Reliability and prognostic value of radiomic features are highly dependent on choice of feature extraction platform. Eur Radiol. (2020) 30(11):6241–50. doi: 10.1007/s00330-020-06957-9
232. McNitt-Gray M, Napel S, Jaggi A, Mattonen SA, Hadjiiski L, Muzi M, et al. Standardization in quantitative imaging: a multicenter comparison of radiomic features from different software packages on digital reference objects and patient data sets. Tomography. (2020) 6(2):118–28. doi: 10.18383/j.tom.2019.00031
233. Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. Cham: Springer International Publishing (2019). (Statistics for Biology and Health). Available online at: http://link.springer.com/10.1007/978-3-030-16399-0 (Accessed August 10, 2023).
234. Ramspek CL, Jager KJ, Dekker FW, Zoccali C, van Diepen M. External validation of prognostic models: what, why, how, when and where? Clin Kidney J. (2021) 14(1):49–58. doi: 10.1093/ckj/sfaa188
235. Efron B. Bootstrap methods: another Look at the jackknife. Ann Stat. (1979) 7(1):1–26. doi: 10.1214/aos/1176344552
236. Ramella S, Fiore M, Greco C, Cordelli E, Sicilia R, Merone M, et al. A radiomic approach for adaptive radiotherapy in non-small cell lung cancer patients. PLoS One. (2018) 13(11):e0207455. doi: 10.1371/journal.pone.0207455
237. Steyerberg EW, Vergouwe Y. Towards better clinical prediction models: seven steps for development and an ABCD for validation. Eur Heart J. (2014) 35(29):1925–31. doi: 10.1093/eurheartj/ehu207
238. Sun R, Limkin EJ, Vakalopoulou M, Dercle L, Champiat S, Han SR, et al. A radiomics approach to assess tumour-infiltrating CD8 cells and response to anti-PD-1 or anti-PD-L1 immunotherapy: an imaging biomarker, retrospective multicohort study. Lancet Oncol. (2018) 19(9):1180–91. doi: 10.1016/S1470-2045(18)30413-3
239. Panth KM, Leijenaar RT, Carvalho S, Lieuwes NG, Yaromina A, Dubois L, et al. Is there a causal relationship between genetic changes and radiomics-based image features? An in vivo preclinical experiment with doxycycline inducible GADD34 tumor cells. Radiother Oncol. (2015) 116(3):462–6. doi: 10.1016/j.radonc.2015.06.013
240. Brown KH, Payan N, Osman S, Ghita M, Walls GM, Patallo IS, et al. Development and optimisation of a preclinical cone beam computed tomography-based radiomics workflow for radiation oncology research. Phys Imaging Radiat Oncol. (2023) 26:100446. doi: 10.1016/j.phro.2023.100446
241. Müller J, Leger S, Zwanenburg A, Suckert T, Lühr A, Beyreuther E, et al. Radiomics-based tumor phenotype determination based on medical imaging and tumor microenvironment in a preclinical setting. Radiother Oncol. (2022) 169:96–104. doi: 10.1016/j.radonc.2022.02.020
242. Allphin AJ, Mowery YM, Lafata KJ, Clark DP, Bassil AM, Castillo R, et al. Photon counting CT and radiomic analysis enables differentiation of tumors based on lymphocyte burden. Tomography. (2022) 8(2):740–53. doi: 10.3390/tomography8020061
243. Zinn PO, Singh SK, Kotrotsou A, Hassan I, Thomas G, Luedi MM, et al. A coclinical radiogenomic validation study: conserved magnetic resonance radiomic appearance of periostin-expressing glioblastoma in patients and Xenograft models. Clin Cancer Res. (2018) 24(24):6288–99. doi: 10.1158/1078-0432.CCR-17-3420
244. Polevoy D, Kazimirov D, Gilmanov M, Nikolaev D. No reproducibility, No progress: rethinking CT benchmarking. Journal of Imaging. (2025) 11(10):344. doi: 10.3390/jimaging11100344
245. COCIR: COCIR Analysis on AI in medical Device Legislation - May 2021. (2021). Available online at: https://www.cocir.org/latest-news/publications/article/cocir-analysis-on-ai-in-medical-device-legislation-may-2021 (Accessed September 04, 2024).
246. Ebers M, Hoch VRS, Rosenkranz F, Ruschemeier H, Steinrötter B. The European commission’s proposal for an artificial intelligence act—a critical assessment by members of the robotics and AI law society (RAILS). J. (2021) 4(4):589–603. doi: 10.3390/j4040043
247. Health C for D and R. Good Machine Learning Practice for Medical Device Development: Guiding Principles. FDA. (2023). Available online at: https://www.fda.gov/medical-devices/software-medical-device-samd/good-machine-learning-practice-medical-device-development-guiding-principles (Accessed September 04, 2024).
248. Chinzei K, Shimizu A, Mori K, Harada K, Takeda H, Hashizume M, et al. Regulatory science on AI-based medical devices and systems. Advanced Biomedical Engineering. (2018) 7:118–23. doi: 10.14326/abe.7.118
249. NMPA Announcement on Guidance for the Classification Defining of AI-Based Medical Software Products. Available online at: http://english.nmpa.gov.cn/2021-07/08/c_660267.htm (Accessed August 03, 2023).
250. Woznicki P, Laqua FC, Al-Haj A, Bley T, Baeßler B. Addressing challenges in radiomics research: systematic review and repository of open-access cancer imaging datasets. Insights Imaging. (2023) 14(1):216. doi: 10.1186/s13244-023-01556
251. Xu Y, Li Y, Wang F, Zhang Y, Huang D. Addressing the current challenges in the clinical application of AI-based radiomics for cancer imaging. Front Med (Lausanne). (2025) 12:1674397. doi: 10.3389/fmed.2025.1674397
252. Maleki F, Ovens K, Gupta R, Reinhold C, Spatz A, Forghani R. Generalizability of machine learning models: quantitative evaluation of three methodological pitfalls. Radiol Artif Intell. (2022) 5(1):e220028. doi: 10.1148/ryai.220028
253. Kocak B, Pinto dos Santos D, Dietzel M. The widening gap between radiomics research and clinical translation: rethinking current practices and shared responsibilities. Eur J Radiol Artif Intell. (2025) (1):100004. doi: 10.1016/j.ejrai.2025.100004
254. Sanduleanu S, Woodruff HC, de Jong EEC, van Timmeren JE, Jochems A, Dubois L, et al. Tracking tumor biology with radiomics: a systematic review utilizing a radiomics quality score. Radiother Oncol. (2018) 127(3):349–60. doi: 10.1016/j.radonc.2018.03.033
255. Barry N, Kendrick J, Molin K, Li S, Rowshanfarzad P, Hassan GM, et al. Evaluating the impact of the radiomics quality score: a systematic review and meta-analysis. Eur Radiol. (2025) 35(3):1701–13. doi: 10.1007/s00330-024-11341-y
256. Li X, Yang L, Jiao X. Comparison of traditional radiomics, deep learning radiomics and fusion methods for axillary lymph node metastasis prediction in breast cancer. Acad Radiol. (2023) 30(7):1281–7. doi: 10.1016/j.acra.2022.10.015
257. Raza A, Guzzo A, Ianni M, Lappano R, Zanolini A, Maggiolini M, et al. Federated Learning in radiomics: a comprehensive meta-survey on medical image analysis. Comput Methods Programs Biomed. (2025) 267:108768. doi: 10.1016/j.cmpb.2025.108768
258. Ashrafinia S. Quantitative Nuclear Medicine Imaging Using Advanced Image Reconstruction and Radiomics. (2019). Available online at: https://jscholarship.library.jhu.edu/bitstream/handle/1774.2/61551/ASHRAFINIA-DISSERTATION-2019.pdf?sequence=1 (Accessed December 18, 2025).
Keywords: radiomics, machine learning, artificial intelligence, guidelines & recommendations, medical imaging
Citation: Avanzo M, Soda P, Bertolini M, Bettinelli A, Rancati T, Stancanello J, Rampado O, Pirrone G and Drigo A (2026) Robust radiomics: a review of guidelines for radiomics in medical imaging. Front. Radiol. 5:1701110. doi: 10.3389/fradi.2025.1701110
Received: 18 September 2025; Revised: 22 November 2025;
Accepted: 29 November 2025;
Published: 12 January 2026.
Edited by:
Abhirup Banerjee, University of Oxford, United KingdomReviewed by:
Luigi La Via, Gaspare Rodolico Hospital, ItalyStephane Chauvie, Azienda Sanitaria Ospedaliera S.Croce e Carle Cuneo, Italy
Jesutofunmi Fajemisin, University of South Florida, Tampa, United States
Yen Cho Huang, Keelung Chang Gung Memorial Hospital, Taiwan
Copyright: © 2026 Avanzo, Soda, Bertolini, Bettinelli, Rancati, Stancanello, Rampado, Pirrone and Drigo. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Michele Avanzo, bWF2YW56b0Bjcm8uaXQ=