Radiomics for Predicting Response of Neoadjuvant Chemotherapy in Nasopharyngeal Carcinoma: A Systematic Review and Meta-Analysis

Purpose This study examined the methodological quality of radiomics to predict the effectiveness of neoadjuvant chemotherapy in nasopharyngeal carcinoma (NPC). We performed a meta-analysis of radiomics studies evaluating the bias risk and treatment response estimation. Methods Our study was conducted through a literature review as per the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. We included radiomics-related papers, published prior to January 31, 2022, in our analysis to examine the effectiveness of neoadjuvant chemotherapy in NPC. The methodological quality was assessed using the radiomics quality score. The intra-class correlation coefficient (ICC) was employed to evaluate inter-reader reproducibility. The pooled area under the curve (AUC), pooled sensitivity, and pooled specificity were used to assess the ability of radiomics to predict response to neoadjuvant chemotherapy in NPC. Lastly, the Quality Assessment of Diagnostic Accuracy Studies technique was used to analyze the bias risk. Results A total of 12 studies were eligible for our systematic review, and 6 papers were included in our meta-analysis. The radiomics quality score was set from 7 to 21 (maximum score: 36). There was satisfactory ICC (ICC = 0.987, 95% CI: 0.957–0.996). The pooled sensitivity and specificity were 0.88 (95% CI: 0.71–0.95) and 0.82 (95% CI: 0.68–0.91), respectively. The overall AUC was 0.91 (95% CI: 0.88–0.93). Conclusion Prediction response of neoadjuvant chemotherapy in NPC using machine learning and radiomics is beneficial in improving standardization and methodological quality before applying it to clinical practice.


INTRODUCTION
Nasopharyngeal carcinoma (NPC) is a malignant head and neck cancer that occurs in the nasopharyngeal space and can spread to the base of the skull and other organs (1)(2)(3). Its anatomical location is relatively hidden, causing nearly 70% of NPC patients to be diagnosed at a locally advanced stage (4)(5)(6). The pathological subtypes of nasopharyngeal tumors mostly include poorly differentiated or undifferentiated squamous cell carcinomas, which are more sensitive to chemoradiotherapy (7)(8)(9). Therefore, definitive concurrent chemoradiotherapy has become the standard of care for NPC patients with locally advanced diseases (10,11). However, the efficacy of neoadjuvant chemotherapy has been shown to vary greatly in clinical practice, and approximately 30% of patients will develop chemoradiotherapy-related adverse events (12)(13)(14)(15).
Radiomics is a highly efficient extraction feature that obtains massive amounts of data from medical images. It transforms imaging data into a high-resolution mineable data space using automated or semiautomated analysis methods (16)(17)(18). Given its precise and systematic nature, radiomics can retrieve data that enable the detection of minimal lesions and the prediction of treatment outcomes (19)(20)(21)(22)(23)(24). As a result, radiomics is widely used in the study of NPC, and there is huge interest in employing radiomics to predict neoadjuvant chemotherapy efficacy in NPC patients. This information can assist physicians in selecting an optimal scheme and in achieving the maximal anticancer effect. Nevertheless, radiologic data analysis is highly reliant on the subjective interpretation of skilled radiologists. The quantitative data and autonomous imaging markers can serve as an adjunct to expert clinical opinion, thus increasing the prognostic precision (25,26).
The purpose of this research was to evaluate the methodological quality and analyze the effectiveness of neoadjuvant chemotherapy in NPC among the published radiomics papers. We also performed a meta-analysis of relevant studies to predict the treatment response of neoadjuvant chemotherapy, using the radiomics method, in NPC.

Study Protocol and Literature Search
This study was conducted as per the Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Diagnostic Test Accuracy (PRISMA-DTA) guidelines (27). Four databases (Web of Science, PubMed, Embase, and Cochrane Library) were screened to select relevant articles published prior to January 31, 2022. The search terms included were as follows: (Nasopharyngeal Carcinoma OR Carcinomas, Nasopharyngeal) AND (Machine Learning OR Artificial Intelligence OR radiomics) AND (CT OR MRI OR Magnetic Resonance Imaging). Please refer to the Supplementary Material for more details on the medical subject heading (MESH terms).
Two independent researchers screened article titles and abstracts to determine inclusion in this study. Case reports, non-original publications, and research on topics of interest other than the effectiveness of neoadjuvant chemotherapy in NPC were excluded. To further evaluate relevant articles, the full texts of articles were retrieved and read to determine eligibility for analysis. The reference list of included papers was also reviewed for potential eligible inclusion. The types of images included in our study included MRI, CT, and PET.

Data Collection
The main endpoints were extracted and adjusted to the largest area under the curve (AUC) in the verification dataset and also prioritized external validation datasets. Among the articles with no external verification dataset, the internal verification dataset (i.e., the test set) was employed. In the absence of an internal verification dataset, the validation set from the training dataset (e.g., leave-one-cross-validation, fivefold cross-validation, and tenfold cross-validation) was employed. The collected models contained radiologically relevant characteristics and sometimes contained characteristics, such as clinical information, pathological types, radiotherapy dose, region of interest (ROI), and imaging features extracted.

Study Evaluation
The radiomics quality score (RQS) assessed the methodological quality of eligible publications, and the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) determined the bias risk (28)(29)(30).
RQS assesses an investigation's methodological quality by examining protocols, images and segmentation reproducibility, feature reduction and verification, biological verification, clinical application, and model performance, with enhanced evidence and open science (28). The detailed RQS report is provided in the Supplementary Material. Overall, 16 items were included in the Abbreviations: AUC, area under curve; CR, complete response; ICC, intra-class correlation coefficient; NPC, nasopharyngeal carcinoma; OR, odds ratio; PD, progressive disease; PR, partial response; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses; PRISMA-DTA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Diagnostic Test Accuracy; QUADAS-2, Quality Assessment of Diagnostic Accuracy Studies; RECIST, Response Evaluation Criteria in Solid Tumors; REML, restricted maximum likelihood; RQS, radiomics quality score; SD, stable disease; SROC, summary receiver operating characteristic. RQS, with scores ranging from −8 to 36. The RQS scores were then converted to percentages, whereby −8 to 0 was 0% and 36 was 100% (28). Two experienced physicians independently scored the RQS of eligible articles.
QUADAS-2 evaluates the bias risk in varying domains ("Patient Selection", "Index Test", "Reference Standard", and "Flow and Timing") and can be customized to a particular study question. The bias risk for each included study was determined by the QUADAS item of Review Manager 5.4 in order to determine the quality of diagnostic articles (31).

Meta-Analysis
A meta-analysis of investigations related to the prediction of the treatment response of neoadjuvant chemotherapy in NPC patients was further performed. The data were retrieved by 2 independent reviewers. The internal validity was assessed by a third reviewer. Only studies that provided a two-by-two contingency table or enough data to reconstruct such a table were eligible for analysis. In cases where multiple models were presented, only models with the largest AUC were chosen in our analysis.

Statistical Analysis
Random-effects meta-analyses were conducted with the restricted maximum likelihood (REML) and presented as a log odds ratio (OR). The threshold effect was determined by calculating the sensitivity and specificity of Spearman's correlation coefficients. Forest plots and summary receiver operating characteristic (SROC) curve were generated. The pooled AUC, sensitivity, and specificity were used to assess the ability of radiomics to predict the treatment response of neoadjuvant chemotherapy in NPC patients. A funnel plot assessed publication bias. Cochran's Q test and I 2 score evaluated heterogeneity among eligible studies. An I 2 value of 0%-25% meant unremarkable heterogeneity, 25%-50% meant reduced heterogeneity, 50%-75% meant moderate heterogeneity, and >75% meant high heterogeneity (32).

Literature Search
We initially identified 613 relevant articles; 317 articles were considered for careful evaluation after the elimination of duplicate publications. Upon screening of the titles and abstracts, 18 relevant articles were extracted for further analysis. Four articles that did not contain a radiomics-based model and two conference abstracts were excluded from the analysis. A total of twelve articles that used radiomics-based prediction models were selected for the final systematic review (33)(34)(35)(36)(37)(38)(39)(40)(41)(42)(43)(44). Five of the articles examined survival analysis, and seven articles examined the prediction of treatment response. One article that predicted treatment response did not provide enough information to reconstruct a contingency table and calculate the overall outcome (43). Therefore, six articles were included in our meta-analysis. Our PRISMA flowchart is presented in Figure 1. The detailed information on all eligible publications is provided in Table 1. We summarized detailed information about the selected articles, such as institution, study duration, and type of radiomics features used. The detailed summary table is available in Tables S4 and S5.

Evaluation Criteria for Neoadjuvant Chemotherapy
The response evaluation of neoadjuvant chemotherapy in all included studies was based on the Response Evaluation Criteria in Solid Tumors 1.1 (RECIST 1.1) (45). Complete response (CR) and partial response (PR) were defined as response to treatment, while stable disease (SD) and progressive disease (PD) were defined as no response to treatment.

Study Evaluation
The RQS scores, ranging from 7 to 21 (maximum score: 36), are summarized in Table 2. The publication with the highest RQS percentage was 58.3%. The intra-class correlation coefficient (ICC) between independent reviewers who assessed the publications was 0.987 (95% CI: 0.957-0.996, p < 0.001), which showed excellent reproducibility among readers. The RQS scores examined by the two readers are presented in the Supplementary Material. Elevated intra-class association represented the high reliability of quality assessment. Lastly, reviewers reassessed any disagreements until a consensus was reached. The bias risk, as assessed by QUADAS-2, is presented in Figure 2. The publications with high, unclear, or low bias risk in the four domains of patient selection, index test, reference standard, and flow and timing were 0, 4, and 2, respectively. Particularly, three publications failed to present a clear report of the patient selection process. Therefore, they received an unclear bias risk in the patient selection domain (34,38,42). One study received an unclear bias risk in the index test domain (33). Three studies received an unclear bias risk in the flow and timing domain (33,34,38). All studies in the meta-analysis displayed relatively reduced concerns regarding applicability in the three domains (patient selection, index test, and reference standard).

Meta-Analysis
Seven, out of twelve, selected systematic studies discussed the use of radiomics in predicting the treatment response of neoadjuvant chemotherapy. Only six studies provided sufficient data to allow the reconstruction of a contingency table to compute an overall outcome. Hence, only six studies were included in the meta-analysis.
Spearman's correlation analysis revealed no threshold effect (r = 0.486, p = 0.3556). The SROC curve, pooled AUC, pooled sensitivity, and pooled specificity were used to assess the ability of radiomics to predict the response of neoadjuvant chemotherapy in NPC patients. Based on our data analysis, the pooled sensitivity and specificity were 0.88 (95% CI: 0.71-0.95) and 0.82 (95% CI: 0.68-0.91), respectively, as evidenced by the corresponding forest plots in Figure 3. The pooled AUC was 0.91 (95% CI: 0.88-0.93). Cochran's Q was 29.16 (p < 0.01), and the I 2 score was 85.8%, which represented a high level of heterogeneity within eligible studies with statistically significant heterogeneity. Figure 4 depicts the forest plot of the treatment outcome, computed as log OR. The log OR of the radiomics model predicting the neoadjuvant chemotherapy treatment response in NPC patients was 0.31 (95% CI: -1.58-2.21). The SROC curve is provided in Figure 5. The funnel plot correlating the outcome to standard error is presented in Figure 6. Given that we had less than 10 eligible articles in our meta-analysis, Egger's test was not applicable, as suggested by the Cochrane guidelines (46).

DISCUSSION
Radiomics has excellent prospects in multiple applications and can potentially aid in retrieving more quantitative data from standard medical images (47). In recent years, radiomics has developed rapidly in NPC research. However, despite ongoing FIGURE 2 | Assessment of the methodological quality of publications included in the meta-analysis, based on the bias risk and applicability using the QUADAS-2 tool. Green, yellow, and red circles denote low, unclear, and high bias risks, respectively.   efforts to standardize radiomics extraction features and analysis, their usage outside research is not yet justified (48). We found several articles based on radiomics to predict response to neoadjuvant chemotherapy in NPC patients, mostly over the last 3 years. This suggested that the use of radiomics in neoadjuvant chemotherapy for NPC patients is novel and remains groundbreaking. Based on our analysis, the characteristics of radiomics investigations were similar among all eligible publications. First, the ROI was manually segmented by two radiologists. Second, the radiomics features were extracted, and relevant features were selected. Third, a model predicting neoadjuvant chemotherapy effectiveness in treating NPC was constructed and evaluated. The texture features were deemed as the most frequent type of radiomics features in the twelve selected articles, and the detailed information is presented in Table S4.
Ten of the twelve articles employed texture features in their highest AUC models. The wavelet features were deemed as a frequent occurrence, and others included first-order features and shape features. During the prediction of neoadjuvant   chemotherapy efficacy in NPC, radiomics features including texture, wavelet, first order, and other features extracted from images by artificial intelligence algorithms were able to show a lot of hidden information. With an increasing number of radiomics studies, several studies also revealed that textural features could provide additional predictive information (49)(50)(51)(52)(53). This systematic analysis found that the Gray Level Run Length Matrix features, the Gray Level Size Zone Matrix features, and the Gray Level Co-occurrence Matrix features are more frequently used. The textural features were shown to provide good results in predicting the efficacy of neoadjuvant chemotherapy treatment for NPC. One possible reason is that texture features contain information related to the efficacy of neoadjuvant chemotherapy treatment. The advent of radiomics has made great contributions to overcoming limitations of user-dependent interpretation, thus assisting physicians in solving clinical problems. However, it was undeniable that the quality of our current research on radiomics is uneven. RQS is a common method for assessing the quality of radiomics studies and has been shown to accurately evaluate the methodological quality of radiomics studies. This is essential for the critical appraisal of a massive amount of research articles and prioritizing the verification of high-quality data. Since the first RQS application produced certain variations in inter-rater agreements (54), our independent RQS scoring was conducted by two independent readers experienced in radiomics. This way, we achieved a good level of agreement in terms of overall rating (ICC=0.987) and all scoring elements. The ICC, corresponding to each score category, was greater than 0.75. The RQS of our eligible studies were between 7 and 21 points, with a maximum of 36. But all eligible studies were retrospective in nature; therefore, 7 points was lost. We recommend future prospective studies to obtain higher-quality evidence. Moreover, none of the studies we analyzed conducted a cost-benefit analysis, and no phantom investigations were performed in terms of scan images. These deficiencies in research should be resolved in future radiomics research.
Our meta-analysis examined the prediction accuracy of neoadjuvant chemotherapy efficacy in NPC patients, based on radiomics. The SROC curve, obtained from the meta-analysis, is a ROC curve drawn from the OR of different radiomics studies. We demonstrated an enhanced prediction with a pooled AUC of 0.91. Our pooled sensitivity and pooled specificity reached 0.88 and 0.82, respectively. In terms of the publications that were eligible for meta-analysis, our QUADAS-2 assessment revealed a reduced bias risk while highlighting some critical matters. Particularly, three articles exhibited incomplete reporting of the inclusion-exclusion criteria, which can inadvertently introduce bias in the patient selection process (34,38,42). Moreover, one study received an unclear bias risk in the index test domain (33), due to the low number of features analyzed to the point of potential bias. In addition, three studies received an unclear bias risk in the flow and timing domain. Among them, one study failed to report the neoadjuvant chemotherapy duration (38). The remaining two studies showed less standardized processes (33,34), and neither study employed a test set to validate the radiomics model. One study (33) employed a leave-one-out cross-validation method to evaluate the model, and another (34) used the bootstrapvalidated model. Although internal validation in the training set can evaluate the performance of the radiomics model, this validation method may have introduced bias. All these concerns are sources of possible bias and should be clearly stated to eliminate bias.
The limitations of our work include the following. First, all studies were retrospective, and no prospective radiomics studies were found. Second, the radiomics features may have been affected by imaging technology. In the future, multicenter prospective investigations should be conducted to fully examine the predictability of radiomics studies (55). Third, RQS is a purely methodological scoring system that does not account for alterations in the study aim. Fourth, our sample size was relatively low, and the included studies were all from China. Fifth, although the QUADAS-2 assessment provided some unclear bias risks, no high bias sources were found. Moreover, being a qualitative score, the QUADAS-2 interpretation is not easily interpretable. Given our small sample size, our publication bias assessment is open to question. Sixth, we noted a high study heterogeneity, but this is typically common among machine learning meta-analyses and diagnostic meta-analyses (56)(57)(58)(59).

CONCLUSION
Radiomics studies investigating the efficacy of neoadjuvant chemotherapy in NPC patients demonstrated promising results. We, therefore, recommend properly designed prospective trials in the future, including the validation and standardization of methodological data analysis.

DATA AVAILABILITY STATEMENT
The original contributions presented in the study are included in the article/Supplementary Material. Further inquiries can be directed to the corresponding authors.