Radiomic biomarkers of locoregional recurrence: prognostic insights from oral cavity squamous cell carcinoma preoperative CT scans

Abstract

Introduction:

This study aimed to identify CT-based imaging biomarkers for locoregional recurrence (LR) in Oral Cavity Squamous Cell Carcinoma (OSCC) patients.

Methods:

Computed tomography scans were collected from 78 patients with OSCC who underwent surgical treatment at a single medical center. We extracted 1,092 radiomic features from gross tumor volume in each patient’s pre-treatment CT. Clinical characteristics were also obtained, including race, sex, age, tobacco and alcohol use, tumor staging, and treatment modality. A feature selection algorithm was used to eliminate the most redundant features, followed by a selection of the best subset of the Logistic regression model (LRM). The best LRM model was determined based on the best prediction accuracy in terms of the area under Receiver operating characteristic curve. Finally, significant radiomic features in the final LRM model were identified as imaging biomarkers.

Results and discussion:

Two radiomics biomarkers, Large Dependence Emphasis (LDE) of the Gray Level Dependence Matrix (GLDM) and Long Run Emphasis (LRE) of the Gray Level Run Length Matrix (GLRLM) of the 3D Laplacian of Gaussian (LoG σ=3), have demonstrated the capability to preoperatively distinguish patients with and without LR, exhibiting exceptional testing specificity (1.00) and sensitivity (0.82). The group with LRE > 2.99 showed a 3-year recurrence-free survival rate of 0.81, in contrast to 0.49 for the group with LRE ≤ 2.99. Similarly, the group with LDE > 120 showed a rate of 0.82, compared to 0.49 for the group with LDE ≤ 120. These biomarkers broaden our understanding of using radiomics to predict OSCC progression, enabling personalized treatment plans to enhance patient survival.

1 Introduction

Oral Cavity Squamous Cell Carcinoma (OSCC) is the most common malignancy in the head and neck region and is characterized by a poor prognosis (1). Surgery is the primary treatment of OSCC, followed by cisplatin-based chemotherapy and/or radiotherapy depending on pathologic features and individualized risk of recurrence. Regional recurrence is the most common cause of failure after treatment of oral carcinoma (2, 3). Despite advancements in surgical techniques and adjuvant therapies, the 5-year overall survival rate hovers between 45-50%, contingent upon the stage and metastasis status of the disease (4, 5). Locoregional recurrence (LR), as indicated by prior studies (6, 7), represents a significant clinical challenge, with some patients cohorts demonstrating extremely high rates of LR even following surgery and appropriate adjuvant therapy (8). Given that disease recurrence is devastating for patients, and adjuvant therapies are associated with significant economic and quality of life detriment, identification of patients at higher risk of LR who would benefit most from adjuvant treatment is paramount. It is, therefore, relevant to identify patients who are at a higher risk of locoregional recurrence before their primary surgery to guide treatment plans and increase the therapeutic window. By utilizing noninvasive imaging information and cutting-edge machine learning algorithms, post-treatment failure can be better screened, enabling medical professionals to tailor treatment plans accordingly.

Histopathologic factors are used for OSCC diagnosis and prognosis staging evaluation (9). Studies (10–15) reported that tumor size, depth of invasion (DOI), stromal, vascular, and nerve invasion are significantly different between the groups with and without metastasis. The dysregulation of specific miRNAs in OSCC, such as miRNA-184 (16), miR-31 (17), and miR-27b (18), are implicated in malignant transformation and disease progression. Other proteins and peptides, such as Leukotriene A4 hydrolase (LTA4H) and its peptide, Pep8_LTA4H, among other proteins and peptides, may distinguish individuals with metastasis (N+) from individuals metastasis-free (N0) (19). Studies (20–22) implicated that the amplification of CCND1 and overexpression of cyclin D1 are significantly correlated with OSCC metastasis. Soluble factors, such as IL- ,TNF- , and MIP- , that can be detected in saliva, may also play a significant role in detecting metastasis (23). The other study (24) reveals an association between primary site recurrence and a high ratio of ITGA3/CD9. Elevated levels of squamous cell carcinoma antigen (SCC-Ag) in serum are significantly associated with tumor progression (25). While these parameters evaluated in pre-clinical settings hold promise in enhancing disease detection, prognosis, and personalized treatment, those findings need to be confirmed by larger and more rigorous studies. One of the limitations of histopathologic biopsy is that it may not capture the full heterogeneity of the tumor due to sampling bias (26). Furthermore, factors such as DOI are only available on the resection specimen. The extraction and analysis of biomarkers, such as H&E staining, tissue microarray, and sequencing, can be technically complex and expensive, requiring specialized resources, which may restrict their practicality in specific circumstances. Additionally, challenges with reproducibility and standardization across laboratories and the potential for false positives and negatives further complicate their practicality (27, 28). Furthermore, validating a biomolecule-based assay, from its initial discovery to clinical implementation, is often arduous and lengthy. A significant number of potential markers prove to be ineffective across various populations (29). Additionally, it is crucial to reduce the overall processing time to avoid LR in patients who require adjuvant therapy. Also, while some biomarkers may indicate the presence of a disease, they might not offer actionable information for treatment plans, thereby restricting their practical clinical use (30, 31). Lastly, employing genetic and other biomolecular markers raises ethical, legal, and societal concerns (32). This is a primary reason these biomarkers have not been introduced in clinical settings and lack FDA approval.

Imaging-based biomarkers have been investigated for different modalities, such as Computed Tomography (CT), Positron Emission Tomography (PET), and Magnetic Resonance Imaging (MRI), by extracting quantitative imaging features known as radiomics features. In contrast to biomolecule-based assays, imaging techniques are non-invasive. The imaging information is readily available from routine diagnostic scans without incurring additional costs. Moreover, imaging provides unique 3D information about neoplasm. These radiomics features can be leveraged to develop predictive models for survival and treatment failure (33–42). The rationale behind this approach is that these images capture crucial information about the neoplasm phenotype and microenvironment (43). In fact, the American College of Radiology has developed a standardized Neck Imaging Reporting and Data System (NI-RADS) (44) to manage and surveil the posttreatment course. Studies (45, 46) demonstrated a strong association between NI-RADS category and treatment failure in HNSCC patients. Over recent decades, imaging factors have demonstrated their capacity to furnish accurate prognostic information for posttreatment recurrence screening. A study (47) found a significant association between PET/CT radiomic features and Head and Neck locoregional recurrence. Our pilot study (48) demonstrated two potential Radiomic overall survival biomarkers. However, the identification of non-invasive factors for 2-year locoregional recurrence after primary surgery in OSCC patients remains lacking. The distinction between our study and similar research lies in the emphasis on the susceptibility/risk associated with the biomarker, specifically an increased likelihood of developing locoregional recurrence (LR) within 2 years post-surgery. Consequently, we focus more on specificity and sensitivity to minimize the incidence of false positives and false negatives.

This study aimed to identify CT-based imaging risk factors for locoregional recurrence in patients with OSCC at an academic health network serving a diverse population, which enabled the development of machine learning classifiers that could accurately distinguish patients with locoregional recurrence from those without prior to treatment. A retrospective study design was used, with high-dimensional radiomics, pathological, and clinical information collected from this diverse cohort of OSCC cases. The primary endpoint was 2-year locoregional recurrence (defined as locoregional recurrence occurring within 2 years of surgery). The findings of this study lay the foundation for the implementation of pre-treatment screening for LR and risk assessment using non-invasive risk factors in this diverse patient population, which could ultimately impact the management of high-risk OSCC patients by helping physicians customize treatment planning and reduce the chance of distant metastasis.

2 Materials and methods

2.1 Data preparation and overall workflow

The workflow outlining our approach is illustrated in Figure 1. In this workflow, the neoplasm volume serves as the region of interest (ROI) from which all radiomics features are computed. The contouring of the ROI was performed manually by experienced Radiation Oncologists, not directly involved in the study, using the Varian Medical System Eclipse software environment. These features underwent a selection process to minimize redundancy and were combined with clinical data. A logistic regression model, optimized via five-repeated 10-fold cross-validation, was then applied. The model’s predictive performance was evaluated using the Area Under the ROC Curve (AUC). All statistical analyses were performed using R programming language, with a significance level (alpha) set at 0.05 for all tests.

Figure 1

2.2 Radiomic feature acquisition and extraction

Uniformity in voxel sizes is essential for precise and dependable feature calculations in radiomics (49). Given the original CT scan resolutions varied significantly, from 0.3×0.3×0.5 mm to 1.3×1.3×5 mm, we resampled the voxel values to a uniform resolution of 1×1×1mm using interpolation. For this purpose, we employed the Bspline algorithm (50), suitable for smoothly adjusting voxel values. Concurrently, to maintain the binary nature of the tumor masks, we applied the nearest neighbor interpolation algorithm (51, 52), which assigns the mask label of the closest voxel in the original grid to each voxel in the interpolation grid. Another critical step employed was normalization, computed by , where x represents voxel intensity, is the average intensity, and is the standard deviation of intensity. As demonstrated in a previous study (49), this normalization minimizes variance and boosts feature robustness of radiomic features, especially against different discretization levels, ensured normalized intensities generally spanned from [-3, 3] after outlier exclusion (53, 54). These intensities were then scaled to a range of approximately [-300, 300]. Moreover, to capture detailed textural information, we discretized the intensities within the ROI using a uniform bin width of 5, starting from a normalized minimum HU value of 0. We chose a bin width of 5 to ensure an adequate number of bins (between 1 and 400), allowing for the capture of more detailed textural information (55). This discretization assigns new values to each voxel according to the formula: floor((original intensity)/5) + 1. This method not only suppresses noise but also enhances the robustness of radiomic features by smoothing out minor variations.

Medical images provide insights into the phenotypic traits of neoplasms. These images typically contain data from tens of thousands of voxel intensities per neoplasm, leading to a scenario where the number of features (p) greatly exceeds the sample size (n). In our study, we extracted features from each image set using the PyRadiomics library in Python. According to the Imaging Biomarker Standardization Initiative (IBSI) (56), we extracted features across six categories: shape, first-order statistics, gray level co-occurrence matrix (GLCM), gray level run length matrix (GLRLM) (57), grey level size zone matrix (GLSZM) (58), gray level dependence matrix (GLDM) (59), and neighborhood grey tone difference matrix (NGTDM) (60). Additional calculations were performed on images processed with wavelet, Laplacian of Gaussian (LoG), square, square root, logarithm, exponential, and gradient filters, culminating in 1,092 features.

2.3 Feature selection and modeling

Logistic Regression Models (LRM) were used in assessing the features of discriminative power. Algorithms are available in the R stats, glmulti package. A logistic model can be mathematically described as follows: , where is the probability of recurrence, and the coefficient vector for the independent variables (61). The guidelines outlined in (62) recommend that the number of predictors used in fitting LRM should not exceed 10% of the events in the sample. In our cohort, with 10% of events in the training sample, the optimal number of predictors for model fitting should be 2 to 3, as 10% of the total recurrence equals 2.1. Thus, we aimed to limit the final Logistic Regression Model (LRM) to a maximum of 3 degrees of freedom. We used the Best Subset Selection (BSS) Modeling strategy to identify the most effective LRM based on validation performance to achieve this. BSS, known for its efficiency in finding the most parsimonious model, outperforms methods like stepwise selection and Lasso, although its high computational demand is a limitation. For instance, fitting LRMs with 2, 3, or 4 degrees of freedom using 17 radiomics and six clinical features requires BSS to estimate a minimum of 41,262 coefficients, making exhaustive evaluation impractical. Therefore, we reduced the number of input variables before employing BSS.

Radiomics data often faces the challenge of high multicollinearity, where variables are highly correlated, affecting the significance of individual variables in the model. For example, sphericity, minor axis length, and elongation show strong multicollinearity. Multicollinearity can lead to the phenomenon where a variable is not deemed significant when correlated features are also present in the model. Figure 2 uses a color scheme where white represents no correlation, blue represents a perfect negative correlation, and red represents a perfect positive correlation. The heatmap illustrates the correlation coefficients prior to the feature selection process, revealing the initial relationships between features. The heatmap revealed numerous red and blue shades, indicating strong positive and negative correlations, respectively, among the data. There is now a substantial body of research on mitigating multicollinearity, such as Principal Component Analysis (PCA), Sparse PCA (63), and Kernel PCA (KPCA) (64). We employed Recursive Feature Elimination (RFE) (65), an iterative procedure to refine the input data for BSS LRM. Using RFE with repeated 10-fold cross-validation (Table 1), we narrowed down from 1,092 radiomic features to a subset of eight active features (Figure 3). We then fitted degree-2 LRMs, considering all combinations of these eight features across 5,000 data shuffles. The models were trained on 63 samples (80% of the cohort) using 10-fold cross-validation and evaluated based on AUC. The remaining 15 samples (20% of the cohort) were used for prognostic validation. We also used the ROC to visualize the classifiers’ performance. Figure 4 shows that the 3,480 models out of 5,000 data shufflings unveiled the most distinctive factor among the extensive array of 1,092 radiomic features: specifically, Large Dependence Emphasis (LDE) of the Gray Level Dependence Matrix (GLDM) and Long Run Emphasis (LRE) of the Gray Level Run Length Matrix (GLRLM) of the 3D Laplacian of Gaussian (LoG _σ=3) filtered ROIs (Figures 5, 6). Statistical evidence was quantified by the p-value of the hypotheses’ tests. If the p-value is less than α = 0.05, then the null hypothesis will be rejected in favor of the alternative hypothesis.

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

2.4 Assessment of the prognosis

We stratified the cohort into high and low LRE and LDE subgroups according to the threshold selected by using respective median. Kaplan-Meier curve analyses were conducted to assess the impacts of the biomarkers on RFS. RFS was defined as the time from surgery to locoregional recurrence.

2.5 Constructing nomogram

The primary end-point of the analysis was the time to peritoneal locoregional recurrence. The follow-up duration to peritoneal LR was calculated from the date of surgery to the date when peritoneal LR was diagnosed or to the last follow-up, and information about the survival status and recurrence type was also documented. Finally, a radiomic nomogram was constructed. A model containing both radiomic and clinical factors was also constructed for comparison.

2.6 Code availability

The code used in this study is available at https://doi.org/10.5281/zenodo.10460030.

3 Results

3.1 Oral cavity squamous cell carcinoma cohort features

This retrospective biomarker analysis examines a group of oral cavity squamous cell carcinoma (OSCC) patients who underwent surgical/curative/elective neck/selective neck resection at the institution between 2006 and 2017. The study involved 78 patients, with 21 experiencing locoregional recurrence (LR), while 57 remained disease-free within a 2-year period after the end of the initial treatment course. Demographic and clinicopathological features of patients are detailed in Table 2. The mean age at the initial surgery was 60, ranging from 30 to 98 years. The median follow-up time for recurrence-free survival (RFS) was 56.2 months. A locoregional recurrence was defined as a positive biopsy in the primary site or the cervical lymphatic region after treatment. We collected six clinical characteristics of interest, including age, gender, tobacco usage, alcohol consumption, T-stage, N-stage, and race. All patients were in the first 2 years of follow-up after surgery. Patients were categorized into four T stages (1, 2, 3, and 4) based on the size and extent of the primary tumor. Smoking and alcohol status were self-reported and coded as 1 for Yes and 2 for No. The missing values for smoking and alcohol status were hard coded as 3 due to their substantial representation within the dataset. Smoking status revealed that 60% of the total cohort were smokers, with this number rising to 72% in the LR subgroup. For alcohol consumption, 40% of the total group reported alcohol use, compared to 48% in the LR subgroup. All patients underwent surgery treatment as primary treatment, along with chemoradiotherapy (CRT) or radiotherapy (RT). The endpoint in this study was 2-year LR status, defined as whether an LR happened within 2 years after curative treatment. Here, the proportion increased in the LR subgroup (17% in the total group, 29% in LR), indicating a higher prevalence of this intermediate stage in the LR subgroup. T3 tumors constituted 17% of the cases. The proportion of T3 tumors increases to 29% in the LR subgroup. T4 tumors, which represent the most advanced stage of tumor size and extent, accounted for 23% of the total cohort. The representation of T4 tumors is notably higher in the LR subgroup, constituting 33%. In total, T3 and T4 stages comprise only 31% of the entire group, in contrast to 62% in the LR subgroup.

3.2 Radiomic factors selection and validation

A preliminary feature selection algorithm identified 8 radiomic factors of discriminative power in LR depicted in Figure 7. To ascertain the independence of these two radiomic factors from clinical factors and their potential as clinical alternatives, we investigated their interaction with clinical factors, smoking, alcohol I(ETOH), N stage, and T stage. Logistic regression modeling then incorporated the radiomic features with demographic and clinicopathological characteristics to define the final radiomic risk factors. In this comparative analysis of five logistic regression models in Table 3, denoted as Models 1 through 5, we have assessed their performance based on a range of statistical metrics. The Akaike Information Criterion (AIC) is employed as a model selection criterion. At the same time, accuracy (ACC), area under the receiver operating characteristic curve (AUC), sensitivities (Sens), and specificities (Spec) are utilized to evaluate the models’ predictive capabilities. It is evident that the 2nd model (clinical-only) exhibits the highest AIC of 80, suggesting a worse fit to the training data (80%) compared to the other models. When considering the training measurements, the model demonstrates the lowest accuracy (0.75) and AUC values (0.46). In regard to the testing measurements on the held-out 20% data, the 2nd model shows the lowest accuracy (0.6) and AUC (0.67) among all models, showcasing its deficiency in generalization. Moreover, the model consistently maintains an unbalanced sensitivity (0.45) and specificity (1), highlighting its inability to make accurate predictions while minimizing false positives and false negatives. These findings collectively underscore the suboptimal performance of the 2nd model and establish it as the least favorable choice when contrasted with the other models in this analysis.

Figure 7

Numerous studies have identified smoking and drinking as risk factors for OSCC patients. Further analysis indicated that including clinical factors didn’t significantly enhance the model’s explanatory power (based on deviance analysis in Table 4 via chi-square test, P=0.43). Table 4 presents the Analysis of Deviance results for eight pairs of nested model comparisons, testing the null hypothesis that additional factors have no effects on outcome. Our analysis yielded robust evidence (via χ2-test, p-values< 0.0001) supporting the significance of adding radiomic factors in each pair. Augmenting radiomics to include Smoke, ETOH, and T (the first pair) decreased the deviance by 27.68, indicating a significantly better fit of the larger model to the data. The larger model’s AUC showed a noteworthy improvement over the smaller model, as depicted in Figure 8.

Figure 8

3.3 Risk stratification and prognostic ability

Patients were stratified into high- and low-end groups for recurrence-free survival based on the median value of two factors (Figure 9). The Kaplan-Meier curves provide compelling evidence of a significant difference in RFS between the high- and low-end groups (Log-rank p< 0.05). Furthermore, the AUCs of logistic regression, incorporating radiomic factors (0.93), corroborate the significant enhancement in discriminative power when compared to clinical factors-only models (Figure 8). Nomograms were constructed with radiomic and clinical factors respectively in Figure 10. Notably, the addition of clinical features in the full model demonstrates minimal influence on the predicted RFS probability when compared to the radiomic feature-only model.

Figure 9

Figure 10

3.4 Radiomic features uncover hidden textural patterns

Tumor heterogeneity is widely acknowledged as a significant factor associated with tumor progression. The quantification of tumor heterogeneity has assumed a pivotal role in pathological assessments. Radiomic texture analysis presents distinct advantages, such as non-invasiveness and cost-effectiveness, compared to conventional pathological evaluations. Multiple studies (66–69) have underscored the prognostic potential of GLDM (Gray Level Dependence Matrix) and GLRLM (Gray Level Run Length Matrix) features in the evaluation of tumor progression. These features, GLDM and GLRLM, quantify the degree of local variation within an image (70). LRE (Long Run Emphasis) serves as a metric for assessing the distribution of long run lengths, with higher values indicative of longer run lengths and coarser structural textures. Conversely, LDE (Long Dependence Emphasis) quantifies the distribution of large dependencies, with elevated values denoting larger dependencies and more homogeneous textures. In light of the findings presented in Table 5, it is noteworthy that, when maintaining LDE at a constant value, each unit increment in LRE corresponds to a 79% (1-0.21) decrease in the odds of recurrence as opposed to non-recurrence. Conversely, when keeping LRE at a fixed value, every unit increasing in LDE results in a 5.8% increase in the odds of recurrence. It is essential to recognize that the estimate for the intercept represents the log odds of a patient with hypothetical zero values for LRE and LDE experiencing recurrence, which is calculated to be 0.095. This observation underscores a robust association between these two radiomic factors and the risk of locoregional recurrence.

Figure 11 compares two sets of images processed using the Laplacian of Gaussian (LoG) filter. The first three rows depict LoG-filtered results on an Oral Squamous Cell Carcinoma (OSCC) occurring in the tongue area, with varying sigma (σ) values. The last three rows display results for a non-OSCC area of the tongue. Lower sigma values highlight finer structures, while higher sigma values accentuate larger clusters in the tissue (71). We observed a pattern in the OSCC images: with increasing sigma, there is a reduction in highlights (white regions) in the filtered images, contrary to the non-OSCC images, which maintain a consistent level of highlights. This suggests that squamous cells in OSCC may be more homogeneous than normal cells. Furthermore, we observed circular artifacts in the OSCC images when filtered with larger sigma (σ > 0.4mm), whereas the normal set presents relatively random structures. to normal tissues. These findings support a fundamental histopathological principle: tumor tissues typically exhibit a more anaplastic and infiltrative pattern than normal tissues (72). This occurs because tumor cells grow unregulated and clonally, leading to the loss of normal differentiation and organization characteristic of healthy tissues.

Figure 11

4 Discussion

Several studies (57, 73, 74) have demonstrated the statistical significance of the discriminating ability of radiomic features. The Laplacian of an image highlights regions of rapid intensity change (75). LDE and LRE measure the distribution of low gray-level values, with a higher value indicating a greater concentration of low gray-level values in the tumor CT scan. SRLGLE measures the joint distribution of shorter run lengths with lower gray-level values despite the universal adoption of CT modality in OSCC diagnosis, automatic imaging prognostic evaluation is lacking and subjective. We present a fully automated prognostic evaluation tool to preoperatively detect locoregional failure in oral cavity cancer. The present study aimed to assess the prognostic capabilities of radiomic features in OSCC locoregional recurrence. Our findings demonstrate that analyzing radiomics from pre-treatment CT scans offers valuable insights into risk factors for locoregional failure and serves as prognostic biomarkers in this patient population. Non-invasive risk factors play a crucial role in personalizing treatment planning, particularly in OSCC, due to the involvement of critical neck surgeries. It is well-known that neck surgeries potentially significantly impact the quality of a patient’s life. Thoughtful treatment planning has the potential to mitigate the side effects of unnecessary neck surgery. Key findings of our study include two significant radiomic risk factors: Large Dependence Emphasis (LDE) of the Gray Level Dependence Matrix (GLDM) and Long Run Emphasis (LRE) of the Gray Level Run Length Matrix (GLRLM) of the 3D Laplacian of Gaussian (LoG σ=3) filtered ROIs.

The AUC showed a stable and approximate value of 0.8 with a sensitivity 0.8 and specificity of 0.8 at the optimal threshold, indicating good prognosis accuracy of the classifier. These results highlight the potential of radiomic features, as a biomarker indicator for treatment failure prognosis. Our study standardized voxel spacing in CT images across patients for precise feature calculation and applied gray-level normalization to enhance feature comparability. These steps are crucial for consistent radiomics analysis. Combining Correlation Analysis, Recursive Feature Selection, and Logistic Regression Best Subset Selection, our feature selection process effectively reduced feature space dimensionality while retaining critical prognostic information. This approach helps mitigate bias, overfitting, and multicollinearity in high-throughput data analysis.

The cost of missing a positive diagnosis (Type 2 error) is often higher than false alarms. On the contrary, since neck dissection significantly decreases the quality of life, reducing the false positive rate (Type 1 error) shall be necessary. In fact, study (76) demonstrate that ROC plots in the context of imbalanced datasets can be deceptive. Therefore, our modeling emphasis was placed on increasing sensitivity and specificity with due consideration to AUC. The threshold for positive event classification plays a pivotal role in predictive accuracy. While a threshold of 0.5 is commonly employed in default, this value is often suboptimal for practical applications in real-world studies, particularly in clinical settings where the distribution of positive cases may have an inherent prevalence, thereby elevating the risks of Type I and Type II errors. Both types of errors are of concern in the study since both overtreatment and undertreatment may lead to escalating healthcare costs and potential harm to patients. To mitigate these risks, we propose adopting a threshold that aligns with the natural prevalence of our cohort, specifically a value of 0.28 for this cohort, for final classification. This calibrated threshold aims to optimize two key metrics: high sensitivity, crucial for minimizing Type II errors and thereby maximizing the identification of LR, and high precision, vital for minimizing Type I errors to reduce false alarms. A number of studies (77, 78) have shown that the chance of an OSCC postoperative locoregional disease being diagnosed positive (Sensitivity) after surgery is only 29%. Our approach demonstrated a noteworthy testing AUC of 0.84, prioritizing both high sensitivity (0.82) and specificity (1). This significantly reduces Type I and Type II errors in post-treatment disease screening tests, effectively minimizing overtreatment and undertreatment.

Our study underscores radiomics’ promise in OSCC classification, yet it’s crucial to consider its limitations. The small sample size and the classification study’s nature might influence our model’s radiomics feature stability. For a low-biased, variance classification model with two effects, at least 20 events per training set are advisable, necessitating 27 events for a training set comprising 75% of the sample. This requirement could limit our model’s flexibility, potentially impacting the diagnostic capability of the radiomics. Moreover, our analysis only involved radiomics features from CT imaging. Future research should explore features from various imaging techniques, like CT and MRI, to heighten prediction precision. Notably, the observed correlation between certain radiomics features and overall survival hints that these features may mirror the tumors’ molecular traits. Upcoming studies should integrate genetic data, such as TP53 (13) mutations and P16 overexpression (28), with radiomics to more comprehensively characterize head and neck squamous cell carcinoma and offer a non-invasive, multimodal approach to OSCC outcome prediction. It is important to acknowledge the challenges posed by the sensitivity of the data involved. To our knowledge, there is no public dataset available that could be directly applied to our validation needs. As a result, we are actively seeking to collaborate with multiple institutions to gather data for validation purposes, aiming to mitigate the limitation and ensure the robustness of findings.

In conclusion, our study demonstrated the potential of radiomics as an effective tool to predict treatment response in OSCC patients. Incorporating radiomics analysis into clinical practice could improve decision support and enhance patient stratification, reducing both over-treatment and under-treatment to improve outcomes. Moreover, processing the ROI at the level of small tiles provides an additional non-invasive avenue for assessing the spatial heterogeneity within the tumor. The findings from the study pave the way for future investigations through a larger clinical trial to further evaluate the clinical efficacy of radiomics biomarkers for overall survival prediction for OSCC patients.

References

• 1

  da SilvaSDHierMMlynarekAKowalskiLPAlaoui-JamaliMA. Recurrent oral cancer: current and emerging therapeutic approaches. Front Pharmacol. (2012) 3:149. doi: 10.3389/fphar.2012.00149

  • CrossRef
  • Google Scholar

• 2

  YuenAPWWeiWIWongYMTangKC. Elective neck dissection versus observation in the treatment of early oral tongue carcinoma. Head Neck: J Sci Specialties Head Neck. (1997) 19:583–8. doi: 10.1002/(ISSN)1097-0347

  • CrossRef
  • Google Scholar

• 3

  ContreraKJHuangATShensonJATangCRobertsDMyersJNet al. Primary and recurrent regional metastases for lateralized oral cavity squamous cell carcinoma. Surg Oncol. (2022) 44:101804. doi: 10.1016/j.suronc.2022.101804

  • CrossRef
  • Google Scholar

• 4

  LicitraLLocatiLBossiP. Head and neck cancer. Ann Oncol. (2004) 15:iv267–73. doi: 10.1093/annonc/mdh937

  • CrossRef
  • Google Scholar

• 5

  DuprezFBerwoutsDDe NeveWBonteKBoterbergTDeronPet al. Distant metastases in head and neck cancer. Head Neck. (2017) 39:1733–43. doi: 10.1002/hed.v39.9

  • CrossRef
  • Google Scholar

• 6

  WoolgarJAet al. Survival, metastasis and recurrence of oral cancer in relation to pathological features. Ann R Coll Surgeons Engl. (1995) 77:325.

  • Google Scholar

• 7

  WoolgarJARogersSWestCRErringtonRDBrownJSVaughanED. Survival and patterns of recurrence in 200 oral cancer patients treated by radical surgery and neck dissection. Oral Oncol. (1999) 35:257–65. doi: 10.1016/S1368-8375(98)00113-4

  • CrossRef
  • Google Scholar

• 8

  LeemansCRTiwariRNautaJJWaalIVDSnowGB. Recurrence at the primary site in head and neck cancer and the significance of neck lymph node metastases as a prognostic factor. Cancer. (1994) 73:187–90. doi: 10.1002/(ISSN)1097-0142

  • CrossRef
  • Google Scholar

• 9

  FullerCCamilonRNguyenSJenningsJDayTGillespieMB. Adjunctive diagnostic techniques for oral lesions of unknown Malignant potential: Systematic review with meta-analysis. Head Neck. (2015) 37:755–62. doi: 10.1002/hed.v37.5

  • CrossRef
  • Google Scholar

• 10

  WoolgarJ. T2 carcinoma of the tongue: the histopathologist’s perspective. Br J Oral Maxillofac Surg. (1999) 37:187–93. doi: 10.1054/bjom.1999.0034

  • CrossRef
  • Google Scholar

• 11

  ShingakiSSuzukiINakajimaTKawasakiT. Evaluation of histopathologic parameters in predicting cervical lymph node metastasis of oral and oropharyngeal carcinomas. Oral Surg Oral Med Oral Pathol. (1988) 66:683–8. doi: 10.1016/0030-4220(88)90318-0

  • CrossRef
  • Google Scholar

• 12

  CrissmanJDLiuWGluckmanJLCummingsG. Prognostic value of histopathologic parameters in squamous cell carcinoma of the oropharynx. Cancer. (1984) 54:2995–3001. doi: 10.1002/(ISSN)1097-0142

  • CrossRef
  • Google Scholar

• 13

  FukanoHMatsuuraHHasegawaYNakamuraS. Depth of invasion as a predictive factor for cervical lymph node metastasis in tongue carcinoma. Head Neck: J Sci Specialties Head Neck. (1997) 19:205–10. doi: 10.1002/(ISSN)1097-0347

  • CrossRef
  • Google Scholar

• 14

  SpiroRHHuvosAGWongGYSpiroJDGneccoCAStrongEW. Predictive value of tumor thickness in squamous carcinoma confined to the tongue and floor of the mouth. Am J Surg. (1986) 152:345–50. doi: 10.1016/0002-9610(86)90302-8

  • CrossRef
  • Google Scholar

• 15

  JonesKRLodge-RigalRDReddickRLTudorGEShockleyWW. Prognostic factors in the recurrence of stage I and II squamous cell cancer of the oral cavity. Arch Otolaryngology–Head Neck Surg. (1992) 118:483–5. doi: 10.1001/archotol.1992.01880050029006

  • CrossRef
  • Google Scholar

• 16

  MeniniMDe GiovanniEBagnascoFDelucchiFPeraFBaldiD. Salivary micro-RNA and oral squamous cell carcinoma: A systematic review. J Pers Med. (2021) 11:101. doi: 10.3390/jpm11020101

  • CrossRef
  • Google Scholar

• 17

  LiuC-JLinS-CYangC-CChengH-WChangK-W. Exploiting salivary miR-31 as a clinical biomarker of oral squamous cell carcinoma. Head Neck. (2012) 34:219–24. doi: 10.1002/hed.21713

  • CrossRef
  • Google Scholar

• 18

  Momen-HeraviFTrachtenbergAKuoWChengY. Genomewide study of salivary microRNAs for detection of oral cancer. J Dental Res. (2014) 93:86S–93S. doi: 10.1177/0022034514531018

  • CrossRef
  • Google Scholar

• 19

  CarnielliCMMacedoCCDe RossiTGranatoDCRiveraCDominguesRRet al. Combining discovery and targeted proteomics reveals a prognostic signature in oral cancer. Nat Commun. (2018) 9:3598. doi: 10.1038/s41467-018-05696-2

  • CrossRef
  • Google Scholar

• 20

  GotoHKawanoKKobayashiISakaiHYanagisawaS. Expression of cyclin D1 and GSK-3β and their predictive value of prognosis in squamous cell carcinomas of the tongue. Oral Oncol. (2002) 38:549–56. doi: 10.1016/S1368-8375(01)00121-X

  • CrossRef
  • Google Scholar

• 21

  MyoKUzawaNMiyamotoRSonodaIYukiYAmagasaT. Cyclin D1 gene numerical aberration is a predictive marker for occult cervical lymph node metastasis in TNM Stage I and II squamous cell carcinoma of the oral cavity. Cancer: Interdiscip Int J Am Cancer Soc. (2005) 104:2709–16. doi: 10.1002/cncr.21491

  • CrossRef
  • Google Scholar

• 22

  ChienH-TChengS-DLiaoC-TWangH-MHuangS-F. Amplification of the EGFR and CCND1 are coordinated and play important roles in the progression of oral squamous cell carcinomas. Cancers. (2019) 11:760. doi: 10.3390/cancers11060760

  • CrossRef
  • Google Scholar

• 23

  LeeLTWongYKHsiaoHYWangYWChanMYChangKW. Evaluation of saliva and plasma cytokine biomarkers in patients with oral squamous cell carcinoma. Int J Oral Maxillofac Surg. (2018) 47:699–707. doi: 10.1016/j.ijom.2017.09.016

  • CrossRef
  • Google Scholar

• 24

  NagataMNomanAASuzukiKKuritaHOhnishiMOhyamaTet al. ITGA3 and ITGB4 expression biomarkers estimate the risks of locoregional and hematogenous dissemination of oral squamous cell carcinoma. BMC Cancer. (2013) 13:1–9. doi: 10.1186/1471-2407-13-410

  • CrossRef
  • Google Scholar

• 25

  KatoHTamaiKMoriokaHNagaiMNagayaTTorigoeT. Tumor-antigen TA-4 in the detection of recurrence in cervical squamous cell carcinoma. Cancer. (1984) 54:1544–6. doi: 10.1002/(ISSN)1097-0142

  • CrossRef
  • Google Scholar

• 26

  TakesRPBaatenburg de JongRJKeuningJHermansJSchuuringEVan KriekenHJ. Protein expression of cancer associated genes: biopsy material compared to resection material in laryngeal cancer. Anticancer Res. (1998) 18:4787–91.

  • Google Scholar

• 27

  BayramogluNKaakinenMEklundLHeikkilaJ. Towards virtual H&E staining of hyperspectral lung histology images using conditional generative adversarial networks, in: 2017 IEEE International Conference on Computer Vision Workshops (ICCVW). (Venice, Italy: IEEE International Conference on Computer Vision Workshops (ICCVW)). (2017). pp. 64–71.

  • Google Scholar

• 28

  RanaAYauneyGLoweAShahP. Computational histological staining and destaining of prostate core biopsy RGB images with generative adversarial neural networks, in: 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA). (Orlando, FL: IEEE International Conference on Machine Learning and Applications (ICMLA)). (2018). pp. 828–34.

  • Google Scholar

• 29

  RedstonMNoffsingerAKimAAkarcaFGRaraMStapletonDet al. Abnormal TP53 predicts risk of progression in patients with Barrett’s esophagus regardless of a diagnosis of dysplasia. Gastroenterology. (2022) 162:468–81. doi: 10.1053/j.gastro.2021.10.038

  • CrossRef
  • Google Scholar

• 30

  FlahertyKTGrayRChenALiSPattonDHamiltonSRet al. The molecular analysis for therapy choice (NCI-MATCH) trial: lessons for genomic trial design. JNCI: J Natl Cancer Institute. (2020) 112:1021–9. doi: 10.1093/jnci/djz245

  • CrossRef
  • Google Scholar

• 31

  BlucherASMillsGBTsangYH. Precision oncology for breast cancer through clinical trials. Clin Exp Metastasis. (2022) 39:71–8. doi: 10.1007/s10585-021-10092-0

  • CrossRef
  • Google Scholar

• 32

  PrudenteSDallapiccolaBPellegriniFDoriaATrischittaV. Genetic prediction of common diseases. Still no help for the clinical diabetologist! Nutrition Metab Cardiovasc Dis. (2012) 22:929–36. doi: 10.1016/j.numecd.2012.04.010

  • CrossRef
  • Google Scholar

• 33

  MartensRMKoopmanTNoijDPPfaehlerEÜbelhörCSharmaSet al. Predictive value of quantitative 18F-FDG-PET radiomics analysis in patients with head and neck squamous cell carcinoma. EJNMMI Res. (2020) 10:1–15. doi: 10.1186/s13550-020-00686-2

  • CrossRef
  • Google Scholar

• 34

  SalmanpourMRHosseinzadehMRezaeijoSMRahmimA. Fusion-based tensor radiomics using reproducible features: application to survival prediction in head and neck cancer. Comput Methods Programs Biomedicine. (2023) 240:107714. doi: 10.1016/j.cmpb.2023.107714

  • CrossRef
  • Google Scholar

• 35

  NieKHuPZhengJZhangYYangPJabbourSKet al. Incremental value of radiomics in 5-year overall survival prediction for stage II–III rectal cancer. Front Oncol. (2022) 12:779030. doi: 10.3389/fonc.2022.779030

  • CrossRef
  • Google Scholar

• 36

  ZhengJChakrabortyJChapmanWCGerstSGonenMPakLMet al. Preoperative prediction of microvascular invasion in hepatocellular carcinoma using quantitative image analysis. J Am Coll Surgeons. (2017) 225:778–88. doi: 10.1016/j.jamcollsurg.2017.09.003

  • CrossRef
  • Google Scholar

• 37

  AttiyehMAChakrabortyJGazitLLangdon-EmbryLGonenMBalachandranVPet al. Preoperative risk prediction for intraductal papillary mucinous neoplasms by quantitative CT image analysis. Hpb. (2019) 21:212–8. doi: 10.1016/j.hpb.2018.07.016

  • CrossRef
  • Google Scholar

• 38

  HuangYQLiangCHHeLTianJLiangCSChenXet al. Development and validation of a radiomics nomogram for preoperative prediction of lymph node metastasis in colorectal cancer. J Clin Oncol. (2016) 34:2157–64. doi: 10.1200/JCO.2015.65.9128

  • CrossRef
  • Google Scholar

• 39

  YuYTanYXieCHuQOuyangJChenYet al. Development and validation of a preoperative magnetic resonance imaging radiomics–based signature to predict axillary lymph node metastasis and disease-free survival in patients with early-stage breast cancer. JAMA network Open. (2020) 3:e2028086. doi: 10.1001/jamanetworkopen.2020.28086

  • CrossRef
  • Google Scholar

• 40

  WuSZhengJLiYWuZShiSHuangMet al. Development and validation of an MRI-based radiomics signature for the preoperative prediction of lymph node metastasis in bladder cancer. EBioMedicine. (2018) 34:76–84. doi: 10.1016/j.ebiom.2018.07.029

  • CrossRef
  • Google Scholar

• 41

  ZhangZHuangMJiangZChangYTorokJYinFF. 4D radiomics: impact of 4D-CBCT image quality on radiomic analysis. Phys Med Biol. (2021) 66(4):045023.

  • Google Scholar

• 42

  ZhangZHuangMJiangZChangYLuKYinFF. Patient-specific deep learning model to enhance 4D-CBCT image for radiomics analysis. Phys Med Biol. (2022) 67(8):085003.

  • Google Scholar

• 43

  GilliesRJKinahanPEHricakH. Radiomics: images are more than pictures, they are data. Radiology. (2016) 278:563–77. doi: 10.1148/radiol.2015151169

  • CrossRef
  • Google Scholar

• 44

  AikenAHRathTJAnzaiYBranstetterBFHoangJKWigginsRHet al. ACR Neck Imaging Reporting and Data Systems (NI-RADS): a white paper of the ACR NI-RADS Committee. J Am Coll Radiol. (2018) 15:1097–108. doi: 10.1016/j.jacr.2018.05.006

  • CrossRef
  • Google Scholar

• 45

  DinkelborgPRoSRShnayienSSchaafsLAKoerdtSKreutzerKet al. Retrospective evaluation of NI-RADS for detecting postsurgical recurrence of oral squamous cell carcinoma on surveillance CT or MRI. Am J Roentgenol. (2021) 217:198–206. doi: 10.2214/AJR.20.24209

  • CrossRef
  • Google Scholar

• 46

  HsuDChokshiFHHudginsPAKunduSBeitlerJJPatelMRet al. Predictive value of first posttreatment imaging using standardized reporting in head and neck cancer. Otolaryngology–Head Neck Surg. (2019) 161:978–85. doi: 10.1177/0194599819865235

  • CrossRef
  • Google Scholar

• 47

  VallieresMKay-RivestEPerrinLJLiemXFurstossCAertsHJet al. Radiomics strategies for risk assessment of tumour failure in head-and-neck cancer. Sci Rep. (2017) 7:10117. doi: 10.1038/s41598-017-10371-5

  • CrossRef
  • Google Scholar

• 48

  LingXAlexanderGSMolitorisJChoiJSchumakerLMehraRet al. Identification of CT-based non-invasive radiomic biomarkers for overall survival prediction in oral cavity squamous cell carcinoma. Sci Rep. (2023) 13:21774. doi: 10.1038/s41598-023-48048-x

  • CrossRef
  • Google Scholar

• 49

  Shafiq-ul-HassanMZhangGGLatifiKUllahGHuntDCBalagurunathanYet al. Intrinsic dependencies of CT radiomic features on voxel size and number of gray levels. Med Phys. (2017) 44:1050–62. doi: 10.1002/mp.12123

  • CrossRef
  • Google Scholar

• 50

  Bagher-EbadianHSiddiquiFLiuCMovsasBChettyIJ. On the impact of smoothing and noise on robustness of CT and CBCT radiomics features for patients with head and neck cancers. Med Phys. (2017) 44:1755–70. doi: 10.1002/mp.12188

  • CrossRef
  • Google Scholar

• 51

  HosseiniSAShiriIHajianfarGGhafarianPKaramMBAyMR. The impact of preprocessing on the PET-CT radiomics features in non-small cell lung cancer. Front Biomed Technol. (2021) 8:261–72. doi: 10.18502/fbt.v8i4.7754

  • CrossRef
  • Google Scholar

• 52

  ThévenazPBluTUnserM. Image interpolation and resampling. Handb Med imaging Process Anal. (2000) 1:393–420.

  • Google Scholar

• 53

  CollewetGStrzeleckiMMarietteF. Influence of MRI acquisition protocols and image intensity normalization methods on texture classification. Magnetic resonance Imaging. (2004) 22:81–91. doi: 10.1016/j.mri.2003.09.001

  • CrossRef
  • Google Scholar

• 54

  GianniniVPanicJReggeDBalestraGRosatiS. Could normalization improve robustness of abdominal MRI radiomic features? Biomed Phys Eng Express. (2023) 9:055002. doi: 10.1088/2057-1976/ace4ce

  • CrossRef
  • Google Scholar

• 55

  LarueRTvan TimmerenJEde JongEEFelicianiGLeijenaarRTSchreursWMet al. Influence of gray level discretization on radiomic feature stability for different CT scanners, tube currents and slice thicknesses: a comprehensive phantom study. Acta Oncol. (2017) 56:1544–53. doi: 10.1080/0284186X.2017.1351624

  • CrossRef
  • Google Scholar

• 56

  ZwanenburgAVallièresMAbdalahMAAertsHJAndrearczykVApteAet al. The image biomarker standardization initiative: standardized quantitative radiomics for high-throughput image-based phenotyping. Radiology. (2020) 295:328–38. doi: 10.1148/radiol.2020191145

  • CrossRef
  • Google Scholar

• 57

  GallowayMM. Texture analysis using gray level run lengths. Comput Graphics image Process. (1975) 4:172–9. doi: 10.1016/S0146-664X(75)80008-6

  • CrossRef
  • Google Scholar

• 58

  ThibaultGFertilBNavarroC. Texture indexes and gray level size zone matrix: application to cell nuclei classification, in: International Conference on Pattern Recognition and Information Processing (PRIP’09), Minsk, Belarus, (2009). pp. 140–5.

  • Google Scholar

• 59

  SunCWeeWG. Neighboring gray level dependence matrix for texture classification. Comput Vision Graphics Image Process. (1983) 23:341–52. doi: 10.1016/0734-189X(83)90032-4

  • CrossRef
  • Google Scholar

• 60

  AmadasunMKingR. Textural features corresponding to textural properties. IEEE Trans systems man Cybernetics. (1989) 19:1264–74. doi: 10.1109/21.44046

  • CrossRef
  • Google Scholar

• 61

  MenardS. Applied Logistic Regression Analysis. London UK: Sage (2002). doi: 10.4135/9781412983433

  • CrossRef
  • Google Scholar

• 62

  PeduzziPConcatoJKemperEHolfordTRFeinsteinAR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. (1996) 49:1373–9. doi: 10.1016/S0895-4356(96)00236-3

  • CrossRef
  • Google Scholar

• 63

  LingXBrooksJP. L1-norm regularized L1-norm best-fit line problem. (2020).

  • Google Scholar

• 64

  LingXBuiABrooksP. Kernel l1-norm principal component analysis for denoising. Optimization Lett. (2023). doi: 10.1007/s11590-023-02051-3

  • CrossRef
  • Google Scholar

• 65

  GuyonIWestonJBarnhillSVapnikV. Gene selection for cancer classification using support vector machines. Mach Learn. (2002) 46:389–422. doi: 10.1023/A:1012487302797

  • CrossRef
  • Google Scholar

• 66

  KimHParkJKimYKimK. GLRLM feature by using CT texture analysis is prognostic factor, evaluation of pancreatic cancer heterogeneity. Ann Oncol. (2018) 29:ix63–4. doi: 10.1093/annonc/mdy432.053

  • CrossRef
  • Google Scholar

• 67

  NovitasariDCRLubabASawijiAAsyharAH. Application of feature extraction for breast cancer using one order statistic, GLCM, GLRLM, and GLDM. Adv Science Technol Eng Syst J (ASTESJ). (2019) 4:115–20. doi: 10.25046/astesj

  • CrossRef
  • Google Scholar

• 68

  FilipczukPFevensTKrzyżakAObuchowiczA. GLCM and GLRLM based texture features for computer-aided breast cancer diagnosis. J Med Inf Technol. (2012) 19.

  • Google Scholar

• 69

  DurgamahanthiVAnita ChristalineJShirly EdwardA. GLCM and GLRLM based texture analysis: application to brain cancer diagnosis using histopathology images. In: Intelligent Computing and Applications: Proceedings of ICICA 2019. Singapore: Springer (2021). p. 691–706.

  • Google Scholar

• 70

  HaralickRMShanmugamKDinsteinIH. Textural features for image classification. IEEE Trans systems man cybernetics. (1973) 6:610–21. doi: 10.1109/TSMC.1973.4309314

  • CrossRef
  • Google Scholar

• 71

  DepeursingeAAndrearczykVWhybraPGriethuysenJVMüllerHSchaerRet al. The image biomarker standardization initiative II: standardized quantitative radiomics for high-throughput image-based phenotyping. Radiology. (2023) 295:328–38.

  • Google Scholar

• 72

  TronconeGVigliarE. Histopathology of the Tumors. In: Practical Medical Oncology Textbook (Switzerland: Springer Nature) (2021). p. 33–41.

  • Google Scholar

• 73

  XieDWangTTHuangSJDengJJRenYJYangYet al. Radiomics nomogram for prediction disease-free survival and adjuvant chemotherapy benefits in patients with resected stage I lung adenocarcinoma. Trans Lung Cancer Res. (2020) 9:1112. doi: 10.21037/tlcr

  • CrossRef
  • Google Scholar

• 74

  MazzaschiGQuainiFMilaneseGMadedduDBocchialiniGAmpolliniLet al. Differentially regulated high-throughput CT imaging features correlate to distinct tumor immune contextures portraying a radiomic signature with prognostic impact on surgically resected NSCLC. Ann Oncol. (2019) 30:ii1. doi: 10.1093/annonc/mdz072.001

  • CrossRef
  • Google Scholar

• 75

  MarrD. Representing the image: zero-crossings and the raw primal sketch. In: Vision. WH Freeman, San Francisco, CA (1982). p. 54–68.

  • Google Scholar

• 76

  SaitoTRehmsmeierM. The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PloS One. (2015) 10:e0118432. doi: 10.1371/journal.pone.0118432

  • CrossRef
  • Google Scholar

• 77

  JonesAPhillipsDHelliwellTRolandN. Occult node metastases in head and neck squamous carcinoma. Eur Arch oto-rhino-laryngology. (1993) 250:446–9. doi: 10.1007/BF00181087

  • CrossRef
  • Google Scholar

• 78

  RoepmanPWesselsLFKettelarijNKemmerenPMilesAJLijnzaadPet al. An expression profile for diagnosis of lymph node metastases from primary head and neck squamous cell carcinomas. Nat Genet. (2005) 37:182–6. doi: 10.1038/ng1502

  • CrossRef
  • Google Scholar