Automated Machine-Learning Framework Integrating Histopathological and Radiological Information for Predicting IDH1 Mutation Status in Glioma

Diffuse gliomas are the most common malignant primary brain tumors. Identification of isocitrate dehydrogenase 1 (IDH1) mutations aids the diagnostic classification of these tumors and the prediction of their clinical outcomes. While histology continues to play a key role in frozen section diagnosis, as a diagnostic reference and as a method for monitoring disease progression, recent research has demonstrated the ability of multi-parametric magnetic resonance imaging (MRI) sequences for predicting IDH genotypes. In this paper, we aim to improve the prediction accuracy of IDH1 genotypes by integrating multi-modal imaging information from digitized histopathological data derived from routine histological slide scans and the MRI sequences including T1-contrast (T1) and Fluid-attenuated inversion recovery imaging (T2-FLAIR). In this research, we have established an automated framework to process, analyze and integrate the histopathological and radiological information from high-resolution pathology slides and multi-sequence MRI scans. Our machine-learning framework comprehensively computed multi-level information including molecular level, cellular level, and texture level information to reflect predictive IDH genotypes. Firstly, an automated pre-processing was developed to select the regions of interest (ROIs) from pathology slides. Secondly, to interactively fuse the multimodal complementary information, comprehensive feature information was extracted from the pathology ROIs and segmented tumor regions (enhanced tumor, edema and non-enhanced tumor) from MRI sequences. Thirdly, a Random Forest (RF)-based algorithm was employed to identify and quantitatively characterize histopathological and radiological imaging origins, respectively. Finally, we integrated multi-modal imaging features with a machine-learning algorithm and tested the performance of the framework for IDH1 genotyping, we also provided visual and statistical explanation to support the understanding on prediction outcomes. The training and testing experiments on 217 pathologically verified IDH1 genotyped glioma cases from multi-resource validated that our fully automated machine-learning model predicted IDH1 genotypes with greater accuracy and reliability than models that were based on radiological imaging data only. The accuracy of IDH1 genotype prediction was 0.90 compared to 0.82 for radiomic result. Thus, the integration of multi-parametric imaging features for automated analysis of cross-modal biomedical data improved the prediction accuracy of glioma IDH1 genotypes.

Diffuse gliomas are the most common malignant primary brain tumors. Identification of isocitrate dehydrogenase 1 (IDH1) mutations aids the diagnostic classification of these tumors and the prediction of their clinical outcomes. While histology continues to play a key role in frozen section diagnosis, as a diagnostic reference and as a method for monitoring disease progression, recent research has demonstrated the ability of multi-parametric magnetic resonance imaging (MRI) sequences for predicting IDH genotypes. In this paper, we aim to improve the prediction accuracy of IDH1 genotypes by integrating multi-modal imaging information from digitized histopathological data derived from routine histological slide scans and the MRI sequences including T1-contrast (T1) and Fluid-attenuated inversion recovery imaging (T2-FLAIR). In this research, we have established an automated framework to process, analyze and integrate the histopathological and radiological information from high-resolution pathology slides and multi-sequence MRI scans. Our machine-learning framework comprehensively computed multi-level information including molecular level, cellular level, and texture level information to reflect predictive IDH genotypes. Firstly, an automated pre-processing was developed to select the regions of interest (ROIs) from pathology slides. Secondly, to interactively fuse the multimodal complementary information, comprehensive feature information was extracted from the pathology ROIs and segmented tumor regions (enhanced tumor, edema and non-enhanced tumor) from MRI sequences. Thirdly, a Random Forest (RF)-based algorithm was employed to identify and quantitatively characterize histopathological and radiological imaging origins, respectively. Finally, we integrated multi-modal imaging features with a machine-learning algorithm and tested the performance of the framework for IDH1 genotyping, we also provided visual and statistical explanation to support the understanding on prediction outcomes. The training and testing experiments on 217 pathologically verified IDH1 genotyped glioma cases from multi-resource validated that our fully automated machine-learning model predicted IDH1 genotypes with greater accuracy and reliability than models that were based on radiological imaging data only. The accuracy of IDH1 genotype prediction was 0.90 compared to 0.82 for radiomic result. Thus, the integration of multi-parametric imaging features for automated analysis of cross-modal biomedical data improved the prediction accuracy of glioma IDH1 genotypes.

INTRODUCTION
The current WHO classification of CNS tumors not only considers histopathological phenotypes but also molecular genetic parameters, e.g., DNA methylome profiling (Louis et al., 2021;Lopes 2017;Chang et al., 2018;Lee et al., 2019). IDH mutations in glioma have been found to be associated with better outcomes and are therefore of great relevance in the clinical assessment of glioma patients (Louis et al., 2016b). Recently, some attempts have been made to use radiological images for the pre-surgical prediction of IDH1 genotypes (Gillies et al., 2015;Kesler et al., 2019;Lee et al., 2019;Tatekawa et al., 2021).
Pathological and radiological imaging results are increasingly available in digitized format (Nance et al., 2013;Farahani and Pantanowitz 2015;Griffin and Treanor 2017). It has become apparent that fully utilizing the data of digital radiology and pathology images through machine-learning can facilitate the identification of biomarkers that reflect information on the basic biology and physiology of various malignancies (Deo 2015;Gillies et al., 2016). Although tumor diagnoses increasingly consider molecular genetics markers, histology continues to play a key role in frozen section diagnosis, as a diagnostic reference and as a method for monitoring disease progression. In addition, compared to the indirect visualization of disease phenotypes by means of imaging, histology provides direct information at high resolution (Kinjo et al., 2008;Missbach-Guentner et al., 2018;Vågberg et al., 2018;Rathore et al., 2020). Competing with the computer-aid image technique used in radiology and pathology, clinical practice demands professional knowledge and long-term training to obtain useful information from the image with the naked eye for diagnosis and evaluation (Harezlak and Kasprowski 2018).
Computerized image analysis can reduce subjective interobserver bias that is known to limit all human observation including in histopathology (Emblem et al., 2014;Zhang et al., 2016;Choi et al., 2019). Machine-learning algorithms are already widely used in glioma research, and most are based on the analysis of features extracted from MRIs (Ellingson et al., 2011;Zacharaki et al., 2012;Emblem et al., 2014;Macyszyn et al., 2015). Zhou et al. (2017) have recently demonstrated the ability to predict IDH genotypes in cases of primary grade II and III glioma using clinical and pathological variables and textual features extracted from regions of interest (ROI) in four sequences of MRI, including T1W, T2W, T1CE and T2-FLAIR, achieving an accuracy of 0.86. Compared to the results obtained by Zhou and colleagues, Eichinger et al. (2017) were able to increase the accuracy of IDH genotyping to 92% by designing an algorithm based on feature extraction of local binary pattern, which represent texture features extracted from multimodal MRI data. In addition to traditional machine-learning techniques, the method employed by Xing et al. (2017) classifies IDH mutations and IDH wild type (IDH-wt) using conventional machinelearning algorithms in order to extract deep features from four sequences of MRI (T1W, T2W, T1CE, T2-FLAIR). Zhang et al. (2016) aimed to distinguish the presence of an IDH mutation and IDH-wt in primary grade III and IV gliomas by means of additional features (intensity, texture and shape features) extracted from multimodality MRIs (T1, T1CE, T2, T2-FLAIR and DWI), achieving an accuracy of 0.883 using the Random Forest algorithm.
Recently, the improvements in deep learning is capable to overcome the previous challenges by learning high-dimensional representations of imaging data. Novel, fully automated postprocessing analyses of standard and advanced MR images are clearly rapidly approaching. These fully automated analyses are especially appealing because they provide unbiased evaluations independent of operator, training or experience. Fully automated postprocessing with deep learning analyses of standard and advanced MR images have achieved high accuracy even at 92.8% accuracy, 93.1% specificity, and 92.6% sensitivity . Although they can be very powerful for the prediction of IDH for glioma, deep CNN models are vulnerable to overfitting to their given training dataset and inherent difficult for interpretation which is the most crucial for decision support system. Comprehensive understanding of the mechanism of deep and machine-learning is necessary, however, to best develop and then apply these algorithms to clinical practice we need to avoid Frontiers in Bioinformatics | www.frontiersin.org October 2021 | Volume 1 | Article 718697 their potential pitfalls. It is unlikely to replace tissue sampling for now; therefore, the continued improvement in model performance and consistency across diverse imaging modality brings us closer to the precise molecular diagnosis (Gutman and Young 2021).
In this study, we introduce an improved approach to IDH prediction, which integrates radiological and histopathological data analyses in a single combined framework. Radiological data analysis in this context refers to the extraction and analysis of high-throughput features from tomographic images (MR images) whereas histopathological data analysis refers to the features extraction and analysis from whole slide images. We envision that this model could set a pathway for the noninvasive evaluation of IDH mutation in gliomas and may provide a quantitative result analysis for the researchers. Compared with deep learning-based method, we aim to provide doctors with an intuitive, interpretable, and costeffective mechanism through machine-learning based method to support the decision on IDH status prediction.

Patient Enrolment
The imaging data of 217 subjects that had been diagnosed with glioma were collected from two different sources. 126 cases were from Shandong Provincial Hospital that is affiliated with Shandong University, comprising 41 histological grade III cases and 85 histological grade IV cases. The remaining 91 cases were retrieved from The Cancer Imaging Archive (TCIA), comprising 25 histological grade III cases and 66 histological grade IV cases ( Table 1). The criteria for image acquisition in this study are as follows: I) available histology, age at diagnosis, sex, and IDH status; II) MR imaging data, including post-contrast T1-weighted images (T1CE), and T2-FLAIR, and III) histopathological images.

Histopathological Images
Shandong provincial hospital's cohort: cases were diagnosed according to WHO criteria (Louis et al., 2021). Paraffinembedded tissue samples were cut into 3 μm thick slides and stained with H&E stain for all patients in this cohort. All H&E stained images were scanned on a Leica SCN400 slide scanner (Leica Biosystems, Nussloch, Germany) with multi-resolution varying from 20× to 40× for analysis.
Genomic DNA was isolated from formalin-fixed paraffinembedded glioma tissues. DNA was extracted using the QIAamp DNA Micro kit (Qiagen GmbH, Hilden, Germany) as previously described (Perizzolo et al., 2012).
Expression of IDH-R132H mutant was firstly analyzed by IHC as previously described (Reyes-Botero et al., 2014). For IDH R132H-negative tumors, multiple-gene Sanger sequencing was performed to identify alternative IDH mutations (Sanson et al., 2009). IDH status was defined according to the absence of IDH-R132H immunopositivity and/or mutations in IDH1 and IDH2 genes identified by sequencing.
The Cancer Genome Atlas (TCGA) cohort: Digital pathology slides of diagnosed diffuse gliomas were downloaded from TCGA Data Portal (http://cancergenome.nih.gov.) including information on IDH status, and the corresponding MRI images were acquired from the Cancer Imaging Archive (TCIA) Data Portal (https://www.cancerimagingarchive.net).
All MRI sequences of each patient from our own datasets and from TCIA have the same imaging scale, position, slice anatomy and slice thickness.

COMPUTER ANALYSIS
An automated framework was designed to predict IDH genotype, consisting of the following steps, which were carried out in sequence: I) automated image pre-processing to select the regions of interest (ROIs), II) feature extraction, III) feature selection, and IV) automated IDH genotype prediction and results interpretation (Figure 1).

Automatic Extraction of Region of Interest
Our computational algorithm used for the analysis of histological images approaches the region of interest at two different levels. First, one tile with the highest cell density (5,120 * 5,120) (Sertel et al., 2009;Mobadersany et al., 2018) is extracted employing the watershed nuclei detection algorithm (Al-Kofahi et al., 2010;Kumar et al., 2017;Wang et al., 2018). Then, based on Hue, Saturation, and Value of Brightness (HSV channel), five tiles representing the whole image (Al-Kofahi et al., 2010;Kumar et al., 2017) at 40X resolution are identified. Third, based on the HSV channel, five tiles representing the entire image at 4X resolution are identified.
For the analysis of radiological images, we have segmented edema and non-enhanced tumors from T2-FLAIR image. In addition, T1CE images were used for enhancing tumor volume segmentation. The lesions were separated into three parts, enhancing tumor, tumor necrosis and peritumoral edema. The process of tumor segmentation was performed manually using the ITK-SNAP software (version 3.6.0; www.itksnap.org). First, all MRI sequences were retrieved from the Picture Archiving and Communication System (PACS). Then we applied N4 bias field correction to remove the presence of low frequency intensity nonuniformity. Inter-modality co-registration with different 2D MRI sequences was achieved by means of ITK-SNAP. Using this method, ROIs of enhancing tumor were delineated on postcontrast T1WI images by a semi-automatic method, in which only the enhancing area was selected. Tumor necrosis was defined as the non-enhancing area within enhancing area on postcontrast T1WI. ROIs of peritumoral edema were delineated on T2-FLAIR, which was defined as the high-signal region beyond the enhancing area. The process was performed by a consultant neuro-radiologist. Finally, the ROIs were registered on each slice of each 2D MRI sequence.

Feature Extraction
In this step, we extracted quantitative features from histopathology images and MRIs. In case of the histopathology images, we extracted two types, visual features and sub-visual features, at two different resolutions. The visual features quantitatively describe the morphology of nuclei such as the mean area occupied and the pattern of staining. Sub-visual features are derived from a high-throughput intensity and texture matrix, which reflects the intensity distribution at the single pixel level.
In the case of MRIs, we obtained shape features from the volume of interest (VOI) reflecting tumor area and volume. Subsequently, we extracted first-order, second-order and high order features from the ROIs. The shape-based features describe the three-dimensional (3D) properties of the tumor, such as tumor volume, sphericity, and 3D diameter. First-order statistical features reflect the distribution of voxel intensities within the tumor area, including energy and entropy. Secondorder statistical features were obtained from the relationships between adjacent voxels (Balagurunathan et al., 2014) to describe the second-order joint probability function of the tumor region as a gray-level co-occurrence matrix (GLCM) and Gray-level run-length matrix (GLRLM), respectively, which reflect intra-tumoral heterogeneity. High-order features were calculated with the help of different filters such as the wavelet transform.

Feature Selection
Although a large number of image features can be used to construct a model that fully reflects the characteristics of gliomas, removing redundant information can improve the efficacy of the model for glioma genotyping (Guyon and Elisseeff 2003). In order to reduce the amount of redundant information inherent to quantitative features, we built a Random Forest algorithm enhanced by a recursive feature elimination (RF-RFE) procedure in order to identify the relevant and important characteristics before implementation in a classification model all (Saeys et al., 2007). As shown in Figure 1, the feature with the lowest importance for classification calculated by the algorithm will be eliminated.

Modeling and Validation
We are proposing a binary classification model to differentiate patients with an IDH mutation from wild type high-grade gliomas (HGGs) based on clinical features (age and sex), digital histopathological image features and MRI Radiomics features.
The Random Forest algorithm employed in this study is widely used in medical imaging analysis. The corresponding model is able to accommodate a very large set of features. All machinelearning methods were implemented with the Statistics and Machine-Learning package on Python 3.6.
As discussed by Guyon and Elisseeff (2003), although a large number of image features can be used to construct a model to better reflects the characteristics of gliomas, the model may face over-fitting problems, and therefore redundant information needs to be carefully removed to improve the efficacy of the model for genotyping gliomas. In order to ensure the stability and efficiency of the selected features, the 10-Fold cross-validation is nested with the RF training model to select a valuable feature set. The random forest algorithm enhanced by the recursive feature elimination (RF-RFE) process is used to identify relevant and important features before all implementations in the classification model (Saeys et al., 2007).

Feature Extraction and Selection
We extracted a total of 22 morphological features, which were identified in the glioma cases studied (Supplementary Material I. Extraction of histopathologic features), including nuclear shape and staining intensity (Figure 1). In addition, we extracted 171 sub-visual features (Wang et al., 2018) from the high resolution digital histopathology images, including intensity features and GLCM features.
As for results of IDH status prediction for HGGs, the histopathological features extracted from histopathology images, which reached an accuracy of 0.81 ± 0.03 with 10-fold

Comparison of Performances When Using Different Modalities and Feature Types
In order to assess the differential relevance of the modalities tested (T1CE, FLAIR and digital pathology images) in the prediction of IDH genotype, a Random Forest machinelearning model with 10-fold cross-validation was established. In general, scans of histopathological images yielded more accurate results in the IDH genotype prediction than other image types ( Figure 2C). Considering quantitative features obtained from the different modalities, our morphologically defined visual features also showed high accuracy and stability ( Figure 2B). With the multiparameter imaging features minded from different modalities images, our quantitative and objective analysis platform achieved high diagnostic accuracy ( For IDH genotype prediction, optimal features were selected from the different modalities of medical images, including seven features from the digital histopathological images, four features form the T1CE images and five from the T2-FLAIR images. As shown in Figure 2B, GLRLM, Shape-based and GLCM features had the greatest power in predicting glioma IDH status. Age, counts of nuclei and first-order features were the most important factors that contributed to this result. Top-performers within different groups of image features contributed to IDH status prediction as summarized in Table 2.
The accuracy of IDH status prediction was as high as 0.88 ± 0.03 when multi-parametric features were extracted from Frontiers in Bioinformatics | www.frontiersin.org October 2021 | Volume 1 | Article 718697 6 different histopathological and radiomics images through our implementation of the Random Forest algorithm. Table 3 shows the features that play important roles in our classification model.

Quantitative Isocitrate Dehydrogenase Status Prediction and Results
Interpretation LIME (Local Interpretable Model-agnostic Explanations) is a tool for facilitating local model interpretability. The technique perturbs the input data to understand how the predictions are affected. Figures 3, 4 illustrate two representative cases from visual analysis and the machine-learning model. The first case is an IDH-wt patient (Figure 3), who is 43 years old (age at diagnosis), female with a histopathological grade IV glioma. The second one is a patient with an IDH mutation (Figure 4), who is 22 years old (age at diagnosis), male with histopathological grade III glioma.
Machine-learning models taking into account the extracted features' different contributions, then quantitatively predict the results by combining these different features according to their corresponding contributions. During model training, the LIME model can generate weighted coefficients to illustrate the contributions made by different features. The predictive ability of LIME algorithm made the most important contribution to achieve a higher weight value. Positive weights reflect the increase in the corresponding features may make a positive contribution to the IDH status prediction. In contrast, negative weights would have negative predictive power. As shown in Figures 3, 4, the feature contribution for IDH genotyping for two representative cases have been listed, which are derived from the LIME model to obtain the linear combination of feature values and weights.

Discussion
Determination of IDH status has become a standard for glioma diagnosis as it helps to guide clinical decision-making. In this study, we have developed a Random Forest algorithm-based  genotype classifier that allows the prediction of IDH mutation status in glioma patients from pre-surgical MRI scans (Zhang et al., 2016) with improved accuracy. The Random Forest algorithm-based genotype classifier aims to employ the machine-learning algorithm to do the IDH genotype and a stable and efficient prediction result of IDH genotype. In this situation, the Random Forest machine-learning model with 10-fold cross-validation was implement into this experiment.
To be more specific, we apply the RF algorithm into the experiment due to the following advantages: I) in specific experiments, training can be highly parallelized and run efficiently on large data sets; II) since the partition features of decision tree nodes can be selected randomly, which leads to the input samples with high-dimensional features can be processed without dimensionality reduction; III) the algorithm is able to calculate the importance of each feature to the prediction result and IV) due to the adoption of random sampling and random feature selection, the model has small variance at the training location and strong induction ability. We adopt the RF into the experiment, due to the advantages the model have which match the height of our datasets. In order to improve the accuracy of IDH phenotype prediction, visual and sub-visual features extracted from digital histopathological images and quantitative radiomics feature extracted from different multimodality MRIs were implemented into our Random-Forest-Recursive Feature Elimination (RF-RFE) feature selection model to identify optimal criteria for further analysis. In this task, the "visual features" refer to not only the basic features including the color and appearance of nuclear staining, but also non-basic features including different directions. On the other hand, the "sub-visual features" allude to the computerized high-throughput first-order and second-order features, which includes intensity and texture information. In this experiment, features were extracted from different conduits of the H&E images, which aims to improve the prediction accuracy of the IDH phenotype.
Our novel integrated approach, which combined multiparametric biomedical imaging features, was found to be a more accurate predictor of IDH genotype than either radiomics or histopathological feature recognition alone. Multi-parametric biomedical imaging characterizes tumor properties at different biological levels, it meets the need to understand correlations between image features, genomics, and clinical outcomes.
Specifically, the IDH predictive performance of histopathological images was found to be superior to T2-FLAIR and T1CE (0.86 vs. 0.71, 0.75). Among the leading histomorphometrical features, the mean cell area and the mean cell axis were most significant. These top identified features mirror the fact that gliomas with an IDH mutation have a more coherent nuclear architecture, i.e., they are FIGURE 3 | IDH status prediction result explanation of a representative case with the LIME algorithm for the RF model (IDH-Wild type case).
Frontiers in Bioinformatics | www.frontiersin.org October 2021 | Volume 1 | Article 718697 8 morphologically less atypical than IDH wild type, which is associated with a higher risk of recurrence.
Our IDH genotype prediction achieved high accuracy for mainly two reasons: First, we integrated MRIs, digital histopathological images and clinical information for IDH prediction. Second, we used the selected features to significantly reduce the number of parameters in the model to avoid overfitting while making our model more robust. To the best of our knowledge, this is the first study to integrate MRI and digital pathology images in a computerized model for predicting IDH genotype. It is worth noting that T1CE and T2-FLAIR images conferred a higher predictive value than other MR sequences.

Conclusion
In conclusion, our work is a step towards a more effective use of radiomic and histopathological data. It should be particularly helpful for retrospective studies on gliomas where imaging results are available but also to point of care that do not have timely access to a molecular genetics laboratory. To sum up, our results i) demonstrate that machine-learning is capable of indirectly identifying genetic information within structural MR images and histopathological datasets, ii) suggest a complementary method for the IDH genotyping of gliomas suitable for patient screening, and iii) demonstrate the potential for algorithmic tools to support clinical decision-making. Taken together, it is expected that the integration of multimodal biomedical data analysis will become more popular in oncology research and practice as technology evolves, with significant potential for the future clinical management of brain tumor patients.

DATA AVAILABILITY STATEMENT
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

ETHICS STATEMENT
This study was approved by the institutional review board (IRB) of Shandong Provincial Hospital affiliated to Shandong First Medical University (SFMU) and conducted in accordance with ethical guidelines, including the Declaration of Helsinki and Belmont Report. Informed consent was waived for the TCIA given the use of existing public datasets; and for the 126 patients from Shandong First Medical University (SFMU), written informed consents to participate in this study was provided by the patient/next of kin. Frontiers in Bioinformatics | www.frontiersin.org October 2021 | Volume 1 | Article 718697