AUTHOR=Lu Guoxiu , Tian Ronghui , Yang Wei , Liu Ruibo , Liu Dongmei , Xiang Zijie , Zhang Guoxu 

TITLE=Deep learning radiomics based on multimodal imaging for distinguishing benign and malignant breast tumours

JOURNAL=Frontiers in Medicine

VOLUME=Volume 11 - 2024

YEAR=2024

URL=https://www.frontiersin.org/journals/medicine/articles/10.3389/fmed.2024.1402967

DOI=10.3389/fmed.2024.1402967

ISSN=2296-858X

ABSTRACT=Objectives: To develop a deep learning radiomics model using multimodal imaging to differentiate benign and malignant breast tumours.Methods: Multimodality imaging data, including ultrasonography (US), mammography (MG) and magnetic resonance imaging (MRI), from 322 patients (112 with benign breast tumours and 210 with malignant breast tumours) with histopathologically confirmed breast tumours were retrospectively collected between December 2018 and May 2023. Based on multimodal imaging, the experiment was divided into three parts: traditional radiomics, deep learning radiomics, and feature fusion. We tested the performance of seven classifiers, namely, SVM, KNN, random forest, extra trees, XGBoost, LightGBM, and LR, on different feature models. In terms of feature fusion, under the ensemble and stacking strategies, we obtained the optimal classification model for benign and malignant breast tumours.In terms of traditional radiomics, the ensemble fusion strategy achieved the highest accuracy, AUC, and specificity, with values of 0.892, 0.942 [0.886-0.996], and 0.956 [0.873-1.000], respectively. The early fusion strategy with US, MG, and MRI achieved the highest sensitivity of 0.952 [0.887-1.000]. In terms of deep learning radiomics, the stacking fusion strategy achieved the highest accuracy, AUC, and sensitivity, with values of 0.937, 0.947 [0.887-1.000], and 1.000 [0.999-1.000], respectively. The early fusion strategies of US+MRI and US+MG achieved the highest specificity of 0.954 [0.867-1.000]. In terms of feature fusion, the ensemble and stacking approaches of the late fusion strategy achieved the highest accuracy of 0.968. In addition, stacking achieved the highest AUC and specificity, which were 0.997 [0.990-1.000] and 1.000 [0.999-1.000], respectively. The traditional radiomics and depth features of US+MG+MR achieved the highest sensitivity of 1.000 [0.999-1.000] under the early fusion strategy.This study demonstrated the potential of integrating deep learning and radiomics features with multimodal images. As a single modality, MRI based on radiomics features achieved greater accuracy than US or MG. The US and MG models achieved higher accuracy with transfer learning than the single-mode or radiomics models. The traditional radiomics and depth features of US+MG+MR achieved the highest sensitivity under the early fusion strategy, showed higher diagnostic performance and provided more valuable information for differentiation between benign and malignant breast tumours.