AUTHOR=Zhu Dongmei , Li Junyu , Li Yan , Wu Ji , Zhu Lin , Li Jian , Wang Zimo , Xu Jinfeng , Dong Fajin , Cheng Jun 

TITLE=Multimodal ultrasound fusion network for differentiating between benign and malignant solid renal tumors

JOURNAL=Frontiers in Molecular Biosciences

VOLUME=Volume 9 - 2022

YEAR=2022

URL=https://www.frontiersin.org/journals/molecular-biosciences/articles/10.3389/fmolb.2022.982703

DOI=10.3389/fmolb.2022.982703

ISSN=2296-889X

ABSTRACT=Abstract
Objective: We aim to establish a deep learning model called Multimodal Ultrasound Fusion Network (MUF-Net) based on gray-scale and contrast-enhanced ultrasound (CEUS) images for classifying benign and malignant solid renal masses automatically and to compare the model's performance with the assessments by radiologists with different levels of experience. 
Methods: A retrospective study evaluated CEUS videos of 181 patients with solid renal tumors (81 benign and 100 malignant tumors) from June 2012 to June 2021. A total of 9794 B-mode and CEUS-mode images were cropped from the CEUS diagnosis videos of 181 patients, respectively. The MUF-Net was proposed to combine on gray-scale ultrasound and CEUS images to diagnose the nature of solid renal masses. In this network, two independent branches were designed to extract features from each of the two modalities, and the features were fused using adaptive weight-based fusion. Finally, the classifier output a classification score based on the fused features. The model's performance was evaluated with five-fold cross-validation and compared with the assessments of the two groups of radiologists with different levels of experience.
Results: For the discrimination between benign and malignant renal masses, the accuracy, sensitivity, and specificity of the junior radiologist, senior radiologist group, and MUF-Net were 70.6%, 89.2%, and 58.7%; 75.7%, 95.9%, and 62.9%; 80.0%, 80.4%, and 79.1%, respectively; the area under receiver operating characteristic curve was 0.740(95%CI:0.70-0.75), 0.793(95%CI:0.72-0.83), and 0.880(95%CI:0.83-0.93), respectively.
Conclusions: The MUF-Net model can accurately classify benign and malignant solid renal masses and achieve better performance than senior radiologists. 
Key points:
The CEUS video data contains the entire tumor microcirculation perfusion characteristics. The MUF-Net, the multimodal fusion convolutional Network model based on B-mode images and CEUS-mode images, could be trained to accurately distinguish benign and malignant solid renal lesions with 80% accuracy, which is better than radiologists.