Explainable Multi-modal Deep Learning for Transparent Cancer Diagnosis: Integrating Radiology, Clinical Features, and Decision Visualization

Sital Dash^1*Laxmi Bewoor²Yashwant Dongre²Amol Bhosle³Kailas PATIL^1*Shrikant Jadhav^4*Banani Mohapatra⁵Bhavnish Walia⁶

• ¹Vishwakarma University, Pune, India
• ²Vishwakarma Institute of Technology, Pune, India
• ³MIT Art Design and Technology University, Pune, India
• ⁴San Jose State University, San Jose, United States
• ⁵Walmart, California, Canada
• ⁶Amazon, New York, United States

Abstract. Introduction:Although artificial intelligence–based cancer diagnostic models have demonstrated strong predictive performance, their lack of transparency and reliance on single-modality data continue to limit clinical trust and adoption. Effectively integrating multi-modal data with interpret-able decision-making remains a key challenge. Methods:We propose an explainable multi-modal deep learning framework that integrates radiological imaging and structured clinical features using attention-based fusion. Image-level explanations are generated using Grad-CAM++, while SHAP is employed to quantify clinical feature contributions, enabling unified and cross-modal aligned interpretation rather than independent uni-modal explanations. The framework was evaluated on publicly available datasets, including CBIS-DDSM mammography, Duke Breast Cancer MRI, and TCGA cohorts (BRCA, LUAD, and GBM), comprising a total of 3,842 images from 2,917 patients. Results: The proposed model consistently outperformed uni-modal approaches and simple fusion baselines, achieving an improved balance between sensitivity and specificity. Attention-based fusion demonstrated superior performance compared with feature concatenation, and the integration of explainability did not compromise predictive accuracy. Visual and clinical explanations highlighted diagnostically relevant tumor regions and established oncological risk factors. Stable performance across datasets indicates strong generalization capability. Discussion: These results demonstrate that explainable multi-modal learning can effectively combine accuracy, interpret-ability, and robustness, supporting the development of reliable AI-based decision-support systems for cancer diagnosis. Keywords: Explainable artificial intelligence; Multi-modal deep learning; Cancer diagnosis; Medical imaging; Clinical data integration; Attention-based fusion; Model interpretability.