An explainable hybrid deep learning framework for precise skin lesion segmentation and multi-class classification

muhammad Fiaz^1,2Bilal Shoaib¹Dr Anas Bilal³Abdul Hannan Khan¹Monir Abdullah Kaid⁴Abdulbasit Abdullah Darem⁵Raheem Sarwar^6*

• ¹green international university lahore, Lahore, Pakistan
• ²University of Central Punjab, Lahore, Pakistan
• ³Hainan Normal University, Haikou, China
• ⁴University of Bisha, Bishah, Saudi Arabia
• ⁵Center for Scientific Research and Entrepreneurship, Northern Border University, Arar 73213, Saudi Arabia, Arar, Saudi Arabia
• ⁶Manchester Metropolitan University, Manchester, United Kingdom

Skin diseases, ranging from benign conditions to malignant tumors such as melanoma, present substantial diagnostic challenges due to their visual complexity and the inherent subjectivity in manual examination. This paper introduces a hybrid deep learning framework specifically designed for skin lesion segmentation and multi-class classification using dermoscopic images. The proposed model integrates a dual-task architecture, which combines a U-Net-based segmentation network with a classification module based on the EfficientNet-B0 backbone. To improve model interpretability and foster clinical trust, Grad-CAM is incorporated, allowing clinicians to visualize heatmaps that highlight the regions influencing the model's decisions. The model was trained and evaluated on the HAM10000 dataset, demonstrating robust performance, with a Dice coefficient surpassing 0.85 for segmentation and classification accuracy nearing 85%. Despite challenges such as class imbalance and the variety of lesion types, the model provides reliable results across different skin conditions. The use of explainable AI (XAI) enhances transparency, a crucial factor in the clinical acceptance of AI-based diagnostic tools. This approach shows promise in improving diagnostic accuracy and supporting dermatologists, especially in resource-constrained settings, by providing both accurate lesion delineation and reliable class predictions. Future research will focus on improving the model's generalizability, addressing underrepresented classes, and validating its effectiveness in real-world clinical environments.