Skin Disease Diagnosis using Decision and Feature Level Fusion of Deep Features

Md. Zasim Uddin^1*Md. Arif Shahriar¹Björn W. Schuller^2,3*Md. Nadim Mahamood¹Md Atiqur Rahman Ahad⁴

• ¹Begum Rokeya University, Rangpur City, Bangladesh
• ²Universitätsbibliothek, Technische Universität München, Munich, Bavaria, Germany
• ³Imperial College London, London, England, United Kingdom
• ⁴University of East London, London, United Kingdom

Early skin disease diagnosis is essential and one of the challenging tasks for a dermatologist. Manual diagnosis by healthcare providers is subjective, costly, and may yield inconsistent results. In contrast, automated skin disease detection and classification using traditional machine learning and deep learning approaches have shown promise in addressing this problem. In this study, we propose a hybrid ensemble framework that integrates both feature-level fusion (FLF) and decision-level fusion (DLF) to leverage complementary strengths for detecting and classifying skin diseases. We employ two convolutional neural network (CNN)-based models, i.e., a modified DenseNet201 and VGG19, along with an attention-based model vision transformer (ViT) to identify and classify skin diseases. In FLF, feature representations from these models are point-wise added and passed through a shared classification head to make the final prediction. In DLF, decisions from each base model are collected, and the majority voting scheme is used to make a final decision. Furthermore, we incorporate a generative adversarial network (GAN)-based approach for offline-based training data augmentation to reduce overfitting and improve performance. Based on different evaluation metrics (i.e., accuracy, precision, recall, and F1-score), our proposed framework demonstrates superior performance on four benchmark datasets: the PH2, HAM10000, ISIC 2018, and ISIC 2019 datasets, with an accuracy of 99.3\%/99.2\%, 92.7\%/96.1\%, 86.7\%/89.0\%, and 94.5\%/95.0\%, respectively, for FLF/DLF. These results demonstrate that while both fusion strategies are effective, DLF slightly outperforms FLF, emphasizing the value of ensemble decision aggregation for robust skin disease classification.