Enhancing Bangla Handwritten Character Recognition Using Vision Transformers, VGG-16, and ResNet-50: A Performance Analysis

A H M Shahariar Parvez¹Md Samiul Islam²Fahmid Al Farid³Tashida Yeasmin⁴Md. Monirul Islam⁵Md. Shafiul Azam⁵Hezerul Abdul Karim³JIA UDDIN^6*

• ¹Daffodil International University, Dhaka, Bangladesh
• ²State University of Bangladesh, Dhaka, Bangladesh
• ³Multimedia University, Malacca, Malaysia
• ⁴Atish Dipankar University of Science and Technology, Dhaka, Bangladesh
• ⁵Pabna University of Science and Technology, Pabna, Bangladesh
• ⁶Woosong University, Daejeon, Republic of Korea

Bangla Handwritten Character Recognition (BHCR) remains challenging due to complex alphabets, and handwriting variations. In this study, we present a comparative evaluation of three deep learning architectures—Vision Transformer (ViT), VGG-16, and ResNet-50—on the CMATERdb 3.1.2 dataset comprising 24,000 images of 50 basic Bangla characters. Our work highlights the effectiveness of ViT in capturing global context and long-range dependencies, leading to improved generalization. Experimental results show that ViT achieves a state-of-the-art accuracy of 98.26%, outperforming VGG-16 (94.54%) and ResNet-50 (93.12%). We also analyze model behavior, discuss overfitting in CNNs, and provide insights into character-level misclassifications. This study demonstrates the potential of transformer-based architectures for robust BHCR and offers a benchmark for future research.