VMDU-Net: A Dual Encoder Network with Transformer and Mamba Fusion for Enhanced Long-Distance Dependency in Polyp Segmentation

Peng Li^1*Jianhua Ding²Chia S Lim¹

• ¹Asia Pacific University of Technology & Innovation, Kuala Lumpur, Malaysia
• ²Gansu Provincial Tumor Hospital, Gan Su WuWei, China

Rectal cancer is typically manifested as polyps. Early screening and timely removal of polyps can effectively prevent colorectal cancer and help halt its progression to malignancy. Although polyp segmentation algorithms play a key role in polyp removal, accurate segmentation remains challenging due to the diverse shapes, indistinct boundaries, and varying sizes of polyps. Furthermore, these algorithms need to capture long-range dependencies, but current polyp segmentation algorithms often struggle to converge when attempting this, posing challenges in practical applications. To address these issues, this study proposes a Transformer and Mamba dual encoder fusion network structure-VMDU-Net. In this model, one encoder integrates the Vision Mamba component, while the other employs the designed Cross-Shaped Transformer. By combining the Mamba structure with the Cross-Shaped Transformer, the network's ability to extract semantic information about polyp shapes and boundaries is enhanced. Additionally, to promote dual encoder fusion, we design the a feature fusion module named Mamba-Transformer-Merge (MTM), which performs attention-weighted fusion from both spatial and channel dimensions, fully leveraging the advantages of both Transformer and Mamba features. To address potential convergence issues during model training, this study employs depthwise separable convolutions for multiscale feature extraction and accelerates convergence using the inductive bias of convolution. Furthermore, experiments were conducted on five widely used polyp datasets, and the results demonstrated outstanding performance in segmentation accuracy and edge detail preservation. Notably, our method achieved a Dice score of 0.934 on the Kvasir-SEG dataset and 0.951 on the CVC-ClinicDB dataset, surpassing existing state-of-the-art algorithms.