HMA -Net: A Hybrid Mixer Framework with Multihead Attention for Breast Ultrasound Image Segmentation

Soumya Sara KoshyJANI ANBARASI L^*

• Vellore Institute of Technology (VIT), Chennai, India

Breast cancer is a severe illness predominantly affecting women, and in most cases, it leads to loss of life if left undetected. Early detection can significantly reduce the mortality rate associated with breast cancer. Ultrasound imaging has been widely used for effectively detecting the disease, and segmenting breast ultrasound images aids in the identification and localization of tumors, thereby enhancing disease detection accuracy. Numerous computer aided methods have been proposed for the segmentation of breast ultrasound images. A deep learning based architecture utilizing ConvMixer based encoder and ConvNeXT based decoder coupled with convolution enhanced multihead attention has been proposed for segmenting breast ultrasound images. The enhanced ConvMixer modules utilizes spatial filtering and channel-wise integration to efficiently capture local and global contextual features, enhancing feature relevance thus increasing segmentation accuracy through dynamic channel recalibration and residual connections. The bottleneck with the attention mechanism enhances segmentation by utilizing multihead attention to capture long range dependencies thus enabling the model to focus on relevant features across distinct regions. The enhanced ConvNeXT modules with squeeze and excitation utilizes depthwise convolution for efficient spatial filtering, layer normalization for stabilizing training, and residual connections to ensure the preservation of relevant features for accurate segmentation. A combined loss function, integrating binary cross entropy and dice loss, is used to train the model. The proposed model has exceptional performance in segmenting intricate structures, as confirmed by comprehensive experiments conducted on two datasets namely BUSI dataset and BrEaST dataset of breast ultrasound images. The model achieved a Jaccard index of 98.04% and 94.84%, and a Dice similarity coefficient of 99.01% and 97.35% on the BUSI and BrEaST datasets respectively.