BLoss-DDNet: bending loss and dual-task decoding network for overlap-ping cell nucleus segmentation of cervical clinical LBC images

Guihua Yang¹Ziran Chen^2,3Peng Guo⁴Junchi Ma¹Jinjie Huang^5*Jin Cong⁶Xiaona Yang⁷Kai Zhao¹Yibo Wang¹Qi Gao⁸Chengcheng Liu⁹Tianqi Wu⁴Yong Li⁴Yingwei Guo^10*Jie Zheng^4*Xiang-Ran Cai^11*Yingjian Yang^4*

• ¹School of Mechanical and Electrical Engineering, Daqing Normal University, Daqing, China
• ²College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China
• ³Department of Radiological Research and Development, Shenzhen Lanmage Medical Technology Co., Ltd, Shenzhen, China
• ⁴Department of Radiological Research and Development,, Shenzhen Lanmage Medical Technology Co., Ltd, Shenzhen, China
• ⁵School of Automation, Harbin University of Science and Technology, Harbin, China
• ⁶School of Computer, Xi'an Aeronautical University, Xi'an, China
• ⁷School of Information Science and Engineering, Zaozhuang University, Zaozhuang, China
• ⁸Department of Medical Image Processing Algorithm, Research and Development Center of Smart Imaging Software, Neusoft Medical Systems Co Ltd, Shenyang, China
• ⁹School of Life and Health Management, Shenyang City University, Shenyang, China
• ¹⁰School of Electrical and Information Engineering, Northeast Petroleum University, Daqing, China
• ¹¹Medical Imaging Center, The First Affiliated Hospital of Jinan University, Guangzhou, China

Abstract: Cervical cancer has become one of the most malignant tumors that threatens women's health worldwide. Liquid-based cytology (LBC) examination has become the most common screening method for detecting cervical cancer early and preventing it. Currently, nuclear segmentation technology for cervical clinical LBC images based on convolutional neural networks has become a vital means of assisting in the diagnosis of cervical cancer. However, the existing nuclear segmentation techniques fail to segment the nuclei of severely overlapping nuclei in highly aggregated cell clusters, which will inevitably lead to the misdiagnosis of cervical cancer pathology. Therefore, a novel bending loss and dual-task decoding network (Bloss-DDNet) is proposed for overlapping cell nucleus segmentation of cervical clinical LBC images. First, the network architecture search method is introduced to search and optimize the architecture of the decoding module in the dual-task branch, determining the mask and boundary decoding modules (dual-task decoding modules) of the Bloss-DDNet. Second, two feature maps, separately generated from dual-task decoding branches composed of a shared encoder module and dual-task decoder modules, are fused to enhance the sensitivity to cell nucleus boundaries. Third, a bending loss is introduced to the loss function to focus on the curvature variation characteristics of the intersection of overlapping cell nucleus boundaries, thereby constraining the training process of the dual-task decoding branch and increasing the constraint on the cell nucleus boundary. The results show that all evaluation metrics of the proposed Bloss-DDNet achieved the best performance on public data sets. Therefore, the proposed Bloss-DDNet can effectively solve the segmentation problem of overlapping cell clusters and nuclei in clinical LBC images, providing strong support for subsequent clinical auxiliary diagnosis of cervical cancer.