AUTHOR=Zhang Guo , Huang Zhiwei , Lin Jinzhao , Li Zhangyong , Cao Enling , Pang Yu , sun Weiwei 

TITLE=A 3D reconstruction based on an unsupervised domain adaptive for binocular endoscopy

JOURNAL=Frontiers in Physiology

VOLUME=Volume 13 - 2022

YEAR=2022

URL=https://www.frontiersin.org/journals/physiology/articles/10.3389/fphys.2022.994343

DOI=10.3389/fphys.2022.994343

ISSN=1664-042X

ABSTRACT=Endoscopic imaging plays a crucial role in minimally invasive surgery. Therefore, we propose an unsupervised adaptive neural network to address the lack of true parallax in binocular endoscopic images. The network combines adaptive smoke removal, depth estimation of binocular endoscopic images, and 3D display of high-quality endoscopic images. We simulate the fog generated during surgery by artificial fogging. The training images of U-Net fused by Laplacian pyramid are introduced to improve the network's ability to extract intermediate features. After adding the Convolutional Block Attention Module, the optimal parameters of each layer of the network are obtained. We use the virtual right-eye image obtained by combining the left-eye image with disparity as a label for self-supervised training in HS-Resnet, and the whole network is trained in an end-to-end manner. This method extracts fused disparity images at different scale levels of the decoder, and the generated disparity images are more complete and smoother. The experimental results show that light and heavy smoke can be removed in real laparoscopic surgery; 3D images of binocular endoscopic tissue structures are effectively reconstructed; contours, edges, details, and vessel textures in medical images are preserved; and real-time surgery needs are met. The model significantly outperforms existing similar schemes in objective indicators and subjective assessments by clinicians, highlighting the broad prospects of 3D binocular endoscopic imaging in clinical applications.