CGA-ASNet: An RGB-D Amodal Segmentation Network for Restoring Occluded Tomato Regions

Zhaoyang LiYong YinZhihong XingHanbing Deng^*

• Shenyang Agricultural University, Shenyang, China

Obtaining the complete morphology of the fruit under non-destructive conditions during tomato growth is crucial for tomato phenotype research. However, occlusions among tomato fruits can impede deep learning-based image segmentation methods from accurately capturing the true shape of the occluded areas during training. This issue results in the model being unable to accurately predict the complete morphology of the fruit, which in turn affects the model's ability to obtain accurate tomato phenotype data. To address this issue, we proposed CGA-ASNet, an RGB-D amodal segmentation network based on a Contextual and Global Attention (CGA) module. First of all, we constructed a synthetic tomato dataset (Tomato-sim) using NVIDIA Isaac Sim's Replicator Composer (ISRC). This dataset accurately simulated tomato morphology and greenhouse environments and had been trained by CGA-ASNet. To evaluate the generalization of model, we tested CGA-ASNet on both the synthetic and a separate real-world dataset. Although no explicit domain adaptation techniques were used, we adopted an implicit strategy by simulating diverse lighting conditions (e.g., strong, normal, and weak illumination) to narrow the domain gap.Additionally, a mean coordinate fusion algorithm was applied to improve annotation completeness in real-world occlusion scenarios. CGA-ASNet leveraged contextual information between feature input keys for self-attention learning, captured global information, and expanded the receptive field, thus enhancing the network's representation capacity and enabling more accurate semantic understanding and precise localization. The experimental results demonstrated that CGA-ASNet achieved an F@.75 score of 94.2 and a mIoU of 82.4% in amodal segmentation accuracy in greenhouse scenarios. The results demonstrate that training with well-designed synthetic data can effectively support accurate occlusion-aware segmentation in real environments that offers a practical solution for tomato phenotyping in greenhouse.