AUTHOR=Qiao Yuhui , Liao Qingxi , Zhang Moran , Han Binbin , Peng Chengli , Huang Zhenhao , Wang Shaodong , Zhou Guangsheng , Xu Shengyong 

TITLE=Point clouds segmentation of rapeseed siliques based on sparse-dense point clouds mapping

JOURNAL=Frontiers in Plant Science

VOLUME=Volume 14 - 2023

YEAR=2023

URL=https://www.frontiersin.org/journals/plant-science/articles/10.3389/fpls.2023.1188286

DOI=10.3389/fpls.2023.1188286

ISSN=1664-462X

ABSTRACT=Rapeseed yield is determined by the number of siliques per plant, the number of seeds per silique, and the thousand seed weight. However, currently, manual counting of rapeseed siliques is timeconsuming, laborious, and expensive, which limits the rapid progress of rapeseed-related research. Therefore, we propose a high-throughput and low-cost automatic detection method based on deep learning. Firstly, we use a smartphone to shoot a video around the rapeseed plants during the silique period. Based on the SIFT operator, feature point detection and matching, the principle of epipolar geometry, and triangulation, we apply sparse point clouds recovery to the extracted video frames. Using the method of image matching calculates the disparity of the image to obtain the depth value. The depth map is fused to obtain a dense point cloud. The Structure from Motion (SFM) algorithm is used to reconstruct the model of the whole rapeseed plant during the silique period. Then, we use a pass-through filter to remove the background from the rapeseed plant model and process the downsampled 3D point clouds data using the Deep Graph Convolutional Neural Network (DGCNN). The point clouds is divided into two categories: sparse rapeseed crown siliques and rapeseed stems. The sparse-dense point clouds mapping method is used to segment the sparse crown siliques and obtain the original whole rapeseed silique point clouds, which effectively saves running time and improves efficiency. On the one hand, Euclidean clustering segmentation is used to segment the rapeseed crown siliques. On the other hand, the random sampling consensus algorithm is used to segment the siliques that are stuck together after clustering. The three-dimensional (3D) spatial position of each silique is obtained. Then, the number of siliques is counted. Finally, we identified 1,457 siliques from 12 rapeseed plants. The experimental results show that the recognition accuracy is greater than 97.80%. The method proposed in this paper achieved good results in rapeseed silique recognition and provided a useful example for applying deep learning networks for segmentation of dense 3D point clouds.