A Dual-task Segmentation Network Based on Multi-head Hierarchical Attention for 3D Plant Point Cloud

Dan Pan¹Baijing Liu²Lin Luo³An Zeng^3*Yuting Zhou¹Kaixin Pan¹Zhiheng Xian⁴Yulun Xian⁵Licheng Liu²

• ¹School of Electronics and Information, Guangdong Polytechnic Normal University, Guangzhou, Guangdong, China
• ²School of Information Engineering, Guangdong University of Technology, Guangzhou, China
• ³School of Computer Science and Engineering, Guangdong University of Technology, Guangzhou, China
• ⁴Guangzhou Huitong Agricultural Technology Co.,Ltd., Guangzhou, China
• ⁵Guangzhou iGrowLite Agricultural Technology Co.,Ltd., Guangzhou, China

The development of automated high-throughput plant phenotyping systems with non-destructive characteristics fundamentally relies on achieving accurate segmentation of botanical structures at both semantic and instance levels. However, most existing approaches rely heavily on empirically determined threshold parameters and rarely integrate semantic and instance segmentation within a unified framework. To address these limitations, this study introduces a methodology leveraging 2D image data of real plants, i.e., Caladium bicolor, captured using a custom-designed plant cultivation platform. A high-quality 3D point cloud dataset was generated through reconstruction.Building on this foundation, we propose a streamlined Dual-Task Segmentation Network (DSN) incorporating a multi-head hierarchical attention mechanism to achieve superior segmentation performance. Also, the dual-task framework employs Multi-Value Conditional Random Field (MV-CRF) to enable semantic segmentation of stem-leaf and individual leaf identification through the DSN architecture when processing manually-annotated 3D point cloud data. The network features a dual-branch architecture: one branch predicts the semantic class of each point, while the other embeds points into a high-dimensional vector space for instance clustering. Multi-task joint optimization is facilitated through the MV-CRF model. Benchmark evaluations validate the novel framework's segmentation efficacy, yielding 99.16% macro-averaged precision, 95.73% class-wise recognition rate, and an average Intersection over Union of 93.64%, while comparative analyses confirm its superiority over nine benchmark architectures in 3D point cloud analytics.For instance segmentation, the model achieved leading metrics of 87.94%, 72.36%, and 71.61%, respectively. Furthermore, ablation studies validated the effectiveness of the network's design and substantiated the rationale behind each architectural choice.