AUTHOR=Pan Dan , Liu Baijing , Luo Lin , Zeng An , Zhou Yuting , Pan Kaixin , Xian Zhiheng , Xian Yulun , Liu Licheng 

TITLE=A dual-task segmentation network based on multi-head hierarchical attention for 3D plant point cloud

JOURNAL=Frontiers in Plant Science

VOLUME=Volume 16 - 2025

YEAR=2025

URL=https://www.frontiersin.org/journals/plant-science/articles/10.3389/fpls.2025.1610443

DOI=10.3389/fpls.2025.1610443

ISSN=1664-462X

ABSTRACT=IntroductionThe development of automated high-throughput plant phenotyping systems with non-destructive characteristics fundamentally relies on achieving accurate segmentation of botanical structures at both semantic and instance levels. However, most existing approaches rely heavily on empirically determined threshold parameters and rarely integrate semantic and instance segmentation within a unified framework.MethodsTo address these limitations, this study introduces a methodology leveraging 2D image data of real plants, i.e., Caladium bicolor, captured using a custom-designed plant cultivation platform. A high-quality 3D point cloud dataset was generated through reconstruction. Building on this foundation, we propose a streamlined Dual-Task Segmentation Network (DSN) incorporating a multi-head hierarchical attention mechanism to achieve superior segmentation performance. Also, the dual-task framework employs Multi-Value Conditional Random Field (MV-CRF) to enable semantic segmentation of stem-leaf and individual leaf identification through the DSN architecture when processing manually-annotated 3D point cloud data. The network features a dual-branch architecture: one branch predicts the semantic class of each point, while the other embeds points into a high-dimensional vector space for instance clustering. Multi-task joint optimization is facilitated through the MV-CRF model.Results and discussionBenchmark evaluations validate the novel framework’s segmentation efficacy, yielding 99.16% macro-averaged precision, 95.73% class-wise recognition rate, and an average Intersection over Union of 93.64%, while comparative analyses confirm its superiority over nine benchmark architectures in 3D point cloud analytics. For instance segmentation, the model achieved leading metrics of 87.94%, 72.36%, and 71.61%, respectively. Furthermore, ablation studies validated the effectiveness of the network’s design and substantiated the rationale behind each architectural choice.