AUTHOR=Song Guozhu , Wang Jian , Ma Rongting , Shi Yan , Wang Yaqi 

TITLE=Study on the fusion of improved YOLOv8 and depth camera for bunch tomato stem picking point recognition and localization

JOURNAL=Frontiers in Plant Science

VOLUME=Volume 15 - 2024

YEAR=2024

URL=https://www.frontiersin.org/journals/plant-science/articles/10.3389/fpls.2024.1447855

DOI=10.3389/fpls.2024.1447855

ISSN=1664-462X

ABSTRACT=When picking ripe cluster tomatoes, accurately identifying specific fruiting stems is challenging as they may be obstructed by branches and leaves or have similar colors to branches, main vines, and lateral vines. Furthermore, irregularities in the growth patterns of fruiting pedicels further complicate the precise localization of picking points, thus affecting harvesting efficiency.Moreover, when fruit stalks are too short or slender, it poses an obstacle, making it impossible for the depth camera to accurately obtain depth information during depth value acquisition.To address these challenges, this paper proposes an enhanced YOLOv8 model integrated with a depth camera for string tomato fruit stalk picking point identification and localization research.Initially, the Fasternet bottleneck in YOLOv8 is replaced with the c2f bottleneck, and the MLCA attention mechanism is added after the backbone network to construct the FastMLCA-YOLOv8 model for fruit stalk recognition.Subsequently, the optimized K-means algorithm, which utilizes K-means++ for clustering center initialization and determines the optimal number of clusters via Silhouette coefficients, is employed to segment the fruit stalk region.Following this, the corrosion operation and Zhang refinement algorithm are employed to denoise the segmented fruit stalk region and extract the refined skeletal line, thus determining the coordinate position of the fruit stalk picking point in the binarized image.Finally, a secondary extraction method is employed to resolve the missing depth values, obtaining depth values and 3D coordinate information for the picking points in the RGB-D camera coordinates. The experimental results demonstrate that the algorithm can accurately identify and locate the picking points of ripe cluster tomatoes under complex background conditions. The success rate of picking point identification reaches 91.3%.Compared with the YOLOv8 model, the accuracy is improved by 2.8%, and the depth value error of the picking points is only ±2.5 mm.This research meets the requirements of string tomato picking robots for fruit stalk target detection and provides strong support for the development of string tomato picking technology.