AUTHOR=Zhu Xiaoying , Pang Guangyao , He Xi , Chen Yue , Yu Zhenming 

TITLE=A segmentation-combination data augmentation strategy and dual attention mechanism for accurate Chinese herbal medicine microscopic identification

JOURNAL=Frontiers in Plant Science

VOLUME=Volume 15 - 2024

YEAR=2024

URL=https://www.frontiersin.org/journals/plant-science/articles/10.3389/fpls.2024.1442968

DOI=10.3389/fpls.2024.1442968

ISSN=1664-462X

ABSTRACT=Chinese Herbal Medicine (CHM), with its deep-rooted history and increasing global recognition, faces significant obstacles in widespread application due to the limitations inherent in traditional microscopic identification methods. The challenges in automating Chinese herbal medicine microscopic identification are multifaceted, encompassing the scarcity of publicly accessible datasets, imbalanced class distributions, small and unevenly distributed features, and the frequent occurrence of incomplete or blurred cell structures in microscopic images. To overcome these obstacles, this paper introduces a novel deep learning-based approach for automated Chinese Herbal Medicine Microscopic Identification (CHMMI). Our CHMMI employs a segmentation-combination data augmentation strategy to expand and balance the dataset, capturing comprehensive feature sets. A shallow-deep dual attention module enhances the model's focus on relevant features across different layers, enabling effective processing of small, uneven, incomplete, and blurred features. Multi-scale inference integrates features at different scales to improve object detection and identification accuracy. The proposed CHMMI approach achieved an Average Precision (AP) of 0.841, a mean Average Precision at IoU=.50 (mAP@.5) of 0.887, a mean Average Precision at IoU from .50 to .95 (mAP@.5:.95) of 0.551, and a Matthews Correlation Coefficient of 0.898, demonstrating superior performance compared to state-of-theart methods like YOLOv5, SSD, Faster R-CNN, and ResNet. These results highlight CHMMI's potential for practical application in automating CHM microscopic identification, addressing the limitations of traditional methods, and supporting the modernization and growth of the CHM industry.