In-field estimation of vertical distribution of total nitrogen and nicotine content for tobacco plants based on multispectral and texture feature fusion

Wenwu Liu¹Weimin Guo¹Junying Li²Yanling Zhang¹Hanping Zhou¹Aiguo Wang¹Yuxin Hou¹Qi Guo¹Qiang Xu^1*Xuan Song^3*

• ¹Zhengzhou Tobacco Research Institute of CNTC, Zhengzhou, China
• ²Pingdingshan Branch of Henan Provincial Tobacco Company, Pingdingshan, China
• ³Zhengzhou University, Zhengzhou, China

To obtain the total nitrogen and nicotine content of tobacco plants and their vertical distribution within canopy in the field is beneficial for smart management and quality assessment. The complex environment and uneven vertical distribution of these components bring great challenges for accurate estimation. Therefore, this study propose a precise leaf segmentation method combining a deep learning-based spectral and texture feature fusion method for estimation accuracy improvement of total nitrogen and nicotine content of tobacco. To accurately extract leaf region features, the improved YOLOv8 model (AO-YOLOv8) was proposed for instance segmentation of the tobacco leaves. After segmentation, the average spectral features from six channels of the leaf images were extracted, and 474 texture features were obtained using five methods: Gray Level Co-occurrence Matrix (GLCM), Local Binary Pattern (LBP), Fourier Transform, Gabor Filter, and Wavelet Transform. Four deep neural network models, including LSTM, RNN, MLP, and FCNN, were then used to establish total nitrogen and nicotine estimation models based on the fusion of multispectral and texture data at both the leaf and plant scales. The results showed that: (i) The AO-YOLOv8 model achieved an mAP50 of 87.3 and an mIoU of 83.4 in the leaf instance segmentation task, improving by 6.99% and 8.88%, respectively, compared to the original YOLOv8 model, and can better detect and separate the overlapping leaves under complex environment; (ii) The fusion of spectral and texture data contributed to an accuracy increase of models for predicting total nitrogen and nicotine content, with the LSTM network achieving the highest accuracy for both total nitrogen and nicotine predictions. In the laboratory environment, the R² of the total nitrogen and nicotine prediction models for leaf samples was 0.8634 and 0.8735, respectively; (iii) In the field environment, the R² for the total nitrogen and nicotine estimation models based on LSTM network for tobacco plants was 0.6771 and 0.5735, respectively, and improved compared to single spectral feature. This approach enabled the accurate estimation and visualization of nitrogen and nicotine content vertical distribution in field tobacco plants, providing an efficient, low-cost, non-destructive detection solution for tobacco leaf production and quality control.