The experiment was conducted in 2018, 2019, and 2022 at the Korla Experimental Station of the Institute of Plant Protection, Chinese Academy of Agricultural Sciences (41°44′59″N, 85°48′30″E). The experimental station is located in Heshilike Township, Korla City, Bayingol Mongolian Autonomous Prefecture, Xinjiang, China, which is located in the central part of Xinjiang and on the northeastern edge of the Tarim Basin, near the Tianshan Branch to the north and the Taklimakan Desert to the south. Cotton is the main crop grown in this area, and is cultivated with large-scale and simple cropping structures. A. gossypii is the main cotton pest in the region, and its peak season occurs from late June to early July (Lu et al., 2022). Experimental plots with severe occurrence of A. gossypii were selected in the field for data acquisition. No pesticides were applied to suppress the population growth of this pest during the experiment. The cotton crops selected for the test were the experimental cultivars ‘Zhongmiansuo49’ and ‘Xinluzhong66’ from the Cotton Insect Group of the Institute of Plant Protection, Chinese Academy of Agricultural Sciences. The cotton was sown in mid to late April with a film mulching cultivation mode, and with spot sowing on the film. Standard water and fertilizer management was carried out through drip irrigation under the film.

Cotton image data were collected at the Korla Experimental Station of the Institute of Plant Protection, Chinese Academy of Agricultural Sciences in 2018, 2019 and 2022 ( Table 1 ). The collection dates for 2018 and 2019 were from late June to mid-July, and the collection dates for 2022 were from early June to early July. The collections were made on sunny days and with low light intensity to avoid image overexposure. To allow the model to learn more features of A. gossypii infestation during training, multiple smartphones were used to acquire the cotton images. Image data acquisition was conducted in 2018 and 2019 with the HUAWEI Nova, OnePlus7pro, iPhone 8 Plus, and Mi Note 3 smartphones and in 2022 with the iPhone 8 Plus, iPhone 12, iPhone 13, iPhone XR, and Redmi 5 Plus smartphones. The image acquisition method was overhead vertical shooting. The researchers stood next to the cotton plants with mobile equipment in hand and vertically shot images of the cotton seedlings from a vantage of 1.2-1.5 meters. The data acquired in 2018 and 2019 were used for training, validation, and testing of the A. gossypii infestation recognition model, and the cotton images collected all featured the ‘Zhongmiansuo49’ cultivar. In 2022, in addition to photographing plants of the ‘Zhongmiansuo49’ cultivar, some images of plants from the ‘Xinluzhong66’ cultivar were also collected for testing the A. gossypii infestation level recognition model detection capabilities on images of other cultivars.

Directly inputting original images into a model for training can interfere with training by causing problems such as taking up a large amount of memory on the device, slowing down the training speed of the model, and causing memory overflow. Therefore, the original images must be preprocessed. When inputting images into the model, the image formats are adjusted to a certain size. To prevent soil and crop backgrounds from interfering with object detection, the original image is cropped to remove redundant information such as soil from the image. To crop the images, we first started from the center of each original cotton image, cropping the image to 3000×3000 and uniformly adjusting the image resolution to 1024×1024. Then the data were annotated. The data preprocessing process of this paper is shown in Figure 1 .

Because the training data of the object detection model need to be manually labeled, LabelImg (Tzutalin, 2015) was chosen as the image labeling tool in this experiment. Examples of acquired images for this experiment are shown in Figure 2 . When labeling the image data, the A. gossypii infestation grading standard referred to the national grading standard (GB/T 15799-2011, 2011) ( Table 2 ). Since the level of infestation suffered by cotton leaves at the seedling stage rarely reaches level 4, individual level 4s were classified as level 3s when conducting data annotation to avoid serious imbalance in the data set. The annotation entailed using rectangular boxes to annotate the cotton images, and according to the grading standard in Table 2 , the leaves in the central region of the cotton plant were annotated as level 0, level 1, level 2 and level 3 according to their different morphological characteristics, and the generated annotation files were all saved as XML files in PASCAL VOC format. The annotated images included the images collected by the various collection devices in 2018 and 2019, and a total of 3051 ‘Zhongmiansuo49’ cotton images were annotated, including 295 from the HUAWEI Nova, 1237 from the OnePlus7pro, 1270 from the iPhone 8 Plus and 249 from the Mi Note 3.

To balance the number of labels, we also performed a left-right flip operation on 188 annotated images from the iPhone 8 Plus and 151 annotated images from the Mi Note 3, resulting in 3390 annotated images. After data enhancement by flipping up and down, adding Gaussian noise and changing image brightness, the number of images was 16,950, and 16,950 annotated images were used as the data set for this experiment. The data set was first divided into a training validation set and a test set at a ratio of 9:1, and then the training validation set was divided into a training set and a validation set at a ratio of 9:1. The final training set was 13,729, the validation set was 1,526, the test set was 1,695, and the number of labels for each infestation level was 10,554 for level 0, 10,854 for level 1, 11,066 for level 2, and 11,025 for level 3.

In this study, three classical object detection models, Faster R-CNN, YOLOv5 and SSD, were chosen to conduct the study of A. gossypii infestation level recognition.

The structure of the Faster R-CNN (Ren et al., 2015) model is shown in Figure 3 . The model first used the feature extraction network to extract the feature map of each input cotton image, which was shared by the subsequent RPN with the Fast R-CNN network. RPN performs the binary classification task through a softmax classifier, determines whether the anchor belongs to the foreground or background, and obtains the candidate box position through anchor regression. The Fast R-CNN synthesizes the information from the feature maps and candidate boxes, determines the category to which the foreground belongs, and generates the exact location of the final detection box.

The structure of the YOLOv5 (Jocher, 2020) model is shown in Figure 4 . The network structure of the model is divided into four parts according to the processing stage: input, backbone, neck and prediction. The input part completes basic processing tasks such as data enhancement, adaptive image scaling and anchor box calculation. The backbone network mainly uses a common spatial pattern (CSP) structure to extract the main information from the input samples for use in subsequent stages. The neck part uses feature pyramid network (FPN) and path aggregation network (PAN) structures and uses the information extracted from the backbone to enhance feature fusion. The prediction component makes predictions and calculates the value of each loss. YOLOv5 has four model styles, s, m, l and x. They have the same network structure, and only the depth and width of the models are different.

The structure of the SSD (Liu et al., 2016) model is shown in Figure 5 . The model inputted cotton images into the backbone network, which could obtain feature maps of different sizes from the pretrained base network, and feature maps of six convolutional layers of different sizes, Conv4_3, Conv7, Conv8_2, Conv9_2, Conv10_2, and Conv11_2, were the output. Six default candidate boxes with different aspect ratios were constructed from each pixel point of these feature maps and then detected and classified separately to generate multiple initial eligible default candidate boxes. Finally, the nonmaximum suppression method was used to screen out the eligible candidate boxes to generate the final set of detected boxes, that is, the A. gossypii infestation level.

On the basis of the construction model, the hardware environment was as follows: graphics processor, NVIDIA A 100-PCIE-40 GB; CUDA Cores, 6912; total memory, 4060 MB; and memory interface, 5120-bit. The software environment included PyCharm (2020.3, JetBrains, Prague, Czech), Linux (Ubuntu 20.04.4 LTS, Linus Benedict Torvalds, Helsinki, Suomi), Python (Python 3.8.12, Python Software Foundation, State of Delaware, USA), PyTorch (PyTorch 1.10.2 or PyTorch 1.10.0+CUDA 11.3, Facebook AI Research, California, USA), and Android Studio (2021.2.1, Google, California, USA).

To quantitatively analyze the performance of a detection algorithm, researchers have formulated many evaluation metrics, each reflecting different aspects of the performance to some extent. The object detection performance evaluation metrics used in this experiment were Precision (Yang, 1999). Recall (Yang, 1999), Accuracy, Average Precision (AP), mean Average Precision (mAP), and Frames Per Second (FPS).

For each category, a curve can be drawn according to the precision and recall rate, and the AP value is the area under the curve. The mAP value is the average value of AP for each category. The classification and localization ability of the object detection model is its main performance representation, and the mAP value is its most intuitive expression. The larger the mAP value is, the higher the precision of the model; the detection speed represents the computational performance of the object detection model and is represented by the FPS value, and the larger the FPS value is, the better the detection speed of the algorithm model (Xu et al., 2021).

The formula for calculating each evaluation metric is shown below:

Where P is precision, R is recall, TP is positive samples correctly predicted as positive samples, FP is negative samples incorrectly predicted as positive samples, FN is positive samples incorrectly predicted as negative samples, TN is negative samples correctly predicted as negative samples, n is the number of categories, frameNum is number of images, and elapsedTime is detection time.

In this experiment, three models, Faster R-CNN, YOLOv5 and SSD, were selected, and the same data set was used with each model. Since the width and depth of Yolov5x model were the largest among the four model styles, x model was chosen here to participate in the comparison test. Before training, the image resolution was uniformly set to 512×512, the learning rate was set to 0.0005, the iteration rounds were set to 300 rounds, and the batch size of each iteration was set to 16. The training model was saved, and then the model was tested and evaluated with the test set. The AP, mAP, and FPS obtained for the three models’ tests are shown in Table 3 .

As shown in Table 3 , it can be seen that the mAP value after testing was 87.4% for the Faster R-CNN model, 95.7% for the YOLOv5 model, and 61.5% for the SSD model. Through the test results, it was revealed that the mAP value of the YOLOv5 model for recognizing A. gossypii infestation levels was higher than those of the Faster R-CNN and SSD models, which indicates that this model has the highest precision. The YOLOv5 model took the least amount of time to train, at 15.62 hours, under the same conditions that were used with all three models, while the Faster R-CNN and SSD models took 34.32 hours and 33.24 hours to train, respectively. It can also be seen from Table 3 that the YOLOv5 model had the fastest detection speed, with an FPS value of 61.73, which was much faster than the other two models, indicating that this model has the best detection speed.

In summary, the test results showed that the YOLOv5 model requires the shortest training time and has the highest mAP value and the fastest detection speed, with a mAP value of 95.7% and an FPS value of 61.73. Therefore, this model has the best performance.

After the comparison of the three models, it was found that the best performance at an image resolution of 512×512 was the YOLOv5 model. To verify the effect of different image resolutions on model performance, two model styles, s and x, in the YOLOv5 model were selected to study the performance of the model at image resolutions of 1024×1024, 640×640, 512×512, 256×256, and 128×128.

The data set used in this experiment was the same as the data set used in the construction of the three models, the learning rate was uniformly set to 0.0005, the number of iterations rounds was set to 10,000, the batch size of each iteration was set to 16, and the image resolution was set appropriately for each according to the requirements of the experiment before training. There is an early stop mechanism in the YOLOv5 model; that is, after a certain quantity of training iteration rounds, if the model effectiveness has not improved, then the model training is stopped early. The patience parameter set in this experiment was 100; that is, during the model training, training was stopped if the model effectiveness did not improve within 100 consecutive rounds. The training models with different image resolutions were saved. The training results for YOLOv5x and YOLOv5s are shown in Figures 6 , 7 . Then, the training models for each image resolution were evaluated with the test set. The test results obtained are shown in Table 4 .

As shown in Figure 6 , the convergence speed of the YOLOv5x model was the slowest when the image resolution was 128×128, and the convergence speed for several other image resolutions did not differ much. When the image resolution was 1024×1024, the mAP value of the validation set reached 96%, which was the highest mAP value among the five image resolutions, indicating that this was the resolution at which the model had the best train effect.

As shown in Figure 7 , the convergence speed of the YOLOv5s model was the slowest when the image resolution was 128×128, and the convergence speed of the remaining image resolutions was relatively similar. When the image resolution was 640×640, the mAP value of the validation set reached 97.1%, which was the highest mAP value among the five image resolutions, indicating that this was the resolution at which the model had the best train effect.

Comparing Figures 6 , 7 , it can be seen that the convergence speed of the YOLOv5s model was significantly lower than that of the YOLOv5x model. Except for at the 128×128 image resolution in the YOLOv5x model, the YOLOv5s model generally had more training rounds than the YOLOv5x model for all image resolutions.

As shown in Table 4 , it can be seen that in the YOLOv5x model, when the image resolution was 1024×1024, the model took the longest time from training to stopping, at 141.55 hours, and the test set had the highest mAP value, at 95.9% but the lowest FPS value, at 34.65. When the image resolution was 256×256, the model took the least amount of time from training to stopping, at 33.48 hours, and the test set had the lowest mAP value, at 93.7%. When the image resolution was 128×128, the test set had the highest FPS value, at 81.22. Overall, the performance was best when the image resolution was 640×640, as the mAP value for that image resolution was 0.1% lower than the highest mAP value, which was for an image resolution of 1024×1024, but the FPS value was 17.17 higher. In the YOLOv5s model, when the image resolution was 640×640, the test set had the highest mAP value, at 96.8%. When the image resolution was 128×128, the model took the longest time from training to stopping, at 132.21 hours, and the test set had the lowest mAP value, at 91.6%, but the highest FPS value, at 92.32. When the image resolution was 1024×1024, the test set had the lowest FPS value, at 62.71. When the image resolution was 512×512, the model took the least amount of time from training to stopping, at 52.16 hours. Overall, the performance was best when the image resolution was 640×640 because although the FPS value for this image resolution was not the highest, the mAP value was the highest, at 96.8%.

Comparing the YOLOv5x and YOLOv5s models, as seen in Table 4 , the YOLOv5s model performed better than the YOLOv5x model, both in terms of mAP values and FPS values. The model performed best when the image resolution was 640×640 with a mAP value of 96.8% and an FPS value of 71.43.

The best performance of YOLOv5s was found in the previous study comparison, and its authors updated YOLOv8 version in early 2023, so this paper conducts a supplementary experiment to compare the YOLOv5s model in this paper with the latest YOLOv8s model, and its comparison results are shown in Table 5 .

As shown in Table 5 , the YOLOv5s model achieves the mAP value of 96.8% and the FPS value of 71.43 on the test set. The YOLOv8s model achieved the mAP value of 97.5% and the FPS value of 67.75 on the test set. Compared to the YOLOv5s model, the mAP value of YOLOv8s improved by 0.7% and the FPS value decreased by 3.68. Although YOLOv8s improved 0.7 percentage points in detection precision over YOLOv5s, the detection speed dropped by 3.68. So all in all, YOLOv5s performed better than YOLOv8s.

To facilitate real-time detection in the field, the YOLOv5 model with the best test results was deployed to Android mobile. Based on the test results presented in Table 4 , we chose to deploy the trained YOLOv5s models of each image resolution in Figure 7 to the Android mobile platform one by one and compare their detection effects. The model files saved after training were first converted into the corresponding files, then the code was debugged in Android studio and finally deployed to the Android mobile platform, and the test machine used in this experiment was a Redmi 5 Plus (Android Version:8.1.0; GPU Model: Adreno 506; Operating Memory:3GB; Storage Capacity:32GB). After deployment, the Android Package Kit (APK) for each resolution had an application size of 1.62 GB and required 16.38 KB of user data.

In this experiment, 20 original images of ‘Zhongmiansuo49’ cultivar plants from the image data collected in 2018 and 2019 were reselected for mobile platform detection, and the results of the mobile platform detection for different image resolutions are shown in Figure 8 ; Table 6 . The total number of leaves detected at different resolutions were between 35 and 50. The confusion matrix indicates whether the model confounds different categories by comparing the actual infestation grades of the blades in the mobile platform test data with the predicted infestation grades. As shown in Table 6 , when the image resolution was 256×256, the accuracy of the mobile platform was the highest, at 81.0%, and the detection speed was also the fastest, with an FPS value of 6.98. Therefore, the detection of the mobile platform is the best at this image resolution.

Separately selecting 20 cotton images from the ‘Zhongmiansuo49’ and ‘Xinluzhong66’ cultivars from the new data collected in 2022, the accuracy and usefulness of the YOLOv5 model, which had been successfully deployed on the Android platform, were evaluated. The test results are shown in Figure 9 ; Table 7 , and an example of the detection results is shown in Figure 10 .

By comparing the actual infestation levels and predicted infestation levels of the leaves in the mobile platform test data ( Figure 9 ), it can be seen that the adjacent levels are easily confused with each other, and that level 3 is confused the least, probably because the leaf features at levels 0 and 1 are similar, and that the curl of the leaves at level 3 is very obvious and easy to judge. As presented in Table 7 , the mobile-side detection accuracy for infested leaves of the ‘Zhongmiansuo49’ cultivar was 84.6%, and the detection speed FPS value was 8.61; the mobile-side detection accuracy for infested leaves of the ‘Xinluzhong66’ cultivar was 85.2%, and the detection speed FPS value was 8.19. As shown in Table 7 , the accuracy and usefulness of the YOLOv5 model deployed on the Android platform are relatively ideal and can provide a more convenient and faster means for field investigators to use the model.