To select the best detection box from a set of overlapping boxes, the NMS used in YOLOv5s only considers the overlapping area of the detection frame, which often leads to false suppression and reduces the effect of object detection. To solve this above problem, the research proposes adding the Soft-NMS module to realize the NMS function, which accepts the intersection ratio of two object detection candidate boxes through the Gaussian function and makes different degrees of punishment. The Soft-NMS combines different penalties to modify the confidence of the object so that the best detection box is selected from a set of overlapping boxes.

NMS is mainly used for post-processing the output of the object detection model. In the prediction stage of object detection, many candidate anchor boxes will be output, among which some prediction detection boxes are overlapped around the same object. At this time, NMS can be used to merge similar detection boxes of the same object to remove redundant detection boxes and get correct detection results. The NMS processing method can be intuitively represented by the following score reset function,

In Equation 1, where i stands for the serial number of the remaining boxes with the highest score from high to low except for the box Λ. Λ represents the maximum score of the detection boxes. Δ_i is the score of detection boxes with the slightly lower score Λ. N_t stands for the threshold. s represents the confidence score.

However, the main drawback of NMS is that it only retains the highest confidence prediction box when objects are highly overlapped, which may lead to the deletion of similar but different objects. Therefore, the Soft-NMS is proposed to optimize the selection of detection boxes:

In Equation 2, one can see that the Soft-NMS does not directly delete the remaining detection boxes with the highest confidence ratio of the detection box, but it reduces the confidence of the remaining detection boxes and retains more prediction frames (Bodla et al., 2017). In this way, the situation of mistakenly deleting overlapping objects is avoided through great suppression. Equation 3 is then derived as follows:

where P is the final set of anchor frames, and ζ is the super parameter of Soft-NMS.

When the threshold of NMS is reached, a sudden penalty will be taken and the penalty function is continuous. The continuous penalty function will take a penalty when the anchor frame is highly overlapped, and it will not take a penalty when there is no overlap (Bodla et al., 2017). In addition, when the overlap is low, the penalty will gradually increase. An example of detecting overlapping boxes is shown in Figure 2.

As shown in Figure 2, there are two detection boxes in this image, highlighted in red and purple, with respective scores of 0.95 and 0.80. Notably, the purple detection box exhibits substantial overlap with the red one. When using NMS, the choice of a threshold is critical. The threshold determines which bounding boxes are retained and which are suppressed. If the threshold is too low, too many frames may be retained, increasing false detections; if the threshold is too high, correct detections may be suppressed, leading to missed detections. Therefore, experimentally determining the optimal threshold is the key to balancing false detections and missed detections. Soft-NMS is an improvement of NMS, which does not suppress overlapping frames completely but rather reduces their scores, thus avoiding the problem of hard threshold setting.

In the traditional convolution pooling process, equal importance is typically assigned to each channel of the feature map. However, various channels have varying degrees of importance, and specific problems should be analyzed specifically in practice. To solve the loss problem stemming from the varying importance of feature map for different channels during convolution pooling, SUS-YOLOv5 incorporates SENet. This addition enhances the model’s capability to capture correlations of features and improves the representation of feature information.

The SENet introduces attention to the different dimensions of channels through key operations known as Squeeze and Excitation (Huang et al., 2022). Through automatic learning, the SENet makes the neural network focus on some feature channels. SENet can improve the channels of the feature map that are helpful for the current task, and it can suppress the unuseful feature channels for the current task. The SENet can bring a significant improvement in model performance with a slight increase in computing cost (Hu et al., 2018a, b).

Figure 3 shows the schematic diagram for the SENet module. Before the feature map from the backbone network is input into the SENet attention module. Before adding SENet attention, all channels in the feature map are treated equally despite of importance. However, after adding the SENet, the importance of individual feature channels becomes different. Different colors stands for different weights, which makes the neural network focus on some channels which have large weight values.

In Figure 4, X is the input feature map, and F_ex represents the transformation of the feature map. U stands for C feature maps with H × W, and X ˜ represents the feature map scaled by the activation function. F_sq represents the Squeeze operation, that is, as shown in Equation 4:

where z_c represents the output operated by F_sq , the subscript c represents the channel. u_c stands for the cth two-dimensional matrix of U. F_ex in Figure 4 is the Excitation operation that corresponds to the two fully connected operations, namely, as shown in Equation 5:

where s is the output operated by F_ex . δ and σ represent the activation function of ReLU and Sigmoid, respectively.

After obtaining s, SENet operates F_scale , that is, by the channel product of s_c and u_c the output feature can be obtained, as shown in Equation 6:

As the depth of network layers becomes deeper, the feature information will be lost to some extent. Therefore, multi-scale feature fusion is widely used in detection networks to improve the detection performance of networks. At present, the main feature fusion structures commonly include Feature Pyramid Networks (FPN) (Lin et al., 2017), Path Aggregation Network (Liu et al., 2018), and Bidirectional Feature Pyramid Network (Tan et al., 2020). FPN is a top-down feature pyramid structure, which is limited by unidirectional feature information transmission although it combines deep and shallow layers of features as well as multi-scale features. The Path Aggregation Network adds a bottom-up secondary fusion path based on FPN (Tan et al., 2020). BiFPN is a weighted bi-directional feature pyramid network, which realizes the simple and fast multi-scale feature fusion. To reduce the loss of feature information and improve the detection accuracy of objects, this research proposes a SE-BiFPN network structure, which combines the SENet and BiFPN structures, whose diagrammatic sketch is shown in Figure 5.

Firstly, multiscale feature maps are derived from the SENet layer of the backbone network. Secondly, the first feature fusion is achieved with the lower sampling layer through a transverse connection, and the second feature fusion is achieved with the lower sampling layer and the upper sampling layer of the same scale of feature maps through a skip connection. Then, the feature map is obtained after these multi-scale fusions. Take the example of the fourth channel, the intermediate feature map and the final feature map are output as follows.

In Equations 7 and 8, ∂ ( g ) denotes the convolution operation on g. R(g) stands for an up-sampling or down-sampling operation on g. O 4 t d represents the intermediate feature map between O 4 i n and O 4 o u t . ω and ε respectively represent the learnable weight and a preset small positive value to avoid numerical instability, which usually are set to 0.0001. The weights ω are dynamically adjusted by the optimization algorithm during the training process, and their initial values are usually randomly generated and updated by back-propagation of the loss function. The purpose of presetting a small positive value ε is to prevent numerical instability caused by manipulating 0 or infinity during numerical calculations. We choose 0.0001 as the value of ε, which is an empirical value widely used in numerous experiments and is sufficient to cope with the need for numerical stability in most cases, without significantly affecting the performance of the model.

The SE-BiFPN network enhances the fusion ability of image shallow feature information and deep feature information. Each new feature layer of information is processed by SENet, so the SE-BiFPN can play a core role in the fusion of global and local information between the deep layers and shallow layers. In conclusion, the SE-BiFPN network structure enhances the expression ability of the feature pyramid and enables the model to achieve optimal feature fusion. After the SENet processing, the feature map in the backbone network of SUS-YOLOv5 is input into the SE-BiFPN structure multiple times to realize bidirectional multi-scale feature fusion, which improves the learning ability of the model on the whole feature and reduces the missing rate.

The SUS-YOLOv5 integrates the DeepSORT algorithm to achieve multi-object behavior tracking of sea cucumbers, and the schematic diagram of MO-Tracking is shown in Figure 6.

In Figure 7, the DeepSORT algorithm is on the basis of the SORT algorithm (Bewley et al., 2016) and introduces a re-identification model, which reduces the frequency of identity document (ID) switching by the appearance information and the movement information. In addition, DeepSORT combines the Kalman filter for movement prediction and the Hungarian algorithm for data association to solve the multi-object tracking problems.

The video of sea cucumbers is input into SUS-YOLOv5. After the object detector detects the sea cucumber objects, the DeepSORT algorithm can match the sea cucumber objects between the front and back frames based on the Kalman filter and the Hungarian algorithm, and it can also continuously track the object. When tracking the object, the movement model established by DeepSORT will calculate the Mahalanobis distance between the detection box and the filter prediction box, the distance is expressed as:

where d_j stands for the distance of the jth detection box, y_i is the predicted position of the i tracking object, and s_i stands for the covariance matrix between the detection position and the predicted position. d⁽¹⁾(i,j) denotes the Mahalanobis distance between the Kalman prediction result and the detection result of the motion state of the existing moving object.

When the uncertainty of the object motion is very low, the Mahalanobis distance represented by Equation 9 is a suitable correlation measure method. However, when the camera is moving, the correlation method of the Mahalanobis distance will fail, which results in the phenomenon of ID switch. For solving this problem, we propose to use the following second kind of correlation method to identify a feature vector for each detection block d_j , which is appointed by the DeepSORT.

where r_j is an appearance descriptor with the restriction condition ||r_j ||=1. For each track frame k, the algorithm preserves the set ∂ k of the last L_k =100 related to appearance descriptors, namely ∂ k = { r k ( i ) } k = 1 L k . d⁽²⁾(i,j) is the minimal cosine distance of the ith track and jth detection.

The above two kinds of metrics between motion features and appearance features can complement each other in matching problems. On one hand, the metric of the Mahalanobis distance to motion characteristics is helpful for short-term predictions. On the other hand, the cosine distance considering appearance features is a very useful metric for ID recovery after prolonged occlusion. To describe the correlation problem, the above two kinds of metrics from Equations 9 and 10 are weighted and summed as shown in Equation 11:

where c_i,j stands for the weighted sum of the Mahalanobis distance and the cosine distance. λ is the hyperparameter that affects the association effect.

The experiment is conducted on an experimental platform equipped with Intel (R) Core (TM) i7-6800K CPU @ 3.40GHz (64G RAM) and NVIDIA GeForce RTX 2080Ti 48G. We use CUDA 11.1.0, CUDNN 11.1, and Python 3.8.8 as the configuration environment of software. Table 2 shows the detail training parameters.

The loss function in YOLOv5s is employed to measure the discrepancy between the model’s predictions and the actual ground truth, and it comprises three losses: object loss, detection box loss and classification loss. To avoid overfitting and underfitting during the training process, the loss values are monitored for the training and validation sets and the best model is selected based on these values.

The Intersection over Union (IoU) metric is often employed to appraise the accuracy of the model’s prediction. IoU gives the overlap ratio between the predicted detection boxes and the ground truth detection boxes. Using this metric, the values of precision (P) and recall (R) can be computed based on the model’s predictions. The expressions for R, P, and loss are shown in Equations 12–14.

where l_bbox , l_object and l_{classification} represent detection box loss, object loss, and classification loss, respectively. TP presents the number of positive samples correctly predicted as positive samples, FP is the number of negative samples incorrectly predicted as positive samples, and FN stands for the number of positive samples incorrectly predicted as negative samples.

The Average Precision (AP) serves as an evaluative metric for assessing the accuracy of an detection model for object. AP is determined by calculating the area under the Precision-Recall (P-R) curve, which is generated by plotting precision and recall values across various threshold levels of the model’s confidence scores. AP is the average value of precision at different recall levels, and it represents the ability of the model to correctly detect objects and avoid false positives. The mAP is the average of the AP values calculated for each category in the dataset. the mAP is a commonly used evaluation metric for object detection models, as it provides a single performance score that takes into account the detection accuracy for all object categories in the dataset. A higher mAP value indicates better performance of the model. In summary, AP and mAP are important evaluation metrics for object detection models, and they are computed based on the P-R curve. They are defined as shown in Equations 15 and 16:

where Q_R stands for the number of categories.

The training results of the model is shown in Figure 9, including the training loss curve, verification loss curve, mAP@0.5:0.95 curve and mAP@0.5 curve. From Figures 9A, B, one can see that the loss function value of training and verification of SUS-YOLOv5 gradually decreases with the increase of the number of iterations and finally stabilizes at a value close to 0. In this process, no obvious overfitting phenomenon occurs in the model. From Figures 9C, D, one can see that the mAP@0.5 and mAP@0.5:0.95 indicators of SUS-YOLOv5 have obvious advantages over other models, and the detection performance of object has been improved.

In the experimental process, several methods for improving detection performance are tested and their results are compared in Table 3.

From Table 3, the proposed SUS-YOLOv5 has the highest accuracy whose mAP@0.5 is 0.954 and the mAP@0.5:0.95 is 0.838, which are 3.3% and 4.1% higher than the original YOLOv5s, respectively. Although the SUS-YOLOv5 model is more relatively complex than the original YOLOv5s, the size of the SUS-YOLOv5 model only increases by 0.4 MB compared with 14.1 MB of the YOLOv5s. The above results demonstrate that that SUS-YOLOv5 achieves a favorable balance between model size and detection accuracy.

The detection results are shown in Figure 10 by using YOLOv5s and SUS-YOLOv5 on the sea cucumber dataset, respectively. To facilitate an understanding of the detection performance of the models, this research visualizes the 80 × 80 detection layer of the models by using heatmaps. Heatmaps are usually used to highlight key areas with colors of different depths in object detection. Generally, the brighter the color of the heatmap, the more confident the model is in detecting the object.

From Figure 10A, it can be seen that the YOLOv5s misses an object in the upper right corner, whose anchor box has a lower score. In contrast, from Figure 10B, it can be seen that SUS-YOLOv5 successfully detects all sea cucumber objects and obtains higher confidence scores. In addition, the SUS-YOLOv5 has higher brightness than YOLOv5s in the object area, which indicates that SUS-YOLOv5 has a higher focus on the object area.

To further compare the performances of YOLOv5s and SUS-YOLOv5, the P-R curves are given in the training process of the sea cucumber object detection, shown in Figure 11.

From Figure 11, it can be observed that the SUS-YOLOv5 appears higher detection accuracy for sea cucumbers infected with Vibrio alginolyticus compared to YOLOv5s. Additionally, the improvements made on YOLOv5s are effective because the SENet attention mechanism, optimized Non-Maximum Suppression algorithm, and enhanced feature fusion capability improve the overall performance of the model.

To access the performance of SUS-YOLOv5, this research compares it with traditional object detection models, including YOLOv3, YOLOv4, SSD, Faster-RCNN, EfficientDet, and YOLOv5s. The results of the comparison are shown in Table 4. From Table 4, one can see that the mAP@0.5 of SUS-YOLOv5 achieves 95.40% among the one-stage object detection algorithms, which is not only higher than other YOLO series algorithms but also 6.30% and 10.94% higher than mAP@0.5 of SSD and EfficientDet, respectively. The accuracy of SUS-YOLOv5 can meet the requirements of sea cucumber detection.

To make the performances of different detection algorithms more visual and clear, Figure 12 shows the comparison results of performance for different detection algorithms in the form of a histogram.

As seen in the above figure, the mAP@0.5 of Faster-RCNN is the highest in the seven models and it is 0.06% higher than that of SUS-YOLOv5, but the model size of SUS-YOLOv5 reduces 86.94% than that of Faster-RCNN. As we all know, the large size of the model makes it difficult to deploy the model in embedded devices. As shown in Figure 12, the processing time of SUS-YOLOv5 is only 0.016s for a single image, satisfying the real-time detection requirement for sea cucumber objects. On the other hand, the P and R of SUS-YOLOv5 both increase by 1.80% compared with those before the improvement. In summary, the SUS-YOLOv5 achieves a good balance among detection accuracy, detection speed, and model size, which can lay a good foundation for the following sea cucumber object tracking task.

This research analyzes the behavioral changes of sea cucumbers in the early stage of suffering from SUS disease to observe their behavior within one hour after immersion by setting different experiment groups (normal environment, the low concentration of Vibrio alginolyticus, and the high concentration of Vibrio alginolyticus). In the sea cucumber object tracking experiment, the MO-Tracking algorithm combines the ideas of cascade matching and object re-identification to effectively avoid the object ID loss problem and the ID switching problem caused by object occlusion. The ID identification results of the MO-Tracking algorithm are shown in Figure 13.

From Figure 13, one can see that the MO-Tracking algorithm assigns an ID to each sea cucumber and accurately locks the identity of each sea cucumber based on the object’s ID, while the corresponding coordinate information is continuously output.

After obtaining the coordinate information of sea cucumber objects by using the MO-Tracking algorithm, the pixel coordinates are transformed proportionally to obtain the actual coordinates of sea cucumbers in the 40 ×40 ×40 cm³ fish tank. Table 5 shows the detailed statistical results of sea cucumber movement quantity in the three experiments.

To more intuitively reflect the movement status of sea cucumbers, the trajectories of movement for sea cucumbers are plotted based on the obtained coordinates of the objects in the three groups. Figure 14 shows the results.

Combining Table 5 and Figure 14, we can conduct a detailed analysis of the movement status of sea cucumbers within one hour after the experiment. Figure 14A shows the trajectories of the sea cucumbers in group 1 (normal environment). From Figure 14A, it can be seen that: besides ID 3 which shows obvious movement, the other three sea cucumbers stay stationary near their original locations in the normal environment. According to Table 6, the movement quantity of ID 3 is 63.67 cm, the mean movement quantity of the other three sea cucumbers is 22.99 cm, and the overall mean movement quantity is 33.17 cm. Figure 14B shows the trajectories of sea cucumbers in group 2 (low concentration of Vibrio alginolyticus). ID 3 and ID 2 are the ones with the stronger reaction, and the movement quantity reaches 81.21 cm and 77.38 cm, respectively, while ID 1 has the least strong reaction. The overall mean movement quantity is 56.48 cm in group 2, which has an increase of 23.31 cm compared to that in the normal environment. Figure 14C shows the behavior trajectories of sea cucumbers in group 3 (high concentration of Vibrio alginolyticus). It can be seen that the movement quantity of sea cucumbers increases significantly and the movement quantity of ID 1 is the largest which reaches 135.44 cm. The movement quantity of ID 3 increases by 40.83 cm and 58.37 cm compared with that in group 2 and group 1, respectively. The mean movement quantity of sea cucumbers reaches 94.36 cm in group 3, which is obviously higher than those of the other two groups.

In the whole experiment, sea cucumbers do not appear a strong reaction or death in low concentrations of Vibrio alginolyticus, and they only appear with some symptoms such as shaking their heads and spitting out their guts. To further explore infection in the situation of sea cucumbers in different concentrations of Vibrio alginolyticus, the mean movement quantity of sea cucumbers is calculated in the first three hours of each experiment, as shown in Table 6.

In Table 6, A, B, and C represent the movement quantity in the first, second, and third hours, respectively, and D represents the mean movement quantity in the first three hours. From Table 6, it can be seen that ID 1 and ID 4 in group 2 do not appear significant differences in mean movement quantity compared to sea cucumbers in group 1. The occurrence of this phenomenon may be because of the concentration of Vibrio alginolyticus is low and the bacterial inoculation adopts the immersion infection method in the experiment, so that the sea cucumbers may be incompletely infected with Vibrio alginolyticus. When sea cucumbers are exposed to high concentrations of Vibrio alginolyticus, their reactions become more fidgety, and more symptoms occur, such as shaking their heads, blackening their tentacles, spitting out their guts, fully extending their trunk, and floating their heads. On the 5th day of these experiments, the high-concentration group of sea cucumbers begins to appear dead.

According to the analysis for the experimental results, sea cucumbers become irritable in the early stage of infection with Vibrio alginolyticus, and the movement quantity increases significantly compared with the normal environment. Therefore, in the early infection stages of sea cucumbers with Vibrio alginolyticus, the increase in the movement quantity of sea cucumbers can be used as one of the characteristics to determine whether the sea cucumber is diseased.

According to the experimental results of sea cucumbers infection with Vibrio alginolyticus, one can classify the infection level into three categories: low infection level (mean movement quantity ranging from 25 cm/h to 45 cm/h, but not including 45 cm/h), moderate infection level (mean movement quantity ranging from 45 cm/h to 85 cm/h, but not including 85 cm/h), and severe infection level (mean movement quantity greater than 85 cm/h). We can judge the infection level by the movement quantity of sea cucumber, then a alarm of the infection level can be provided to facilitate the appropriate measures to be taken.