Given a leaf disease dataset D = { ( x i , y i c , y i s ) } i = 1 N containing N images, where x i ∈ ℝ C × H × W is the i-th leaf image with C, H, and W denoting the number of channels, height, and width of the image, respectively. Each image is labeled with two types of annotations: y i c ∈ { 1 , 2 , · · · , K c is the disease category label, with K^c being the number of disease categories, and y i s ∈ { 1 , 2 , · · · , K s } is the disease degree label, with K^s being the number of severity levels.

In MTDL, there are three basic tasks denoted as T c for disease category recognition, T s for severity estimation, and T h for the hybrid task that jointly performs T c and T s . As shown in Figure 2 , each task uses a standard ResNet50 (He et al., 2016) as the backbone for feature extraction. In particular, the two single tasks T c and T s , each uses a multi-layer perceptron (MLP) to output the logits of its corresponding task, denoted as z i c ∈   ℝ K c and z i s ∈   ℝ K s , respectively. For T h , two separate MLPs are used to perform two tasks simultaneously on a shared backbone, and the output is denoted as z i h = [ z i h c : z i h s ] ∈   ℝ K c + K s , where z i h c and z i h s corresponding to the logits for the disease category and severity, respectively. Usually, a softmax function is applied to the output of each task to produce the predicted probabilities, p i c ∈   ℝ K c , p i s ∈   ℝ K s and p i h = [ p i h c : p i h s ] ∈   ℝ K c + K s , respectively. Guided by these three basic tasks, MTDL employs a designed knowledge routing mechanism to build a tomato disease diagnosis model. The process begins with the distillation of multi-task knowledge from T h back to the corresponding task models T c and T s (as shown in Figure 2A ). These two models then engage in mutual learning (as shown in Figure 2B ). Finally, the knowledge from these two models is integrated to output an enhanced multi-task model, namely T ′   h (as shown in Figure 2C ). The detailed learning process is described in the following sections, including, knowledge decomposition (Section 2.3.2), mutual knowledge tranfer (Section 2.3.3), and knowledge integration (Section 2.3.4).

Multi-task learning has demonstrated its advantages in leveraging shared information among related tasks to improve performance on individual tasks. However, directly training a multi-tasking model can be suboptimal, as the tasks may have different levels of difficulty. For instance, the task of severity estimation is more challenging than the leaf disease classification task because it typically necessitates a finer analysis of the leaf and disease spot attributes (Wang et al., 2017). Therefore, given a multi-task model T h pre-trained on dataset D, as shown in Figure 2A , it is reasonable to disentangle the shared knowledge and transfer it back to the single-task models, i.e., T c and T s , using knowledge distillation (Hinton et al., 2015). Specifically, when distilling knowledge from T h to T c , we first soften the probability p i h c by:

where T is the temperature hyperparameter that controls the sharpness of q i h c , p i , j h c is the j-th element of p i h c , and q i , j h c denotes the softened probability distribution of the j-th class for the i-th input data. The formulation of the knowledge distillation process from T h to T c involves minimizing the loss function ℒ h → c , which is defined as follows:

where   ℒ C E is the cross-entropy loss, which measures the dissimilarity between the predicted probability distribution p i c and the one-hot ground-truth label vector y i c for the single-task model T c . It can be written as shown in Equation 3:

And ℒ K D , the knowledge distillation loss, which quantifies the divergence between q i h c and p i c , is defined as shown in Equation 4:

Similar to Equation 2, we can define a loss function from

T h to T s , denoted as ℒ h → s , which is given by:

where q i h s is the probability distribution obtained by softening the severity prediction output p i h s from T h (referred to in Equation 1), and p i s is the output from T s .

Upon completing the knowledge disentanglement process, the shared knowledge from the hybrid tasks T h is individually transferred back to the corresponding subtasks, i.e., T c for disease species classification and T s for disease severity identification. We then employ mutual distillation to further investigate the complementarity of the two subtasks. Here, we assume that T c and T s use the same backbones, such as ResNet50. Motivated by Komodakis and Zagoruyko (2016), as shown in Figure 2B , the commonality of knowledge between subtasks is reflected in the consistency of attention maps from the middle layer. Specifically, given two feature mappings, F l c and F l s , which are the outputs of layer l of the models T c and T s , respectively, we can calculate the attention maps, A l c and A l s , as shown in Equation 6:

where C_i is the number of channels in the feature mappings of

F l c and F l s , and (k,x,y) specifies the location and channel of an activation value within the feature mapping. The attention maps A l c and A l s are computed by averaging the activation values across the channels of the respective feature mappings, F l c and F l s . For stability of optimization, we first reshape the A l c and A l s into a vector form as a l c = vec ( A l c ) and a l s = vec ( A l s ) , where vec(.) is an operation that transforms a matrix into a vector by concatenating its columns. Then, we normalize the vectors using l ₂ norm as shown in Equation 7:

The attention transfer loss for layer l is written as shown in Equation 8:

And the total loss for mutual learning between subtasks is defined as follows:

where L denotes the number of layers considered for attention transfer loss.

The primary objective of the proposed MTDL is to enhance multi-task learning capabilities. In the final step of this learning framework, we consider the two sub-tasks after mutual learning, T c and T s , and reintegrate them into the original multi-tasking model, denoted as … As shown in Figure 2C , this reintegration process results in an enhanced multi-task model T h ' .The knowledge integration loss is formulated as follows:

where q i c and q i s represent the output of softened probability distributions of T c and T s , respectively, which are obtained by applying the process described in Equation 1. The whole process of MTDL is summarized in Algorithm 1 .

The MTDL framework consists of three main components: knowledge disentanglement, subtask mutual learning, and knowledge integration. To ensure simplicity and generality of the framework, we employ a consistent training strategy for different learning components. Specifically, the framework is trained using the SGD optimizer with a batch size of 32 and a momentum of 0.9. The initial learning rate is set to 0.001, and it is reduced by a factor of 0.1 every 20 epochs. The weight decay is set to 1e-4. The maximum number of training epochs is set to 100, and an early stopping strategy is used based on the validation performance. If the validation loss does not improve for 5 consecutive epochs, the training process is stopped.

The MTDL framework involves three main stages of knowledge distillation, which correspond to the objective functions in Equations 2, 9, and 10. During the process, we use a temperature parameter T to smooth the output of the teacher model. This hyperparameter is determined through cross-validation using the validation set. A comprehensive analysis of hyperparameter selection can be found in Section 3.3.4.

To evaluate the performance of the proposed MTDL method, we employ four commonly used evaluation metrics, namely Accuracy, Precision, Recall, and F1-score. Given true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN), the specific definitions of these metrics are as shown in Equations 20 and 21:

The MTDL framework is a flexible knowledge distillation approach designed for tomato disease diagnosis. It aims to improve the performance of recognition models while reducing their parameter size and can be combined with various existing neural network architectures. To ensure the versatility of the MTDL framework, we incorporate four conventional network models, including ResNet101 (He et al., 2016), ResNet50 (He et al., 2016), DenseNet121 (Huang et al., 2017), and VGG16 (Simonyan and Zisserman, 2014), as well as four lightweight network models such as EfficientNet (Tan and Le, 2019), ShuffleNetV2 (Zhang et al., 2018b), MobileNetV3 (Howard et al., 2019), and SqueezeNet (Iandola et al., 2016). Detailed information about these models can be found in Table 2 . These backbone models serve as the learning components in different stages of the MTDL framework. We use the original classification results of these models as a baseline and compare the results before and after the multi-task distillation process to validate the effectiveness of the proposed framework.

In this section, we report the results from two experimental settings. The first setting, referred to as unified MTDL, employs the same network architecture for teacher and student modules. This setting aims to verify the effectiveness of the multi-stage distillation architecture proposed in this paper. The second setting, termed heterogeneous MTDL, involves using lightweight network architectures for all student models within the MTDL framework. This setting is designed to demonstrate the advantages of the proposed architecture in achieving a balance between performance and efficiency. As a reference, Table 2 lists the baseline results of the initial two single tasks T c and T s , as well as the multi-task model T h , where T h c and T h s correspond to the results of T h for disease classification and severity estimation tasks, respectively. The results in Table 2 demonstrate that the multi-task learning approach effectively enhances performance across various network architectures.

The results for MTDL with a unified architecture are presented in Table 3 . We can observe that all models show improvement when using MTDL for knowledge learning. This indicates that the MTDL framework effectively leverages the staged learning of knowledge and the complementarity between different tasks. In terms of specific models, ResNet101 achieves the highest performance in both tasks under the MTDL setting, with Accuracy scores of 98.92% for T c and 95.32% for T s , respectively. The corresponding F1-scores are 98.78% and 96.32%, respectively. These results can be attributed to both the ResNet101’s powerful feature extraction capabilities and MTDL’s effective multi-task learning strategy. On the other hand, SqueezeNet shows significant improvement with an increase of 1.08% and 2.53% in Accuracy of T c and T s respectively, and an increase of 0.68% and 2.26% in F1-scoref or each task. This suggests that the MTDL allows the lightweight model to learn more robust and comprehensive features. Furthermore, Table 3 also provides a comparison between the MTDL, MTDL-PTF, and MTDL-TF methods across various architectures. The results indicate that while the overall performance of MTDL-PTF and MTDL-TF decreases when the dependence on the teacher model is reduced, the introduction of a virtual teacher model significantly improves the accuracy of both methods compared to the original multitask learning. This indeed validates the effectiveness of the decoupled teacher-free knowledge distillation approach that we proposed. We also display the confusion matrices for results using ResNet50 as the backbone. As shown in Figure 4 , it is evident that our proposed MTDL method either maintains or improves performance across all individual classes for both disease classification and severity estimation tasks. This demonstrates MTDL’s ability to achieve a balanced enhancement in both overall performance and category-specific outcomes.

Furthermore, to investigate the impact of using teacher and student models with different architectures on the performance of the MTDL framework, we employ complex models like DenseNet121 for the teacher and lightweight models such as EfficientNet for the student. The results presented in Table 4 substantiate the effectiveness of this heterogeneous MTDL approach. For instance, when using ResNet101 as the teacher model, the SqueezeNet student model shows an improvement of 1.95% and 3.07% in T c and T s respectively, which are higher than the result obtained under the unified architecture MTDL setting. These results suggest that a more powerful teacher model enriches the student model’s learning.

Finally, to ensure the effectiveness of our proposed method, we conduct a comprehensive comparison with four well-established approaches in the field to validate its performance:

To ensure fair comparisons among KD, DKD, AT, and MTDL, we consistently used ResNet-101 as the teacher and MobileNetV3Small as the student model. This approach enables a reliable assessment of knowledge distillation efficacy. Additionally, we present MTDL results using ResNet-101 as both teacher and student, aligning with DHBP’s backbone, to effectively demonstrate its multi-tasking capabilities.

The results are shown in Table 5 . In our experiments, MTDL with ResNet-101 as both teacher and student models achieve the best results, outperforming DHBP in disease classification by 0.53% in Accuracy and 0.29% in F1-score, and in severity prediction by 0.86% in Accuracy and 1.08% in F1-score. These improvements validate MTDL’s phased multi-task learning approach. Moreover, when compared under the same teacher-student model setup with other distillation methods (KD, DKD, AT), MTDL excelled, particularly surpassing DKD by 0.37% in Accuracy and 0.16% in F1-score for disease classification, and by 0.62% in Accuracy and 0.38% in F1-score for severity prediction. This indicates the effectiveness of MTDL’s proposed mutual distillation learning between teachers and students.

In this subsection, we conduct a Wilcoxon Signed-Rank Test (Corder and Foreman, 2014) to evaluate the significance of the performance improvements across all CNN architectures. We provide the detailed significance analysis corresponding to the results originally presented in Tables 3 and 4 in the following Table 6 and 7 . In Table 6 , we present a comparison of the performance of our MTDL model and its variants against several baseline CNN architectures. This table focuses on scenarios within our MTDL framework where both the teacher and student models utilize identical architecture. The results from this table demonstrate statistically significant improvements across all comparisons in both disease classification and severity prediction tasks. The p-values obtained are consistently well below the 0.05 threshold, indicating robust enhancements attributed to our MTDL approach. Similarly, Table 7 showcases the results in a heterogeneous setting, where the MTDL model employs a more complex architecture as the teacher model and a lightweight network as the student model. In these comparisons, the results again confirm significant improvements across all evaluated aspects.

In addition, we also perform the significance of the results in comparison with other multi-task and distillation learning methods. with the results recorded in Table 8 . It can be seen that in most cases, the MTDL framework shows statistically significant differences when compared with methods like DHBP, KD, DKD, and AT, with p-values well beneath the 0.05 significance threshold. However, there is one exception to note: in the case of MTDL (ResNet101-MobileNetV3Small) vs DHBP for severity prediction, the p-value is slightly above the conventional threshold for significance. This exception likely stems from MTDL employing lightweight MobileNetV3Small as the distillation target, whereas DHBP uses the more substantial ResNet101 as its base model.

We assess the effectiveness of the three stages in our MTDL framework: knowledge disentanglement, mutual knowledge transfer, and knowledge integration. To do so, we employ single-task and multi-task models as our baselines and incorporate the results obtained after each stage of learning. As illustrated in Figure 5 , the results in terms of Accuracy and F1-score align with our expectations. The results clearly demonstrate that each stage of learning contributes to the final performance improvement, thereby validating the effectiveness of staged distillation in the MTDL framework.

We investigate the balance between performance and efficiency within the context of our MTDL framework. Performance is measured by Accuracy, while efficiency is represented by the number of parameters and floating-point operations (FLOPs). We use the single-task ResNet101 model and the multi-task ResNet101 model as baselines due to their superior performance across all single-task and multi-task models, as shown in Table 3 . The results are presented in Figure 6 , and the size of each model’s marker in the figure represents the number of parameters used by the model.

It can be observed that there is a similar trend in both task of disease classification ( Figure 6A ) and disease severity estimation ( Figure 6B ). Our MTDL-enhanced ResNet101 notably surpasses the single-task baseline with an Accuracy improvement of 0.81% for disease classification and 1.71% for severity estimation, and it outperforms the multi-task baseline with 0.36% and 0.99% improvements respectively. When using MobileNetV3Large as the MTDL-optimized model, we achieved significant performance gains with reduced parameter count and FLOPs, while still enhancing Accuracy over both baselines. For example, the MobileNetV3Large model, enhanced by our MTDL framework, outperforms the ResNet101 baseline by 0.63% and 1.44% in the two tasks, respectively. Remarkably, this is achieved with only 12.81% of the parameters (5.450M vs. 42.529M) and 2.87% of the FLOPs (0.225G vs. 7.832G). These findings highlight the MTDL framework’s capability to improve performance significantly while maintaining computational efficiency, thereby reinforcing its advantage over conventional models.

Therefore, we need to select the appropriate distillation model for each specific scenario. The choice depends on balancing computational resources and performance. Typically, complex teachers like ResNet101 outperform compact students such as MobileNet, owing to deeper architectures. MTDL promotes mutual learning between teachers and students, simultaneously enhancing both models. With abundant resources, an MTDL-optimized teacher offers substantial performance gains. In contrast, for limited-resource scenarios like mobile inference, MTDL can distill a lightweight yet performant student model. Additionally, the teacher-free MTDL-TF variant reduces dependency on complex teachers, offering an alternative when resources are constrained.

In this section, we use Grad-CAM (Selvaraju et al., 2017) for visual analysis to gain deeper insights into the learning process of our MTDL framework. We examine three severity levels of Early Blight: healthy, general, and severe. Visualizations for single-task and multi-task models, as well as for each stage of MTDL learning, are provided. Figure 7 shows that the model’s attention shifts toward task-relevant areas as it learns. For healthy leaves, the MTDL-enhanced model more precisely identifies the leaf as a whole, aligning with human visual systems. For leaves at a general severity level, the model focuses on localized disease spots for classification but expands its attention to surrounding regions for severity estimation. In cases of severe disease levels, the disease spots typically exhibit a widespread distribution across the leaf area. The knowledge integration model, in its pursuit to accurately recognize both the disease type and severity, tends to produce a Grad-CAM sensitivity map covering the entire leaf area. This comprehensive coverage contrasts with the single-task model, which primarily focuses on localized diseased regions, and the multi-task model, which, although it expands the area of interest, does not distribute sensitivity intensity as effectively. Moreover, the distribution of sensitivity intensity in the knowledge integration model offers a more realistic representation of the disease’s extensive impact, thereby enhancing the model’s explanatory power for Severe Early Blight. This analysis highlights the MTDL framework’s adaptability in shifting its focus based on the task and severity, thereby improving performance and interpretability.

The temperature parameter T adjusts the softmax output in the neural network, smoothing the probability distribution and revealing more nuanced information about the model’s predictions. This is crucial for knowledge distillation, where it aids in transferring detailed information from a teacher to a student model. This concept is introduced and utilized in Equation 1. To assess the sensitivity of our model to T, we vary T within the interval [0.1,50] and record the Accuracy of the disease classification and severity estimation tasks for each value. The results of nine common network architectures are shown in Figure 8 . Despite the differences in architecture, a similar trend is observed: as T increases, the model’s performance improves, but rapidly declines when T exceeds 10. Notably, the model’s performance remains relatively stable for T within the interval [3,8]. This indicates that our model is robust to the choice of T within this range, providing flexibility in practical applications.

One the other hand, the selection of a batch size of 32, momentum of 0.9, and learning rate decay factor of 0.1 was guided by a combination of empirical conventions and experimental validation aimed at striking a balance between computational efficiency and model performance. To validate the impact of different parameter settings on performance, we analyzed MTDL and its variants on the validation set for varying batch sizes ( Figures 9A, B ), momentum ( Figures 9C, D ), and learning rate decay factors ( Figures 9E, F ), detailing their effects on Accuracy. We can see that Accuracy remains relatively stable across batch sizes that varies (8, 16, 32, 64, 128), with the optimal average Accuracy achieved at 32. This is likely because a moderate batch size balances gradient estimation Accuracy and the beneficial noise of stochasticity, optimizing learning. As momentum increases from 0.1 to 0.9, Accuracy generally improves. A higher momentum, like 0.9, effectively uses past gradients to accelerate convergence and navigate through local minima, leading to better performance compared to a lower setting like 0.1. Moreover, increasing decay factors tend to lower Accuracy, potentially due to a swift reduction in the learning rate and premature convergence. An optimal decay factor is one that slowly decreases the learning rate, facilitating precise adjustments as the model converges to the best solution.