Dynamic transfer learning (Hoffman et al., 2014; Bitarafan et al., 2016; Mancini et al., 2019) refers to the knowledge transfer from a static source task to a dynamic target task. Compared to standard transfer learning on the static source and target tasks (Pan and Yang, 2009; Zhou et al., 2017, 2019a,b; Tripuraneni et al., 2020; Wu and He, 2021), dynamic transfer learning is a more challenging but realistic problem setting due to its time evolving task relatedness. More recently, various dynamic transfer learning frameworks are built from the following aspects: self-training (Kumar et al., 2020; Chen and Chao, 2021; Wang et al., 2022), incremental distribution alignment (Bobu et al., 2018; Wulfmeier et al., 2018; Wang H. et al., 2020; Wu and He, 2020, 2022a), meta-learning (Liu et al., 2020; Wu and He, 2022b), contrastive learning (Tang et al., 2021; Taufique et al., 2022), etc. Specifically, most existing works assume that the task distribution is continuously evolving over time. Very little effort has been devoted to studying dynamic transfer learning when this assumption is violated in real scenarios. Compared to previous works (Liu et al., 2020; Wang et al., 2022; Wu and He, 2022b), in this paper, we focus on a more realistic dynamic transfer learning with a relaxed assumption that the task distribution could be suddenly changed at some time stamp.

Meta-learning (Hospedales et al., 2021) leverages the knowledge from a set of prior meta-training tasks for fast adaptation to new tasks. In the context of few-shot classification, meta-learning aims to find the optimal model initialization (Finn et al., 2017, 2018; Wang L. et al., 2020; Yao et al., 2021) from previously seen tasks such that this model can be fine-tuned on a new task by performing a few gradient steps. It assumes that all the tasks follow a stationary task distribution. More recently, this meta-learning paradigm has been extended into the online learning setting where a sequence of tasks is sampled from non-stationary task distributions (Finn et al., 2019; Acar et al., 2021). Following previous work (Wu and He, 2022b), we formulate dynamic transfer learning as a meta-learning problem, which aims to learn the optimal model initialization for knowledge transfer across any meta-pair of tasks. In contrast to Wu and He (2022b) where the meta-pairs of tasks are simply constructed from tasks at consecutive time stamps, we propose to learn the sampling probability for meta-pairs of tasks based on the task relatedness during model training. This can help our meta-learning framework avoid the negative transfer induced by the meta-pairs of tasks sampled from suddenly shifted task distribution.

Let X and Y be the input feature space and output label space respectively. We consider the dynamic transfer learning problem (Hoffman et al., 2014; Bobu et al., 2018) with a static source task Ds and a dynamic target task {Djt}j=1N with time stamp j. In this case, we assume that there are m^s labeled training examples Ds={(xis,yis)}i=1ms in the source task. Let mjt be the number of unlabeled training examples Djt={xijt}i=1mjt in the j^th target task. Let H be the hypothesis class on X where a hypothesis is a function h:X→Y. L(·,·) is the loss function such that L:Y×Y→ℝ. The expected classification error on the source task Ds is defined as ϵs(h)=E(x,y)~Ds[L(h(x),y)] for any h∈H, and its empirical estimate is given by ϵ^s(h)=1ms∑i=1msL(h(xi),yi). The expected error ϵjt(h) and empirical error ϵ^jt(h) of the target task at the j^th time stamp can also be defined similarly.

Following previous works (Hoffman et al., 2014; Bitarafan et al., 2016; Bobu et al., 2018), we formally define the problem of dynamic transfer learning as follows.

Definition 3.1. (Dynamic Transfer Learning) Given a labeled static source task Ds and an unlabeled dynamic target task {Djt}j=1N, dynamic transfer learning aims to learn the prediction function for the newest target task DN+1t by leveraging the knowledge from historical source and target tasks.

The key challenge of dynamic transfer learning is the time evolving task relatedness between source and target tasks. Recent works (Liu et al., 2020; Wang et al., 2022; Wu and He, 2022b) showed the generalization error bounds by assuming that the data distribution of the target task is continuously changing over time. Intuitively, in this case, the expected error bound on the newest target task is bounded in terms of the largest distribution gap [e.g., max0≤j≤NdHΔH(Djt,Dj+1t)] across time stamps. As a result, these generalization error bounds are not tight when the task distribution is significantly shifted at some time stamp. As shown in Figure 2, the task distribution is shifted smoothly from time stamp 1 to time stamp 2. However, it changes sharply from time stamp 2 to time stamp 3. In real scenarios, this sharp distribution shift might be induced by some unexpected issues, e.g., adversarial manipulation (Wu and He, 2021). This thus motivates us to study dynamic transfer learning with a much more relaxed assumption that the task distribution could be suddenly shifted at some time stamp.

We derive the generalization error bound of dynamic transfer learning as follows. Following Ben-David et al. (2010) and Liu et al. (2020), we use H-divergence to measure the distribution shift across tasks and Vapnik-Chervonenkis (VC) dimension to measure the complexity of a class of functions H. Without loss of generality, we would like to consider a binary classification problem (i.e., Y={0,1}) with the loss function L(ŷ,y)=|ŷ-y|. The following theorem showed that the expected error of the newest target task DN+1t can be bounded in terms of the historical source and target knowledge.

Theorem 4.1. (Generalization Error Bound) Let H be a hypothesis space of VC dimension d. If there are m labeled source examples i.i.d. drawn from Ds (denoted as D0t as well) and m unlabeled target examples i.i.d. drawn from Djt for each time stamp j = 1, ⋯ , N+1², then for any δ>0 and h∈H, with probability at least 1−δ, the expected error of the newest target task DN+1t can be bounded as follows.

where ∑i=0N∑j=i+1N+1wij=1, and w_ij≥0 if i<j, w_ij = 0 otherwise. ηij=12 if 1 ≤ j ≤ N and i<j, and ηij=12(1+∑k=0i-1wkiwij) if j = N+1 and i<j, η_ij = 0 otherwise. Here λ denotes the combined error of the ideal hypothesis over all the tasks, i.e., λ=minh∈H∑i=0N+1ϵit(h), and d^HΔH(·,·) denotes the empirical estimate of H-divergence over finite examples.

Note that this error bound holds with other existing distribution discrepancy measures (see Corollary 4.3), though we consider H-divergence (Ben-David et al., 2010) in Theorem 4.1. Furthermore, we show the generalization error bound of dynamic transfer learning from the perspective of meta-learning. That is, instead of sharing the hypothesis h∈H for all the tasks, we learn a common initialized model h̄∈H across tasks. Then the task-specific model h_i via one-step gradient update for the target at the i^th time stamp, i.e., θi=θ̄-β∇θLmeta, where θi,θ̄ denotes the parameters of hi,h̄ respectively and Lmeta is the meta-learning loss for updating the task-specific model parameters. If we let Lmeta=ϵ^it(h̄)=1m∑k=1mL[h̄(xki),yki], the following theorem provides the generalization error bound based on meta-learning.

Theorem 4.2. (Meta-Learning Generalization Error Bound) Let H be a hypothesis space of VC dimension d. If there are m labeled source examples i.i.d. drawn from Ds (denoted as D0t as well) and m unlabeled target examples i.i.d. drawn from Djt for each time stamp j = 1, ⋯ , N+1, then for any δ>0 and a proper inner learning rate β, with probability at least 1−δ, the expected error of the newest target task DN+1t can be bounded in the following.

where ∑i=0N∑j=i+1N+1wij=1, and w_ij≥0 if i<j, w_ij = 0 otherwise. ηij=12 if 1 ≤ j ≤ N and i<j, and ηij=12(1+∑k=0i-1wkiwij) if j = N+1 and i<j, η_ij = 0 otherwise. Here λ denotes the combined error of the ideal hypothesis over all the tasks, i.e., λ=minh∈H∑i=0N+1ϵit(h), and d^HΔH(·,·) denotes the empirical estimate of H-divergence over finite examples.

We observe from Theorem 4.2 that the parameter w_ij plays an important role in the generalization error bound of dynamic transfer learning. Intuitively, it is more likely to assign higher value w_ij for the easy meta-pair of tasks Di→Dj with stronger class discrimination over Di [i.e., smaller ϵ^it(hi)] and smaller distribution shift between Di and Dj [i.e., smaller d^HΔH(Dit,Djt)].

The following corollary shows that the error bound in Theorem 4.1 can be generalized by considering various domain discrepancy measures.

Corollary 4.3. With the same assumptions in Theorem 4.1, for any δ>0 and h∈H, there exist w_ij≥0 and η_ij≥0, with probability at least 1−δ, the expected error of the newest target task DN+1t can be bounded in the following.

where d^(·,·) can be instantiated with existing distribution discrepancy measures, including discrepancy distance (Mansour et al., 2009), maximum mean discrepancy (Long et al., 2015), Wasserstein distance (Shen et al., 2018), f-divergence (Acuna et al., 2021), etc. Here Ω denotes the corresponding sample complexity when the distribution discrepancy measure is selected.

Corollary 4.3 shows the flexibility in generalizing existing static transfer learning theories (Mansour et al., 2009; Ben-David et al., 2010; Ghifary et al., 2016; Shen et al., 2018; Zhang et al., 2019; Acuna et al., 2021) into the dynamic transfer learning setting. Moreover, it is observed that Corollary 4.3 is closely related to the existing generalization error bounds (Wang et al., 2022; Wu and He, 2022b) of dynamic transfer learning, under different parameters w_ij and η_ij.

where τ∈ℝ. Then, when τ → 0, Corollary 4.3 recovers the generalization error bound (Wang et al., 2022).

where H is the hypothesis class of R-Lipschitz L-layer fully-connected neural networks with 1-Lipschitz activation function.

Then, Corollary 4.3 recovers the generalization error bound (Wu and He, 2022b).

where Ω_L is a Rademacher complexity term.

Compared to existing theoretical results (Wang et al., 2022; Wu and He, 2022b), with appropriate w_ij, our generalization error bound in Corollary 4.3 is much more tighter when there exists some time stamp i such that d^HΔH(Di-1t,Dit) is large. It thus motivates us to develop a progressive meta-task scheduler in the meta-learning framework for dynamic transfer learning. The crucial idea is to automatically learn the values w_ij, based on the intuition that assigning large value w_ij on easy meta-pair of tasks Di→Dj would make our error bound much tighter.

We used the following publicly available image data sets:

Following Bobu et al. (2018) and Wu and He (2022b), we report both the classification accuracy on the newest target task (Acc) and the average classification accuracy on the historical target tasks (H-Acc) in the experiments. The comparison baselines we used in the experiments include: (1) static transfer learning approaches: SourceOnly, DAN (Long et al., 2015), DANN (Ganin et al., 2016), and MDD (Zhang et al., 2019); and (2) dynamic transfer learning: CUA (Bobu et al., 2018), GST (Kumar et al., 2020), L2E (Wu and He, 2022b), and our proposed L2S framework. For a fair comparison, all the methods use the same base models for feature extraction, e.g., LeNet for Rotating MNIST and ResNet-18 (He et al., 2016) for ImageCLEF-DA. In addition, we set η = 1, β = 0.01 and the number of inner epochs in M_ij(θ) as 1. All the experiments are performed on a Windows machine with four 3.80GHz Intel Cores, 64GB RAM and two NVIDIA Quadro RTX 5000 GPUs.

Figures 3, 4 show the distribution shift in the dynamic transfer learning tasks, where “S-T" denotes the distribution difference d(Ds,Djt) between the source and the target at every time stamp and “T-T" denotes the distribution difference d(Dj-1t,Djt) of the target at consecutive time stamp. Here we use maximum mean discrepancy (MMD) (Gretton et al., 2012) to measure the distribution difference across tasks. We see that when the target task is continuously evolving over time, d(Dj-1t,Djt) is small. This enables gradual knowledge transferability in the target task. If there exists a large distribution shift at some times, i.e., d(Dj-1t,Djt) is large, the strategy of gradual knowledge transferability might fail. In Figures 3, 4, the large distribution shift happened in the time stamps 17–35 on Rotating MNIST and time stamp 1 on I → C/P.

Tables 1, 2 provides the experimental results of L2S as well as baselines on Rotating MNIST and Image-CLEF data sets. We have the following observations from the results. On the one hand, when the target task is continuously evolving over time, most dynamic transfer learning baselines can achieve satisfactory performance on both the newest and historical target tasks. The baseline GST (Kumar et al., 2020) fails on Rotating MNIST, because the self-training approach might be more likely to accumulate the classification error when the target task is evolving for a long time. On the other hand, the performance of CUA (Bobu et al., 2018) and L2E (Wu and He, 2022b) drops significantly when there is a large distribution shift within the target task at some time stamp. In contrast, by adaptively selecting the meta-pairs of tasks, the proposed L2S framework can mitigate the issue of the potential large distribution shift in the targe task. Specifically, compared to L2E (Wu and He, 2022b), L2S improves the performance by a large margin. This confirms the efficacy of the proposed progressive meta-pair scheduler.

We provide the ablation study of our L2S framework with respect to the number of inner training epochs. The results on the newest target task of Rotating MNIST are shown in Figure 5, where we use 1 or 5 inner epochs for our meta-learning framework. We see that using more inner epochs can improve the convergence of L2S but it sacrifices the classification accuracy on the historical target task. This is because L2S with more inner epochs would enforce the fine-tuned model to be more task-specific. Thus, we set the number of inner epochs as 1 in our experiments.