Continual Learning of Achieving Forgetting-free and Positive Knowledge Transfer

TL;DR

The paper addresses the gap in continual learning by coupling forgetting-free learning with positive forward and backward knowledge transfer (FKT/BKT) in task-incremental settings. It introduces Enhanced Task Continual Learning (ETCL), a non-expansion method that uses task-specific binary masks (inspired by the Lottery Ticket Hypothesis), gradient alignment with similar tasks, and a bi-objective optimization to enable positive KT while preventing catastrophic forgetting. The authors provide theoretical bounds for forward and backward KT, a Wasserstein-distance-based online task similarity detector, and extensive experiments showing ETCL outperforms state-of-the-art baselines across dissimilar, similar, and mixed task sequences, with strong KT gains on similar tasks. The work advances practical CL by embedding KT directly into the learning objective, offering scalable, dataset- and backbone-agnostic performance gains with rigorous analysis. Overall, ETCL demonstrates that forgetting-free CL and positive KT can be simultaneously achieved in a non-expansive framework, enabling more versatile continual learning systems.

Abstract

Existing research on continual learning (CL) of a sequence of tasks focuses mainly on dealing with catastrophic forgetting (CF) to balance the learning plasticity of new tasks and the memory stability of old tasks. However, an ideal CL agent should not only be able to overcome CF, but also encourage positive forward and backward knowledge transfer (KT), i.e., using the learned knowledge from previous tasks for the new task learning (namely FKT), and improving the previous tasks' performance with the knowledge of the new task (namely BKT). To this end, this paper first models CL as an optimization problem in which each sequential learning task aims to achieve its optimal performance under the constraint that both FKT and BKT should be positive. It then proposes a novel Enhanced Task Continual Learning (ETCL) method, which achieves forgetting-free and positive KT. Furthermore, the bounds that can lead to negative FKT and BKT are estimated theoretically. Based on the bounds, a new strategy for online task similarity detection is also proposed to facilitate positive KT. To overcome CF, ETCL learns a set of task-specific binary masks to isolate a sparse sub-network for each task while preserving the performance of a dense network for the task. At the beginning of a new task learning, ETCL tries to align the new task's gradient with that of the sub-network of the previous most similar task to ensure positive FKT. By using a new bi-objective optimization strategy and an orthogonal gradient projection method, ETCL updates only the weights of previous similar tasks at the classification layer to achieve positive BKT. Extensive evaluations demonstrate that the proposed ETCL markedly outperforms strong baselines on dissimilar, similar, and mixed task sequences.

Figures (9)

• Figure 1: The architecture and pipeline of the proposed ETCL (on the left), where the proposed new techniques are embedded in the pink components. To the right of the dotted line separation, let $i$ and $t$ be two tasks similar to each other ($i<t$). (a) The selected sub-network $\hat{\mathbf{w}}_i^*$ (indicated by masks $\mathbf{m}^*_{i}$) of the previous task $i$ represented with blue arrows. (b) The selected initial sub-network $\hat{\mathbf{w}}_t$ (masks $\mathbf{m}_{t}$) of task $t$ represented by the selected new or unused weights by previous tasks (red arrows) and reused weights of previous similar task $i$ (green arrows) leading to automatic forward KT. (c) During task $t$ training, the weights corresponding to $\mathbf{m}_{t}$ are constantly updated and optimized. With the bi-objective optimization of the classification layer, the knowledge from task $t$ is backward transferred to previous task $i$ (those arrows with a circular point at the tails). (d) The optimized sub-network $\hat{\mathbf{w}}_t^*$ (masks $\mathbf{m}^*_{t}$) of task $t$ with newly selected and reused weights.
• Figure 2: The schematic diagram of the difference between Euclidean distance and Wasserstein distance. The figure shows three distributions $f_1$(red), $f_2$(green) and $f_3$(blue). Each pair has the same distance in the Euclidean space. But in the Wasserstein space, $f_1$ and $f_2$ are closer as the shapes/geometries of $f_1$ and $f_2$ are more similar overall.
• Figure 3: The performances of $A_{i,i}$ and $A_{t,i}$, where (a) $t \in [6,20]$ and $i=5$ on the dissimilar task dataset MiniImageNet (20 tasks), (b) $t \in [5,10]$ and $i=4$ on the similar task dataset F-EMNIST-1 (10 tasks), (c) $t \in [4,20]$ and $i=3$ on the similar task dataset F-CelebA-2 (20 tasks) and (d) $t \in [3,20]$ and $i=2$ on the mixed task dataset (CIFAR 100, F-CelebA-1) (20 tasks).
• Figure 4: The FWT and BWT performances of SOTA TIL methods on various dissimilar/similar/mixed task datasets.
• Figure 5: The average time and memory usage comparisons of ETCL and SOTA baselines on 11 benchmark datasets.

Theorems & Definitions (1)

• proof