CIP-Net: Continual Interpretable Prototype-based Network

TL;DR

CIP-Net introduces an exemplar-free, self-explainable prototype-based network for continual learning that relies on a single shared prototype pool to mitigate catastrophic forgetting while maintaining interpretability. It combines alignment-uniformity self-supervision with targeted prototype regularization to curb interference and explain drift across tasks, achieving state-of-the-art results on CUB and CARS in both task- and class-incremental settings with substantially reduced memory overhead. The model provides both global and local explanations, linking prototypes to classes and local patch activations, and demonstrates stability of explanations across task sequences. Overall, CIP-Net offers a scalable, interpretable solution for continual learning with strong performance and practical memory efficiency, while outlining clear avenues for further refinement in sparsity, regularization, and out-of-distribution handling.

Abstract

Continual learning constrains models to learn new tasks over time without forgetting what they have already learned. A key challenge in this setting is catastrophic forgetting, where learning new information causes the model to lose its performance on previous tasks. Recently, explainable AI has been proposed as a promising way to better understand and reduce forgetting. In particular, self-explainable models are useful because they generate explanations during prediction, which can help preserve knowledge. However, most existing explainable approaches use post-hoc explanations or require additional memory for each new task, resulting in limited scalability. In this work, we introduce CIP-Net, an exemplar-free self-explainable prototype-based model designed for continual learning. CIP-Net avoids storing past examples and maintains a simple architecture, while still providing useful explanations and strong performance. We demonstrate that CIPNet achieves state-of-the-art performances compared to previous exemplar-free and self-explainable methods in both task- and class-incremental settings, while bearing significantly lower memory-related overhead. This makes it a practical and interpretable solution for continual learning.

Figures (6)

• Figure 1: CIP-Net overview. Two augmented views are processed by a CNN to produce a prototype feature map $\mathbf{z}$; channel-wise softmax and maxpooling operation yield prototype-presence scores $\mathbf{p}$. Contrastive alignment loss $\mathcal{L}'_{A}$ aligns prototypes across views, $\mathcal{L}'_{T}$ promotes rarely used prototypes, and $\mathcal{L}_{R}$ limits drift of important ones over tasks. Task-specific sparse, non-negative heads map prototypes to classes, trained with negative log-likelihood loss $\mathcal{L}_{C}$ plus sparsity $\mathcal{L}_{H}$ and head-decorrelation $\mathcal{L}_{D}$.
• Figure 2: Example of suppressed prototype activation drift for one of the regularized prototypes from task 1 to task 4 (4-tasks setting) on CUB. More details in Supplementary Material.
• Figure 3: Visualization of a portion of prototypes for each task in CUB's 4-task setting), obtained after training on the whole task sequence.
• Figure 4: (a) Average difference of activation values over classes between the current task and the first task. (b) Average difference of activation values over classes between the current task and the immediate previous task.
• Figure 5: Example of local explanation of CIP-Net for the correct class. CIP-Net learns part-prototypes visualized as patches from the training data, and localizes similar image patches in an unseen input image.