ReReLRP -- Remembering and Recognizing Tasks with LRP

TL;DR

ReReLRP introduces a replay-free continual learning method that uses Layerwise Relevance Propagation to identify and freeze task-critical neurons, thereby mitigating catastrophic forgetting while maintaining adaptability for new tasks. By constructing relevance-based datasets and employing a classifier chain for inter-task inference, the approach delivers explainable, task-specific representations and memory-efficient operation. Empirical results across diverse datasets show competitive accuracy and notably reduced forgetting, with strong performance in medical imaging tasks where privacy and interpretability are essential. The work highlights the potential of relevance-driven pruning for scalable, privacy-preserving continual learning, while noting challenges in network capacity saturation and suggesting avenues for dynamic capacity management.

Abstract

Deep neural networks have revolutionized numerous research fields and applications. Despite their widespread success, a fundamental limitation known as catastrophic forgetting remains, where models fail to retain their ability to perform previously learned tasks after being trained on new ones. This limitation is particularly acute in certain continual learning scenarios, where models must integrate the knowledge from new domains with their existing capabilities. Traditional approaches to mitigate this problem typically rely on memory replay mechanisms, storing either original data samples, prototypes, or activation patterns. Although effective, these methods often introduce significant computational overhead, raise privacy concerns, and require the use of dedicated architectures. In this work we present ReReLRP (Remembering and Recognizing with LRP), a novel solution that leverages Layerwise Relevance Propagation (LRP) to preserve information across tasks. Our contribution provides increased privacy of existing replay-free methods while additionally offering built-in explainability, flexibility of model architecture and deployment, and a new mechanism to increase memory storage efficiency. We validate our approach on a wide variety of datasets, demonstrating results comparable with a well-known replay-based method in selected scenarios.

Figures (14)

• Figure 1: A step-by-step illustration of the proposed method. Samples labeled as ID or OOD stand for in-distribution and out-of-distribution, respectively. Relevance datasets built for tasks $1$, $2$, and $3$ are marked as $\mathcal{D}_{R_{1}}$, $\mathcal{D}_{R_{2}}$, and $\mathcal{D}_{R_{3}}$. Probabilities based solely on the relevance subnetworks are expressed as $P_{R_{1}}$ and $P_{R_{2}}$, while $P_{T_{1}}$ and $P_{T_{2}}$ denote probabilities based on all subnetwork relevances. Images used to illustrate the classes are taken from MNIST deng2012mnist.
• Figure 2: Average task-agnostic accuracy of different approaches.
• Figure 3: Average percentage of available neurons in experiments conducted on various datasets.
• Figure 4: Average task-agnostic accuracy of ablation experiments.
• Figure 5: Average task-agnostic accuracy on real-world medical datasets.