Pre-training for Recommendation Unlearning

TL;DR

This work tackles the challenge of unlearning in graph neural network–based recommender systems under privacy and regulatory pressures. It introduces UnlearnRec, a model-agnostic, pretraining-based framework featuring a Learnable Influence Encoder (IE) that predicts embedding updates in response to unlearning requests, enabling fast, directed adjustments without full retraining. The core contributions are the Influence Dependency Matrix to capture interdependent unlearning effects, a trainable IE, and a multi-task pretraining objective that jointly optimizes model fidelity, unlearning efficacy, preservation of distribution, and contrastive generalization. Extensive experiments on Movielens-1M, Gowalla, and Yelp2018 across backbone models show that UnlearnRec achieves strong unlearning effectiveness with more than $>10\times$ speedup over retraining, while maintaining competitive predictive performance and reducing information leakage. The approach offers practical impact for compliant recommender systems by delivering efficient, robust unlearning compatible with SSL-based architectures and real-world privacy requirements.

Abstract

Modern recommender systems powered by Graph Neural Networks (GNNs) excel at modeling complex user-item interactions, yet increasingly face scenarios requiring selective forgetting of training data. Beyond user requests to remove specific interactions due to privacy concerns or preference changes, regulatory frameworks mandate recommender systems' ability to eliminate the influence of certain user data from models. This recommendation unlearning challenge presents unique difficulties as removing connections within interaction graphs creates ripple effects throughout the model, potentially impacting recommendations for numerous users. Traditional approaches suffer from significant drawbacks: fragmentation methods damage graph structure and diminish performance, while influence function techniques make assumptions that may not hold in complex GNNs, particularly with self-supervised or random architectures. To address these limitations, we propose a novel model-agnostic pre-training paradigm UnlearnRec that prepares systems for efficient unlearning operations. Our Influence Encoder takes unlearning requests together with existing model parameters and directly produces updated parameters of unlearned model with little fine-tuning, avoiding complete retraining while preserving model performance characteristics. Extensive evaluation on public benchmarks demonstrates that our method delivers exceptional unlearning effectiveness while providing more than 10x speedup compared to retraining approaches. We release our method implementation at: https://github.com/HKUDS/UnlearnRec.

Figures (6)

• Figure 1: Overall framework of the proposed UnlearnRec paradigm.
• Figure 2: Visualization for the predictions of positive, negative, and adversarial edges in the Movielens-1M and Yelp2018 datasets. Four methods with three backbone models are compared, including the IF-based method GIF, the partition-based method GraphEraser, the retraining-based exact unlearning, and our proposed , on the GCN, SGL, SimGCL backbones.
• Figure 3: Visualization of the prediction distributions for Movielens-1M and Yelp2018 datasets before unlearning.
• Figure 4: Comparison of GPU memory usage with different methods on various backbones based on two datasets.
• Figure 5: Unlearning efficacy and performance v.s. pre-training and fine-tuning processes of .