LEGO: A Lightweight and Efficient Multiple-Attribute Unlearning Framework for Recommender Systems

TL;DR

This work tackles the problem of protecting user privacy in recommender systems when multiple sensitive attributes must be unlearned, and where privacy requirements can evolve over time. It introduces LEGO, a two-step framework that first calibrates embeddings to erase information about individual attributes via mutual information minimization, then flexibly combines the resulting embeddings to protect all requested attributes. The approach provides a theoretical bound ensuring simultaneous unlearning and leverages vCLUB for MI estimation, enabling efficient parallelizable updates and rapid adaptation to changing privacy needs. Experiments across three real-world datasets and three recommender models demonstrate LEGO’s strong multi-attribute unlearning performance while preserving recommendation quality and offering superior efficiency in dynamic scenarios compared with existing methods.

Abstract

With the growing demand for safeguarding sensitive user information in recommender systems, recommendation attribute unlearning is receiving increasing attention. Existing studies predominantly focus on single-attribute unlearning. However, privacy protection requirements in the real world often involve multiple sensitive attributes and are dynamic. Existing single-attribute unlearning methods cannot meet these real-world requirements due to i) CH1: the inability to handle multiple unlearning requests simultaneously, and ii) CH2: the lack of efficient adaptability to dynamic unlearning needs. To address these challenges, we propose LEGO, a lightweight and efficient multiple-attribute unlearning framework. Specifically, we divide the multiple-attribute unlearning process into two steps: i) Embedding Calibration removes information related to a specific attribute from user embedding, and ii) Flexible Combination combines these embeddings into a single embedding, protecting all sensitive attributes. We frame the unlearning process as a mutual information minimization problem, providing LEGO a theoretical guarantee of simultaneous unlearning, thereby addressing CH1. With the two-step framework, where Embedding Calibration can be performed in parallel and Flexible Combination is flexible and efficient, we address CH2. Extensive experiments on three real-world datasets across three representative recommendation models demonstrate the effectiveness and efficiency of our proposed framework. Our code and appendix are available at https://github.com/anonymifish/lego-rec-multiple-attribute-unlearning.

Figures (4)

• Figure 1: (a) Privacy protection requirements often involve multiple attributes and are dynamic: they may increase, decrease, and alter. (b) Single-attribute unlearning cannot meet dynamic privacy protection requirements. The dashed arrow indicates the storage of the intermediate model can accelerate sequential unlearning.
• Figure 2: An overview of LEGO. Our proposed LEGO splits multiple-attribute unlearning into two steps: embedding calibration and flexible combination. To illustrate the goal of each step, we provide a sketch of the embedding distribution. In the sketch, the shape, color, and border of the data points represent age, gender, and occupation information, respectively. The lines and plane represent the decision boundaries of the classifier.
• Figure 3: Results of efficiency in adapting to dynamic requirements. We present the running time of compared methods on NCF model across three datasets. We run all models 10 times and report the average results in seconds (s). The dashed line represents the training time of the original recommendation model.
• Figure 4: Effect of hyperparameter $\epsilon$. We conduct experiments on all three models across three datasets. We use BAcc and NDCG@10 to represent the performance of unlearning and recommendation respectively. We report the results of unlearning all user attributes recorded in the dataset.

Theorems & Definitions (2)

• definition 1
• theorem 1