Post-Training Attribute Unlearning in Recommender Systems

TL;DR

This article focuses on a strict but practical setting of AU, namely Post-Training Attribute Unlearning (PoT-AU), where unlearning can only be performed after the training of the recommendation model is completed, and proposes a two-component loss function.

Abstract

With the growing privacy concerns in recommender systems, recommendation unlearning is getting increasing attention. Existing studies predominantly use training data, i.e., model inputs, as unlearning target. However, attackers can extract private information from the model even if it has not been explicitly encountered during training. We name this unseen information as \textit{attribute} and treat it as unlearning target. To protect the sensitive attribute of users, Attribute Unlearning (AU) aims to make target attributes indistinguishable. In this paper, we focus on a strict but practical setting of AU, namely Post-Training Attribute Unlearning (PoT-AU), where unlearning can only be performed after the training of the recommendation model is completed. To address the PoT-AU problem in recommender systems, we propose a two-component loss function. The first component is distinguishability loss, where we design a distribution-based measurement to make attribute labels indistinguishable from attackers. We further extend this measurement to handle multi-class attribute cases with efficient computational overhead. The second component is regularization loss, where we explore a function-space measurement that effectively maintains recommendation performance compared to parameter-space regularization. We use stochastic gradient descent algorithm to optimize our proposed loss. Extensive experiments on four real-world datasets demonstrate the effectiveness of our proposed methods.

Figures (9)

• Figure 1: Illustrations of different unlearning targets.
• Figure 2: An overview of Post-Training Attribute Unlearning (PoT-AU) vs In-Training Attribute Unlearning (InT-AU) in recommender systems. $\mathcal{L}_u$ denotes the distinguishability loss designed for Goal #1, $\mathcal{L}_r$ denotes the regularization loss designed for Goal #2. The orange dots represent positive items which are in the top-$l$ positions of recommended list, while the gray dots represent the opposite. We omit other parameters in the collaborative filtering model besides embeddings for conciseness.
• Figure 3: Correlation between two types of regularization losses and RBO (similarity in recommendation performance), where the x-axis and y-axis represent values of losses and RBO, respectively. Note that $\ell_2$ is a parameter-space regularization, and $\ell_r$ is a function-space regularization. (a) Adding perturbation and calculating $\ell_2$; (b) Adding perturbation and calculating $\ell_r$. The Pearson correlation coefficients for (a) and (b) are -0.255 and -0.766 respectively.
• Figure 4: Distribution of user embedding in the first dimension on NMF
• Figure 5: Effect of the hyper-parameter $\alpha$.

Theorems & Definitions (1)

• Definition 1: Distribution-to-Distribution Distinguishability li2023making