Recommendation Unlearning via Influence Function

TL;DR

A new Influence Function-based Recommendation Unlearning framework, which efficiently updates the model without retraining by estimating the influence of the unusable data on the model via the influence function, and an importance-based pruning algorithm to reduce the cost of the influence function.

Abstract

Recommendation unlearning is an emerging task to serve users for erasing unusable data (e.g., some historical behaviors) from a well-trained recommender model. Existing methods process unlearning requests by fully or partially retraining the model after removing the unusable data. However, these methods are impractical due to the high computation cost of full retraining and the highly possible performance damage of partial training. In this light, a desired recommendation unlearning method should obtain a similar model as full retraining in a more efficient manner, i.e., achieving complete, efficient and harmless unlearning. In this work, we propose a new Influence Function-based Recommendation Unlearning (IFRU) framework, which efficiently updates the model without retraining by estimating the influence of the unusable data on the model via the influence function. In the light that recent recommender models use historical data for both the constructions of the optimization loss and the computational graph (e.g., neighborhood aggregation), IFRU jointly estimates the direct influence of unusable data on optimization loss and the spillover influence on the computational graph to pursue complete unlearning. Furthermore, we propose an importance-based pruning algorithm to reduce the cost of the influence function. IFRU is harmless and applicable to mainstream differentiable models. Extensive experiments demonstrate that IFRU achieves more than 250 times acceleration compared to retraining-based methods with recommendation performance comparable to full retraining. Codes are avaiable at https://github.com/baiyimeng/IFRU.

Figures (7)

• Figure 1: Illustration of recommendation unlearning process.
• Figure 2: The neighborhood aggregation process before and after erasing an interaction in LightGCN. $\mathcal{N}_{u_1}$ represents the set of items interacted by user $u_1$.
• Figure 3: Illustration of how erasing an interaction between $u_1$ and $i_1$ affects users and items. The value around a node denotes its importance score.
• Figure 4: Comparison of recommendation accuracy among methods for erasing an empty data set from a trained model. The reference method "Original" denotes the backbone model without performing unlearning.
• Figure 5: Performance of Original, Retrain, and IFRU when erasing unusable data of different sizes on the metrics $AUC_{0}$ and $AUC_{1}$w.r.t. BookCrossing. 'Ratio’ denotes $|\mathcal{D}_{r}|/|\mathcal{D}|$.