Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graph Unlearning Meets Influence-aware Negative Preference Optimization

Qiang Chen, Zhongze Wu, Ang He, Xi Lin, Shuo Jiang, Shan You, Chang Xu, Yi Chen, Xiu Su

TL;DR

The paper tackles graph unlearning by reframing edge forgetting as a negative preference optimization task and addressing topological coupling that degrades model utility during unlearning. It introduces INPO, which combines an influence-aware message passing scheme, a fast edge-influence estimator, and a topological entropy loss, forming the holistic objective $Loss = \lambda_1 L_{NPO} + \lambda_2 L_{GD} + \lambda_3 TE$. Empirical results on five real-world datasets show that INPO achieves state-of-the-art forget quality while preserving utility, with ablations confirming the contribution of each component. The work advances practical graph unlearning by delivering robust forgetting under varying deletion scenarios and provides code to enable reproducibility and wider adoption.

Abstract

Recent advancements in graph unlearning models have enhanced model utility by preserving the node representation essentially invariant, while using gradient ascent on the forget set to achieve unlearning. However, this approach causes a drastic degradation in model utility during the unlearning process due to the rapid divergence speed of gradient ascent. In this paper, we introduce \textbf{INPO}, an \textbf{I}nfluence-aware \textbf{N}egative \textbf{P}reference \textbf{O}ptimization framework that focuses on slowing the divergence speed and improving the robustness of the model utility to the unlearning process. Specifically, we first analyze that NPO has slower divergence speed and theoretically propose that unlearning high-influence edges can reduce impact of unlearning. We design an influence-aware message function to amplify the influence of unlearned edges and mitigate the tight topological coupling between the forget set and the retain set. The influence of each edge is quickly estimated by a removal-based method. Additionally, we propose a topological entropy loss from the perspective of topology to avoid excessive information loss in the local structure during unlearning. Extensive experiments conducted on five real-world datasets demonstrate that INPO-based model achieves state-of-the-art performance on all forget quality metrics while maintaining the model's utility. Codes are available at \href{https://github.com/sh-qiangchen/INPO}{https://github.com/sh-qiangchen/INPO}.

Graph Unlearning Meets Influence-aware Negative Preference Optimization

TL;DR

The paper tackles graph unlearning by reframing edge forgetting as a negative preference optimization task and addressing topological coupling that degrades model utility during unlearning. It introduces INPO, which combines an influence-aware message passing scheme, a fast edge-influence estimator, and a topological entropy loss, forming the holistic objective . Empirical results on five real-world datasets show that INPO achieves state-of-the-art forget quality while preserving utility, with ablations confirming the contribution of each component. The work advances practical graph unlearning by delivering robust forgetting under varying deletion scenarios and provides code to enable reproducibility and wider adoption.

Abstract

Recent advancements in graph unlearning models have enhanced model utility by preserving the node representation essentially invariant, while using gradient ascent on the forget set to achieve unlearning. However, this approach causes a drastic degradation in model utility during the unlearning process due to the rapid divergence speed of gradient ascent. In this paper, we introduce \textbf{INPO}, an \textbf{I}nfluence-aware \textbf{N}egative \textbf{P}reference \textbf{O}ptimization framework that focuses on slowing the divergence speed and improving the robustness of the model utility to the unlearning process. Specifically, we first analyze that NPO has slower divergence speed and theoretically propose that unlearning high-influence edges can reduce impact of unlearning. We design an influence-aware message function to amplify the influence of unlearned edges and mitigate the tight topological coupling between the forget set and the retain set. The influence of each edge is quickly estimated by a removal-based method. Additionally, we propose a topological entropy loss from the perspective of topology to avoid excessive information loss in the local structure during unlearning. Extensive experiments conducted on five real-world datasets demonstrate that INPO-based model achieves state-of-the-art performance on all forget quality metrics while maintaining the model's utility. Codes are available at \href{https://github.com/sh-qiangchen/INPO}{https://github.com/sh-qiangchen/INPO}.
Paper Structure (33 sections, 2 theorems, 29 equations, 16 figures, 20 tables)

This paper contains 33 sections, 2 theorems, 29 equations, 16 figures, 20 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Graph Unlearning
  4. Graph Neural Network
  5. Negative Preference Optimization
  6. Robustness Against Unlearning
  7. Methodology
  8. Fast Estimation of Edge Influence
  9. Influence-Enhanced MPNN
  10. Topological Entropy
  11. Complexity Analysis
  12. Experiments
  13. Experimental Settings
  14. Datasets.
  15. Baseline Models.
  16. ...and 18 more sections

Key Result

lemma 1

If the predicted probability of unlearned model much less than the counterpart of original pre-trained model on the forget set, i.e., $\pi_\theta(y \mid x) \ll \pi_{ref}(y \mid x)$, the performance decrease on the retain set is slow.

Figures (16)

  • Figure 1: The accuracy and probability curve using NPO on RT and FT of DBLP. RT and FT denote the retain set and the forget set, respectively.
  • Figure 2: Overview of our INPO. Achieving graph unlearning by preference optimization considering topological decoupling.
  • Figure 3: The predicted probability after unlearning high-influence edges on DBLP.
  • Figure 4: The prediction probability change curve of NPO and INPO on Cora validation dataset.
  • Figure 5: The AUC change curve of INPO-S and GNNDelete on retain set and forget set for DBLP validation dataset.
  • ...and 11 more figures

Theorems & Definitions (2)

  • lemma 1
  • proposition 1