Scalable and Certifiable Graph Unlearning: Overcoming the Approximation Error Barrier
Lu Yi, Zhewei Wei
TL;DR
ScaleGUN tackles the bottleneck of certifiable graph unlearning on billion-edge graphs by integrating an approximate, lazy local propagation framework with Generalized PageRank into the certified unlearning pipeline. It derives theoretical bounds showing that approximation-induced model error remains bounded and thus can be masked by noise to preserve $(oldsymbol{ psilon},oldsymbol{ elta})$ guarantees for node feature, edge, and node unlearning. The approach achieves remarkable practical speedups (e.g., 20 seconds total for 5,000 random edge removals on ogbn-papers100M, with 5 seconds for embeddings) while maintaining competitive accuracy on large graphs, outperforming retraining in propagation cost. Empirically, ScaleGUN demonstrates strong unlearning efficacy, data-dependent bounds, and robustness under various unlearning settings, and extends to PPR-based models and spectral GNNs as backbones. This work paves the way for scalable privacy-preserving graph learning in industrial-scale networks, balancing privacy, utility, and computational efficiency.
Abstract
Graph unlearning has emerged as a pivotal research area for ensuring privacy protection, given the widespread adoption of Graph Neural Networks (GNNs) in applications involving sensitive user data. Among existing studies, certified graph unlearning is distinguished by providing robust privacy guarantees. However, current certified graph unlearning methods are impractical for large-scale graphs because they necessitate the costly re-computation of graph propagation for each unlearning request. Although numerous scalable techniques have been developed to accelerate graph propagation for GNNs, their integration into certified graph unlearning remains uncertain as these scalable approaches introduce approximation errors into node embeddings. In contrast, certified graph unlearning demands bounded model error on exact node embeddings to maintain its certified guarantee. To address this challenge, we present ScaleGUN, the first approach to scale certified graph unlearning to billion-edge graphs. ScaleGUN integrates the approximate graph propagation technique into certified graph unlearning, offering certified guarantees for three unlearning scenarios: node feature, edge, and node unlearning. Extensive experiments on real-world datasets demonstrate the efficiency and unlearning efficacy of ScaleGUN. Remarkably, ScaleGUN accomplishes $(ε,δ)=(1,10^{-4})$ certified unlearning on the billion-edge graph ogbn-papers100M in 20 seconds for a 5,000 random edge removal request -- of which only 5 seconds are required for updating the node embeddings -- compared to 1.91 hours for retraining and 1.89 hours for re-propagation. Our code is available at https://github.com/luyi256/ScaleGUN.