Convergent Privacy Framework for Multi-layer GNNs through Contractive Message Passing
Yu Zheng, Chenang Li, Zhou Li, Qingsong Wang
TL;DR
The paper tackles the challenge of applying differential privacy to deep GNNs, where noise must typically grow with depth, harming utility. It introduces CARIBOU, a framework built on Contractive Graph Layers (CGL) and three modules—Contractive Aggregation Module, Privacy Allocation Module, and Privacy Auditing Module—to achieve a convergent privacy budget via contractive message passing and privacy amplification. A Contractive Noisy Iteration-based analysis underpins formal edge- and node-level DP guarantees, while empirical results on nine datasets show improved privacy-utility trade-offs and robust privacy auditing against membership-inference attacks. This work enables reliable, deeper private GNNs for graph learning tasks with long-range interactions, providing practical mechanisms and theoretical guarantees for privacy-preserving graph learning at scale.
Abstract
Differential privacy (DP) has been integrated into graph neural networks (GNNs) to protect sensitive structural information, e.g., edges, nodes, and associated features across various applications. A prominent approach is to perturb the message-passing process, which forms the core of most GNN architectures. However, existing methods typically incur a privacy cost that grows linearly with the number of layers (e.g., GAP published in Usenix Security'23), ultimately requiring excessive noise to maintain a reasonable privacy level. This limitation becomes particularly problematic when multi-layer GNNs, which have shown better performance than one-layer GNN, are used to process graph data with sensitive information. In this paper, we theoretically establish that the privacy budget converges with respect to the number of layers by applying privacy amplification techniques to the message-passing process, exploiting the contractive properties inherent to standard GNN operations. Motivated by this analysis, we propose a simple yet effective Contractive Graph Layer (CGL) that ensures the contractiveness required for theoretical guarantees while preserving model utility. Our framework, CARIBOU, supports both training and inference, equipped with a contractive aggregation module, a privacy allocation module, and a privacy auditing module. Experimental evaluations demonstrate that CARIBOU significantly improves the privacy-utility trade-off and achieves superior performance in privacy auditing tasks.