Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Polynomial Contrastive Learning for Privacy-Preserving Representation Learning on Graphs

Daksh Pandey

TL;DR

This work tackles privacy concerns in graph representation learning by enabling self-supervised learning to run over encrypted data. It introduces Poly-GRACE, a fully polynomial-friendly framework that replaces non-polynomial components of GRACE with polynomial equivalents, including a quadratic activation and a polynomial self-supervised objective suitable for homomorphic encryption. The authors analyze the HE compatibility, emphasizing fixed-depth computation and noise management, and validate the approach on Cora, CiteSeer, and PubMed, showing private pre-training can be competitive and even advantageous on certain datasets. The study advances practical privacy-preserving graph learning and outlines directions for scaling to larger graphs and extending to more complex architectures such as Graph Transformers.

Abstract

Self-supervised learning (SSL) has emerged as a powerful paradigm for learning representations on graph data without requiring manual labels. However, leading SSL methods like GRACE are fundamentally incompatible with privacy-preserving technologies such as Homomorphic Encryption (HE) due to their reliance on non-polynomial operations. This paper introduces Poly-GRACE, a novel framework for HE-compatible self-supervised learning on graphs. Our approach consists of a fully polynomial-friendly Graph Convolutional Network (GCN) encoder and a novel, polynomial-based contrastive loss function. Through experiments on three benchmark datasets -- Cora, CiteSeer, and PubMed -- we demonstrate that Poly-GRACE not only enables private pre-training but also achieves performance that is highly competitive with, and in the case of CiteSeer, superior to the standard non-private baseline. Our work represents a significant step towards practical and high-performance privacy-preserving graph representation learning.

Polynomial Contrastive Learning for Privacy-Preserving Representation Learning on Graphs

TL;DR

This work tackles privacy concerns in graph representation learning by enabling self-supervised learning to run over encrypted data. It introduces Poly-GRACE, a fully polynomial-friendly framework that replaces non-polynomial components of GRACE with polynomial equivalents, including a quadratic activation and a polynomial self-supervised objective suitable for homomorphic encryption. The authors analyze the HE compatibility, emphasizing fixed-depth computation and noise management, and validate the approach on Cora, CiteSeer, and PubMed, showing private pre-training can be competitive and even advantageous on certain datasets. The study advances practical privacy-preserving graph learning and outlines directions for scaling to larger graphs and extending to more complex architectures such as Graph Transformers.

Abstract

Self-supervised learning (SSL) has emerged as a powerful paradigm for learning representations on graph data without requiring manual labels. However, leading SSL methods like GRACE are fundamentally incompatible with privacy-preserving technologies such as Homomorphic Encryption (HE) due to their reliance on non-polynomial operations. This paper introduces Poly-GRACE, a novel framework for HE-compatible self-supervised learning on graphs. Our approach consists of a fully polynomial-friendly Graph Convolutional Network (GCN) encoder and a novel, polynomial-based contrastive loss function. Through experiments on three benchmark datasets -- Cora, CiteSeer, and PubMed -- we demonstrate that Poly-GRACE not only enables private pre-training but also achieves performance that is highly competitive with, and in the case of CiteSeer, superior to the standard non-private baseline. Our work represents a significant step towards practical and high-performance privacy-preserving graph representation learning.

Paper Structure

This paper contains 20 sections, 1 equation, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Graph Neural Networks
  4. Self-Supervised Learning on Graphs
  5. Homomorphic Encryption in Machine Learning
  6. Methodology
  7. The Poly-GRACE Framework
  8. Polynomial-Friendly Encoder
  9. Polynomial Self-Supervised Objective
  10. Homomorphic Encryption Compatibility Analysis
  11. Experiments
  12. Experimental Setup
  13. Datasets
  14. Baselines
  15. Implementation Details
  16. ...and 5 more sections

Figures (3)

  • Figure 1: The Poly-GRACE Framework. A graph is augmented into two views, which are encoded by a polynomial-friendly GCN. The resulting embeddings are compared using our novel polynomial loss function to train the encoder.
  • Figure 2: t-SNE visualization of node embeddings learned on the Cora dataset. Both methods learn to separate the classes into distinct clusters, demonstrating the effectiveness of the self-supervised pre-training. The quality of the Poly-GRACE embeddings is visually comparable to the non-private baseline.
  • Figure 3: Sensitivity analysis of Poly-GRACE's performance on the Cora dataset with respect to the regularization hyperparameter $\lambda$.