Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Conformal Inductive Graph Neural Networks

Soroush H. Zargarbashi, Aleksandar Bojchevski

TL;DR

This work tackles uncertainty quantification for inductive graph neural networks by adapting conformal prediction to dynamic, non-iid graph sequences. By introducing NodeEx CP for node-exchangeable graphs and EdgeEx CP for edge-exchangeable graphs, the authors restore the $1 - \alpha$ coverage guarantee despite score shifts caused by new nodes or edges, ensuring validity at any prediction time. The approach is supported by theory that conditioning conformity scores on the current subgraph yields exact or near-exact coverage, and by experiments showing robust coverage with improved efficiency (smaller prediction sets and higher singleton hits) across multiple datasets and scoring schemes. The practical impact is reliable, model-agnostic uncertainty quantification for evolving graphs, enabling safer deployment of GNNs in real-world, time-evolving networks.

Abstract

Conformal prediction (CP) transforms any model's output into prediction sets guaranteed to include (cover) the true label. CP requires exchangeability, a relaxation of the i.i.d. assumption, to obtain a valid distribution-free coverage guarantee. This makes it directly applicable to transductive node-classification. However, conventional CP cannot be applied in inductive settings due to the implicit shift in the (calibration) scores caused by message passing with the new nodes. We fix this issue for both cases of node and edge-exchangeable graphs, recovering the standard coverage guarantee without sacrificing statistical efficiency. We further prove that the guarantee holds independently of the prediction time, e.g. upon arrival of a new node/edge or at any subsequent moment.

Conformal Inductive Graph Neural Networks

TL;DR

This work tackles uncertainty quantification for inductive graph neural networks by adapting conformal prediction to dynamic, non-iid graph sequences. By introducing NodeEx CP for node-exchangeable graphs and EdgeEx CP for edge-exchangeable graphs, the authors restore the coverage guarantee despite score shifts caused by new nodes or edges, ensuring validity at any prediction time. The approach is supported by theory that conditioning conformity scores on the current subgraph yields exact or near-exact coverage, and by experiments showing robust coverage with improved efficiency (smaller prediction sets and higher singleton hits) across multiple datasets and scoring schemes. The practical impact is reliable, model-agnostic uncertainty quantification for evolving graphs, enabling safer deployment of GNNs in real-world, time-evolving networks.

Abstract

Conformal prediction (CP) transforms any model's output into prediction sets guaranteed to include (cover) the true label. CP requires exchangeability, a relaxation of the i.i.d. assumption, to obtain a valid distribution-free coverage guarantee. This makes it directly applicable to transductive node-classification. However, conventional CP cannot be applied in inductive settings due to the implicit shift in the (calibration) scores caused by message passing with the new nodes. We fix this issue for both cases of node and edge-exchangeable graphs, recovering the standard coverage guarantee without sacrificing statistical efficiency. We further prove that the guarantee holds independently of the prediction time, e.g. upon arrival of a new node/edge or at any subsequent moment.
Paper Structure (23 sections, 6 theorems, 11 equations, 18 figures, 2 tables)

This paper contains 23 sections, 6 theorems, 11 equations, 18 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background: Conformal Prediction and Transductive GNNs
  3. Conformal Prediction for Transductive Node-Classification
  4. Inductive GNNs under Node and Edge Exchangeability
  5. Conformal Prediction for Exchangeable Graph Sequences
  6. Node-exchangeable Sequences
  7. Edge-exchangeable Sequences
  8. Node Exchangeable and Edge Exchangeable CP
  9. Experimental Evaluation
  10. Experimental setup and discussion of metrics
  11. Evaluation of empirical coverage, set size and singleton hit
  12. Conclusion
  13. Algorithm for NodeEx CP and Weighted EdgeEx CP
  14. Addition to the Theory
  15. Limitations of NAPS
  16. ...and 8 more sections

Key Result

Theorem 1

Let $\{({\bm{x}}_i, y_i)\}_{i=1}^{n}$, and $({\bm{x}}_{n+1}, y_{n+1})$ be exchangeable. With any continuous function $s: {\mathcal{X}} \times {\mathcal{Y}} \mapsto {\mathbb{R}}$ measuring the agreement between ${\bm{x}}$, and $y$, and user-specified significance level $\alpha \in (0, 1)$, with predi

Figures (18)

  • Figure 1: [Left] Each vertical line on the heatmap shows sorted true test scores at each timestep. The dashed line shows the true (unknown) $\alpha$-quantile and the quantile from each approach is also shown alongside. NodeEx CP (ours) closely tracks the true quantile, while naive CP deviates over time. [Upper right] Distributions from selected timesteps marked by the same color on the heatmap. The distribution shift is observable over time with new nodes appearing. [Lower right] The earth mover distance (EMD) between naive CP calibration scores and shifted true scores, denoted as "Test"; and EMD between naive and NodeEx CP scores, denoted as "Cal". Details in \ref{['subsec:other-expr']}.
  • Figure 2: $1-{\bm{C}}$ for Cora. Details in \ref{['sec:proofs']}
  • Figure 3: [Upper left] Coverage over time under node exchangeability when predicting upon node arrival (diagonals of ${\bm{C}}$). [Upper right] Coverage when we instead predict at a fixed time (columns of ${\bm{C}}$). [Lower right] Same as upper right but under edge-exchangeability. [Lower left] The empirical distribution of coverage when we predict at node-specific times (fixed entries of ${\bm{C}}$), compared to the theoretical distribution. Sample size results in a slight expected shift. The transparent lines show a particular sequence, and the thick solid lines shows the average over 10 (15) sequences.
  • Figure 4: [Left] Average set size (lower is better) and [Right] singleton hits ratio (higher is better) of naive CP vs. NodeEx CP for CiteSeer and GCN. Our approach improves both metrics.
  • Figure 5: NodeEx and EdgeEx CP for inductive node classification
  • ...and 13 more figures

Theorems & Definitions (13)

  • Theorem 1: Vovk2005AlgorithmicLI
  • Theorem 2: Rephrasing of Theorem 3 by huang2023uncertainty
  • Proposition 1
  • Theorem 3
  • Lemma 1
  • Theorem 4
  • Definition 1: Node-exchangeability
  • Definition 2: Edge-exchangeability
  • Definition 3: Node-inductive sequence
  • Definition 4: Edge-inductive sequence
  • ...and 3 more