Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Uncertainty for Active Learning on Graphs

Dominik Fuchsgruber, Tom Wollschläger, Bertrand Charpentier, Antonio Oroz, Stephan Günnemann

TL;DR

This work addresses the data-inefficiency of graph-based node classification by re-examining Uncertainty Sampling (US) through the lens of uncertainty disentanglement. It formalizes ground-truth uncertainty from the data-generating process using a Bayesian classifier and proves that selecting nodes with maximal epistemic uncertainty optimizes the posterior gain on all unobserved labels. Since true ground-truth uncertainty is unavailable in practice, the authors propose tractable approximations (MP and ESP) that effectively estimate epistemic uncertainty and demonstrate competitive performance against traditional AL baselines on CSBMs and real datasets. The results indicate that accurate epistemic uncertainty estimation—especially when coupled with a faithful generative model—can make US a viable, principled strategy for active learning on graphs, guiding future uncertainty estimators toward graph-specific AL gains.

Abstract

Uncertainty Sampling is an Active Learning strategy that aims to improve the data efficiency of machine learning models by iteratively acquiring labels of data points with the highest uncertainty. While it has proven effective for independent data its applicability to graphs remains under-explored. We propose the first extensive study of Uncertainty Sampling for node classification: (1) We benchmark Uncertainty Sampling beyond predictive uncertainty and highlight a significant performance gap to other Active Learning strategies. (2) We develop ground-truth Bayesian uncertainty estimates in terms of the data generating process and prove their effectiveness in guiding Uncertainty Sampling toward optimal queries. We confirm our results on synthetic data and design an approximate approach that consistently outperforms other uncertainty estimators on real datasets. (3) Based on this analysis, we relate pitfalls in modeling uncertainty to existing methods. Our analysis enables and informs the development of principled uncertainty estimation on graphs.

Uncertainty for Active Learning on Graphs

TL;DR

This work addresses the data-inefficiency of graph-based node classification by re-examining Uncertainty Sampling (US) through the lens of uncertainty disentanglement. It formalizes ground-truth uncertainty from the data-generating process using a Bayesian classifier and proves that selecting nodes with maximal epistemic uncertainty optimizes the posterior gain on all unobserved labels. Since true ground-truth uncertainty is unavailable in practice, the authors propose tractable approximations (MP and ESP) that effectively estimate epistemic uncertainty and demonstrate competitive performance against traditional AL baselines on CSBMs and real datasets. The results indicate that accurate epistemic uncertainty estimation—especially when coupled with a faithful generative model—can make US a viable, principled strategy for active learning on graphs, guiding future uncertainty estimators toward graph-specific AL gains.

Abstract

Uncertainty Sampling is an Active Learning strategy that aims to improve the data efficiency of machine learning models by iteratively acquiring labels of data points with the highest uncertainty. While it has proven effective for independent data its applicability to graphs remains under-explored. We propose the first extensive study of Uncertainty Sampling for node classification: (1) We benchmark Uncertainty Sampling beyond predictive uncertainty and highlight a significant performance gap to other Active Learning strategies. (2) We develop ground-truth Bayesian uncertainty estimates in terms of the data generating process and prove their effectiveness in guiding Uncertainty Sampling toward optimal queries. We confirm our results on synthetic data and design an approximate approach that consistently outperforms other uncertainty estimators on real datasets. (3) Based on this analysis, we relate pitfalls in modeling uncertainty to existing methods. Our analysis enables and informs the development of principled uncertainty estimation on graphs.
Paper Structure (27 sections, 3 theorems, 32 equations, 18 figures, 6 tables)

This paper contains 27 sections, 3 theorems, 32 equations, 18 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Related Work
  4. Benchmarking Uncertainty Sampling Approaches for Active Learning on Graphs
  5. Ground-Truth Uncertainty from the Data Generating Process
  6. Uncertainty Sampling with Ground-Truth Uncertainty
  7. Disentangling Uncertainty on Real Data
  8. Applicability to Indepenent Data
  9. Limitations
  10. Conclusion
  11. Proofs
  12. Experimental Setup
  13. Active Learning
  14. Hyperparameters
  15. Datasets
  16. ...and 12 more sections

Key Result

Theorem 5.6

Epistemic uncertainty ${\mathop{\mathrm{u}}\nolimits^{\mathrm{epi}}(i, \mathbf{y}_i^{\mathrm{gt}})}$ of a node $i$ is equivalent to the relative gain its acquisition provides to the posterior over the remaining true labels: Hence, acquiring the most epistemically uncertain node is an optimal AL strategy for $f^*_{\theta}$.

Figures (18)

  • Figure 1: US can be realized by acquiring the label of a node with with maximal total, aleatoric or epistemic uncertainty. The former two include irreducible effects leading to label node $a$ while the latter isolates epistemic factors and queries the label of node $b$, thereby increasing the confidence in correctly predicting the remaining unlabeled nodes the most.
  • Figure 2: Accuracy of AL strategies on Citeseer using a GCN (left) / SGC (right) classifier. Except for GEEM, which is only tractable for SGCs, traditional AL can not significantly outperform random selection.
  • Figure 3: US on Citeseer. No method significantly outperforms random selection.
  • Figure 4: US on a CSBM with 100 nodes and 7 classes using $f^*_\theta$. Ground-truth epistemic uncertainty significantly outperforms other estimators and random queries.
  • Figure 5: Approximation of disentangled uncertainty against existing epistemic US methods on CoraML. Our framework works well both using multiple pseudo-labels (MP) or taking an expectation over each of them (ESP).
  • ...and 13 more figures

Theorems & Definitions (11)

  • Definition 5.1
  • Definition 5.2
  • Definition 5.3
  • Definition 5.4
  • Remark 5.5
  • Theorem 5.6
  • Proposition 6.1
  • Proposition 6.2
  • proof
  • proof
  • ...and 1 more