Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improving Active Learning with a Bayesian Representation of Epistemic Uncertainty

Jake Thomas, Jeremie Houssineau

TL;DR

This work tackles the challenge of reducing epistemic uncertainty (EU) in active learning by moving beyond purely probabilistic uncertainty representations and adopting a hybrid probabilistic-possibilistic framework. It introduces outer probability measures (OPMs) and possibility functions to explicitly model EU, and defines a possibilistic Gaussian process (PGP) to enable Bayesian-like inference within this framework. Two local, update-free acquisition strategies are developed—one based on a novel EU measure and another on the necessity of correct classification—together with probabilistic class-conditional updates via Laplace approximations. Empirical results on synthetic and real datasets demonstrate that the proposed approaches yield strong performance across binary and multiclass classification tasks, often outperforming standard baselines while maintaining computational efficiency, thereby offering a robust alternative for uncertainty-aware active learning.

Abstract

A popular strategy for active learning is to specifically target a reduction in epistemic uncertainty, since aleatoric uncertainty is often considered as being intrinsic to the system of interest and therefore not reducible. Yet, distinguishing these two types of uncertainty remains challenging and there is no single strategy that consistently outperforms the others. We propose to use a particular combination of probability and possibility theories, with the aim of using the latter to specifically represent epistemic uncertainty, and we show how this combination leads to new active learning strategies that have desirable properties. In order to demonstrate the efficiency of these strategies in non-trivial settings, we introduce the notion of a possibilistic Gaussian process (GP) and consider GP-based multiclass and binary classification problems, for which the proposed methods display a strong performance for both simulated and real datasets.

Improving Active Learning with a Bayesian Representation of Epistemic Uncertainty

TL;DR

This work tackles the challenge of reducing epistemic uncertainty (EU) in active learning by moving beyond purely probabilistic uncertainty representations and adopting a hybrid probabilistic-possibilistic framework. It introduces outer probability measures (OPMs) and possibility functions to explicitly model EU, and defines a possibilistic Gaussian process (PGP) to enable Bayesian-like inference within this framework. Two local, update-free acquisition strategies are developed—one based on a novel EU measure and another on the necessity of correct classification—together with probabilistic class-conditional updates via Laplace approximations. Empirical results on synthetic and real datasets demonstrate that the proposed approaches yield strong performance across binary and multiclass classification tasks, often outperforming standard baselines while maintaining computational efficiency, thereby offering a robust alternative for uncertainty-aware active learning.

Abstract

A popular strategy for active learning is to specifically target a reduction in epistemic uncertainty, since aleatoric uncertainty is often considered as being intrinsic to the system of interest and therefore not reducible. Yet, distinguishing these two types of uncertainty remains challenging and there is no single strategy that consistently outperforms the others. We propose to use a particular combination of probability and possibility theories, with the aim of using the latter to specifically represent epistemic uncertainty, and we show how this combination leads to new active learning strategies that have desirable properties. In order to demonstrate the efficiency of these strategies in non-trivial settings, we introduce the notion of a possibilistic Gaussian process (GP) and consider GP-based multiclass and binary classification problems, for which the proposed methods display a strong performance for both simulated and real datasets.

Paper Structure

This paper contains 38 sections, 9 theorems, 51 equations, 1 figure, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Possibility Theory and OPMs
  4. Introduction and properties
  5. Gaussian Possibility Function
  6. Inference with OPMs
  7. Active Learning with Possibility Theory
  8. Measures of Epistemic Uncertainty
  9. Possibilistic Active Learning for Classification
  10. Necessity of classification
  11. Possibilistic Gaussian Processes
  12. Covariance
  13. Definition of the Possibilistic Gaussian Process
  14. Modelling and Prediction with Possibilistic Gaussian Processes
  15. Experimental Results
  16. ...and 23 more sections

Key Result

Proposition 1

The measure of uncertainty $U^{\Theta}$ satisfies Properties A0-A2, Property A3 in the multinomial setting, as well as the strict version of Property A4, and $U^{\mathsf{Y}}$ satisfies Properties A0-A3 as well as the weak version of Property A4.

Figures (1)

  • Figure 1: Performance of BALD, Random, Standard and our approaches on six binary classification problems on real-world datasets. The shaded area corresponds to the interval between the first quantile (Q1) and third quantile (Q3).

Theorems & Definitions (15)

  • Proposition 1
  • Proposition 2
  • Proposition 3
  • Proposition 4
  • Proposition 5
  • Proposition 6: Regression with Systematic/Random Error
  • Proposition 7: Binary classification
  • Proposition 8: Multiclass classification
  • Proposition 5
  • Remark 1: On the reformulation of Property A3
  • ...and 5 more