Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Maxitive Donsker-Varadhan Formulation for Possibilistic Variational Inference

Jasraj Singh, Shelvia Wongso, Jeremie Houssineau, Badr-Eddine Chérief-Abdellatif

TL;DR

This paper extends variational inference to the realm of possibility theory by deriving a maxitive Donsker–Varadhan analogue, enabling principled inference under epistemic uncertainty. It introduces a maxitive posterior $g^{\star}_{\text{max}}$ and a pair of consistency bounds (CBOs) that parallel ELBO/VI bounds in probability theory, along with a unifying framework for exponential-family possibilistic VI. The work shows how conjugate priors and exponential-family forms translate into the possibilistic setting, yielding tractable updates and gradient-based optimisation rules. Overall, the approach provides a robust, non-additive alternative to probabilistic VI with practical gradient-based methods for exponential-family models.

Abstract

Variational inference (VI) is a cornerstone of modern Bayesian learning, enabling approximate inference in complex models that would otherwise be intractable. However, its formulation depends on expectations and divergences defined through high-dimensional integrals, often rendering analytical treatment impossible and necessitating heavy reliance on approximate learning and inference techniques. Possibility theory, an imprecise probability framework, allows to directly model epistemic uncertainty instead of leveraging subjective probabilities. While this framework provides robustness and interpretability under sparse or imprecise information, adapting VI to the possibilistic setting requires rethinking core concepts such as entropy and divergence, which presuppose additivity. In this work, we develop a principled formulation of possibilistic variational inference and apply it to a special class of exponential-family functions, highlighting parallels with their probabilistic counterparts and revealing the distinctive mathematical structures of possibility theory.

Maxitive Donsker-Varadhan Formulation for Possibilistic Variational Inference

TL;DR

This paper extends variational inference to the realm of possibility theory by deriving a maxitive Donsker–Varadhan analogue, enabling principled inference under epistemic uncertainty. It introduces a maxitive posterior and a pair of consistency bounds (CBOs) that parallel ELBO/VI bounds in probability theory, along with a unifying framework for exponential-family possibilistic VI. The work shows how conjugate priors and exponential-family forms translate into the possibilistic setting, yielding tractable updates and gradient-based optimisation rules. Overall, the approach provides a robust, non-additive alternative to probabilistic VI with practical gradient-based methods for exponential-family models.

Abstract

Variational inference (VI) is a cornerstone of modern Bayesian learning, enabling approximate inference in complex models that would otherwise be intractable. However, its formulation depends on expectations and divergences defined through high-dimensional integrals, often rendering analytical treatment impossible and necessitating heavy reliance on approximate learning and inference techniques. Possibility theory, an imprecise probability framework, allows to directly model epistemic uncertainty instead of leveraging subjective probabilities. While this framework provides robustness and interpretability under sparse or imprecise information, adapting VI to the possibilistic setting requires rethinking core concepts such as entropy and divergence, which presuppose additivity. In this work, we develop a principled formulation of possibilistic variational inference and apply it to a special class of exponential-family functions, highlighting parallels with their probabilistic counterparts and revealing the distinctive mathematical structures of possibility theory.

Paper Structure

This paper contains 21 sections, 7 theorems, 97 equations, 1 table.

Table of Contents

  1. Introduction
  2. Primer on Possibility Theory
  3. Marginalization and conditioning:
  4. Possibilistic moments:
  5. Notations:
  6. Bayesian Inference and the Donsker-Varadhan Variational Formula
  7. (Generalized) Bayesian inference via Donsker-Varadhan.
  8. (Generalized) Variational inference via Donsker-Varadhan.
  9. Possibilistic Inference and a Maxitive Donsker--Varadhan Analogue
  10. Canonical choice of maximizer:
  11. Recovering the posterior:
  12. Relationship with VI:
  13. Relationship with Generalized VI:
  14. Possibilistic VI with Exponential Families
  15. Bayesian Inference with Conjugate Family
  16. ...and 6 more sections

Key Result

Theorem 1

Let $(\Theta,\mathcal{T})$ be a measurable space and let $\nu$ be a probability measure on $\Theta$. For any measurable function $h:\Theta\to\mathbb{R}$ such that $\int e^{h} \mathrm{d}\nu<+\infty$, we have where $\mathcal{P}(\Theta)$ denotes the set of probability measures on $\Theta$ and $\mathrm{KL}(\rho\Vert\nu)$ is the Kullback--Leibler divergence of $\rho$ with respect to $\nu$, with the co

Theorems & Definitions (22)

  • Example 1
  • Theorem 1: Donsker and Varadhan's variational formula
  • Theorem 2: Maxitive Donsker-Varadhan formula
  • Example 2
  • Example 3
  • Example 4
  • Proposition 1
  • Proposition 2
  • Proposition 3
  • Corollary 1
  • ...and 12 more