Towards representation agnostic probabilistic programming

TL;DR

The paper tackles the limitation in probabilistic programming where model representations are tightly coupled to inference algorithms, hindering experimentation with new or hybrid representations. It introduces a representation-agnostic factor API built around core operations (multiplication, sum-out, marginalization, conditioning, division) with optional smoothing, addition, and mixture, enabling a universal algebra over factors $f_{XY}: X\times Y \rightarrow \mathbb{R}$. Through a toy hybrid model with a continuous state $F_t\in\mathbb{R}^2$ and a discrete state $S_t\in\{0,1,2,3\}$, it demonstrates how the same factor operations can realize both parametric and sampling-based representations, facilitating mixing within a single model. This decoupling of representation from inference supports practical inference in complex hybrid models and points toward representation-flexible probabilistic programming and richer probabilistic program typing for future research.

Abstract

Current probabilistic programming languages and tools tightly couple model representations with specific inference algorithms, preventing experimentation with novel representations or mixed discrete-continuous models. We introduce a factor abstraction with five fundamental operations that serve as a universal interface for manipulating factors regardless of their underlying representation. This enables representation-agnostic probabilistic programming where users can freely mix different representations (e.g. discrete tables, Gaussians distributions, sample-based approaches) within a single unified framework, allowing practical inference in complex hybrid models that current toolkits cannot adequately express.

Figures (2)

• Figure 1: (Left) Graphical representation of dependencies of the model and (Right) an illustration of the quadrants and their linear transition models.
• Figure 2: Sampling versus parametric based representation. The mean marginal $F_{1:T}$ state trajectory given circular observations.