Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DeGAS: Gradient-Based Optimization of Probabilistic Programs without Sampling

Francesca Randone, Romina Doz, Mirco Tribastone, Luca Bortolussi

TL;DR

DeGAS provides a sample-free, gradient-based optimization framework for probabilistic programs by introducing a differentiable Gaussian-approximate semantics that smoothly relaxes discrete and measure-zero constructs. It builds on the SOGA Gaussian-mixture semantics to yield closed-form posterior and path probabilities under a vanishing smoothing parameter $oldsymbol{}$, with differentiability in program parameters guaranteed via a moment-matching operator. The authors prove differentiability and convergence, implement the approach in PyTorch, and demonstrate competitive accuracy and runtimes against VI and MCMC on thirteen benchmarks, including challenging continuous-conditioned branching. This enables end-to-end optimization of probabilistic programs without Monte Carlo estimators, offering a principled tool for optimizing discontinuous models and cyber-physical-system-style applications where sampling struggles to converge.

Abstract

We present DeGAS, a differentiable Gaussian approximate semantics for loopless probabilistic programs that enables sample-free, gradient-based optimization in models with both continuous and discrete components. DeGAS evaluates programs under a Gaussian-mixture semantics and replaces measure-zero predicates and discrete branches with a vanishing smoothing, yielding closed-form expressions for posterior and path probabilities. We prove differentiability of these quantities with respect to program parameters, enabling end-to-end optimization via standard automatic differentiation, without Monte Carlo estimators. On thirteen benchmark programs, DeGAS achieves accuracy and runtime competitive with variational inference and MCMC. Importantly, it reliably tackles optimization problems where sampling-based baselines fail to converge due to conditioning involving continuous variables.

DeGAS: Gradient-Based Optimization of Probabilistic Programs without Sampling

TL;DR

DeGAS provides a sample-free, gradient-based optimization framework for probabilistic programs by introducing a differentiable Gaussian-approximate semantics that smoothly relaxes discrete and measure-zero constructs. It builds on the SOGA Gaussian-mixture semantics to yield closed-form posterior and path probabilities under a vanishing smoothing parameter , with differentiability in program parameters guaranteed via a moment-matching operator. The authors prove differentiability and convergence, implement the approach in PyTorch, and demonstrate competitive accuracy and runtimes against VI and MCMC on thirteen benchmarks, including challenging continuous-conditioned branching. This enables end-to-end optimization of probabilistic programs without Monte Carlo estimators, offering a principled tool for optimizing discontinuous models and cyber-physical-system-style applications where sampling struggles to converge.

Abstract

We present DeGAS, a differentiable Gaussian approximate semantics for loopless probabilistic programs that enables sample-free, gradient-based optimization in models with both continuous and discrete components. DeGAS evaluates programs under a Gaussian-mixture semantics and replaces measure-zero predicates and discrete branches with a vanishing smoothing, yielding closed-form expressions for posterior and path probabilities. We prove differentiability of these quantities with respect to program parameters, enabling end-to-end optimization via standard automatic differentiation, without Monte Carlo estimators. On thirteen benchmark programs, DeGAS achieves accuracy and runtime competitive with variational inference and MCMC. Importantly, it reliably tackles optimization problems where sampling-based baselines fail to converge due to conditioning involving continuous variables.
Paper Structure (12 sections, 1 theorem, 11 equations, 3 figures)

This paper contains 12 sections, 1 theorem, 11 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Structure of the paper.
  3. Related Work
  4. Syntax and Semantics
  5. Notation.
  6. Syntax
  7. Semantics
  8. Non-differentiability of SOGA semantics.
  9. Smoothing operator for Boolean predicates.
  10. DeGAS.
  11. Example 1.
  12. Properties of $\llbracket{\cdot}\rrbracket^{\Delta, \epsilon}$

Key Result

theorem thmcountertheorem

For any valid program $P(\Theta)$, $\llbracket{P(\Theta)}\rrbracket^{\Delta, \epsilon}$ returns $(p'(\Theta), D'(\Theta), V')$ such that $p'(\Theta)$ and $D'(\Theta)$ are differentiable in $\Theta$.

Figures (3)

  • Figure 1: Nondifferentiability of SOGA semantics. Examples yielding singular covariance matrices.
  • Figure 2: Example program $P$ (left) and its cfg (right).
  • Figure 3: Marginal posterior densities of program $P$ under different semantics. The orange lines highlight the different components of the mixtures.

Theorems & Definitions (1)

  • theorem thmcountertheorem