Multi-Language Probabilistic Programming

TL;DR

MultiPPL addresses the challenge of mixing disparate probabilistic inference strategies by embedding two languages, Disc for exact discrete inference and Cont for approximate continuous inference, into a single multi-language framework. It provides a formal syntax, typing, and dual semantic models (high-level and low-level) and proves a soundness result ensuring that inter-language execution preserves the intended probability distribution. Empirical evaluation on networking and Bayesian network tasks demonstrates improved sample quality and competitive efficiency, confirming practical viability and guiding boundary placement trade-offs. The work advances a rigorous, modular approach to heterogeneous probabilistic programming and lays groundwork for extending interoperation to richer inference techniques.

Abstract

There are many different probabilistic programming languages that are specialized to specific kinds of probabilistic programs. From a usability and scalability perspective, this is undesirable: today, probabilistic programmers are forced up-front to decide which language they want to use and cannot mix-and-match different languages for handling heterogeneous programs. To rectify this, we seek a foundation for sound interoperability for probabilistic programming languages: just as today's Python programmers can resort to low-level C programming for performance, we argue that probabilistic programmers should be able to freely mix different languages for meeting the demands of heterogeneous probabilistic programming environments. As a first step towards this goal, we introduce \textsc{MultiPPL}, a probabilistic multi-language that enables programmers to interoperate between two different probabilistic programming languages: one that leverages a high-performance exact discrete inference strategy, and one that uses approximate importance sampling. We give a syntax and semantics for \textsc{MultiPPL}, prove soundness of its inference algorithm, and provide empirical evidence that it enables programmers to perform inference on complex heterogeneous probabilistic programs and flexibly exploits the strengths and weaknesses of two languages simultaneously.%

Figures (20)

• Figure 1: A small network and a fragment of a probabilistic program encoding of the packet arrival problem.
• Figure 2: Syntax of MultiPPL. In Disc, we require ${{\color{orange40} e}} \in [0,1]$ for ${\color{indigo50} \mathsf{flip}}$. In Cont, the syntax of distributions ${\color{orange40} d}$ denotes probability distributions. In ${\color{orange40} \mathsf{obs}}$ these terms condition, otherwise they are immediately sampled.
• Figure 3: Motivating example showing the compilation of the Disc program in \ref{['fig:etwocoins:program']} to BDDs in \ref{['fig:etwocoins:bdds']}. On the left of \ref{['fig:etwocoins:bdds']} is a BDD representing the distribution formula $\varphi = f_X$; on the right is the BDD representing the accepting formula $\alpha = f_X \lor f_Y$. T and F represent true and false values, respectively.
• Figure 4: Interpreting Cont values in Disc.
• Figure 5: Rules for convertibility between Disc types ${\color{indigo50} A}$ and Cont types ${\color{orange40} \tau}$.

Theorems & Definitions (37)

• Lemma 3.1: natural embedding
• proof
• Definition 3.2
• Definition 3.3
• Definition 3.4
• Definition 3.5
• Theorem 3.6: soundness
• Lemma 3.7
• Lemma 3.8
• Lemma A.1