Modular Probabilistic Models via Algebraic Effects

TL;DR

The paper tackles modular, reusable probabilistic modeling and multimodal inference in probabilistic programming languages by introducing ProbFX, a deeply embedded PPL in Haskell that uses algebraic effects to separate syntax from semantics. Models are first-class and multimodal, with simulation and inference implemented as composable effect handlers that operate over model environments binding observable variables. A SIR/HMM case study demonstrates how to compose modular submodels and extend them with higher order components, as well as how to perform Bayesian bootstrapping by reusing simulated outputs for inference. The embedding relies on a Freer monad style representation with Dist and ObsReader effects, and it provides core handlers for specialization, simulation, and inference that yield competitive performance relative to state of the art PPLs while offering strong type safety and expressivity for multimodal and higher order models.

Abstract

Probabilistic programming languages (PPLs) allow programmers to construct statistical models and then simulate data or perform inference over them. Many PPLs restrict models to a particular instance of simulation or inference, limiting their reusability. In other PPLs, models are not readily composable. Using Haskell as the host language, we present an embedded domain specific language based on algebraic effects, where probabilistic models are modular, first-class, and reusable for both simulation and inference. We also demonstrate how simulation and inference can be expressed naturally as composable program transformations using algebraic effect handlers.

Figures (10)

• Figure 1: Visualising Linear Regression
• Figure 2: Hidden Markov Model
• Figure 3: A Modular Hidden Markov Model
• Figure 4: SIR Hidden Markov Model Simulation
• Figure 5: SIR Inference (Metropolis Hastings)