Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Pyro: Deep Universal Probabilistic Programming

Eli Bingham, Jonathan P. Chen, Martin Jankowiak, Fritz Obermeyer, Neeraj Pradhan, Theofanis Karaletsos, Rohit Singh, Paul Szerlip, Paul Horsfall, Noah D. Goodman

TL;DR

Pyro presents a Python-embedded probabilistic programming framework built on PyTorch, unifying expressive model construction with scalable, gradient-based variational inference through SVI and modular inference via Poutine. It argues for balancing expressivity, scalability, flexibility, and minimality, and demonstrates practicality through VAE and DMM experiments that reproduce prior results while maintaining competitive performance. The paper also discusses openness (MIT-licensed, community-driven) and situates Pyro among existing PPLs, highlighting its design choices and extensibility. Overall, Pyro offers a flexible, scalable platform for developing advanced AI models with probabilistic reasoning. The combination of Python ergonomics, GPU acceleration, and effect-handling abstractions positions Pyro as a versatile tool for research-driven probabilistic modeling.

Abstract

Pyro is a probabilistic programming language built on Python as a platform for developing advanced probabilistic models in AI research. To scale to large datasets and high-dimensional models, Pyro uses stochastic variational inference algorithms and probability distributions built on top of PyTorch, a modern GPU-accelerated deep learning framework. To accommodate complex or model-specific algorithmic behavior, Pyro leverages Poutine, a library of composable building blocks for modifying the behavior of probabilistic programs.

Pyro: Deep Universal Probabilistic Programming

TL;DR

Pyro presents a Python-embedded probabilistic programming framework built on PyTorch, unifying expressive model construction with scalable, gradient-based variational inference through SVI and modular inference via Poutine. It argues for balancing expressivity, scalability, flexibility, and minimality, and demonstrates practicality through VAE and DMM experiments that reproduce prior results while maintaining competitive performance. The paper also discusses openness (MIT-licensed, community-driven) and situates Pyro among existing PPLs, highlighting its design choices and extensibility. Overall, Pyro offers a flexible, scalable platform for developing advanced AI models with probabilistic reasoning. The combination of Python ergonomics, GPU acceleration, and effect-handling abstractions positions Pyro as a versatile tool for research-driven probabilistic modeling.

Abstract

Pyro is a probabilistic programming language built on Python as a platform for developing advanced probabilistic models in AI research. To scale to large datasets and high-dimensional models, Pyro uses stochastic variational inference algorithms and probability distributions built on top of PyTorch, a modern GPU-accelerated deep learning framework. To accommodate complex or model-specific algorithmic behavior, Pyro leverages Poutine, a library of composable building blocks for modifying the behavior of probabilistic programs.

Paper Structure

This paper contains 5 sections, 4 figures.

Table of Contents

  1. Introduction
  2. Design Principles
  3. Project Openness and Development
  4. Existing Systems
  5. Experiments

Figures (4)

  • Figure 1: A complete Pyro example: the generative model (model), approximate posterior (guide), constraint specification (conditioned_model), and stochastic variational inference (svi, loss) in a variational autoencoder. encoder is a torch.nn.Module object. pyro.module calls pyro.param on every parameter of a torch.nn.Module.
  • Figure 2: Simplified summary of design principles of Pyro and some other PPLs.
  • Figure 3: Times per update of VAE in Pyro versus PyTorch
  • Figure 4: Test ELBOs for DMM and extension with IAF guide