Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Categorical Stochastic Processes and Likelihood

Dan Shiebler

TL;DR

The work develops a category-theoretic framework for stochastic modeling and learning by integrating probability, stochastic processes, and parametric statistics. It introduces two core constructions, the co-Kleisli category under a product comonad and Lawvere-parameterized Para, to model compositional stochastic processes and their relationship to Stoch and CEucMeas. It then extends stochastic processes to parametric statistical models via thePara construction, defines a Radon-Nikodym semifunctor to map models to likelihoods, and builds Marginal Likelihood Factorization categories to support likelihood-based learning and backpropagation functors that map models to Learn. The approach yields a frequentist, likelihood-driven route to composing stochastic layers, deriving backpropagation mechanisms from first principles, and linking maximum likelihood with categorical structure. This framework offers a principled path toward building compositional neural architectures with stochastic layers and principled learning dynamics grounded in category theory.

Abstract

In this work we take a Category Theoretic perspective on the relationship between probabilistic modeling and function approximation. We begin by defining two extensions of function composition to stochastic process subordination: one based on the co-Kleisli category under the comonad (Omega x -) and one based on the parameterization of a category with a Lawvere theory. We show how these extensions relate to the category Stoch and other Markov Categories. Next, we apply the Para construction to extend stochastic processes to parameterized statistical models and we define a way to compose the likelihood functions of these models. We conclude with a demonstration of how the Maximum Likelihood Estimation procedure defines an identity-on-objects functor from the category of statistical models to the category of Learners. Code to accompany this paper can be found at https://github.com/dshieble/Categorical_Stochastic_Processes_and_Likelihood

Categorical Stochastic Processes and Likelihood

TL;DR

The work develops a category-theoretic framework for stochastic modeling and learning by integrating probability, stochastic processes, and parametric statistics. It introduces two core constructions, the co-Kleisli category under a product comonad and Lawvere-parameterized Para, to model compositional stochastic processes and their relationship to Stoch and CEucMeas. It then extends stochastic processes to parametric statistical models via thePara construction, defines a Radon-Nikodym semifunctor to map models to likelihoods, and builds Marginal Likelihood Factorization categories to support likelihood-based learning and backpropagation functors that map models to Learn. The approach yields a frequentist, likelihood-driven route to composing stochastic layers, deriving backpropagation mechanisms from first principles, and linking maximum likelihood with categorical structure. This framework offers a principled path toward building compositional neural architectures with stochastic layers and principled learning dynamics grounded in category theory.

Abstract

In this work we take a Category Theoretic perspective on the relationship between probabilistic modeling and function approximation. We begin by defining two extensions of function composition to stochastic process subordination: one based on the co-Kleisli category under the comonad (Omega x -) and one based on the parameterization of a category with a Lawvere theory. We show how these extensions relate to the category Stoch and other Markov Categories. Next, we apply the Para construction to extend stochastic processes to parameterized statistical models and we define a way to compose the likelihood functions of these models. We conclude with a demonstration of how the Maximum Likelihood Estimation procedure defines an identity-on-objects functor from the category of statistical models to the category of Learners. Code to accompany this paper can be found at https://github.com/dshieble/Categorical_Stochastic_Processes_and_Likelihood

Paper Structure

This paper contains 28 sections, 14 theorems, 158 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Probability
  4. Probability Measures, Random Variables and Markov Kernels
  5. Categories in Probability
  6. Parameterized Morphisms
  7. Random Variables and Independence
  8. Random Variables and Independence in $\mathbf{BorelStoch}$
  9. The co-Kleisli Construction
  10. Independence and Dependence in $\mathbf{C}\mathbf{EucMeas}$
  11. The Parameterization Construction
  12. An Subcategory of $\mathbf{Para}(\mathbf{C})$
  13. A Category of Parametric Measurable Maps
  14. Lawvere Parameterization
  15. Applying $\mathbf{Para}$ to $\mathbf{EucMeas}$
  16. ...and 13 more sections

Key Result

Proposition 1

We can form a strict symmetric monoidal subcategory $\mathbf{EucMeas}$ of $\mathbf{BorelMeas}$ in which objects are restricted to be $(\mathbb{R}^n, \mathcal{B}(\mathbb{R}^n))$ for some $n \in \mathbb{N}$, the tensor of the objects $(\mathbb{R}^n, \mathcal{B}(\mathbb{R}^n))$ and $(\mathbb{R}^m, \mat

Figures (3)

  • Figure :
  • Figure :
  • Figure :

Theorems & Definitions (62)

  • Definition 1.1
  • Definition 1.2
  • Definition 2.1
  • Definition 2.2
  • Definition 2.3
  • Definition 2.4
  • Definition 2.5
  • Definition 2.6
  • Definition 2.7
  • Definition 2.8
  • ...and 52 more