Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Schrödinger's control and estimation paradigm with spatio-temporal distributions on graphs

Asmaa Eldesoukey, Tryphon T. Georgiou

TL;DR

The paper extends Schrödinger's bridge to regulate stopping-time marginals for random walks on directed graphs with absorbing states, framing the problem as a large-deviation/maximum-likelihood task. It proves that, when the prior is Markov, the optimal posterior remains Markov for both classic SBP and SBP with stopping times, and it provides explicit Sinkhorn-type constructions to obtain the optimal path law that matches initial and stopping-time marginals. The authors further generalize to regularized transport on graphs via entropic penalties, and demonstrate the approach with concrete examples such as De Moivre's martingale and congestion-control networks, illustrating practical applicability to uncertainty control and space-probe planning.

Abstract

The problem of reconciling a prior probability law on paths with data was introduced by E. Schrödinger in 1931/32. It represents an early formulation of a maximum likelihood problem. This specific formulation can also be seen as the control problem to modify the law of a diffusion process so as to match specifications on marginal distributions at given times. Thereby, in recent years, this so-called Schrödinger's bridge problem has been at the center of the uncertainty control development. However, an understudied facet of this program has been to address uncertainty in space (state) and time, modeling the effect of tasks being completed contingent on meeting a certain condition at some random time instead of imposing specifications at fixed times. The present work is a study to extend Schrödinger's paradigm on such an issue, and herein, it is tackled in the context of random walks on directed graphs. Specifically, we study the case where one marginal is the initial probability distribution on a Markov chain, while others are marginals of stopping (first-arrival) times at absorbing states, signifying completion of tasks. We show when the prior law on paths is Markov, a Markov policy is once again optimal to satisfy those marginal constraints with respect to a likelihood cost following Schrödinger's dictum. Based on this, we present the mathematical formulation involving a Sinkhorn-type iteration to construct the optimal probability law on paths matching the spatio-temporal marginals.

Schrödinger's control and estimation paradigm with spatio-temporal distributions on graphs

TL;DR

The paper extends Schrödinger's bridge to regulate stopping-time marginals for random walks on directed graphs with absorbing states, framing the problem as a large-deviation/maximum-likelihood task. It proves that, when the prior is Markov, the optimal posterior remains Markov for both classic SBP and SBP with stopping times, and it provides explicit Sinkhorn-type constructions to obtain the optimal path law that matches initial and stopping-time marginals. The authors further generalize to regularized transport on graphs via entropic penalties, and demonstrate the approach with concrete examples such as De Moivre's martingale and congestion-control networks, illustrating practical applicability to uncertainty control and space-probe planning.

Abstract

The problem of reconciling a prior probability law on paths with data was introduced by E. Schrödinger in 1931/32. It represents an early formulation of a maximum likelihood problem. This specific formulation can also be seen as the control problem to modify the law of a diffusion process so as to match specifications on marginal distributions at given times. Thereby, in recent years, this so-called Schrödinger's bridge problem has been at the center of the uncertainty control development. However, an understudied facet of this program has been to address uncertainty in space (state) and time, modeling the effect of tasks being completed contingent on meeting a certain condition at some random time instead of imposing specifications at fixed times. The present work is a study to extend Schrödinger's paradigm on such an issue, and herein, it is tackled in the context of random walks on directed graphs. Specifically, we study the case where one marginal is the initial probability distribution on a Markov chain, while others are marginals of stopping (first-arrival) times at absorbing states, signifying completion of tasks. We show when the prior law on paths is Markov, a Markov policy is once again optimal to satisfy those marginal constraints with respect to a likelihood cost following Schrödinger's dictum. Based on this, we present the mathematical formulation involving a Sinkhorn-type iteration to construct the optimal probability law on paths matching the spatio-temporal marginals.
Paper Structure (16 sections, 8 theorems, 89 equations, 3 figures)

This paper contains 16 sections, 8 theorems, 89 equations, 3 figures.

Table of Contents

  1. Introduction
  2. De Moivre's Martingale
  3. Notation and Rudiments on Large Deviations
  4. On Markovian structure of posteriors
  5. The Classical SBP
  6. The SBP with Stopping Times
  7. Transition rates & spatio-temporal control
  8. Problem Formulation
  9. Building Space-Time Bridges I: Reconciling Marginals
  10. Building Space-Time Bridges II: Obtaining the Optimal Markov Kernel
  11. Optimality of the Law
  12. Regularized transport on Graphs
  13. Examples
  14. De Moivre's Martingale
  15. Congestion Control: Traffic Flow Regulation
  16. ...and 1 more sections

Key Result

Theorem III.1

(Sanov's Theorem in Finite Dimensions) Let $\mathbf X_1, \hdots, \mathbf X_N$ be independent and identically distributed (i.i.d.) random vectors taking values in $\mathcal{X}$ with $|\mathcal{X}|<\infty$ and law $Q(\mathbf x)$. For any set $\Gamma$ of probability distributions over $\mathcal{X}$ tha where $Q^N$ is the joint probability distribution on $\mathcal{X}^N$ and$Q^N(\mathbf X_1 =\mathbf x

Figures (3)

  • Figure 1: Abstract illustration of De Moivre's Martingale problem setting -- matching the marginal $\hat{\mu}_0$ (spatial) and the two marginals $\hat{\nu}^{\rm win}$ and $\hat{\nu}^{\rm ruin}$ (temporal). The requirement is to find the most likely law on paths.
  • Figure 2: Network topology with prior transition probabilities.
  • Figure 3: Network representation of vital neighborhoods in a city and associated transition probabilities under normal operation of the city roads.

Theorems & Definitions (18)

  • Theorem III.1
  • Theorem IV.1
  • proof
  • Theorem IV.2
  • proof
  • Remark IV.1
  • Remark V.1
  • Proposition V.1
  • proof
  • Proposition V.2
  • ...and 8 more