Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Wasserstein error bounds for aggregations of continuous-time Markov chains

Fabian Michel

TL;DR

This work develops Wasserstein-distance error bounds for aggregating finite-state CTMCs onto a reduced state space, introducing a Wasserstein matrix norm to bound the error due to dynamics reduction and using coarse Ricci curvature to control error propagation over time. The central result provides a differential bound on the Wasserstein distance between the true and aggregated transient distributions, linking it to the aggregation mismatch $\|\Theta A - A Q\|_W$ and to a curvature term. The bounds’ practical usefulness depends on curvature: nonnegative/positive curvature (as in translation-invariant CTMCs or discrete metric cases) yields non-explosive or improved bounds, while negative curvature can cause exponential growth in the bound, highlighting limitations in some models. The paper also demonstrates the approach with toy and RSVP examples and sketches extensions toward continuous-state Markov processes, outlining future directions and challenges for broader applicability.

Abstract

We study the approximation of a (finite) continuous-time Markov chain by a Markov chain on a reduced state space, and we provide formal error bounds for the approximated transient distributions in the Wasserstein distance. These bounds extend previous work on error bounds in the total variation distance, and are the first step towards a generalization to continuous-time Markov processes with continuous state spaces. A Wasserstein matrix norm is used to bound the error caused by the lower-dimensional approximation of the dynamics. In order to control the propagation of the accumulated error, we rely on the concept of coarse Ricci curvature of a Markov chain. The practical applicability of the presented bounds depends strongly on the curvature of the chain. Examples for CTMCs taken from the literature (where we added a metric on the state space) show that a negative curvature results in exponentially exploding bounds. On the other hand, certain CTMCs which we call translation-invariant always have non-negative curvature. When measuring the error in the total variation distance (a special case of the Wasserstein distance with the discrete metric), the curvature is also always non-negative. If it is strictly positive, the bounds presented in this paper are an improvement over previous work.

Wasserstein error bounds for aggregations of continuous-time Markov chains

TL;DR

This work develops Wasserstein-distance error bounds for aggregating finite-state CTMCs onto a reduced state space, introducing a Wasserstein matrix norm to bound the error due to dynamics reduction and using coarse Ricci curvature to control error propagation over time. The central result provides a differential bound on the Wasserstein distance between the true and aggregated transient distributions, linking it to the aggregation mismatch and to a curvature term. The bounds’ practical usefulness depends on curvature: nonnegative/positive curvature (as in translation-invariant CTMCs or discrete metric cases) yields non-explosive or improved bounds, while negative curvature can cause exponential growth in the bound, highlighting limitations in some models. The paper also demonstrates the approach with toy and RSVP examples and sketches extensions toward continuous-state Markov processes, outlining future directions and challenges for broader applicability.

Abstract

We study the approximation of a (finite) continuous-time Markov chain by a Markov chain on a reduced state space, and we provide formal error bounds for the approximated transient distributions in the Wasserstein distance. These bounds extend previous work on error bounds in the total variation distance, and are the first step towards a generalization to continuous-time Markov processes with continuous state spaces. A Wasserstein matrix norm is used to bound the error caused by the lower-dimensional approximation of the dynamics. In order to control the propagation of the accumulated error, we rely on the concept of coarse Ricci curvature of a Markov chain. The practical applicability of the presented bounds depends strongly on the curvature of the chain. Examples for CTMCs taken from the literature (where we added a metric on the state space) show that a negative curvature results in exponentially exploding bounds. On the other hand, certain CTMCs which we call translation-invariant always have non-negative curvature. When measuring the error in the total variation distance (a special case of the Wasserstein distance with the discrete metric), the curvature is also always non-negative. If it is strictly positive, the bounds presented in this paper are an improvement over previous work.

Paper Structure

This paper contains 30 sections, 13 theorems, 100 equations, 15 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Our contribution
  3. Paper structure
  4. Preliminaries
  5. Markov chains and state space reduction
  6. Wasserstein distance
  7. Wasserstein norm for matrices
  8. Linear programs and the Wasserstein distance
  9. Wasserstein distance in an example
  10. Ricci curvature
  11. Wasserstein error bounds
  12. The CTMC case
  13. Bounding the growth of the accumulated errror
  14. Bounding the approximation error
  15. Overall error bound
  16. ...and 15 more sections

Key Result

proposition 4

Let $p, q \in \mathbb{R}^n$ be probability measures on the state space $S = \{1, \ldots, n\}$ with metric $\mathop{\mathrm{dist}}\nolimits$. Then, we have and, equivalently (by linear programming duality), Furthermore, there is a pair of optimal solutions $f^\ast, \gamma^\ast$ of eq:wasserstein_finite_primallp and eq:wasserstein_finite_duallp which satisfies all of the following:

Figures (15)

  • Figure 1: A line metric for the state space $S = \{1, \ldots, 6\}$
  • Figure 2: A toy CTMC used for illustrating some of the concepts
  • Figure 3: Evolution of the Wasserstein distance between the two transient distributions obtained when starting in states $1$ and $2$ of the toy CTMC
  • Figure 4: Evolution of the Wasserstein distance between two transient distributions of the toy CTMC: on the one hand the transient distribution obtained when starting in state $1$; on the other hand the initial distribution $\alpha \delta_2 + (1 - \alpha) \delta_3$ for $\alpha \in [0, 1]$ is used. The orange lines show the bound on the initial derivative of the Wasserstein distance given by \ref{['lem:wdderiv_error_ricciinfbound']}.
  • Figure 5: Error evolution for the toy CTMC using the given aggregation and the initial distribution $p_0 = (\frac{1}{2}, \; \frac{1}{2}, \; 0)^{ \mathsf{T}}$. The red and orange lines show the error bounds obtained from \ref{['thm:wasserstein_error_growth_bound']}.
  • ...and 10 more figures

Theorems & Definitions (36)

  • definition 1
  • remark
  • remark
  • definition 2
  • remark
  • definition 3
  • proposition 4
  • Proof
  • definition 5
  • definition 6
  • ...and 26 more