Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Formal Error Bounds for the State Space Reduction of Markov Chains

Fabian Michel, Markus Siegle

TL;DR

This work develops formal error bounds for reducing Markov chains to smaller state spaces, covering both discrete- and continuous-time settings and a highly general aggregation framework that allows arbitrary disaggregation and aggregated dynamics. It proves tight transient-distribution bounds, connects them to dynamic-exact and exact lumpability concepts, and analyzes stationary-distribution bounds, showing when aggregation preserves or approximates stationarity. The authors introduce and compare practical aggregation algorithms (notably SVD-based methods and almost-exact lumpability) and demonstrate substantial speedups across diverse models while quantifying the trade-offs between accuracy and reduced dimension. The results offer a unified theoretical and algorithmic toolkit for reliable state-space reduction in stochastic models, with clear guidance on selecting aggregations and understanding their impact on transient and stationary behavior.

Abstract

We study the approximation of a Markov chain on a reduced state space, for both discrete- and continuous-time Markov chains. In this context, we extend the existing theory of formal error bounds for the approximated transient distributions. As a special case, we consider aggregated (or lumped) Markov chains, where the state space reduction is achieved by partitioning the state space into macro states. In the discrete-time setting, we bound the stepwise increment of the error, and in the continuous-time setting, we bound the rate at which the error grows. In addition, the same error bounds can also be applied to bound how far an approximated stationary distribution is from stationarity. Subsequently, we compare these error bounds with relevant concepts from the literature, such as exact and ordinary lumpability, as well as deflatability and aggregatability. These concepts define stricter than necessary conditions to identify settings in which the aggregation error is zero. We also consider possible algorithms for finding suitable aggregations for which the formal error bounds are low, and we analyse first experiments with these algorithms on a range of different models.

Formal Error Bounds for the State Space Reduction of Markov Chains

TL;DR

This work develops formal error bounds for reducing Markov chains to smaller state spaces, covering both discrete- and continuous-time settings and a highly general aggregation framework that allows arbitrary disaggregation and aggregated dynamics. It proves tight transient-distribution bounds, connects them to dynamic-exact and exact lumpability concepts, and analyzes stationary-distribution bounds, showing when aggregation preserves or approximates stationarity. The authors introduce and compare practical aggregation algorithms (notably SVD-based methods and almost-exact lumpability) and demonstrate substantial speedups across diverse models while quantifying the trade-offs between accuracy and reduced dimension. The results offer a unified theoretical and algorithmic toolkit for reliable state-space reduction in stochastic models, with clear guidance on selecting aggregations and understanding their impact on transient and stationary behavior.

Abstract

We study the approximation of a Markov chain on a reduced state space, for both discrete- and continuous-time Markov chains. In this context, we extend the existing theory of formal error bounds for the approximated transient distributions. As a special case, we consider aggregated (or lumped) Markov chains, where the state space reduction is achieved by partitioning the state space into macro states. In the discrete-time setting, we bound the stepwise increment of the error, and in the continuous-time setting, we bound the rate at which the error grows. In addition, the same error bounds can also be applied to bound how far an approximated stationary distribution is from stationarity. Subsequently, we compare these error bounds with relevant concepts from the literature, such as exact and ordinary lumpability, as well as deflatability and aggregatability. These concepts define stricter than necessary conditions to identify settings in which the aggregation error is zero. We also consider possible algorithms for finding suitable aggregations for which the formal error bounds are low, and we analyse first experiments with these algorithms on a range of different models.
Paper Structure (26 sections, 21 theorems, 108 equations, 7 figures, 1 table, 3 algorithms)

This paper contains 26 sections, 21 theorems, 108 equations, 7 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Linear Algebra
  4. Markov chains and state space reduction
  5. Bounding the approximation error
  6. Error bounds for DTMCs
  7. Error bounds for CTMCs
  8. When is the error bound zero?
  9. Error bound for the approximated stationary distribution
  10. Tightness of the error bound
  11. Reduction to arbitrarily low dimension
  12. Summary of results
  13. Lumpability and aggregatability
  14. An alternative characterization of dynamic-exactness
  15. Dynamic-exactness compared to other lumpability concepts
  16. ...and 11 more sections

Key Result

Lemma 1

For a vector $v \in \mathbb{R}^m$ and a matrix $A \in \mathbb{R}^{m \times n}$, we have:

Figures (7)

  • Figure 1: A Venn diagram summarizing the relation between the different types of lumpability
  • Figure 2: SVDsgn, SVDdir, SVDseba and \ref{['alg:ealmostexlump']} executed on 100 randomly generated almost aggregatable DTMCs with 200 states, 20 aggregates and a probability of 0.5 to have no transition between a particular pair of aggregates. The almost aggregatable DTMCs were obtained by random perturbation (with a magnitude of $0.002$) of the transition matrix of a randomly generated aggregatable DTMC. Each plotted point is an average resulting from running the algorithms with a particular fixed input parameter $\varepsilon$ on the 100 DTMCs.
  • Figure 3: SVDdir and \ref{['alg:ealmostexlump']} executed on a prokaryotic gene expression model already used in the experiments in adaptformalagg, originally from prokgeneexpr. The maximum population size was set to 5, resulting in 43 957 states. The CTMC was uniformised using the maximal exit rate $16.78$ as uniformisation rate.
  • Figure 4: SVDdir and \ref{['alg:ealmostexlump']} executed on the Lotka-Volterra model already used in the experiments in adaptformalagg, described in more detail e.g. in stochsimcouplchem. The maximum number of species was set to 100, resulting in 10 201 states. The CTMC was uniformised using the maximal exit rate $2078$ as uniformisation rate.
  • Figure 5: SVDdir and \ref{['alg:ealmostexlump']} executed on a workstation cluster model already used in the experiments in adaptformalagg, originally from modcheckdependability. The number of workstations per cluster was set to 20, resulting in 15 540 states. The CTMC was uniformised using the maximal exit rate $50.08$ as uniformisation rate.
  • ...and 2 more figures

Theorems & Definitions (54)

  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Definition 3
  • Theorem 4
  • proof
  • Theorem 5
  • Lemma 6
  • proof
  • ...and 44 more