Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Skolem and Positivity Completeness of Ergodic Markov Chains

Mihir Vahanwala

TL;DR

This paper tackles the decidability of Markov Reachability problems viewed as Linear Dynamical Systems, linking them to the Skolem, Positivity, and Ultimate Positivity problems for Linear Recurrence Sequences (LRS). It introduces an elementary reduction that maps LRS of order $k$ to ergodic Markov Chains of order $k+1$ via a decomposition $M = S + D$ with $S = \mathbf{s}\mathbf{1}^T$ and a disturbance $D$ designed so $DS = SD = 0$ and $d_{ij}^{(n)} = \eta u_n / \rho^n$. This yields a substantial improvement over prior reductions that produced reducible, periodic chains of order $4k+5$, by achieving ergodic chains of order $k+1$. The result indicates that the inherent hardness of Verifying Linear Dynamical Systems persists under spectral constraints and provides a pathway toward more robust verification for ergodic Markov processes.

Abstract

We consider the following Markov Reachability decision problems that view Markov Chains as Linear Dynamical Systems: given a finite, rational Markov Chain, source and target states, and a rational threshold, does the probability of reaching the target from the source at the $n^{th}$ step: (i) equal the threshold for some $n$? (ii) cross the threshold for some $n$? (iii) cross the threshold for infinitely many $n$? These problems are respectively known to be equivalent to the Skolem, Positivity, and Ultimate Positivity problems for Linear Recurrence Sequences (LRS), number-theoretic problems whose decidability has been open for decades. We present an elementary reduction from LRS Problems to Markov Reachability Problems that improves the state of the art as follows. (a) We map LRS to ergodic (irreducible and aperiodic) Markov Chains that are ubiquitous, not least by virtue of their spectral structure, and (b) our reduction maps LRS of order $k$ to Markov Chains of order $k+1$: a substantial improvement over the previous reduction that mapped LRS of order $k$ to reducible and periodic Markov chains of order $4k+5$. This contribution is significant in view of the fact that the number-theoretic hardness of verifying Linear Dynamical Systems can often be mitigated by spectral assumptions and restrictions on order.

Skolem and Positivity Completeness of Ergodic Markov Chains

TL;DR

This paper tackles the decidability of Markov Reachability problems viewed as Linear Dynamical Systems, linking them to the Skolem, Positivity, and Ultimate Positivity problems for Linear Recurrence Sequences (LRS). It introduces an elementary reduction that maps LRS of order to ergodic Markov Chains of order via a decomposition with and a disturbance designed so and . This yields a substantial improvement over prior reductions that produced reducible, periodic chains of order , by achieving ergodic chains of order . The result indicates that the inherent hardness of Verifying Linear Dynamical Systems persists under spectral constraints and provides a pathway toward more robust verification for ergodic Markov processes.

Abstract

We consider the following Markov Reachability decision problems that view Markov Chains as Linear Dynamical Systems: given a finite, rational Markov Chain, source and target states, and a rational threshold, does the probability of reaching the target from the source at the step: (i) equal the threshold for some ? (ii) cross the threshold for some ? (iii) cross the threshold for infinitely many ? These problems are respectively known to be equivalent to the Skolem, Positivity, and Ultimate Positivity problems for Linear Recurrence Sequences (LRS), number-theoretic problems whose decidability has been open for decades. We present an elementary reduction from LRS Problems to Markov Reachability Problems that improves the state of the art as follows. (a) We map LRS to ergodic (irreducible and aperiodic) Markov Chains that are ubiquitous, not least by virtue of their spectral structure, and (b) our reduction maps LRS of order to Markov Chains of order : a substantial improvement over the previous reduction that mapped LRS of order to reducible and periodic Markov chains of order . This contribution is significant in view of the fact that the number-theoretic hardness of verifying Linear Dynamical Systems can often be mitigated by spectral assumptions and restrictions on order.
Paper Structure (6 sections, 2 theorems, 8 equations)

This paper contains 6 sections, 2 theorems, 8 equations.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Linear Recurrence Sequences
  4. Stochastic Matrices and Markov Chains
  5. The Reduction
  6. Discussion

Key Result

Theorem 1

The Skolem, Positivity, and Ultimate Positivity Problems for LRS of order $k$ reduce to Markov Reachability Problems for ergodic Markov Chains of order $k+1$.

Theorems & Definitions (3)

  • Definition 1: Markov Reachability Problems
  • Theorem 1: Main Result
  • Theorem 2: Standard