Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Cover Time of a (Multiple) Markov Chain with Rational Transition Probabilities is Rational

John Sylvester

TL;DR

The paper proves that the expected cover time of a finite-state Markov chain with rational transition probabilities is rational, provided the cover time is finite from the given start state. It achieves this by converting the cover-time problem into a hitting-time problem on a higher-dimensional auxiliary chain Q(P,k) and then applying a rationality result for hitting times. The methods combine a linear-algebraic rationality argument for hitting times with a coupling-based encoding of multi-walker cover times as hitting times to a specific set. The findings extend to k independent copies and do not require irreducibility, offering a precise algebraic characterization of finite-state cover times and a framework for analyzing them via rational linear systems and auxiliary chains.

Abstract

The cover time of a Markov chain on a finite state space is the expected time until all states are visited. We show that if the cover time of a discrete-time Markov chain with rational transitions probabilities is bounded, then it is a rational number. The result is proved by relating the cover time of the original chain to the hitting time of a set in another higher dimensional chain. We also extend this result to the setting where $k\geq 1 $ independent copies of a Markov chain are run simultaneously on the same state space and the cover time is the expected time until each state has been visited by at least one copy of the chain.

The Cover Time of a (Multiple) Markov Chain with Rational Transition Probabilities is Rational

TL;DR

The paper proves that the expected cover time of a finite-state Markov chain with rational transition probabilities is rational, provided the cover time is finite from the given start state. It achieves this by converting the cover-time problem into a hitting-time problem on a higher-dimensional auxiliary chain Q(P,k) and then applying a rationality result for hitting times. The methods combine a linear-algebraic rationality argument for hitting times with a coupling-based encoding of multi-walker cover times as hitting times to a specific set. The findings extend to k independent copies and do not require irreducibility, offering a precise algebraic characterization of finite-state cover times and a framework for analyzing them via rational linear systems and auxiliary chains.

Abstract

The cover time of a Markov chain on a finite state space is the expected time until all states are visited. We show that if the cover time of a discrete-time Markov chain with rational transitions probabilities is bounded, then it is a rational number. The result is proved by relating the cover time of the original chain to the hitting time of a set in another higher dimensional chain. We also extend this result to the setting where independent copies of a Markov chain are run simultaneously on the same state space and the cover time is the expected time until each state has been visited by at least one copy of the chain.

Paper Structure

This paper contains 4 sections, 6 theorems, 9 equations, 2 figures.

Table of Contents

  1. Introduction and Results
  2. Proofs
  3. Rationality of Hitting Times
  4. Encoding Cover Times as Hitting Times

Key Result

Theorem 1

Let $(X_t)_{t\geq 0}$ be a discrete-time rational Markov chain on a finite state space $\Omega$. Then, for any $x \in \Omega$ such that $\mathbb{E}_{x}\!\left[\tau_{\mathsf{cov}}\right]<\infty$, we have $\mathbb{E}_{x}\!\left[\tau_{\mathsf{cov}}\right]\in \mathbb{Q}$.

Figures (2)

  • Figure 1: Example of a non-irreducible Markov chain on seven states where the cover time from $x$ is finite and from any other vertex the cover time is unbounded/undefined.
  • Figure 2: This figure shows an example of a Markov chain $\mathbf{P}$ on three states (bottom right) and its associated auxiliary chain $\mathbf{Q}(\mathbf{P},1)$, where the set $C$ from Lemma \ref{['multicovashit']} is shown in the red shaded ellipse.

Theorems & Definitions (10)

  • Theorem 1
  • Corollary 2
  • Theorem 3
  • Proposition 4
  • Lemma 5
  • proof
  • proof : Proof of Proposition \ref{['hitrat+']}
  • Lemma 6
  • proof : Proof of Lemma \ref{['multicovashit']}
  • proof : Proof of Theorem \ref{['multicovrat']}