Nearest Reversible Markov Chains with Sparsity Constraints: An Optimization Approach

TL;DR

This framework provides a principled way to enforce reversibility and sparsity patterns in Markov chains with applications in MCMC, computational chemistry, and data-driven modeling.

Abstract

Reversibility is a key property of Markov chains, central to algorithms such as Metropolis-Hastings and other MCMC methods. Yet many applications yield non-reversible chains, motivating the problem of approximating them by reversible ones with minimal modification. We formulate this task as a matrix nearness problem and focus on the practically relevant case of sparse transition matrices. The resulting optimization problem is a quadratic programming problem, and numerical experiments illustrate the effectiveness of the approach. This framework provides a principled way to enforce reversibility and sparsity patterns in Markov chains with applications in MCMC, computational chemistry, and data-driven modeling.

Figures (7)

• Figure 1.1: Boxplots summarizing the distribution of solution times and condition numbers across all test cases, grouped by solver. Each boxplot displays the median and interquartile range, with individual test cases shown as jittered dots.
• Figure 1.2: Comparison with the Riemannian solver. Left: sparsity pattern of the transition matrix of the initial Markov chain. Center: sparsity pattern of the nearest sparse reversible transition matrix obtained with Alg. \ref{['alg:summming_it_up']}. Right: the Riemannian solver produces a dense matrix (not shown as a sparsity pattern). The top bar plot reports the perturbation norm, while the bottom bar plot shows the computational time required, respectively, by the approach here proposed (Sparse Nearest) and by the Riemannian solver.
• Figure 1.3: Reversibilization of the transition matrix $P$ obtained from the count matrix of the butane configuration. The four panels, from left to right, show: (i) the adjacency matrix of the original Markov chain $P$; (ii) the admissible modification pattern $\mathbb{S}(P + P^\top + I)$; (iii) the sparsity pattern of the perturbation $\Delta$ obtained from the Nearest Sparse Reversible problem; (iv) the sign of the modification.
• Figure 1.4: Illustration of the MSM construction for the Fs-peptide system. Panel \ref{['fig:molecular_dynamic_examples_a']} reports the projection of the trajectories onto the first two tICA components, and is followed by panel \ref{['fig:molecular_dynamic_examples_b']} which gives discretization into clusters/microstates obtained by using MiniBatch $k$-Means 10.1145/1772690.1772862---cluster centers are represtend as large white dots. These clusters serve as the discrete states on which the Markov State Model is estimated.
• Figure 1.5: Reversibilization of the transition matrix $P$ from \ref{['eq:t_peptide']}. The four panels, from left to right, show: (i) the sparsity pattern of the original matrix $P$, which is not symmetric; (ii) the admissible modification pattern $\mathbb{S}(P + P^\top + I)$; (iii) the sparsity pattern of the perturbation $\Delta$; (iv) sign of the obtained $\Delta$ matrix.