Towards Markov-State Holography

TL;DR

This work introduces a model-free, histogram-based method to detect memory locally in observed transitions of lumped Markov processes. By conditioning transition probabilities on history sequences, it reveals hidden microscopic paths and defines a weak Markov order that quantifies memory duration via the convergence of history-conditioned histograms. The approach provides a practical test for the local Markov property and offers insights into hidden transitions not captured by standard Markov-state models, with validation on a toy protein-like example. It has potential applications to single-molecule trajectories and other partially observed systems, and it connects memory detection to broader memory-kernel and thermodynamic inference concepts.

Abstract

Experiments, in particular on biological systems, typically probe lower-dimensional observables which are projections of high-dimensional dynamics. In order to infer consistent models capturing the relevant dynamics of the system, it is important to detect and account for the memory in the dynamics. We develop a method to infer the presence of hidden states and transition pathways based on observable transition probabilities conditioned on history sequences of visited states for projected (i.e. observed) dynamics of Markov processes. Histograms conditioned on histories reveal information on the transition probabilities of hidden paths locally between any specific pair of observed states. The convergence rate of these histograms towards a stationary distribution provides a local quantification of the duration of memory, which reflects how distinct microscopic paths projecting onto the same observed transition decorrelate in path space. This motivates the notion of "weak Markov order" and provides insight about the hidden topology of microscopic paths in a holography-like fashion. The method can be used to test for the local Markov property of observables. The information extracted is also helpful in inferring relevant hidden transitions which are not captured by a Markov-state model.

Figures (7)

• Figure 1: A schematic illustrating the definition of a microscopic full sequence $\{\sigma_{\tau}\}_{1\leqslant \tau\leqslant k'}$ (above), a reduced sequence $\{\alpha_{t}\}_{1\leqslant t\leqslant k+2}$ (middle), and the observable sequence $\{s_{t}\}_{1\leqslant t\leqslant k+2}$ (below).
• Figure 2: (a) Schematic of the full Markov network representing a toy protein model: the $s$-axis is the observable and the $z$-axis the hidden dimension. Microscopic states $2,3,4$ and $5,6,7$ are projected onto observable states $b$ and $c$, respectively. State $a$ and $d$ in pink are observable Markov states, as they correspond to a single microscopic state $1$ and $8$, respectively. (b) Example of a microscopic (blue) and the corresponding observed (black) trajectory. The latter is a projection of the microscopic trajectory onto the $s$-$t$ plane; the corresponding observed sequence of states is shown below. (c) Histogram of transition probability from state $c$ to $b$ conditioned on one step in the history $\mathbbm{P}(b|cS^c_1)$, where $S^c_1\in\{b,d\}$. The two bars are due to $\mathbbm{P}(b|cd)$ and $\mathbbm{P}(b|cb)$. (d) Histogram of transition probability conditioned on two steps in the history $\mathbbm{P}(b|cS^c_2)$, where $S^c_2\in\{dc,bc,bd\}$. Note that the sequences in the conditional probability are visited from right to left ordered in time, e.g. $\mathbbm{P}(b|cd)$ corresponds to the sequence $d\rightarrow c\rightarrow b$.
• Figure 3: (a)-(c) Histograms of transition probabilities between different pairs of states conditioned on history of different lengths $k$. Each row represents a histogram for a fixed history length $k$. The two rows in gray rectangle in panel (c) correspond to the histograms in Figure \ref{['fig1']}(c-d). The width of bars in all histogram is $2\delta=0.05$. The color bar is the same for all plots. (d) The points show the TVD determined from simulation results, and the solid line is the theoretical bound in Eq. \ref{['bound']} which is independent of the pair of states.
• Figure C1: A part of a network for the dynamics of the reduced microscopic sequence. The values on arrows represent elements in the transition matrix $\mathbf{\Omega}$ for the reduced sequence. The two microscopic states in the observable state $j$ both satisfy $\mathbbm{P}(i|\alpha_{k+1})=p$.
• Figure F1: (a)-(c) Histograms of transition probabilities between different pairs of states conditioned on history of different lengths $k$ for driving parameter $\varphi=0.05$. Each row represents a histogram for a fixed history length $k$. The bar width in each histogram is $2\delta=0.05$. The color bar is the same for all plots. (d) The points show the TVD determined from simulation results, and the solid line is the theoretical bound which is independent of the pair of states. (e)-(g) Histograms of transition probabilities for driving parameter $\varphi=-0.05$. (h) The TVD determined from simulation results and the theoretical bound.