Coherent set identification via direct low rank maximum likelihood estimation

TL;DR

This paper analyzes two low-rank approaches to dynamical data: the classical coherence problem, which seeks partitions of states that remain highly distinguishable under a stochastic transition, and Direct Bayesian Model Reduction (DBMR), which directly estimates a low-rank factorization of the transition. It demonstrates that DBMR outputs can be expressed as the full transition composed with a projection, $\Lambda = P\Pi$, and derives bounds ensuring the reduced model’s coherence controls that of the full model. A key theoretical contribution is a bound connecting the Frobenius-norm error between the normalized full and reduced transitions to the relaxed likelihood gap $\hat{\ell}$, supported by a Pinsker-type inequality, thereby relating projection-based and likelihood-based estimations and linking Frobenius and KL objectives. Numerically, DBMR provides interpretable, structure-preserving reduced models and, despite potential local maxima, shows broad alignment with the classical coherence approach, suggesting complementary utility and motivating further exploration of symmetrized DBMR and alternative objective formulations.

Abstract

We analyze connections between two low rank modeling approaches from the last decade for treating dynamical data. The first one is the coherence problem (or coherent set approach), where groups of states are sought that evolve under the action of a stochastic transition matrix in a way maximally distinguishable from other groups. The second one is a low rank factorization approach for stochastic matrices, called Direct Bayesian Model Reduction (DBMR), which estimates the low rank factors directly from observed data. We show that DBMR results in a low rank model that is a projection of the full model, and exploit this insight to infer bounds on a quantitative measure of coherence within the reduced model. Both approaches can be formulated as optimization problems, and we also prove a bound between their respective objectives. On a broader scope, this work relates the two classical loss functions of nonnegative matrix factorization, namely the Frobenius norm and the generalized Kullback--Leibler divergence, and suggests new links between likelihood-based and projection-based estimation of probabilistic models.

Figures (8)

• Figure 1: Coherent set identification for Example \ref{['example:1_three_coherent_sets']} with $r=3$ clusters and $3$ different levels of perturbation: Top: Full transition matrix $P$. Middle: Reduced transition matrix $P_{\textup{red}}$ obtained within the classical Algorithm \ref{['alg:classical_approach_coherence']}. Bottom: Reduced transition matrix $\Lambda = \lambda \Gamma$ of the DBMR Algorithm \ref{['alg:the_alg']}. The coloring at the bottom line of each plot corresponds to the clustering given by the associated affiliation matrix $\Gamma$ (or partition $\mathcal{E}$): default-$\Gamma$ (top), SVD-$\Gamma$ (middle), and DBMR-$\Gamma$ (bottom). The coloring on the left margin of the plots represents the corresponding partitions $\mathcal{F}$. For the middle row, $\mathcal{F}$ is obtained by clustering and matching (lines 5--6 of Algorithm \ref{['alg:classical_approach_coherence']}), and for the bottom row by \ref{['eq:Sets_F_k']}.
• Figure 2: Coherent set identification for Example \ref{['example:piecewise_expanding_interval_map']} with $r=3$ clusters and $3$ different levels of perturbation: Top: Full transition matrix $P$. Middle: Reduced transition matrix $P_{\textup{red}}$ obtained within the classical Algorithm \ref{['alg:classical_approach_coherence']}. Bottom: Reduced transition matrix $\Lambda = \lambda \Gamma$ of the DBMR Algorithm \ref{['alg:the_alg']}. The coloring at the bottom line of each plot corresponds to the clustering given by the associated affiliation matrix $\Gamma$ (or partition $\mathcal{E}$): default-$\Gamma$ (top), SVD-$\Gamma$ (middle) and DBMR-$\Gamma$ (bottom). The coloring on the left margin of the plots represents the corresponding partitions $\mathcal{F}$. For the middle row, $\mathcal{F}$ is obtained by clustering and matching (lines 5--6 of Algorithm \ref{['alg:classical_approach_coherence']}), and for the bottom row by \ref{['eq:Sets_F_k']}.
• Figure 3: Contour plots of the stream function $\psi$ from \ref{['eq:stream_fct_double_gyre']} at time $t = 1/4$ (left) and time $t = 3/4$ (right). At any time, the velocity field is tangential to level sets of the stream function.
• Figure 4: Coherent sets for the double gyre flow from Example \ref{['example:periodically_perturbed_double_gyre']}. Top two rows: The classical approach to coherence, clustering of singular vectors into $r=3$ (first row) and $r=5$ (second row) clusters to yield coherent sets at initial time (left) and at final time (right). Bottom two rows: DBMR with $r=3$ (third row) and $r=5$ (fourth row), coloring by the latent states from $\Gamma$ (left) and the sets $F_k$ given by \ref{['eq:Sets_F_k']} (right).
• Figure 5: Top: First $5$ singular values of ${\widetilde{P}}$ and ${\widetilde{\Lambda}} = {\widetilde{P}}\widetilde{\Pi}$ for the transition matrix of Example \ref{['example:1_three_coherent_sets']}. Bottom: Likelihood bound $\hat{\ell}(\lambda,\Gamma)$ in dependence of the degree of coherence $\mathcal{C}_{3}(\Lambda)$. Results are shown for $100$ runs of DBMR with $r=3$ latent states. Left: unperturbed; center: slightly perturbed right: strongly perturbed.

Theorems & Definitions (38)

• Definition 1: Affiliation matrix
• Definition 2: Degree of coherence
• Remark 3
• Remark 4
• Lemma 5
• proof
• Remark 6: Connection to Kullback--Leibler divergences
• Remark 7
• Example 8
• Remark 9