A convex approach for Markov chain estimation from aggregate data via inverse optimal transport
Michele Mascherpa, Axel Ringh, Amirhossein Taghvaei, Johan Karlsson
TL;DR
The paper tackles learning a discrete-time Markov law from aggregate, indistinguishable-particle data by casting the problem as a convex, likelihood-based inverse transport problem. It jointly optimizes over the transport plans $M_t$ and the prior transition matrix $A$ under marginal constraints, using an entropy-regularized objective ${\mathcal D}(M_t\,|\,A)$ that links to entropic OT and the Schrödinger bridge. An entropic proximal algorithm with Sinkhorn-style updates is developed, with a duality-based analysis showing existence and conditions for uniqueness. Numerical experiments on independent and sequential observations demonstrate accurate recovery of $A$ when states are sufficiently excited, while highlighting identifiability limitations tied to mixing time and data diversity. The approach offers a convex, scalable alternative to nonconvex inverse OT formulations and suggests extensions to partial identification and hidden Markov models.
Abstract
We address the problem of identifying the dynamical law governing the evolution of a population of indistinguishable particles, when only aggregate distributions at successive times are observed. Assuming a Markovian evolution on a discrete state space, the task reduces to estimating the underlying transition probability matrix from distributional data. We formulate this inverse problem within the framework of entropic optimal transport, as a joint optimization over the transition matrix and the transport plans connecting successive distributions. This formulation results in a convex optimization problem, and we propose an efficient iterative algorithm based on the entropic proximal method. We illustrate the accuracy and convergence of the method in two numerical setups, considering estimation from independent snapshots and estimation from a time series of aggregate observations, respectively.