Joint Manifold Learning and Optimal Transport for Dynamic Imaging

TL;DR

The paper addresses limited time-series data in dynamic imaging by integrating a learnable image-manifold latent space with a dynamical OT regularizer. An autoencoder maps images to latent codes, which are evolved via a neural ODE and decoded back, while a Riemannian OT term enforces latent trajectories to follow OT geodesics through barycentric interpolations $B(\mu_{t_j},\mu_{t_{j+1}},\frac{t-t_j}{t_{j+1}-t_j})$ after normalization $n(I)$. Experiments on synthetic Gaussian sequences and HeLa cell data show that the OT prior preserves structure and yields smoother, more faithful interpolations under temporal undersampling, while the latent-manifold model provides robust reconstruction. The work highlights a synergistic framework where latent manifold learning and dynamical OT regularization reinforce each other, with future extensions to unbalanced OT and forward-model incorporation.

Abstract

Dynamic imaging is critical for understanding and visualizing dynamic biological processes in medicine and cell biology. These applications often encounter the challenge of a limited amount of time series data and time points, which hinders learning meaningful patterns. Regularization methods provide valuable prior knowledge to address this challenge, enabling the extraction of relevant information despite the scarcity of time-series data and time points. In particular, low-dimensionality assumptions on the image manifold address sample scarcity, while time progression models, such as optimal transport (OT), provide priors on image development to mitigate the lack of time points. Existing approaches using low-dimensionality assumptions disregard a temporal prior but leverage information from multiple time series. OT-prior methods, however, incorporate the temporal prior but regularize only individual time series, ignoring information from other time series of the same image modality. In this work, we investigate the effect of integrating a low-dimensionality assumption of the underlying image manifold with an OT regularizer for time-evolving images. In particular, we propose a latent model representation of the underlying image manifold and promote consistency between this representation, the time series data, and the OT prior on the time-evolving images. We discuss the advantages of enriching OT interpolations with latent models and integrating OT priors into latent models.

Figures (4)

• Figure 1: Model overview. An initial image is encoded into latent space, where a trajectory is predicted and decoded back into image space. To address the scarcity of time series data and time points, existing methods often rely on priors such as a low-dimensional manifold assumption or a dynamic OT prior. Our model combines these approaches by constructing a low-dimensional latent space and regularizing the predicted image trajectories to align with the OT prior.
• Figure 2: Comparison of manifold-based interpolations of Gaussians. The ground truth images and the estimated images by the neural ODE model when trained with different regularizers: a penalty on the time derivative of latent vectors, a penalty on the time derivative of decoded images, or our OT regularization. During training, the models only see images subsampled at an interval of 5 time points. OT regularization is the only regularizer maintaining consistent Gaussian shapes across time.
• Figure 3: Comparison of manifold-based interpolations of HeLa cells. The ground truth images and the estimated images by the neural ODE model when trained with different regularizers: a penalty on the time derivative of latent vectors, a penalty on the time derivative of decoded images, or our OT regularization. During training, the models only see images subsampled at an interval of 5 time points. Only OT regularization obtains a clear and smooth cell division.
• Figure 4: Barycentric vs. Manifold interpolation. The ground truth images, the estimated images by $l_2$ or Wasserstein $\mathcal{W}_2$ interpolation, and the estimated images by the neural ODE manifold model. $l_2$ interpolation does not resemble a (divided) cell, $\mathcal{W}_2$ splits the cells prematurely, and manifold interpolation avoids both issues.

Theorems & Definitions (3)

• definition thmcounterdefinition: Wasserstein space $\mathcal{W}_2(\mathbb{R}^d)$ villani2009optimal
• theorem thmcountertheorem: Ambrosio2013ambrosio2005gradient
• definition thmcounterdefinition: Wasserstein barycenters and interpolations