Shape Modeling of Longitudinal Medical Images: From Diffeomorphic Metric Mapping to Deep Learning

TL;DR

This paper surveys spatiotemporal shape modeling in longitudinal medical imaging, contrasting Large Deformation Diffeomorphic Metric Mapping (LDDMM) with deep learning (DL) approaches. It explains that shapes are modeled as diffeomorphic transformations on a template within an infinite-dimensional shape space, where geodesics minimize energy $\frac{1}{2} \int_{0}^{1} \| v_t \|^2_{G_{c_t}} dt$ and geodesic regression extends to population trajectories on a Riemannian manifold. It reviews DL architectures—autoencoders, GANs, RNNs, and transformers—and discusses their potential to capture nonlinear shape changes from imaging data, alongside challenges of data hunger and lack of physical interpretability. The authors advocate hybrid physics-informed and causal DL, diffusion models, and multimodal datasets as promising directions to improve generalization and interpretability, while highlighting the need for open longitudinal datasets for robust benchmarking. They conclude that combining geometric models with modern DL and mechanistic insights offers a practical path toward accurate, interpretable predictions of shape evolution in diseases and development.

Abstract

Living biological tissue is a complex system, constantly growing and changing in response to external and internal stimuli. These processes lead to remarkable and intricate changes in shape. Modeling and understanding both natural and pathological (or abnormal) changes in the shape of anatomical structures is highly relevant, with applications in diagnostic, prognostic, and therapeutic healthcare. Nevertheless, modeling the longitudinal shape change of biological tissue is a non-trivial task due to its inherent nonlinear nature. In this review, we highlight several existing methodologies and tools for modeling longitudinal shape change (i.e., spatiotemporal shape modeling). These methods range from diffeomorphic metric mapping to deep-learning based approaches (e.g., autoencoders, generative networks, recurrent neural networks, etc.). We discuss the synergistic combinations of existing technologies and potential directions for future research, underscoring key deficiencies in the current research landscape.

Figures (8)

• Figure 1: A) Longitudinal phase contrast imaging of 3D cell cultured cervical cancer spheroids Muniandy2021*. B) Neurodegradation of brain structure with progression of AD, from healthy to moderate AD (top to bottom). Adapted from Pasnoori2024*. C) Longitudinal MRI imaging of the morphogenesis of a femur during the embryonic and fetal periods. Figure adapted from Suzuki2019*. *Images obtained from referenced sources and licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)
• Figure 2: A) An illustration of diffeomorphisms $[\phi_{1},...,\phi_{N}]$ acting on a baseline reference shape $y_{0}$ to transform it to a shape within a dataset $[y_{1},...,y_{N}]$. A diffeomorphism constitutes $p$ momentum vectors $m_{t}$ acting on a similar number of control points $c_{t}$. B) Diffeomorphisms within the LDDMM framework lie on a Riemannian manifold $M$. The shortest paths (i.e., geodesics) connecting the reference shape and other shapes are used to describe the transformation and are determined based on minimal deformational energy.
• Figure 3: A) Each shape in a longitudinal dataset of $N$ shapes spanning $[t_{0},t_{N}]$ can be described with a corresponding diffeomorphism at time $t$, $\phi_{t}$, acting on reference shape $y_{0}$. These diffeomorphisms are obtained from the estimation of an underlying group-average geodesic. Thus, the action of $\phi_{t}$ on $y_{0}$ leads to an estimate for the corresponding shape $\hat{y_{t}}$B) Each diffeomorphism lies on a Riemannian manifold $M$, and an underlying group-average geodesic, which describes the trajectory of diffeomorphisms, can be estimated via geodesic regression.
• Figure 4: A) A dataset of various shapes spanning $[t_{0},t_{N}]$ can be described as diffeomorphic transformations of an underlying baseline template shape $y_{0}$. B) Individual shape trajectories can be modeled by individual geodesic regressions, which can be used to estimate a group-average geodesic or vice-versa (i.e., group-average geodesic used to estimate individual trajectories).
• Figure 5: A) Autoencoder structure consisting of an encoder ($\theta_{E}$) which translates an input image $x_{i}$ into a vector of latent variables $\mathbf{z_{r}}$. A decoder ($\theta_{D}$) then attempts to reconstruct input data $\hat{x}_{i}$ from $\mathbf{z_{r}}$. B) A variational autoencoder consists of similar components, however $\theta_{E}$ maps $x_{i}$ instead to deterministic parameters $z_{\mu}$ and $z_{\sigma}$ which describe a probabilistic distribution. These are then used to obtain $\mathbf{z_{r}}$ and similarly decoded.