Decoupling actions of finite-dimensional Lie groups and of groups of diffeomorphisms in the large deformation framework
Rayane Mouhli, Thomas Pierron
TL;DR
This paper addresses the problem of disentangling actions of finite-dimensional Lie groups from the diffeomorphic deformations in the Large Deformation Diffeomorphic Metric Mapping (LDDMM) framework. It introduces a semidirect product $G\ltimes\operatorname{Diff}_{C_0^k}(\mathbb{R}^d)$ equipped with a right-invariant sub-Riemannian structure and augments the shape space to $\tilde{\mathcal{Q}}=G\times\mathcal{Q}$, along with a change of variable $\tilde{q}=g^{-1}\cdot q$ to decouple the two deformation modes via a momentum constraint $\mu=0$, yielding a reduced Hamiltonian on $T^*G\oplus T^*(\mathcal{Q}/G)$. It further introduces anisotropic Gaussian kernels with $\Sigma\in S_d^{++}$, allowing the deformation bias to be transported with the shape, and demonstrates joint optimization over $(X_t,v_t)$ to improve registration accuracy compared to two-stage approaches. The theoretical framework is complemented by applications to rigid and non-rigid motions (curves) and to landmark-based registration, with numerical results showing improved decoupling and reduced rotation leakage. The practical impact lies in providing a coherent, geometry-preserving method to separate and quantify coarse (group) and fine (diffeomorphic) deformation modes, together with anisotropic priors that can be transported across transformations for more faithful shape analysis and statistics.
Abstract
In computational anatomy, the Large Deformation Diffeomorphic Metric Mapping (LDDMM) framework has become a central tool for modeling smooth, invertible transformations between shapes such as curves or landmarks. In this paper, we extend this framework by enriching diffeomorphic deformations with transformations induced by finite-dimensional Lie groups (e.g. isometries, scalings), and we develop a registration model that decouples the actions of these two types of deformation on the shape during the matching process. To achieve this, we consider semidirect products between finite-dimensional groups and groups of diffeomorphisms, endowed with a right-invariant sub-Riemannian structure that give rise to new variational problems for shape registration. By exploiting symmetries and reduction theory, we decouple the contributions of each group throughout the matching process. We further extend the framework to incoroporate anisotropic deformations that preferentially favor certain directions during registration. On the numerical side, we propose an algorithm based on a joint optimization over both deformation groups, in contrast to the standard twostage approach that optimizes first over the finite-dimensional component and then over the diffeomorphic one. Experiments on curves and landmarks demonstrate that the proposed joint optimization improves registration accuracy and more effectively disentangles the contributions of the two deformation groups.