Consistent Approximation of Interpolating Splines in Image Metamorphosis

TL;DR

This work develops a variational framework for smooth interpolation (splines) in the image metamorphosis model, formulating a spline energy that jointly penalizes Eulerian flow acceleration and the second material derivative of image intensity. A time-discrete approximation is introduced and shown to Mosco-converge to the time-continuous spline energy, guaranteeing existence of metamorphosis splines for a given set of key frames. The authors also provide a fully discrete spatial discretization, a multiresolution strategy, and an iPALM-based optimizer to compute spline paths in practice. Numerical experiments demonstrate that metamorphosis splines yield superior temporal smoothness and reduced artifacts compared to piecewise geodesic interpolation, while illustrating the influence of surrounding key frames on the interpolated sequence. Overall, the paper offers a rigorous, implementable pathway to smooth image morphing trajectories with strong theoretical grounding and practical efficacy.

Abstract

This paper investigates a variational model for splines in the image metamorphosis model for the smooth interpolation of key frames in the space of images. The Riemannian manifold of images based on the metamorphosis model defines shortest geodesic paths interpolating two images as minimizers of the path energy which measures the viscous dissipation caused by the motion field and dissipation caused by the material derivative of the image intensity along motion paths. In this paper, we aim at smooth interpolation of multiple key frame images picking up the general observation of cubic splines in Euclidean space which minimize the squared acceleration along the interpolation path. To this end, we propose the spline functional which combines quadratic functionals of the Eulerian motion acceleration and of the second material derivative of the image intensity as the proper notion of image intensity acceleration. We propose a variational time discretization of this model and study the convergence to a suitably relaxed time continuous model via $Γ$-convergence methodology. As a byproduct, this also allows to establish the existence of metamorphosis splines for given key frame images as minimizers of the time continuous spline functional. The time discretization is complemented by effective spatial discretization based on finite differences and a stable B-spline interpolation of deformed quantities. A variety of numerical examples demonstrates the robustness and versatility of the proposed method in applications. For the minimization of the fully discrete energy functional a variant of the iPALM algorithm is used.

Figures (8)

• Figure 1: Left: Schematic drawing of the Hermite interpolation (blue) on the time interval $\color{blue}[(k-\tfrac{1}{2})/K, (k+\tfrac{1}{2})/K]$ together with the discrete acceleration $a_k^K(x)$. Right: Image extension $\mathcal{U}^K[\mathbf{u}^K,\mathbf{\Phi}^K](\cdot, x)$ along a path $(\psi_t^K(x))_{t \in [0,1]}$ from the left, plotted against time. Dots represent the values $u_k^K$, and crosses the "half-way" values $\tfrac{1}{2} (u_k^K+u_{k-1}^K)$ along the discrete transport path.
• Figure 2: Left: Time discrete spline with framed key frame images (first row), color-coded displacement field (second row), discrete second order material derivative (third row) and color-coded discrete acceleration field (fourth row) for the Gaussians example and values of the parameters $\delta=5\cdot 10^{-3}$, $\sigma=1$, $\theta=5\cdot 10^{-5}$. The colors and their intensities indicate the direction and the intensity of the field, as indicated by the color wheel on the left. Right: Euclidean splines in $(x,y,m)$ coordinates for the input parameters versus splines for metamorphosis in $(x,y,m)$ extracted from the numerical results in post-processing, with $(x,y)$ denoting the center of mass and $m$ the mass of the distribution.
• Figure 3: Left: Time discrete spline (top row) and piecewise geodesic (middle row) interpolation with framed key frames. The bottom row shows the difference in intensity between the different interpolations, using the color map $-0.35\space$$\space0.35$. Right: Width of the interpolated shape measured at the horizontal axis of symmetry (in number of pixels) for a spline interpolation (orange) and piecewise geodesic interpolation (green) showing the concavities ($\delta=5\cdot 10^{-3}$, $\sigma=1$, $\theta=5\cdot 10^{-4}$).
• Figure 4: Time discrete spline with framed fixed images (first and second row), first order material derivative slack variable $\bar{\mathbf{z}}$ (third and fourth row), second order material derivative with energies comparison (fifth and sixth row) and color-coded acceleration field with energies comparison (seventh and eight row) for values of the parameters $\delta=2\cdot 10^{-2},\,\sigma=2,\,\theta=8\cdot 10^{-4}$. The graphics on the right in row four and six show for the spline (orange) time plots of the $L^2$-norm of the actual second order material derivative $\hat{\mathbf{w}}$ and the dissipation energy density reflecting motion acceleration $\|W_A(\nabla_{{MN}}\mathbf{a}_k)\|_{L^1_{{MN}}}$, respectively. This is compared to the corresponding piecewise geodesic interpolation (green) (not visualized here, cf. Figure \ref{['fig:human_faces_comparison']}).
• Figure 5: Left: Time discrete spline with framed fixed images (top and bottom row). Right: Energy density norm of acceleration flow $\|W_A(\nabla_{{MN}}\mathbf{a}_k)\|_{L^1_{{MN}}}$ (top), and $L^2$-norm of the actual second order material derivative $\hat{\mathbf{w}}$ (bottom). Parameter values: $\delta=10^{-3}, \sigma=2, \theta=2\cdot 10^{-5}$.

Theorems & Definitions (24)

• Proposition 1
• proof
• Definition 1: Regularized spline energy
• Definition 2: Continuous time regularized spline interpolation
• Remark 1
• Definition 3: Discrete time regularized spline interpolation
• Lemma 1
• proof
• Remark 2
• Proposition 2