Image Interpolation with Score-based Riemannian Metrics of Diffusion Models
Shinnosuke Saito, Takashi Matsubara
TL;DR
The paper introduces a score-based Riemannian metric on the data space of pre-trained diffusion models, defined by $G_{x_t}=J_{x_t}^ op J_{x_t}$ with $J_{x_t}= abla_{x_t}s_ heta(x_t,t)$, and performs geodesic interpolation under this metric to traverse the learned data manifold. To avoid unstable score landscapes at $t=0$, the method uses DDIM inversion to map samples to time $t=\tau>0$, computes the geodesic there, and then inverts back to the original data space, effectively producing smooth, semantically meaningful transitions between images. Experiments on MNIST and Stable Diffusion demonstrate that geometry-aware geodesics yield more realistic and faithful interpolations with fewer reconstruction errors than baseline methods such as Linear and Spherical interpolation, NAO, and NoiseDiffusion, as evidenced by CLIP-IQA and related metrics. The approach opens avenues for broader applications, including video editing via parallel transport across frames, by leveraging the manifold geometry of diffusion models for consistent content transformation.
Abstract
Diffusion models excel in content generation by implicitly learning the data manifold, yet they lack a practical method to leverage this manifold - unlike other deep generative models equipped with latent spaces. This paper introduces a novel framework that treats the data space of pre-trained diffusion models as a Riemannian manifold, with a metric derived from the score function. Experiments with MNIST and Stable Diffusion show that this geometry-aware approach yields image interpolations that are more realistic, less noisy, and more faithful to prompts than existing methods, demonstrating its potential for improved content generation and editing.