Diffeomorphic Measure Matching with Kernels for Generative Modeling

TL;DR

Diffeomorphic Measure Matching with Kernels for Generative Modeling introduces KODE, a kernel-based ODE transport framework that learns diffeomorphic maps to push a reference measure toward a target by minimizing the MMD between pushforwards. The velocity field lives in an RKHS and is regularized via inducing points, yielding provable error bounds that decompose into approximation, statistical, and misspecification components. The paper provides algorithmic details, including kernel choices and a triangular transport variant (T-KODE) for conditional simulation, and validates the approach on 2D and high-dimensional benchmarks, MNIST, and a Lotka–Volterra parameter inference task, often matching or outperforming neural-ODE-based OT methods. Overall, the work offers a theoretically grounded, practically implementable alternative to neural-network–based transport models with diffeomorphic guarantees and flexible conditioning capabilities. The insights advance transport-based generative modeling by integrating RKHS theory, explicit error control, and efficient inducing-point strategies, with potential impact on sampling, inference, and conditional generation in high dimensions.

Abstract

This article presents a general framework for the transport of probability measures towards minimum divergence generative modeling and sampling using ordinary differential equations (ODEs) and Reproducing Kernel Hilbert Spaces (RKHSs), inspired by ideas from diffeomorphic matching and image registration. A theoretical analysis of the proposed method is presented, giving a priori error bounds in terms of the complexity of the model, the number of samples in the training set, and model misspecification. An extensive suite of numerical experiments further highlights the properties, strengths, and weaknesses of the method and extends its applicability to other tasks, such as conditional simulation and inference.

Figures (7)

• Figure 1: Transport experiments on 2D benchmarks using KODE: (Top row) The empirical samples from the target $\nu$; (Middle row) Samples generated by transporting a standard Gaussian reference $\eta$; (Bottom row) Samples generated by transporting $\nu$ backward towards $\eta$.
• Figure 1: KODE sample trajectories for 2D benchmarks with different choices of $\lambda_2$ in \ref{['sobolev-in-time-RKHS-norm']}: (Top row) using $\lambda_2 \neq 0$ so the time derivatives of the coefficients are penalized; (Bottom row) using $\lambda_2 =0$.
• Figure 2: Conditional sampling using Triangular KODE for two-dimensional benchmarks: The left-most column shows the target measure $\nu$ while the red, green, and blue lines denote slices along which conditional samples are generated. The next column shows generated samples by KODE. The remaining panels compare ground truth (solid lines) and generated (dashed lines) kernel density estimators of the requisite conditional distributions using triangular KODE.
• Figure 2: Transport experiments on GAS benchmark using non-autonomous KODE. (First row) Marginal distributions of the data measure $\nu$. (Second row) Marginal distributions of the learned pushforward measure. (Third row) Marginal distributions of the pullback measure obtained from the backward flow of KODE. (Fourth row) Marginal distributions of the pullback measure obtained from the backward flow of KODE trained to transport $\eta$ and $\nu$ between each other simultaneously.
• Figure 3: Transport experiments on HEPMASS benchmark using non-autonomous KODE. (First row) 2D marginals of the true data set; (Second row) 2D marginals of samples generated by KODE.

Theorems & Definitions (16)

• Lemma 2.1
• Remark 2.2
• Lemma 2.3
• Lemma 2.4
• Proposition 2.5
• Proof 1
• Proposition 2.6
• Proof 2
• Lemma 2.7
• Remark 2.8