Generator Matching: Generative modeling with arbitrary Markov processes

TL;DR

This work introduces Generator Matching (GM), a general framework for scalable generative modeling on arbitrary state spaces via Markov process generators. By decomposing generative dynamics into conditional and marginal generators and leveraging the Kolmogorov Forward Equation, GM unifies diffusion, flow-matching, and discrete diffusion while enabling jump processes and multimodal models. It provides a tractable training objective (CGM) with a broad family of Bregman divergences and demonstrates practical gains through jump models and Markov superpositions in image and protein generation. The framework offers a rigorous theoretical foundation and a rich design space for creating flexible, multimodal generative models across diverse domains.

Abstract

We introduce Generator Matching, a modality-agnostic framework for generative modeling using arbitrary Markov processes. Generators characterize the infinitesimal evolution of a Markov process, which we leverage for generative modeling in a similar vein to flow matching: we construct conditional generators which generate single data points, then learn to approximate the marginal generator which generates the full data distribution. We show that Generator Matching unifies various generative modeling methods, including diffusion models, flow matching and discrete diffusion models. Furthermore, it expands the design space to new and unexplored Markov processes such as jump processes. Finally, Generator Matching enables the construction of superpositions of Markov generative models and enables the construction of multimodal models in a rigorous manner. We empirically validate our method on image and multimodal generation, e.g. showing that superposition with a jump process improves performance.

Figures (11)

• Figure 1: Overview of the Generator Matching (GM) framework to construct generative models. GM works on any state space (including multi-modal) and Markov processes. Flower image source:vecteezy.com
• Figure 2: Illustration of Markov models trained with different KFE solutions for the same probability path. The paths for individual samples are plotted across time in one plot. 2d histograms of generated samples are plotted per time point. Although the individual sample paths look very different, the marginal probability path (histogram) are the same up to approximation error (Geometric average $\sim$ example 1, mixture $\sim$ example 2).
• Figure 3: Examples of generated proteins with $SO(3)$ jumps model and MultiFlow. Each protein passes the designability filter check and is structurally unique.
• Figure 4: . Examples of generated images on CIFAR10 (top) and ImageNet32 (bottom).
• Figure 5: Illustration of Markov models in \ref{['table:novel_kfe_sols']} trained on a checkerboard pattern data distribution with a mixture probability path. As one can see, the flow model does not perform well on this probability path (due to the high Lipschitz constant), while the jump model performs very well because it can "teleport" mass due to its discontinuity.

Theorems & Definitions (7)

• Theorem 1: Universal characterization of generators
• Proposition 1
• Proposition 2
• Proposition 3: Combining models
• Proposition 4: Multimodal generative models - Informal version
• Lemma 1
• Proposition 5: Multimodal generative models - Formal version