Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On The Hidden Biases of Flow Matching Samplers

Soon Hoe Lim

TL;DR

The paper analyzes empirical flow matching (FM) and conditional FM (CFM) in generative modeling, highlighting that empirical minimization yields velocity fields that are typically not gradient fields and thus fail to reproduce optimal transport (OT) maps or minimize kinetic energy. By framing FM/CFM within the Benamou–Brenier dynamical OT perspective and employing Helmholtz–Hodge decomposition, it shows that empirical flows generically contain non-conservative components, leading to energetic inefficiency and memorization tendencies. Despite this, the kinetic energy of generated samples exhibits sharp tail concentration: exponential under Gaussian sources and polynomial under heavy-tailed sources, with the tail behavior largely dictated by the source distribution rather than the data. These results clarify structural biases in empirical FM and motivate design principles for improved sampler dynamics.

Abstract

We study the implicit bias of flow matching (FM) samplers via the lens of empirical flow matching. Although population FM may produce gradient-field velocities resembling optimal transport (OT), we show that the empirical FM minimizer is almost never a gradient field, even when each conditional flow is. Consequently, empirical FM is intrinsically energetically suboptimal. In view of this, we analyze the kinetic energy of generated samples. With Gaussian sources, both instantaneous and integrated kinetic energies exhibit exponential concentration, while heavy-tailed sources lead to polynomial tails. These behaviors are governed primarily by the choice of source distribution rather than the data. Overall, these notes provide a concise mathematical account of the structural and energetic biases arising in empirical FM.

On The Hidden Biases of Flow Matching Samplers

TL;DR

The paper analyzes empirical flow matching (FM) and conditional FM (CFM) in generative modeling, highlighting that empirical minimization yields velocity fields that are typically not gradient fields and thus fail to reproduce optimal transport (OT) maps or minimize kinetic energy. By framing FM/CFM within the Benamou–Brenier dynamical OT perspective and employing Helmholtz–Hodge decomposition, it shows that empirical flows generically contain non-conservative components, leading to energetic inefficiency and memorization tendencies. Despite this, the kinetic energy of generated samples exhibits sharp tail concentration: exponential under Gaussian sources and polynomial under heavy-tailed sources, with the tail behavior largely dictated by the source distribution rather than the data. These results clarify structural biases in empirical FM and motivate design principles for improved sampler dynamics.

Abstract

We study the implicit bias of flow matching (FM) samplers via the lens of empirical flow matching. Although population FM may produce gradient-field velocities resembling optimal transport (OT), we show that the empirical FM minimizer is almost never a gradient field, even when each conditional flow is. Consequently, empirical FM is intrinsically energetically suboptimal. In view of this, we analyze the kinetic energy of generated samples. With Gaussian sources, both instantaneous and integrated kinetic energies exhibit exponential concentration, while heavy-tailed sources lead to polynomial tails. These behaviors are governed primarily by the choice of source distribution rather than the data. Overall, these notes provide a concise mathematical account of the structural and energetic biases arising in empirical FM.

Paper Structure

This paper contains 13 sections, 7 theorems, 89 equations.

Table of Contents

  1. Introduction
  2. Flow Matching (FM) and Conditional Flow Matching (CFM)
  3. Empirical Flow Matching
  4. FM Through the Lens of Energetics
  5. Conclusion
  6. Acknowledgment.
  7. Related Work
  8. Proof of Theoretical Results
  9. Proof of Proposition \ref{['prop1']}
  10. Proof of Proposition \ref{['prop2']}
  11. Proof of Proposition \ref{['prop3']}
  12. Proof of Theorem \ref{['thm1']}
  13. Proof of Theorem \ref{['thm2_polydecay']}

Key Result

Proposition 1

For the family of affine flows, the minimizer $\hat{v}^*$ of the empirical FM objective admits a closed-form formula: where $a_t$ and $b_t$ are given in eq_ab, and $w_i(t,z)$ is a kernel-dependent weighting function given by: with

Theorems & Definitions (23)

  • Example 1: Rectified Flow
  • Example 2: Affine Flows
  • Example 3: Empirical Rectified Flow
  • Example 4: Empirical Affine Flows
  • Proposition 1
  • Remark 1
  • Example 5: Optimal Velocity of Rectified Flow Can Be a Gradient Field
  • Example 6: Explicit Examples mena2025statistical
  • Proposition 2
  • Proposition 3: Population setting, OT case
  • ...and 13 more