Constant Acceleration Flow

TL;DR

Constant Acceleration Flow (CAF) is introduced, a novel framework based on a simple constant acceleration equation that outperforms state-of-the-art baselines for one-step generation and dramatically improves few-step coupling preservation and inversion over Rectified flow.

Abstract

Rectified flow and reflow procedures have significantly advanced fast generation by progressively straightening ordinary differential equation (ODE) flows. They operate under the assumption that image and noise pairs, known as couplings, can be approximated by straight trajectories with constant velocity. However, we observe that modeling with constant velocity and using reflow procedures have limitations in accurately learning straight trajectories between pairs, resulting in suboptimal performance in few-step generation. To address these limitations, we introduce Constant Acceleration Flow (CAF), a novel framework based on a simple constant acceleration equation. CAF introduces acceleration as an additional learnable variable, allowing for more expressive and accurate estimation of the ODE flow. Moreover, we propose two techniques to further improve estimation accuracy: initial velocity conditioning for the acceleration model and a reflow process for the initial velocity. Our comprehensive studies on toy datasets, CIFAR-10, and ImageNet 64x64 demonstrate that CAF outperforms state-of-the-art baselines for one-step generation. We also show that CAF dramatically improves few-step coupling preservation and inversion over Rectified flow. Code is available at \href{https://github.com/mlvlab/CAF}{https://github.com/mlvlab/CAF}.

Figures (16)

• Figure 1: Initial Velocity Conditioning (IVC). We illustrate the importance of IVC to address the flow crossing problem, which hinders the learning of straight ODE trajectories during training. In Fig. \ref{['fig:fig3a']}, Rectified flow suffers from approximation errors at the overlapping point $\mathbf{x}_t$ (where $\mathbf{x}_t^1 = \mathbf{x}_t^2$), resulting in curved sampling trajectories due to flow crossing. Conversely, Fig. \ref{['fig:fig3b']} demonstrates that CAF, utilizing IVC, successfully estimates ground-truth trajectories by minimizing the ambiguity at $\mathbf{x}_t$.
• Figure 2: 2D synthetic dataset. We compare results between 2-Rectified flow and our Constant Acceleration Flow (CAF) on 2D synthetic data. $\pi_0$ (blue) and $\pi_1$ (green) are source and target distributions parameterized by Gaussian mixture models. Here, the number of sampling steps is $N=1$. While 2-Rectified flow frequently generates samples that deviate from $\pi_1$, CAF more accurately estimates the target distribution $\pi_1$. The generated samples (orange) from CAF form a more similar distribution as the target distribution $\pi_1$.
• Figure 3: Sampling trajectories of CAF with different $h$. The sampling trajectories of CAF are displayed for different values of $h$, which determines the initial velocity and acceleration. $\pi_0$ and $\pi_1$ are mixtures of Gaussian distributions. We sample across sampling steps of $N=7$ to show how sampling trajectories change with $h$.
• Figure 4:
• Figure 5:

Theorems & Definitions (3)

• Definition C.1
• Definition C.2
• Theorem 1