Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bezier Distillation

Ling Feng, SK Yang

TL;DR

The paper tackles error accumulation in Rectified Flow's distribution-to-distribution transport by introducing Bezier distillation, which inserts one or more guiding distributions and uses Bezier curve interpolation to shape the transport path. It formalizes quadratic and cubic Bezier-guided distillation with multi-teacher support, and derives drift-fitting objectives that enable direct prediction of the end state from the current state. The approach yields faster inference and improved sample quality with fewer Rectified Flow iterations, while maintaining alignment to the real data distribution $\\pi_1$. This framework offers a stable, efficient mechanism for distribution transfer in continuous-time generative models and suggests promising extensions to multimodal generation and optimal transport-based methods.

Abstract

In Rectified Flow, by obtaining the rectified flow several times, the mapping relationship between distributions can be distilled into a neural network, and the target distribution can be directly predicted by the straight lines of the flow. However, during the pairing process of the mapping relationship, a large amount of error accumulation will occur, resulting in a decrease in performance after multiple rectifications. In the field of flow models, knowledge distillation of multi - teacher diffusion models is also a problem worthy of discussion in accelerating sampling. I intend to combine multi - teacher knowledge distillation with Bezier curves to solve the problem of error accumulation. Currently, the related paper is being written by myself.

Bezier Distillation

TL;DR

The paper tackles error accumulation in Rectified Flow's distribution-to-distribution transport by introducing Bezier distillation, which inserts one or more guiding distributions and uses Bezier curve interpolation to shape the transport path. It formalizes quadratic and cubic Bezier-guided distillation with multi-teacher support, and derives drift-fitting objectives that enable direct prediction of the end state from the current state. The approach yields faster inference and improved sample quality with fewer Rectified Flow iterations, while maintaining alignment to the real data distribution . This framework offers a stable, efficient mechanism for distribution transfer in continuous-time generative models and suggests promising extensions to multimodal generation and optimal transport-based methods.

Abstract

In Rectified Flow, by obtaining the rectified flow several times, the mapping relationship between distributions can be distilled into a neural network, and the target distribution can be directly predicted by the straight lines of the flow. However, during the pairing process of the mapping relationship, a large amount of error accumulation will occur, resulting in a decrease in performance after multiple rectifications. In the field of flow models, knowledge distillation of multi - teacher diffusion models is also a problem worthy of discussion in accelerating sampling. I intend to combine multi - teacher knowledge distillation with Bezier curves to solve the problem of error accumulation. Currently, the related paper is being written by myself.

Paper Structure

This paper contains 11 sections, 9 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Background
  3. Rectified Flow
  4. Bezier Curve
  5. Methods
  6. Quartic Bezier Curve
  7. Cubic Quartic Bezier Curve/multi teacher
  8. Transport
  9. Experiment
  10. Conclusion
  11. You can have an appendix here.

Figures (1)

  • Figure 1: $X_0\sim\pi_1$ and $X_1\sim\pi_1$ are connected by the Bezier curve. Through learning the trajectory, under the guidance of the guiding distribution $X_T\sim\pi_T$, the model is mapped to the target distribution of $X_1\sim\pi_1$ in one step.