Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Quantized Adaptive Conditions for Diffusion Models

Yuchen Liang, Yuchuan Tian, Lei Yu, Huao Tang, Jie Hu, Xiangzhong Fang, Hanting Chen

TL;DR

This paper proposes a novel and effective approach to reduce trajectory curvature by utilizing adaptive conditions by employing a extremely light-weight quantized encoder, which accelerates ODE sampling while preserving the downstream task image editing capabilities of SDE techniques.

Abstract

The curvature of ODE trajectories in diffusion models hinders their ability to generate high-quality images in a few number of function evaluations (NFE). In this paper, we propose a novel and effective approach to reduce trajectory curvature by utilizing adaptive conditions. By employing a extremely light-weight quantized encoder, our method incurs only an additional 1% of training parameters, eliminates the need for extra regularization terms, yet achieves significantly better sample quality. Our approach accelerates ODE sampling while preserving the downstream task image editing capabilities of SDE techniques. Extensive experiments verify that our method can generate high quality results under extremely limited sampling costs. With only 6 NFE, we achieve 5.14 FID on CIFAR-10, 6.91 FID on FFHQ 64x64 and 3.10 FID on AFHQv2.

Learning Quantized Adaptive Conditions for Diffusion Models

TL;DR

This paper proposes a novel and effective approach to reduce trajectory curvature by utilizing adaptive conditions by employing a extremely light-weight quantized encoder, which accelerates ODE sampling while preserving the downstream task image editing capabilities of SDE techniques.

Abstract

The curvature of ODE trajectories in diffusion models hinders their ability to generate high-quality images in a few number of function evaluations (NFE). In this paper, we propose a novel and effective approach to reduce trajectory curvature by utilizing adaptive conditions. By employing a extremely light-weight quantized encoder, our method incurs only an additional 1% of training parameters, eliminates the need for extra regularization terms, yet achieves significantly better sample quality. Our approach accelerates ODE sampling while preserving the downstream task image editing capabilities of SDE techniques. Extensive experiments verify that our method can generate high quality results under extremely limited sampling costs. With only 6 NFE, we achieve 5.14 FID on CIFAR-10, 6.91 FID on FFHQ 64x64 and 3.10 FID on AFHQv2.
Paper Structure (17 sections, 1 theorem, 16 equations, 5 figures, 7 tables, 2 algorithms)

This paper contains 17 sections, 1 theorem, 16 equations, 5 figures, 7 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Diffusion Model
  4. Rectified Flow
  5. Coupling Optimization
  6. Adaptive Conditions
  7. Discretization Error Control
  8. Quantized Condition Encoder
  9. Online Sampling Weight Collection
  10. Training and Sampling
  11. Experiments
  12. Quantized Condition Encoder
  13. Working with Reparamized Noise Encoder
  14. Comparison with state-of-the-arts
  15. Zero-Shot Image Editing
  16. ...and 2 more sections

Key Result

theorem thmcountertheorem

Let $S_{t, y}\sim \tilde{p}_t$ be a simulation of $X_t|_{Y=y} \sim p_{t,y}$ and be the one-step further simulation of $X_{t-\Delta t, y}\sim p_{t-\Delta t,y}$. And $d_{t,\Delta t}(S_{t,Y},Y)\sim \tilde{p}_{t-\Delta t}$ denote the overall simulation of $X_{t-\Delta t}\sim p_{t-\Delta t}$. Then we can contral the Wasserstein distance where $L$ is the Lipschitz constant for $d_{t,\Delta t}(\cdot, y

Figures (5)

  • Figure 1: (a) Denoising diffusion models with nonlinear forward flow ho2020denoisingsong2020score have complex ODE trajectors. (b) Linear Flow models liu2022flowkarras2022elucidatingsong2020denoisingnichol2021improved still have highly curved ODE trajectories. (c) Coupling optimizationlee2023minimizing methods try to reduce curvature by trajectory relocation, but are limited by the difficulty of keeping the noise distribution unchanged. (d) Adaptive Conditions untie the crossover between forward trajectories without compromising the full simulation accuracy.
  • Figure 2: A visual schematic of our approach.
  • Figure 3: Visualization of intermediate samples. Adaptive conditions allow for sharper initial predictions at high noise level, as indicated by red boxes.
  • Figure 4: Qualitative comparison between our method and baseline on CIFAR-10, FFHQ and AFHQv2.
  • Figure 5: Our method allows SDE-based zero-shot image editing applications such as super-resolution, colorization and inpainting. In the experiments, we used Karras's schedule with steps $N=40$.

Theorems & Definitions (2)

  • theorem thmcountertheorem
  • proof