Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Direct Discriminative Optimization: Your Likelihood-Based Visual Generative Model is Secretly a GAN Discriminator

Kaiwen Zheng, Yongxin Chen, Huayu Chen, Guande He, Ming-Yu Liu, Jun Zhu, Qinsheng Zhang

TL;DR

This work addresses the inherent mode-covering limitation of forward KL in likelihood-based visual generative models by introducing Direct Discriminative Optimization (DDO), which implicitly encodes a GAN-style discriminator through the likelihood ratio between a learnable target and a fixed reference. By enabling multi-round self-play finetuning without architectural changes or extra discriminators, DDO substantially improves diffusion and autoregressive models, achieving state-of-the-art (guidance-free) FID scores on CIFAR-10, ImageNet-64, and ImageNet-512, and delivering strong results on autoregressive VAR models without sampling tricks. The approach unifies likelihood-based training with adversarial discrimination, connects to DPO and guidance methods, and offers a practical, efficient path to push pretrained models closer to the data distribution with minimal computational overhead. The findings highlight the practical potential of density-ratio based discrimination to augment high-fidelity visual generation at scale, with avenues for extending to $f$-divergences and more complex tasks like text-to-image synthesis.

Abstract

While likelihood-based generative models, particularly diffusion and autoregressive models, have achieved remarkable fidelity in visual generation, the maximum likelihood estimation (MLE) objective, which minimizes the forward KL divergence, inherently suffers from a mode-covering tendency that limits the generation quality under limited model capacity. In this work, we propose Direct Discriminative Optimization (DDO) as a unified framework that integrates likelihood-based generative training and GAN-type discrimination to bypass this fundamental constraint by exploiting reverse KL and self-generated negative signals. Our key insight is to parameterize a discriminator implicitly using the likelihood ratio between a learnable target model and a fixed reference model, drawing parallels with the philosophy of Direct Preference Optimization (DPO). Unlike GANs, this parameterization eliminates the need for joint training of generator and discriminator networks, allowing for direct, efficient, and effective finetuning of a well-trained model to its full potential beyond the limits of MLE. DDO can be performed iteratively in a self-play manner for progressive model refinement, with each round requiring less than 1% of pretraining epochs. Our experiments demonstrate the effectiveness of DDO by significantly advancing the previous SOTA diffusion model EDM, reducing FID scores from 1.79/1.58/1.96 to new records of 1.30/0.97/1.26 on CIFAR-10/ImageNet-64/ImageNet 512x512 datasets without any guidance mechanisms, and by consistently improving both guidance-free and CFG-enhanced FIDs of visual autoregressive models on ImageNet 256x256.

Direct Discriminative Optimization: Your Likelihood-Based Visual Generative Model is Secretly a GAN Discriminator

TL;DR

This work addresses the inherent mode-covering limitation of forward KL in likelihood-based visual generative models by introducing Direct Discriminative Optimization (DDO), which implicitly encodes a GAN-style discriminator through the likelihood ratio between a learnable target and a fixed reference. By enabling multi-round self-play finetuning without architectural changes or extra discriminators, DDO substantially improves diffusion and autoregressive models, achieving state-of-the-art (guidance-free) FID scores on CIFAR-10, ImageNet-64, and ImageNet-512, and delivering strong results on autoregressive VAR models without sampling tricks. The approach unifies likelihood-based training with adversarial discrimination, connects to DPO and guidance methods, and offers a practical, efficient path to push pretrained models closer to the data distribution with minimal computational overhead. The findings highlight the practical potential of density-ratio based discrimination to augment high-fidelity visual generation at scale, with avenues for extending to -divergences and more complex tasks like text-to-image synthesis.

Abstract

While likelihood-based generative models, particularly diffusion and autoregressive models, have achieved remarkable fidelity in visual generation, the maximum likelihood estimation (MLE) objective, which minimizes the forward KL divergence, inherently suffers from a mode-covering tendency that limits the generation quality under limited model capacity. In this work, we propose Direct Discriminative Optimization (DDO) as a unified framework that integrates likelihood-based generative training and GAN-type discrimination to bypass this fundamental constraint by exploiting reverse KL and self-generated negative signals. Our key insight is to parameterize a discriminator implicitly using the likelihood ratio between a learnable target model and a fixed reference model, drawing parallels with the philosophy of Direct Preference Optimization (DPO). Unlike GANs, this parameterization eliminates the need for joint training of generator and discriminator networks, allowing for direct, efficient, and effective finetuning of a well-trained model to its full potential beyond the limits of MLE. DDO can be performed iteratively in a self-play manner for progressive model refinement, with each round requiring less than 1% of pretraining epochs. Our experiments demonstrate the effectiveness of DDO by significantly advancing the previous SOTA diffusion model EDM, reducing FID scores from 1.79/1.58/1.96 to new records of 1.30/0.97/1.26 on CIFAR-10/ImageNet-64/ImageNet 512x512 datasets without any guidance mechanisms, and by consistently improving both guidance-free and CFG-enhanced FIDs of visual autoregressive models on ImageNet 256x256.

Paper Structure

This paper contains 31 sections, 3 theorems, 46 equations, 18 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Likelihood-Based Generative Models
  4. GANs
  5. Direct Discriminative Optimization
  6. Your Likelihood-Based Generative Model is Secretly a Discriminator
  7. Theoretical Analysis
  8. Practical Implementation
  9. Discussion
  10. Comparison with Existing Methods
  11. Direct Preference Optimization (DPO)
  12. Guidance Methods
  13. Experiments
  14. Experimental Setups
  15. Results on Diffusion Models
  16. ...and 16 more sections

Key Result

Theorem 3.1

With unlimited model capacity, the optimal likelihood-based model $p_\theta$ under the objective satisfies $p_{\theta^*}=p_\text{data}$.

Figures (18)

  • Figure 1: Samples on ImageNet 512$\times$512, without any guidance.
  • Figure 2: Toy example illustrating DDO. (a) Models pretrained via maximum likelihood estimation (MLE) exhibit dispersed density, while DDO imposes contrastive forces toward the data distribution. (b) The finetuned model concentrates better on the main mode.
  • Figure 3: Illustration of DDO. (1) Models. $\theta_{\text{ref}}$ is the (pretrained) reference model frozen during training. $\theta$ is the learnable model initialized as $\theta_{\text{ref}}$. (2) Data. Samples from $p_\text{data}$ are drawn from the training dataset. Samples from $p_{\theta_{\text{ref}}}$ are generated by the reference model, either offline or online. (3) Objective. The target model $\theta$ is optimized by applying the GAN discriminator loss with the implicitly parameterized discriminator $d_\theta$ to distinguish between real samples from $p_\text{data}$ and fake samples from $p_{\theta_{\text{ref}}}$.
  • Figure 4: Comparison with DPO.
  • Figure 5: Comparison of model parameter counts and inference time across different guidance methods and DDO. For DG, we measure the statistics on class-conditional CIFAR-10. For AG, we measure the statistics on ImageNet-64.
  • ...and 13 more figures

Theorems & Definitions (3)

  • Theorem 3.1: Optimality
  • Theorem 3.2: Divergence Bounds
  • Theorem 3.3