Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ProxT2I: Efficient Reward-Guided Text-to-Image Generation via Proximal Diffusion

Zhenghan Fang, Jian Zheng, Qiaozi Gao, Xiaofeng Gao, Jeremias Sulam

TL;DR

ProxT2I tackles slow sampling and misalignment in diffusion-based text-to-image generation for human imagery by using backward proximal diffusion with learned proximal operators. It integrates reinforcement learning via Group Relative Policy Optimization to optimize for rewards, improving human-preference alignment while preserving fast sampling. The approach learns conditional proximal functions, extends proximal diffusion to text conditioning with a proximal CFG, and uses an auxiliary variable trick to enable tractable RL updates. A new LAION-Face-T2I-15M dataset (15M images with fine-grained captions and a 3M hand-focused subset) supports training and evaluation, and experiments show competitive performance with lower compute and smaller models.

Abstract

Diffusion models have emerged as a dominant paradigm for generative modeling across a wide range of domains, including prompt-conditional generation. The vast majority of samplers, however, rely on forward discretization of the reverse diffusion process and use score functions that are learned from data. Such forward and explicit discretizations can be slow and unstable, requiring a large number of sampling steps to produce good-quality samples. In this work we develop a text-to-image (T2I) diffusion model based on backward discretizations, dubbed ProxT2I, relying on learned and conditional proximal operators instead of score functions. We further leverage recent advances in reinforcement learning and policy optimization to optimize our samplers for task-specific rewards. Additionally, we develop a new large-scale and open-source dataset comprising 15 million high-quality human images with fine-grained captions, called LAION-Face-T2I-15M, for training and evaluation. Our approach consistently enhances sampling efficiency and human-preference alignment compared to score-based baselines, and achieves results on par with existing state-of-the-art and open-source text-to-image models while requiring lower compute and smaller model size, offering a lightweight yet performant solution for human text-to-image generation.

ProxT2I: Efficient Reward-Guided Text-to-Image Generation via Proximal Diffusion

TL;DR

ProxT2I tackles slow sampling and misalignment in diffusion-based text-to-image generation for human imagery by using backward proximal diffusion with learned proximal operators. It integrates reinforcement learning via Group Relative Policy Optimization to optimize for rewards, improving human-preference alignment while preserving fast sampling. The approach learns conditional proximal functions, extends proximal diffusion to text conditioning with a proximal CFG, and uses an auxiliary variable trick to enable tractable RL updates. A new LAION-Face-T2I-15M dataset (15M images with fine-grained captions and a 3M hand-focused subset) supports training and evaluation, and experiments show competitive performance with lower compute and smaller models.

Abstract

Diffusion models have emerged as a dominant paradigm for generative modeling across a wide range of domains, including prompt-conditional generation. The vast majority of samplers, however, rely on forward discretization of the reverse diffusion process and use score functions that are learned from data. Such forward and explicit discretizations can be slow and unstable, requiring a large number of sampling steps to produce good-quality samples. In this work we develop a text-to-image (T2I) diffusion model based on backward discretizations, dubbed ProxT2I, relying on learned and conditional proximal operators instead of score functions. We further leverage recent advances in reinforcement learning and policy optimization to optimize our samplers for task-specific rewards. Additionally, we develop a new large-scale and open-source dataset comprising 15 million high-quality human images with fine-grained captions, called LAION-Face-T2I-15M, for training and evaluation. Our approach consistently enhances sampling efficiency and human-preference alignment compared to score-based baselines, and achieves results on par with existing state-of-the-art and open-source text-to-image models while requiring lower compute and smaller model size, offering a lightweight yet performant solution for human text-to-image generation.

Paper Structure

This paper contains 27 sections, 29 equations, 14 figures.

Table of Contents

  1. Introduction
  2. Related Works
  3. Diffusion Models.
  4. Diffusion models and reinforcement learning for T2I generation.
  5. Methods
  6. Warm-up: Proximal Diffusion Models
  7. Proximal Diffusion Models.
  8. Proximal Diffusion for Conditional Generation
  9. Learning Conditional Proximal Functions.
  10. Proximal Diffusion Policy Optimization
  11. Integrating RL with proximal diffusion.
  12. LAION-Face-T2I-15M Dataset
  13. Experiments and Results
  14. Comparison with competing methods
  15. Results at $256^2$ Resolution.
  16. ...and 12 more sections

Figures (14)

  • Figure 1: HPSv2.1 score wu2023human vs. number of sampling steps at $256^2$ resolution. Our ProxT2I achieves more efficient and human-preference-aligned T2I generation than competing methods.
  • Figure 2: Samples generated by ProxT2I and competing methods at $256^2$ resolution using 10 sampling steps.
  • Figure 3: Comparison between ProxT2I and competing methods at $512^2$ resolution.
  • Figure 4: Comparison with Stable Diffusion (SD) 3.5 Medium esser2024scaling and its Flow-GRPO-finetuned variant liu2025flow.
  • Figure 5: Example images and corresponding fine-grained captions in the LAION-Face-T2I-15M dataset.
  • ...and 9 more figures