Improving Text-to-Image Generation with Input-Side Inference-Time Scaling

TL;DR

The paper addresses suboptimal outputs in text-to-image generation caused by underspecified prompts by introducing input-side inference-time scaling: an LLM-based prompt rewriter trained entirely with iterative Direct Preference Optimization (DPO) while keeping T2I backbones frozen. The method uses a multimodal LLM judge to provide pairwise preferences across four rewards—image quality, general alignment, physical alignment, and aesthetics—and builds two rewriters (general and aesthetics) that optimize different objectives. Empirical results demonstrate consistent improvements in image-text alignment, image quality, and aesthetics across multiple backbones and benchmarks, with clear scalability as LLM size grows and strong transferability across T2I models. The approach is practical, model-agnostic, and data-efficient (no SFT), and the authors plan to release code and trained rewrites to facilitate adoption.

Abstract

Recent advances in text-to-image (T2I) generation have achieved impressive results, yet existing models often struggle with simple or underspecified prompts, leading to suboptimal image-text alignment, aesthetics, and quality. We propose a prompt rewriting framework that leverages large language models (LLMs) to refine user inputs before feeding them into T2I backbones. Our approach introduces a carefully designed reward system and an iterative direct preference optimization (DPO) training pipeline, enabling the rewriter to enhance prompts without requiring supervised fine-tuning data. We evaluate our method across diverse T2I models and benchmarks. Results show that our prompt rewriter consistently improves image-text alignment, visual quality, and aesthetics, outperforming strong baselines. Furthermore, we demonstrate strong transferability by showing that a prompt rewriter trained on one T2I backbone generalizes effectively to others without needing to be retrained. We also systematically study scalability, evaluating how performance gains scale with the capacity of the large LLM used as the rewriter. These findings highlight that prompt rewriting is an effective, scalable, and practical model-agnostic strategy for improving T2I systems. We plan to release the code and trained prompt rewriters soon.

Figures (11)

• Figure 1: Rewritten prompts lead to images that are more detailed, visually pleasing, and better aligned with the input text, with quality improving through successive rounds of DPO training.
• Figure 2: The Iterative DPO Training Pipeline. Given a user input prompt, the rewriter generates multiple candidate refinements, which are used by a frozen T2I model to synthesize images. Reward models evaluate these images to produce pairwise preferences. The chosen and rejected prompt pairs are then used to update the rewriter via Direct Preference Optimization, and the process is repeated iteratively for multiple rounds. The T2I model is kept frozen.
• Figure 3: Win rates for rewriters trained on various T2I backbones but evaluated on a fixed target, either (a) FLUX.1-schnell or (b) SD-3.5-medium. Evaluation is on the Pick-a-pic v2 dataset using a GPT-4o judge. Results show strong transferability.
• Figure 4: Win rate for different base models in different rounds. The prompt rewriters are evaluated on Pick-a-pic v2 with GPT-4o.
• Figure 5: Average length of rewritten prompts across different DPO training rounds, computed on the Pick-a-Pic v2 dataset.