Self-Evaluation Unlocks Any-Step Text-to-Image Generation

TL;DR

This work introduces the Self-Evaluating Model (Self-E), a teacher-free pretraining framework for text-to-image generation that supports any-step inference by combining data-driven local supervision with a self-evaluation mechanism that uses the model’s own score estimates as a global teacher. By parameterizing a dual-time denoiser G_theta with two time inputs and optimizing a hybrid objective that blends L_data with a self-evaluation term L_self, Self-E achieves strong few-step performance and scales gracefully as the inference budget increases. The approach bridges diffusion/flow-matching and distillation paradigms without a pretrained teacher, achieving state-of-the-art GenEval results across budgets and offering robust, scalable generation from scratch. Extensive ablations confirm the importance of energy-preserving target normalization and training schedules that favor classifier-score guidance early and refinement later. The work opens new directions for teacher-free pretraining and flexible inference in high-fidelity image synthesis, with potential extensions to video and unconditional settings.

Abstract

We introduce the Self-Evaluating Model (Self-E), a novel, from-scratch training approach for text-to-image generation that supports any-step inference. Self-E learns from data similarly to a Flow Matching model, while simultaneously employing a novel self-evaluation mechanism: it evaluates its own generated samples using its current score estimates, effectively serving as a dynamic self-teacher. Unlike traditional diffusion or flow models, it does not rely solely on local supervision, which typically necessitates many inference steps. Unlike distillation-based approaches, it does not require a pretrained teacher. This combination of instantaneous local learning and self-driven global matching bridges the gap between the two paradigms, enabling the training of a high-quality text-to-image model from scratch that excels even at very low step counts. Extensive experiments on large-scale text-to-image benchmarks show that Self-E not only excels in few-step generation, but is also competitive with state-of-the-art Flow Matching models at 50 steps. We further find that its performance improves monotonically as inference steps increase, enabling both ultra-fast few-step generation and high-quality long-trajectory sampling within a single unified model. To our knowledge, Self-E is the first from-scratch, any-step text-to-image model, offering a unified framework for efficient and scalable generation.

Figures (10)

• Figure 1: Qualitative Any-Step Generation. We showcase diverse text-to-image results from our model at different inference step counts, demonstrating coherent semantics, strong text alignment. Text prompts are provided in the supplementary material.
• Figure 2: Self-Evaluating Model. (a) Overview. The model is trained with two complementary objectives: learning from data (b) and self-evaluation (c). (b) Learning from data. Given a real sample $\mathbf{x}_0$, we add noise to obtain $\mathbf{x}_t$ and train $G_\theta^{t \rightarrow s}$ with an $x_0$-prediction loss, providing local trajectory supervision. (c) Self-evaluation with classifier score. When $s<t$, we re-noise the generated $\hat{\mathbf{x}}_0$ to $\hat{\mathbf{x}}_s$ and run the same network in evaluation mode (stop-gradient) twice: once with condition $\mathbf{c}$ and once with the null prompt $\phi$. The difference between these outputs yields a self-evaluation score, which is treated as a feedback gradient on $\hat{\mathbf{x}}_0$ and back-propagated through the denoising path, enforcing global distribution matching in a teacher-free manner.
• Figure 3: Qualitative Any-Step Comparison. Generated images from all methods at various inference steps. Our approach consistently produces detailed, semantically accurate, and visually appealing images aligned with textual prompts at all step counts. In extremely few-step scenarios (e.g., 2-step), FLUX, SANA, and SDXL fail to generate recognizable results, while LCM and TiM exhibit semantic and structural degradation. When using more inference steps, all methods improve, but our method retains superior quality, realism, and text alignment. At 50 steps, normal Flow Matching realm, our method is competitive with FLUX, despite FLUX being a much larger model.
• Figure 4: Controlled Ablation Study. We compare our method to alternative pretraining methods - Flow Matching and IMM. Full prompts appear in supplementary. Our method produces favorable results across all step budgets.
• Figure 5: Training Progress Comparison. GenEval scores across different inference steps (2, 4, 8, and 50) for our method and Flow Matching over training iterations (from 50k to 300k). Our approach consistently outperforms Flow Matching at all inference steps, indicating its superior effectiveness and robustness.