CoF-T2I: Video Models as Pure Visual Reasoners for Text-to-Image Generation

TL;DR

CoF-T2I addresses the gap between video-based Chain-of-Frame reasoning and text-to-image generation by reusing a video foundation model to perform a structured three-frame refinement, decoding only the final frame as the image. It introduces a frame-wise latent encoding to minimize motion artifacts and trains on a 64K CoF-Evol-Instruct dataset that provides stepwise, defect-aware supervision for semantic grounding and aesthetic refinement. The approach yields strong results on GenEval (0.86) and Imagine-Bench (7.468), outperforming the base video model and competing with unified, text-planning methods, demonstrating the viability of pure visual reasoning for high-quality T2I generation. This work suggests a promising direction for leveraging video-based reasoning to improve controllability and fidelity in image synthesis, with potential extensions to longer video tasks and reinforcement learning-based refinements.

Abstract

Recent video generation models have revealed the emergence of Chain-of-Frame (CoF) reasoning, enabling frame-by-frame visual inference. With this capability, video models have been successfully applied to various visual tasks (e.g., maze solving, visual puzzles). However, their potential to enhance text-to-image (T2I) generation remains largely unexplored due to the absence of a clearly defined visual reasoning starting point and interpretable intermediate states in the T2I generation process. To bridge this gap, we propose CoF-T2I, a model that integrates CoF reasoning into T2I generation via progressive visual refinement, where intermediate frames act as explicit reasoning steps and the final frame is taken as output. To establish such an explicit generation process, we curate CoF-Evol-Instruct, a dataset of CoF trajectories that model the generation process from semantics to aesthetics. To further improve quality and avoid motion artifacts, we enable independent encoding operation for each frame. Experiments show that CoF-T2I significantly outperforms the base video model and achieves competitive performance on challenging benchmarks, reaching 0.86 on GenEval and 7.468 on Imagine-Bench. These results indicate the substantial promise of video models for advancing high-quality text-to-image generation.

Figures (7)

• Figure 1: Comparison of Inference-time Reasoning Models. (a) Equipping image models with external verifier. (b) Interleaving textual planning within unified multimodal large language models. (c) CoF-T2I: Our proposed video-based CoF reasoning model.
• Figure 2: Visualizations of CoF-T2I output. We visualize the reasoning trajectories generated by CoF-T2I. For each example, the final output is shown in large, and the intermediate latent frames are shown in small.
• Figure 3: Overview of CoF-T2I. CoF-T2I builds on a video generation backbone, reframing inference-time reasoning for T2I generation as a CoF refinement process. Training. Given a CoF trajectory, we employ a video VAE to encode each frame, and optimize a vanilla flow matching objective. Inference. Starting from noisy initialization, the model denoises to sample a progressively refined reasoning trajectory internalized during training, only the final-frame latent is fully decoded and taken as the output image. Quality assessment. Along the CoF trajectory, text-image alignment and aesthetic quality continue to improve.
• Figure 4: Curation Pipeline for CoF-Evol-Instruct. A quality-aware construction pipeline to curate reasoning data. We generate an initial pool of images across diverse distributions and dynamically route valid samples. These images are then expanded into complete CoF sequences through targeted construction strategies. Our pipeline ensures both sample-level diversity and frame-wise consistency.
• Figure 5: Visualization of CoF-Evol-Instruct Dataset. We showcase the prompt and corresponding CoF trajectories in our data, including five categories: Attribute Binding, Object Combination, Spatial Arrangement, Context Manipulation, and Quantity Control.