REVEALER: Reinforcement-Guided Visual Reasoning for Element-Level Text-Image Alignment Evaluation

TL;DR

The paper tackles the need for fine-grained, interpretable evaluation of text-to-image alignment. It presents REVEALER, a reinforcement-guided visual reasoning framework that uses a grounding–reasoning–conclusion paradigm, supported by a cold-start supervised fine-tuning phase and a GRPO-based reinforcement learning stage to produce element-level alignment judgments. Key contributions include a 25K visual reasoning trajectory dataset constructed from EvalMuse-40K, a two-stage training pipeline, and state-of-the-art results on EvalMuse-40K, RichHF, MHaluBench, and GenAI-Bench, along with improved efficiency relative to iterative reasoning approaches. The work delivers interpretable, human-aligned evaluation signals that generalize across benchmarks, offering a scalable tool for validating and improving T2I models in practical deployments.

Abstract

Evaluating the alignment between textual prompts and generated images is critical for ensuring the reliability and usability of text-to-image (T2I) models. However, most existing evaluation methods rely on coarse-grained metrics or static QA pipelines, which lack fine-grained interpretability and struggle to reflect human preferences. To address this, we propose REVEALER, a unified framework for element-level alignment evaluation based on reinforcement-guided visual reasoning. Adopting a structured "grounding-reasoning-conclusion" paradigm, our method enables Multimodal Large Language Models (MLLMs) to explicitly localize semantic elements and derive interpretable alignment judgments. We optimize the model via Group Relative Policy Optimization(GRPO) using a composite reward function that incorporates structural format, grounding accuracy, and alignment fidelity. Extensive experiments across four benchmarks-EvalMuse-40K, RichHF, MHaluBench, and GenAI-Bench-demonstrate that REVEALER achieves state-of-the-art performance. Our approach consistently outperforms both strong proprietary models and supervised baselines while demonstrating superior inference efficiency compared to existing iterative visual reasoning methods.

Figures (5)

• Figure 1: Revealer performs element-level text-to-image alignment evaluation via structured visual reasoning, following a grounding–reasoning–conclusion paradigm.
• Figure 2: Our work consists of three components: (a) Training data is constructed using Grounding DINO and GPT-4o to generate structured alignment annotations; (b) A two-stage training pipeline performs reinforcement-guided visual reasoning via GRPO; (c) The model is evaluated on four fine-grained alignment benchmarks.
• Figure 3: Accuracy across different element categories on the EvalMuse-40K benchmark.
• Figure 4: (a) GRPO training yields significant MIOU gains (+0.15/+0.11) for both models, narrowing the gap with the expert Grounding DINO. (b) A strong positive correlation ($r=0.8873$) between box and element rewards confirms that precise visual localization is essential for accurate alignment evaluation.
• Figure 5: Comparison of training stability and final accuracy between binary and continuous reward designs. Continuous rewards result in more stable training and better alignment performance on EvalMuse-40K, achieving an ACC of 0.7964 compared to 0.7746 with binary rewards.