Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Image

TL;DR

MMRB2 delivers the first comprehensive, multi-task benchmark for reward models on omni models that interleave text and image data. It combines diverse prompts, frontier-model responses, and high-consensus human preferences to evaluate four subtasks: text-to-image generation, image editing, interleaved generation, and multimodal reasoning. The study reveals that top API models approach human performance for rewards on generation tasks but lag on reasoning, with strong downstream correlations to GenAI benchmarks, underscoring MMRB2's value as a predictor of downstream success. The results also highlight limitations of current evaluators and reward-model training, and propose directions such as improved scaling strategies and safety-aware extensions. Overall, MMRB2 provides a robust foundation for advancing reward modeling in omni, interleaved multimodal settings.

Abstract

Reward models (RMs) are essential for training large language models (LLMs), but remain underexplored for omni models that handle interleaved image and text sequences. We introduce Multimodal RewardBench 2 (MMRB2), the first comprehensive benchmark for reward models on multimodal understanding and (interleaved) generation. MMRB2 spans four tasks: text-to-image, image editing, interleaved generation, and multimodal reasoning ("thinking-with-images"), providing 1,000 expert-annotated preference pairs per task from 23 models and agents across 21 source tasks. MMRB2 is designed with: (1) practical but challenging prompts; (2) responses from state-of-the-art models and agents; and (3) preference pairs with strong human-expert consensus, curated via an ensemble filtering strategy. Using MMRB2, we study existing judges for each subtask, including multimodal LLM-as-a-judge and models trained with human preferences. The latest Gemini 3 Pro attains 75-80% accuracy. GPT-5 and Gemini 2.5 Pro reach 66-75% accuracy, compared to >90% for humans, yet surpass the widely used GPT-4o (59%). The best performing open-source model Qwen3-VL-32B achieves similar accuracies as Gemini 2.5 Flash (64%). We also show that MMRB2 performance strongly correlates with downstream task success using Best-of-N sampling and conduct an in-depth analysis that shows key areas to improve the reward models going forward.

Figures (17)

• Figure 1: Examples of multimodal preference pairs in MMRB2 across four subtasks: text-to-image generation, interleaved generation, image editing, and multimodal reasoning, showing human and model judgments on challenging prompts.
• Figure 2: Breakdown of MMRB2 by task type and source, and detailed categories under each task.
• Figure 3: Overview of the MMRB2 data pipeline. The process combines ensemble MLLM judging, human verification, and multi-stage filtering to ensure high-quality, reasoning-consistent preference pairs across tasks.
• Figure 4: Downstream best-of-N score v.s. MMRB2 performance. We perform best-of-N sampling with 2 base models each on 4 tasks (GenAI-Bench GENAI-Bench, GEdit-Bench liu2025step1x, ISG-Bench ISG, and EMMA EMMA). A judge's score on MMRB2 strongly correlates with improvement in downstream tasks when it is used in best-of-N sampling, highlighting MMRB2's utility for downstream task success.
• Figure 5: Majority-vote accuracy of each MLLM as the number of samples $K$ varies. Test-time scaling yields small gains for GPT and Gemini models but no improvement for Qwen3-VL.