OmniQuality-R: Advancing Reward Models Through All-Encompassing Quality Assessment

TL;DR

OmniQuality-R introduces a structured, reasoning-driven reward model for all-encompassing image quality assessment, unifying technical quality, aesthetic quality, and text-image alignment under a Plan-then-Reason framework. It combines a cold-start supervised fine-tuning stage on a deliberately constructed CoT dataset with reinforcement learning via Group Relative Policy Optimization using a Gaussian reward, augmented by STD filtering and entropy gating to stabilize learning and improve generalization. Empirical results across diverse IQA tasks and test-time T2I generation demonstrate strong cross-domain performance, few-shot generalization, and the ability to guide generation quality at inference. The approach offers a scalable path to robust, interpretable reward models for multimodal vision-language systems and downstream image generation workflows.

Abstract

Current visual evaluation approaches are typically constrained to a single task. To address this, we propose OmniQuality-R, a unified reward modeling framework that transforms multi-task quality reasoning into continuous and interpretable reward signals for policy optimization. Inspired by subjective experiments, where participants are given task-specific instructions outlining distinct assessment principles prior to evaluation, we propose OmniQuality-R, a structured reward modeling framework that transforms multi-dimensional reasoning into continuous and interpretable reward signals. To enable this, we construct a reasoning-enhanced reward modeling dataset by sampling informative plan-reason trajectories via rejection sampling, forming a reliable chain-of-thought (CoT) dataset for supervised fine-tuning (SFT). Building on this, we apply Group Relative Policy Optimization (GRPO) for post-training, using a Gaussian-based reward to support continuous score prediction. To further stabilize the training and improve downstream generalization, we incorporate standard deviation (STD) filtering and entropy gating mechanisms during reinforcement learning. These techniques suppress unstable updates and reduce variance in policy optimization. We evaluate OmniQuality-R on three key IQA tasks: aesthetic quality assessment, technical quality evaluation, and text-image alignment.

Figures (11)

• Figure 1: Overview of the challenges in multimodal image quality assessment and the proposed OmniQuality-R framework. Left: Existing methods lack a structured reasoning process, often yielding incomplete analysis, whereas OmniQuality-R introduces a step-by-step, interpretable reasoning structure. Right: OmniQuality-R achieves improved generalization performance, particularly with few-shot data through reinforcement learning (RL), outperforming existing SOTA models across multiple benchmarks. Bottom: OmniQuality-R supports multi-dimensional evaluation—technical quality, aesthetic appeal, and text-image alignment—each guided by a transparent think process and yielding final quality predictions.
• Figure 2: Overview of the OmniQuality-R Framework. The framework consists of three stages: (1) Generating Chain-of-Thought (CoT) data with quality level prediction based on structured image analysis; (2) Reasoning-aware supervised fine-tuning (SFT) using reformatted CoT samples, emphasizing hard cases; (3) Gaussian-reward guided GRPO that applies a standard deviation-based filter and high-entropy token gating to optimize policy learning, improving reasoning robustness under rule-based supervision.
• Figure 3: The T2I optimization process visualization guided by our proposed OmniQuality-R.
• Figure 4: The visualization of think process synthesis for text-image alignment.
• Figure 5: The visualization of think process synthesis for image technical quality assessment.