SRUM: Fine-Grained Self-Rewarding for Unified Multimodal Models

TL;DR

SRUM introduces a fine-grained post-training framework that lets a Unified Multimodal Model's understanding module serve as an internal evaluator to improve its own generation. By implementing a global-local dual reward system and reward-weighted training, SRUM provides multi-scale feedback and stable optimization without external labeled data. Empirical results show SRUM achieves state-of-the-art performance on complex compositional and reasoning benchmarks and demonstrates robust in-domain and out-of-domain generalization. The work highlights a paradigm where internal critique guides self-improvement, with potential for scalable, data-efficient enhancement of multimodal systems.

Abstract

Recently, remarkable progress has been made in Unified Multimodal Models (UMMs), which integrate vision-language generation and understanding capabilities within a single framework. However, a significant gap exists where a model's strong visual understanding often fails to transfer to its visual generation. A model might correctly understand an image based on user instructions, yet be unable to generate a faithful image from text prompts. This phenomenon directly raises a compelling question: Can a model achieve self-improvement by using its understanding module to reward its generation module? To bridge this gap and achieve self-improvement, we introduce SRUM, a self-rewarding post-training framework that can be directly applied to existing UMMs of various designs. SRUM creates a feedback loop where the model's own understanding module acts as an internal ``evaluator'', providing corrective signals to improve its generation module, without requiring additional human-labeled data. To ensure this feedback is comprehensive, we designed a global-local dual reward system. To tackle the inherent structural complexity of images, this system offers multi-scale guidance: a \textbf{global reward} ensures the correctness of the overall visual semantics and layout, while a \textbf{local reward} refines fine-grained, object-level fidelity. SRUM leads to powerful capabilities and shows strong generalization, boosting performance on T2I-CompBench from 82.18 to \textbf{88.37} and on T2I-ReasonBench from 43.82 to \textbf{46.75}. Overall, our work establishes a powerful new paradigm for enabling a UMMs' understanding module to guide and enhance its own generation via self-rewarding.

Figures (6)

• Figure 1: The example on the left suggests that the current UMMs' understanding module has exceeded the capability of its generation module: the generation module is prone to producing incorrect candidate images based on a given prompt in relevant scenarios, a situation which the understanding module can reasonably identify. This not only highlights a gap between understanding and generation but also reveals the potential for understanding to guide generation. Inspired by this insight, we propose SRUM to bridge this gap, particularly in complex generation domains.
• Figure 2: Showcase of the SRUM pipeline. It consists of two main stages: Self-Rewarding Data Generation and Reward-Weighted Training. The first stage generates high-quality data and scores it to produce a reward signal for the next training stage for self-improvement.
• Figure 3: Left: Module Evaluation. We report the accuracy drop ($\Delta$ Acc. %) from our SRUM. Specifically, 0-1 Reward represents the sparse reward. Right: Hyperparameters Evaluation on T2I-CompBench. We report the accuracy in different $\lambda$ under two modes: CoT and without CoT.
• Figure 4: Score per step during inference in Bagel with its ablation models.
• Figure 5: Functional cluster activation patterns of the different models (Bagel, SFT and SRUM) on understanding and generation tasks. The average activation strength of Understanding and Generation clusters is shown.