Endogenous Reprompting: Self-Evolving Cognitive Alignment for Unified Multimodal Models

TL;DR

This work tackles the persistent cognitive gap between understanding and generation in Unified Multimodal Models by introducing Endogenous Reprompting. It presents SEER, a self-evolving framework that converts latent model understanding into explicit, self-aligned reprompts through a two-stage loop: RLVR to cultivate a high-fidelity internal evaluator, and RLMT to optimize a reprompting policy using endogenous rewards, all bootstrapped from a compact Visual Instruction Elaboration proxy dataset of 300 samples. By freezing the generation module and training only the understanding/reasoning components, SEER achieves model-specific alignment, producing reprompts that strictly match the generator’s priors and substantially improve instruction compliance and visual generation quality. Across rigorous benchmarks, SEER outperforms state-of-the-art baselines in evaluation accuracy, reprompting efficiency, and generation quality while preserving general multimodal capabilities. This approach shifts the learning focus from improving pixel-level generation to evolving cognitive reasoning within a unified model framework, enabling efficient, self-guided alignment in UMMs.

Abstract

Unified Multimodal Models (UMMs) exhibit strong understanding, yet this capability often fails to effectively guide generation. We identify this as a Cognitive Gap: the model lacks the understanding of how to enhance its own generation process. To bridge this gap, we propose Endogenous Reprompting, a mechanism that transforms the model's understanding from a passive encoding process into an explicit generative reasoning step by generating self-aligned descriptors during generation. To achieve this, we introduce SEER (Self-Evolving Evaluator and Reprompter), a training framework that establishes a two-stage endogenous loop using only 300 samples from a compact proxy task, Visual Instruction Elaboration. First, Reinforcement Learning with Verifiable Rewards (RLVR) activates the model's latent evaluation ability via curriculum learning, producing a high-fidelity endogenous reward signal. Second, Reinforcement Learning with Model-rewarded Thinking (RLMT) leverages this signal to optimize the generative reasoning policy. Experiments show that SEER consistently outperforms state-of-the-art baselines in evaluation accuracy, reprompting efficiency, and generation quality, without sacrificing general multimodal capabilities.

Figures (9)

• Figure 1: Bridging the cognitive gap. While direct generation fails to reflect the model's understanding (Left), and external reprompters often cause misalignment by generating descriptors that mismatch the generator's priors (Right), SEER (Middle) leverages Endogenous Reprompting. It produces concrete, self-aligned descriptors that strictly match the generator's generative priors, successfully bridging understanding and generation.
• Figure 2: Overview of SEER. The framework bridges the cognitive gap via a two-stage endogenous loop. RLVR: (Step 1) collecting endogenous image pairs to (Step 2) optimize the internal evaluator. RLMT: (Step 3) optimizing the Reprompting Policy using (Step 4) relative rewards derived from the activated evaluator.
• Figure 3: Stage 1: RLVR. We employ curriculum learning to transform the model into a robust internal critic. By training on pairwise comparisons (from basic alignment to instruction discrimination) using GRPO, we activate a high-fidelity internal evaluator $E(x; a, p_0)$ capable of assessing user intention.
• Figure 4: Stage 2: RLMT. The model optimizes its reasoning policy $\pi_\theta$ via a "Think-Generate-Evaluate" loop. The endogenous reward is computed by comparing the reasoned generation ($x_{\text{pol}}$) against a naive baseline ($x_{\text{ref}}$), effectively teaching the model to "think" (reprompt) for superior visual results.
• Figure 5: Qualitative Comparison. While the Base Model (Left) fails to execute visual instructions due to the cognitive gap, and External Reprompters (Right) cause representation mismatch, SEER (Center) generates self-aligned reprompts that strictly match the generator's priors, yielding superior visual fidelity.