Unified Personalized Understanding, Generating and Editing

TL;DR

OmniPersona tackles the challenge of personalized, end-to-end capabilities in unified multimodal models by introducing structurally decoupled concept tokens and an explicit knowledge replay mechanism, enabling consistent personalized understanding, generation, and the novel capability of personalized editing. The framework is complemented by OmniPBench, a cross-task benchmark extending UnifyBench with personalized editing tasks to systematically evaluate synergy across tasks. Empirical results show competitive performance across understanding, generation, and notably superior personalized editing, with ablations highlighting the importance of token decoupling and knowledge replay for reducing cross-task interference and enhancing attribute-grounded generation. This work advances controllable, personalized AI assistants and provides a strong baseline for future research in unified personalization and editing-enabled multimodal systems.

Abstract

Unified large multimodal models (LMMs) have achieved remarkable progress in general-purpose multimodal understanding and generation. However, they still operate under a ``one-size-fits-all'' paradigm and struggle to model user-specific concepts (e.g., generate a photo of \texttt{<maeve>}) in a consistent and controllable manner. Existing personalization methods typically rely on external retrieval, which is inefficient and poorly integrated into unified multimodal pipelines. Recent personalized unified models introduce learnable soft prompts to encode concept information, yet they either couple understanding and generation or depend on complex multi-stage training, leading to cross-task interference and ultimately to fuzzy or misaligned personalized knowledge. We present \textbf{OmniPersona}, an end-to-end personalization framework for unified LMMs that, for the first time, integrates personalized understanding, generation, and image editing within a single architecture. OmniPersona introduces structurally decoupled concept tokens, allocating dedicated subspaces for different tasks to minimize interference, and incorporates an explicit knowledge replay mechanism that propagates personalized attribute knowledge across tasks, enabling consistent personalized behavior. To systematically evaluate unified personalization, we propose \textbf{\texttt{OmniPBench}}, extending the public UnifyBench concept set with personalized editing tasks and cross-task evaluation protocols integrating understanding, generation, and editing. Experimental results demonstrate that OmniPersona delivers competitive and robust performance across diverse personalization tasks. We hope OmniPersona will serve as a strong baseline and spur further research on controllable, unified personalization.

Figures (12)

• Figure 1: Capability Overview of OmniPersona. OmniPersona leverages decoupled learnable prompts to achieve unified personalized understanding, generation, and editing from only a few concept images paired with textual descriptions. Notably, OmniPersona is the first framework to enable personalized image editing, a critical capability overlooked by previous works.
• Figure 2: Current state-of-the-art personalized unified models fail to stably leverage textual knowledge during generation, resulting in images misaligned with concept descriptions (top). Moreover, existing methods neglect personalized image editing, producing outputs unrelated to the input image (bottom).
• Figure 3: Model Overview of OmniPersona. We employ an end-to-end dual-branch routing mechanism to train two sets of parameter-decoupled prompts.
• Figure 4: Explicit knowledge replay pipeline.
• Figure 5: Qualitative comparison with Yo'Chameleon and Unictoken on OmniPBench. OmniPersona (Ours) demonstrates more precise, personalized image generation and excellent capability of personalized image editing.