TUNA: Taming Unified Visual Representations for Native Unified Multimodal Models

TL;DR

Tuna introduces a native unified multimodal model that constructs a single, continuous visual representation by cascading a VAE encoder with a representation encoder, enabling end-to-end handling of images and videos for understanding and generation. The three-stage training pipeline and end-to-end fusion with an LLM decoder and flow-matching head enable strong performance across image/video understanding, generation, and editing, beating decoupled baselines and showing mutual task enhancement. Comprehensive ablations and qualitative results support the superiority of unified representations over dual-path designs like Show-o2, with evidence that stronger pretrained representation encoders further boost performance. The work demonstrates scalable, efficient integration of understanding and generation in a single framework, advancing native unified multimodal modeling.

Abstract

Unified multimodal models (UMMs) aim to jointly perform multimodal understanding and generation within a single framework. We present TUNA, a native UMM that builds a unified continuous visual representation by cascading a VAE encoder with a representation encoder. This unified representation space allows end-to-end processing of images and videos for both understanding and generation tasks. Compared to prior UMMs with decoupled representations, TUNA's unified visual space avoids representation format mismatches introduced by separate encoders, outperforming decoupled alternatives in both understanding and generation. Moreover, we observe that stronger pretrained representation encoders consistently yield better performance across all multimodal tasks, highlighting the importance of the representation encoder. Finally, in this unified setting, jointly training on both understanding and generation data allows the two tasks to benefit from each other rather than interfere. Our extensive experiments on multimodal understanding and generation benchmarks show that TUNA achieves state-of-the-art results in image and video understanding, image and video generation, and image editing, demonstrating the effectiveness and scalability of its unified representation design.

Figures (8)

• Figure 1: We present Tuna, a native unified multimodal model built on a unified visual representation, enabling diverse multimodal understanding and generation capabilities such as image and video understanding, image and video generation, and image editing.
• Figure 2: Overview of the Tuna architecture. Our model employs a VAE encoder and a representation encoder to construct unified visual representations, which are then combined with text tokens and processed by an LLM decoder. The decoder performs autoregressive text generation for understanding tasks and flow-matching-based visual generation for generation tasks. $^*$During visual generation, noise is added to the visual tokens to enable diffusion-based generation.
• Figure 3: Attention masks in the LLM decoder for understanding and generation tasks. $^*$ indicates that the visual tokens are noised.
• Figure 4: Comparison between Tuna and Show-o2 on how unified visual representations are produced.
• Figure 5: Representation alignment analysis with SigLIP 2 and SD3-Medium. For both Tuna and Show-o2, we extract visual representations at the input layer of the LLM decoder.