Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

VARGPT-v1.1: Improve Visual Autoregressive Large Unified Model via Iterative Instruction Tuning and Reinforcement Learning

Xianwei Zhuang, Yuxin Xie, Yufan Deng, Dongchao Yang, Liming Liang, Jinghan Ru, Yuguo Yin, Yuexian Zou

TL;DR

VARGPT-v1.1 advances a unified visual autoregressive model by integrating iterative visual instruction tuning with Direct Preference Optimization, expanding the training corpus to 8.3M visual-generative instruction pairs, upgrading the LLM backbone to Qwen2-7B-Instruct, and progressively scaling image resolution while enabling image editing without architectural changes. The approach blends supervised fine-tuning, RLHF-like policy optimization, and editing data to achieve state-of-the-art performance in multimodal understanding and text-to-image instruction-following across diverse benchmarks. Key contributions include a detailed three-stage training pipeline, a novel RL objective over image tokens, and architecture-agnostic visual editing capabilities, all within a single, coherent framework. The results demonstrate strong improvements in both comprehension and generation, with practical impact on unified multimodal systems that can understand, generate, and edit images from mixed-modal prompts.

Abstract

In this work, we present VARGPT-v1.1, an advanced unified visual autoregressive model that builds upon our previous framework VARGPT. The model preserves the dual paradigm of next-token prediction for visual understanding and next-scale generation for image synthesis. Specifically, VARGPT-v1.1 integrates: (1) a novel training strategy combining iterative visual instruction tuning with reinforcement learning through Direct Preference Optimization (DPO), (2) an expanded training corpus containing 8.3M visual-generative instruction pairs, (3) an upgraded language model backbone using Qwen2, (4) enhanced image generation resolution, and (5) emergent image editing capabilities without architectural modifications. These advancements enable VARGPT-v1.1 to achieve state-of-the-art performance in multimodal understanding and text-to-image instruction-following tasks, demonstrating significant improvements in both comprehension and generation metrics. Notably, through visual instruction tuning, the model acquires image editing functionality while maintaining architectural consistency with its predecessor, revealing the potential for unified visual understanding, generation, and editing. Our findings suggest that well-designed unified visual autoregressive models can effectively adopt flexible training strategies from large language models (LLMs), exhibiting promising scalability. The codebase and model weights are publicly available at https://github.com/VARGPT-family/VARGPT-v1.1.

VARGPT-v1.1: Improve Visual Autoregressive Large Unified Model via Iterative Instruction Tuning and Reinforcement Learning

TL;DR

VARGPT-v1.1 advances a unified visual autoregressive model by integrating iterative visual instruction tuning with Direct Preference Optimization, expanding the training corpus to 8.3M visual-generative instruction pairs, upgrading the LLM backbone to Qwen2-7B-Instruct, and progressively scaling image resolution while enabling image editing without architectural changes. The approach blends supervised fine-tuning, RLHF-like policy optimization, and editing data to achieve state-of-the-art performance in multimodal understanding and text-to-image instruction-following across diverse benchmarks. Key contributions include a detailed three-stage training pipeline, a novel RL objective over image tokens, and architecture-agnostic visual editing capabilities, all within a single, coherent framework. The results demonstrate strong improvements in both comprehension and generation, with practical impact on unified multimodal systems that can understand, generate, and edit images from mixed-modal prompts.

Abstract

In this work, we present VARGPT-v1.1, an advanced unified visual autoregressive model that builds upon our previous framework VARGPT. The model preserves the dual paradigm of next-token prediction for visual understanding and next-scale generation for image synthesis. Specifically, VARGPT-v1.1 integrates: (1) a novel training strategy combining iterative visual instruction tuning with reinforcement learning through Direct Preference Optimization (DPO), (2) an expanded training corpus containing 8.3M visual-generative instruction pairs, (3) an upgraded language model backbone using Qwen2, (4) enhanced image generation resolution, and (5) emergent image editing capabilities without architectural modifications. These advancements enable VARGPT-v1.1 to achieve state-of-the-art performance in multimodal understanding and text-to-image instruction-following tasks, demonstrating significant improvements in both comprehension and generation metrics. Notably, through visual instruction tuning, the model acquires image editing functionality while maintaining architectural consistency with its predecessor, revealing the potential for unified visual understanding, generation, and editing. Our findings suggest that well-designed unified visual autoregressive models can effectively adopt flexible training strategies from large language models (LLMs), exhibiting promising scalability. The codebase and model weights are publicly available at https://github.com/VARGPT-family/VARGPT-v1.1.

Paper Structure

This paper contains 17 sections, 4 equations, 11 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Model Architecture
  5. Training
  6. Visual Instruction Tuning
  7. Reinforcement Learning Human Feedback
  8. SFT for Visual Editing
  9. Instruction-following and Preference Data
  10. Instruction-following Data
  11. Preference Instruction Data
  12. Image Editing Data
  13. Experiments
  14. Main Results
  15. Conclusion, Limitation and Future Work
  16. ...and 2 more sections

Figures (11)

  • Figure 1: Some generated 512$\times$512 samples by VARGPT-v1.1. Our VARGPT-v1.1 supports text-and-image instructions from user and outputs both text-and-image mixed modal data simultaneously.
  • Figure 2: A comparative analysis of various MLLMs across multiple visual comprehension benchmarks is presented. The remaining metrics are derived from standard visual question-answering benchmarks and multi-modal comprehension benchmarks. Notably, our VARGPT-v1.1 model demonstrates significant superiority over the compared baselines across all comprehension benchmarks.
  • Figure 3: Comparison of different model architectures, where, 'AR' denotes autoregressive, while 'VAR' signifies visual autoregressive. We present a comparative analysis of architectures designed for comprehension-only tasks, generation-only tasks, and unified comprehension and generation, alongside our proposed VARGPT-v1.1 an VARGPT ding2021cogviewmasteringtexttoimagegeneration model. Our VARGPT-v1.1 and VARGPT are conceptualized as purely autoregressive multimodel model, achieving visual comprehension through next-token prediction and visual generation through next-scale prediction paradigms.
  • Figure 4: The illustration of the proposed VARGPT-v1.1 framework similar to VARGPT zhuang2025vargptunifiedunderstandinggeneration, which consists of (1) an LLM (Qwen2-7B-Instruct Qwen2VLqwen2), visual encoder and a understanding projector for visual understanding; (2) a visual decoder and dual generation projectors for visual generation. VARGPT-v1.1 employs causal attention in the LLM backbone while utilizing block causal attention in the visual decoder.
  • Figure 5: The three training stages of the VARGPT, including stage-1 pretraining, stage-2 visual instruction tuning and stage-3 iterative training.
  • ...and 6 more figures