Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignment

Ziang Yan, Zhilin Li, Yinan He, Chenting Wang, Kunchang Li, Xinhao Li, Xiangyu Zeng, Zilei Wang, Yali Wang, Yu Qiao, Limin Wang, Yi Wang

TL;DR

Task Preference Optimization (TPO) introduces differentiable task preferences to bridge multimodal large language models with specialized vision heads. By attaching learnable task tokens to region, temporal, and mask heads and training in a three-stage local-to-global scheme, MLLMs achieve significant gains in fine-grained visual perception while preserving multimodal dialogue performance. The method yields about 14.6% average improvements across video-language benchmarks and shows strong zero-shot capabilities; ablations confirm data scaling and co-training benefits. TPO demonstrates scalability across models (VideoChat, LLaVA) and tasks, and highlights a practical path to fuse expert vision signals with generalist MLLMs. Limitations include focus on discriminative tasks and supervision-dependent data, suggesting future work in self-supervised extensions.

Abstract

Current multimodal large language models (MLLMs) struggle with fine-grained or precise understanding of visuals although they give comprehensive perception and reasoning in a spectrum of vision applications. Recent studies either develop tool-using or unify specific visual tasks into the autoregressive framework, often at the expense of overall multimodal performance. To address this issue and enhance MLLMs with visual tasks in a scalable fashion, we propose Task Preference Optimization (TPO), a novel method that utilizes differentiable task preferences derived from typical fine-grained visual tasks. TPO introduces learnable task tokens that establish connections between multiple task-specific heads and the MLLM. By leveraging rich visual labels during training, TPO significantly enhances the MLLM's multimodal capabilities and task-specific performance. Through multi-task co-training within TPO, we observe synergistic benefits that elevate individual task performance beyond what is achievable through single-task training methodologies. Our instantiation of this approach with VideoChat and LLaVA demonstrates an overall 14.6% improvement in multimodal performance compared to baseline models. Additionally, MLLM-TPO demonstrates robust zero-shot capabilities across various tasks, performing comparably to state-of-the-art supervised models. The code will be released at https://github.com/OpenGVLab/TPO

Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignment

TL;DR

Task Preference Optimization (TPO) introduces differentiable task preferences to bridge multimodal large language models with specialized vision heads. By attaching learnable task tokens to region, temporal, and mask heads and training in a three-stage local-to-global scheme, MLLMs achieve significant gains in fine-grained visual perception while preserving multimodal dialogue performance. The method yields about 14.6% average improvements across video-language benchmarks and shows strong zero-shot capabilities; ablations confirm data scaling and co-training benefits. TPO demonstrates scalability across models (VideoChat, LLaVA) and tasks, and highlights a practical path to fuse expert vision signals with generalist MLLMs. Limitations include focus on discriminative tasks and supervision-dependent data, suggesting future work in self-supervised extensions.

Abstract

Current multimodal large language models (MLLMs) struggle with fine-grained or precise understanding of visuals although they give comprehensive perception and reasoning in a spectrum of vision applications. Recent studies either develop tool-using or unify specific visual tasks into the autoregressive framework, often at the expense of overall multimodal performance. To address this issue and enhance MLLMs with visual tasks in a scalable fashion, we propose Task Preference Optimization (TPO), a novel method that utilizes differentiable task preferences derived from typical fine-grained visual tasks. TPO introduces learnable task tokens that establish connections between multiple task-specific heads and the MLLM. By leveraging rich visual labels during training, TPO significantly enhances the MLLM's multimodal capabilities and task-specific performance. Through multi-task co-training within TPO, we observe synergistic benefits that elevate individual task performance beyond what is achievable through single-task training methodologies. Our instantiation of this approach with VideoChat and LLaVA demonstrates an overall 14.6% improvement in multimodal performance compared to baseline models. Additionally, MLLM-TPO demonstrates robust zero-shot capabilities across various tasks, performing comparably to state-of-the-art supervised models. The code will be released at https://github.com/OpenGVLab/TPO
Paper Structure (49 sections, 1 equation, 8 figures, 23 tables)

This paper contains 49 sections, 1 equation, 8 figures, 23 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Vision Foundation Model.
  4. Multimodal Large Language Model.
  5. Alignment in MLLM.
  6. Method
  7. Task Preference Models
  8. Task Preference Optimization
  9. Stage 1: Task Assignment.
  10. Stage 2: Vision Task Training.
  11. Stage 3: Multi-task Training.
  12. Experiment
  13. Implementation Details.
  14. Benchmarks.
  15. General Understanding Evaluation
  16. ...and 34 more sections

Figures (8)

  • Figure 1: TPO uses differentiable task preferences from dense visual supervisions via task-specific heads to enhance MLLMs in fine-grained understanding.
  • Figure 2: Comparison of Learning Method. A solid line indicates data flow, and a dotted line represents feedback. and denote modules that are frozen and unfrozen.
  • Figure 3: Overall Pipeline of TPO. The architecture of Task Preference Optimization (TPO) consists of four main components: (1) a vision encoder, (2) a connector, (3) a large language model, and (4) a series of visual task heads. Differently colored flame symbols indicate which components are unfrozen at various stages of the training process.
  • Figure 4: Qualitative Results of Spatial Grounding.
  • Figure 5: Qualitative Results of Referring Segmentation.
  • ...and 3 more figures