MetaCaptioner: Towards Generalist Visual Captioning with Open-source Suites

TL;DR

Open-source visual captioning has lagged behind commercial systems, limiting applications. The authors propose CapFlow, a multi-agent, domain-routing framework that synthesizes high-quality captions with open-source MLLMs, and MetaCaptioner, fine-tuned on CapFlow-generated data to become a generalist captioner. They demonstrate that CapFlow achieves GPT-4.1-level caption quality across domains at an 89.5% reduction in cost, and that MetaCaptioner delivers top-tier multimodal performance among open-source models. This work provides a scalable, cost-effective path for multimodal research and data synthesis, enabling broader and cheaper access to high-quality visual captioning and reasoning capabilities.

Abstract

Generalist visual captioning goes beyond a simple appearance description task, but requires integrating a series of visual cues into a caption and handling various visual domains. In this task, current open-source models present a large performance gap with commercial ones, which limits various applications such as data synthesis. To bridge the gap, this paper proposes CapFlow, a novel multi-agent collaboration workflow. CapFlow demonstrates for the first time that, by capitalizing on open-source models, it is possible to achieve caption quality on par with GPT-4.1 in various domains with an 89.5% reduction in costs. By leveraging CapFlow as the data synthesizer, we produce high-quality visual captions from image and video domains at scale, and obtain a generalist visual captioner via fine-tuning, namely MetaCaptioner. Through extensive experiments, we show that MetaCaptioner not only achieves comparable captioning capabilities with commercial models but also reaches top-tier multimodal performance in the open-source community. We hope CapFlow and MetaCaptioner can benefit future multimodal research by providing a strong and cost-effective visual captioning solution.

Figures (9)

• Figure 1: Comparison of caption quality between GPT-4.1, OmniCaptioner Omnicaptioner and MetaCaptioner. These metrics are computed by GPT-5. MetaCaptioner is able to generate informative captions for a variety of visual domains with comparable quality to GPT-4.1.
• Figure 2: Comparison of existing captioning systems with our our methods. Our CapFlow adopts a multi-agent collaboration workflow to produce high-quality captions for training MetaCaptioner-8B. Under the setting of visual reasoning with LLMs, CapFlow and MetaCaptioner-8B achieve comparable performance with GPT-4.1 using open-source suites.
• Figure 3: Overview of CapFlow Framework. CapFlow can dynamically select an appropriate caption workflow for the input image, and each workflow is equipped with multiple agents for close collaboration. With this pipeline, CapFlow can produce high-quality captions of various visual domains with open-source MLLMs, e.g., Qwen2.5-VL qwen2_5_vl.
• Figure 4: Training pipeline of MetaCaptioner. We leverage CapFLow as the data synthesizer to generate various captions, upon which a strict reject sampling pipeline is used to filter out low-quality ones. The obtained captions are combined with common instruction data to train MetaCaptioner.
• Figure 5: Category distribution of MetaCaption-4.1M. a) The distribution of 9 main domain; b) The data distribution in each subdomain. We report the percentage of each subdomain of its corresponding main domain.