LongWriter-V: Enabling Ultra-Long and High-Fidelity Generation in Vision-Language Models

TL;DR

The paper tackles the bottleneck of ultra-long output generation in vision-language models by showing that the scarcity of long-output fine-tuning data limits generation length. It introduces LongWriter-V-22k, a 22k+ SFT dataset with inputs spanning single and multi-image prompts and outputs up to 10,000 words, and an IterDPO framework that breaks long outputs into segments for iterative human and AI feedback. A new benchmark, MMLongBench-Write, evaluates long-generation capabilities across professional and creative tasks, revealing that a 7B LongWriter-V model with IterDPO can outperform GPT-4o on long-output tasks. The work demonstrates that data distribution and segmented preference data are crucial for long, high-fidelity generation, and highlights the potential for scalable improvements in long-form multimodal generation, while noting limitations in dataset size and language coverage.

Abstract

Existing Large Vision-Language Models (LVLMs) can process inputs with context lengths up to 128k visual and text tokens, yet they struggle to generate coherent outputs beyond 1,000 words. We find that the primary limitation is the absence of long output examples during supervised fine-tuning (SFT). To tackle this issue, we introduce LongWriter-V-22k, a SFT dataset comprising 22,158 examples, each with multiple input images, an instruction, and corresponding outputs ranging from 0 to 10,000 words. Moreover, to achieve long outputs that maintain high-fidelity to the input images, we employ Direct Preference Optimization (DPO) to the SFT model. Given the high cost of collecting human feedback for lengthy outputs (e.g., 3,000 words), we propose IterDPO, which breaks long outputs into segments and uses iterative corrections to form preference pairs with the original outputs. Additionally, we develop MMLongBench-Write, a benchmark featuring six tasks to evaluate the long-generation capabilities of VLMs. Our 7B parameter model, trained with LongWriter-V-22k and IterDPO, achieves impressive performance on this benchmark, outperforming larger proprietary models like GPT-4o. Code and data: https://github.com/THU-KEG/LongWriter-V

Figures (11)

• Figure 1: Left: Six examples for each type of task in MMLongBench-Write. They are divided into two categories: professional writing and creative writing. The former requires professional knowledge, while the latter does not. Right: The joint distribution of the number of input images and the expected output length for data in both categories. Most data requires a 1000+ word output with given images, challenging the long-generation capabilities of VLMs.
• Figure 2: LongWriter-V-Ruler test across different output length requirements. The horizontal line show the overall upper bound for current VLMs.
• Figure 3: LongWriter-V-Ruler test for Qwen2-VL-7B-Instruct trained on 10k SFT data samples with different average output lengths.
• Figure 4: SFT and DPO data collection pipeline of LongWriter-V. The SFT data includes both single-image and multi-image input for long text output. The DPO data contains human revision over each paragraph of VLM's long output. We conduct iterative direct preference optimization to learn the fine-grained human feedback.
• Figure 5: Output length statistics of LongWrite-V-22k.