Bridging Writing Manner Gap in Visual Instruction Tuning by Creating LLM-aligned Instructions

TL;DR

This work identifies a writing manner gap between visual instruction data and the inner LLMs used in large multi-modal models, showing that misalignment can degrade both the base LLM and the LMM. It introduces a straightforward, self-contained approach that uses the inner LLM to rewrite and then review soft-format visual instructions, aligning their writing style to the LLM without changing semantics. Across LLaVA-7B and QwenVL, this LLM-aligned training regime yields reduced hallucinations and broad improvements on 12–15 benchmarks, including visual and textual tasks, with ablations confirming the value of rewriting and review stages. The method relies solely on internal data and models, offering a practical path to more robust, instruction-following LMMs and highlighting the importance of matching discourse style between training data and model priors.

Abstract

In the realm of Large Multi-modal Models (LMMs), the instruction quality during the visual instruction tuning stage significantly influences the performance of modality alignment. In this paper, we assess the instruction quality from a unique perspective termed \textbf{Writing Manner}, which encompasses the selection of vocabulary, grammar and sentence structure to convey specific semantics. We argue that there exists a substantial writing manner gap between the visual instructions and the base Large Language Models (LLMs) within LMMs. This gap forces the pre-trained base LLMs to deviate from their original writing styles, leading to capability degradation of both base LLMs and LMMs. To bridge the writing manner gap while preserving the original semantics, we propose directly leveraging the base LLM to align the writing manner of soft-format visual instructions with that of the base LLM itself, resulting in novel LLM-aligned instructions. The manual writing manner evaluation results demonstrate that our approach successfully minimizes the writing manner gap. By utilizing LLM-aligned instructions, the baseline models LLaVA-7B and QwenVL demonstrate enhanced resistance to hallucinations and non-trivial comprehensive improvements across all $15$ visual and language benchmarks.

Figures (8)

• Figure 1: The instance of word-level and sentence-level writing manner gap.
• Figure 2: The brief diagram of our LLM-aligned trainset construction.
• Figure 3: One positive instance generation process for the illustration of writing manner alignment. The important information are highlighted in non-black colors for clarity.
• Figure 4: Positive LLM-aligned samples in different revision types.
• Figure 5: Unqualified samples filtered out during the review stage.