Towards Self-Refinement of Vision-Language Models with Triangular Consistency

TL;DR

The paper investigates self-refinement of Vision-Language Models by leveraging a Triangular Consistency principle to generate and filter synthetic image-question-answer data from unlabeled images. The proposed Self-Refinement Framework (SRF) comprises three stages: enhance instruction generation via multitask fine-tuning, Triangular Consistency-based filtering, and iterative instruction-tuning on filtered data, enabling multi-round improvement without external supervision. A theoretical causal analysis explains why additional unlabeled data can improve $P(X|Y)$ through refinement of $P(Y)$ under an Additive Noise Model, and experiments using the LLaVA-1.5 baseline show modest yet consistent gains across eight vision-language benchmarks, with notable improvements in real-world visual dialogue tasks. The approach generalizes across architectures and data scales, and code is released.

Abstract

Vision-Language Models (VLMs) integrate visual knowledge with the analytical capabilities of Large Language Models (LLMs) through supervised visual instruction tuning, using image-question-answer triplets. However, the potential of VLMs trained without supervised instruction remains largely unexplored. This study validates that VLMs possess inherent self-refinement capabilities, enabling them to generate high-quality supervised data without external inputs and thereby learn autonomously. Specifically, to stimulate the self-refinement ability of VLMs, we propose a self-refinement framework based on a Triangular Consistency principle: within the image-query-answer triangle, any masked elements should be consistently and accurately reconstructed. The framework involves three steps: (1) We enable the instruction generation ability of VLMs by adding multi-task instruction tuning like image$\rightarrow$question-answer or image-answer$\rightarrow$question. (2) We generate image-query-answer triplets from unlabeled images and use the Triangular Consistency principle for filtering. (3) The model is further updated using the filtered synthetic data. To investigate the underlying mechanisms behind this self-refinement capability, we conduct a theoretical analysis from a causal perspective. Using the widely recognized LLaVA-1.5 as our baseline, our experiments reveal that the model can autonomously achieve consistent, though deliberately modest, improvements across multiple benchmarks without any external supervision, such as human annotations or environmental feedback. We expect that the insights of this study on the self-refinement ability of VLMs can inspire future research on the learning mechanism of VLMs. Code is available at https://github.com/dengyl20/SRF-LLaVA-1.5.

Figures (10)

• Figure 1: An overview of the Self-Refinement framework. The Self-Refinement framework comprises three stages. First, we design multi-task fine-tuning to enhance the model's instruction generation capabilities. Next, we apply the Triangular Consistency principle to filter high-quality instructions based on their scores. Finally, the selected data is used to refine the model. By using the updated model and synthetic data as the starting point for the next iteration, the framework naturally supports an iterative process.
• Figure 2: Causal modeling and understanding of involved data-generating processes. Panel (a) presents causal relations among the language $X$, the semantic concept $S$, and the image $Y$. Panel (b) summarizes the input-output relation of VLMs.
• Figure 3: Two comparison examples between our SRF-LLaVA-1.5 and LLaVA-1.5 in visual chat. Red highlights indicate factual errors or irrelevant content in the response, while green highlights emphasize image details critical for providing an accurate answer.
• Figure 4: Ablation study on specific synthetic data types. The numbers denote the percentage increase compared to LLaVA-1.5.
• Figure :