CoFFT: Chain of Foresight-Focus Thought for Visual Language Models

TL;DR

CoFFT addresses the challenge of visual input clutter in Vision-Language Models by introducing a training-free, iterative framework that jointly optimizes reasoning and visual focus. It cycles through Diverse Sample Generation, Dual Foresight Decoding, and Visual Focus Adjustment, guided by a Relative Attention mechanism to emphasize task-relevant regions. By integrating the first step of the best reasoning sample and adaptively cropping attention-rich regions, CoFFT achieves consistent gains across diverse visual reasoning benchmarks and model scales, with a favorable efficiency profile relative to search-based baselines. The approach demonstrates strong fine-grained understanding and scalable improvements without retraining, while acknowledging some limitations and the potential for further robustness improvements.

Abstract

Despite significant advances in Vision Language Models (VLMs), they remain constrained by the complexity and redundancy of visual input. When images contain large amounts of irrelevant information, VLMs are susceptible to interference, thus generating excessive task-irrelevant reasoning processes or even hallucinations. This limitation stems from their inability to discover and process the required regions during reasoning precisely. To address this limitation, we present the Chain of Foresight-Focus Thought (CoFFT), a novel training-free approach that enhances VLMs' visual reasoning by emulating human visual cognition. Each Foresight-Focus Thought consists of three stages: (1) Diverse Sample Generation: generates diverse reasoning samples to explore potential reasoning paths, where each sample contains several reasoning steps; (2) Dual Foresight Decoding: rigorously evaluates these samples based on both visual focus and reasoning progression, adding the first step of optimal sample to the reasoning process; (3) Visual Focus Adjustment: precisely adjust visual focus toward regions most beneficial for future reasoning, before returning to stage (1) to generate subsequent reasoning samples until reaching the final answer. These stages function iteratively, creating an interdependent cycle where reasoning guides visual focus and visual focus informs subsequent reasoning. Empirical results across multiple benchmarks using Qwen2.5-VL, InternVL-2.5, and Llava-Next demonstrate consistent performance improvements of 3.1-5.8% with controllable increasing computational overhead.

Figures (4)

• Figure 1: An example from the SeekWorld seekworld2025. (a) is the reasoning process of o3, and (b) is the reasoning process of o3 after human visual cognition. The correct answer is Jiangsu, China.
• Figure 2: The overall approach of CoFFT, where Dual Foresight Decoding and Visual Focus Adjustment will be introduced in detail later.
• Figure 3: The two primary components of CoFFT: (a) Dual Foresight Decoding, which evaluates different reasoning samples and selects the best sample from both visual focus and reasoning progression to enhance decision robustness, and (b) Visual Focus Adjustment, which adaptively modulates visual focus adjustment to reasoning-relevant regions for optimized information understanding.
• Figure 4: Illustrative cases demonstrating the reasoning process of CoFFT. (a), (b), and (c) show an example from Charxiv, SeekWorld-China, and MathVista benchmarks, respectively.