CASHEW: Stabilizing Multimodal Reasoning via Iterative Trajectory Aggregation

TL;DR

Cashew introduces an inference-time framework that stabilizes multimodal reasoning by iteratively aggregating a population of candidate reasoning trajectories with explicit grounding in visual evidence. Cashew-RL extends this idea into a learnable policy via GSPO, trained with a composite reward that balances accuracy, grounded evidence, and efficient reasoning. Across 13 image and video benchmarks, the methods yield substantial gains and outperform existing test-time scaling baselines, demonstrating improved reasoning stability and grounding. The approach has practical impact for robust multimodal understanding and lays groundwork for further integration of perception and aggregation.

Abstract

Vision-language models achieve strong performance across a wide range of multimodal understanding and reasoning tasks, yet their multi-step reasoning remains unstable. Repeated sampling over the same input often produces divergent reasoning trajectories and inconsistent final predictions. To address this, we introduce two complementary approaches inspired by test-time scaling: (1) CASHEW, an inference-time framework that stabilizes reasoning by iteratively aggregating multiple candidate trajectories into higher-quality reasoning traces, with explicit visual verification filtering hallucinated steps and grounding reasoning in visual evidence, and (2) CASHEW-RL, a learned variant that internalizes this aggregation behavior within a single model. CASHEW-RL is trained using Group Sequence Policy Optimization (GSPO) with a composite reward that encourages correct answers grounded in minimal yet sufficient visual evidence, while adaptively allocating reasoning effort based on task difficulty. This training objective enables robust self-aggregation at inference. Extensive experiments on 13 image understanding, video understanding, and video reasoning benchmarks show significant performance improvements, including gains of up to +23.6 percentage points on ScienceQA and +8.1 percentage points on EgoSchema.

Figures (7)

• Figure 1: Cashew enables robust reasoning through visually grounded iterative aggregation. Unlike standard vision-language models that rely on single-path reasoning and are prone to hallucinations, Cashew aggregates multiple reasoning trajectories with explicit visual verification. Cashew-RL further internalizes this aggregation behavior via reinforcement learning.
• Figure 2: Overview of the Cashew and Cashew-RL frameworks.Left:Cashew performs test-time iterative aggregation by generating a population of candidate trajectories from a frozen VLM, verifying object-level claims with Grounding DINO, and synthesizing subsets into refined trajectories over multiple iterations to produce a consolidated trajectory $\tau^{*}$. Right:Cashew-RL extends this framework via post-training with GSPO, teaching the VLM to internally aggregate multiple candidate trajectories into high-quality, visually grounded reasoning traces.
• Figure 3: Performance across Cashew population ($N$) for different values of $T$. All results with fixed $K=4$.
• Figure 4: Prompt templates for different stages of Cashew, including population initialization, grounded aggregation, and final aggregation.
• Figure 5: Prompt used for supervised fine-tuning (SFT) in Cashew-RL. The prompt enforces a structured output format consisting of a reasoning chain, a list of visual keys, and a final answer, enabling the model to learn aggregation and grounding behaviors from demonstrations.