HopChain: Multi-Hop Data Synthesis for Generalizable Vision-Language Reasoning

Abstract

Vision-language models (VLMs) show strong multimodal capabilities but still struggle with fine-grained vision-language reasoning. We find that long chain-of-thought (CoT) reasoning exposes diverse failure modes, including perception, reasoning, knowledge, and hallucination errors, which can compound across intermediate steps. However, most existing vision-language data used for reinforcement learning with verifiable rewards (RLVR) does not involve complex reasoning chains that rely on visual evidence throughout, leaving these weaknesses largely unexposed. We therefore propose HopChain, a scalable framework for synthesizing multi-hop vision-language reasoning data for RLVR training of VLMs. Each synthesized multi-hop query forms a logically dependent chain of instance-grounded hops, where earlier hops establish the instances, sets, or conditions needed for later hops, while the final answer remains a specific, unambiguous number suitable for verifiable rewards. We train Qwen3.5-35B-A3B and Qwen3.5-397B-A17B under two RLVR settings: the original data alone, and the original data plus HopChain's multi-hop data, and compare them across 24 benchmarks spanning STEM and Puzzle, General VQA, Text Recognition and Document Understanding, and Video Understanding. Although this multi-hop data is not synthesized for any specific benchmark, it improves 20 of 24 benchmarks on both models, indicating broad and generalizable gains. Consistently, replacing full chained queries with half-multi-hop or single-hop variants reduces the average score across five representative benchmarks from 70.4 to 66.7 and 64.3, respectively. Notably, multi-hop gains peak in long-CoT vision-language reasoning, exceeding 50 points in the ultra-long-CoT regime. These experiments establish HopChain as an effective, scalable framework for synthesizing multi-hop data that improves generalizable vision-language reasoning.

Figures (8)

• Figure 1: Overview of HopChain and the motivation for multi-hop vision-language reasoning data. (a) HopChain synthesizes multi-hop data through four stages: category identification, instance segmentation, multi-hop query generation, and ground-truth annotation with difficulty calibration. (b) Typical vision-language training data often does not require complex reasoning chains that rely on visual evidence throughout; on hard problems, minor mistakes introduced at intermediate long-CoT steps can compound into final errors. (c) In contrast, the multi-hop data synthesized by HopChain forms logically dependent hop chains in which later hops depend on instances, sets, or conditions established by earlier hops, while nearly every hop requires fresh visual re-grounding. This training signal encourages continual visual evidence seeking during long-CoT reasoning, improving robustness and reducing compounding errors.
• Figure 2: Error-type distributions before and after adding multi-hop data in RLVR. Subfigure (a) shows the error-type distribution of RLVR w/o Multi-Hop on the benchmarks in Tables \ref{['tab:exp-35b']} and \ref{['tab:exp-397b']}. Subfigure (b) shows the distribution of baseline error types mitigated after adding multi-hop data (i.e., cases corrected by RLVR w/ Multi-Hop). In (a), errors are diverse, with perception and reasoning errors as the dominant categories. In (b), the distribution remains similar to (a), indicating that multi-hop data mitigates failures in a generalizable way rather than improving only a narrow error type. \ref{['fig:qualitative_examples']} provides representative perception reasoning failures, together with responses corrected after adding multi-hop data.
• Figure 3: Qualitative examples of unreliable visual perception in long CoT reasoning. We show failure cases from the benchmarks in \ref{['tab:exp-35b', 'tab:exp-397b']} using "RVLR w/o Multi-Hop" (baseline), alongside correct answers from "RVLR w/ Multi-Hop" (ours). For brevity, unimportant parts of the long CoT are omitted with $[\dots]$. The baseline error is highlighted in red, and the failure reason is given in the error analysis.
• Figure 4: Examples of synthesized multi-hop data. In each query, purple-highlighted text denotes the instance chain, meaning that the instance required at the current hop can only be identified from instances established in earlier hops. In the corresponding image, we mark the instance regions involved in each hop with colored rectangles, and the rectangle colors are aligned with the text colors of the corresponding hop-wise ground-truth answers.
• Figure 5: Benchmark-level comparison on Qwen3.5-35B-A3B under three training-query settings. RLVR w/ Single Hop simplifies each multi-hop training query to only its final hop, RLVR w/ Half-Multi-Hop removes the first half of the hops and keeps only the latter half, and RLVR w/ Multi-Hop uses the full multi-hop training queries. We evaluate the resulting models on five representative benchmarks, namely MathVision, MMMU Pro, RealWorldQA, ERQA, and VideoMMMU, and plot the benchmark score for each setting. Across all five benchmarks, the full multi-hop setting performs best, showing that preserving the complete multi-hop structure during training is more effective than shortening the query chain.