MMFineReason: Closing the Multimodal Reasoning Gap via Open Data-Centric Methods

TL;DR

MMFineReason presents an open, data-centric approach to closing the multimodal reasoning gap by constructing a large, high-quality reasoning dataset (MMFineReason) with 1.77–1.8 million samples and 5.1–8.8 billion tokens. The authors implement a three-stage pipeline—data collection/processing, reasoning Distillation via a strong teacher (Qwen3-VL-235B-A22B-Thinking), and rigorous data selection with difficulty-aware filtering—yielding both a rich full dataset (MMFineReason-1.8M) and compact, challenging subsets (MMFineReason-123K and MMFineReason-586K). Fine-tuning Qwen3-VL-Instruct on MMFineReason yields model families (2B/4B/8B) that achieve state-of-the-art results for their sizes, with notable parameter efficiency (e.g., MMFineReason-4B beating 8B-Thinking; MMFineReason-8B approaching 32B-Thinking). The work demonstrates a synergistic effect where reasoning-focused data improves general capabilities and shows extreme data efficiency—a small, high-quality subset can match full-dataset performance—while releasing data and models to spur reproducible, data-centric progress in open-source multimodal reasoning.

Abstract

Recent advances in Vision Language Models (VLMs) have driven significant progress in visual reasoning. However, open-source VLMs still lag behind proprietary systems, largely due to the lack of high-quality reasoning data. Existing datasets offer limited coverage of challenging domains such as STEM diagrams and visual puzzles, and lack consistent, long-form Chain-of-Thought (CoT) annotations essential for eliciting strong reasoning capabilities. To bridge this gap, we introduce MMFineReason, a large-scale multimodal reasoning dataset comprising 1.8M samples and 5.1B solution tokens, featuring high-quality reasoning annotations distilled from Qwen3-VL-235B-A22B-Thinking. The dataset is established via a systematic three-stage pipeline: (1) large-scale data collection and standardization, (2) CoT rationale generation, and (3) comprehensive selection based on reasoning quality and difficulty awareness. The resulting dataset spans STEM problems, visual puzzles, games, and complex diagrams, with each sample annotated with visually grounded reasoning traces. We fine-tune Qwen3-VL-Instruct on MMFineReason to develop MMFineReason-2B/4B/8B versions. Our models establish new state-of-the-art results for their size class. Notably, MMFineReason-4B succesfully surpasses Qwen3-VL-8B-Thinking, and MMFineReason-8B even outperforms Qwen3-VL-30B-A3B-Thinking while approaching Qwen3-VL-32B-Thinking, demonstrating remarkable parameter efficiency. Crucially, we uncover a "less is more" phenomenon via our difficulty-aware filtering strategy: a subset of just 7\% (123K samples) achieves performance comparable to the full dataset. Notably, we reveal a synergistic effect where reasoning-oriented data composition simultaneously boosts general capabilities.

Figures (9)

• Figure 1: Average score across mathematical reasoning and multimodal understanding benchmarks. MMFineReason-2B/4B/8B demonstrates strong performance relative to thinking models with significantly more parameters.
• Figure 2: MMFineReason data pipeline and the two-stage training. Illustrating data construction, annotation, selection, mixing, and model training (SFT and RL) in our framework.
• Figure 3: Consistency analysis across visual instruction tuning datasets. The chart displays the ratio of samples where the predictions generated by Qwen3-VL-235-A22B-Thinking align with the original ground truth answers ("Consistent") versus cases of disagreement ("Inconsistent").
• Figure 4: Dataset composition of MMFineReason-1.8M. The outer ring represents the proportion of major categories, and the inner ring shows the distribution of specific datasets. Note: To ensure the visual legibility of diverse domains, the segment sizes in this chart are scaled by the square root of sample counts ($\sqrt{N}$). The actual data distribution is dominated by Mathematics (79.4%), followed by Science (13.8%), Puzzle/Game (4.6%), and General/OCR (2.2%).
• Figure 5: Pass rate distribution across sub-datasets. Datasets are sorted by descending mean pass rate (easiest to hardest). The bubble chart encodes sample proportion via size and color intensity, overlaid with a mean pass rate trendline.