MathCanvas: Intrinsic Visual Chain-of-Thought for Multimodal Mathematical Reasoning

TL;DR

MathCanvas tackles the gap in mathematical reasoning by enabling intrinsic Visual Chain-of-Thought within a unified Large Multimodal Model. It introduces two large data pipelines (MathCanvas-Edit/MathCanvas-Imagen) for diagram generation/editing and a third (MathCanvas-Instruct) for interleaved visual–text reasoning, plus a 3K-problem MathCanvas-Bench to rigorously evaluate progress. The two-stage training (Visual Manipulation followed by Strategic Visual-Aided Reasoning) yields BAGEL-Canvas, which shows substantial improvements over strong baselines and generalizes to other public math benchmarks. This work provides a practical framework, datasets, and benchmarks to advance human-like visual-aided reasoning in multimodal models.

Abstract

While Large Language Models (LLMs) have excelled in textual reasoning, they struggle with mathematical domains like geometry that intrinsically rely on visual aids. Existing approaches to Visual Chain-of-Thought (VCoT) are often limited by rigid external tools or fail to generate the high-fidelity, strategically-timed diagrams necessary for complex problem-solving. To bridge this gap, we introduce MathCanvas, a comprehensive framework designed to endow unified Large Multimodal Models (LMMs) with intrinsic VCoT capabilities for mathematics. Our approach consists of two phases. First, a Visual Manipulation stage pre-trains the model on a novel 15.2M-pair corpus, comprising 10M caption-to-diagram pairs (MathCanvas-Imagen) and 5.2M step-by-step editing trajectories (MathCanvas-Edit), to master diagram generation and editing. Second, a Strategic Visual-Aided Reasoning stage fine-tunes the model on MathCanvas-Instruct, a new 219K-example dataset of interleaved visual-textual reasoning paths, teaching it when and how to leverage visual aids. To facilitate rigorous evaluation, we introduce MathCanvas-Bench, a challenging benchmark with 3K problems that require models to produce interleaved visual-textual solutions. Our model, BAGEL-Canvas, trained under this framework, achieves an 86% relative improvement over strong LMM baselines on MathCanvas-Bench, demonstrating excellent generalization to other public math benchmarks. Our work provides a complete toolkit-framework, datasets, and benchmark-to unlock complex, human-like visual-aided reasoning in LMMs. Project Page: https://mathcanvas.github.io/

Figures (13)

• Figure 1: MathCanvas demonstrates the first successful application of intrinsic Visual Chain-of-Thought (VCoT) for complex mathematical reasoning. Prior attempts fail by generating incorrect (BAGEL-Zebra-CoT) or strategically poor (Nano-Banana) visuals, leading to wrong solutions. In contrast, MathCanvas correctly generates an intermediate visual step that unlocks a simpler, elegant solution path.
• Figure 2: The curation pipeline for the MathCanvas-Edit and MathCanvas-Imagen dataset.
• Figure 3: Statistical analysis of the MathCanvas-Bench test set. Left: Knowledge types distribution. Middle: Distribution of questions and solutions containing varying numbers of images. Right: Text length distribution of questions and solutions (measured in text tokens).
• Figure 4: The two-stage training recipe for MathCanvas. (Left) Stage I: Visual Manipulation. The model's Generation Expert is pretrained on our MathCanvas-Edit and MathCanvas-Imagen corpora to instill foundational diagram generation and editing skills. (Right) Stage II: Strategic Visual-Aided Reasoning. The entire model is then fine-tuned on MathCanvas-Instruct to learn the strategic interleaving of visual actions with textual reasoning.
• Figure 5: Distribution of knowledge type of MathCanvas-Instruct dataset.