Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Milestones over Outcome: Unlocking Geometric Reasoning with Sub-Goal Verifiable Reward

Jianlong Chen, Daocheng Fu, Shengze Xu, Jiawei Chen, Yuan Feng, Yue Yang, Junchi Yan, Hongyuan Zha, Renqiu Xia

TL;DR

This work tackles the gap between reasoning quality and final-answer accuracy in multimodal geometric reasoning by introducing GeoGoal, a verifiable sub-goal benchmark, and SGVR, a sub-goal verifiable reward framework. GeoGoal converts proofs into dense, numeric subgoals and defines Skeleton Rate, Skeleton Completion, and Consistency Ratio to quantify reasoning fidelity. SGVR uses GRPO to optimize policies based on verifiable sub-goal signals, yielding substantial improvements in geometric reasoning and transfer to general math and reasoning tasks. The approach highlights the value of dense, verifiable supervision for robust, out-of-domain generalization in complex, diagram-rich reasoning tasks.

Abstract

Multimodal Large Language Models (MLLMs) struggle with complex geometric reasoning, largely because "black box" outcome-based supervision fails to distinguish between lucky guesses and rigorous deduction. To address this, we introduce a paradigm shift towards subgoal-level evaluation and learning. We first construct GeoGoal, a benchmark synthesized via a rigorous formal verification data engine, which converts abstract proofs into verifiable numeric subgoals. This structure reveals a critical divergence between reasoning quality and outcome accuracy. Leveraging this, we propose the Sub-Goal Verifiable Reward (SGVR) framework, which replaces sparse signals with dense rewards based on the Skeleton Rate. Experiments demonstrate that SGVR not only enhances geometric performance (+9.7%) but also exhibits strong generalization, transferring gains to general math (+8.0%) and other general reasoning tasks (+2.8%), demonstrating broad applicability across diverse domains.

Milestones over Outcome: Unlocking Geometric Reasoning with Sub-Goal Verifiable Reward

TL;DR

This work tackles the gap between reasoning quality and final-answer accuracy in multimodal geometric reasoning by introducing GeoGoal, a verifiable sub-goal benchmark, and SGVR, a sub-goal verifiable reward framework. GeoGoal converts proofs into dense, numeric subgoals and defines Skeleton Rate, Skeleton Completion, and Consistency Ratio to quantify reasoning fidelity. SGVR uses GRPO to optimize policies based on verifiable sub-goal signals, yielding substantial improvements in geometric reasoning and transfer to general math and reasoning tasks. The approach highlights the value of dense, verifiable supervision for robust, out-of-domain generalization in complex, diagram-rich reasoning tasks.

Abstract

Multimodal Large Language Models (MLLMs) struggle with complex geometric reasoning, largely because "black box" outcome-based supervision fails to distinguish between lucky guesses and rigorous deduction. To address this, we introduce a paradigm shift towards subgoal-level evaluation and learning. We first construct GeoGoal, a benchmark synthesized via a rigorous formal verification data engine, which converts abstract proofs into verifiable numeric subgoals. This structure reveals a critical divergence between reasoning quality and outcome accuracy. Leveraging this, we propose the Sub-Goal Verifiable Reward (SGVR) framework, which replaces sparse signals with dense rewards based on the Skeleton Rate. Experiments demonstrate that SGVR not only enhances geometric performance (+9.7%) but also exhibits strong generalization, transferring gains to general math (+8.0%) and other general reasoning tasks (+2.8%), demonstrating broad applicability across diverse domains.
Paper Structure (66 sections, 21 equations, 8 figures, 7 tables)

This paper contains 66 sections, 21 equations, 8 figures, 7 tables.

Table of Contents

  1. Introduction
  2. GeoGoal: A Verifiable Benchmark for Sub-Goal Reasoning
  3. Construction Pipeline
  4. Step 1: Data Engine: Formal Skeleton Generation
  5. Step 2: Decomposition and Sub-goal Mapping
  6. Step 3: Reorganization and Sequence Construction
  7. Sub-goal Evaluation Metrics
  8. Dataset Characteristics
  9. Benchmark Evaluation of Existing Models
  10. Sub-goal baselines.
  11. How aligned are outcome accuracy and step-wise consistency?
  12. What failure modes are exposed by the relationship between SR and SC?
  13. Sub-Goal Verifiable Reward
  14. RL Formulation
  15. Sub-goal Reward Signal.
  16. ...and 51 more sections

Figures (8)

  • Figure 1: Our main goal: Decomposing the "black box" of complex geometric reasoning into a verifiable chain of fine-grained intermediate milestones.
  • Figure 2: Overall framework: (Top) Benchmark construction: Formally verified skeletons from TrustGeoGen trustgeogen2025 are decomposed into numeric sub-goals to enable subgoal-level metrics. (Bottom) SGVR training: The model generates structured traces; predicted sub-goals are verified against ground truth to formulate dense rewards for policy optimization via GRPO.
  • Figure 3: Skeleton Completion (SC) versus Final Answer accuracy on our benchmark. Each point denotes a multimodal model. The light blue background indicates SC < FA. The closer the model is to the line where SC = FA, the more rigorous its reasoning logic is.
  • Figure 4: Skeleton Completion (SC) v.s. Skeleton Rate (SR) on our benchmark. Points are color-coded by the Consistency Ratio (CR), revealing distinct trade-offs between step-wise correctness and end-to-end consistency.
  • Figure 5: Performance comparison of our trained models against baselines on final answer accuracy. Solid bars represent baseline performance; patterned sections indicate improvements from our training. Our method achieves consistent gains across model sizes and task domains, with particularly strong improvements on some datasets (GeoPQA-Test: +6.4% for 7B, +7.2% for 3B; AMC: +12.1% for 7B, +6.0% for 3B; LiveBench-Reasoning: +3.5% for 7B, +7.0% for 3B).
  • ...and 3 more figures