Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond Symbolic Solving: Multi Chain-of-Thought Voting for Geometric Reasoning in Large Language Models

Md. Abu Bakor Siddique, Shahrin Hossain, Sadman Ahmed Siam, Syed Rifat Raiyan, Hasan Mahmud, Md Kamrul Hasan

Abstract

Geometric Problem Solving (GPS) remains at the heart of enhancing mathematical reasoning in large language models because it requires the combination of diagrammatic understanding, symbolic manipulation and logical inference. In existing literature, researchers have chiefly focused on synchronising the diagram descriptions with text literals and solving the problem. In this vein, they have either taken a neural, symbolic or neuro-symbolic approach. But this solves only the first two of the requirements, namely diagrammatic understanding and symbolic manipulation, while leaving logical inference underdeveloped. The logical inference is often limited to one chain-of-thought (CoT). To address this weakness in hitherto existing models, this paper proposes MARS-GPS, that generates multiple parallel reasoning rollouts augmented with Python code execution for numerical verification, ranks them using token-level entropy as a confidence signal, and aggregates answers through a multi-stage voting and self-verification pipeline. Empirical results show that MARS-GPS with 8 parallel rollouts achieves 88.8% on Geometry3K, a nearly +11% improvement over the prior state-of-the-art, with accuracy scaling consistently as the number of rollouts increases from 1 to 16 (+6.0% on ablation subset). We provide our code and data in an anonymous repository: https://anonymous.4open.science/r/MARS-GPS-DE55.

Beyond Symbolic Solving: Multi Chain-of-Thought Voting for Geometric Reasoning in Large Language Models

Abstract

Geometric Problem Solving (GPS) remains at the heart of enhancing mathematical reasoning in large language models because it requires the combination of diagrammatic understanding, symbolic manipulation and logical inference. In existing literature, researchers have chiefly focused on synchronising the diagram descriptions with text literals and solving the problem. In this vein, they have either taken a neural, symbolic or neuro-symbolic approach. But this solves only the first two of the requirements, namely diagrammatic understanding and symbolic manipulation, while leaving logical inference underdeveloped. The logical inference is often limited to one chain-of-thought (CoT). To address this weakness in hitherto existing models, this paper proposes MARS-GPS, that generates multiple parallel reasoning rollouts augmented with Python code execution for numerical verification, ranks them using token-level entropy as a confidence signal, and aggregates answers through a multi-stage voting and self-verification pipeline. Empirical results show that MARS-GPS with 8 parallel rollouts achieves 88.8% on Geometry3K, a nearly +11% improvement over the prior state-of-the-art, with accuracy scaling consistently as the number of rollouts increases from 1 to 16 (+6.0% on ablation subset). We provide our code and data in an anonymous repository: https://anonymous.4open.science/r/MARS-GPS-DE55.

Paper Structure

This paper contains 42 sections, 8 equations, 3 figures, 7 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Problem Formulation
  5. Diagram Understanding and Parsing
  6. Text Parser.
  7. Diagram Parser.
  8. Inference-Time Reasoning Strategy
  9. Parallel Rollout Sampling
  10. Confidence Estimation via Token Entropy
  11. Code-Augmented Reasoning
  12. Verification and Self-Consistency
  13. Step 1: Early Consensus.
  14. Step 2: Hard Accept.
  15. Step 3: Candidate Selection.
  16. ...and 27 more sections

Figures (3)

  • Figure 1: Overview of the Multi-path Aggregated Reasoning System for Geometry Problem Solving (MARS-GPS) pipeline. Left: the problem parsing stage takes the diagram and problem text as input and produces a unified formal context $\mathcal{F}^*$ via PGDPNet and a rule-based semantic parser. Right: the inference-time ensemble reasoning stage samples $k$ parallel rollouts from $f_\theta$, each augmented with a Python sandbox $\mathcal{E}$ for numerical computation. The rollout outputs feed into the answer aggregation pipeline, which applies majority voting, entropy-ranked self-verification, and a weighted fallback to produce the final answer $a^*$.
  • Figure 2: Ablation studies on a subset of Geometry3K. (a) Entropy-weighted voting outperforms majority voting and entropy sorting by 2.0 percentage points. (b) Removing self-verification causes the largest single-component accuracy drop ($-4.5$pp), followed by code augmentation ($-2.5$pp).
  • Figure 3: Accuracy vs. number of CoT samples.