Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

X-RAY: Mapping LLM Reasoning Capability via Formalized and Calibrated Probes

Gao Tianxi, Cai Yufan, Yuan Yusi, Dong Jin Song

TL;DR

X-RAY, an explainable reasoning analysis system that maps the LLM reasoning capability using calibrated, formally verified probes, and reveals a systematic asymmetry in LLM reasoning: models are relatively robust to constraint refinement, but degrade sharply under solution-space restructuring, where modifications alter the underlying structural form of the solution manifold.

Abstract

Large language models (LLMs) achieve promising performance, yet their ability to reason remains poorly understood. Existing evaluations largely emphasize task-level accuracy, often conflating pattern matching with reasoning capability. We present X-RAY, an explainable reasoning analysis system that maps the LLM reasoning capability using calibrated, formally verified probes. We model reasoning capability as a function of extractable \textit{structure}, operationalized through formal properties such as constraint interaction, reasoning depth, and solution-space geometry. X-Ray generates probes via formal tools with controlled structural variations, enabling precise isolation of incremental structural information through formal calibration and verification. We evaluate state-of-the-art LLMs on problems ranging from junior-level to advanced in mathematics, physics, and chemistry. Our analysis reveals a systematic asymmetry in LLM reasoning: models are relatively robust to constraint refinement, where additional conditions shrink an existing solution space, but degrade sharply under solution-space restructuring, where modifications alter the underlying structural form of the solution manifold. Moreover, calibrated formal probes differentiate models that appear indistinguishable on standard benchmarks and reveal failure modes that are structurally interpretable rather than opaque. Beyond evaluation, our framework is contamination-free and supports the training and testing of reasoning models.

X-RAY: Mapping LLM Reasoning Capability via Formalized and Calibrated Probes

TL;DR

X-RAY, an explainable reasoning analysis system that maps the LLM reasoning capability using calibrated, formally verified probes, and reveals a systematic asymmetry in LLM reasoning: models are relatively robust to constraint refinement, but degrade sharply under solution-space restructuring, where modifications alter the underlying structural form of the solution manifold.

Abstract

Large language models (LLMs) achieve promising performance, yet their ability to reason remains poorly understood. Existing evaluations largely emphasize task-level accuracy, often conflating pattern matching with reasoning capability. We present X-RAY, an explainable reasoning analysis system that maps the LLM reasoning capability using calibrated, formally verified probes. We model reasoning capability as a function of extractable \textit{structure}, operationalized through formal properties such as constraint interaction, reasoning depth, and solution-space geometry. X-Ray generates probes via formal tools with controlled structural variations, enabling precise isolation of incremental structural information through formal calibration and verification. We evaluate state-of-the-art LLMs on problems ranging from junior-level to advanced in mathematics, physics, and chemistry. Our analysis reveals a systematic asymmetry in LLM reasoning: models are relatively robust to constraint refinement, where additional conditions shrink an existing solution space, but degrade sharply under solution-space restructuring, where modifications alter the underlying structural form of the solution manifold. Moreover, calibrated formal probes differentiate models that appear indistinguishable on standard benchmarks and reveal failure modes that are structurally interpretable rather than opaque. Beyond evaluation, our framework is contamination-free and supports the training and testing of reasoning models.
Paper Structure (62 sections, 6 equations, 6 figures, 5 tables)

This paper contains 62 sections, 6 equations, 6 figures, 5 tables.

Table of Contents

  1. Introduction
  2. From pattern matching to structured reasoning.
  3. Why formal verification.
  4. Reasoning capability of LLMs.
  5. Our approach.
  6. Contributions.
  7. Motivating Examples
  8. Postage Stamp Problem
  9. Autoformalization.
  10. Formal probe calibration.
  11. Probing results.
  12. N-Primable Numbers
  13. Autoformalization.
  14. Formal probe calibration.
  15. Probing results.
  16. ...and 47 more sections

Figures (6)

  • Figure 1: Capability of GPT-4o and o4-mini measured on stamp-coverage probes as a solution space restructuring example.
  • Figure 2: Capability of GPT-4o and o4-mini measured on N-primable problems as a constraint refinement example.
  • Figure 3: GSM8K Dataset: Heatmap visualization of metric correlations in model success rates.
  • Figure 4: MATH Dataset: Heatmap visualization of metric correlations in model success rates.
  • Figure 5: Physics Dataset: Heatmap visualization of metric correlations in model success rates.
  • ...and 1 more figures