On the Reasoning Capacity of AI Models and How to Quantify It

TL;DR

This paper interrogates the claim that modern AI models perform genuine reasoning by introducing a phenomenological framework that separates observed behavior into memorization, reasoning, and guessing. By leveraging positional bias as a controlled perturbation, it couples a Probabilistic Mixture Model (PMM) with an Information-Theoretic Consistency (ITC) analysis to quantify how strategy selection, confidence, and accuracy interrelate. Empirical results on GPQA with a GPT-4o-mini model reveal that many high-accuracy outcomes stem from memorization and pattern-matching rather than true deduction, and that accuracy alone fails to capture the underlying cognitive mix. The work provides quantitative criteria and a geometric phase-space view for evaluating model reliability in real-world deployments, offering a principled path toward more robust benchmarks and responsible AI use.

Abstract

Recent advances in Large Language Models (LLMs) have intensified the debate surrounding the fundamental nature of their reasoning capabilities. While achieving high performance on benchmarks such as GPQA and MMLU, these models exhibit limitations in more complex reasoning tasks, highlighting the need for more rigorous evaluation methodologies. We propose a novel phenomenological approach that goes beyond traditional accuracy metrics to probe the underlying mechanisms of model behavior, establishing a framework that could broadly impact how we analyze and understand AI systems. Using positional bias in multiple-choice reasoning tasks as a case study, we demonstrate how systematic perturbations can reveal fundamental aspects of model decision-making. To analyze these behaviors, we develop two complementary phenomenological models: a Probabilistic Mixture Model (PMM) that decomposes model responses into reasoning, memorization, and guessing components and an Information-Theoretic Consistency (ITC) analysis that quantifies the relationship between model confidence and strategy selection. Through controlled experiments on reasoning benchmarks, we show that true reasoning remains challenging for current models, with apparent success often relying on sophisticated combinations of memorization and pattern matching rather than genuine logical deduction. More fundamentally, we demonstrate that accuracy alone often overstates a model's reasoning abilities, as model behavior can be characterized through underlying mechanisms in the phase space of cognitive strategies, revealing how models dynamically balance different approaches when responding to queries. This framework enables quantitative criteria for real-world deployments, allowing applications to specify reliability thresholds based on strategy distributions rather than aggregate performance metrics.

Figures (12)

• Figure 1: Phenomenological decomposition of language model behavior through strategy space, illustrating the mappings between input queries and output responses.
• Figure 2: Theoretical entropy-accuracy frontiers for multiple-choice questions with varying numbers of options ($k$). The black curve represents the four-option case ($k=4$ where questions have multiple choices such A,B,C,D) relevant to GPQA, where maximum entropy ($H_{\text{max}} = 2$ bits) occurs at random guessing accuracy ($A = \frac{1}{k}=0.25$). The dotted blue line shows the relationship $A = \frac{1}{k}$, intersecting each frontier at its maximum entropy point. Higher values of $k$ result in larger maximum entropy due to increased uncertainty in the option space. The frontiers demonstrate how prediction uncertainty (entropy) varies with accuracy, reaching zero at perfect accuracy ($A = 1$) for all $k$.
• Figure 3: Wrong answer distributions conditional on correct answer position. Each subplot shows the probability mass of incorrect choices when the correct answer was at positions A, B, C, or D respectively. Position D exhibits markedly lower selection probability ( 0.1) as an incorrect choice compared to other positions ( 0.2-0.25), suggesting systematic positional bias. Probability mass is calculated as frequency of selection normalized by total number of trials.
• Figure 4: Evolution of position-dependent model behavior under controlled randomization protocols. Upper panels track mean accuracies as functions of randomization parameter $\theta$, while lower panels show corresponding variance dynamics. The protocols differ fundamentally: inclusive randomization (a) permits correct answers at any position, while exclusive randomization (b) systematically prevents correct answers at original positions, enabling isolation of position-specific effects. This controlled comparison reveals both universal features (convergence to random chance) and protocol-specific dynamics (critical points, variance patterns) in the model's position-dependent behavior.
• Figure 5: Difference $\Delta\mu_o(\theta)$ between inclusive and exclusive randomization accuracies. Position D shows strongest bias with monotonically decreasing difference reaching -0.04 at $\theta = 1$, while positions A, B, C maintain near-zero differences ($|\Delta\mu_o(\theta)| < 0.01$), suggesting more uniform treatment.