Comprehension Without Competence: Architectural Limits of LLMs in Symbolic Computation and Reasoning

TL;DR

The paper investigates why large language models exhibit strong explanations of symbolic procedures yet fail to reliably execute them. It formalizes the computational split-brain syndrome, driven by three architectural constraints: contextual averaging that impedes domain binding, FFNs that resort to pattern storage rather than exact symbolic computation, and a decoupled instruction-execution pathway caused by next-token prediction. Through embedding analyses, layer-wise computation tracking, and cross-domain experiments in arithmetic and relational reasoning, it shows that these constraints persist across model families and scales, and that compensatory strategies (self-scaffolding, tool delegation, and hybrid architectures) only shift the bottleneck without resolving it. The work argues for architectural innovations—metacognitive control, lifted representations, and grounded execution—to achieve robust symbolic reasoning, and provides testable predictions and implications for interpretability research. Overall, it reframes LLM capabilities as pattern completion strengths that do not generalize to principled computation, calling for fundamental design changes to reach true generalizable intelligence.

Abstract

Large Language Models (LLMs) display striking surface fluency yet systematically fail at tasks requiring symbolic reasoning, arithmetic accuracy, and logical consistency. This paper offers a structural diagnosis of such failures, revealing a persistent gap between \textit{comprehension} and \textit{competence}. Through controlled experiments and architectural analysis, we demonstrate that LLMs often articulate correct principles without reliably applying them--a failure rooted not in knowledge access, but in computational execution. We term this phenomenon the computational \textit{split-brain syndrome}, where instruction and action pathways are geometrically and functionally dissociated. This core limitation recurs across domains, from mathematical operations to relational inferences, and explains why model behavior remains brittle even under idealized prompting. We argue that LLMs function as powerful pattern completion engines, but lack the architectural scaffolding for principled, compositional reasoning. Our findings delineate the boundary of current LLM capabilities and motivate future models with metacognitive control, principle lifting, and structurally grounded execution. This diagnosis also clarifies why mechanistic interpretability findings may reflect training-specific pattern coordination rather than universal computational principles, and why the geometric separation between instruction and execution pathways suggests limitations in neural introspection and mechanistic analysis.

Figures (5)

• Figure 1: Numeric embeddings in LLaMA2-7B-chat fail to preserve symbolic geometry: (a) irregular cosine distances between digit tokens; (b) bifurcation at "10.0" due to tokenization boundaries; (c) large distances between equivalent representations like "5" and "five".
• Figure 2: Computational flow through transformer layers. When processing arithmetic operations (e.g., "43 × 78 = ?"), attention aggregates the operation context (operator and operands) while FFNs must generate the result. Due to causal masking, attention at the "=" position can only compute weighted averages of the embeddings it sees, creating a representation that cannot contain the novel direction for the result token.
• Figure 3: Experimental validation of hierarchical pattern assembly in arithmetic computation. Layer-by-layer convergence analysis for multiplication (left) and addition (right) shows progressive refinement from initial similarity of $\sim$0.07 to late-layer similarity of $\sim$0.48. Error bars represent standard error across 10 examples per operation. The consistent improvement patterns (0.404 for multiplication, 0.407 for addition) provide direct neural evidence for residual pattern fitting rather than algorithmic computation. Shaded regions indicate early (blue), middle (orange), and late (green) processing phases.
• Figure 4: t-SNE projection of instruction, symbolic execution, and word-form execution embeddings from LLaMA2-7B-chat, across 50 problems each in five arithmetic operations. Instructions (circles), symbolic expressions (squares), and word-form results (triangles) occupy geometrically distinct regions, despite describing the same operation. This illustrates that LLaMA2-7B-chat stores instructional knowledge and executional know-how in separate regions of its latent space.
• Figure 5: Cosine distances between cluster centroids for each operation-role pair, grouped by role: instructional texts (top block), execution strings (middle), and worded executions (bottom). Each centroid represents the average embedding of 50 samples per category, computed using LLaMA2-7B-chat. Clear geometric separation is observed between instructional and execution-related forms, with execution and worded execution clusters being closer to each other than either is to instruction. This provides quantitative support for the hypothesis that instruction-following and execution are learned as distinct representational pathways within the model's embedding space.

Theorems & Definitions (5)

• Definition 1
• Theorem 1: FFN Piecewise Linearity
• proof
• Theorem 2: Multiplication Impossibility
• proof