Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Geometric Price of Discrete Logic: Context-driven Manifold Dynamics of Number Representations

Long Zhang, Dai-jun Lin, Wei-neng Chen

Abstract

Large language models (LLMs) generalize smoothly across continuous semantic spaces, yet strict logical reasoning demands the formation of discrete decision boundaries. Prevailing theories relying on linear isometric projections fail to resolve this fundamental tension. In this work, we argue that task context operates as a non-isometric dynamical operator that enforces a necessary "topological distortion." By applying Gram-Schmidt decomposition to residual-stream activations , we reveal a dual-modulation mechanism driving this process: a class-agnostic topological preservation that anchors global structure to prevent semantic collapse, and a specific algebraic divergence that directionally tears apart cross-class concepts to forge logical boundaries. We validate this geometric evolution across a gradient of tasks, from simple mapping to complex primality testing. Crucially, targeted specific vector ablation establishes a strict causal binding between this topology and model function: algebraically erasing the divergence component collapses parity classification accuracy from 100% to chance levels (38.57%). Furthermore, we uncover a three-phase layer-wise geometric dynamic and demonstrate that under social pressure prompts, models fail to generate sufficient divergence. This results in a "manifold entanglement" that geometrically explains sycophancy and hallucination. Ultimately, our findings revise the linear-isometric presumption, demonstrating that the emergence of discrete logic in LLMs is purchased at an irreducible cost of topological deformation.

The Geometric Price of Discrete Logic: Context-driven Manifold Dynamics of Number Representations

Abstract

Large language models (LLMs) generalize smoothly across continuous semantic spaces, yet strict logical reasoning demands the formation of discrete decision boundaries. Prevailing theories relying on linear isometric projections fail to resolve this fundamental tension. In this work, we argue that task context operates as a non-isometric dynamical operator that enforces a necessary "topological distortion." By applying Gram-Schmidt decomposition to residual-stream activations , we reveal a dual-modulation mechanism driving this process: a class-agnostic topological preservation that anchors global structure to prevent semantic collapse, and a specific algebraic divergence that directionally tears apart cross-class concepts to forge logical boundaries. We validate this geometric evolution across a gradient of tasks, from simple mapping to complex primality testing. Crucially, targeted specific vector ablation establishes a strict causal binding between this topology and model function: algebraically erasing the divergence component collapses parity classification accuracy from 100% to chance levels (38.57%). Furthermore, we uncover a three-phase layer-wise geometric dynamic and demonstrate that under social pressure prompts, models fail to generate sufficient divergence. This results in a "manifold entanglement" that geometrically explains sycophancy and hallucination. Ultimately, our findings revise the linear-isometric presumption, demonstrating that the emergence of discrete logic in LLMs is purchased at an irreducible cost of topological deformation.
Paper Structure (24 sections, 13 equations, 3 figures, 1 table)

This paper contains 24 sections, 13 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Theoretical Framework & Hypotheses
  3. Base State: Initial Spatial Mapping of Concepts
  4. Vector Decomposition of Task Instructions (Gram-Schmidt Orthogonalization)
  5. State Updates and Equivalent Rotation under RMSNorm
  6. Core Similarity Evolution Formula and Algebraic Cost Deconstruction
  7. Core Theoretical Hypotheses
  8. Experimental Design
  9. Programmatic Dataset Synthesis
  10. Knowledge-Based Filtering
  11. Hidden Representation Extraction and Centered Geometric Decomposition
  12. Category-Aware Metrics
  13. Manifold Healing via Specific Vector Ablation
  14. Layer-wise Geometric Dynamics Tracking
  15. Results & Discussion
  16. ...and 9 more sections

Figures (3)

  • Figure 1: Panoramic analysis of geometric evolution across task levels. The scatter plots illustrate the transition from isometric translation to specific divergence, while the bottom density plots confirm class-agnostic topological preservation ($C_{ij}$).
  • Figure 2: Comparison of specific vectors pre- and post-ablation. The specific vector erasure successfully collapses the cross-class clusters back into the entangled isomorphic zone.
  • Figure 3: Layer-wise evolution of representation manifolds. Top row: $U_{sim}$ dynamics showing the three-phase mechanism; Middle row: $C_{ij}$ tracking; Bottom row: 2D phase portraits illustrating the scissor-like bifurcation in logical tasks.