Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

WavePhaseNet: A DFT-Based Method for Constructing Semantic Conceptual Hierarchy Structures (SCHS)

Kiyotaka Kasubuchi, Kazuo Fukiya

TL;DR

This paper demonstrates that reduction from 24,576 to 3,000 dimensions preserves meaning and intent while enabling rigorous reasoning and suppressing hallucination, and derives that approximately 3,000 dimensions constitute the lower bound for"complete representation."

Abstract

This paper reformulates Transformer/Attention mechanisms in Large Language Models (LLMs) through measure theory and frequency analysis, theoretically demonstrating that hallucination is an inevitable structural limitation. The embedding space functions as a conditional expectation over a σ-algebra, and its failure to be isomorphic to the semantic truth set fundamentally causes logical consistency breakdown. WavePhaseNet Method The authors propose WavePhaseNet, which explicitly constructs a Semantic Conceptual Hierarchy Structure (SCHS) using Discrete Fourier Transform (DFT). By applying DFT along the sequence dimension, semantic information is decomposed into frequency bands: low-frequency components capture global meaning and intent, while high-frequency components represent local syntax and expression. This staged separation enables precise semantic manipulation in diagonalized space. Dimensionality Reduction GPT-4's 24,576-dimensional embedding space exhibits a 1/f spectral structure based on language self-similarity and Zipf's law. Through cumulative energy analysis, the authors derive that approximately 3,000 dimensions constitute the lower bound for "complete representation." This demonstrates that reduction from 24,576 to 3,000 dimensions preserves meaning and intent while enabling rigorous reasoning and suppressing hallucination. Cohomological Consistency Control The reduced embedding space, constructed via cohomological regularization over overlapping local windows, allows defining a graph structure and cochain complex. This quantifies inconsistencies among local inferences as coboundary-based losses. Applying harmonic projection based on Hodge theory positions cohomology as a computable regularization principle for controlling semantic consistency, extracting maximally consistent global representations.

WavePhaseNet: A DFT-Based Method for Constructing Semantic Conceptual Hierarchy Structures (SCHS)

TL;DR

This paper demonstrates that reduction from 24,576 to 3,000 dimensions preserves meaning and intent while enabling rigorous reasoning and suppressing hallucination, and derives that approximately 3,000 dimensions constitute the lower bound for"complete representation."

Abstract

This paper reformulates Transformer/Attention mechanisms in Large Language Models (LLMs) through measure theory and frequency analysis, theoretically demonstrating that hallucination is an inevitable structural limitation. The embedding space functions as a conditional expectation over a σ-algebra, and its failure to be isomorphic to the semantic truth set fundamentally causes logical consistency breakdown. WavePhaseNet Method The authors propose WavePhaseNet, which explicitly constructs a Semantic Conceptual Hierarchy Structure (SCHS) using Discrete Fourier Transform (DFT). By applying DFT along the sequence dimension, semantic information is decomposed into frequency bands: low-frequency components capture global meaning and intent, while high-frequency components represent local syntax and expression. This staged separation enables precise semantic manipulation in diagonalized space. Dimensionality Reduction GPT-4's 24,576-dimensional embedding space exhibits a 1/f spectral structure based on language self-similarity and Zipf's law. Through cumulative energy analysis, the authors derive that approximately 3,000 dimensions constitute the lower bound for "complete representation." This demonstrates that reduction from 24,576 to 3,000 dimensions preserves meaning and intent while enabling rigorous reasoning and suppressing hallucination. Cohomological Consistency Control The reduced embedding space, constructed via cohomological regularization over overlapping local windows, allows defining a graph structure and cochain complex. This quantifies inconsistencies among local inferences as coboundary-based losses. Applying harmonic projection based on Hodge theory positions cohomology as a computable regularization principle for controlling semantic consistency, extracting maximally consistent global representations.
Paper Structure (35 sections, 1 theorem, 32 equations, 1 table)

This paper contains 35 sections, 1 theorem, 32 equations, 1 table.

Table of Contents

  1. Introduction
  2. Autoregressive Models and Embedding Spaces as $\sigma$-algebras
  3. Measure-Theoretic Formulation of Autoregressive Models
  4. Norm Structure of $L^p$ Spaces
  5. Norms and the Minkowski Inequality
  6. Mathematical Definition of Hallucination
  7. Definition
  8. Impossibility of Isomorphism and Incompleteness
  9. Reformulation as a Defect on the Truth Manifold
  10. Zipf's Law and $1/f$ Spectral Structure
  11. Zipf's Law in Natural Language
  12. Spectral Structure of Embedding Spaces
  13. Dimensional Bound via Cumulative Energy Analysis
  14. DFT-Based Semantic Conceptual Hierarchy Structure
  15. Discrete Fourier Transform on Token Sequences
  16. ...and 20 more sections

Key Result

Theorem 2

For the embedding space $V$ and the truth manifold $T$, if there exists a non-negligible region where then there necessarily exists a Cauchy sequence in $V$ whose limit does not belong to $T$. In other words, hallucination is inevitable.

Theorems & Definitions (3)

  • Definition 1: Hallucination
  • Theorem 2: Structural Inevitability of Hallucination
  • proof