The Exploratory Study on the Relationship Between the Failure of Distance Metrics in High-Dimensional Space and Emergent Phenomena

TL;DR

The paper proposes a unified information-theoretic–statistical-mechanical framework to explain how distance metrics fail in high-dimensional spaces and how emergent global features arise in complex systems. It introduces the Information Dilution Theorem, showing the mutual information efficiency $\eta(D)=\frac{I(D;S)}{H(S)}$ decays as $O\left(\frac{1}{d}\right)$, and the Emergence Critical Theorem, which links the system's information structural complexity $\mathcal{C}(S)$ to an encoding-capacity threshold $C'$ to predict the appearance of new features $f_{\mathrm{new}}$ with $\frac{I(f_{\mathrm{new}};S)}{H(S)}>\theta$. The framework provides an operational lens for self-organization and phase transitions, and is extended to cross-disciplinary applications including MI-based manifold learning (UMAP+) and neural network hierarchy emergence, with proposed validations in Ising models and gene-expression data. It also outlines future directions and limitations, such as handling strong coupling and dynamic systems, and calls for empirical studies to substantiate the theory across domains.

Abstract

This paper presents a unified framework, integrating information theory and statistical mechanics, to connect metric failure in high-dimensional data with emergence in complex systems. We propose the "Information Dilution Theorem," demonstrating that as dimensionality ($d$) increases, the mutual information efficiency between geometric metrics (e.g., Euclidean distance) and system states decays approximately as $O(1/d)$. This decay arises from the mismatch between linearly growing system entropy and sublinearly growing metric entropy, explaining the mechanism behind distance concentration. Building on this, we introduce information structural complexity ($C(S)$) based on the mutual information matrix spectrum and interaction encoding capacity ($C'$) derived from information bottleneck theory. The "Emergence Critical Theorem" states that when $C(S)$ exceeds $C'$, new global features inevitably emerge, satisfying a predefined mutual information threshold. This provides an operational criterion for self-organization and phase transitions. We discuss potential applications in physics, biology, and deep learning, suggesting potential directions like MI-based manifold learning (UMAP+) and offering a quantitative foundation for analyzing emergence across disciplines.