Compression, Regularity, Randomness and Emergent Structure: Rethinking Physical Complexity in the Data-Driven Era

TL;DR

The paper presents a unified framework that positions statistical, algorithmic, and dynamical complexity measures along three axes—regularity, randomness, and complexity—and discusses their computability and practical approximations. It argues that modern data‑driven methods, especially latent representations learned by autoencoders, latent ODEs, and PINNs, operationalize classical complexity ideals by compressing data, extracting regularities, and modeling dynamics within latent spaces. By introducing conceptual maps and a depth–accessibility trade‑off, the authors illuminate why certain measures are theoretically rich yet practically uncomputable and how proxies can be used to bridge theory and practice. The work outlines a roadmap for integrating compression, regularity extraction, and complexity negotiation into latent‑space learning, with implications for physics‑informed AI and AI‑guided discovery in complex physical systems.

Abstract

Complexity science offers a wide range of measures for quantifying unpredictability, structure, and information. Yet, a systematic conceptual organization of these measures is still missing. We present a unified framework that locates statistical, algorithmic, and dynamical measures along three axes (regularity, randomness, and complexity) and situates them in a common conceptual space. We map statistical, algorithmic, and dynamical measures into this conceptual space, discussing their computational accessibility and approximability. This taxonomy reveals the deep challenges posed by uncomputability and highlights the emergence of modern data-driven methods (including autoencoders, latent dynamical models, symbolic regression, and physics-informed neural networks) as pragmatic approximations to classical complexity ideals. Latent spaces emerge as operational arenas where regularity extraction, noise management, and structured compression converge, bridging theoretical foundations with practical modeling in high-dimensional systems. We close by outlining implications for physics-informed AI and AI-guided discovery in complex physical systems, arguing that classical questions of complexity remain central to next-generation scientific modeling.