An Algorithmic Information-Theoretic Perspective on the Symbol Grounding Problem
Zhangchi Liu
TL;DR
The paper reframes the Symbol Grounding Problem through Algorithmic Information Theory by treating symbols as programs and worlds as data strings, defining grounding as data compression measured by Kolmogorov complexity $K(g)$ and conditional complexity $K(g|\mathcal{S})$. It establishes four information-theoretic stages: most worlds are incompressible, static grounding is incomplete against adversarial worlds, the grounding act is non-inferable and requires new information, and fixed learning algorithms face Chaitin-type limits that bound their provable grasp of world complexity. It further proves the uncomputability of Optimal Grounding due to the inherent uncomputability of $K(g)$, thereby arguing that meaning is an ongoing search for increasingly compact descriptions rather than a fixed attainment. Collectively, the results unify Gödelian and NFL perspectives under a single information-theoretic constraint, reframing meaning as an open-ended engagement with potentially infinite complexity.
Abstract
This paper provides a definitive, unifying framework for the Symbol Grounding Problem (SGP) by reformulating it within Algorithmic Information Theory (AIT). We demonstrate that the grounding of meaning is a process fundamentally constrained by information-theoretic limits, thereby unifying the Gödelian (self-reference) and No Free Lunch (statistical) perspectives. We model a symbolic system as a universal Turing machine and define grounding as an act of information compression. The argument proceeds in four stages. First, we prove that a purely symbolic system cannot ground almost all possible "worlds" (data strings), as they are algorithmically random and thus incompressible. Second, we show that any statically grounded system, specialized for compressing a specific world, is inherently incomplete because an adversarial, incompressible world relative to the system can always be constructed. Third, the "grounding act" of adapting to a new world is proven to be non-inferable, as it requires the input of new information (a shorter program) that cannot be deduced from the system's existing code. Finally, we use Chaitin's Incompleteness Theorem to prove that any algorithmic learning process is itself a finite system that cannot comprehend or model worlds whose complexity provably exceeds its own. This establishes that meaning is the open-ended process of a system perpetually attempting to overcome its own information-theoretic limitations.