On the Limits of Self-Improving in LLMs and Why AGI, ASI and the Singularity Are Not Near Without Symbolic Model Synthesis
Hector Zenil
TL;DR
The paper formalizes recursive self-improvement in large language models as a discrete-time dynamical system and proves that training on increasingly self-generated data inexorably degrades diversity and drifts toward degenerate solutions when external grounding vanishes. It shows entropy decay and variance drift under self-referential updates, and (in the open-loop limit) convergence to a fixed point that is misaligned with the true data distribution. The authors argue that purely statistical objectives (e.g., KL-based training) cannot escape the Data Processing Inequality and thus cannot produce new, synthetic knowledge; to overcome this, they propose a neurosymbolic framework that combines causal correction and algorithmic information theory (via CTM/BDM) to recover mechanism-level understanding and symbolic grounding. The work presents a principled case for integrating symbolic regression, causal models, and program synthesis to enable robust self-improvement, rather than an unbounded analytic expansion, with significant implications for the feasibility of a Singularity and for designing safer, more grounded AI systems.
Abstract
We formalise recursive self-training in Large Language Models (LLMs) and Generative AI as a discrete-time dynamical system and prove that, as training data become increasingly self-generated ($α_t \to 0$), the system undergoes inevitably degenerative dynamics. We derive two fundamental failure modes: (1) Entropy Decay, where finite sampling effects cause a monotonic loss of distributional diversity (mode collapse), and (2) Variance Amplification, where the loss of external grounding causes the model's representation of truth to drift as a random walk, bounded only by the support diameter. We show these behaviours are not contingent on architecture but are consequences of distributional learning on finite samples. We further argue that Reinforcement Learning with imperfect verifiers suffers similar semantic collapse. To overcome these limits, we propose a path involving symbolic regression and program synthesis guided by Algorithmic Probability. The Coding Theorem Method (CTM) allows for identifying generative mechanisms rather than mere correlations, escaping the data-processing inequality that binds standard statistical learning. We conclude that while purely distributional learning leads to model collapse, hybrid neurosymbolic approaches offer a coherent framework for sustained self-improvement.