Can Intelligence Explode?
Marcus Hutter
TL;DR
The paper interrogates whether the technological singularity necessarily entails an intelligence explosion, distinguishing speed from true increases in intelligence and grounding the discussion in a virtual, rational-agent framework. It surveys competing narratives of rapid acceleration, contrasts outsider and insider experiences, and formalizes intelligence using the universal intelligence measure $\Upsilon$ and the $AIXI$ model to assess theoretical bounds. By examining both observable dynamics (speedups) and intrinsic limits (bounded intelligence), it analyzes how a society of near- or at-singularity agents might evolve and what this implies for the value of virtual life and ethical governance. The work highlights that while speed explosions are plausible and observable, an unchecked intelligence explosion is not guaranteed, and rigorous analysis via $\Upsilon$-based frameworks offers meaningful insights for future research and policy.
Abstract
The technological singularity refers to a hypothetical scenario in which technological advances virtually explode. The most popular scenario is the creation of super-intelligent algorithms that recursively create ever higher intelligences. It took many decades for these ideas to spread from science fiction to popular science magazines and finally to attract the attention of serious philosophers. David Chalmers' (JCS 2010) article is the first comprehensive philosophical analysis of the singularity in a respected philosophy journal. The motivation of my article is to augment Chalmers' and to discuss some issues not addressed by him, in particular what it could mean for intelligence to explode. In this course, I will (have to) provide a more careful treatment of what intelligence actually is, separate speed from intelligence explosion, compare what super-intelligent participants and classical human observers might experience and do, discuss immediate implications for the diversity and value of life, consider possible bounds on intelligence, and contemplate intelligences right at the singularity.