A theory of understanding for artificial intelligence: composability, catalysts, and learning

TL;DR

The paper addresses the hard problem of defining understanding in AI and other subjects by proposing a verifier-based, minimal framework centered on composability. It formalizes understanding as the set $S$ of input-output tuples $(\vec{I}, I_{\rm out})$ that a verifier deems related to an object, and introduces catalysts and subject decomposition as tools to analyze and reveal internal structure and learning. Key contributions include a practical operationalization of understanding via composition, the concepts of inner catalysts and acquisition of understanding, and an argument that autocatalysis enabled by large language models (LLMs) is a promising path toward general intelligence. The work highlights how catalysts (e.g., explanations, RAG, and chain-of-thought prompts) can improve outputs, how decomposing subjects reveals cognitive architecture, and how learning corresponds to updating inner catalysts, suggesting a framework for building more general, AI-enabled learners with broad, cross-domain applicability.

Abstract

Understanding is a crucial yet elusive concept in artificial intelligence (AI). This work proposes a framework for analyzing understanding based on the notion of composability. Given any subject (e.g., a person or an AI), we suggest characterizing its understanding of an object in terms of its ability to process (compose) relevant inputs into satisfactory outputs from the perspective of a verifier. This highly universal framework can readily apply to non-human subjects, such as AIs, non-human animals, and institutions. Further, we propose methods for analyzing the inputs that enhance output quality in compositions, which we call catalysts. We show how the structure of a subject can be revealed by analyzing its components that act as catalysts and argue that a subject's learning ability can be regarded as its ability to compose inputs into its inner catalysts. Finally we examine the importance of learning ability for AIs to attain general intelligence. Our analysis indicates that models capable of generating outputs that can function as their own catalysts, such as language models, establish a foundation for potentially overcoming existing limitations in AI understanding.

Figures (3)

• Figure 1: A diagrammatic depiction of composition and the verifier of understanding. We define composition as a general process that produces an output based on inputs, where the inputs can be any reasonable entity in the verifier's view. In our framework of understanding, verifiers characterize a subject's understanding of an object $O$ by its outputs in composing inputs that are related to $O$.
• Figure 2: A replication of a student's drawing of the layout of Dr. Jekyll's house with Nabokov's annotations. nabokov2017lectures The student demonstrates an understanding of reading by composing the novel into a diagram. The diagram, as well as the process of making it, also helps the student understand the novel.
• Figure 3: Subject decomposition for a general subject. We reveal a subject's structure by decomposing it into inner catalysts and a primitive subject so that the composition done by the subject can be viewed as the primitive subject composing inner catalysts and inputs from outside. The learning ability of the subject is determined by its ability to produce inner catalysts.

Theorems & Definitions (6)

• Definition 1: Composition
• Definition 2: Understanding as composability
• Definition 3: Catalyst
• Definition 4: Subject decomposition
• Claim 1: Acquisition of understanding
• Claim 2: Learning ability