Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Counterfactual Basis Extension and Representational Geometry: An MDL-Constrained Model of Conceptual Growth

Chainarong Amornbunchornvej

TL;DR

The paper tackles how a representational basis can grow, not just adapt, by proposing a geometric framework where new dimensions arise via admissible, low-rank basis extensions evaluated under a Minimum Description Length (MDL) criterion. Imagination is reframed as counterfactual exploration over representations, constrained by residual structure so that novelty is admitted only when it yields meaningful compression gains. The main contributions include formalizing an admissible extension class, deriving an MDL acceptance rule, and defining representational counterfactuals that govern conceptual growth, with normative rather than phenomenological implications. Key findings show that new concepts emerge only along residual-aligned directions and within low-rank limits, while orthogonal or unstructured novelty is rejected; imagination helps only by exposing or amplifying residual structure. The framework offers a principled account of theory change and provides interpretive insights for AI systems by explaining how structured residuals constrain the growth of abstractions.

Abstract

Concept learning becomes possible only when existing representations fail to account for experience. Most models of learning and inference, however, presuppose a fixed representational basis within which belief updating occurs. In this paper, I address a prior question: under what structural conditions can the representational basis itself expand in a principled and selective way? I propose a geometric framework in which conceptual growth is modeled as admissible basis extension evaluated under a Minimum Description Length (MDL) criterion. Experience, whether externally observed or internally simulated, is represented as vectors relative to a current conceptual subspace. Residual components capture systematic representational failure, and candidate conceptual extensions are restricted to low-rank, admissible transformations. I show that any MDL-accepted extension can be chosen so that its novel directions lie entirely within the residual span induced by experience, while extensions orthogonal to this span strictly increase description length and are therefore rejected. This yields a conservative account of imagination and conceptual innovation. Internally generated counterfactual representations contribute to learning only insofar as they expose or amplify structured residual error, and cannot introduce arbitrary novelty. I further distinguish representational counterfactuals--counterfactuals over an agent's conceptual basis--from causal or value-level counterfactuals, and show how MDL provides a normative selection principle governing representational change. Overall, the framework characterizes conceptual development as an error-driven, geometry-constrained process of basis extension, clarifying both the role and the limits of imagination in learning and theory change.

Counterfactual Basis Extension and Representational Geometry: An MDL-Constrained Model of Conceptual Growth

TL;DR

The paper tackles how a representational basis can grow, not just adapt, by proposing a geometric framework where new dimensions arise via admissible, low-rank basis extensions evaluated under a Minimum Description Length (MDL) criterion. Imagination is reframed as counterfactual exploration over representations, constrained by residual structure so that novelty is admitted only when it yields meaningful compression gains. The main contributions include formalizing an admissible extension class, deriving an MDL acceptance rule, and defining representational counterfactuals that govern conceptual growth, with normative rather than phenomenological implications. Key findings show that new concepts emerge only along residual-aligned directions and within low-rank limits, while orthogonal or unstructured novelty is rejected; imagination helps only by exposing or amplifying residual structure. The framework offers a principled account of theory change and provides interpretive insights for AI systems by explaining how structured residuals constrain the growth of abstractions.

Abstract

Concept learning becomes possible only when existing representations fail to account for experience. Most models of learning and inference, however, presuppose a fixed representational basis within which belief updating occurs. In this paper, I address a prior question: under what structural conditions can the representational basis itself expand in a principled and selective way? I propose a geometric framework in which conceptual growth is modeled as admissible basis extension evaluated under a Minimum Description Length (MDL) criterion. Experience, whether externally observed or internally simulated, is represented as vectors relative to a current conceptual subspace. Residual components capture systematic representational failure, and candidate conceptual extensions are restricted to low-rank, admissible transformations. I show that any MDL-accepted extension can be chosen so that its novel directions lie entirely within the residual span induced by experience, while extensions orthogonal to this span strictly increase description length and are therefore rejected. This yields a conservative account of imagination and conceptual innovation. Internally generated counterfactual representations contribute to learning only insofar as they expose or amplify structured residual error, and cannot introduce arbitrary novelty. I further distinguish representational counterfactuals--counterfactuals over an agent's conceptual basis--from causal or value-level counterfactuals, and show how MDL provides a normative selection principle governing representational change. Overall, the framework characterizes conceptual development as an error-driven, geometry-constrained process of basis extension, clarifying both the role and the limits of imagination in learning and theory change.

Paper Structure

This paper contains 46 sections, 10 theorems, 38 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Scope.
  4. Related Work
  5. Geometric and Vector-Space Models of Concepts
  6. Representation Learning and Feature Discovery
  7. MDL and Simplicity Principles
  8. Counterfactual Reasoning
  9. Theory: Counterfactual Basis Extension
  10. Conceptual Space and Residual Structure
  11. MDL-Governed Conceptual Change
  12. Counterfactual Basis Extension
  13. Interpretation.
  14. When Is Novelty Accepted?
  15. Intuition.
  16. ...and 31 more sections

Key Result

Theorem 1

Let $\mathcal{C}\subseteq\mathcal{H}$ be the agent’s current conceptual subspace and $D$ a finite multiset of experience vectors. Define the residual span Where $C^\perp$ is the set of all vectors orthogonal to every vector in $C$. Under mild structural admissibility conditions on admissible extensions (formalized in Appendix app:ptc-proof), any MDL-accepted conceptual update must satisfy:

Figures (1)

  • Figure 1: Schematic illustration of counterfactual basis extension under MDL. Residual error induces candidate novelty directions, which are filtered by description length reduction. Conceptual growth occurs only when a proposed basis extension compresses structured residuals.

Theorems & Definitions (21)

  • Definition 1: Conceptual Space
  • Definition 2: Residual
  • Definition 3: MDL Acceptance
  • Theorem 1: Counterfactual Basis Extension under MDL
  • Proposition 1: MDL Acceptance Threshold
  • Proposition 2: Two Mechanisms by which Imagination Affects Learning
  • Definition 4: MDL objective and residual span
  • Definition 5: Admissible extension class
  • Proposition 3: Low-rank novelty
  • proof
  • ...and 11 more