Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

What Machine Learning Tells Us About the Mathematical Structure of Concepts

Jun Otsuka

TL;DR

This paper investigates how concepts across philosophy, cognitive science, and ML can be captured by four mathematical frameworks: Abstractionism (lattice/poset), Similarity (metric spaces and vector representations), Functional (laws/functional relations and manifolds), and Invariance (group actions, invariance/equivariance). It surveys historical and contemporary models, shows how each approach yields different representations—lattice of intents/extents, cluster-based metric spaces, submanifolds on latent spaces, and invariant representations—and discusses cross-domain connections and potential unifications via non-Euclidean geometry and group theory. The main contribution is a structured taxonomy linking philosophical and computational concept representations and highlighting interdisciplinary synthesis to guide future research. It emphasizes interpretability, robustness, and the design of representations that reflect both data structure and transformation invariances. It notes limitations and future directions, including attention to newer models like Attention mechanisms and Diffusion models.

Abstract

This paper examines the connections among various approaches to understanding concepts in philosophy, cognitive science, and machine learning, with a particular focus on their mathematical nature. By categorizing these approaches into Abstractionism, the Similarity Approach, the Functional Approach, and the Invariance Approach, the study highlights how each framework provides a distinct mathematical perspective for modeling concepts. The synthesis of these approaches bridges philosophical theories and contemporary machine learning models, providing a comprehensive framework for future research. This work emphasizes the importance of interdisciplinary dialogue, aiming to enrich our understanding of the complex relationship between human cognition and artificial intelligence.

What Machine Learning Tells Us About the Mathematical Structure of Concepts

TL;DR

This paper investigates how concepts across philosophy, cognitive science, and ML can be captured by four mathematical frameworks: Abstractionism (lattice/poset), Similarity (metric spaces and vector representations), Functional (laws/functional relations and manifolds), and Invariance (group actions, invariance/equivariance). It surveys historical and contemporary models, shows how each approach yields different representations—lattice of intents/extents, cluster-based metric spaces, submanifolds on latent spaces, and invariant representations—and discusses cross-domain connections and potential unifications via non-Euclidean geometry and group theory. The main contribution is a structured taxonomy linking philosophical and computational concept representations and highlighting interdisciplinary synthesis to guide future research. It emphasizes interpretability, robustness, and the design of representations that reflect both data structure and transformation invariances. It notes limitations and future directions, including attention to newer models like Attention mechanisms and Diffusion models.

Abstract

This paper examines the connections among various approaches to understanding concepts in philosophy, cognitive science, and machine learning, with a particular focus on their mathematical nature. By categorizing these approaches into Abstractionism, the Similarity Approach, the Functional Approach, and the Invariance Approach, the study highlights how each framework provides a distinct mathematical perspective for modeling concepts. The synthesis of these approaches bridges philosophical theories and contemporary machine learning models, providing a comprehensive framework for future research. This work emphasizes the importance of interdisciplinary dialogue, aiming to enrich our understanding of the complex relationship between human cognition and artificial intelligence.
Paper Structure (8 sections, 3 equations, 4 figures)

This paper contains 8 sections, 3 equations, 4 figures.

Table of Contents

  1. Introduction
  2. The Abstractionism
  3. The Similarity Approach
  4. The Functional Approach
  5. The Invariance Approach
  6. Discussion
  7. Conclusion
  8. Acknowledgment

Figures (4)

  • Figure 1: An example of a conceptual lattice. Each link represents 'is-a' relationship between the relata. The upward arrows (dotted and orange) indicate the join operation that abstracts various concepts, while the downward arrows (solid and blue) indicate the meet operation that combines concepts.
  • Figure 2: Manifold learning with VAE. The left plot shows the latent representation space of a VAE model trained on the MNIST dataset, where each digit is represented by a different color, forming (roughly) continuous subregions. As a generative model, the VAE can produce images corresponding to points within this representation space. The right image displays figures generated from the black dots arranged in a grid in the left plot, illustrating the continuous morphing of digit shapes.
  • Figure 3: An image of face morphing on a conceptual manifold. If a concept is defined as a submanifold in the representation space, a smooth curve connecting two data points represents a "morphing" of one instance (her, a face with glasses) into another (one without). Images are adapted and modified from Higgins2016-rx.
  • Figure 4: Invariant (left) and equivariant (right) representation.