The Process of Categorical Clipping at the Core of the Genesis of Concepts in Synthetic Neural Cognition

TL;DR

This work investigates how language-model neurons perform synthetic categorical segmentation to generate new concepts by outlining subdimensions from precursor categories. It centers on three aggregation-driven factors—categorical priming ($x$-effect), categorical attention ($w$-effect), and categorical phasing ($\Sigma$-effect)—and maps Piagetian reflective abstraction to the neural formation of form versus background. Through GPT-2XL experiments across two layers, the study provides empirical evidence of categorical reduction, selectivity, and the separation of embedding dimensions, supported by PCA and t-SNE analyses that reveal semantically coherent, human-interpretable lexical zones. The findings advance a pragmatically grounded epistemology of synthetic concepts-in-act and theorems-in-act, emphasizing functional, action-oriented representations rather than attempting to inflate AI subdimensions with human-like meaning, and point to ongoing cross-layer analysis of how subdimensions converge to form novel categorical structures.

Abstract

This article investigates, within the field of neuropsychology of artificial intelligence, the process of categorical segmentation performed by language models. This process involves, across different neural layers, the creation of new functional categorical dimensions to analyze the input textual data and perform the required tasks. Each neuron in a multilayer perceptron (MLP) network is associated with a specific category, generated by three factors carried by the neural aggregation function: categorical priming, categorical attention, and categorical phasing. At each new layer, these factors govern the formation of new categories derived from the categories of precursor neurons. Through a process of categorical clipping, these new categories are created by selectively extracting specific subdimensions from the preceding categories, constructing a distinction between a form and a categorical background. We explore several cognitive characteristics of this synthetic clipping in an exploratory manner: categorical reduction, categorical selectivity, separation of initial embedding dimensions, and segmentation of categorical zones.

Figures (12)

• Figure : Graph n$^{\circ}$1: Descriptive comparison of mean cosine similarity distances between taken-tokens and core-tokens, for each relevant precursor neuron of each destination neuron (Layer 1 ; $N=9007$).
• Figure : Graph n$^{\circ}$2: Descriptive comparison of mean cosine similarity distances between taken-tokens and core-tokens, for one precursor neuron of neuron 3000 (Layer 1).
• Figure : Graph n$^{\circ}$3: Descriptive comparison of mean cosine similarity distances between taken-tokens and left-tokens, for each relevant precursor neuron of each destination neuron (Layer 1 ; $N=9007$).
• Figure : Graph n$^{\circ}$4: Descriptive comparison of mean cosine similarity distances between taken-tokens and left-tokens, for one precursor neuron (Layer 1 ; Control neuron 3000).
• Figure : Graph n$^{\circ}$5: PCA correlation circle on embedding variables and group membership variable (left/taken) associated with left-tokens and taken-tokens (Weighting=1% ; Target neuron 816 ; Precursor neuron 12 ; on the 78.91% of variables with cos$^{2} > .6$).