How Do Artificial Intelligences Think? The Three Mathematico-Cognitive Factors of Categorical Segmentation Operated by Synthetic Neurons
Michael Pichat, William Pogrund, Armanush Gasparian, Paloma Pichat, Samuel Demarchi, Michael Veillet-Guillem
TL;DR
The paper probes how artificial neurons construct and segment categories, proposing three mathematico-cognitive factors—categorical priming, attention, and phasing—embedded in the aggregation function $\sum(w_{i,j} x_{i,j}) + a$ that shape a neuron's categorical extension. Using GPT-2XL, it analyzes 6,400 destination neurons in layer 1 with 100 core-tokens each and 10 precursor neurons per destination, validating the three factors through both quantitative (Spearman correlations: $\rho = 0.94$, $p < 0.001$ for priming; $\rho = 0.999$, $p < 0.001$ for attention; $\rho = 0.989$, $p < 0.001$ for phasing) and qualitative analyses. The study demonstrates that prior precursor activations propagate to destination activations, that connection weights regulate information uptake and token extraction (taken-tokens), and that co-activation across precursors strengthens activations via categorical intersections, collectively enabling a mechanistic account of artificial categorization. These findings illuminate how synthetic cognition constructs and refines representations at a microscopic level, with implications for explainability, bias, and alignment in language models, and lay groundwork for exploring higher-level abstraction in subsequent layers. The work advances a formal, explainable view of how neural architectures generate meaning through internally constructed categories and their content.
Abstract
How do the synthetic neurons in language models create "thought categories" to segment and analyze their informational environment? What are the cognitive characteristics, at the very level of formal neurons, of this artificial categorical thought? Based on the mathematical nature of algebraic operations inherent to neuronal aggregation functions, we attempt to identify mathematico-cognitive factors that genetically shape the categorical reconstruction of the informational world faced by artificial cognition. This study explores these concepts through the notions of priming, attention, and categorical phasing.