A neural network for modeling human concept formation, understanding and communication

TL;DR

The study tackles how humans form abstract concepts from sensorimotor experience and flexibly apply them for perception and communication. It introduces CATS Net, a dual-module architecture with a concept-abstraction (CA) module that yields low-dimensional concept vectors and a task-solving (TS) module gated hierarchically by these concepts, trained to perform visual judgments and transfer knowledge across networks. Across extensive analyses, the authors show that the emergent concept spaces align with human semantic structures (Binder65, SPOSE49) and with brain representations in the ventral occipitotemporal cortex, while gating aligns with the semantic control network; a translation module enables cross-network concept transfer that preserves semantic relationships. The work provides a mechanistic, modular account of concept formation and understanding with implications for human-like conceptual AI and advances in neuroscience through model-brain representational alignment.

Abstract

A remarkable capability of the human brain is to form more abstract conceptual representations from sensorimotor experiences and flexibly apply them independent of direct sensory inputs. However, the computational mechanism underlying this ability remains poorly understood. Here, we present a dual-module neural network framework, the CATS Net, to bridge this gap. Our model consists of a concept-abstraction module that extracts low-dimensional conceptual representations, and a task-solving module that performs visual judgement tasks under the hierarchical gating control of the formed concepts. The system develops transferable semantic structure based on concept representations that enable cross-network knowledge transfer through conceptual communication. Model-brain fitting analyses reveal that these emergent concept spaces align with both neurocognitive semantic model and brain response structures in the human ventral occipitotemporal cortex, while the gating mechanisms mirror that in the semantic control brain network. This work establishes a unified computational framework that can offer mechanistic insights for understanding human conceptual cognition and engineering artificial systems with human-like conceptual intelligence.

Figures (15)

• Figure 1: $\vert$ Motivation, experimental approach, and architecture of CATS net for concept decoupling and formation. a, The key characteristic of concepts is their decoupling from complex, high-dimensional sensory-motor information into lower-dimensional representations. For instance, the concept conveyed by a simple word like “dinner” can evoke neural population activity patterns associated with dining scenes, even without direct sensory stimulation. b, A possible solution for concept formation is to compress sensory-motor neural circuits, independent of direct inputs, into low-dimensional representations. If these concepts can subsequently activate proper circuits to effectively accomplish the desired functions, it can be regarded as concept understanding. c, Illustration of our concept abstraction task approach. After training from random CATS Net parameter weights and initial concept vectors (all same or totally random), the system gets a set of well-trained parameter weights and well-trained concept vectors, which further support successfully making binary judgement for a given image under a given concept. d, Schematic illustration of the dual-module architecture in CATS net: the CA module receives low-dimensional conceptual inputs to generate controlling signal for TS module; The TS module performs “Yes/No” judgement according to sensory inputs and gating operation by CA module. All images were adopted from PublicDomainPictures and Free-Images under a Creative Commons license CC0.
• Figure 2: $\vert$ Model performance and conceptual space semantic structure analyses. a, The performance of concept abstraction by CATS Net on ImageNet-1k dataset. The purple histogram depict the accuracy distribution of CATS Net for 1000 initial concept vectors before learning, while blue ones are after learning. In the inset, the purple and blue bar represents the average of mean accuracy across 30 models before and after training, and each pair point represents the corresponding mean accuracy of each category. b, Visualization of selective attentions on the same input modulated by different concept. c&d, The functional specificity of the unit basis vectors on hyper-categories before (c) and after learning (d). The height of bar indicates the number of 'yes' response of the input basis vectors to these five hyper-categories. This result is a single case randomly chosen from 30 instances. e, Calculation pipeline of “functional entropy”, which quantitatively measures the functional specificity on the task. f, Probability density distribution of functional entropy in the trained concept space (blue) and task-solving parameter space (purple). All images were adopted from PublicDomainPictures under a Creative Commons license CC0.
• Figure 3: $\vert$ Alignment of CATS concept layer with human semantic models. a, Representational dissimilarity matrices (RDMs) for the CATS concept layer, SPOSE49 hebart_revealing_2020, and Binder65 binder_toward_2016 models, computed on 332 concepts overlapping between ImageNet-1k (deng_imagenet_2009) and the THINGS dataset (hebart_things_2019) using Pearson distance. The CATS RDM represents the average across 30 independently initialized instances. Warmer colors (red) indicate greater dissimilarity between concept pairs, while cooler colors (blue) indicate greater similarity. b, Correlations of best-match conceptual dimensions in each CATS Net instance with given four SPOSE49 exemplar dimensions. Each bar represents the maximum Pearson correlation between the 20 concept dimensions of a given CATS instance and a specific SPOSE49 dimension (labeled around the perimeter; dimension names from Hebart et al. hebart_revealing_2020). The red circle denotes the significance threshold for correlation coefficients ($r = 0.107$, two-tailed $p < 0.05$, df = 330).
• Figure 4: $\vert$ Knowledge transfer via communication between independently trained CATS Nets. a&b, Semantic maps of the concept space formed by the teacher (a) and the student networks (b). Colors represent clusters at a given hierarchical clustering threshold. Manual adjustments were then made to achieve the closest possible visual alignment between the teacher Net and student Net clusters in the visualization. c, Pipeline for knowledge acquisition via communication between the teach and student nets, consisting of three phases: independent concept abstracting, concept alignment and concept transmission. d, Layer-wise RDMs of the translation module (in order of arrows: input layer, $3^{rd}$ layer, $7^{th}$ layer, and output layer; see Supplementary Figure 3a for a complete view of all layers). e, Performance of transferred concepts on CIFAR-100 for student net through communication. Each dot represents the accuracy of an independent model instance (n=100 independent experimental units), where each was trained on a unique 99-category subset and evaluated on the corresponding held-out category. For all violin plots, individual dots represent independent model instances. The unit of analysis is a single model. Violin plots show the kernel density estimation of the data distribution. Overlaid box plots indicate the median (center line), interquartile range (IQR; 25th–75th percentiles), and min–max range (whiskers). Statistical significance for accuracy was determined by a one-tailed one-sample t-test against chance level (0.5). $***$, $p < 0.001$; $**$, $p < 0.01$; $*$, $p < 0.05$. For all comparisons, the statistic values, degrees of freedom and exact P values are provided in the Source data. No technical replicates were used. Unless otherwise specified, the sample violin conventions are applied in all figures. All images were adopted from PublicDomainPictures and Free-Images under a Creative Commons license CC0.
• Figure 5: $\vert$ Concept acquisition in CATS Net using Word2Vec embeddings. a, Pipeline for novel concept acquisition in CATS Net on CIFAR-100 using Word2Vec. In Phase 1, images from 99 categories and their name embeddings (as predefined concept vectors) are used to train a randomly initialized CATS Net by updating only network parameters. In Phase 2, the remaining category and its Word2Vec embedding (as a unseen concept vector) is used to evaluate the model’s understanding of the novel concept. b, Performance on unseen concepts under the leave-one-out approach described in (a). Each dot represents the accuracy of an independent model instance ($n = 100$ independent experimental units), where each was trained on a unique 99-category subset and evaluated on the corresponding held-out category. c, RDMs of learned concept vectors (left) versus Word2Vec vectors (right). All images were adopted from PublicDomainPictures and Free-Images under a Creative Commons license CC0.