How Do Large Language Models Learn Concepts During Continual Pre-Training?

TL;DR

This work investigates how large language models acquire, retain, and forget concepts during continual pretraining by introducing the Fico dataset of fictional concepts derived from ConceptNet, and by extracting Concept Circuits—computational subgraphs that encode concept-specific knowledge. It links circuit topology to concept learning and forgetting via four graph metrics, reveals a stage-wise consolidation trajectory, and shows that stronger learning can lead to greater forgetting due to interference. The study further demonstrates cross-concept interference driven by semantic similarity and asymmetric transfer between knowledge types, offering actionable guidance for interference-aware curricula and concept-aware training. Collectively, these findings provide a circuit-level, time-resolved view of concept learning in LLMs and pave the way for more interpretable and robust continual pretraining strategies.

Abstract

Human beings primarily understand the world through concepts (e.g., dog), abstract mental representations that structure perception, reasoning, and learning. However, how large language models (LLMs) acquire, retain, and forget such concepts during continual pretraining remains poorly understood. In this work, we study how individual concepts are acquired and forgotten, as well as how multiple concepts interact through interference and synergy. We link these behavioral dynamics to LLMs' internal Concept Circuits, computational subgraphs associated with specific concepts, and incorporate Graph Metrics to characterize circuit structure. Our analysis reveals: (1) LLMs concept circuits provide a non-trivial, statistically significant signal of concept learning and forgetting; (2) Concept circuits exhibit a stage-wise temporal pattern during continual pretraining, with an early increase followed by gradual decrease and stabilization; (3) concepts with larger learning gains tend to exhibit greater forgetting under subsequent training; (4) semantically similar concepts induce stronger interference than weakly related ones; (5) conceptual knowledge differs in their transferability, with some significantly facilitating the learning of others. Together, our findings offer a circuit-level view of concept learning dynamics and inform the design of more interpretable and robust concept-aware training strategies for LLMs.

Figures (19)

• Figure 1: (A) Construct the Fico dataset based on ConceptNet. (B) Extract Concept Circuits, LLM computational subgraph associated with individual concepts, and characterize their structure using graph metrics. (C) Analyze concept learning and forgetting dynamics in two-stage continual pretraining. (D) Study synergy and interference across concepts (e.g., $A$, $B$, $Y$) and knowledge type (e.g., $T1$ and $T2$).
• Figure 2: Distribution of learning and forgetting degree across concepts.
• Figure 3: Correlation between learning dynamics and LLM circuit pattern.
• Figure 4: Graph Metric over training steps
• Figure 5: Spearman Correlations between learning and forgetting of concepts

Theorems & Definitions (2)

• Definition 1: Knowledge Learning/Forgetting De- gree
• Definition 2: Concept Learning/Forgetting De- gree