Differential syntactic and semantic encoding in LLMs

TL;DR

This work analyzes how syntax and meaning are encoded in inner representations of a very large LLM by constructing syntactic and semantic centroids and applying linear ablations to sentence vectors. It shows that both syntactic and semantic information are, to a significant extent, linearly encoded and that semantics centers in inner layers while syntax is more broadly distributed across layers. The results reveal partial decoupling between syntax and semantics and demonstrate robust effects across multiple models and languages, with implications for controllability and interpretability of LLMs. The approach provides a geometric lens on linguistic information and suggests avenues for steering model behavior by manipulating centroid components.

Abstract

We study how syntactic and semantic information is encoded in inner layer representations of Large Language Models (LLMs), focusing on the very large DeepSeek-V3. We find that, by averaging hidden-representation vectors of sentences sharing syntactic structure or meaning, we obtain vectors that capture a significant proportion of the syntactic and semantic information contained in the representations. In particular, subtracting these syntactic and semantic ``centroids'' from sentence vectors strongly affects their similarity with syntactically and semantically matched sentences, respectively, suggesting that syntax and semantics are, at least partially, linearly encoded. We also find that the cross-layer encoding profiles of syntax and semantics are different, and that the two signals can to some extent be decoupled, suggesting differential encoding of these two types of linguistic information in LLM representations.

Figures (15)

• Figure 1: Syntax similarity and its ablation. Similarity between equal-syntax sentences (syntax twins), such as those presented in Table \ref{['tab:syn-similarity']}. Panels a) and b) represent sentences by token concatenation and average, respectively. The shaded colored areas represent 1 standard deviation, calculated by subsampling five times half of the samples.
• Figure 2: Semantic similarity and its ablation. Similarity between English sentences and their (English) paraphrases. Panels a) and b) represent sentences by token concatenation and average, respectively. The shaded colored areas represent 1 standard deviation, calculated by subsampling five times half of the samples.
• Figure 3: Syntactic similarity with semantic ablation. The shaded colored areas represent 1 standard deviation, calculated by subsampling five times half of the samples.
• Figure 4: Semantic similarity with syntax ablation. The shaded colored areas represent 1 standard deviation, calculated by subsampling five times half of the samples.
• Figure 5: Decomposition of sentence vectors. Average fraction of squared norm from sentence activations $\boldsymbol{X}_i$ contained in syntax centroids $\boldsymbol{S}_i$ (blue) and semantic centroids $\boldsymbol{T}_i$ (green), across the network. The gray sections represent the residual fraction of norm that is not captured by any either centroid. The vertical axis is cut to 0.6 for visualization purposes only.