Understanding Post-Training Structural Changes in Large Language Models

TL;DR

This work reveals two consistent regularities in large language models after post-training: near-uniform geometric scaling of singular values and highly coordinated orthogonal rotations of left and right singular vectors across layers. By modeling post-training as a reparameterization of pretrained subspaces, it shows that singular-value scaling acts as a temperature-like effect while the core functional change arises from coordinated subspace rotations. Experimental validations, including singular value replacement and ablation/restoration tests, demonstrate the necessity and sufficiency of orthogonal coherence for maintaining performance. The findings provide a global, parameter-space perspective on post-training dynamics, with implications for initialization, convergence, and model fingerprinting. Together, they offer a principled framework for understanding and predicting how post-training reshapes LLM parameter space.

Abstract

Post-training fundamentally alters the behavior of large language models (LLMs), yet its impact on the internal parameter space remains poorly understood. In this work, we conduct a systematic singular value decomposition (SVD) analysis of principal linear layers in pretrained LLMs, focusing on two widely adopted post-training methods: instruction tuning and long-chain-of-thought (Long-CoT) distillation. Our analysis reveals two consistent and unexpected structural changes:(1) a near-uniform geometric scaling of singular values across layers, which theoretically modulates attention scores; and (2) highly consistent orthogonal transformations are applied to the left and right singular vectors of each matrix. Disrupting this orthogonal consistency leads to catastrophic performance degradation. Based on these findings, we propose a simple yet effective framework that interprets post-training as a reparameterization of fixed subspaces in the pretrained parameter space. Further experiments reveal that singular value scaling behaves as a secondary effect, analogous to a temperature adjustment, whereas the core functional transformation lies in the coordinated rotation of singular vectors. These results challenge the prevailing view of the parameter space in large models as a black box, uncovering the first clear regularities in how parameters evolve during training, and providing a new perspective for deeper investigation into model parameter changes.

Figures (26)

• Figure 1: A simple but effective mathematical approximation to describe the effect of post-training on the parameter space. Performing SVD on weight matrices in the base model, post-training is equivalent to performing linear scaling on singular values and performing consistent orthogonal transformations on left and right singular vectors.
• Figure 2: The heatmaps of SVSMs comparing $\mathcal{M}_\text{base}$ with $\mathcal{M}'_\text{base}$, $\mathcal{M}_\text{Instruct}$ and $\mathcal{M}_\text{reasoning}$. (a) indicates no regular pattern in the distribution of scaling factors between $\mathcal{M}'_\text{base}$ and $\mathcal{M}_\text{base}$. In both (b) and (c), the principal scaling exhibits a near-uniform distribution. While in (c), scaling factors of $W_O$ are significantly higher than those of other matrix types.
• Figure 3: An example showing the orthogonality of singular vector similarity to the transformation performed. Only the first 25 dimensions are retained for clearer visualization. (a) shows the singular vector behavior of $W_O$ in the first Transformer block. Difference matrix (③) represents ${|sim^{(0)}_{U}-sim^{(0)}_{V}|}$, which is almost a zero matrix. ④ is ${I^{(0)}_{orth}}$ of ${W^{(0)}_O}$. Most of its diagonal elements are close to 1, and the rest are basically 0. (b) extensively verifies the approximate equality of ${Q^{(i)}_1}$ and ${Q^{(i)}_2}$ comparing $\mathcal{M}_\text{base}$ to ${\mathcal{M}'_\text{base}}$ and $\mathcal{M}_\text{post}$.
• Figure 4: Visualization of the average attention patterns before and after replacing the singular values. ① in (a) shows the original attention heads, while ② presents the averaged attention heads from the modified model. ③ illustrates the differences between the original and modified attention patterns. Panel (b) suggests that this behavior corresponds to a modulation of attention entropy.
• Figure 5: An example of model responses under three different settings. The ablation model outputs all garbled characters, while the restoration model reconstructs the features of the original model through the orthogonal matrix of the input subspaces.