CAST: Compositional Analysis via Spectral Tracking for Understanding Transformer Layer Functions

TL;DR

This work presents CAST, a probe-free framework that analyzes transformer layer functions through direct linear transformation estimation and spectral tracking. By estimating layer-to-layer transformation matrices with $T_i = \widetilde{H}_i^{\dagger} \widetilde{H}_{i+1}$ and applying six spectral metrics, CAST reveals distinct encoder-vs-decoder processing strategies and a three-phase layer architecture, further enriched by kernel analysis via Random Fourier Features and CKA to capture nonlinearities and cross-layer similarity. Across GPT-2, RoBERTa, Llama, and DeepSeek-R1, the results show decoders exhibit compression-expansion cycles while encoders maintain high-rank processing, with middle layers showing the strongest nonlinear transformations. These insights can guide layer pruning and architecture design, offering a complementary, mathematically grounded view to existing interpretability methods, though CAST remains a supplementary tool alongside probe-based and mechanistic analyses. Overall, CAST advances interpretable language model development by formalizing transformation dynamics as a central object of study and demonstrating robust, architecture-dependent patterns across multiple models.

Abstract

Large language models have achieved remarkable success but remain largely black boxes with poorly understood internal mechanisms. To address this limitation, many researchers have proposed various interpretability methods including mechanistic analysis, probing classifiers, and activation visualization, each providing valuable insights from different perspectives. Building upon this rich landscape of complementary approaches, we introduce CAST (Compositional Analysis via Spectral Tracking), a probe-free framework that contributes a novel perspective by analyzing transformer layer functions through direct transformation matrix estimation and comprehensive spectral analysis. CAST offers complementary insights to existing methods by estimating the realized transformation matrices for each layer using Moore-Penrose pseudoinverse and applying spectral analysis with six interpretable metrics characterizing layer behavior. Our analysis reveals distinct behaviors between encoder-only and decoder-only models, with decoder models exhibiting compression-expansion cycles while encoder models maintain consistent high-rank processing. Kernel analysis further demonstrates functional relationship patterns between layers, with CKA similarity matrices clearly partitioning layers into three phases: feature extraction, compression, and specialization.

Figures (11)

• Figure 1: Overview of the CAST framework. Given a large language model with multiple layers, we focus on layer $L_i$ and estimate the linear transformation between its input $H_i$ and output $H_{i+1}$ representations. Spectral analysis reveals patterns across layer types: early layers maintain high effective rank, middle layers perform aggressive compression, late layers show moderate specialization.
• Figure 2: Comparison of CAST with complementary methods across four architectures. From left to right: CAST Effective Rank reveals architecture-specific compression patterns, Logit Lens Entropy tracks prediction-space evolution, Tuned Lens Entropy captures refined prediction dynamics, Attention Entropy shows attention focusing patterns, and DirectProbe Anisotropy measures representation isotropy. The comparison demonstrates that CAST captures unique structural properties.
• Figure 3: Singular value distributions across layers. GPT-2 shows pronounced compression at middle layers with steep spectral decay, RoBERTa maintains gentle decay curves preserving information, Llama exhibits moderate compression patterns, while DeepSeek-R1 shows sustained high-rank representations across its layers with gradual spectral evolution.
• Figure 4: Comparison of CAST metrics. Decoder models exhibit compression-expansion cycles while encoder models maintain consistent high-rank processing.
• Figure 5: Random Fourier Features analysis reveals complementary patterns to linear analysis with consistently lower residual norms, demonstrating enhanced nonlinear transformation modeling capabilities.