StructLens: A Structural Lens for Language Models via Maximum Spanning Trees

TL;DR

The findings demonstrate that StructLens yields an inter-layer similarity pattern that is distinctively different from conventional cosine similarity, and proves to be beneficial for practical tasks, such as layer pruning, highlighting the effectiveness of structural analysis for understanding and optimizing language models.

Abstract

Language exhibits inherent structures, a property that explains both language acquisition and language change. Given this characteristic, we expect language models to manifest internal structures as well. While interpretability research has investigated the components of language models, existing approaches focus on local inter-token relationships within layers or modules (e.g., Multi-Head Attention), leaving global inter-layer relationships largely overlooked. To address this gap, we introduce StructLens, an analytical framework designed to reveal how internal structures relate holistically through their inter-token connection within a layer. StructLens constructs maximum spanning trees based on the semantic representations in residual streams, analogous to dependency parsing, and leverages the tree properties to quantify inter-layer distance (or similarity) from a structural perspective. Our findings demonstrate that StructLens yields an inter-layer similarity pattern that is distinctively different from conventional cosine similarity. Moreover, this structure-aware similarity proves to be beneficial for practical tasks, such as layer pruning, highlighting the effectiveness of structural analysis for understanding and optimizing language models. Our code is available at https://github.com/naist-nlp/structlens.

Figures (14)

• Figure 1: Inter-layer similarity samples of Llama3.1 8B for each metric on MMLU. Bright color represents high similarity, while dark color represents low similarity.
• Figure 2: Inter-layer similarity samples of Qwen2.5 7B for each metric on MMLU. Bright color represents high similarity, while dark color represents low similarity.
• Figure 3: Samples of a contiguous and non-contiguous subtree.
• Figure 4: Visualization of the layer-wise evolution of the contiguous subtrees and tokens in them. The x-axis denotes layer depth as a percentage, and the y-axis denotes the contiguous subtree and token ratio.
• Figure 5: Logit lens visualization on MMLU. We visualize the token predictions for each of the last eight tokens in the input. The lower rows represent predictions from the lower layers, while the upper rows show predictions from the higher layers. Color intensity represents prediction probability.