Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SCALPEL: Selective Capability Ablation via Low-rank Parameter Editing for Large Language Model Interpretability Analysis

Zihao Fu, Xufeng Duan, Zhenguang G. Cai

TL;DR

SCALPEL tackles the challenge of understanding and controlling distributed capability encoding in large language models by modeling capabilities as low-rank subspaces within the parameter space. It uses LoRA adapters with a probability equalization loss and a trio of regularizers to selectively ablate a target capability while preserving general language performance, revealing the low-dimensional structure of capabilities across layers. Experiments across 24 capability tasks and 67 BLiMP linguistic tasks show effective ablation with minimal perplexity increase and consistent cross-architecture generalization, uncovering distinct layer-wise distribution patterns for different cognitive functions. The work provides a principled, fine-grained interpretability framework with practical implications for safe deployment and targeted knowledge editing in high-stakes domains.

Abstract

Large language models excel across diverse domains, yet their deployment in healthcare, legal systems, and autonomous decision-making remains limited by incomplete understanding of their internal mechanisms. As these models integrate into high-stakes systems, understanding how they encode capabilities has become fundamental to interpretability research. Traditional approaches identify important modules through gradient attribution or activation analysis, assuming specific capabilities map to specific components. However, this oversimplifies neural computation: modules may contribute to multiple capabilities simultaneously, while single capabilities may distribute across multiple modules. These coarse-grained analyses fail to capture fine-grained, distributed capability encoding. We present SCALPEL (Selective Capability Ablation via Low-rank Parameter Editing for Large language models), a framework representing capabilities as low-rank parameter subspaces rather than discrete modules. Our key insight is that capabilities can be characterized by low-rank modifications distributed across layers and modules, enabling precise capability removal without affecting others. By training LoRA adapters to reduce distinguishing correct from incorrect answers while preserving general language modeling quality, SCALPEL identifies low-rank representations responsible for particular capabilities while remaining disentangled from others. Experiments across diverse capability and linguistic tasks from BLiMP demonstrate that SCALPEL successfully removes target capabilities while preserving general capabilities, providing fine-grained insights into capability distribution across parameter space. Results reveal that capabilities exhibit low-rank structure and can be selectively ablated through targeted parameter-space interventions, offering nuanced understanding of capability encoding in LLMs.

SCALPEL: Selective Capability Ablation via Low-rank Parameter Editing for Large Language Model Interpretability Analysis

TL;DR

SCALPEL tackles the challenge of understanding and controlling distributed capability encoding in large language models by modeling capabilities as low-rank subspaces within the parameter space. It uses LoRA adapters with a probability equalization loss and a trio of regularizers to selectively ablate a target capability while preserving general language performance, revealing the low-dimensional structure of capabilities across layers. Experiments across 24 capability tasks and 67 BLiMP linguistic tasks show effective ablation with minimal perplexity increase and consistent cross-architecture generalization, uncovering distinct layer-wise distribution patterns for different cognitive functions. The work provides a principled, fine-grained interpretability framework with practical implications for safe deployment and targeted knowledge editing in high-stakes domains.

Abstract

Large language models excel across diverse domains, yet their deployment in healthcare, legal systems, and autonomous decision-making remains limited by incomplete understanding of their internal mechanisms. As these models integrate into high-stakes systems, understanding how they encode capabilities has become fundamental to interpretability research. Traditional approaches identify important modules through gradient attribution or activation analysis, assuming specific capabilities map to specific components. However, this oversimplifies neural computation: modules may contribute to multiple capabilities simultaneously, while single capabilities may distribute across multiple modules. These coarse-grained analyses fail to capture fine-grained, distributed capability encoding. We present SCALPEL (Selective Capability Ablation via Low-rank Parameter Editing for Large language models), a framework representing capabilities as low-rank parameter subspaces rather than discrete modules. Our key insight is that capabilities can be characterized by low-rank modifications distributed across layers and modules, enabling precise capability removal without affecting others. By training LoRA adapters to reduce distinguishing correct from incorrect answers while preserving general language modeling quality, SCALPEL identifies low-rank representations responsible for particular capabilities while remaining disentangled from others. Experiments across diverse capability and linguistic tasks from BLiMP demonstrate that SCALPEL successfully removes target capabilities while preserving general capabilities, providing fine-grained insights into capability distribution across parameter space. Results reveal that capabilities exhibit low-rank structure and can be selectively ablated through targeted parameter-space interventions, offering nuanced understanding of capability encoding in LLMs.
Paper Structure (22 sections, 8 equations, 5 figures, 7 tables)

This paper contains 22 sections, 8 equations, 5 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Problem Formulation
  5. Low-Rank Adaptation Architecture
  6. Probability Equalization Loss
  7. Regularization Framework
  8. Analysis Methods
  9. Experiments
  10. Experiment Setup
  11. Datasets
  12. Baseline Methods
  13. Main Results
  14. Ablation Study on Regularization Components
  15. Cross-Architecture Generalization
  16. ...and 7 more sections

Figures (5)

  • Figure 1: Overview of the SCALPEL framework. Given a target capability, we train low-rank LoRA adapters to make the model equally confused between correct and incorrect answers, while text regularization preserves general language modeling quality. The resulting low-rank modifications reveal how the target capability is encoded across the model.
  • Figure 2: Multi-dimensional comparison of interpretability methods on the language translation task. The visualization shows the relationship between target accuracy degradation, model perplexity, and overall capability preservation across all baseline methods. SCALPEL (highlighted) achieves the optimal balance, positioned in the region of low perplexity and high capability retention while achieving the most effective capability removal.
  • Figure 3: Ablation study comparing SCALPEL configurations on language translation task. Left: Accuracy degradation shows targeted capability removal effectiveness. Right: Overall capability preservation demonstrates the impact of each regularization component on maintaining general language abilities. The full SCALPEL method (with all regularizations) achieves the optimal balance.
  • Figure 4: Peak layer analysis for capability tasks (left) and BLiMP tasks (right). Capability tasks show a progression from basic language tasks in early layers to complex reasoning in middle layers and creative tasks in late layers. BLiMP tasks reveal morphological processing in early layers, syntactic processing in later layers, and semantic tasks distributed throughout.
  • Figure 5: Dimensionality reduction visualization of task similarity in LoRA weight space. Left: Capability tasks showing clustering patterns among reasoning, knowledge, and linguistic domains. Right: BLiMP linguistic tasks (67 fine-grained linguistic phenomena) revealing structural relationships among syntax, semantics, morphology, and syntax-semantics interfaces.