Towards Understanding and Improving Refusal in Compressed Models via Mechanistic Interpretability

TL;DR

This work investigates how safety and refusal mechanisms change when large language models are compressed. Using a mechanistic interpretability pipeline, it shows that refusal behavior in both pruning- and quantization-based compressions is mediated by a single direction in the residual stream, with pruning altering the direction and source more than quantization. The authors introduce AIRD, a lightweight method that orthogonalizes projections to the original refusal direction, significantly reducing attack success on AdvBench without harming zero-shot performance. They also demonstrate that quantization tends to preserve the original refusal mechanism, offering a mechanistic explanation for why quantized models maintain safety better than pruned ones. The study provides a practical safety-enhancement technique for compressed models and highlights directions for future work in extending mechanistic interpretability to new architectures.

Abstract

The rapid growth of large language models has spurred significant interest in model compression as a means to enhance their accessibility and practicality. While extensive research has explored model compression through the lens of safety, findings suggest that safety-aligned models often lose elements of trustworthiness post-compression. Simultaneously, the field of mechanistic interpretability has gained traction, with notable discoveries, such as the identification of a single direction in the residual stream mediating refusal behaviors across diverse model architectures. In this work, we investigate the safety of compressed models by examining the mechanisms of refusal, adopting a novel interpretability-driven perspective to evaluate model safety. Furthermore, leveraging insights from our interpretability analysis, we propose a lightweight, computationally efficient method to enhance the safety of compressed models without compromising their performance or utility.

Figures (7)

• Figure 1: Interpretability Pipeline for comparing refusal in Compressed vs Base models.
• Figure 2: Artificially Inducing Refusal Direction (AIRD) pipeline for increasing safety of compressed models.
• Figure 3: Attack score (ASR) after directional ablation in Llama2-7b compressed model. Ablating the refusal direction increases the attack score significantly.
• Figure 4: Refusal Score on harmless prompts after activation addition (ActAdd) in compressed Llama2-7b model. Activation addition causes the model to refuse to answer.
• Figure 5: Necessity Test for Llama3-8b-instruct: Attack Score(ASR) after direction ablation vs no intervention on harmful instructions