Understanding Refusal in Language Models with Sparse Autoencoders

TL;DR

This work investigates how refusals are encoded in instruction-tuned LLMs by identifying latent refusal features with sparse autoencoders and validating their causal influence on generation. By combining Attribution Patching and Activation Steering within an SAE feature space, the authors extract a minimal, faithful set of refusal mediators and demonstrate that harm features can act as upstream triggers for refusal, while adversarial jailbreaks suppress these features. The study shows that refusal features are separable from harm features and that these features improve generalization to out-of-distribution adversarial samples when used in linear probes. The results offer a mechanistic view of refusal, reveal interactions between harm and refusal, and provide practical tools for debugging safety behavior, with code released for reproducibility.

Abstract

Refusal is a key safety behavior in aligned language models, yet the internal mechanisms driving refusals remain opaque. In this work, we conduct a mechanistic study of refusal in instruction-tuned LLMs using sparse autoencoders to identify latent features that causally mediate refusal behaviors. We apply our method to two open-source chat models and intervene on refusal-related features to assess their influence on generation, validating their behavioral impact across multiple harmful datasets. This enables a fine-grained inspection of how refusal manifests at the activation level and addresses key research questions such as investigating upstream-downstream latent relationship and understanding the mechanisms of adversarial jailbreaking techniques. We also establish the usefulness of refusal features in enhancing generalization for linear probes to out-of-distribution adversarial samples in classification tasks. We open source our code in https://github.com/wj210/refusal_sae.

Figures (16)

• Figure 1: Intervening on upstream harmful features suppresses the downstream refusal features, leading to refusal. The features often correlates with the token which they activates on.
• Figure 2: Jailbreak scores between feature search baselines across the $3$ harmful datasets.
• Figure 3: Normalized jailbreak scores across different harmful behaviors by intervening on different subsets of feature. Common features appears to enable jailbreak well across different behaviors as opposed to specific ones.
• Figure 4: (a) depicts the suppression of each token and the effects of clamp $F_H$. (b) shows the values across each token.
• Figure 5: Process layout for attribution patching. [Left]: Two forward passes to retrieve corrupted and clean feature activations. One backward pass for the clean gradients.[Top Right]: Instead of using a separate $d_{corrupt}$ input, we approximate feature activations derived from $d_{corrupt}$ with $\text{AS}(d_{clean})$ at every residual state. [Bottom Right]: Eq. \ref{['eq:ap']}.