Dynamically Scaled Activation Steering

TL;DR

DSAS introduces a universal framework that decouples when to apply activation steering from how to apply it, enabling context-aware, per-layer and per-token scaling of steering interventions. It learns per-layer strength regulators via PCA-regularized logistic regression on activation statistics and can be combined with existing steering methods, including end-to-end training with LinEAS using a Wasserstein-based objective and a control regularizer. Empirically, DSAS improves the toxicity-versus-utility Pareto front across large language models and steering families and generalizes to diffusion models for selective concept modulation, all with minimal runtime overhead. The work also demonstrates end-to-end variants (E2E-DSAS) and discusses robustness to noisy supervision, reproducibility, and cross-modal applicability. Overall, DSAS offers a practical, interpretable, and broadly applicable approach to fine-grained, input-aware behavioral control in generative systems.

Abstract

Activation steering has emerged as a powerful method for guiding the behavior of generative models towards desired outcomes such as toxicity mitigation. However, most existing methods apply interventions uniformly across all inputs, degrading model performance when steering is unnecessary. We introduce Dynamically Scaled Activation Steering (DSAS), a method-agnostic steering framework that decouples when to steer from how to steer. DSAS adaptively modulates the strength of existing steering transformations across layers and inputs, intervening strongly only when undesired behavior is detected. At generation time, DSAS computes context-dependent scaling factors that selectively adjust the strength of any steering method. We also show how DSAS can be jointly optimized end-to-end together with the steering function. When combined with existing steering methods, DSAS consistently improves the Pareto front with respect to steering alone, achieving a better trade-off between toxicity mitigation and utility preservation. We further demonstrate DSAS's generality by applying it to a text-to-image diffusion model, showing how adaptive steering allows the modulation of specific concepts. Finally, DSAS introduces minimal computational overhead while improving interpretability, pinpointing which tokens require steering and by how much.

Figures (24)

• Figure 1: DSAS dynamically scales the intervention strengths applied to each token. Vanilla activation steering with the common strategy of applying a global strength $\lambda$, induces pear regardless of the input prompt (left). Our DSAS adapts the per-token strength of any steering technique to work only conditional to some aspect of the input, in this example, only when the concept fruit is present (right). Note how pear does not appear in (middle) since the prompt is not about fruits.
• Figure 2: Pareto fronts for toxicity mitigation vs. capability retention.Left:$\text{Tox}_\text{TET}$ vs. MMLU accuracy for CAA, ITI and, LinEAS, both with and without DSAS. Right:$\text{Tox}_\text{TET}$ vs. $\text{PPL}_{\text{Wik}}$ for the same methods. For each original method, and in each DSAS-conditioned model, we vary the global intervention strength $\lambda$ to draw the Pareto front. We set $\lambda = [0,...,10]$ for all methods and clip the Y axis when perplexity increases by 3 points to discard nonsensical generations. In both models, applying DSAS consistently improves the trade-off between toxicity reduction and capability retention.
• Figure 3: Left: Average CLIPScore and IMGScore for 6 target concepts toward blurriness under increasing global steering strengths $\lambda$ with CAA. Right: CLIPScore and IMGScore under CAA+DSAS. Whereas CAA affects both concept and non-concept-related images equally, CAA+DSAS blurs concept-related images while better preserving non-concept-related ones.
• Figure 4: Examples of 6 generated images from validation prompts: 3 banana-related (top) and 3 non-banana-related (bottom). For each prompt, the first row shows generations with CAA across $\lambda \in \{0, 0.25, 0.5, 0.75, 1\}$, and the second row shows the same for CAA+DSAS. While CAA introduces blurriness in all cases, CAA+DSAS selectively blurs banana-related images only.
• Figure 5: Effect of the number of training samples on validation performance. The figure reports the mean validation accuracy and its standard deviation over 100 random repetitions for each sample size.

Theorems & Definitions (3)

• Definition 1: Source set
• Definition 2: Target set
• Definition 3: Control set