Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Keep Calm and Avoid Harmful Content: Concept Alignment and Latent Manipulation Towards Safer Answers

Ruben Belo, Marta Guimaraes, Claudia Soares

TL;DR

The paper tackles jailbreak and harmful-content generation in large language models by introducing CALM, an inference-time method that suppresses harmful concepts through latent-space manipulation. By extending Concept Whitening to language models and coupling it with an orthogonal rotation $Q$ and a diagonal projection $P$, CALM aligns and then silences harmful latent directions in the final-layer embeddings, incurring only $O(d^2)$ per decoding step without retraining. Empirical results across Harmful Q&A, Harmful Chat, and AdvBench demonstrate that CALM increases the perplexity of unsafe outputs while preserving safe content, often outperforming ProFS baselines and offering interpretable concept axes. The approach provides a lightweight, modular safety mechanism that can augment existing safeguards, with future work focusing on finer-grained concepts, model-specific tuning, and multilingual generalization.

Abstract

Large Language Models are susceptible to jailbreak attacks that bypass built-in safety guardrails (e.g., by tricking the model with adversarial prompts). We propose Concept Alignment and Concept Manipulation CALM, an inference-time method that suppresses harmful concepts by modifying latent representations of the last layer of the model, without retraining. Leveraging concept whitening technique from Computer Vision combined with orthogonal projection, CALM removes unwanted latent directions associated with harmful content while preserving model performance. Experiments show that CALM reduces harmful outputs and outperforms baseline methods in most metrics, offering a lightweight approach to AI safety with no additional training data or model fine-tuning, while incurring only a small computational overhead at inference.

Keep Calm and Avoid Harmful Content: Concept Alignment and Latent Manipulation Towards Safer Answers

TL;DR

The paper tackles jailbreak and harmful-content generation in large language models by introducing CALM, an inference-time method that suppresses harmful concepts through latent-space manipulation. By extending Concept Whitening to language models and coupling it with an orthogonal rotation and a diagonal projection , CALM aligns and then silences harmful latent directions in the final-layer embeddings, incurring only per decoding step without retraining. Empirical results across Harmful Q&A, Harmful Chat, and AdvBench demonstrate that CALM increases the perplexity of unsafe outputs while preserving safe content, often outperforming ProFS baselines and offering interpretable concept axes. The approach provides a lightweight, modular safety mechanism that can augment existing safeguards, with future work focusing on finer-grained concepts, model-specific tuning, and multilingual generalization.

Abstract

Large Language Models are susceptible to jailbreak attacks that bypass built-in safety guardrails (e.g., by tricking the model with adversarial prompts). We propose Concept Alignment and Concept Manipulation CALM, an inference-time method that suppresses harmful concepts by modifying latent representations of the last layer of the model, without retraining. Leveraging concept whitening technique from Computer Vision combined with orthogonal projection, CALM removes unwanted latent directions associated with harmful content while preserving model performance. Experiments show that CALM reduces harmful outputs and outperforms baseline methods in most metrics, offering a lightweight approach to AI safety with no additional training data or model fine-tuning, while incurring only a small computational overhead at inference.
Paper Structure (27 sections, 12 equations, 12 figures, 10 tables)

This paper contains 27 sections, 12 equations, 12 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work.
  3. Our work.
  4. CALM: Concept Alignment and Latent Manipulation
  5. How to remove concepts?
  6. Experimental Setup
  7. Datasets.
  8. Models.
  9. Results
  10. Harmful Q&A
  11. Harmful Chat
  12. Effect of CALM in generation
  13. Toxicity and Harmfulness.
  14. How Receptive Models Are to CALM
  15. Interpretability
  16. ...and 12 more sections

Figures (12)

  • Figure 1: Example of a harmful prompt and the corresponding answers. The Baseline Response (left/red) provides a neutral analysis, while the calm Response (right/green) reframes the prompt with context, consequences, and alternative solutions. This is a summarized version, with the full answers shown in Fig. \ref{['fig:front_page_full_example']}, and additional examples provided in App. \ref{['sec:appendix_strings']}.
  • Figure 2: calm is applied to token embeddings from the final decoder layer, using whitening for decorrelation and an orthogonal rotation to align concept directions with canonical axes. The aligned representations enable (1) interpretability and (2) projection to remove undesired concepts, after which inverse transformations restore the embeddings for continued generation.
  • Figure 3: Activations of two example answers projected onto 10 concept axes. The left example shows a detailed, multi-step terrorist attack plan, with arrows indicating the exact position where the word “bomb” appears. The right example illustrates potential uses of stolen credit cards.
  • Figure 4: CALM vs CALM-noAlign
  • Figure 5: Perplexity (PPL) results on the Harmful Q&A dataset. This breakdown shows how varying the number of learned concepts in profs and calm affects the PPL of safe and unsafe answers. Higher PPL for unsafe responses, combined with lower PPL for safe ones and reduced Unsafe Win Rate (UWR), indicates better alignment. calm consistently yields sharper increases in harmful PPL while preserving safe PPL, highlighting the benefits of whitening and decorrelation for disentangling concepts. "--" indicates PPL values exceeding 150.
  • ...and 7 more figures