Concept-Level Explainability for Auditing & Steering LLM Responses

TL;DR

This work introduces ConceptX, a concept-level coalition-based attribution framework for LLMs that identifies semantically rich input concepts influencing outputs. It uses ConceptNet-based concept extraction and a Monte Carlo Shapley-like procedure with context-preserving replacement strategies to produce faithful, human-aligned explanations and enable prompt-level steering without retraining. The method demonstrates improved faithfulness over token-based baselines and effective auditing of gender bias, as well as practical steering capabilities for sentiment control and jailbreak defense across multiple models. Its model-agnostic design and focus on semantics offer a transparent, scalable path toward safer and better-aligned LLMs, with potential extensions to combine concept- and token-level explanations and scale to global explanations.

Abstract

As large language models (LLMs) become widely deployed, concerns about their safety and alignment grow. An approach to steer LLM behavior, such as mitigating biases or defending against jailbreaks, is to identify which parts of a prompt influence specific aspects of the model's output. Token-level attribution methods offer a promising solution, but still struggle in text generation, explaining the presence of each token in the output separately, rather than the underlying semantics of the entire LLM response. We introduce ConceptX, a model-agnostic, concept-level explainability method that identifies the concepts, i.e., semantically rich tokens in the prompt, and assigns them importance based on the outputs' semantic similarity. Unlike current token-level methods, ConceptX also offers to preserve context integrity through in-place token replacements and supports flexible explanation goals, e.g., gender bias. ConceptX enables both auditing, by uncovering sources of bias, and steering, by modifying prompts to shift the sentiment or reduce the harmfulness of LLM responses, without requiring retraining. Across three LLMs, ConceptX outperforms token-level methods like TokenSHAP in both faithfulness and human alignment. Steering tasks boost sentiment shift by 0.252 versus 0.131 for random edits and lower attack success rates from 0.463 to 0.242, outperforming attribution and paraphrasing baselines. While prompt engineering and self-explaining methods sometimes yield safer responses, ConceptX offers a transparent and faithful alternative for improving LLM safety and alignment, demonstrating the practical value of attribution-based explainability in guiding LLM behavior.

Figures (8)

• Figure 1: ConceptX methodology illustrated with ConceptX$_{\text{B/A/R}}$-n: (1) extract input concepts, (2) use GPT-4o-mini to generate neutral replacements, and (3) compute the attribution $\varphi(c)$ of a concept $c$ by evaluating its contribution across concept coalitions S, based on how much it drives the LLM output toward the target response t. (3) is repeated N times (number of input concepts).
• Figure 2: Faithfulness scores on the Alpaca dataset. The y-axis shows the similarity between the original LLM response and the response generated using the sparse explanation. The sparsity threshold, varied from 0 to 1 along the x-axis, controls the fraction of the explanation that is retained.
• Figure 3: Rank distribution of the gender input concept by the explainability methods on our created GenderBias dataset (see details in \ref{['sec:general_settings']}).
• Figure 4: Summary of LLM steering after perturbing ConceptX's explanatory concept.
• Figure 5: Faithfulness scores on the SST-2 dataset. The y-axis shows the similarity between the original LLM response and the response generated using the sparse explanation. The sparsity threshold, varied from 0 to 1 along the x-axis, controls the fraction of the explanation that is retained.