Dual Mechanisms of Value Expression: Intrinsic vs. Prompted Values in LLMs

TL;DR

The paper investigates intrinsic versus prompted value expression in LLMs by mechanistically comparing their underlying representations. It introduces value vectors and value neurons as dual probes, extracting and orthogonalizing directions in the residual stream to separate shared and unique components. Across multiple models and languages, the study shows substantial overlap but also distinct components, with intrinsic mechanisms promoting lexical diversity and prompted mechanisms enhancing instruction-following and prompt compliance, including jailbreaking scenarios. The findings illuminate how value steering operates at vector and neuron levels, offering insights for value alignment and the trade-offs between intrinsic preference and prompt-driven control, with robust cross-language generalization and ecological evaluations like PVQ and Value Portrait. These insights have practical implications for designing safer, more controllable LLMs and for understanding the mechanistic consequences of system prompts versus intrinsic training signals.

Abstract

Large language models (LLMs) can express different values in two distinct ways: (1) intrinsic expression, reflecting the model's inherent values learned during training, and (2) prompted expression, elicited by explicit prompts. Given their widespread use in value alignment and persona steering, it is paramount to clearly understand their underlying mechanisms, particularly whether they mostly overlap (as one might expect) or rely on substantially different mechanisms, but this remains largely understudied. We analyze this at the mechanistic level using two approaches: (1) value vectors, feature directions representing value mechanisms extracted from the residual stream, and (2) value neurons, MLP neurons that contribute to value expressions. We demonstrate that intrinsic and prompted value mechanisms partly share common components that are crucial for inducing value expression, but also possess unique elements that manifest in different ways. As a result, these mechanisms lead to different degrees of value steerability (prompted > intrinsic) and response diversity (intrinsic > prompted). In particular, components unique to the intrinsic mechanism seem to promote lexical diversity in responses, whereas those specific to the prompted mechanism primarily strengthen instruction following, taking effect even in distant tasks like jailbreaking.

Figures (45)

• Figure 1: Overview of the extraction pipeline of intrinsic and prompted value vectors.
• Figure 2: Cosine similarity between intrinsic and prompted value vectors (layer 14). Highlighted sections show the overlap.
• Figure 3: Distribution of shared and unique neurons from layers 0 to 14, for the Conformity value. Distributions for other values can be found in Appendix \ref{['appendix_overlap_neuron']}.
• Figure 4: Example of a PVQ dataset steering experiment using the Universalism value vector (English). For more results, see Appendix \ref{['appendix:steering_experiments_pvq']}.
• Figure 5: Steering on the English version of the situational dilemmas dataset with Qwen2.5-7B-Instruct. Other languages and models are visible in Appendix \ref{['appendix:steering_experiments_situational_dilemmas']}.