Tracing Moral Foundations in Large Language Models

TL;DR

The results provide mechanistic evidence that moral concepts in LLMs are distributed, layered, and partly disentangled, suggesting that pluralistic moral structure can emerge as a latent pattern from the statistical regularities of language alone.

Abstract

Large language models (LLMs) often produce human-like moral judgments, but it is unclear whether this reflects an internal conceptual structure or superficial ``moral mimicry.'' Using Moral Foundations Theory (MFT) as an analytic framework, we study how moral foundations are encoded, organized, and expressed within two instruction-tuned LLMs: Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct. We employ a multi-level approach combining (i) layer-wise analysis of MFT concept representations and their alignment with human moral perceptions, (ii) pretrained sparse autoencoders (SAEs) over the residual stream to identify sparse features that support moral concepts, and (iii) causal steering interventions using dense MFT vectors and sparse SAE features. We find that both models represent and distinguish moral foundations in a structured, layer-dependent way that aligns with human judgments. At a finer scale, SAE features show clear semantic links to specific foundations, suggesting partially disentangled mechanisms within shared representations. Finally, steering along either dense vectors or sparse features produces predictable shifts in foundation-relevant behavior, demonstrating a causal connection between internal representations and moral outputs. Together, our results provide mechanistic evidence that moral concepts in LLMs are distributed, layered, and partly disentangled, suggesting that pluralistic moral structure can emerge as a latent pattern from the statistical regularities of language alone.

Figures (13)

• Figure 1: Overview of the experimental pipeline. (i) Relative moral concept vectors are constructed from extended Moral Foundations vignettes and serve as a central representational hub. These vectors are validated in parallel through (ii) topological alignment with human-labeled Reddit post distributions and (iii) mechanistic decomposition into sparse autoencoder features. (iv) We then causally intervene on model activations via macro- and micro-steering and assess behavioral shifts and capability preservation.
• Figure 2: The geometry of moral alignment in LLAMA. We project human-labeled Reddit posts for each moral foundation, and non-moral data, onto the corresponding foundation-vs.-Social Norm vectors.
• Figure 3: Layer-wise Alignment of Moral Features in LLAMA. Average cosine similarity of the top-3 most aligned SAE features for each Moral Foundation across every 4 layers vs random baselines. Similarity is calculated between the SAE decoder weights and the corresponding Foundation vs. Social Norms concept vector.
• Figure 4: Steering results. For each foundation, we plot the MFQ-2 score change $\Delta \text{Score}(\alpha)$ relative to the unsteered baseline ($\alpha=0$) as a function of steering strength $\alpha$, evaluated at that foundation's best layer. Points show measured $\Delta$ scores and the solid line shows the corresponding linear trend. The gray dashed line reports general performance (MMLU) under the same interventions. See Appendix \ref{['appx:Steering']} for details.
• Figure 5: The geometry of moral alignment in Qwen-2.5-7B-Instruct. We project human-labeled Reddit posts for each moral foundation, and non-moral data, onto the corresponding foundation-vs.-Social Norm vectors.