Morality is Contextual: Learning Interpretable Moral Contexts from Human Data with Probabilistic Clustering and Large Language Models

TL;DR

This work tackles context-dependent moral judgments by introducing COMETH, a framework that learns context-specific reward models from human judgments through a Probabilistic Context Learner, while leveraging LLM-derived semantic representations and an interpretable generalization module. The approach is validated on a 300-scenario dataset spanning six core actions, with 101 participants, and demonstrates a substantial improvement in alignment with human judgments over end-to-end LLM prompts. Key contributions include an empirically grounded moral-context dataset, a reproducible pipeline combining human data with model-based context learning and LLM semantics, and an interpretable mechanism to explain context-driven moral predictions. The findings suggest that structuring morality around context and features enables more reliable, transparent alignment of AI systems with human values, especially for smaller models, and offers a practical path toward context-aware moral reasoning in real-world applications.

Abstract

Moral actions are judged not only by their outcomes but by the context in which they occur. We present COMETH (Contextual Organization of Moral Evaluation from Textual Human inputs), a framework that integrates a probabilistic context learner with LLM-based semantic abstraction and human moral evaluations to model how context shapes the acceptability of ambiguous actions. We curate an empirically grounded dataset of 300 scenarios across six core actions (violating Do not kill, Do not deceive, and Do not break the law) and collect ternary judgments (Blame/Neutral/Support) from N=101 participants. A preprocessing pipeline standardizes actions via an LLM filter and MiniLM embeddings with K-means, producing robust, reproducible core-action clusters. COMETH then learns action-specific moral contexts by clustering scenarios online from human judgment distributions using principled divergence criteria. To generalize and explain predictions, a Generalization module extracts concise, non-evaluative binary contextual features and learns feature weights in a transparent likelihood-based model. Empirically, COMETH roughly doubles alignment with majority human judgments relative to end-to-end LLM prompting (approx. 60% vs. approx. 30% on average), while revealing which contextual features drive its predictions. The contributions are: (i) an empirically grounded moral-context dataset, (ii) a reproducible pipeline combining human judgments with model-based context learning and LLM semantics, and (iii) an interpretable alternative to end-to-end LLMs for context-sensitive moral prediction and explanation.

Figures (19)

• Figure 1: COMETH Pipeline. 101 participants answered an online survey which permit us to obtain moral judgments distributions for the 300 scenarios we generated. Then a pre-processing algorithm extracts the core action of the scenarios and group the scenarios sharing the same core action. The Probabilistic Context Learner then clustered scenarios with the same action into distinct moral contexts based on human judgment distributions, using adding and merging modules. To interpret and generalize, an LLM-based module extracted descriptive contextual features, which were binarized into feature vectors. Aggregate feature profiles were computed for each context, and feature weights were learned via a likelihood-based model. Finally, predictions of moral judgments were evaluated using a softmax-based scoring function. Colors in the figure represent different core actions.
• Figure 2: Clustering of moral scenarios by the Probabilistic Context Learner. Each subplot corresponds to a specific action type and displays the clustering performed by the agent. Points represent individual states derived from moral scenarios, while squares denote the clusters (or contexts) created by the agent. The size of each square is proportional to the number of states it contains. The coordinates of all elements are based on the probability distribution over the outcomes Support and Blame; the third outcome (Neutral) is implicitly defined as $1-P(Support)-P(Blame)$, allowing for a two-dimensional representation. The number and shape of the clusters emerge dynamically from the data and align with human intuition based on visual inspection of the same distributions.
• Figure 3: Temporal evolution of clusters for the action “To lie by interest.” This figure illustrates the agent’s continual learning process as new scenarios are introduced. Subplots show successive clustering states over time. When a new scenario is similar to an existing cluster, it is integrated and the cluster is updated (e.g., left panel). If the scenario is dissimilar, a new cluster is created (e.g., center panel, second cluster). When two clusters become sufficiently close, they are merged by the agent (e.g., right panel, last cluster). Clusters containing more scenarios exhibit increased stability and undergo smaller changes during updates.
• Figure 4: Comparison of alignment rates between the COMETH pipeline and end-to-end LLMs per action. Mean alignment rates across the prompts used (3 for the COMETH pipeline, 5 for end-to-end methods) are shown with standard deviations for each action, as well as the overall average across all actions. Results are presented for MistralAI/Mistral-7B-Instruct-v0.3, Meta-LLaMA/LLaMA-3.1-8B-Instruct, and Qwen/Qwen-3-Next-80B-A3B-Instruct (qwen2.5-1m), alongside human baseline data collected via an online survey.
• Figure 5: Feature weights obtained for the action "Practice Euthanasia" with Qwen/Qwen-3-Next-80B-A3B-Instruct (qwen2.5-1m). The plot reports the relative importance of each feature in shaping cluster assignments, highlighting how different attributes influence whether the scenario is more likely to be assigned to each cluster and then judged with support or blame.