Dropouts in Confidence: Moral Uncertainty in Human-LLM Alignment

TL;DR

This work investigates moral uncertainty in human–LLM alignment using binary trolley-problem judgments across 32 open-source models and 9 moral dimensions. It introduces an information-theoretic framework that decomposes uncertainty into total entropy, conditional entropy, and mutual information, and it demonstrates that inference-time attention dropout increases mutual information and total entropy, which in turn improves alignment with human moral preferences. The key finding is that higher model–input information flow (MI) correlates with closer human alignment, suggesting that intentionally injecting uncertainty can reduce overconfident, misaligned decisions. The results highlight a practical path toward uncertainty-aware AI that better reflects the variability of human moral judgments, while also emphasizing the need to manage resulting decision variability in high-stakes settings.

Abstract

Humans display significant uncertainty when confronted with moral dilemmas, yet the extent of such uncertainty in machines and AI agents remains underexplored. Recent studies have confirmed the overly confident tendencies of machine-generated responses, particularly in large language models (LLMs). As these systems are increasingly embedded in ethical decision-making scenarios, it is important to understand their moral reasoning and the inherent uncertainties in building reliable AI systems. This work examines how uncertainty influences moral decisions in the classical trolley problem, analyzing responses from 32 open-source models and 9 distinct moral dimensions. We first find that variance in model confidence is greater across models than within moral dimensions, suggesting that moral uncertainty is predominantly shaped by model architecture and training method. To quantify uncertainty, we measure binary entropy as a linear combination of total entropy, conditional entropy, and mutual information. To examine its effects, we introduce stochasticity into models via "dropout" at inference time. Our findings show that our mechanism increases total entropy, mainly through a rise in mutual information, while conditional entropy remains largely unchanged. Moreover, this mechanism significantly improves human-LLM moral alignment, with correlations in mutual information and alignment score shifts. Our results highlight the potential to better align model-generated decisions and human preferences by deliberately modulating uncertainty and reducing LLMs' confidence in morally complex scenarios.

Figures (16)

• Figure 1: Moral machine scenario and LLM uncertainty in binary choices. (a) Example trolley problem with binary collision choice. (b) LLM probabilities across scenarios $x_n$ under utilitarianism (top) and age (bottom) dimensions: overall 0.5, varying $\Delta p = |p_1 - p_2|$ (top: high-confidence $\uparrow$; bottom: low-confidence $\downarrow$). (c) Uncertainty decomposition: total entropy, conditional entropy, and mutual information.
• Figure 2: Distributions of LLM output probabilities, confidence, and uncertainty across data and models. Results are shown for two moral dimensions (Age, Utilitarianism) and three LLMs (Llama3‑8B, Qwen3‑14B, Gemma3‑12B) in (a) binary probabilities $p(y|x)$, (b) confidence, measured by $\Delta p^2$, (c) uncertainty, measured by binary entropy $\mathbb{H}(p)$. Mean values $\mu$ are indicated.
• Figure 3: Confidence ($\Delta{p}^2$) variation by models and moral dimensions, represented by the size and color of circles. Relative uncertainty variations differ significantly across models but exhibit little difference across moral dimensions.
• Figure 4: Dropout effects on uncertainty components and human-LLM moral alignment. (a) Effects of increasing dropout rate (0, 0.05, 0.1) on average total entropy (blue), conditional entropy (orange), and mutual information (green), with trend lines and p-values from paired t-tests (ns: non-significant; *: $p<0.05$; two-sided, Bonferroni corrected). Error bars represent standard errors across scenario–model combinations (n=9×32). Total entropy and mutual information increase with dropout, while conditional entropy remains almost unchanged. (b) Changes in human-LLM moral decision alignment ($\Delta L_2$) for models, sorted by decreasing $\Delta L_2$ (increased alignment), at dropout rates 0.05 (left, teal bars) and 0.1 (right, orange bars). (c) Example radar chart illustrating improved Alignment with AMCE values across nine moral dimensions. Human: gray, dashed line; Llama-3.1-70B (without dropout): red, solid line; Llama-3.1-70B (with dropout=0.1), blue, solid line.
• Figure 5: Uncertainty changes predict shifts in moral decision alignment under dropout. (a) Scatterplots of $\Delta$ uncertainty components (scenario-averaged) vs. $\Delta L_2$ between human and LLM. Per-model points at dropout=0.05 (blue) and 0.1 (orange). Pearson $r$, and Bonferroni-corrected $p$-values are shown at the top of the figures. (b) Model-wise trajectories of $\Delta$ mutual information vs. $\Delta L_2$ from dropout 0$\to$0.05 and 0$\to$0.1; gray lines connect points, showing larger mutual information increases link to better alignment.