Scaling Laws for Moral Machine Judgment in Large Language Models

TL;DR

The paper tests whether moral-judgment capabilities in large language models scale with model size. Using 75 configurations (0.27B–1000B parameters) and the Moral Machine AMCE framework, it finds a consistent power-law relationship: larger models align more closely with human moral preferences, with $D \propto S^{-0.10}$ and $R^2 = 0.50$. Extended reasoning models provide an additional ~16% improvement beyond size alone, while temporal improvements in data show no substantial effect. These findings establish a quantitative foundation for deploying AI with value-based judgments and suggest that scale, along with reasoning architectures, governs the emergence and reliability of moral alignment.

Abstract

Autonomous systems increasingly require moral judgment capabilities, yet whether these capabilities scale predictably with model size remains unexplored. We systematically evaluate 75 large language model configurations (0.27B--1000B parameters) using the Moral Machine framework, measuring alignment with human preferences in life-death dilemmas. We observe a consistent power-law relationship with distance from human preferences ($D$) decreasing as $D \propto S^{-0.10\pm0.01}$ ($R^2=0.50$, $p<0.001$) where $S$ is model size. Mixed-effects models confirm this relationship persists after controlling for model family and reasoning capabilities. Extended reasoning models show additional 16\% improvement beyond scale effects. The relationship holds across diverse architectures, while variance decreases at larger scales, indicating systematic emergence of more reliable moral judgment with computational scale. These findings extend scaling law research to value-based judgments and provide empirical foundations for artificial intelligence governance.

Figures (4)

• Figure 1: Scaling relationship between model size ($S$) and moral judgment alignment with human preferences (distance from human, $D$). Each point represents one LLM, colored by model family. The dashed line shows the fitted power-law relationship ($D \propto S^{-0.10\pm0.01}$)
• Figure S1: Family-specific scaling relationships. Log-log plot showing the relationship between model size and distance from human preferences for each model family. Points represent individual models, lines show linear regression fits with 95% confidence intervals. All families exhibit negative scaling relationships, demonstrating that the power-law pattern is not driven by any single architectural approach. Model families: DeepSeek (red circles), Gemma (blue triangles), Llama (green squares), Qwen (purple diamonds), Other (brown crosses).
• Figure S2: Effect of extended reasoning capabilities on moral alignment. Comparison of scaling relationships between standard models (gray circles) and extended reasoning models (red triangles). Extended reasoning models show systematically better alignment (lower distance) at comparable sizes, with a significant effect of approximately 16% improvement after controlling for model size ($p = 0.008$). Lines show linear regression fits with 95% confidence intervals.
• Figure S3: Temporal trends in moral alignment. Residual distance from human preferences (after controlling for model size) plotted against release date. The absence of a significant trend (Spearman $\rho = 0.047$, $p = 0.69$) suggests that improvements in training data quality or methods over time do not contribute substantially to alignment beyond the effects of scale and reasoning capabilities. Points represent individual models, line shows linear regression fit with 95% confidence interval.