Expected Moral Shortfall for Ethical Competence in Decision-making Models

TL;DR

This work tackles the problem of embedding ethical competence into AI decision-making by introducing the Expected Moral Shortfall (EMS), a tail-risk–based loss derived from $ES$ that penalizes morally weak actions. It combines a discretized moral framework with normative ethics weights and context-sensitive thresholds to compute an ethical judgment score $EJ$ and a cutoff $\tau$, and integrates EMS into learning via the loss $L = L_{BCE} + \lambda L_{EMS}$. The authors compare three strategies for moral integration (penalized weights, post-hoc overrides, and EMS loss) across diverse model families on two real-world datasets, highlighting tradeoffs between accuracy, complexity, and ethical competence. Findings show EMS can closely approximate hard ethical constraints while preserving performance in many scenarios, with larger tail-risk emphasis producing stronger ethical constraints but potential performance costs; overriding achieves the strongest ethics but eliminates learning. This framework offers a tunable, theory-aware path toward practically impactful ethically aware AI, particularly in high-stakes domains like admissions and lending.

Abstract

Moral cognition is a crucial yet underexplored aspect of decision-making in AI models. Regardless of the application domain, it should be a consideration that allows for ethically aligned decision-making. This paper presents a multifaceted contribution to this research space. Firstly, a comparative analysis of techniques to instill ethical competence into AI models has been presented to gauge them on multiple performance metrics. Second, a novel mathematical discretization of morality and a demonstration of its real-life application have been conveyed and tested against other techniques on two datasets. This value is modeled as the risk of loss incurred by the least moral cases, or an Expected Moral Shortfall (EMS), which we direct the AI model to minimize in order to maximize its performance while retaining ethical competence. Lastly, the paper discusses the tradeoff between preliminary AI decision-making metrics such as model performance, complexity, and scale of ethical competence to recognize the true extent of practical social impact.