Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Uncertain Machine Ethics Planning

Simon Kolker, Louise A. Dennis, Ramon Fraga Pereira, Mengwei Xu

TL;DR

The paper tackles planning under both outcome and moral uncertainty in machine ethics by formalizing uncertain ethical planning as MMMDP/MMSSP and applying MEHR within a Multi-Objective AO* framework. It introduces a running insulin-theft case to demonstrate how conflicting moral theories can be modeled, reasoned about, and comparatively evaluated using history-based arguments and attacks. The main contributions are the MMMDP/MMSSP formalism, the MEHR-based argumentation mechanism for policy selection, and a two-stage Moral Planner that first finds Pareto-undominated policies and then selects the ethically preferable one under a budgeted, non-moral cost. The work advances explainability in ethical AI by grounding decisions in reproducible argumentative structures, while highlighting computational challenges and avenues for efficiency improvements and broader application.

Abstract

Machine Ethics decisions should consider the implications of uncertainty over decisions. Decisions should be made over sequences of actions to reach preferable outcomes long term. The evaluation of outcomes, however, may invoke one or more moral theories, which might have conflicting judgements. Each theory will require differing representations of the ethical situation. For example, Utilitarianism measures numerical values, Deontology analyses duties, and Virtue Ethics emphasises moral character. While balancing potentially conflicting moral considerations, decisions may need to be made, for example, to achieve morally neutral goals with minimal costs. In this paper, we formalise the problem as a Multi-Moral Markov Decision Process and a Multi-Moral Stochastic Shortest Path Problem. We develop a heuristic algorithm based on Multi-Objective AO*, utilising Sven-Ove Hansson's Hypothetical Retrospection procedure for ethical reasoning under uncertainty. Our approach is validated by a case study from Machine Ethics literature: the problem of whether to steal insulin for someone who needs it.

Uncertain Machine Ethics Planning

TL;DR

The paper tackles planning under both outcome and moral uncertainty in machine ethics by formalizing uncertain ethical planning as MMMDP/MMSSP and applying MEHR within a Multi-Objective AO* framework. It introduces a running insulin-theft case to demonstrate how conflicting moral theories can be modeled, reasoned about, and comparatively evaluated using history-based arguments and attacks. The main contributions are the MMMDP/MMSSP formalism, the MEHR-based argumentation mechanism for policy selection, and a two-stage Moral Planner that first finds Pareto-undominated policies and then selects the ethically preferable one under a budgeted, non-moral cost. The work advances explainability in ethical AI by grounding decisions in reproducible argumentative structures, while highlighting computational challenges and avenues for efficiency improvements and broader application.

Abstract

Machine Ethics decisions should consider the implications of uncertainty over decisions. Decisions should be made over sequences of actions to reach preferable outcomes long term. The evaluation of outcomes, however, may invoke one or more moral theories, which might have conflicting judgements. Each theory will require differing representations of the ethical situation. For example, Utilitarianism measures numerical values, Deontology analyses duties, and Virtue Ethics emphasises moral character. While balancing potentially conflicting moral considerations, decisions may need to be made, for example, to achieve morally neutral goals with minimal costs. In this paper, we formalise the problem as a Multi-Moral Markov Decision Process and a Multi-Moral Stochastic Shortest Path Problem. We develop a heuristic algorithm based on Multi-Objective AO*, utilising Sven-Ove Hansson's Hypothetical Retrospection procedure for ethical reasoning under uncertainty. Our approach is validated by a case study from Machine Ethics literature: the problem of whether to steal insulin for someone who needs it.
Paper Structure (9 sections, 11 equations, 4 figures, 2 tables, 2 algorithms)

This paper contains 9 sections, 11 equations, 4 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Related Work
  4. Multi-Moral Markov Decision Process
  5. Machine Ethics Hypothetical Retrospection over Policies
  6. A Non-moral Extension
  7. Moral Planner
  8. Experiment
  9. Discussion

Figures (4)

  • Figure 1: A company may produce a beauty care product or research a drug with 50% probability of saving lives. Beauty care has negative retrospection (like regret) for missing the chance to save lives; if research is unsuccessful, there is no negative retrospection since risk was accepted at decision-time. Arguments are generated from each outcome. Directed edges are attacks indicating negative retrospection.
  • Figure 2: In $s_0$ Hal has no insulin; $s_1$ Hal dies waiting; $s_2$ both live; $s_3$ Hal dies; $s_4$ Carla dies; $s_5$ both die.
  • Figure 3: Shows notation. Worth function $W$ maps state-time to worth. Multi-worth function $\boldsymbol{W}$ holds many worth functions. Worth-vectors $\vec{w}$ store worth across considerations. From Section \ref{['sec:algorithm']}, $\vec{W}$ maps state-time to a set of worth-vectors.
  • Figure 4: MEHR argumentation graph. Boxes are arguments from policy-history pairs, $Arg(\pi,h)$. Directed edges are attacks, meaning $\Psi_m(Arg(\pi', h'), Arg(\pi', h'))$. Red/solid edge for utility attack; blue/dashed line for no-stealing attack. Probabilities on attacked arguments sum to policy's non-acceptability $\mathcal{N}(\pi )$.

Theorems & Definitions (5)

  • Definition 1: Multi-Moral Markov Decision Process
  • Definition 2: Moral Consideration
  • Definition 3: Moral Theory
  • Definition 4: Multi-Moral Stochastic Shortest Path Problem
  • Definition 5: Worth Vector Pareto Dominance