Towards Developing Ethical Reasoners: Integrating Probabilistic Reasoning and Decision-Making for Complex AI Systems

TL;DR

The paper tackles the problem of building robust ethical reasoners for AI by proposing a holistic framework that combines intermediate representations with probabilistic reasoning and knowledge representation to handle real-world uncertainty. It introduces circumstantial dicta and ethical prescripts as core, formal elements, and develops a probabilistic structure with $P(C)$, $P(E|C)$, and $P(C,E)$ to drive context-aware decisions. A normalized collection of ethical profiles encoded as matrices with entries $m_{ij} = w(e_i,c_j) P(e_i|c_j)$ supports efficient retrieval via ensemble coding and clustering, while non-deterministic action selection, expected utility maximization, and multi-objective optimization balance contextual factors and ethical priorities. Theoretical foundations include five theorems on consistency, optimality, robustness, convergence, and alignment with human judgments, offering a pathway to verifiable, human-aligned decision-making. The framework aims to enable scalable, ethically reliable AI across single-agent and multi-agent settings, with emphasis on autonomous systems navigating complex moral dilemmas through principled, context-sensitive reasoning.

Abstract

A computational ethics framework is essential for AI and autonomous systems operating in complex, real-world environments. Existing approaches often lack the adaptability needed to integrate ethical principles into dynamic and ambiguous contexts, limiting their effectiveness across diverse scenarios. To address these challenges, we outline the necessary ingredients for building a holistic, meta-level framework that combines intermediate representations, probabilistic reasoning, and knowledge representation. The specifications therein emphasize scalability, supporting ethical reasoning at both individual decision-making levels and within the collective dynamics of multi-agent systems. By integrating theoretical principles with contextual factors, it facilitates structured and context-aware decision-making, ensuring alignment with overarching ethical standards. We further explore proposed theorems outlining how ethical reasoners should operate, offering a foundation for practical implementation. These constructs aim to support the development of robust and ethically reliable AI systems capable of navigating the complexities of real-world moral decision-making scenarios.

Figures (1)

• Figure 1: Three scenarios (Scenario 1, Scenario 2, and Scenario 3) each with a choice between two possible routes (A and B) for a self-driving vehicle are depicted in the figure above.