SEED-SET: Scalable Evolving Experimental Design for System-level Ethical Testing

TL;DR

SEED-SET is proposed, a Bayesian experimental design framework that incorporates domain-specific objective evaluations, and subjective value judgments from stakeholders, and provides an interpretable and efficient trade-off between exploration and exploitation.

Abstract

As autonomous systems such as drones, become increasingly deployed in high-stakes, human-centric domains, it is critical to evaluate the ethical alignment since failure to do so imposes imminent danger to human lives, and long term bias in decision-making. Automated ethical benchmarking of these systems is understudied due to the lack of ubiquitous, well-defined metrics for evaluation, and stakeholder-specific subjectivity, which cannot be modeled analytically. To address these challenges, we propose SEED-SET, a Bayesian experimental design framework that incorporates domain-specific objective evaluations, and subjective value judgments from stakeholders. SEED-SET models both evaluation types separately with hierarchical Gaussian Processes, and uses a novel acquisition strategy to propose interesting test candidates based on learnt qualitative preferences and objectives that align with the stakeholder preferences. We validate our approach for ethical benchmarking of autonomous agents on two applications and find our method to perform the best. Our method provides an interpretable and efficient trade-off between exploration and exploitation, by generating up to $2\times$ optimal test candidates compared to baselines, with $1.25\times$ improvement in coverage of high dimensional search spaces.

Figures (18)

• Figure 1: SEED-SET Overview. Our framework integrates quantitative metrics learned using Objective GP (a) with user preferences learned via a Subjective GP through pairwise elicitation (b). An LLM performs pairwise comparisons of scenario outcomes (c) to inform the acquisition process (d), which generates a pair of scenarios for evaluation, aligned with user-defined ethical criteria. These scenarios are then used for system-level simulations (e) in a sequential manner.
• Figure 2: Environments for the two case studies considered in this work. (Left) Power Grid Allocation - IEEE 5-Bus and 30-Bus ( \ref{['subsec:power']}). (Right)Fire Rescue ( \ref{['subsec:fire']}). Additional case study for Optimal Routing in \ref{['appendix:optimal_routing']}.
• Figure 3: Power Grid Allocation Preference Scores. Preference scores baseline comparison for 5-Bus (left) and 30-Bus (right).
• Figure 4: Bus-30 Different Stakeholder Groups. We show that our learned preference GP is able to adapt to the needs of different potential stakeholder groups. The plots show data point for optimum preference score (shown in red) projected on objective space, with contours of predicted preference score for Stakeholder A (left) and B (right), with optimum value data point.
• Figure 5: Fire Rescue Preference Scores. We report preferences scores for baseline comparisons (left) and acquisition strategy ablations (right).