A Framework for Human-Reason-Aligned Trajectory Evaluation in Automated Vehicles

TL;DR

The paper tackles everyday ethical decision-making in automated vehicles by moving beyond rare trolley-problem tests to model how AV trajectories align with the distinct reasons of multiple human agents under Meaningful Human Control (MHC). It introduces a unified reasons-based evaluation framework that maps agents' reasons to per-time-step scores $f_{ib}$, aggregates them into $F_{ib}$, combines across agents to obtain $S_w(T_a)$, and applies a fairness-adjusted balance function $B(\mathbf{w}, \mathbf{w}^*)$ to produce a final score $S(T_a) = B(\mathbf{w}, \mathbf{w}^*) \cdot S_w(T_a)$ for trajectory selection via $T^* = \arg\max S(T_a)$. The method is demonstrated on a real-world-inspired overtaking scenario with three agents (policymaker, driver, cyclist) and four candidate trajectories, revealing that even small shifts in agent weights can cause discrete changes in the preferred action, thereby highlighting the ethical sensitivity and need for transparent weight specifications. By integrating as an evaluation layer over existing planning stacks, the framework provides transparency, traceability, and a practical path toward Meaningful Human Control in everyday AV operation, with future work aimed at empirical weight elicitation and extension to richer multi-agent settings.

Abstract

One major challenge for the adoption and acceptance of automated vehicles (AVs) is ensuring that they can make sound decisions in everyday situations that involve ethical tension. Much attention has focused on rare, high-stakes dilemmas such as trolley problems. Yet similar conflicts arise in routine driving when human considerations, such as legality, efficiency, and comfort, come into conflict. Current AV planning systems typically rely on rigid rules, which struggle to balance these competing considerations and often lead to behaviour that misaligns with human expectations. This paper introduces a reasons-based trajectory evaluation framework that operationalises the tracking condition of Meaningful Human Control (MHC). The framework represents human agents reasons (e.g., regulatory compliance) as quantifiable functions and evaluates how well candidate trajectories align with them. It assigns adjustable weights to agent priorities and includes a balance function to discourage excluding any agent. To demonstrate the approach, we use a real-world-inspired overtaking scenario, which highlights tensions between compliance, efficiency, and comfort. Our results show that different trajectories emerge as preferable depending on how agents reasons are weighted, and small shifts in priorities can lead to discrete changes in the selected action. This demonstrates that everyday ethical decisions in AV driving are highly sensitive to the weights assigned to the reasons of different human agents.

Figures (6)

• Figure 1: Integration of the proposed human-reasons-based trajectory evaluation module into a hierarchical AV control architecture. The module does not generate or select trajectories; instead, it evaluates candidate trajectories produced by the global planner for alignment with human reasons. The global planner then uses these scores to select the trajectory that best satisfies both motion-planning and human-reason considerations.
• Figure 2: Illustration of the vehicle-cyclist overtaking scenario showing the initial configuration and possible trajectories
• Figure 3: Spatial visualisation of four candidate AV trajectories ($T_1$–$T_4$) relative to a cyclist. The trajectories vary in lateral clearance and lane usage, reflecting different prioritisation patterns across safety, efficiency, and legal compliance.
• Figure 4: Ternary plot showing the output of the balance function $B(\mathbf{w})$ across combinations of agent weights. Maximum balance occurs when all weights are equal.
• Figure 5: Trajectory scores for four candidate trajectories evaluated against agents' reasons. The red region shows historical score progression; the blue region begins when the score drops below 0.7, prompting trajectory reevaluation.