Optimizing Delegation in Collaborative Human-AI Hybrid Teams

TL;DR

The paper addresses how to optimize delegation in hybrid human-AI teams by introducing an RL-based manager that selects which agent should control the team at intervention points under constraint-based safety and performance criteria. It formalizes the manager’s problem using an Intervening MDP (IMDP) built on absorbing Markov chain concepts, enabling the manager to observe only endpoint states and outcomes while remaining independent from agents’ internal rewards. The framework is instantiated in a driving scenario using a CARLO-like 2D simulator, with perception-context degradations (Decayed Sensing and Failed Perception) to stress-test the manager. Empirically, the manager improves team performance, achieving up to approximately 187% of the best solo agent performance in certain contexts, while substantially reducing the need for interventions. The work demonstrates a scalable, constraint-driven approach to managing delegation in heterogeneous, pre-trained agent teams and outlines future directions toward more realistic 3D domains and multi-agent settings.

Abstract

When humans and autonomous systems operate together as what we refer to as a hybrid team, we of course wish to ensure the team operates successfully and effectively. We refer to team members as agents. In our proposed framework, we address the case of hybrid teams in which, at any time, only one team member (the control agent) is authorized to act as control for the team. To determine the best selection of a control agent, we propose the addition of an AI manager (via Reinforcement Learning) which learns as an outside observer of the team. The manager learns a model of behavior linking observations of agent performance and the environment/world the team is operating in, and from these observations makes the most desirable selection of a control agent. We restrict the manager task by introducing a set of constraints. The manager constraints indicate acceptable team operation, so a violation occurs if the team enters a condition which is unacceptable and requires manager intervention. To ensure minimal added complexity or potential inefficiency for the team, the manager should attempt to minimize the number of times the team reaches a constraint violation and requires subsequent manager intervention. Therefore our manager is optimizing its selection of authorized agents to boost overall team performance while minimizing the frequency of manager intervention. We demonstrate our manager performance in a simulated driving scenario representing the case of a hybrid team of agents composed of a human driver and autonomous driving system. We perform experiments for our driving scenario with interfering vehicles, indicating the need for collision avoidance and proper speed control. Our results indicate a positive impact of our manager, with some cases resulting in increased team performance up to ~187% that of the best solo agent performance.

Figures (7)

• Figure 1: Collaborative hybrid team dynamic with single agent control. Controlling agent remains in control until constraint violation, which cues the manager to make new a delegation of authority.
• Figure 2: Sample Driving environment (including rectangular buildings, sidewalks, and cars) with sample path.
• Figure 3: Sensing contexts illustrating conditions which may impact the performance/likelihood of detection of other vehicles. Reduced detection increases the likelihood of collision or other driving failures.
• Figure 4: Sensing contexts illustrating conditions which may impact the performance/likelihood of detection of other vehicles. Reduced detection increases the likelihood of collision or other driving failures.
• Figure 5: Sample first three branch steps to generate trajectories $\tau_{s_i}\in\mathcal{T}$