Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Complementary Policies for Human-AI Teams

Ruijiang Gao, Maytal Saar-Tsechansky, Maria De-Arteaga

TL;DR

This work advances human-AI collaboration by introducing Learning Complementary Policies for Human-AI Augmentation (lcp-hai), a deferral-based framework that jointly learns an AI policy and a routing mechanism to maximize team rewards when outcomes are only observed under assigned actions. It formally defines the problem, develops doubly robust estimators for policy evaluation, and proves regret guarantees for the joint policy optimization. Through synethic data, IHDP, and a WebShop shopping assistant task, the authors show that a well-designed complementary AI tailored to human weaknesses and a smart deferral router can significantly outperform both humans and standalone AI, often with only a small fraction of tasks routed to humans. The work highlights practical managerial implications, such as the diversity bonus and robustness in imperfect environments, and discusses future avenues including online adaptation and fairness considerations.

Abstract

This paper tackles the critical challenge of human-AI complementarity in decision-making. Departing from the traditional focus on algorithmic performance in favor of performance of the human-AI team, and moving past the framing of collaboration as classification to focus on decision-making tasks, we introduce a novel approach to policy learning. Specifically, we develop a robust solution for human-AI collaboration when outcomes are only observed under assigned actions. We propose a deferral collaboration approach that maximizes decision rewards by exploiting the distinct strengths of humans and AI, strategically allocating instances among them. Critically, our method is robust to misspecifications in both the human behavior and reward models. Leveraging the insight that performance gains stem from divergent human and AI behavioral patterns, we demonstrate, using synthetic and real human responses, that our proposed method significantly outperforms independent human and algorithmic decision-making. Moreover, we show that substantial performance improvements are achievable by routing only a small fraction of instances to human decision-makers, highlighting the potential for efficient and effective human-AI collaboration in complex management settings.

Learning Complementary Policies for Human-AI Teams

TL;DR

This work advances human-AI collaboration by introducing Learning Complementary Policies for Human-AI Augmentation (lcp-hai), a deferral-based framework that jointly learns an AI policy and a routing mechanism to maximize team rewards when outcomes are only observed under assigned actions. It formally defines the problem, develops doubly robust estimators for policy evaluation, and proves regret guarantees for the joint policy optimization. Through synethic data, IHDP, and a WebShop shopping assistant task, the authors show that a well-designed complementary AI tailored to human weaknesses and a smart deferral router can significantly outperform both humans and standalone AI, often with only a small fraction of tasks routed to humans. The work highlights practical managerial implications, such as the diversity bonus and robustness in imperfect environments, and discusses future avenues including online adaptation and fairness considerations.

Abstract

This paper tackles the critical challenge of human-AI complementarity in decision-making. Departing from the traditional focus on algorithmic performance in favor of performance of the human-AI team, and moving past the framing of collaboration as classification to focus on decision-making tasks, we introduce a novel approach to policy learning. Specifically, we develop a robust solution for human-AI collaboration when outcomes are only observed under assigned actions. We propose a deferral collaboration approach that maximizes decision rewards by exploiting the distinct strengths of humans and AI, strategically allocating instances among them. Critically, our method is robust to misspecifications in both the human behavior and reward models. Leveraging the insight that performance gains stem from divergent human and AI behavioral patterns, we demonstrate, using synthetic and real human responses, that our proposed method significantly outperforms independent human and algorithmic decision-making. Moreover, we show that substantial performance improvements are achievable by routing only a small fraction of instances to human decision-makers, highlighting the potential for efficient and effective human-AI collaboration in complex management settings.
Paper Structure (24 sections, 5 theorems, 21 equations, 2 figures, 8 tables)

This paper contains 24 sections, 5 theorems, 21 equations, 2 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Deferral Collaboration
  4. Background on Policy Learning from Observational Data
  5. Problem Statement
  6. AI Policy for Policy Learning from Observational Data
  7. Learning Complementary Policies for Human-AI Collaboration lcp-hai
  8. Problem Statement
  9. Develop Human-AI collaboration Methods with Observational Data
  10. Regret Guarantees
  11. Best AI may not be the best human teammate
  12. Experiments
  13. Baselines
  14. Double Robustness
  15. Infant Health and Development Program (IHDP)
  16. ...and 9 more sections

Key Result

Proposition 1

Under ass:unconfoundness, ass:overlap, and SUTVA, for a given routing and decision policy $\phi,\pi$, $\hat{\theta}_{\text{DR}}$ is a consistent estimator of $\theta$ if $f$ is a consistent estimator of $Y(X,T)$ or $\hat{\pi}_0$ is a consistent estimator of $\pi_0$.

Figures (2)

  • Figure 1: Learning Complementary Policies for Human-AI augmentation lcp-hai. The routing model and AI policy are jointly optimized on the observational data. After the Human-AI system is deployed, the routing model will select between AI and the human to solve the task. The figure demonstrates a case where a human decision maker is chosen to solve the task, the human then selects a treatment and the requester receives a reward under the assigned treatment.
  • Figure 2: Illustration of Different Policies on the Synthetic Data. (a): The potential outcomes of the data-generating process; (b): Observed reward under the human policy; (c): Observed reward under the best independent AI policy; (d): Observed reward under the Predict-then-Collaborate system; (e): Observed reward under the joint Human-AI system.

Theorems & Definitions (11)

  • Definition 1: Regret
  • Proposition 1: Doubly-Robustness
  • Definition 2
  • Definition 3
  • Definition 4
  • Definition 5
  • Definition 6
  • Lemma 1: Rademacher Complexity
  • Lemma 2
  • Lemma 3
  • ...and 1 more