Human-AI Complementarity: A Goal for Amplified Oversight

TL;DR

This work investigates how humans and AI can complement each other to improve oversight of AI systems, with a focus on fact-verification of AI outputs. It introduces confidence-based hybridization, routing tasks to AI or humans based on AI confidence, and rater assistance where humans can use AI-generated traces, evidence, and judgments. Empirical results show that confidence-based hybridization can surpass AI or human ratings alone, and that AI assistance that presents targeted evidence (without overly leading explanations) best calibrates human reliance and improves accuracy. The findings support Amplified Oversight as a practical approach to supervise growing AI capabilities, highlighting the importance of careful interface design and measurement of reliance to realize human-AI complementarity in real-world evaluation and training pipelines.

Abstract

Human feedback is critical for aligning AI systems to human values. As AI capabilities improve and AI is used to tackle more challenging tasks, verifying quality and safety becomes increasingly challenging. This paper explores how we can leverage AI to improve the quality of human oversight. We focus on an important safety problem that is already challenging for humans: fact-verification of AI outputs. We find that combining AI ratings and human ratings based on AI rater confidence is better than relying on either alone. Giving humans an AI fact-verification assistant further improves their accuracy, but the type of assistance matters. Displaying AI explanation, confidence, and labels leads to over-reliance, but just showing search results and evidence fosters more appropriate trust. These results have implications for Amplified Oversight -- the challenge of combining humans and AI to supervise AI systems even as they surpass human expert performance.

Figures (11)

• Figure 1: Mean Accuracy of different rating protocols by Confidence Threshold. Humans alone (orange) achieved lower accuracy than AI alone (blue). Hybridized accuracy (green) varies by confidence threshold and there is a range of thresholds for which Hybridization achieves a statistically significantly higher accuracy than using only AI ratings. (NB: Human ratings are majority vote labels from the "T1" studies, described in Section \ref{['sec:t1']})
• Figure 2: Mean Individual Rater Accuracy, for the different assistant presentations. Accuracy is calculated over the Post-hybridized Human Set (i.e., restricted to examples where model confidence in the overall rating is <= .62). Dotted horizontal lines are Model Accuracy (60%) and Baseline unassisted accuracy (69.3%) to ease comparison. Error bars are 95% Bootstrapped confidence intervals of the mean. These were from "T2" studies described in Section \ref{['sec:t2']}.
• Figure 3: Mean Individual Rater Accuracy, for the different assistant presentations, split by whether the Overall Judgment generated by the fact-verification assistant was correct or incorrect (note that some forms of assistance did not display this judgment). Accuracy is calculated over the Post-hybridized Human Set (i.e., restricted to examples where model confidence in the overall judgment is <= .62). Dotted lines are Baseline accuracy for correct and incorrect to ease comparison. Error bars are 95% Bootstrapped CIs of the mean. These were from "T2" studies, described in Section \ref{['sec:t2']}.
• Figure 4: Mean accuracy and and 95% confidence intervals on the entire evaluation set of humans alone (yellow), AI alone (light blue), hybridization with unassisted humans (green), and hybridization with Evidence-assisted humans (dark blue). The assisted-hybridized approach is significantly more accurate than the Human alone, AI alone, and the unassisted-hybridized protocol.
• Figure 5: Mean Individual Rater Accuracy for the bonus and select T2 studies described in Sections \ref{['sec:bonus_desc']} and \ref{['sec:t2']}. Restricted to examples where model confidence <= .62.