Task Ambiguity in Humans and Language Models

TL;DR

This work investigates task ambiguity in language models by introducing AmbiBench, a benchmark of six ambiguously-specified sentence classification tasks designed to probe how users' instructions and examples influence task inference. The study shows that very large language models trained with human feedback data can approach or exceed human performance in resolving task ambiguity, while neither scale nor feedback alone suffices. Crucially, finetuning standard, non-HFD models on a small set of ambiguous in-context examples can dramatically improve generalization to unseen ambiguous tasks, offering a practical route to robust ambiguity handling. The results highlight the potential for targeted finetuning to teach models to reason over ambiguous instructions, with implications for safer, more reliable AI deployment across domains such as software engineering, law, and education.

Abstract

Language models have recently achieved strong performance across a wide range of NLP benchmarks. However, unlike benchmarks, real world tasks are often poorly specified, and agents must deduce the user's intended behavior from a combination of context, instructions, and examples. We investigate how both humans and models behave in the face of such task ambiguity by proposing AmbiBench, a new benchmark of six ambiguously-specified classification tasks. We evaluate humans and models on AmbiBench by seeing how well they identify the intended task using 1) instructions with varying degrees of ambiguity, and 2) different numbers of labeled examples. We find that the combination of model scaling (to 175B parameters) and training with human feedback data enables models to approach or exceed the accuracy of human participants across tasks, but that either one alone is not sufficient. In addition, we show how to dramatically improve the accuracy of language models trained without large-scale human feedback training by finetuning on a small number of ambiguous in-context examples, providing a promising direction for teaching models to generalize well in the face of ambiguity.

Figures (19)

• Figure 1: Complex tasks are often hard to specify precisely, leaving important pieces of information missing. Agents should be able to fill in the blanks by combining information from instructions and examples in order to identify the intended behavior.
• Figure 2: The best HFD model (text-davinci-003) approaches human accuracy for both uninformative and informative instructions. Accuracy of humans and other models for tasks prompted with an instruction and two in-context examples. Error bars show 95% bootstrap CIs.
• Figure 3: The best HFD models (text-davinci-002 and text-davinci-003) outperform human participants at disambiguating the intended task. Accuracy as the number of examples in the in-context window grows. Surprisingly, the smaller curie model reliably outperforms the larger davinci model across the examples. In addition, the HFD training hurts at curie scale, but dramatically helps at davinci scale. Shaded regions are 95% bootstrap CIs.
• Figure 4: Finetuning on ambiguous in-context examples dramatically improves accuracy on unseen tasks that are ambiguously specified. Accuracy after finetuning davinci on ambiguous and non-ambiguous (control) in-context examples. Models are finetuned on 272 examples from four tasks, then evaluated on the two held-out tasks (subfigure captions). Shaded regions are 95% bootstrap CIs.
• Figure 5: Even models that perform very well on a task struggle to describe that task in words.