Value of Information: A Framework for Human-Agent Communication

TL;DR

The paper tackles the problem of underspecified user requests in autonomous LLM agents by introducing a decision-theoretic framework based on the Value of Information (VoI). It formalizes Clarify-or-Commit as a sequential decision process and provides an inference-time VoI method that estimates belief distributions, simulates user responses, and selects queries only when the expected downstream utility gain justifies the communication cost, all without hyperparameter tuning. The approach is instantiated with LLMs and validated across four domains—20 Questions, medical diagnosis, flight booking, and e-commerce—showing that VoI consistently matches or exceeds manually tuned baselines, with notable gains in high-cost settings (up to 1.36 utility points). The work demonstrates a practical foundation for adaptive human–AI collaboration, balancing task risk, query ambiguity, and user effort, and it highlights implications for safer, more efficient interaction and future extensions to fully open-ended dialogue.

Abstract

Large Language Model (LLM) agents deployed for real-world tasks face a fundamental dilemma: user requests are underspecified, yet agents must decide whether to act on incomplete information or interrupt users for clarification. Existing approaches either rely on brittle confidence thresholds that require task-specific tuning, or fail to account for the varying stakes of different decisions. We introduce a decision-theoretic framework that resolves this trade-off through the Value of Information (VoI), enabling agents to dynamically weigh the expected utility gain from asking questions against the cognitive cost imposed on users. Our inference-time method requires no hyperparameter tuning and adapts seamlessly across contexts-from casual games to medical diagnosis. Experiments across four diverse domains (20 Questions, medical diagnosis, flight booking, and e-commerce) show that VoI consistently matches or exceeds the best manually-tuned baselines, achieving up to 1.36 utility points higher in high-cost settings. This work provides a parameter-free framework for adaptive agent communication that explicitly balances task risk, query ambiguity, and user effort.

Figures (19)

• Figure 1: Illustration of different communication methods and user reaction. Given user flight history, an LLM agent is able to infer user latent preferences with some probability. Excessive questions that asks about every aspect of preference would lead to user dissatisfaction (A) while directly acting without communication could lead to unexpected consequences (B). Decision-theoretic reasoning can balance expected utility gain via asking user questions against communication cost to achieve efficient but effective communication at inference time (C).
• Figure 2: Utility vs. Communication Rounds. Final utility as a function of the number of clarification questions asked across our three tasks, for GPT-4 (top two rows) and Gemini-2.5-Flash (bottom two rows), with communication costs $c=0.01$ and $c=0.05$. Utility is defined as $U(\theta,a) - T \cdot c$. The curves for Fixed Round and Confidence Thresholding represent Pareto frontiers generated by varying their respective hyperparameters ($k$ and $\tau$). In contrast, our VoI agent (starred) is a parameter-free method. In nearly all settings, VoI automatically identifies an operating point that matches or exceeds the performance of the best-tuned baseline, demonstrating its superior adaptability and practical value.
• Figure 3: Calibration Analysis The figure presents the calibration analysis of GPT-4 and Gemini-2.5-Flash on Animal Guessing, Medical Diagnoiss, and Flight Recommendation. (In (c) the accuracy for predicted probability between 0 and 0.2 is omitted because very few samples fall in that range.
• Figure 4: A side by side comparison for different methods for Mixed 20 Question task. The figure contrasts four controllers---No-Ask, Fixed-Round, Confidence Thresholding ($\tau=0.90$), and our VOI policy---on a single Mixed 20Q instance with communication cost $c=0.05$. Task stakes are encoded directly in the terminal utility: a correct animal guess yields reward $1$ (low stakes), whereas a correct medical diagnosis yields reward $10$ (high stakes). The objective maximizes decision utility minus dialogue cost, $U(\theta, a) - c(\xi)$.
• Figure 5: GPT-4: results for different methods and thresholds across three tasks. For Webshop, LLM is normalized by 10 and utilities are $\text{Util}=\text{LLM}-\#T \times \{0.01,0.05\}$. Mixed 20Q utilities are recomputed per spec. Within each method, the best utility is underlined. The global best per task/cost is bold+italic and the second best is bold.