Do Reasoning Models Ask Better Questions? A Formal Information-Theoretic Analysis on Multi-Turn LLM Games

TL;DR

This work tackles the problem of how to make LLM-based agents ask better questions in multi-turn settings. It introduces a formal, information-theoretic framework in which a hierarchical knowledge graph constrains a Guess My City game, and a triad of Seeker, Oracle, and Pruner agents collaborates to maximize Information Gain (IG) per turn. By computing $IG$ as the difference in Shannon entropy before and after each turn, the paper enables fine-grained, turn-level and cumulative evaluation of query efficiency across different model sizes and whether Chain-of-Thought reasoning is enabled, under fully vs. partially observable conditions. Across experiments, larger models with explicit reasoning consistently achieve higher IG and shorter solution paths, while smaller models compensate with broader exploration; the results offer insights into how reasoning strategies shape information-seeking behavior in LLM agents and how to design training or prompting schemes to maximize IG in interactive tasks.

Abstract

Large Language Models (LLMs) excel at many tasks but still struggle with a critical ability for LLM-based agents: asking good questions for resolving ambiguity in user requests. While prior work has explored information-seeking behavior through word games, existing benchmarks lack comprehensive evaluation frameworks that provide both final and intermediate signals based on Information Gain (IG). Moreover, they rarely provide systematic comparisons between models that use chain-of-thought reasoning and those that do not. We propose a multi-turn dialogue framework that quantitatively measures how effectively LLMs gather information through yes/no questions in a hierarchical knowledge graph environment. Our framework employs a triad of interacting LLM agents that ask questions, answer them, and update the hypothesis space. We adopt IG as the main metric, grounded in Shannon entropy, to assess query effectiveness at each turn and cumulatively. We instantiate our framework in a geographical Guess My City game setting organized in a five-level taxonomy and evaluate multiple LLM variants under fully and partially observable conditions, with and without Chain-of-Thought reasoning. Our experiments demonstrate that, among the evaluated models, the ones with explicit reasoning capabilities achieve higher IG per turn and reach solutions in fewer steps, particularly in partially observable settings. Analysis of reasoning traces reveals that smaller models compensate for limited capacity through more aggressive exploration of candidate questions, while larger models exhibit higher assertiveness in selecting optimal queries, generating candidates with greater potential IG.

Figures (3)

• Figure 1: Framework architecture. The numbered arrows indicate the sequential flow between the three agents and the hypothesis space.
• Figure 2: Example dialogue between the Seeker and Oracle agents. The Seeker asks yes/no questions to narrow down the hypothesis space, while the Oracle provides answers based on its knowledge of the target. The game terminates when the Seeker identifies the target.
• Figure 3: Aggregated Information Gain Over Time. Performance comparison of LLMs under fully observable (FO) and partially observable (PO) conditions. Each subplot reports average information gain per turn. Models with stronger reasoning yield higher information gain, especially early; performance drops as the hypothesis space narrows.