Uncertainty of Thoughts: Uncertainty-Aware Planning Enhances Information Seeking in Large Language Models

TL;DR

Uncertainty of Thoughts (UoT) introduces an uncertainty-aware planning framework that enables large language models to actively seek information by asking informative questions. By coupling an uncertainty-aware future simulation, information-gain rewards, and a reward-propagation scheme, UoT selects questions that maximize expected uncertainty reduction. Across five datasets and three tasks (20 Questions, medical diagnosis, and troubleshooting), UoT consistently improves task success rates (average +38.1%) and reduces the number of questions, demonstrating robust information-seeking capabilities across diverse LLMs. The work also provides an open-set benchmark and analyzes reward design and efficiency, offering a practical path toward more reliable AI assistants in uncertain real-world settings.

Abstract

In the face of uncertainty, the ability to *seek information* is of fundamental importance. In many practical applications, such as medical diagnosis and troubleshooting, the information needed to solve the task is not initially given and has to be actively sought by asking follow-up questions (for example, a doctor asking a patient for more details about their symptoms). In this work, we introduce Uncertainty of Thoughts (UoT), an algorithm to augment large language models with the ability to actively seek information by asking effective questions. UoT combines 1) an *uncertainty-aware simulation approach* which enables the model to simulate possible future scenarios and how likely they are to occur, 2) *uncertainty-based rewards* motivated by information gain which incentivizes the model to seek information, and 3) a *reward propagation scheme* to select the optimal question to ask in a way that maximizes the expected reward. In experiments on medical diagnosis, troubleshooting, and the `20 Questions` game, UoT achieves an average performance improvement of 38.1% in the rate of successful task completion across multiple LLMs compared with direct prompting and also improves efficiency (i.e., the number of questions needed to complete the task). Our code has been released [here](https://github.com/zhiyuanhubj/UoT)

Figures (6)

• Figure 1: The importance of information seeking in medical diagnosis. The patient initially only complains of a headache, but by asking the right questions, the doctor uncovers the critical information needed for a correct diagnosis.
• Figure 2: UoT Overview: UoT includes three components: (a) Question Generation and Simulation, where an LLM proposes candidate questions and simulates future scenarios; (b) Uncertainty-based Rewards, measuring the uncertainty reduction from answers to a question, and (c) Reward Propagation computing accumulated rewards $R_a$ over past questions, and expected rewards $R_e$ capturing expected future gains. The process ends by choosing questions with the highest expected reward.
• Figure 2: Average success rates for 20Q, MD, and TB at comparable efficiency, measured by GPT-4 token use. $k$ is sampling count, $D$ is tree depth.
• Figure 3: Case studies from the 20 Questions game (left) and simplified medical diagnosis (right).
• Figure 4: Success rate comparison between Adapted-ToT and Adapted-ToT using uncertainty reward, and between UoT and UoT without uncertainty reward.

Theorems & Definitions (2)

• Proposition 1
• proof