Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Partial Reasoning in Language Models: Search and Refinement Guided by Uncertainty

Murilo da Luz, Bruno Brandão, Luana Martins, Gustavo Oliveira, Bryan de Oliveira, Luckeciano Melo, Telma Soares

TL;DR

The paper tackles the challenge of reliable multi-step reasoning in large language models by introducing PREGU, an entropy-driven framework that interrupts autoregressive generation at uncertainty hotspots and triggers a localized latent-space refinement via Soft Reasoning. By monitoring the Shannon entropy of next-token distributions, $H_t$, and applying a threshold $\tau$, PREGU generates multiple partial reasoning prefixes and refines them in latent space to select the best result, combining breadth with depth through Bayesian Optimization guided by Expected Improvement. Evaluations on LLaMA-3-8B, Mistral-7B, and Qwen2-7B across GSM8K, GSM-Hard, SVAMP, and StrategyQA show that PREGU matches or exceeds Soft Reasoning, with strong gains on long, multi-step tasks like GSM-Hard. The work demonstrates that internal uncertainty signals can serve as effective metacognitive cues to allocate computational effort toward the most informative refinements, improving reasoning reliability and efficiency in practical settings. It also provides insights into the linguistic manifestations of uncertainty, linking high-entropy moments to structural transitions in reasoning.

Abstract

The use of Large Language Models (LLMs) for reasoning and planning tasks has drawn increasing attention in Artificial Intelligence research. Despite their remarkable progress, these models still exhibit limitations in multi-step inference scenarios, particularly in mathematical and logical reasoning. We introduce PREGU (Partial Reasoning Guided by Uncertainty). PREGU monitors the entropy of the output distribution during autoregressive generation and halts the process whenever entropy exceeds a defined threshold, signaling uncertainty. From that point, a localized search is performed in the latent space to refine the partial reasoning and select the most coherent answer, using the Soft Reasoning method. Experiments conducted with LLaMA-3-8B, Mistral-7B, and Qwen2-7B across four reasoning benchmarks (GSM8K, GSM-Hard, SVAMP, and StrategyQA) showed performance greater than or similar to Soft Reasoning, indicating that entropy can serve as an effective signal to trigger selective refinement during reasoning.

Partial Reasoning in Language Models: Search and Refinement Guided by Uncertainty

TL;DR

The paper tackles the challenge of reliable multi-step reasoning in large language models by introducing PREGU, an entropy-driven framework that interrupts autoregressive generation at uncertainty hotspots and triggers a localized latent-space refinement via Soft Reasoning. By monitoring the Shannon entropy of next-token distributions, , and applying a threshold , PREGU generates multiple partial reasoning prefixes and refines them in latent space to select the best result, combining breadth with depth through Bayesian Optimization guided by Expected Improvement. Evaluations on LLaMA-3-8B, Mistral-7B, and Qwen2-7B across GSM8K, GSM-Hard, SVAMP, and StrategyQA show that PREGU matches or exceeds Soft Reasoning, with strong gains on long, multi-step tasks like GSM-Hard. The work demonstrates that internal uncertainty signals can serve as effective metacognitive cues to allocate computational effort toward the most informative refinements, improving reasoning reliability and efficiency in practical settings. It also provides insights into the linguistic manifestations of uncertainty, linking high-entropy moments to structural transitions in reasoning.

Abstract

The use of Large Language Models (LLMs) for reasoning and planning tasks has drawn increasing attention in Artificial Intelligence research. Despite their remarkable progress, these models still exhibit limitations in multi-step inference scenarios, particularly in mathematical and logical reasoning. We introduce PREGU (Partial Reasoning Guided by Uncertainty). PREGU monitors the entropy of the output distribution during autoregressive generation and halts the process whenever entropy exceeds a defined threshold, signaling uncertainty. From that point, a localized search is performed in the latent space to refine the partial reasoning and select the most coherent answer, using the Soft Reasoning method. Experiments conducted with LLaMA-3-8B, Mistral-7B, and Qwen2-7B across four reasoning benchmarks (GSM8K, GSM-Hard, SVAMP, and StrategyQA) showed performance greater than or similar to Soft Reasoning, indicating that entropy can serve as an effective signal to trigger selective refinement during reasoning.
Paper Structure (17 sections, 2 equations, 2 figures, 1 table)

This paper contains 17 sections, 2 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Latent Space and Soft Reasoning
  4. Entropy as a Measure of Uncertainty
  5. The PREGU Method
  6. Minimum Prefix ($t_{\min}$):
  7. Experimental Setup
  8. Configuration
  9. Benchmarks.
  10. Baselines.
  11. Hyperparameters.
  12. Results
  13. Empirical Validation of Uncertainty
  14. Conclusion and Future Work
  15. PREGU - Experiments
  16. ...and 2 more sections

Figures (2)

  • Figure 1: Overview of the PREGU architecture. The pre-trained LLM ($M_{\phi}$) generates multiple partial reasoning paths in the token space. When the entropy of the output distribution ($H_t$) exceeds the threshold ($\tau$), the generation halts, marking a region of uncertainty. Each partial reasoning sequence is then refined in the latent space using the Soft Reasoning method, producing candidate responses ($R_1, R_2, \ldots, R_n$), from which the highest-reward answer is selected.
  • Figure 2: Word cloud of tokens with high average entropy observed in the SVAMP dataset. Token size represents frequency and color encodes the average entropy (in bits), highlighting terms such as “For”, “Since”, and “Given”, which are associated with logical transitions in reasoning.