Entropy-Tree: Tree-Based Decoding with Entropy-Guided Exploration

TL;DR

Entropy-Tree addresses the trade-offs in decoding strategies by guiding exploration to high-entropy decision points, enabling structured, prefix-sharing tree search rather than undirected sampling. It introduces branch point filtering and size-controlled tree expansion to balance exploration with efficiency, and uses leaf-derived outcomes to compute predictive entropy for uncertainty quantification. Across diverse reasoning benchmarks and model scales, Entropy-Tree improves pass@k accuracy and enhances calibration (AUROC) relative to Multi-chain, while maintaining fair computational budgets. This approach unifies efficient reasoning exploration with reliable uncertainty estimation, with practical implications for robust decision-making in LLM-based systems.

Abstract

Large language models achieve strong reasoning performance, yet existing decoding strategies either explore blindly (random sampling) or redundantly (independent multi-sampling). We propose Entropy-Tree, a tree-based decoding method that exploits entropy as a signal for branching decisions--expanding the search tree only at positions where the model exhibits genuine uncertainty. Entropy-Tree shows superior accuracy and calibration in reasoning tasks: it achieves better pass@k than Multi-chain across multiple models and datasets, and its predictive entropy demonstrates better AUROC compared to several traditional metrics. Entropy-Tree unifies efficient structured exploration and reliable uncertainty estimation within a single decoding procedure.

Figures (6)

• Figure 1: Entropy-Tree: Branching at high entropy tokens to form multiple decoding paths.
• Figure 2: The complete decoding process of Entropy-Tree.
• Figure 3: Decoding example of Entropy-Tree.
• Figure 4: The pass@k curve of Qwen2.5-7B-Instruct on MATH-500.
• Figure 5: AUROC of different methods on the GPQA-diamond.