Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Martingale Foresight Sampling: A Principled Approach to Inference-Time LLM Decoding

Huayu Li, ZhengXiao He, Siyuan Tian, Jinghao Wen, Ao Li

TL;DR

This work reframes LLM decoding as identifying an optimal stochastic process, addressing the myopic nature of token-by-token generation. It introduces Martingale Foresight Sampling (MFS), which uses Doob Decomposition for principled step valuation, Optional Stopping for adaptive pruning, and Martingale Convergence for adaptive stopping, to guide inference-time decoding. The resulting framework achieves state-of-the-art accuracy and improved efficiency across six reasoning benchmarks, outperforming prior foresight-based methods like φ-Decoding while reducing computational cost. These results demonstrate the viability of a theoretically grounded, principled approach to decoding that leverages core martingale theory to improve robustness and scalability in reasoning tasks.

Abstract

Standard autoregressive decoding in large language models (LLMs) is inherently short-sighted, often failing to find globally optimal reasoning paths due to its token-by-token generation process. While inference-time strategies like foresight sampling attempt to mitigate this by simulating future steps, they typically rely on ad-hoc heuristics for valuing paths and pruning the search space. This paper introduces Martingale Foresight Sampling (MFS), a principled framework that reformulates LLM decoding as a problem of identifying an optimal stochastic process. By modeling the quality of a reasoning path as a stochastic process, we leverage Martingale theory to design a theoretically-grounded algorithm. Our approach replaces heuristic mechanisms with principles from probability theory: step valuation is derived from the Doob Decomposition Theorem to measure a path's predictable advantage, path selection uses Optional Stopping Theory for principled pruning of suboptimal candidates, and an adaptive stopping rule based on the Martingale Convergence Theorem terminates exploration once a path's quality has provably converged. Experiments on six reasoning benchmarks demonstrate that MFS surpasses state-of-the-art methods in accuracy while significantly improving computational efficiency. Code will be released at https://github.com/miraclehetech/EACL2026-Martingale-Foresight-Sampling.

Martingale Foresight Sampling: A Principled Approach to Inference-Time LLM Decoding

TL;DR

This work reframes LLM decoding as identifying an optimal stochastic process, addressing the myopic nature of token-by-token generation. It introduces Martingale Foresight Sampling (MFS), which uses Doob Decomposition for principled step valuation, Optional Stopping for adaptive pruning, and Martingale Convergence for adaptive stopping, to guide inference-time decoding. The resulting framework achieves state-of-the-art accuracy and improved efficiency across six reasoning benchmarks, outperforming prior foresight-based methods like φ-Decoding while reducing computational cost. These results demonstrate the viability of a theoretically grounded, principled approach to decoding that leverages core martingale theory to improve robustness and scalability in reasoning tasks.

Abstract

Standard autoregressive decoding in large language models (LLMs) is inherently short-sighted, often failing to find globally optimal reasoning paths due to its token-by-token generation process. While inference-time strategies like foresight sampling attempt to mitigate this by simulating future steps, they typically rely on ad-hoc heuristics for valuing paths and pruning the search space. This paper introduces Martingale Foresight Sampling (MFS), a principled framework that reformulates LLM decoding as a problem of identifying an optimal stochastic process. By modeling the quality of a reasoning path as a stochastic process, we leverage Martingale theory to design a theoretically-grounded algorithm. Our approach replaces heuristic mechanisms with principles from probability theory: step valuation is derived from the Doob Decomposition Theorem to measure a path's predictable advantage, path selection uses Optional Stopping Theory for principled pruning of suboptimal candidates, and an adaptive stopping rule based on the Martingale Convergence Theorem terminates exploration once a path's quality has provably converged. Experiments on six reasoning benchmarks demonstrate that MFS surpasses state-of-the-art methods in accuracy while significantly improving computational efficiency. Code will be released at https://github.com/miraclehetech/EACL2026-Martingale-Foresight-Sampling.
Paper Structure (24 sections, 2 theorems, 12 equations, 3 figures, 6 tables, 2 algorithms)

This paper contains 24 sections, 2 theorems, 12 equations, 3 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Foresight Sampling for LLM Decoding
  4. Martingale Theory
  5. Problem Formulation: Decoding as a Stochastic Process
  6. Martingale Foresight Sampling
  7. Principled Step Value via Doob Decomposition
  8. Empirical Validation of Step Values
  9. Optimal Path Selection via Optional Stopping Theory
  10. Adaptive Pruning via Martingale Convergence
  11. Experiments
  12. BenchMarks And Reported Metric
  13. Baselines and Backbone LLMs
  14. Experimental Environments And Basic Settings
  15. Main Results
  16. ...and 9 more sections

Key Result

theorem 1

Any adapted $L^1$ process $(X_n)_{n \ge 0}$ can be uniquely decomposed into a martingale $(M_n)_{n \ge 0}$ and a predictable process $(A_n)_{n \ge 0}$ such that $X_n = X_0 + M_n + A_n$. The increment of the predictable process is given by:

Figures (3)

  • Figure 1: The overall framework of our principled MFS algorithm. We visualized the foresight decoding within one step. For clear visualization, we set the beam size as 3 and rollout number as 3.
  • Figure 1: Decoding Loop (Part 1)
  • Figure 2: Illustration of martingale-based early stopping on a GSM8K example. The deficit process triggers a stopping time once the candidate path’s gap exceeds the adaptive threshold, pruning unpromising trajectories and reducing unnecessary token generation. This principled criterion ensures efficient beam management while preserving solution quality.

Theorems & Definitions (5)

  • definition 1: Filtration and Adapted Process
  • definition 2: Martingale
  • theorem 1: Doob Decomposition
  • theorem 2: Doob's Forward Convergence
  • definition 3: Stopping Time