Power-SMC: Low-Latency Sequence-Level Power Sampling for Training-Free LLM Reasoning
Seyedarmin Azizi, Erfan Baghaei Potraghloo, Minoo Ahmadi, Souvik Kundu, Massoud Pedram
TL;DR
This work tackles efficient training-free sampling from the sequence-level power distribution $\pi_\alpha(y\mid x)$ for LLM reasoning. It introduces Power-SMC, a training-free Sequential Monte Carlo approach that runs $N$ particles in parallel, uses a prefix-flow formulation with token-wise incremental corrections, and applies ESS-triggered resampling within a single batched decode, maintaining fidelity to $\pi_\alpha$ while achieving latency close to baseline decoding. The authors prove that among prefix-only proposals, the locally optimal temperature is $\tau=1/\alpha$, and they provide a Rényi-entropy interpretation of residual weight dispersion along with an exact exponent-bridging ($\alpha$-ramping) mechanism. They develop a gas-forced, cache-safe implementation and an engine-independent cost model, showing substantial latency improvements over MH on MATH500 across multiple models. Empirically, Power-SMC matches or surpasses MH in accuracy while reducing inference latency from 16–28× to 1.4–3.3× relative to standard decoding, demonstrating practical viability for training-free sequence-level sharpening.
Abstract
Many recent reasoning gains in large language models can be explained as distribution sharpening: biasing generation toward high-likelihood trajectories already supported by the pretrained model, rather than modifying its weights. A natural formalization is the sequence-level power distribution $π_α(y\mid x)\propto p_θ(y\mid x)^α$ ($α>1$), which concentrates mass on whole sequences instead of adjusting token-level temperature. Prior work shows that Metropolis--Hastings (MH) sampling from this distribution recovers strong reasoning performance, but at order-of-magnitude inference slowdowns. We introduce Power-SMC, a training-free Sequential Monte Carlo scheme that targets the same objective while remaining close to standard decoding latency. Power-SMC advances a small particle set in parallel, corrects importance weights token-by-token, and resamples when necessary, all within a single GPU-friendly batched decode. We prove that temperature $τ=1/α$ is the unique prefix-only proposal minimizing incremental weight variance, interpret residual instability via prefix-conditioned Rényi entropies, and introduce an exponent-bridging schedule that improves particle stability without altering the target. On MATH500, Power-SMC matches or exceeds MH power sampling while reducing latency from $16$--$28\times$ to $1.4$--$3.3\times$ over baseline decoding.