Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Thermodynamic Focusing for Inference-Time Search: Practical Methods for Target-Conditioned Sampling and Prompted Inference

Zhan Zhang

TL;DR

This work tackles the challenge of discovering rare, high-quality solutions in massive candidate spaces by reframing search as target-conditioned sampling. It introduces ICFA, a practical framework that reweights a fixed proposal sampler via adaptive focusing and monitors stability with an ESS-based diagnostic. The authors provide a reproducible algorithm, stability strategies, and demonstrations in constrained generation and sparse-reward navigation, plus a language-level variant called Prompted ICFA. The results show substantial reductions in required samples and broader applicability, including a practical bridge to prompting when algorithmic intervention is not feasible.

Abstract

Finding rare but useful solutions in very large candidate spaces is a recurring practical challenge across language generation, planning, and reinforcement learning. We present a practical framework, \emph{Inverted Causality Focusing Algorithm} (ICFA), that treats search as a target-conditioned reweighting process. ICFA reuses an available proposal sampler and a task-specific similarity function to form a focused sampling distribution, while adaptively controlling focusing strength to avoid degeneracy. We provide a clear recipe, a stability diagnostic based on effective sample size, a compact theoretical sketch explaining when ICFA can reduce sample needs, and two reproducible experiments: constrained language generation and sparse-reward navigation. We further show how structured prompts instantiate an approximate, language-level form of ICFA and describe a hybrid architecture combining prompted inference with algorithmic reweighting.

Thermodynamic Focusing for Inference-Time Search: Practical Methods for Target-Conditioned Sampling and Prompted Inference

TL;DR

This work tackles the challenge of discovering rare, high-quality solutions in massive candidate spaces by reframing search as target-conditioned sampling. It introduces ICFA, a practical framework that reweights a fixed proposal sampler via adaptive focusing and monitors stability with an ESS-based diagnostic. The authors provide a reproducible algorithm, stability strategies, and demonstrations in constrained generation and sparse-reward navigation, plus a language-level variant called Prompted ICFA. The results show substantial reductions in required samples and broader applicability, including a practical bridge to prompting when algorithmic intervention is not feasible.

Abstract

Finding rare but useful solutions in very large candidate spaces is a recurring practical challenge across language generation, planning, and reinforcement learning. We present a practical framework, \emph{Inverted Causality Focusing Algorithm} (ICFA), that treats search as a target-conditioned reweighting process. ICFA reuses an available proposal sampler and a task-specific similarity function to form a focused sampling distribution, while adaptively controlling focusing strength to avoid degeneracy. We provide a clear recipe, a stability diagnostic based on effective sample size, a compact theoretical sketch explaining when ICFA can reduce sample needs, and two reproducible experiments: constrained language generation and sparse-reward navigation. We further show how structured prompts instantiate an approximate, language-level form of ICFA and describe a hybrid architecture combining prompted inference with algorithmic reweighting.
Paper Structure (29 sections, 1 theorem, 4 equations, 1 table, 1 algorithm)

This paper contains 29 sections, 1 theorem, 4 equations, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Framework
  3. Problem statement
  4. Practical challenge: weight collapse
  5. Algorithm
  6. Self-normalized importance sampling with adaptive focusing
  7. Practical details
  8. When and why focusing helps
  9. Prompted ICFA: language-level approximation
  10. Experiments
  11. Constrained text generation
  12. Setup.
  13. Metrics.
  14. Results.
  15. Sparse-reward reinforcement learning
  16. ...and 14 more sections

Key Result

Theorem 1

Assume the proposal $P_0$ and similarity $S$ are such that, for some $\beta>0$, the expected weight of valid solutions exceeds the expected weight of non-solutions by a factor $e^{\kappa}$ with $\kappa\gg\ln M$. Then, with high probability, ICFA identifies a valid solution using samples, where $N$ is the effective space size and $\delta$ the failure probability.

Theorems & Definitions (1)

  • Theorem 1: Informal