Lateral Tree-of-Thoughts Surpasses ToT by Incorporating Logically-Consistent, Low-Utility Candidates

TL;DR

ToT struggles under large budgets due to breadth saturation and depth myopia. Lateral Tree-of-Thoughts (LToT) introduces a dual-frontier controller that separates consistency from utility and channels surplus compute into wide, cheap lateral exploration via LR-SC, while keeping mainlines narrow for exploitation. The approach yields a pseudolinear lateral cost $\Theta(N_0 \log_{\eta} N_0)$ and promotes only when branches cross the mainline bar under verifier-aligned checks, mitigating both saturation and myopia. Across math, code, and puzzle-style tasks, LToT improves success-per-compute, reduces false promotions, and remains robust to noisy evaluators, offering a practical upgrade for inference-time search.

Abstract

Modern deployments increasingly allocate large test-time compute (thousands of tokens or many node expansions) to boost reliability. Under such budgets, standard Tree-of-Thoughts-style search exhibits two pathologies: breadth saturation (additional samples mostly produce near-duplicates, so width stops growing) and depth myopia (noisy short-horizon utilities prune branches whose payoff appears after a few more steps). We propose Lateral Tree-of-Thoughts (LToT), a drop-in controller that separates utility from logical consistency and treats low-utility but consistent candidates as assets rather than waste. The frontier is split into mainlines (high-utility candidates used for exploitation) and laterals (consistent, initially low-utility candidates that receive short, cheap probes before judgment). LToT explores laterals via Lateral Racing with Short-Circuit (LR--SC): a capped successive-halving race that spreads tiny probes across a very wide lateral set, uses width-aware thresholds with repeat-to-confirm, and immediately promotes a branch once its envelope clears the mainline bar; mainlines are kept intentionally narrow so surplus compute is invested where width is cheap. We prove a pseudolinear lateral cost $Θ(N_0 \log_η N_0)$ with logarithmically many rungs (initial lateral width $N_0$; culling factor $η>1$), in contrast to the exponential growth of uncapped mainlines. Empirical evaluations on benchmark tasks are in preparation and will be added in a future revision. In short, LToT turns large test-time budgets into principled diversity while preserving promotion discipline, mitigating saturation and myopia without inflating compute.