Learning to Discover at Test Time

TL;DR

TTT-Discover reframes test-time learning as a single-problem discovery task, training an open LLM online to surpass the state of the art rather than maximize average performance. It combines an adaptive entropic objective with a PUCT-based state reuse strategy to aggressively search for high-impact solutions within a fixed budget. Across mathematics, kernel optimization, algorithm design, and single-cell denoising, it achieves new state-of-the-art results while maintaining reproducibility through open models and public code. The work suggests that problem-specific, online learning at test time can outperform purely prompt-based search, with broad implications for scientific discovery and engineering optimization.

Abstract

How can we use AI to discover a new state of the art for a scientific problem? Prior work in test-time scaling, such as AlphaEvolve, performs search by prompting a frozen LLM. We perform reinforcement learning at test time, so the LLM can continue to train, but now with experience specific to the test problem. This form of continual learning is quite special, because its goal is to produce one great solution rather than many good ones on average, and to solve this very problem rather than generalize to other problems. Therefore, our learning objective and search subroutine are designed to prioritize the most promising solutions. We call this method Test-Time Training to Discover (TTT-Discover). Following prior work, we focus on problems with continuous rewards. We report results for every problem we attempted, across mathematics, GPU kernel engineering, algorithm design, and biology. TTT-Discover sets the new state of the art in almost all of them: (i) Erdős' minimum overlap problem and an autocorrelation inequality; (ii) a GPUMode kernel competition (up to $2\times$ faster than prior art); (iii) past AtCoder algorithm competitions; and (iv) denoising problem in single-cell analysis. Our solutions are reviewed by experts or the organizers. All our results are achieved with an open model, OpenAI gpt-oss-120b, and can be reproduced with our publicly available code, in contrast to previous best results that required closed frontier models. Our test-time training runs are performed using Tinker, an API by Thinking Machines, with a cost of only a few hundred dollars per problem.

Figures (4)

• Figure 1: TTT-Discover continues to train an LLM on a single problem at test time. $\pi_{\theta_i}$ denotes the policy with the updates weights at test-time training step $i$. We plot the reward distribution at step 0, 9, 24, and 49 (final), recorded while test-time training for the GPUMode https://www.gpumode.com/v2/leaderboard/496?tab=rankings competition. We generate $512$ solutions at each step. As training progresses, the LLM generates better solutions that ultimately surpass the prior art (best human). For comparison, we plot the reward distribution of best-of-$N$ with the same total sampling budget.
• Figure 2: We show the normalized step functions including the prior state-of-the-art from AlphaEvolve. The step function $f(x)$ is the limiting density of set $A$. Unlike the previous state-of-the-art, the solution from TTT-Discover is an asymmetric construction. TTT-Discover found a 600-piece step function, while AlphaEvolve construction was 95-piece. The best human result was a 51-piece construction haugland2016minimum.
• Figure 3: We show the prior and new state-of-the-art, with the (normalized) step functions and their autoconvolutions. Both AlphaEvolve and TTT-Discover starts the discovery process from scratch, while ThetaEvolve initializes from the AlphaEvolve construction, and thus is very similar to the AlphaEvolve construction. TTT-Discover found a 30,000-piece step function that certifies that the upper bound $C_{1} \leq 1.50286$, while AlphaEvolve and ThetaEvolve constructions are 1319-piece. We overlay the step functions and their autoconvolution visually for qualitative comparison.
• Figure 4: Reward distributions for each ablation. We match the sampling budget across all ablations. We sample $512$ rollouts in each step. For example, for Best-of-$N$, we have $N = 50 \times 512 = 256000$ rollouts.

Theorems & Definitions (1)

• Definition : Discovery