Test-Time Meta-Adaptation with Self-Synthesis

TL;DR

MASS is introduced, a meta-learning framework that enables LLMs to self-adapt by generating problem-specific synthetic training data and performing targeted self-updates optimized for downstream performance at inference time.

Abstract

As strong general reasoners, large language models (LLMs) encounter diverse domains and tasks, where the ability to adapt and self-improve at test time is valuable. We introduce MASS, a meta-learning framework that enables LLMs to self-adapt by generating problem-specific synthetic training data and performing targeted self-updates optimized for downstream performance at inference time. We train this behavior end-to-end via bilevel optimization: an inner loop adapts on self-generated examples while an outer loop meta-learns data-attribution signals and rewards post-update task performance. The synthetic data is optimized with scalable meta-gradients, backpropagating the downstream loss through the inner updates to reward useful generations. Experiments on mathematical reasoning show that MASS learns to synthesize per-instance curricula that yield effective, data-efficient test-time adaptation.

Figures (2)

• Figure 1: MASS pipeline. The generator parameterized by $\theta$ generates $m$ auxiliary training examples $(p_1,a_1),\dots,(p_m,a_m)$ given the target task $T$. The examples are assigned scores using the scorer parameterized by $\eta$. Then, $\theta$ performs a weighted SFT self-update on the synthetic data to produce an adapted model $\theta'$ before producing resposne $\hat{R}$ in attempting the target problem. Meta-gradients computed by backpropagating target performance through the inner updates are used to update $\eta$ to identify examples that help the downstream loss and reward $\theta$ to generate better examples.
• Figure 2: Performance gains by domain.