You only need 4 extra tokens: Synergistic Test-time Adaptation for LLMs

TL;DR

SyTTA introduces a label-free test-time adaptation framework for LLMs under distribution shift by synergistically coupling input perplexity minimization and output entropy shaping, with dynamic weighting and a reverse KL-based trust region. It supports Static-Ref and Dynamic-Ref deployment and operates on a small prefix budget (4–16 tokens) to adapt cohorts of unlabeled queries. Across DomainBench and InstructBench, SyTTA yields consistent gains across model families (Llama, Qwen) and scales, notably achieving substantial Rouge-Lsum improvements in agriculture with minimal overhead. The approach reduces reliance on labeled data, enabling practical deployment in label-scarce domains, and is complemented by extensive ablations and practical deployment guidance.

Abstract

Large language models (LLMs) are increasingly deployed in specialized domains such as finance, medicine, and agriculture, where they face significant distribution shifts from their training data. Domain-specific fine-tuning can mitigate this challenge but relies on high-quality labeled data that is expensive and slow to collect in expertise-limited settings. We study label-free test-time adaptation for language models and present SyTTA, an inference-time framework that adapts models on-the-fly without additional supervision. SyTTA couples two complementary uncertainty signals that arise under distribution shift: input-side perplexity, indicating mismatch with domain-specific terminology and patterns, and output-side predictive entropy, indicating diffuse and unstable token probabilities during generation. Across diverse model architectures and domain-specific benchmarks, SyTTA delivers consistent gains. Notably, on agricultural question answering, SyTTA improves Rouge-LSum by over 120% on Qwen-2.5-7B with only 4 extra tokens per query. These results show that effective test-time adaptation for language models is achievable without labeled examples, supporting deployment in label-scarce domains. The code will be made available upon acceptance.

Figures (5)

• Figure 1: Illustration of LLM degradation under distribution shift: a Scottish dialect query ("messages and a piece") is misinterpreted as unrelated intent.
• Figure 2: Overview of the SyTTA framework. Input Distribution Adaptation lowers input perplexity, Output Confidence Shaping reduces output entropy, and Dynamic Importance Weighting balances the two signals. We leverage uncertainties as self-supervised signals for test-time adaptation.
• Figure 3: ROUGE-L$_\text{sum}$ scores under different generation lengths (4 vs. 16) and models. Results are shown for both Static-Ref and Dynamic-Ref , with error bars indicating standard deviations.
• Figure 4: Average token-level response entropy computed by averaging across all responses.
• Figure 5: Ablations of KL regularization and Dynamic Importance Weighting on ROUGE-L$_\text{sum}$ across models. Both absolute and relative improvements (%) are shown.