When Less is More: The LLM Scaling Paradox in Context Compression

TL;DR

The paper investigates a paradox in long-context context compression: increasing compressor size often lowers fidelity of verbatim reconstruction even as training loss improves. It introduces two diagnostic QA tasks to isolate knowledge overwriting and semantic drift, and analyzes memory embeddings to reveal mechanisms: growing semantic capacity (higher effective rank) promotes interference from priors, while increasing generative uncertainty (entropy) drives drift. Across 0.6B–90B models from the Qwen and LLaMA families and multiple compression rates, larger compressors show faster optimization but degrade downstream fidelity, challenging the universality of scaling laws for fidelity-critical tasks. The findings motivate rethinking compression-only objectives and advocate evaluation frameworks that explicitly measure factual faithfulness and structural preservation in open-ended generation.

Abstract

Scaling up model parameters has long been a prevalent training paradigm driven by the assumption that larger models yield superior generation capabilities. However, under lossy context compression in a compressor-decoder setup, we observe a Size-Fidelity Paradox: increasing the compressor size can lessen the faithfulness of reconstructed contexts though training loss decreases. Through extensive experiments across models from 0.6B to 90B, we coin this paradox arising from two dominant factors: 1) knowledge overwriting: larger models increasingly replace source facts with their own prior beliefs, e.g., ``the white strawberry'' $\to$ ``the red strawberry''; and 2) semantic drift: larger models tend to paraphrase or restructure content instead of reproducing it verbatim, e.g., ``Alice hit Bob'' $\to$ ``Bob hit Alice''. By holding model size fixed, we reflect on the emergent properties of compressed context representations. We show that the culprit is not parameter count itself, but the excessive semantic capacity and amplified generative uncertainty that accompany scaling. Specifically, the increased rank of context embeddings facilitates prior knowledge intrusion, whereas higher entropy over token prediction distributions promotes rewriting. Our results complement existing evaluations over context compression paradigm, underpinning a breakdown in scaling laws for faithful preservation in open-ended generation.

Figures (4)

• Figure 1: The Size-Fidelity Paradox in context compression.(Left & Right) A qualitative case study illustrating the breakdown of faithfulness. While the Lite compressor preserves factual details (Q1, Q2), the Large compressor succumbs to two distinct failure modes: (1) knowledge overwriting, where source facts are replaced by priors (e.g., hallucinating "honey bee" instead of "blue-banded bee"); and (2) semantic drift, where the causal relationship is distorted (e.g., "bee pollinates flower" $\to$ "flower pollinates bee"). (Center) Quantitative analysis across Qwen and Llama families confirms this paradox is systematic. As model size scales up ($x$-axis), surface-level Reconstruction scores (dashed lines) remain high, yet QA accuracy (solid lines) significantly degrades. This divergence indicates that larger compressors prioritize their own semantic capacity and priors over the faithful preservation of the source context.
• Figure 2: Training loss dynamics for Qwen (top) and LLaMA (bottom) compressors at a $4\times$ compression rate. Larger models exhibit faster convergence and lower final loss, creating a deceptive signal of superior optimization.
• Figure 3: (a) Effective rank increases monotonically with model scale in the Qwen3 family (0.6B–32B). (b) Training dynamics of effective rank. A clear two-phase trajectory emerges: early expansion followed by compression. (c) Effective rank vs. QA performance. Effective rank is negatively correlated with QA accuracy; shaded bands indicate the sample-level distribution.
• Figure 4: (a) Entropy distribution across model scales (0.6B–32B). (b) Training dynamics of conditional entropy (steadily decreasing over optimization). (c) Conditional entropy vs. QA accuracy (strong negative correlation: Pearson $r=-0.823$, Spearman $\rho=-0.876$).