Quantifying Generalization Complexity for Large Language Models

TL;DR

Scylla, a dynamic evaluation framework that quantitatively measures the generalization abilities of large language models, reveals a critical threshold - referred to as critical complexity - where reliance on non-generalizable behavior peaks, indicating the upper bound of LLMs' generalization capabilities.

Abstract

While large language models (LLMs) have shown exceptional capabilities in understanding complex queries and performing sophisticated tasks, their generalization abilities are often deeply entangled with memorization, necessitating more precise evaluation. To address this challenge, we introduce Scylla, a dynamic evaluation framework that quantitatively measures the generalization abilities of LLMs. Scylla disentangles generalization from memorization via assessing model performance on both in-distribution (ID) and out-of-distribution (OOD) data through 20 tasks across 5 levels of complexity. Through extensive experiments, we uncover a non-monotonic relationship between task complexity and the performance gap between ID and OOD data, which we term the generalization valley. Specifically, this phenomenon reveals a critical threshold - referred to as critical complexity - where reliance on non-generalizable behavior peaks, indicating the upper bound of LLMs' generalization capabilities. As model size increases, the critical complexity shifts toward higher levels of task complexity, suggesting that larger models can handle more complex reasoning tasks before over-relying on memorization. Leveraging Scylla and the concept of critical complexity, we benchmark 28LLMs including both open-sourced models such as LLaMA and Qwen families, and close-sourced models like Claude and GPT, providing a more robust evaluation and establishing a clearer understanding of LLMs' generalization capabilities.

Figures (11)

• Figure 1: An illustration of generalization valley, where the reliance on non-generalizable behaviors first increases and then decreases; and critical complexity shift, where the peak of the valley shifts rightward as model size increases.
• Figure 2: Left: Anchor tasks. These tasks form the core of our benchmark, providing a structured set of challenges across varying time complexities. Right: Probe tasks. These tasks are used to evaluate the level of complexity that LLMs adopt to solve them.
• Figure 3: Pipeline for generating ID and OOD dataset for a given task, tailored to each LLM.
• Figure 4: An example of probability distribution of ID elements collected by querying Mistral 7B v0.3 on the task of Find Longest Increasing Subsequence. The histogram shows the most frequent values with probabilities adding up to 90% (top-p=0.9).
• Figure 5: ID/OOD performance of Qwen 1.5 family across five complexity levels.