CL-bench: A Benchmark for Context Learning

TL;DR

The Sol Accord Interplanetary Criminal Code introduces a cross-jurisdictional framework to regulate conduct in space, defining three signatory classes (SSPs, CAPs, DSEs) and establishing a layered liability regime. It details governance, liability allocation, and enforcement cooperation across participating jurisdictions, aiming to harmonize accountability for acts and omissions in a multilateral interplanetary context. The document also outlines procedures for ratification, implementation, and interaction with existing laws, plus reporting and oversight mechanisms. Collectively, it provides a structured scaffold for prosecuting misconduct in space, addressing governance gaps and enabling coordinated responses among states, corporations, and allied entities in the broader interplanetary arena.

Abstract

Current language models (LMs) excel at reasoning over prompts using pre-trained knowledge. However, real-world tasks are far more complex and context-dependent: models must learn from task-specific context and leverage new knowledge beyond what is learned during pre-training to reason and resolve tasks. We term this capability context learning, a crucial ability that humans naturally possess but has been largely overlooked. To this end, we introduce CL-bench, a real-world benchmark consisting of 500 complex contexts, 1,899 tasks, and 31,607 verification rubrics, all crafted by experienced domain experts. Each task is designed such that the new content required to resolve it is contained within the corresponding context. Resolving tasks in CL-bench requires models to learn from the context, ranging from new domain-specific knowledge, rule systems, and complex procedures to laws derived from empirical data, all of which are absent from pre-training. This goes far beyond long-context tasks that primarily test retrieval or reading comprehension, and in-context learning tasks, where models learn simple task patterns via instructions and demonstrations. Our evaluations of ten frontier LMs find that models solve only 17.2% of tasks on average. Even the best-performing model, GPT-5.1, solves only 23.7%, revealing that LMs have yet to achieve effective context learning, which poses a critical bottleneck for tackling real-world, complex context-dependent tasks. CL-bench represents a step towards building LMs with this fundamental capability, making them more intelligent and advancing their deployment in real-world scenarios.

Figures (17)

• Figure 1: Mismatch between how language models are commonly optimized in practice and the capabilities required by real-world tasks. While current LMs primarily elicit reasoning over prompts using pre-trained knowledge, real-world tasks are often context-dependent and require models to learn from context to solve them, a capability we term context learning.
• Figure 2: Solving tasks in CL-bench requires LMs to learn new knowledge from the provided context, rather than relying solely on static pre-trained knowledge. The knowledge is curated by domain experts, either newly created or sourced from niche and emerging long-tail content. New knowledge required for solving each task is provided within corresponding context, with no need for external retrieval. LM solutions are then verified against carefully annotated task-level rubrics. The example task illustrates a charged particle dynamics analysis within the framework of classical electrodynamics (see Table \ref{['table:category4_case1']} in the Appendix for more details).
• Figure 3: Context taxonomy of CL-bench.
• Figure 4: Distribution of context categories in CL-bench. Subcategory distributions are relatively balanced.
• Figure 5: We compare the task solving rates of ten frontier LMs across subcategories. The Darker-colored cells indicate higher values. For brevity, we omit version numbers for some models. All models use thinking or high reasoning effort settings.