Benchmarking World-Model Learning

TL;DR

<3-5 sentence high-level summary>WorldTest reframes world-model learning evaluation by separating reward-free interaction from a derived, task-specific test phase that probes the agent’s understanding of environment dynamics through environment-level queries. The AutumnBench instantiation provides 43 grid-world environments and 129 tasks across masking, change detection, and planning, enabling automated, cross-agent comparison without constraining representations. Empirical results show humans consistently outperform frontier reasoning models, and that simply scaling compute helps only in some environments, revealing headroom and suggesting that advances in metacognitive exploration and belief updating are needed. Overall, the work proposes a general, extensible framework for evaluating world-model learning with broad applicability beyond grid worlds, including physics, robotics, and multi-agent systems.

Abstract

Model-learning agents should gather information to learn world models that support many downstream tasks and inferences, such as predicting unobserved states, estimating near- and far-term consequences of actions, planning action sequences, and detecting changes in dynamics. Current methods for learning and evaluating world models diverge from this goal: training and evaluation are anchored to next-frame prediction, and success is scored by reward maximization in the same environment. We propose WorldTest, a protocol to evaluate model-learning agents that separates reward-free interaction from a scored test phase in a different but related environment. WorldTest is open-ended $\unicode{x2014}$ models should support many different tasks unknown ahead of time $\unicode{x2014}$ and agnostic to model representation, allowing comparison across approaches. We instantiated WorldTest with AutumnBench, a suite of 43 interactive grid-world environments and 129 tasks across three families: masked-frame prediction, planning, and predicting changes to the causal dynamics. We compared 517 human participants and three frontier models on AutumnBench. We found that humans outperform the models, and scaling compute improves performance only in some environments but not others. WorldTest provides a novel template $\unicode{x2014}$ reward-free exploration, derived tests, and behavior-based scoring $\unicode{x2014}$ to evaluate what agents learn about environment dynamics, and AutumnBench exposes significant headroom in world-model learning.

Figures (15)

• Figure 1: Overview of the WorldTest framework and the AutumnBench instantiation. Agents first interact with an environment without external rewards to build a world model, then are evaluated on a derived challenge. Top-left box shows the 9 example AutumnBench environments. Yellow notes in the middle explain the key UI elements in the human interface of AutumnBench.
• Figure 2: Textual representation of the top grid in the "interaction phase" of \ref{['fig:pipeline_overview']}.
• Figure 3: Aggregate scores over all AutumnBench tasks. Left: Reasoning models perform better in stochastic environments than in deterministic ones; humans perform consistently across both. Right: Humans outperform reasoning models across all task types: CD, MFP, and planning (PL).
• Figure 4: Score vs. cost per problem across environments.
• Figure 5: Action type distribution averaged over AutumnBench problems v.s. agents.