Measuring General Intelligence with Generated Games

TL;DR

gg-bench introduces a scalable data-generating benchmark that uses LLMs to design two-player games, implement them as Gym environments, and evaluate target models by playing against RL agents trained via self-play. The pipeline comprises game-description generation, code-generation for environments, PPO-based self-play training, and filtered evaluation against prompted models, with an upper-bound proof of beatability to ensure task viability. Analyses reveal meaningful diversity in generated games, partial faithfulness of implementations, and that current LLMs struggle with multi-turn strategic reasoning compared to reasoning-focused models, while scalability experiments suggest stronger models create harder benchmarks. The work also provides extensive data, code, and evaluation tools to enable future expansion and reproducibility, aiming to measure domain-general reasoning beyond static datasets.

Abstract

We present gg-bench, a collection of game environments designed to evaluate general reasoning capabilities in language models. Unlike most static benchmarks, gg-bench is a data generating process where new evaluation instances can be generated at will. In particular, gg-bench is synthetically generated by (1) using a large language model (LLM) to generate natural language descriptions of novel games, (2) using the LLM to implement each game in code as a Gym environment, and (3) training reinforcement learning (RL) agents via self-play on the generated games. We evaluate language models by their winrate against these RL agents by prompting models with the game description, current board state, and a list of valid moves, after which models output the moves they wish to take. gg-bench is challenging: state-of-the-art LLMs such as GPT-4o and Claude 3.7 Sonnet achieve winrates of 7-9% on gg-bench using in-context learning, while reasoning models such as o1, o3-mini and DeepSeek-R1 achieve average winrates of 31-36%. We release the generated games, data generation process, and evaluation code in order to support future modeling work and expansion of our benchmark.

Figures (6)

• Figure 1: Overview of our benchmark creation process. We start by generating descriptions of two-player strategy games, after which we generate implementations of these games as Gym environments. Lastly, we employ self-play reinforcement learning to train agents on these games
• Figure 2: An environment in gg-bench consists of three components: (a) a game description, (b) a Gym implementation, and (c) an action space description. Both the game description and action space description are available to the language model when prompted to select a move.
• Figure 3: (a) Average winrates of various LLMs on gg-bench (30 games per matchup; 95% CIs in parentheses). (b) Breakdown of GPT-4o failures: “Faults” are invalid-move errors.
• Figure 4: Example trajectory of Cross Over where o1 (labeled LLM) loses to the RL agent. Moves 0-4 are hidden as the game appears balanced until then, with both the LLM and the RL agent advancing their pieces forward. At Move 5, the LLM moves P1-C to position 6, highlighted by the blue arrow.
• Figure 5: Distribution of the highest similarity score for every one of the $126$ games in gg-bench.