ARC-TGI: Human-Validated Task Generators with Reasoning Chain Templates for ARC-AGI

TL;DR

This work introduces ARC-TGI (ARC Task Generators Inventory), an open-source framework for task-family generators: compact Python programs that sample diverse ARC-AGI tasks while preserving a latent rule.

Abstract

The Abstraction and Reasoning Corpus (ARC-AGI) probes few-shot abstraction and rule induction on small visual grids, but progress is difficult to measure on static collections of hand-authored puzzles due to overfitting, dataset leakage, and memorisation. We introduce ARC-TGI (ARC Task Generators Inventory), an open-source framework for task-family generators: compact Python programs that sample diverse ARC-AGI tasks while preserving a latent rule. ARC-TGI is built around a solver-facing representation: each generated task is paired with natural-language input and transformation reasoning chains and partially evaluated Python code implementing sampling, transformation, and episode construction. Crucially, ARC-TGI supports task-level constraints so that training examples collectively expose the variations needed to infer the underlying rule, a requirement for human-solvable ARC tasks that independent per-example sampling often fails to guarantee. All generators undergo human refinement and local verification to keep both grids and reasoning traces natural and consistent under variation. We release 461 generators covering 180 ARC-Mini tasks, 215 ARC-AGI-1 tasks (200 train, 15 test), and 66 ARC-AGI-2 tasks (55 train, 11 test), enabling scalable dataset sampling and controlled benchmarking.

Figures (15)

• Figure 1: Input grid-size heatmaps. Upper row: original tasks. Lower row: ARC-TGI samples (50 per generator).
• Figure 2: ARC-AGI task (Task ID: e509e548) and corresponding ARC-TGI sample.
• Figure 3: Per-grid diversity visualization for one ARC-TGI generator (taskcmBhVbGzL8ZgWXDE5CUS6B), showing variation in spatial position, size, and color across sampled grids while preserving the latent rule.
• Figure 4: Few-shot performance on ARC-TGI-50N. (A) ARC-style episode: infer a latent rule from training input/output pairs and apply it to a test input. (B) Exact-match accuracy across models on ARC-TGI-50N.
• Figure 5: Per-generator performance across models (200 generators). Heatmap of model accuracy by generator. Models are ordered by mean accuracy (y-axis) and generators are ordered by Qwen3-30B accuracy (x-axis), revealing shared easy/hard regions and sparse success on the hardest generators.