LoopViT: Scaling Visual ARC with Looped Transformers

TL;DR

ARC-AGI visual reasoning tasks require iterative, rule-based processing that goes beyond single-pass vision transformers. We introduce Loop-ViT, a looped Vision Transformer with a weight-tied recurrent core, a Hybrid Block that fuses local spatial updates with global reasoning, and an entropy-based Dynamic Exit that halts computation when predictions crystallize. Empirically, an 18M-parameter Large Loop-ViT reaches 65.8% on ARC-1, outperforming a 73M-parameter ensemble, while a 3.8M Small variant reaches 60.1%, illustrating that adaptive iterative computation is a more efficient scaling axis than width. These results demonstrate robust adaptive computation for abstract visual induction and provide a practical, parameter-efficient baseline for future reasoning benchmarks, with code available at the project repository.

Abstract

Recent advances in visual reasoning have leveraged vision transformers to tackle the ARC-AGI benchmark. However, we argue that the feed-forward architecture, where computational depth is strictly bound to parameter size, falls short of capturing the iterative, algorithmic nature of human induction. In this work, we propose a recursive architecture called Loop-ViT, which decouples reasoning depth from model capacity through weight-tied recurrence. Loop-ViT iterates a weight-tied Hybrid Block, combining local convolutions and global attention, to form a latent chain of thought. Crucially, we introduce a parameter-free Dynamic Exit mechanism based on predictive entropy: the model halts inference when its internal state ``crystallizes" into a low-uncertainty attractor. Empirical results on the ARC-AGI-1 benchmark validate this perspective: our 18M model achieves 65.8% accuracy, outperforming massive 73M-parameter ensembles. These findings demonstrate that adaptive iterative computation offers a far more efficient scaling axis for visual reasoning than simply increasing network width. The code is available at https://github.com/WenjieShu/LoopViT.

Figures (11)

• Figure 1: Acc-Params comparisons with recurrent and vision methods. The vertical axis is accuracy (ARC-AGI-1); the horizontal axis is Parameters (memory cost). Our Loop-ViT outperforms previous methods while requiring significantly cheaper Params.
• Figure 2: Illustration of ARC-AGI-1 and ARC-AGI-2 benchmarks. The left two columns display tasks from ARC-AGI-1, characterized by visual priors such as "Object Cohesion" and "Pattern Completion". These tasks primarily test perceptual generalization. The right column showcases an ARC-AGI-2 task, exemplifying higher-order algorithmic challenges such as "Symbolic Interpretation", "Compositional Reasoning", and "Contextual Rule Application". For each task, the top rows show the few-shot demonstrations (Training) used to infer the rule, and the bottom row shows the query input (Inference).
• Figure 3: Comparison of input representations and inference paradigms for ARC. (A) LLMs operate on a 1D textual token sequence obtained by serializing the ARC grids into a prompt (e.g., JSON/ASCII). (B) Recurrent token models also take a 1D sequence, but with a discrete grid-tokenization that pads the grid to a fixed canvas and inserts special boundary tokens (e.g., PAD/EOS), yielding a fixed-length token stream. (C) VARC follows a vision formulation, encoding the grid as a 2D spatial input processed in a single forward pass. (D) Ours combines the vision input with looped/iterative inference, repeatedly refining internal repre- sentations and predictions across multiple steps, bridging spatial inductive bias and recurrent computation.
• Figure 4: The overall pipeline of the proposed LoopViT. (A) Comparison of the standard VARC pipeline versus our Loop-ViT pipeline. Loop-ViT introduces iterative state refinement through a weight-tied core. (B) Detailed unrolled view of the Loop-ViT recurrence, where the state $z_t$ acts as a dynamic memory. (C) Structure of the Hybrid Transformer Block, employing RMSNorm and Rotary Positional Embeddings. (D) The Heterogeneous Feed-Forward Network (ConvGLU), which splits processing pathways to apply depth-wise convolution solely to image tokens while preserving task tokens, reconciling local spatial updates with global rule induction.
• Figure 5: Iterative Prediction Refinement in Loop-ViT. (Top) The model's output progressively approaches the ground truth through successive iterations. (Middle) Pixel-wise difference maps between consecutive steps show decreasing prediction volatility. (Bottom) Entropy measurements demonstrate the stabilization of the model's confidence. This "crystallization effect" reveals how recurrent processing enables gradual convergence to logically consistent solutions.