Recursive Inference Machines for Neural Reasoning

TL;DR

It is shown that TRMs can be expressed as an instance of RIMs, allowing them to extend them through a reweighting component, yielding better performance on challenging reasoning benchmarks, including ARC-AGI-1, ARC-AGI-2, and Sudoku Extreme.

Abstract

Neural reasoners such as Tiny Recursive Models (TRMs) solve complex problems by combining neural backbones with specialized inference schemes. Such inference schemes have been a central component of stochastic reasoning systems, where inference rules are applied to a stochastic model to derive answers to complex queries. In this work, we bridge these two paradigms by introducing Recursive Inference Machines (RIMs), a neural reasoning framework that explicitly incorporates recursive inference mechanisms inspired by classical inference engines. We show that TRMs can be expressed as an instance of RIMs, allowing us to extend them through a reweighting component, yielding better performance on challenging reasoning benchmarks, including ARC-AGI-1, ARC-AGI-2, and Sudoku Extreme. Furthermore, we show that RIMs can be used to improve reasoning on other tasks, such as the classification of tabular data, outperforming TabPFNs.

Figures (3)

• Figure 1: A Recursive Inference Machine (RIM) that solves problems described by $x$, given an initial solution $y^{(0)}$, and an initial state ($z^{(0)}$). It consists of a Solver, a Reweighter, and a Generator. The components solve the problem by alternating between the Solver updating the state recursively for $T$ steps, and the Generator using these state-updates to generate an updated solution. After repeating this $N$ times, the RIM produces the solution $y^{(N)}.$
• Figure 2: Unrolling a Recursive Inference Machine (RIM). The inner loop (indexed by $j=1,\dots,T$) acts as the Solver, which extends the short-term reasoning "scratchpad" $z^j$ by proposing new states $\hat{z}^{j+1}$ based on the problem description $x$ and current solution $y^i$. The outer loop (indexed by $i=1,\dots,N$) implements the Generator, which updates the solution $y^i$ after each phase of local refinement. The Reweighter intercepts the updates proposed by the Solver and Generator and weighs them against their previous values, stabilizing the trajectory.
• Figure 3: Tabular RIM (TabRIM) for Reasoning under Uncertainty To reason about the distribution over the target $Y$ given noisy observations $E=e,$ the TabRIM decomposes the task into a Solver-Reweighter-Generator loop. The Solver iteratively denoises the observation by iteratively resampling latent variables from their full conditionals. Each such conditional is implemented as a single forward pass over TabPFN. The Reweighter weighs these samples against the observed evidence to ensure consistency with it. These consistency-weighted samples are used by the Generator to infer the final target distribution by computing the empirical expectation over them.

Theorems & Definitions (1)

• Definition 3.1