The Deep Equilibrium Algorithmic Reasoner

TL;DR

The paper tackles neural algorithmic reasoning by replacing iterative GNN unrolls with a fixed-point equilibrium approach. By formulating the processor as a Deep Equilibrium Model and solving for the equilibrium state $\mathbf{H}^{(*)}=P_{\mathbf{U}\mathbf{E}}(\mathbf{H}^{(*)})$, the Deep Equilibrium Algorithmic Reasoner (DEAR) learns to implement CLRS-30 algorithms without one-to-one step alignment, achieving strong out-of-distribution performance and notable speedups, especially on sorting. The results show DEAR can match or exceed a strong no-hint baseline, with substantial efficiency gains when relaxing the solver tolerance, suggesting that equilibrium-based reasoning can generalize well and enable scalable algorithmic learning. This work highlights the potential of implicit, fixed-point reasoning for graph-based algorithm execution and opens avenues for more efficient, parallel-like algorithmic inference on large graphs.

Abstract

Recent work on neural algorithmic reasoning has demonstrated that graph neural networks (GNNs) could learn to execute classical algorithms. Doing so, however, has always used a recurrent architecture, where each iteration of the GNN aligns with an algorithm's iteration. Since an algorithm's solution is often an equilibrium, we conjecture and empirically validate that one can train a network to solve algorithmic problems by directly finding the equilibrium. Note that this does not require matching each GNN iteration with a step of the algorithm.

Figures (2)

• Figure 1: The deep equilibrium algorithmic reasoner. Dashed arrows represent constant variables.
• Figure 2: An example input list and predecessors indicating order. Source: ibarz2022generalist