FloydNet: A Learning Paradigm for Global Relational Reasoning

TL;DR

FloydNet introduces a dynamic-programming–style paradigm for graph reasoning by maintaining and iteratively refining a global all-pairs relationship tensor with a learned DP operator. The architecture, centered on FloydBlocks and Pivotal Attention, achieves 3-WL expressiveness and demonstrates state-of-the-art results on algorithmic reasoning (CLRS-30), general TSP, and real-world benchmarks (LRGB, ZINC). Theoretical analysis links FloydNet to k-FWL and shows exponential growth of the effective receptive field with depth, enabling long-range relational reasoning beyond traditional GNNs. Empirically, FloydNet exhibits strong scaling via model and data size, while maintaining permutation equivariance and avoiding positional encodings. The work positions DP-inspired global refinement as a practical alternative to message passing for high-level graph reasoning with broad implications for neural algorithmic reasoning and combinatorial optimization.

Abstract

Developing models capable of complex, multi-step reasoning is a central goal in artificial intelligence. While representing problems as graphs is a powerful approach, Graph Neural Networks (GNNs) are fundamentally constrained by their message-passing mechanism, which imposes a local bottleneck that limits global, holistic reasoning. We argue that dynamic programming (DP), which solves problems by iteratively refining a global state, offers a more powerful and suitable learning paradigm. We introduce FloydNet, a new architecture that embodies this principle. In contrast to local message passing, FloydNet maintains a global, all-pairs relationship tensor and learns a generalized DP operator to progressively refine it. This enables the model to develop a task-specific relational calculus, providing a principled framework for capturing long-range dependencies. Theoretically, we prove that FloydNet achieves 3-WL (2-FWL) expressive power, and its generalized form aligns with the k-FWL hierarchy. FloydNet demonstrates state-of-the-art performance across challenging domains: it achieves near-perfect scores (often >99\%) on the CLRS-30 algorithmic benchmark, finds exact optimal solutions for the general Traveling Salesman Problem (TSP) at rates significantly exceeding strong heuristics, and empirically matches the 3-WL test on the BREC benchmark. Our results establish this learned, DP-style refinement as a powerful and practical alternative to message passing for high-level graph reasoning.

Figures (9)

• Figure 1: FloydBlock and Pivotal Attention Mechanism. Each FloydBlock updates the representation for a given pair $(i,k)$ by using its state as a query to attend to all possible two-hop paths that pass through an intermediate "pivot" node $j$.
• Figure 2: The FloydNet framework.(A) The model transforms various graph inputs into a unified relational representation ${\bm{R}}$. (B) The SuperNode provides a unified representation for graph, node, and edge information within ${\bm{R}}$.
• Figure 3: Probing the Scalability and Reasoning Logic of FloydNet on CLRS.(a) We extend the OOD evaluation to problem sizes up to $n=256$, following the protocol of minder2023salsa. The results indicate that models trained without hints (upper) generally exhibit superior length extrapolation capabilities compared to those trained with hints (lower). (b) Visualization of the internal reasoning process for the Maximum Subarray problem (Kadane's algorithm). The model iteratively predicts the new maximum subarray based on the current maximum subarray and the next number corresponding to the current step. The clear alignment demonstrates that FloydNet has successfully learned to emulate the algorithm's state-update logic.
• Figure 4: Analysis of FloydNet on the TSP. (a) Optimality rate on the benchmark (including multi-solution instances). The yellow region denotes validation on in-distribution sizes ($N \le 100$), while the cyan region demonstrates OOD generalization to larger problem sizes ($100 < N \le 200$). (b) Performance on the subset of single-solution instances, testing the model's precise alignment with a unique optimal ground truth. (c) Distribution of error types. (d-e) Scaling laws showing that FloydNet effectively utilizes increased data and model depth, avoiding the over-smoothing issues typical of deep MPNNs.
• Figure 5: Performance Comparison of Original vs. Optimized PivotAttn Kernel. Training step latency and peak GPU memory usage are shown as sequence length $N$ increases. The original implementation quickly leads to out-of-memory (OOM) errors. At $N=256$, the Original version requires 61.5x the runtime and 31.6x the peak memory of the Optimized version.

Theorems & Definitions (9)

• Theorem 1: k-FloydNet is equivalent to k-FWL
• proof : Proof Sketch
• Theorem 2: Principled Long-Range Information Propagation
• proof : Proof Sketch
• Definition 1: Color Refinement Algorithm
• Theorem 3: k-FloydNet is k-FWL
• proof
• Theorem 4: Principled Long-Range Information Propagation
• proof