Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

KNARsack: Teaching Neural Algorithmic Reasoners to Solve Pseudo-Polynomial Problems

Stjepan Požgaj, Dobrik Georgiev, Marin Šilić, Petar Veličković

TL;DR

This work extends Neural Algorithmic Reasoning to the Knapsack problem by formulating a two-phase NAR approach: (i) DP-table construction and (ii) solution reconstruction, within the CLRS-30 framework. It introduces edge-length encoding and a homogeneous processor to stabilize training and improve out-of-distribution generalization, achieving superior DP-table properties and reconstruction performance in many OOD regimes compared to baselines and Tropical Attention. The findings demonstrate that modeling intermediate DP states, rather than direct input-output mappings, yields better extrapolation to larger pseudo-polynomial instances, and the approach is readily transferable to other pseudo-polynomial problems. The work also outlines limitations—such as instability of deterministic reconstruction—and proposes concrete directions for extending pseudo-polynomial NAR to additional problems and architectures.

Abstract

Neural algorithmic reasoning (NAR) is a growing field that aims to embed algorithmic logic into neural networks by imitating classical algorithms. In this extended abstract, we detail our attempt to build a neural algorithmic reasoner that can solve Knapsack, a pseudo-polynomial problem bridging classical algorithms and combinatorial optimisation, but omitted in standard NAR benchmarks. Our neural algorithmic reasoner is designed to closely follow the two-phase pipeline for the Knapsack problem, which involves first constructing the dynamic programming table and then reconstructing the solution from it. The approach, which models intermediate states through dynamic programming supervision, achieves better generalization to larger problem instances than a direct-prediction baseline that attempts to select the optimal subset only from the problem inputs.

KNARsack: Teaching Neural Algorithmic Reasoners to Solve Pseudo-Polynomial Problems

TL;DR

This work extends Neural Algorithmic Reasoning to the Knapsack problem by formulating a two-phase NAR approach: (i) DP-table construction and (ii) solution reconstruction, within the CLRS-30 framework. It introduces edge-length encoding and a homogeneous processor to stabilize training and improve out-of-distribution generalization, achieving superior DP-table properties and reconstruction performance in many OOD regimes compared to baselines and Tropical Attention. The findings demonstrate that modeling intermediate DP states, rather than direct input-output mappings, yields better extrapolation to larger pseudo-polynomial instances, and the approach is readily transferable to other pseudo-polynomial problems. The work also outlines limitations—such as instability of deterministic reconstruction—and proposes concrete directions for extending pseudo-polynomial NAR to additional problems and architectures.

Abstract

Neural algorithmic reasoning (NAR) is a growing field that aims to embed algorithmic logic into neural networks by imitating classical algorithms. In this extended abstract, we detail our attempt to build a neural algorithmic reasoner that can solve Knapsack, a pseudo-polynomial problem bridging classical algorithms and combinatorial optimisation, but omitted in standard NAR benchmarks. Our neural algorithmic reasoner is designed to closely follow the two-phase pipeline for the Knapsack problem, which involves first constructing the dynamic programming table and then reconstructing the solution from it. The approach, which models intermediate states through dynamic programming supervision, achieves better generalization to larger problem instances than a direct-prediction baseline that attempts to select the optimal subset only from the problem inputs.

Paper Structure

This paper contains 24 sections, 4 equations, 15 figures, 5 tables.

Table of Contents

  1. Introduction
  2. NAR for Knapsack
  3. Problem Representation
  4. Pseudo-Polynomial Modeling
  5. Dynamic Programming Table Construction
  6. Solution Reconstruction
  7. Deterministic Reconstruction
  8. Evaluation
  9. Experimental Setup
  10. Models and Baselines
  11. Results
  12. Properties of Generated Scalar DP Tables
  13. Limitations and Future Work
  14. Related Work
  15. Other Pseudo-Polynomial Problems
  16. ...and 9 more sections

Figures (15)

  • Figure 1: Visualization of the Knapsack problem and its dynamic programming solution. Left: input items with their weights and values. Middle: dynamic programming table and decision table. Right: optimal solution with selected items. dp[i][c] -- optimal value for the first $i$ items with capacity $c$.
  • Figure 2: Categorical edge length encoding for the NAR construction model. Although we use $M=10$ in the implementation, here, due to visualization constraints, the case with $M=6$ is shown.
  • Figure 3: True DP tables
  • Figure 4: No edge length encoding
  • Figure 5: With edge length encoding
  • ...and 10 more figures