NeuralSolver: Learning Algorithms For Consistent and Efficient Extrapolation Across General Tasks

TL;DR

It is shown that NeuralSolver consistently outperforms the prior state-of-the-art recurrent solvers in extrapolating to larger problems, considering smaller training problems and requiring less parameters than other approaches.

Abstract

We contribute NeuralSolver, a novel recurrent solver that can efficiently and consistently extrapolate, i.e., learn algorithms from smaller problems (in terms of observation size) and execute those algorithms in large problems. Contrary to previous recurrent solvers, NeuralSolver can be naturally applied in both same-size problems, where the input and output sizes are the same, and in different-size problems, where the size of the input and output differ. To allow for this versatility, we design NeuralSolver with three main components: a recurrent module, that iteratively processes input information at different scales, a processing module, responsible for aggregating the previously processed information, and a curriculum-based training scheme, that improves the extrapolation performance of the method. To evaluate our method we introduce a set of novel different-size tasks and we show that NeuralSolver consistently outperforms the prior state-of-the-art recurrent solvers in extrapolating to larger problems, considering smaller training problems and requiring less parameters than other approaches.

Figures (31)

• Figure 1: Observations of the 1S-Maze environment of sizes 7$\times$7, 11$\times$11, 33$\times$33 and 129$\times$129, where the agent (green square) must go to the goal (red square). The light green arrow represents the next target action that the model needs to predict, while the purple path represents the sequence of actions required to solve the maze.
• Figure 2: We design NeuralSolver with two fundamental components: (i) a recurrent module (purple), responsible for iteratively processing the input data regardless of its size; (ii) processing module (green), with an optional aggregation layer (A), responsible for generating the output and allowing our architecture to be used both in same-size and different-size tasks. Additionally, we employ a curriculum-based training scheme to improve the extrapolation performance of our architecture.
• Figure 3: Propagation of information in NeuralSolver in a maze-like environment: the goal is for the agent (green) to find the goal position (red). Top: the difference between the value of the internal state of the recurrent module at each iteration step and the value at the final iteration. Larger differences are shown in dark blue and smaller differences in white. Bottom: additionally, we show the action probabilities predicted by the processing module of the model at different iterations, where the agent can move right (R), down (D), left (L), or up (U).
• Figure 4: We introduce a set of different-size classification tasks to evaluate the performance of recurrent solvers. In all tasks, the input is an image observation of the environment with arbitrary size. The output is an $n$-dimensional one-hot vector with: a) $n=4$; b) $n=4$; c) $n=3$; d) $n=4$.
• Figure 5: Training efficiency of NeuralSolver and Bansal2022endtoendalgorithmsynthesis on same-size tasks: we present the accuracy of the learned algorithms on extrapolating to problems with different dimensionality (columns). Each color represents a different training size, specific to each task, detailed in Appendix \ref{['appendix:training_efficiency_sizes_description']} In the dashed line we show the upper-bound on the performance.