Fast and Precise: Adjusting Planning Horizon with Adaptive Subgoal Search

TL;DR

The paper addresses the challenge of solving hard planning problems where the cost of determining an action plan varies across states. It introduces AdaSubS, an adaptive subgoal search algorithm that generates subgoals at multiple horizons, uses a verifier to filter unreachable candidates, and employs a value function within a Best-First Search to prioritize exploration. By combining multiple $k$-subgoal generators, a verifier, a conditional low-level policy, and offline training, AdaSubS achieves superior performance across Sokoban, Rubik's Cube, and the INT theorem prover, often outperforming non-adaptive baselines and exhibiting notable out-of-distribution generalization. The approach offers scalable, data-driven planning that leverages longer horizons in easier regions and shorter ones in harder regions, with strong practical implications for complex reasoning tasks. The work also provides a thorough experimental comparison of adaptive strategies and discusses limitations and avenues for future work, including stochastic dynamics and online learning.

Abstract

Complex reasoning problems contain states that vary in the computational cost required to determine a good action plan. Taking advantage of this property, we propose Adaptive Subgoal Search (AdaSubS), a search method that adaptively adjusts the planning horizon. To this end, AdaSubS generates diverse sets of subgoals at different distances. A verification mechanism is employed to filter out unreachable subgoals swiftly, allowing to focus on feasible further subgoals. In this way, AdaSubS benefits from the efficiency of planning with longer subgoals and the fine control with the shorter ones, and thus scales well to difficult planning problems. We show that AdaSubS significantly surpasses hierarchical planning algorithms on three complex reasoning tasks: Sokoban, the Rubik's Cube, and inequality proving benchmark INT.

Figures (8)

• Figure 1: An example iteration of the search performed by AdaSubS.
• Figure 2: Success rates of AdaSubS, kSubS, and BestFS expressed in terms of graph size. The figure in the bottom right shows the out-of-distribution performance of methods evaluated on INT with proof length $20$ but trained on length $15$. The remaining figures present in-distribution performance. The results were measured on a fixed set of 1000 problems for each domain. Shaded areas indicate 95% confidence intervals.
• Figure 3: Out-of-distribution performance of AdaSubS and kSubS for long proofs in INT with budget of $5000$ nodes. Both methods were trained on proofs of length $15$. Error bars correspond to $95\%$ confidence intervals.
• Figure 4: Comparison of success rates for different subgoal generators for Sokoban. AdaSubS-$k$ describes using a single generator with distance $k$.
• Figure 5: Out-of-distribution performance of AdaSubS and kSubS for Sokoban boards with more boxes, with budget of $5000$ nodes. Both methods were trained on instances with 4 boxes. Error bars correspond to $95\%$ confidence intervals.