Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

What Matters in Hierarchical Search for Combinatorial Reasoning Problems?

Michał Zawalski, Gracjan Góral, Michał Tyrolski, Emilia Wiśnios, Franciszek Budrowski, Marek Cygan, Łukasz Kuciński, Piotr Miłoś

TL;DR

The paper analyzes when hierarchical subgoal search offers tangible advantages over traditional low-level planners for challenging combinatorial reasoning tasks. By training components with imitation learning on large, diverse datasets and evaluating on multiple NP-hard environments, it shows that subgoal methods excel when value estimates are noisy, action spaces are complex, and dead ends are prevalent, while their edge diminishes with homogeneous data or excessively long subgoals. The study introduces a consistent evaluation framework and demonstrates both empirical and theoretical results (including the search-advancement and action-densification analyses) that explain the observed performance gaps. The findings provide practical guidelines for when to deploy hierarchical search, emphasize fair-baseline reporting, and offer a foundation for future theoretical and empirical exploration. The work has implications for robotics and long-horizon planning, where data diversity and distribution shifts are common challenges.

Abstract

Efficiently tackling combinatorial reasoning problems, particularly the notorious NP-hard tasks, remains a significant challenge for AI research. Recent efforts have sought to enhance planning by incorporating hierarchical high-level search strategies, known as subgoal methods. While promising, their performance against traditional low-level planners is inconsistent, raising questions about their application contexts. In this study, we conduct an in-depth exploration of subgoal-planning methods for combinatorial reasoning. We identify the attributes pivotal for leveraging the advantages of high-level search: hard-to-learn value functions, complex action spaces, presence of dead ends in the environment, or using data collected from diverse experts. We propose a consistent evaluation methodology to achieve meaningful comparisons between methods and reevaluate the state-of-the-art algorithms.

What Matters in Hierarchical Search for Combinatorial Reasoning Problems?

TL;DR

The paper analyzes when hierarchical subgoal search offers tangible advantages over traditional low-level planners for challenging combinatorial reasoning tasks. By training components with imitation learning on large, diverse datasets and evaluating on multiple NP-hard environments, it shows that subgoal methods excel when value estimates are noisy, action spaces are complex, and dead ends are prevalent, while their edge diminishes with homogeneous data or excessively long subgoals. The study introduces a consistent evaluation framework and demonstrates both empirical and theoretical results (including the search-advancement and action-densification analyses) that explain the observed performance gaps. The findings provide practical guidelines for when to deploy hierarchical search, emphasize fair-baseline reporting, and offer a foundation for future theoretical and empirical exploration. The work has implications for robotics and long-horizon planning, where data diversity and distribution shifts are common challenges.

Abstract

Efficiently tackling combinatorial reasoning problems, particularly the notorious NP-hard tasks, remains a significant challenge for AI research. Recent efforts have sought to enhance planning by incorporating hierarchical high-level search strategies, known as subgoal methods. While promising, their performance against traditional low-level planners is inconsistent, raising questions about their application contexts. In this study, we conduct an in-depth exploration of subgoal-planning methods for combinatorial reasoning. We identify the attributes pivotal for leveraging the advantages of high-level search: hard-to-learn value functions, complex action spaces, presence of dead ends in the environment, or using data collected from diverse experts. We propose a consistent evaluation methodology to achieve meaningful comparisons between methods and reevaluate the state-of-the-art algorithms.
Paper Structure (73 sections, 5 theorems, 32 equations, 41 figures, 3 tables, 7 algorithms)

This paper contains 73 sections, 5 theorems, 32 equations, 41 figures, 3 tables, 7 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Solving Decision-Making Problems
  4. Subgoal Methods
  5. Low-level Search Algorithms
  6. Empirical Studies on Algorithmic Performance
  7. Combinatorial Environments
  8. Subgoal Methods
  9. Training Components
  10. Performance Metric
  11. Analysis
  12. Subgoal Methods Benefit from Diverse Sources of Data
  13. Takeaway
  14. Subgoal Methods are Value Noise Filters
  15. Takeaway
  16. ...and 58 more sections

Key Result

Theorem 1

Let $g_k: S\to \mathcal{P}(S)$ be a stochastic $k$-subgoal generator that, given a state $s\in S$ samples a set of $b$ subgoals $\{s_i\}$ such that the distances $d(s_i, s)$ are independent, uniformly distributed in the interval $[-k;k]$. Let $V:S\to\mathbb R$ be a value function with approximation where $\tilde{u}(x)$ is CDF of the sum of two uniform variables $U(-k,k)+U(-\sigma,\sigma)$. Additi

Figures (41)

  • Figure 1: Schematic performance comparison of hierarchical methods (AdaSubS, kSubS) and low-level methods ($\rho$-BestFS, $\rho$-A*, $\rho$-MCTS) across six dimensions: handling data collected from diverse sources, learning from clean unimodal demonstrations, avoiding dead ends, performance under high value approximation errors, handling complex action space, and generalizing to out-of-distribution instances.
  • Figure 2: Solving the Rubik's Cube. Components are trained on data from 4 different solvers.
  • Figure 3: Solving the N-Puzzle. Components are trained on data from 2 different solvers.
  • Figure 4: Solving the Rubik's Cube. Components are trained on reversed random shuffles.
  • Figure 5: Solving the Rubik's Cube. Components are trained on the Beginner algorithmic solver.
  • ...and 36 more figures

Theorems & Definitions (9)

  • Theorem 1: Search advancement formula
  • proof
  • Theorem 2: Densification of the action space
  • proof
  • Theorem 3: Search advancement formula, complete statement
  • proof
  • Theorem 4: Densification of the action space
  • Lemma 1
  • proof