Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Count-based Novelty Exploration in Classical Planning

Giacomo Rosa, Nir Lipovetzky

TL;DR

Facing exploration bottlenecks in Classical Planning due to exponential growth of tuples, the paper introduces classical count-based novelty, which leverages the frequency of $k$-tuples in search histories to guide exploration. It provides theoretical links between counts, Hamming distance, and the expected number of novel tuples, and proposes a memory-efficient Trimmed Open List along with the BFNoS frontend for dual-configuration planning. Experimental results on IPC benchmarks show that count-based novelty, especially when combined with partitioning and trimmed lists, yields competitive coverage and complements existing novelty heuristics; memory-threshold hybrid configurations substantially boost instance coverage. The work bridges classical planning with count-based exploration concepts and offers practical, scalable tools for memory-aware search, with potential cross-domain relevance to reinforcement learning.

Abstract

Count-based exploration methods are widely employed to improve the exploratory behavior of learning agents over sequential decision problems. Meanwhile, Novelty search has achieved success in Classical Planning through recording of the first, but not successive, occurrences of tuples. In order to structure the exploration, however, the number of tuples considered needs to grow exponentially as the search progresses. We propose a new novelty technique, classical count-based novelty, which aims to explore the state space with a constant number of tuples, by leveraging the frequency of each tuple's appearance in a search tree. We then justify the mechanisms through which lower tuple counts lead the search towards novel tuples. We also introduce algorithmic contributions in the form of a trimmed open list that maintains a constant size by pruning nodes with bad novelty values. These techniques are shown to complement existing novelty heuristics when integrated in a classical solver, achieving competitive results in challenging benchmarks from recent International Planning Competitions. Moreover, adapting our solver as the frontend planner in dual configurations that utilize both memory and time thresholds demonstrates a significant increase in instance coverage, surpassing current state-of-the-art solvers.

Count-based Novelty Exploration in Classical Planning

TL;DR

Facing exploration bottlenecks in Classical Planning due to exponential growth of tuples, the paper introduces classical count-based novelty, which leverages the frequency of -tuples in search histories to guide exploration. It provides theoretical links between counts, Hamming distance, and the expected number of novel tuples, and proposes a memory-efficient Trimmed Open List along with the BFNoS frontend for dual-configuration planning. Experimental results on IPC benchmarks show that count-based novelty, especially when combined with partitioning and trimmed lists, yields competitive coverage and complements existing novelty heuristics; memory-threshold hybrid configurations substantially boost instance coverage. The work bridges classical planning with count-based exploration concepts and offers practical, scalable tools for memory-aware search, with potential cross-domain relevance to reinforcement learning.

Abstract

Count-based exploration methods are widely employed to improve the exploratory behavior of learning agents over sequential decision problems. Meanwhile, Novelty search has achieved success in Classical Planning through recording of the first, but not successive, occurrences of tuples. In order to structure the exploration, however, the number of tuples considered needs to grow exponentially as the search progresses. We propose a new novelty technique, classical count-based novelty, which aims to explore the state space with a constant number of tuples, by leveraging the frequency of each tuple's appearance in a search tree. We then justify the mechanisms through which lower tuple counts lead the search towards novel tuples. We also introduce algorithmic contributions in the form of a trimmed open list that maintains a constant size by pruning nodes with bad novelty values. These techniques are shown to complement existing novelty heuristics when integrated in a classical solver, achieving competitive results in challenging benchmarks from recent International Planning Competitions. Moreover, adapting our solver as the frontend planner in dual configurations that utilize both memory and time thresholds demonstrates a significant increase in instance coverage, surpassing current state-of-the-art solvers.
Paper Structure (26 sections, 8 theorems, 28 equations, 7 figures, 6 tables, 1 algorithm)

This paper contains 26 sections, 8 theorems, 28 equations, 7 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Background
  2. Classical Planning
  3. Width-Based Search
  4. Count-based exploration
  5. Count Based Novelty
  6. Theoretical results
  7. Parent-child average distance comparison
  8. Estimating novel k-tuples
  9. Trimmed Open List
  10. Experiments
  11. Count-based solvers
  12. Analysis of proposed techniques
  13. Hybrid solvers with BFNoS frontend
  14. Memory threshold
  15. Discussion
  16. ...and 11 more sections

Key Result

Proposition 1

The Hamming distance $H(s, s')$ between states $s$ and $s'$ equals the true Hamming distance $H_{\text{true}}(s_{\text{neg}}, s'_{\text{neg}})$, considering only variables in $s_{\text{neg}}$ that are true.

Figures (7)

  • Figure 1: $\mathbb{E}[\text{\# novel k-tuple}]$ according to Theorem \ref{['prob-theorem-2']}. In (a) $N$ is a variable, and in (b) $\alpha(s)$ is a variable. Otherwise, parameters are set as $L=100$, $t=50000$, $N=5$, $\alpha(s)=0.3$. A realistic $\alpha(s)$ value was determined through simulation$^1$.
  • Figure 2: Number of nodes expanded across instances solved by $\text{BFNoS}$ and $\text{BFWS}_t(f_5)$. Blue crosses represent instances not solved by at least one planner.
  • Figure 3: Cumulative % of all failures attributed to search-time memory failures ($y$-axis) vs. time of failure (sec) ($x$-axis), for BFNoS solvers with 6 GB and 8 GB memory and 1800 sec time limits.
  • Figure 4: Instance coverage ($y$-axis) vs. time (sec) ($x$-axis). Comparison of BFNoS with the three presented hybrid configurations. The solvers are run in an identical configuration to previous sections, with BFNoS using the full 8 GB memory allowance when run on its own, and a 6 GB memory threshold when run as the frontend of hybrid configurations. The vertical lines signal the 1400 sec (green) and 1600 sec (red) time thresholds.
  • Figure 5: Plan cost over instances solved by $\text{BFNoS}$ and $\text{BFWS}_t(f_5)$. Blue crosses represent instances not solved by at least one planner.
  • ...and 2 more figures

Theorems & Definitions (22)

  • Definition 1: Classical count-based novelty
  • Definition 2: Partitioned classical count-based novelty
  • Proposition 1
  • proof
  • Proposition 2
  • proof
  • Theorem 3
  • proof
  • Theorem 4
  • proof
  • ...and 12 more