Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On Parallel External-Memory Bidirectional Search

Lior Siag, Shahaf S. Shperberg, Ariel Felner, Nathan R. Sturtevant

TL;DR

This work addresses scaling bidirectional heuristic search to very large problems by integrating BiHS with parallel external-memory (PEM) frameworks. It introduces a flexible PEM-BiHS framework and provides PEM-BAE*, a PEM variant of the state-of-the-art BiHS algorithm BAE*, which leverages consistent heuristics and a carefully designed bucket structure to outperform unidirectional PEM variants (A*, rA*, MM) and parallel IDA* on large 15-puzzle, 24-puzzle, and Towers of Hanoi problems. The key contributions include a generalized PEM-BiHS framework, a petabyte-scale evaluation of BiHS versus UniHS under strong heuristics, and insights into I/O trade-offs, delayed duplicate detection, and thread scalability. The results demonstrate that BiHS, when coupled with external memory and parallelization, can surpass unidirectional approaches on large, hard problems, informing future directions in PEM search design and optimization.

Abstract

Parallelization and External Memory (PEM) techniques have significantly enhanced the capabilities of search algorithms when solving large-scale problems. Previous research on PEM has primarily centered on unidirectional algorithms, with only one publication on bidirectional PEM that focuses on the meet-in-the-middle (MM) algorithm. Building upon this foundation, this paper presents a framework that integrates both uni- and bi-directional best-first search algorithms into this framework. We then develop a PEM variant of the state-of-the-art bidirectional heuristic search (BiHS) algorithm BAE* (PEM-BAE*). As previous work on BiHS did not focus on scaling problem sizes, this work enables us to evaluate bidirectional algorithms on hard problems. Empirical evaluation shows that PEM-BAE* outperforms the PEM variants of A* and the MM algorithm, as well as a parallel variant of IDA*. These findings mark a significant milestone, revealing that bidirectional search algorithms clearly outperform unidirectional search algorithms across several domains, even when equipped with state-of-the-art heuristics.

On Parallel External-Memory Bidirectional Search

TL;DR

This work addresses scaling bidirectional heuristic search to very large problems by integrating BiHS with parallel external-memory (PEM) frameworks. It introduces a flexible PEM-BiHS framework and provides PEM-BAE*, a PEM variant of the state-of-the-art BiHS algorithm BAE*, which leverages consistent heuristics and a carefully designed bucket structure to outperform unidirectional PEM variants (A*, rA*, MM) and parallel IDA* on large 15-puzzle, 24-puzzle, and Towers of Hanoi problems. The key contributions include a generalized PEM-BiHS framework, a petabyte-scale evaluation of BiHS versus UniHS under strong heuristics, and insights into I/O trade-offs, delayed duplicate detection, and thread scalability. The results demonstrate that BiHS, when coupled with external memory and parallelization, can surpass unidirectional approaches on large, hard problems, informing future directions in PEM search design and optimization.

Abstract

Parallelization and External Memory (PEM) techniques have significantly enhanced the capabilities of search algorithms when solving large-scale problems. Previous research on PEM has primarily centered on unidirectional algorithms, with only one publication on bidirectional PEM that focuses on the meet-in-the-middle (MM) algorithm. Building upon this foundation, this paper presents a framework that integrates both uni- and bi-directional best-first search algorithms into this framework. We then develop a PEM variant of the state-of-the-art bidirectional heuristic search (BiHS) algorithm BAE* (PEM-BAE*). As previous work on BiHS did not focus on scaling problem sizes, this work enables us to evaluate bidirectional algorithms on hard problems. Empirical evaluation shows that PEM-BAE* outperforms the PEM variants of A* and the MM algorithm, as well as a parallel variant of IDA*. These findings mark a significant milestone, revealing that bidirectional search algorithms clearly outperform unidirectional search algorithms across several domains, even when equipped with state-of-the-art heuristics.
Paper Structure (25 sections, 4 equations, 5 figures, 4 tables, 2 algorithms)

This paper contains 25 sections, 4 equations, 5 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Background and Definitions
  2. BiHS: Definitions and Algorithms
  3. Meet in the middle (MM).
  4. BAE*.
  5. Parallel External-Memory Search
  6. The PEM-BiHS Framework
  7. State-space Representation
  8. Bucket Structure
  9. Direction, Prioritization, and Lower-bound
  10. Reading Buckets
  11. In-bucket Duplicate Detection
  12. Duplicate Detection against Closed
  13. Parallel Node Expansion
  14. Solution Detection
  15. Parallel External-Memory BAE*
  16. ...and 10 more sections

Figures (5)

  • Figure 1: 24-puzzle results: (left) runtime, (right) node expansions
  • Figure 2: ToH4 16+4 results: (left) runtime, (right) node expansions.
  • Figure 3: Distribution of node expansions in ToH4 instance with solution length of 184
  • Figure 4: Avg. runtime as a function of number of threads in 24-Puzzle
  • Figure 5: Avg. nodes expanded per second; $^H$ indicates the hard instance in STP.