Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Heuristic Search for Multi-Objective Probabilistic Planning

Dillon Chen, Felipe Trevizan, Sylvie Thiébaux

TL;DR

New heuristic search algorithms MOLAO* and MOLRTDP are designed, which extend well-known SSP algorithms to the multi-objective case and construct a spectrum of domain-independent heuristic functions differing in their ability to take into account the stochastic and multi- objective features of the problem to guide the search.

Abstract

Heuristic search is a powerful approach that has successfully been applied to a broad class of planning problems, including classical planning, multi-objective planning, and probabilistic planning modelled as a stochastic shortest path (SSP) problem. Here, we extend the reach of heuristic search to a more expressive class of problems, namely multi-objective stochastic shortest paths (MOSSPs), which require computing a coverage set of non-dominated policies. We design new heuristic search algorithms MOLAO* and MOLRTDP, which extend well-known SSP algorithms to the multi-objective case. We further construct a spectrum of domain-independent heuristic functions differing in their ability to take into account the stochastic and multi-objective features of the problem to guide the search. Our experiments demonstrate the benefits of these algorithms and the relative merits of the heuristics.

Heuristic Search for Multi-Objective Probabilistic Planning

TL;DR

New heuristic search algorithms MOLAO* and MOLRTDP are designed, which extend well-known SSP algorithms to the multi-objective case and construct a spectrum of domain-independent heuristic functions differing in their ability to take into account the stochastic and multi- objective features of the problem to guide the search.

Abstract

Heuristic search is a powerful approach that has successfully been applied to a broad class of planning problems, including classical planning, multi-objective planning, and probabilistic planning modelled as a stochastic shortest path (SSP) problem. Here, we extend the reach of heuristic search to a more expressive class of problems, namely multi-objective stochastic shortest paths (MOSSPs), which require computing a coverage set of non-dominated policies. We design new heuristic search algorithms MOLAO* and MOLRTDP, which extend well-known SSP algorithms to the multi-objective case. We further construct a spectrum of domain-independent heuristic functions differing in their ability to take into account the stochastic and multi-objective features of the problem to guide the search. Our experiments demonstrate the benefits of these algorithms and the relative merits of the heuristics.
Paper Structure (17 sections, 1 theorem, 7 equations, 4 figures, 3 tables, 5 algorithms)

This paper contains 17 sections, 1 theorem, 7 equations, 4 figures, 3 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Value Iteration for MOSSPs
  4. Heuristic search
  5. Heuristics
  6. (i)MOLAO$^*$
  7. MOLRTDP
  8. Experiments
  9. Benchmarks
  10. $k$-d Exploding Blocksworld
  11. MO Search and Rescue
  12. MO Triangle Tireworld
  13. Probabilistic VisitAll
  14. VisitAllTire
  15. Setup
  16. ...and 2 more sections

Key Result

Theorem 3.1

Given an MOSSP in which the reachability and weak improper policy assumptions hold for an upper bound $\vec{b}$, and given a set of vectors ${\mathbf V}^0$ such that ${\mathbf V}^0 \preceq {\mathbf V}^*$, the sequence ${\mathbf V}^1, \ldots, {\mathbf V}^k$ computed by MOVI converges to the MOSSP sol

Figures (4)

  • Figure 1: A MOSSP with action costs given by $\vec{C}(a_1) = [1,0]$ and $\vec{C}(a_2) = [0, 1]$.
  • Figure 2: An MOSSP with action costs given by $\vec{C}(a_1) = [1,0], {\vec{C}(a_2) = [1, 0]}, \vec{C}(a_g) = [0,1]$.
  • Figure 3: Boxplot of CCS size of instances across domains which have been solved at least once. Number of solved and total instances for each domain is indicated in parentheses.
  • Figure 4: Cumulative coverage of: (a) planners and heuristics combinations (low-performing planners omitted); (b) planners only, i.e., summation across different heuristics; (c) heuristics only, i.e. summation across different planners; (d) PDB approaches considered, i.e., summation across the ideal-point, MO, and MOSSP approaches. Notice that the $x$-axis is the same for all plots but the $y$-axis is different and might not start at 0.

Theorems & Definitions (4)

  • Theorem 3.1
  • proof : Proof sketch
  • Definition 4.1
  • Definition 4.2