Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On Policy Reuse: An Expressive Language for Representing and Executing General Policies that Call Other Policies

Blai Bonet, Dominik Drexler, Hector Geffner

TL;DR

Three extensions of this language aimed at making policies and sketches more flexible and reusable are considered: internal memory states, as in finite state controllers; indexical features, whose values are a function of the state and a number of internal registers that can be loaded with objects; and modules that wrap up policies and sketches and allow them to call each other by passing parameters.

Abstract

Recently, a simple but powerful language for expressing and learning general policies and problem decompositions (sketches) has been introduced in terms of rules defined over a set of Boolean and numerical features. In this work, we consider three extensions of this language aimed at making policies and sketches more flexible and reusable: internal memory states, as in finite state controllers; indexical features, whose values are a function of the state and a number of internal registers that can be loaded with objects; and modules that wrap up policies and sketches and allow them to call each other by passing parameters. In addition, unlike general policies that select state transitions rather than ground actions, the new language allows for the selection of such actions. The expressive power of the resulting language for policies and sketches is illustrated through a number of examples.

On Policy Reuse: An Expressive Language for Representing and Executing General Policies that Call Other Policies

TL;DR

Three extensions of this language aimed at making policies and sketches more flexible and reusable are considered: internal memory states, as in finite state controllers; indexical features, whose values are a function of the state and a number of internal registers that can be loaded with objects; and modules that wrap up policies and sketches and allow them to call each other by passing parameters.

Abstract

Recently, a simple but powerful language for expressing and learning general policies and problem decompositions (sketches) has been introduced in terms of rules defined over a set of Boolean and numerical features. In this work, we consider three extensions of this language aimed at making policies and sketches more flexible and reusable: internal memory states, as in finite state controllers; indexical features, whose values are a function of the state and a number of internal registers that can be loaded with objects; and modules that wrap up policies and sketches and allow them to call each other by passing parameters. In addition, unlike general policies that select state transitions rather than ground actions, the new language allows for the selection of such actions. The expressive power of the resulting language for policies and sketches is illustrated through a number of examples.
Paper Structure (12 sections, 1 theorem, 1 figure, 3 algorithms)

This paper contains 12 sections, 1 theorem, 1 figure, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Planning Problems and Sketches
  4. Serialized Width, Acyclicity, and Termination
  5. Extended Sketches
  6. Finite Memory
  7. Registers, Indexicals, Concepts, and Roles
  8. Formal Semantics and Termination
  9. Termination for Extended Sketches
  10. Reusable Modules
  11. Execution Model: $\text{SIW}_{\text{M}}$
  12. Discussion

Key Result

Theorem 10

If the extended sketch $R$ is reducible, terminating, and has a serialized width bounded by $k$ over the class of problems $\mathcal{Q}\xspace$, then $\text{SIW}^*_{\text{R}}$ finds plans for any problem $P$ in $\mathcal{Q}\xspace$ in polynomial time.

Figures (1)

  • Figure 1: Policy $\pi_{\textit{on}}\xspace^*$ places markers $\mathfrak{r}\xspace_{0}$ and $\mathfrak{r}\xspace_{1}$ on blocks $x$ and $b_3$, respectively, for an example tower with 4 blocks. (a) The rule $r_2$ puts the marker $\mathfrak{r}\xspace_{0}$ on the block $x{\,\in\,}\textsf{\footnotesize N}$; side effect is $\textsf{\footnotesize T}_0{\,=\,}\{b_1\}$. (b) Rule $r_3$ initializes marker $\mathfrak{r}\xspace_{1}$ on block $b_1{\,\in\,}\textsf{\footnotesize T}_0$; side effect is $\textsf{\footnotesize T}_1{\,=\,}\{b_2\}$. (c) Rule $r_4$ moves marker $\mathfrak{r}\xspace_{1}$ one step above to block $b_2{\,\in\,}\textsf{\footnotesize T}_1$; side effect is $\textsf{\footnotesize T}_1{\,=\,}\{b_3\}$. (d) Another application of $r_4$ moves $\mathfrak{r}\xspace_{1}$ on block $b_3{\,\in\,}\textsf{\footnotesize T}_1$; side effect is $\textsf{\footnotesize T}_1{\,=\,}\emptyset$, and $r_5$ is the next rule to apply.

Theorems & Definitions (10)

  • Definition 1: Sketches with Memory
  • Definition 2: Extended Sketch Rules
  • Definition 3: Extended Sketch
  • Definition 4: Augmented states
  • Definition 5: Compatible pairs
  • Definition 6: Subproblems
  • Definition 7: Reduction
  • Definition 8: Induced subproblems
  • Definition 9: Sketch width
  • Theorem 10: Termination