Predicate Invention for Bilevel Planning

TL;DR

This work tackles the core difficulty of scalable planning in continuous spaces by learning predicates that define a compact state abstraction tailored for bilevel planning. It introduces a surrogate objective, $J_{ ext{surr}}$, that tightly couples predicate invention to downstream planning efficiency and guides a grammar-based hill-climbing search to expand the predicate set $Ψ$. Once predicates are learned, operators and samplers are trained to enable fast abstract planning and concrete action refinements, yielding strong performance across four robotic domains with modest demonstration counts ($50$–$200$). The approach demonstrates substantial gains over baselines, including manually designed abstractions, and offers a principled, planning-aware path to richer state representations with potential extensions to active learning and partial observability.

Abstract

Efficient planning in continuous state and action spaces is fundamentally hard, even when the transition model is deterministic and known. One way to alleviate this challenge is to perform bilevel planning with abstractions, where a high-level search for abstract plans is used to guide planning in the original transition space. Previous work has shown that when state abstractions in the form of symbolic predicates are hand-designed, operators and samplers for bilevel planning can be learned from demonstrations. In this work, we propose an algorithm for learning predicates from demonstrations, eliminating the need for manually specified state abstractions. Our key idea is to learn predicates by optimizing a surrogate objective that is tractable but faithful to our real efficient-planning objective. We use this surrogate objective in a hill-climbing search over predicate sets drawn from a grammar. Experimentally, we show across four robotic planning environments that our learned abstractions are able to quickly solve held-out tasks, outperforming six baselines. Code: https://tinyurl.com/predicators-release

Figures (12)

• Figure 1: Overview of our framework. Given a small set of goal predicates (first panel, top), we use demonstration data to learn new predicates (first panel, bottom). In this Blocks example, the learned predicates P1 -- P4 intuitively represent Holding, NotHolding, HandEmpty, and NothingAbove respectively. Collectively, the predicates define a state abstraction that maps continuous states $x$ in the environment to abstract states $s$. Object types are omitted for clarity. After predicate invention, we learn abstractions of the continuous action space and transition model via planning operators (second panel). For each operator, we learn a sampler (third panel), a neural network that maps continuous object features in a given state to continuous action parameters for controllers which can be executed in the environment. In this example, the sampler proposes different placements on the table for the held block. With these learned representations, we perform bilevel planning (fourth panel), with search in the abstract spaces guiding planning in the continuous spaces.
• Figure 2: Predicate invention via hill climbing. (Left) An example task in Blocks. (Middle) Hill climbing over predicate sets, starting with the goal predicates $\Psi_G$. On each iteration, the single predicate that improves $J_{\text{surr}}$ the most is added to the set. The rightmost table column shows success rates on held-out evaluation tasks. Each iteration of hill climbing adds a predicate that causes all abstract plans above the dotted line to be pruned from consideration. At iteration 0, the robot believes it can achieve the goal by simply stacking b2 on b3 and b1 on b2, even though it hasn't picked up either block. The first step of this abstract plan (shown in red) is thus unrefinable. At iteration 1, a predicate with the intuitive meaning Holding is added, which makes the A$^*$ only consider abstract plans that pick up blocks before stacking them. Still, the abstract plan shown is unrefinable on the first step because b4 is obstructing b2 in the initial state. At iteration 2, a predicate with the intuitive meaning NothingAbove is added, which allows the agent to realize that it must move b4 out of the way if it wants to pick up b2. This plan is still unrefinable, though: the second step fails, because the abstraction still does not recognize that the robot cannot be holding two blocks simultaneously. Finally, at iteration 3, a predicate with the intuitive meaning HandEmpty is added, and planning succeeds.
• Figure 3: Ours versus baselines. Percentage of 50 evaluation tasks solved under a 10-second timeout, for all four environments. All results are averaged over 10 seeds. Black bars denote standard deviations. Learning times and additional metrics are reported in Appendix \ref{['app:results']}.
• Figure 4: Decomposing the surrogate objective. In these plots, each column corresponds to one environment. The x-axes correspond to sets of manually designed predicates. The predicate sets grow in size from left to right, starting with the goal predicates alone, adding one predicate at each tick mark, and concluding with the full set of manual predicates for the respective environment. The order that the predicates are added was determined by hill climbing with respect to the surrogate objective. The top row shows the surrogate objective itself; the middle row shows the plan cost error $|\textsc{Cost}(\hat{\pi}) - \textsc{Cost}(\pi^*)|$ minimized over the first 8 skeletons generated by abstract search; and the bottom row shows the total number of nodes created by the abstract search (our measure of abstract search time), cumulative over the 8 skeletons. There are two key takeaways from this plot. (1) The surrogate objective (first row) monotonically decreases in all environments; this smoothness makes local search over candidate predicate sets an attractive option. (2) Neither of the two components that make up the surrogate objective --- plan cost error (second row) or abstract search time (third row) --- has the same monotonically decreasing property on its own, suggesting that both parts are necessary for making our predicate invention pipeline work. All results are means over 10 seeds.
• Figure 5: PickPlace1D learned abstractions (top: predicates, bottom: operators).

Theorems & Definitions (7)

• Definition 1: Abstract state
• Definition 2: Operator
• Definition 3: Ground operator / abstract action
• Definition 4: Abstract transition model
• Definition 5: Sampler
• Definition 6: Ground sampler
• Definition 7: Abstract plan