Learning Type-Generalized Actions for Symbolic Planning

TL;DR

The paper addresses the transferability gap in symbolic planning caused by hand-crafted state/action representations. It introduces a two-component method: learning type-generalized actions from few observations using an entity hierarchy, and on-the-fly imagined actions during planning to cover unseen scenarios. Experiments in a simulated kitchen demonstrate that combining learning and imagination enables robust generalization to novel entities and longer action sequences, while still supporting fast planning on familiar tasks. The findings highlight a path toward incremental, lifelong planning systems that adapt with minimal supervision and human feedback.

Abstract

Symbolic planning is a powerful technique to solve complex tasks that require long sequences of actions and can equip an intelligent agent with complex behavior. The downside of this approach is the necessity for suitable symbolic representations describing the state of the environment as well as the actions that can change it. Traditionally such representations are carefully hand-designed by experts for distinct problem domains, which limits their transferability to different problems and environment complexities. In this paper, we propose a novel concept to generalize symbolic actions using a given entity hierarchy and observed similar behavior. In a simulated grid-based kitchen environment, we show that type-generalized actions can be learned from few observations and generalize to novel situations. Incorporating an additional on-the-fly generalization mechanism during planning, unseen task combinations, involving longer sequences, novel entities and unexpected environment behavior, can be solved.

Figures (4)

• Figure 1: Simplified sketch of the action learning and creating algorithms, showcased on learning to chop different foods. First, Algorithm \ref{['alg:learning']} learns a set of individual actions from observations, i.e., individual actions for cutting lettuce, banana, tomato, and onion with the cutboard. Next, these individual actions are generalized based on their effects and the entity hierarchy to create a generalized cutting action for food. During planning, Algorithm \ref{['alg:imagine']} can create imagined generalized actions, i.e., proposed generalizations based on effect, entity hierarchy, and available entities in the current planning problem. Note, action preconditions (PRE) and effects (EFF) show a simplified subset for visualization and explainability. Shown are the learned lifted actions, where ?x0 is the variable name followed by its type.
• Figure 2: Screenshot of the kitchen scenario, showing the agent and different entities (food and tools) to interact with.
• Figure 3: Visualization of the used entity hierarchy $\mathcal{H}$, with parents on the y-axis and children on the x-axis. A dot indicates a connection, the color indicates the depth levels inside the hierarchy (top to bottom), and * indicates the entity was seen in the training data.
• Figure 4: Fraction of tasks where a successful plan was found per number of demonstrations used for learning. The shaded violion shows the success ratio distribution of the learned action sets, and the colored bar marks the median.