Predictability-Based Curiosity-Guided Action Symbol Discovery

TL;DR

The paper tackles autonomous discovery of high-level symbolic actions and perceptual symbols to enable planning in manipulation tasks. It introduces a predictive encoder-decoder that yields discrete object and action symbols via bottleneck binarization, guided by an entropy-based curiosity signal to explore informative actions. A BFS planner operates on the learned primitives, with a parameter-distillation step to convert symbols back into executable continuous actions. Empirical results show that curiosity-driven exploration yields a richer set of action primitives and higher planning success than baselines, advancing developmental robotics toward autonomous skill discovery and generalizable planning in manipulation tasks.

Abstract

Discovering symbolic representations for skills is essential for abstract reasoning and efficient planning in robotics. Previous neuro-symbolic robotic studies mostly focused on discovering perceptual symbolic categories given a pre-defined action repertoire and generating plans with given action symbols. A truly developmental robotic system, on the other hand, should be able to discover all the abstractions required for the planning system with minimal human intervention. In this study, we propose a novel system that is designed to discover symbolic action primitives along with perceptual symbols autonomously. Our system is based on an encoder-decoder structure that takes object and action information as input and predicts the generated effect. To efficiently explore the vast continuous action parameter space, we introduce a Curiosity-Based exploration module that selects the most informative actions -- the ones that maximize the entropy in the predicted effect distribution. The discovered symbolic action primitives are then used to make plans using a symbolic tree search strategy in single- and double-object manipulation tasks. We compare our model with two baselines that use different exploration strategies in different experiments. The results show that our approach can learn a diverse set of symbolic action primitives, which are effective for generating plans in order to achieve given manipulation goals.

Figures (4)

• Figure 1: Overview of the proposed method. The effect prediction model is an encoder-decoder deep neural network that predicts a distribution over the effect. The entropy of the distribution is given to the curiosity-based exploration module to guide the action selection process. The object is randomly initialized in the environment. The symbols are generated via binarization of the bottleneck layer of the effect prediction model.
• Figure 2: Overview of the proposed effect prediction model. The first image shows the PyBullet environment with an example object. The blue, green, and red dots show the start, middle, and end points of the robot's trajectory, respectively. The action and object encoders generate latent embeddings, which are then concatenated. The decoder uses these combined embeddings to predict a Gaussian distribution over the object displacement caused by the action. The symbols are then extracted from the latent embeddings with the given method in Section \ref{['parameter_distill']}.
• Figure 3: Example of the learned action primitives in the Curiosity-driven model. From top to bottom; pick up, forward push, left push, pull.
• Figure 4: Example plans generated by BFS with the learned primitives in Curiosity-Based Model. The goal state is shown transparently in the first column, and the initial state is shown so in the last column to emphasize the effect.