CLIMB: Language-Guided Continual Learning for Task Planning with Iterative Model Building

TL;DR

The ability of CLIMB to improve performance in common planning environments compared to baseline methods is demonstrated and the BlocksWorld++ domain, a simulated environment with an easily usable real counterpart, is developed.

Abstract

Intelligent and reliable task planning is a core capability for generalized robotics, requiring a descriptive domain representation that sufficiently models all object and state information for the scene. We present CLIMB, a continual learning framework for robot task planning that leverages foundation models and execution feedback to guide domain model construction. CLIMB can build a model from a natural language description, learn non-obvious predicates while solving tasks, and store that information for future problems. We demonstrate the ability of CLIMB to improve performance in common planning environments compared to baseline methods. We also develop the BlocksWorld++ domain, a simulated environment with an easily usable real counterpart, together with a curriculum of tasks with progressing difficulty for evaluating continual learning. Additional details and demonstrations for this system can be found at https://plan-with-climb.github.io/ .

Figures (6)

• Figure 1: CLIMB is able to predict common world constraints without ground truth PDDL and can learn domain-specific relationships through the execution of one or more tasks in the domain. The framework can self-improve by storing domain and predicate information across tasks.
• Figure 2: The CLIMB Planning Framework includes multiple independent modules for problem translation, planning, predicate generation, verification, execution, and perception.
• Figure 3: We evaluate CLIMB across logical, simulated, and real domains. Logical domains (BlocksWorld, Grippers, and Heavy) provide a convenient mechanism for comparing performance across a variety of robot embodiments. Experiments in IsaacLab and Real environments extend the block manipulation setting to include more complex stacking and arranging tasks. The simulated and real also serve to evaluate predicate grounding and learning from real experience.
• Figure 4: Evaluation of the continual learning capabilities of the CLIMB framework on the BlocksWorld logical dataset. The baseline in this case is the same planner without saving domain and predicate information between problems. CLIMB with continual learning significantly outperforms the baseline.
• Figure 5: Expanding on the BlocksWorld problem formulation, we have developed the BlocksWorld++ dataset which adds additional levels of complexity to the traditional BlocksWorld environment. It includes relative 2D placement on the table and stacking across multiple blocks.