RDD: Retrieval-Based Demonstration Decomposer for Planner Alignment in Long-Horizon Tasks

TL;DR

The paper tackles the challenge of aligning high-level VLM planners with low-level visuomotor policies in long-horizon robotic tasks. It introduces the Retrieval-based Demonstration Decomposer (RDD), a training-free method that decomposes demonstrations by retrieving sub-task intervals from the planner's training data and formulating decomposition as an optimal partitioning problem solved by dynamic programming. The interval scoring combines visual embeddings and temporal cues via an approximate nearest-neighbor search, enabling sub-task segments that closely match learned policy capabilities, with an option to handle OOD sub-tasks. Empirical results on RLBench and real-world benchmarks show that RDD surpasses heuristic decomposers like UVD, achieves near-oracle performance, and demonstrates data efficiency and scalability, validating its effectiveness for planner-policy coordination in hierarchical VLA systems.

Abstract

To tackle long-horizon tasks, recent hierarchical vision-language-action (VLAs) frameworks employ vision-language model (VLM)-based planners to decompose complex manipulation tasks into simpler sub-tasks that low-level visuomotor policies can easily handle. Typically, the VLM planner is finetuned to learn to decompose a target task. This finetuning requires target task demonstrations segmented into sub-tasks by either human annotation or heuristic rules. However, the heuristic subtasks can deviate significantly from the training data of the visuomotor policy, which degrades task performance. To address these issues, we propose a Retrieval-based Demonstration Decomposer (RDD) that automatically decomposes demonstrations into sub-tasks by aligning the visual features of the decomposed sub-task intervals with those from the training data of the low-level visuomotor policies. Our method outperforms the state-of-the-art sub-task decomposer on both simulation and real-world tasks, demonstrating robustness across diverse settings. Code and more results are available at rdd-neurips.github.io.

Figures (4)

• Figure 1: The core idea of RDD. (a) Two sub-tasks appear in the visuomotor policy's training set, on which the policy has been optimized. (b) Existing sub-task decomposers, such as UVD zhang2024universal, use heuristic decomposition rules and may generate "unfamiliar" sub-tasks that are difficult to handle for the low-level visuomotor policy. (c) In contrast, RDD decomposes the demonstration into sub-tasks that are visually similar to the ones in the training set of the visuomotor policy. The sub-tasks are then used to finetune the high-level planner, which gives sub-task instructions to the low-level visuomotor policy and guides it to finish the task step-by-step.
• Figure 2: RDD formulates demonstration decomposition as an optimal partitioning problem. Intervals colored in green are proposed segments of the demonstration ${\mathcal{S}}^i$, and ones colored in blue are retrieved from the visuomotor policy's training set ${{\mathcal{D}}^\text{train}_\text{aug}}$.
• Figure 3: Linear scaling of running time of Algorithm \ref{['alg-dp-solver']} with ${L_{\text{max}}}$.
• Figure 4: Qualitative results of RDD and UVD functioning on both real-world (AgiBotWorld) and simulation (RLBench and LIBERO) benchmarks. Blocks outlined in black are sub-tasks decomposed by the same task-specific heuristic used in the visuomotor policy's training set; blocks outlined in green are sub-tasks found by RDD; and blocks outlined in red are sub-tasks found by UVD.

Theorems & Definitions (6)

• Theorem 3.1: Principle of Optimality jackson2005algorithm
• Corollary 3.1.1
• Definition 3.1
• Proposition 3.1
• proof
• proof