Hierarchical DLO Routing with Reinforcement Learning and In-Context Vision-language Models

TL;DR

Long-horizon routing of deformable linear objects is challenging due to nonlinear dynamics and the need for multi-step planning. The authors propose a hierarchical framework that combines a vision-language model–based in-context planner with reinforcement-learning–driven low-level skills (Insert, Pull, Flatten) and a failure-recovery mechanism. The method demonstrates strong generalization to multi-clip setups (including four clips) and robust performance in both simulation and real robots, achieving up to 92.5% success and significant improvements over baselines. The work highlights the value of integrating high-level semantic planning with task-aware low-level control for scalable, reliable deformable manipulation.

Abstract

Long-horizon routing tasks of deformable linear objects (DLOs), such as cables and ropes, are common in industrial assembly lines and everyday life. These tasks are particularly challenging because they require robots to manipulate DLO with long-horizon planning and reliable skill execution. Successfully completing such tasks demands adapting to their nonlinear dynamics, decomposing abstract routing goals, and generating multi-step plans composed of multiple skills, all of which require accurate high-level reasoning during execution. In this paper, we propose a fully autonomous hierarchical framework for solving challenging DLO routing tasks. Given an implicit or explicit routing goal expressed in language, our framework leverages vision-language models~(VLMs) for in-context high-level reasoning to synthesize feasible plans, which are then executed by low-level skills trained via reinforcement learning. To improve robustness in long horizons, we further introduce a failure recovery mechanism that reorients the DLO into insertion-feasible states. Our approach generalizes to diverse scenes involving object attributes, spatial descriptions, as well as implicit language commands. It outperforms the next best baseline method by nearly 50% and achieves an overall success rate of 92.5% across long-horizon routing scenarios.

Figures (6)

• Figure 1: Hierarchical DLO routing framework. Our framework combines high-level planning via a VLM with in-context learning and low-level control via an RL policy. The VLM generates routing plans and handles failure recovery, while the RL policy executes precise manipulation. This framework enables recovery from insertion failures through reinitialization, generalizes from three-clip to multi-clip routing, and produces smooth paths that avoid sharp turns.
• Figure 2: Pipeline of the proposed hierarchical DLO routing framework. The high-level VLM-based planner processes top-down scene images, task prompts, and auxiliary information to select appropriate skills, including routing (insertion and pulling) and failure recovery (flattening). Insertion is performed by a safe, low-level RL–based parameterized motion primitive for precise manipulation. A failure detection and recovery module (inference only) monitors execution to identify action, perception, or motion-planning failures and triggers replanning, enabling robust recovery and successful long-horizon routing. Note that side-view images shown in the second row are for visualization only; the planner receives only top-down views.
• Figure 3: Illustration of the low-level action space for DLO routing skills. The primitive set includes Flatten (top left) and Pull actions (top right), and the Insert action (bottom). For insertion skill, the gripper (in orange) operates within a $0.16m \times 0.16m$ space with orientation ($\textbf{p}_g^t$, $\textbf{q}_g^t$) conditioned on the clip (in black) state ($\textbf{p}_{c}$, $\textbf{q}_{c}$). The DLO is represented by particles $\{\textbf{p}_1, ..., \textbf{p}_n\}$.
• Figure 4: Long-horizon routing examples in simulation using our VLM with failure reasoning and RL-trained low-level policy. (a) Implicit order forming a V-shape. (b–c) Fixed order based on color attributes (red, green, blue). (d) Fixed order determined by spatial reasoning. (e) Fixed order based on color attributes in a four-clip arrangement.
• Figure 5: Real-robot execution of the proposed hierarchical DLO routing framework. The robot receives both a whole-scene view and a zoomed-in view centered on the current clip. During normal execution, it alternates between insertion and pulling actions. When insertion becomes unlikely due to unfavorable DLO configurations (e.g., alignment along the clip’s long axis or sliding past the clip without entering), the system detects the failure, executes a flatten action to reinitialize the DLO, and then resumes insertion successfully.