Intent-Driven LLM Ensemble Planning for Flexible Multi-Robot Disassembly: Demonstration on EV Batteries

TL;DR

This work tackles the challenge of planning executable, precedence-constrained multi-robot disassembly in unstructured environments by using an intent-driven ensemble of LLMs grounded to perception, coupled with a verifier and a deterministic consistency filter. The method maps operator natural-language intent and live perception into safe, executable action sequences for two robotic arms, with adjustable autonomy and a perception–language interface. Key contributions include an end-to-end language-conditioned disassembly pipeline, a text-based scene representation enabling spatial reasoning, an ensemble-of-inference with a dedicated verifier plus a lightweight data-consistency check, and a real-world usability evaluation with non-expert users. Results on 200 real EV-battery scenes and 600 prompts show significant accuracy gains from ensemble-verifier combinations and acceptable user workload, indicating practical potential for adaptable, low-effort robot-assisted disassembly at scale.

Abstract

This paper addresses the problem of planning complex manipulation tasks, in which multiple robots with different end-effectors and capabilities, informed by computer vision, must plan and execute concatenated sequences of actions on a variety of objects that can appear in arbitrary positions and configurations in unstructured scenes. We propose an intent-driven planning pipeline which can robustly construct such action sequences with varying degrees of supervisory input from a human using simple language instructions. The pipeline integrates: (i) perception-to-text scene encoding, (ii) an ensemble of large language models (LLMs) that generate candidate removal sequences based on the operator's intent, (iii) an LLM-based verifier that enforces formatting and precedence constraints, and (iv) a deterministic consistency filter that rejects hallucinated objects. The pipeline is evaluated on an example task in which two robot arms work collaboratively to dismantle an Electric Vehicle battery for recycling applications. A variety of components must be grasped and removed in specific sequences, determined by human instructions and/or by task-order feasibility decisions made by the autonomous system. On 200 real scenes with 600 operator prompts across five component classes, we used metrics of full-sequence correctness and next-task correctness to evaluate and compare five LLM-based planners (including ablation analyses of pipeline components). We also evaluated the LLM-based human interface in terms of time to execution and NASA TLX with human participant experiments. Results indicate that our ensemble-with-verification approach reliably maps operator intent to safe, executable multi-robot plans while maintaining low user effort.

Figures (13)

• Figure 1: A task-planning pipeline that enhances reliability and flexibility by using an ensemble of LLMs (same checkpoint, different seeds) to generate diverse solutions from structured inputs. These candidate plans are then synthesised and validated by a dedicated verification module before robot execution.
• Figure 2: Example perception output used by the LLM-ensemble planner. A YOLOv8 model overlays class labels and confidence scores for the battery components (Leafcell, Cable, Busbar, Service Plug, Screw).
• Figure 3: Data-collection setup: two UR10e collaborative robot arms (ROBOT1 and ROBOT2) equipped with Intel RealSense RGB-D cameras and the components arranged on the workbench.
• Figure 4: Sample scene showing five battery components (Leafcell, Busbar, Cable, Service Plug and Screw)
• Figure 5: Ghost composite of all RGB frames. Frames are stacked and intensity-normalised. High-intensity regions indicate frequent occupancy across scenes; faint structures show sparse placements.