Decentralized Intent-Based Multi-Robot Task Planner with LLM Oracles on Hyperledger Fabric

TL;DR

The paper tackles secure, decentralized translation of natural language intents into executable robotic subtasks across a heterogeneous multi-robot fleet. It proposes a Hyperledger Fabric–based architecture hosting an ensemble of LLM oracles and introduces a sequence-aware aggregation method based on Longest Common Subsequence to enforce temporal correctness. A public benchmark, SkillChain-RTD, is introduced to evaluate performance and robustness, showing the approach outperforms semantic-based aggregation methods and can detect adversarial model behavior. Results indicate the system achieves auditable task planning with around 2.0 seconds of planning latency and lays groundwork for secure cross-organizational coordination of embodied agents.

Abstract

Large language models (LLMs) have opened new opportunities for transforming natural language user intents into executable actions. This capability enables embodied AI agents to perform complex tasks, without involvement of an expert, making human-robot interaction (HRI) more convenient. However these developments raise significant security and privacy challenges such as self-preferencing, where a single LLM service provider dominates the market and uses this power to promote their own preferences. LLM oracles have been recently proposed as a mechanism to decentralize LLMs by executing multiple LLMs from different vendors and aggregating their outputs to obtain a more reliable and trustworthy final result. However, the accuracy of these approaches highly depends on the aggregation method. The current aggregation methods mostly use semantic similarity between various LLM outputs, not suitable for robotic task planning, where the temporal order of tasks is important. To fill the gap, we propose an LLM oracle with a new aggregation method for robotic task planning. In addition, we propose a decentralized multi-robot infrastructure based on Hyperledger Fabric that can host the proposed oracle. The proposed infrastructure enables users to express their natural language intent to the system, which then can be decomposed into subtasks. These subtasks require coordinating different robots from different vendors, while enforcing fine-grained access control management on the data. To evaluate our methodology, we created the SkillChain-RTD benchmark made it publicly available. Our experimental results demonstrate the feasibility of the proposed architecture, and the proposed aggregation method outperforms other aggregation methods currently in use.

Figures (6)

• Figure 1: High-level conceptual framework of the decentralized task planning system. A natural language user intent is decomposed into robot-specific atomic tasks through a Hyperledger Fabric-based consensus across an ensemble of heterogeneous LLM oracles, ensuring a trusted and verifiable plan execution.
• Figure 2: System architecture.
• Figure 3: Distribution of the latency of models over all tasks.
• Figure 4: Average similarity of LLM models with LCS, SBERT, and TF-IDF similarity methods.
• Figure 5: Task 17 pairwise similarity matrix with different similarity measures.