HELP: Hierarchical Embodied Language Planner for Household Tasks

TL;DR

HELP introduces a hierarchical, multi-agent planning framework for embodied household tasks that decomposes natural-language instructions into unambiguous subtasks (HLP) and grounds each subtasks into executable actions via a low-level planner (LLP). It emphasizes running on mid-sized open-source LLMs on-device, with a feedback loop to incorporate environmental information and a feasibility verifier for safety. Across template-based datasets, ALFRED, and real-world robot experiments, HELP demonstrates robust long-horizon planning, generalization to broader task sets, and practical deployment potential, achieving up to 80% real-world success and strong planning-quality metrics. The work highlights the benefits of environmental grounding, similarity-based few-shot prompting, and modular agent design as a path toward autonomous, adaptable robotic planners.

Abstract

Embodied agents tasked with complex scenarios, whether in real or simulated environments, rely heavily on robust planning capabilities. When instructions are formulated in natural language, large language models (LLMs) equipped with extensive linguistic knowledge can play this role. However, to effectively exploit the ability of such models to handle linguistic ambiguity, to retrieve information from the environment, and to be based on the available skills of an agent, an appropriate architecture must be designed. We propose a Hierarchical Embodied Language Planner, called HELP, consisting of a set of LLM-based agents, each dedicated to solving a different subtask. We evaluate the proposed approach on a household task and perform real-world experiments with an embodied agent. We also focus on the use of open source LLMs with a relatively small number of parameters, to enable autonomous deployment.

Figures (11)

• Figure 1: Given a high-level natural language instruction, a high-level LLM agent decomposes it into unambiguous subtasks (e.g., “Pick up the shirt,” “Pick up the jeans,” etc.). Each subtask is passed to a low-level LLM agent, which generates executable pseudocode grounded in the robot’s action space.
• Figure 2: The HELP architecture consists of multiple LLM-based agents operating in a pipeline. The process begins with the High Level Planner (HLP), which resolves ambiguities by querying the environment for feedback if necessary. The HLP then decomposes the initial instruction into a set of individual natural language sub-tasks. These sub-tasks are subsequently passed to the Low Level Planner (LLP). The LLP translates each natural language sub-task into an executable plan, i.e., a sequence of actions grounded in the agent's available skills and the environment. Finally, the embodied agent executes the resulting sequence of actions.
• Figure 3: Illustration of plan evaluation metrics — PEM, PLCSA, and PLCSS — showing how each scores a predicted plan against the ground truth, highlighting sensitivity to action type vs. arguments and sequence continuity
• Figure 4: A comparison of the HELP, LLP, with baseline approaches, tested on plans ranging from 2 to 16 steps in length. Each length category comprised 200 plans.
• Figure 5: The plots show LCSA, LCSS, and EM scores versus plan length. While larger models generally perform better, all models exhibit performance degradation as task length increases