Goal Inference from Open-Ended Dialog

TL;DR

This work tackles goal inference from open-ended dialog for embodied agents by introducing GOOD (Goals fOr Open-ended Dialogue), a Bayesian framework that extracts natural-language goals from conversations and maintains uncertainty over an explicit finite goal set. By prompting LLMs to role-play as users with specific goals, the method defines the likelihood $P(u\mid g)$ and updates the posterior $P(g\mid u)$ online, achieving a balance between offline flexibility and online data efficiency. The architecture comprises four modules (Conversation, Inference, Goal Management, and Action) and introduces the Unspecified Goal mechanism to trigger goal edits, enabling dynamic expansion or pruning of the goal set. Evaluations in a text-based grocery shopping domain and an AI2Thor robotic simulation demonstrate that Bayesian inference over NL goals can robustly capture diverse user preferences, guiding actions with meaningful uncertainty estimates and improving alignment compared to ablations.

Abstract

Embodied AI Agents are quickly becoming important and common tools in society. These embodied agents should be able to learn about and accomplish a wide range of user goals and preferences efficiently and robustly. Large Language Models (LLMs) are often used as they allow for opportunities for rich and open-ended dialog type interaction between the human and agent to accomplish tasks according to human preferences. In this thesis, we argue that for embodied agents that deal with open-ended dialog during task assistance: 1) AI Agents should extract goals from conversations in the form of Natural Language (NL) to be better at capturing human preferences as it is intuitive for humans to communicate their preferences on tasks to agents through natural language. 2) AI Agents should quantify/maintain uncertainty about these goals to ensure that actions are being taken according to goals that the agent is extremely certain about. We present an online method for embodied agents to learn and accomplish diverse user goals. While offline methods like RLHF can represent various goals but require large datasets, our approach achieves similar flexibility with online efficiency. We extract natural language goal representations from conversations with Large Language Models (LLMs). We prompt an LLM to role play as a human with different goals and use the corresponding likelihoods to run Bayesian inference over potential goals. As a result, our method can represent uncertainty over complex goals based on unrestricted dialog. We evaluate in a text-based grocery shopping domain and an AI2Thor robot simulation. We compare our method to ablation baselines that lack either explicit goal representation or probabilistic inference.

Figures (8)

• Figure 1: Offline/RLHF methods are data heavy, but is flexible to accommodate many tasks and domains. Online methods are data efficient, however are very domain specific. To accommodate human preferences from conversations, our method uses the best of both worlds and is data efficient and is generalizable for a broad set of tasks and domains.
• Figure 2: Conversation Module LLM Prompts for LLMs to generate the robot query and the synthetic human response for the conversation module. The robot query is generated based on descriptions of the agent and the task, current status of the task, and the transcript of the conversation so far. The human response is based on the description and current status of the task, a human profile (preferences of the human), and information about the completion requirements for the task.
• Figure 3: Inference Module LLM Prompts for calculating the log likelihood for each goal by using LLM logits that correspond to the human utterance given context.
• Figure 4: LLM prompts involved in the Goal Management Module for proposition of goals and removal of goals that are unlikely or not relevant.
• Figure 5: Method Overview Diagram. The Conversation Module produces single round of dialog. Inference Module Round 1 for determining whether Goal Management Module should add more goals. Unreasonable goals are removed from the goal list. Another round of inference is done to determine the most likely goal(s) to be passed to the action module, which generates an action list for the agent to take. If the task is completed, then the conversation stops.