Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploring Plan Space through Conversation: An Agentic Framework for LLM-Mediated Explanations in Planning

Guilhem Fouilhé, Rebecca Eifler, Antonin Poché, Sylvie Thiébaux, Nicholas Asher

TL;DR

A multi-agent Large Language Model (LLM) architecture is presented that is agnostic to the explanation framework and enables user- and context-dependent interactive explanations and an instantiation of this framework for goal-conflict explanations is described.

Abstract

When automating plan generation for a real-world sequential decision problem, the goal is often not to replace the human planner, but to facilitate an iterative reasoning and elicitation process, where the human's role is to guide the AI planner according to their preferences and expertise. In this context, explanations that respond to users' questions are crucial to improve their understanding of potential solutions and increase their trust in the system. To enable natural interaction with such a system, we present a multi-agent Large Language Model (LLM) architecture that is agnostic to the explanation framework and enables user- and context-dependent interactive explanations. We also describe an instantiation of this framework for goal-conflict explanations, which we use to conduct a user study comparing the LLM-powered interaction with a baseline template-based explanation interface.

Exploring Plan Space through Conversation: An Agentic Framework for LLM-Mediated Explanations in Planning

TL;DR

A multi-agent Large Language Model (LLM) architecture is presented that is agnostic to the explanation framework and enables user- and context-dependent interactive explanations and an instantiation of this framework for goal-conflict explanations is described.

Abstract

When automating plan generation for a real-world sequential decision problem, the goal is often not to replace the human planner, but to facilitate an iterative reasoning and elicitation process, where the human's role is to guide the AI planner according to their preferences and expertise. In this context, explanations that respond to users' questions are crucial to improve their understanding of potential solutions and increase their trust in the system. To enable natural interaction with such a system, we present a multi-agent Large Language Model (LLM) architecture that is agnostic to the explanation framework and enables user- and context-dependent interactive explanations. We also describe an instantiation of this framework for goal-conflict explanations, which we use to conduct a user study comparing the LLM-powered interaction with a baseline template-based explanation interface.
Paper Structure (82 sections, 1 equation, 17 figures, 1 table)

This paper contains 82 sections, 1 equation, 17 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Conversational Explanations
  4. Multi-Agent LLM Approaches
  5. Planning Formalism
  6. Goal-Conflict Explanations
  7. Temporal Goals
  8. Running Example
  9. Iterative Planning with Natural Language
  10. Goals
  11. Questions
  12. Selecting Questions
  13. Explanations
  14. Selecting Explanations
  15. Summarizing Explanations
  16. ...and 67 more sections

Figures (17)

  • Figure 1: Architecture of our approach to interactive planning with explanations. Translator agents based on LLMs translate the user input into the formal language required by the computation agents.
  • Figure 2: Communication protocol between the user, the translators and the explanation framework. In addition to the information provided in the messages, each agent has access to the planning task and the iteration step. For a goal translation (green, top left) the user directly communicates with $T_G$. If the user asks a question (black) the dispatcher chooses one of four routing options (DIRECT, FOLLOW-UP, EFQUERY-noGT and EFQUERY-GT) depending on the question type identified by $T_Q$.
  • Figure 3: Comparison of maximum utility achieved over iteration steps between G$^{\texttt{LLM}}$ and G$^\texttt{TPL}$. Scores are averaged by group at each iteration step.
  • Figure 4: Comparison of question types used by G$^{\texttt{LLM}}$ and G$^\texttt{TPL}$. See Sec. \ref{['sec:goal-onflict-explanation']} for the description of questions.
  • Figure 5: Feedback questionnaire results comparison for the two groups: G$^\texttt{TPL}$ and G$^{\texttt{LLM}}$. Violin plots show the distribution and mean score of the answers on a Likert scale. LLM-based significantly outperforms template-based in helping participants improve their plans and reduce difficulty.
  • ...and 12 more figures

Theorems & Definitions (8)

  • definition 1: Minimal Unsolvable Subset (MUS)
  • definition 2: Minimal Correction Set (MCS)
  • definition 3: Iteration Step
  • definition 4: Goal Translator
  • definition 5: Question
  • definition 6: Question Translator
  • definition 7: Question Suggester
  • definition 8: Explanation Translator