Contrastive explanations of BDI agents

TL;DR

This work extends Belief-Desire-Intention (BDI) explainability to contrastive explanations, enabling agents to justify actions as alternatives to potential foils. It combines a computational evaluation showing substantial reductions in explanation length with a human-subject study demonstrating mixed preferences but generally improved trust and perceived understanding for contrastive explanations. The approach formalizes contrastive factors through modified ancestry predicates and foil-aware filtering, and demonstrates scalability across randomized goal-plan trees and standard IPC scenarios. Practical implications emphasize foil alignment, cautious deployment, and the value of iterative user-centered evaluation for robust explainability. The findings suggest contrastive explanations can enhance trust and comprehension in autonomous agents, albeit with scenario- and user-expectation dependencies that warrant careful design and further study.

Abstract

The ability of autonomous systems to provide explanations is important for supporting transparency and aiding the development of (appropriate) trust. Prior work has defined a mechanism for Belief-Desire-Intention (BDI) agents to be able to answer questions of the form ``why did you do action $X$?''. However, we know that we ask \emph{contrastive} questions (``why did you do $X$ \emph{instead of} $F$?''). We therefore extend previous work to be able to answer such questions. A computational evaluation shows that using contrastive questions yields a significant reduction in explanation length. A human subject evaluation was conducted to assess whether such contrastive answers are preferred, and how well they support trust development and transparency. We found some evidence for contrastive answers being preferred, and some evidence that they led to higher trust, perceived understanding, and confidence in the system's correctness. We also evaluated the benefit of providing explanations at all. Surprisingly, there was not a clear benefit, and in some situations we found evidence that providing a (full) explanation was worse than not providing any explanation.

Figures (6)

• Figure 1: Definition of $\mathcal{E}^T_X$ (ignore green shaded parts) and of $\mathcal{E}^T_{X/F}$ (adding $\fcolorbox{black}{lime}{$\hbox{green shading}$}$). Auxiliary functions and predicates used: $\mathit{pre}(N)$ and $post(N)$ (return pre-/post-condition of an action as a conjunction represented as a set of propositions), $\mathit{cond}(N)$ (returns the condition of node $N$), $parent(N)$ (return the parent node of $N$ in the given goal tree), $children(N)$ (return the children of $N$), $seqn(N)$ (returns the sequence number of $N$), $\mathit{ancest}(A,B)$ (true when $A$ is an ancestor of $B$), $child(N_i,N)$ (true when $N_i$ is a child of $N$), $isOne(N)$ (resp. $isXOne$, $isSOne$, and $isSeq$) which is true when the type of node $N$ is $one$ (resp. $sone$, $xone$, $seq$), and $isAct(N)$ (true when $N$ is an action node). We also use the predicate $held(N)$ which is true when the condition of node $N$ held when the node was reached (or if there is no condition), and $nheld(N)$ which is true when the condition of node $N$ did not hold.
• Figure 2: Computational Evaluation Results: plot of distribution of contrastive size against full explanation size for selected $F$ (left) and of median full size / contrastive size / saving against full explanation size (right).
• Figure 3: Coffee Example Goal-Plan Tree (based on DBLP:journals/ai/WinikoffSDD21). Notation: Nodes are rectangles that include the type and name of the goal. In the case of an action, the pre and post conditions are included in the node. Where a node is the child of a $seq$ or $sor$ node, the sequence number is on the edge outgoing from the parent. When a node has a condition, the condition is a "stadium" shaped node that sits between the goal and its parent.
• Figure 4: Computational Evaluation Results: Generated Trees (top: left is all data, right $F>10$), miconic (bottom left), logistics (bottom right)
• Figure 5: Preferred explanations for scenarios 3 (left), 5 (middle) and 6 (right); responses: 1 = strongly prefer full; 5 = strongly prefer contrastive.