MindDial: Belief Dynamics Tracking with Theory-of-Mind Modeling for Situated Neural Dialogue Generation

TL;DR

MindDial addresses the challenge of inserting theory-of-mind into situated dialogue to manage shared ground in alignment and negotiation. It introduces an explicit mind module that estimates first-order $b_A$ and second-order $b_{BinA}$ beliefs to guide a next utterance produced by either finetuning- or prompting-based generators. Across MutualFriend and CaSiNo, incorporating mind reasoning improves task outcomes, with ablations confirming benefits from both belief levels and evidence of robustness to prompting variations. Human evaluations and case studies corroborate the improvements in cooperative and negotiating interactions, highlighting MindDial’s potential for socially aware dialogue systems.

Abstract

Humans talk in daily conversations while aligning and negotiating the expressed meanings or common ground. Despite the impressive conversational abilities of the large generative language models, they do not consider the individual differences in contextual understanding in a shared situated environment. In this work, we propose MindDial, a novel conversational framework that can generate situated free-form responses with theory-of-mind modeling. We introduce an explicit mind module that can track the speaker's belief and the speaker's prediction of the listener's belief. Then the next response is generated to resolve the belief difference and take task-related action. Our framework is applied to both prompting and fine-tuning-based models, and is evaluated across scenarios involving both common ground alignment and negotiation. Experiments show that models with mind modeling can achieve higher task outcomes when aligning and negotiating common ground. The ablation study further validates the three-level belief design can aggregate information and improve task outcomes in both cooperative and negotiating settings.

Figures (17)

• Figure 1: Left: Single-turn question answering. Right: Multi-turn common ground alignment. Speakers will update their belief estimation based on context and generate the next response to reduce the belief differences.
• Figure 2: Cases of ToM reasoning in MindDial. Top: an alignment task from MutualFriend. Bottom: A negotiation task from CaSiNo. For each task, we first reason over the first- and second-order ToM beliefs of the conversational partner. Then we generate corresponding utterances wrt. the ToM estimation.
• Figure 3: Belief prediction. The precision and F1 when different models predict the first ($b_A$) and second-order ($b_{BinA}$) beliefs.
• Figure 4: Qualitative comparisons between dialogue generation models without (at left) and with mind modeling (at right) when agents A and B are figuring out their mutual friend.
• Figure 5: Annotation example