Any Other Thoughts, Hedgehog? Linking Deliberation Chains in Collaborative Dialogues

TL;DR

This work examines probing questions in collaborative dialogues: questions that explicitly elicit responses from the speaker's interlocutors, and focuses on modeling the causal relations that lead directly from utterances earlier in the dialogue to the emergence of the probing question.

Abstract

Question-asking in collaborative dialogue has long been established as key to knowledge construction, both in internal and collaborative problem solving. In this work, we examine probing questions in collaborative dialogues: questions that explicitly elicit responses from the speaker's interlocutors. Specifically, we focus on modeling the causal relations that lead directly from utterances earlier in the dialogue to the emergence of the probing question. We model these relations using a novel graph-based framework of deliberation chains, and reframe the problem of constructing such chains as a coreference-style clustering problem. Our framework jointly models probing and causal utterances and the links between them, and we evaluate on two challenging collaborative task datasets: the Weights Task and DeliData. Our results demonstrate the effectiveness of our theoretically-grounded approach compared to both baselines and stronger coreference approaches, and establish a standard of performance in this novel task.

Figures (9)

• Figure 1: Example of a deliberation chain, showing the flow of interventions and their causal relationships within a collaborative task. This example is adapted from our model’s output on the DeliData corpus.
• Figure 2: Prompting framework for GPT to select causal interventions given a probing intervention and a dialogue history (example from DeliData). Ground-truth labels for probing and causal interventions are marked in green and brown, respectively.
• Figure 3: Average Scores for Causal Intervention Survey Responses.
• Figure 4: Our joint-learning framework for deliberation chains, learning to assign correct antecedent utterances for every valid intervention using a "probing" score, a "causal" score, and a "linking" score. Pairs of utterances are encoded with global attention (in green between <m> and </m>), further contextualized by past utterances.
• Figure 5: Cluster-level distribution of correctly assigned intervention links for the best-performing cross-encoder baseline compared to Joint - $W$ on both datasets.