How Did We Get Here? Summarizing Conversation Dynamics

TL;DR

The paper defines conversation dynamics summaries (SCDs) as a new task to capture trajectory and tonal evolution beyond topical content, and constructs a CGA-CMV-based dataset of human-written SCDs released via ConvoKit. It evaluates a spectrum of machine-generated SCD baselines, including prompting and finetuning, and demonstrates a downstream derailment-forecasting task where SCDs improve predictive performance and interpretability for both humans and automated systems. Human-written SCDs better convey tone changes and trajectory, while machine-generated SCDs offer strong topical context; the best automated signals come from procedural prompts, supporting a summarize-then-forecast paradigm. The work highlights practical implications for moderation and counseling contexts and provides open resources to spur further development in dynamics-focused summarization.

Abstract

Throughout a conversation, the way participants interact with each other is in constant flux: their tones may change, they may resort to different strategies to convey their points, or they might alter their interaction patterns. An understanding of these dynamics can complement that of the actual facts and opinions discussed, offering a more holistic view of the trajectory of the conversation: how it arrived at its current state and where it is likely heading. In this work, we introduce the task of summarizing the dynamics of conversations, by constructing a dataset of human-written summaries, and exploring several automated baselines. We evaluate whether such summaries can capture the trajectory of conversations via an established downstream task: forecasting whether an ongoing conversation will eventually derail into toxic behavior. We show that they help both humans and automated systems with this forecasting task. Humans make predictions three times faster, and with greater confidence, when reading the summaries than when reading the transcripts. Furthermore, automated forecasting systems are more accurate when constructing, and then predicting based on, summaries of conversation dynamics, compared to directly predicting on the transcripts.

Figures (7)

• Figure 1: Traditional and dynamics summaries for the same conversation (transcript in Appendix \ref{['appendix:miscellaneous']}). Elements of conversational dynamics are colored in blue.
• Figure 2: Example question for derailment forecasting based on summaries.
• Figure 3: Zero-shot prompt and procedural prompt for SCDs.
• Figure 4: Transcript of the conversation used for the introductory example, Part 1 of 4. Its conversation ID is cz2r8ig in the CGA-CMV corpus.
• Figure 5: Transcript of the conversation used for the introductory example, Part 2 of 4.