Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Challenge Dataset and Effective Models for Conversational Stance Detection

Fuqiang Niu, Min Yang, Ang Li, Baoquan Zhang, Xiaojiang Peng, Bowen Zhang

TL;DR

The paper tackles the scarcity of multi-turn conversational stance data by introducing MT-CSD, a large English dataset collected from Reddit with multiple targets and deep discussion threads. It proposes GLAN, a global-local attention network that captures long-range dependencies, local discourse patterns, and structural relations through a global, local, and structural three-branch design followed by a target-attention layer. Experiments across in-target and cross-target settings show GLAN generally outperforms a wide range of baselines, though overall accuracy remains modest at around 50% on this challenging task, underscoring the complexity of conversational stance detection. MT-CSD thus provides a valuable resource for cross-domain stance detection research and real-world applications, with publicly available code and data to foster further progress.

Abstract

Previous stance detection studies typically concentrate on evaluating stances within individual instances, thereby exhibiting limitations in effectively modeling multi-party discussions concerning the same specific topic, as naturally transpire in authentic social media interactions. This constraint arises primarily due to the scarcity of datasets that authentically replicate real social media contexts, hindering the research progress of conversational stance detection. In this paper, we introduce a new multi-turn conversation stance detection dataset (called \textbf{MT-CSD}), which encompasses multiple targets for conversational stance detection. To derive stances from this challenging dataset, we propose a global-local attention network (\textbf{GLAN}) to address both long and short-range dependencies inherent in conversational data. Notably, even state-of-the-art stance detection methods, exemplified by GLAN, exhibit an accuracy of only 50.47\%, highlighting the persistent challenges in conversational stance detection. Furthermore, our MT-CSD dataset serves as a valuable resource to catalyze advancements in cross-domain stance detection, where a classifier is adapted from a different yet related target. We believe that MT-CSD will contribute to advancing real-world applications of stance detection research. Our source code, data, and models are available at \url{https://github.com/nfq729/MT-CSD}.

A Challenge Dataset and Effective Models for Conversational Stance Detection

TL;DR

The paper tackles the scarcity of multi-turn conversational stance data by introducing MT-CSD, a large English dataset collected from Reddit with multiple targets and deep discussion threads. It proposes GLAN, a global-local attention network that captures long-range dependencies, local discourse patterns, and structural relations through a global, local, and structural three-branch design followed by a target-attention layer. Experiments across in-target and cross-target settings show GLAN generally outperforms a wide range of baselines, though overall accuracy remains modest at around 50% on this challenging task, underscoring the complexity of conversational stance detection. MT-CSD thus provides a valuable resource for cross-domain stance detection research and real-world applications, with publicly available code and data to foster further progress.

Abstract

Previous stance detection studies typically concentrate on evaluating stances within individual instances, thereby exhibiting limitations in effectively modeling multi-party discussions concerning the same specific topic, as naturally transpire in authentic social media interactions. This constraint arises primarily due to the scarcity of datasets that authentically replicate real social media contexts, hindering the research progress of conversational stance detection. In this paper, we introduce a new multi-turn conversation stance detection dataset (called \textbf{MT-CSD}), which encompasses multiple targets for conversational stance detection. To derive stances from this challenging dataset, we propose a global-local attention network (\textbf{GLAN}) to address both long and short-range dependencies inherent in conversational data. Notably, even state-of-the-art stance detection methods, exemplified by GLAN, exhibit an accuracy of only 50.47\%, highlighting the persistent challenges in conversational stance detection. Furthermore, our MT-CSD dataset serves as a valuable resource to catalyze advancements in cross-domain stance detection, where a classifier is adapted from a different yet related target. We believe that MT-CSD will contribute to advancing real-world applications of stance detection research. Our source code, data, and models are available at \url{https://github.com/nfq729/MT-CSD}.
Paper Structure (32 sections, 4 equations, 2 figures, 9 tables)

This paper contains 32 sections, 4 equations, 2 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Stance Detection Datasets
  4. Stance Detection Approaches
  5. Dataset Construction
  6. Data Collection
  7. Data Preprocessing
  8. Data Annotation and Quality Assurance
  9. Data Analysis
  10. Challenges
  11. Our Methodology
  12. Text Representation Layer
  13. The Global-Local Attention Layer
  14. Global Layer
  15. Local Layer
  16. ...and 17 more sections

Figures (2)

  • Figure 1: An example of conversational stance detection.
  • Figure 2: The architecture of our GLAN framework.