Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust Stance Detection: Understanding Public Perceptions in Social Media

Nayoung Kim, David Mosallanezhad, Lu Cheng, Michelle V. Mancenido, Huan Liu

TL;DR

This work tackles the challenge of detecting public stances across changing domains and targets in social media. It introduces STANCE-C3, a two-stage framework that (1) generates domain-counterfactual text to bridge domain gaps via a T5-based generator and (2) employs a supervised contrastive loss to learn cross-target, domain-invariant representations. The approach demonstrates consistent improvements over state-of-the-art baselines in cross-domain and cross-target scenarios on COVID-19-related datasets, with ablation studies underscoring the importance of both counterfactual augmentation and the contrastive objective. The results suggest that domain- and target-robust stance detectors can provide reliable, policy-relevant insights in environments where data are scarce or rapidly shifting, offering practical value for public health and governance contexts.

Abstract

The abundance of social media data has presented opportunities for accurately determining public and group-specific stances around policy proposals or controversial topics. In contrast with sentiment analysis which focuses on identifying prevailing emotions, stance detection identifies precise positions (i.e., supportive, opposing, neutral) relative to a well-defined topic, such as perceptions toward specific global health interventions during the COVID-19 pandemic. Traditional stance detection models, while effective within their specific domain (e.g., attitudes towards masking protocols during COVID-19), often lag in performance when applied to new domains and topics due to changes in data distribution. This limitation is compounded by the scarcity of domain-specific, labeled datasets, which are expensive and labor-intensive to create. A solution we present in this paper combines counterfactual data augmentation with contrastive learning to enhance the robustness of stance detection across domains and topics of interest. We evaluate the performance of current state-of-the-art stance detection models, including a prompt-optimized large language model, relative to our proposed framework succinctly called STANCE-C3 (domain-adaptive Cross-target STANCE detection via Contrastive learning and Counterfactual generation). Empirical evaluations demonstrate STANCE-C3's consistent improvements over the baseline models with respect to accuracy across domains and varying focal topics. Despite the increasing prevalence of general-purpose models such as generative AI, specialized models such as STANCE-C3 provide utility in safety-critical domains wherein precision is highly valued, especially when a nuanced understanding of the concerns of different population segments could result in crafting more impactful public policies.

Robust Stance Detection: Understanding Public Perceptions in Social Media

TL;DR

This work tackles the challenge of detecting public stances across changing domains and targets in social media. It introduces STANCE-C3, a two-stage framework that (1) generates domain-counterfactual text to bridge domain gaps via a T5-based generator and (2) employs a supervised contrastive loss to learn cross-target, domain-invariant representations. The approach demonstrates consistent improvements over state-of-the-art baselines in cross-domain and cross-target scenarios on COVID-19-related datasets, with ablation studies underscoring the importance of both counterfactual augmentation and the contrastive objective. The results suggest that domain- and target-robust stance detectors can provide reliable, policy-relevant insights in environments where data are scarce or rapidly shifting, offering practical value for public health and governance contexts.

Abstract

The abundance of social media data has presented opportunities for accurately determining public and group-specific stances around policy proposals or controversial topics. In contrast with sentiment analysis which focuses on identifying prevailing emotions, stance detection identifies precise positions (i.e., supportive, opposing, neutral) relative to a well-defined topic, such as perceptions toward specific global health interventions during the COVID-19 pandemic. Traditional stance detection models, while effective within their specific domain (e.g., attitudes towards masking protocols during COVID-19), often lag in performance when applied to new domains and topics due to changes in data distribution. This limitation is compounded by the scarcity of domain-specific, labeled datasets, which are expensive and labor-intensive to create. A solution we present in this paper combines counterfactual data augmentation with contrastive learning to enhance the robustness of stance detection across domains and topics of interest. We evaluate the performance of current state-of-the-art stance detection models, including a prompt-optimized large language model, relative to our proposed framework succinctly called STANCE-C3 (domain-adaptive Cross-target STANCE detection via Contrastive learning and Counterfactual generation). Empirical evaluations demonstrate STANCE-C3's consistent improvements over the baseline models with respect to accuracy across domains and varying focal topics. Despite the increasing prevalence of general-purpose models such as generative AI, specialized models such as STANCE-C3 provide utility in safety-critical domains wherein precision is highly valued, especially when a nuanced understanding of the concerns of different population segments could result in crafting more impactful public policies.
Paper Structure (13 sections, 1 equation, 3 figures, 4 tables)

This paper contains 13 sections, 1 equation, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Stance Detection
  4. Domain Adaptation
  5. Problem Statement
  6. Proposed Model
  7. Domain-adaptive Single-target
  8. Domain-adaptive Cross-target
  9. Experiments
  10. Baselines
  11. Datasets
  12. Evaluation and Results
  13. Conclusion

Figures (3)

  • Figure 1: Examples of domain-adaptive cross-target stance detection. (a) Tweets from source domain are collected using target-related keywords/hashtags (e.g., get vaccinated) from January 1st, 2020 to August 23rd, 2021, whereas (b) tweets from target domain are collected with target-related hashtags (e.g., #MasksSaveLives) from February 27th, 2020 to August 20th, 2020.
  • Figure 2: The STANCE-C3 architecture consists of two key components: a counterfactual data generation network and a contrastive learning network. The counterfactual network (left) is built on a T5-based model and generates training examples that maintain sentence structure but has increased diversity in semantic context. The contrastive learning network (right) then uses the augmented dataset to learn cross-target representations by minimizing distances between examples with the same stance (positive pairs) and maximizing distances between examples with different stance (negative pairs). This approach helps acquire domain-invariant features, improving stance detection across targets.
  • Figure 3: Impact analysis of model's parameters and components. Figures (a) and (b) show the impact of the target domain data portion and the balance between the loss values, respectively. Figure (c) shows the impact of different components - removing modified contrastive loss (STANCE-C3 \\ CL), removing counterfactual data generation component (STANCE-C3 \\ CF), and using simple contrastive loss (STANCE-C3 \\ CS) - on the model's performance.