Improving Stance Detection by Leveraging Measurement Knowledge from Social Sciences: A Case Study of Dutch Political Tweets and Traditional Gender Role Division

TL;DR

This study reframes stance detection (SD) for political tweets as a textual entailment recognition (TER) problem and augments it with validated social-science survey instruments to measure attitudes toward traditional gender role division. It trains a Dutch TER model on the SICK-NL dataset and applies zero-shot SD to Dutch party tweets, using both simple hypotheses and multiple hypotheses derived from 11 LISS survey items. Across two experiments, hypotheses grounded in survey instruments improve tweet-level entailment predictions, with GPT-3.5 Turbo achieving 0.725 accuracy when survey items are used, though party-level transfer remains weak. The work demonstrates a promising bridge between SD and attitudinal measurement, highlights practical reproducibility resources, and suggests directions for broader TER datasets and multilingual extensions to enhance cross-target SD capabilities.

Abstract

Stance detection (SD) concerns automatically determining the viewpoint (i.e., in favour of, against, or neutral) of a text's author towards a target. SD has been applied to many research topics, among which the detection of stances behind political tweets is an important one. In this paper, we apply SD to a dataset of tweets from official party accounts in the Netherlands between 2017 and 2021, with a focus on stances towards traditional gender role division, a dividing issue between (some) Dutch political parties. To implement and improve SD of traditional gender role division, we propose to leverage an established survey instrument from social sciences, which has been validated for the purpose of measuring attitudes towards traditional gender role division. Based on our experiments, we show that using such a validated survey instrument helps to improve SD performance.