Investigating Counterclaims in Causality Extraction from Text

TL;DR

This work addresses a blind spot in causality extraction by introducing concausality as a first-class concept and presenting the Concausal News Corpus (CCNC), a concausal-augmented version of CNCv2 built via manual rewriting and rigorous annotation ($κ=0.74$). By extending the standard three-step causality extraction pipeline to include concausal labels, the authors demonstrate that models trained only on procausal data misclassify concausal statements, which can distort causal graphs and reasoning. Baseline transformer models show improved discrimination between pro- and concausality when trained on CCNC, underscoring the practical value of balancing causal claims for reliable downstream tasks such as causal QA and graph construction. The CCNC framework enables balanced causal reasoning, supports more accurate causal graphs for decision-support, and opens avenues for concausal data generation and computational argumentation research.

Abstract

Research on causality extraction from text has so far almost entirely neglected counterclaims. Existing causality extraction datasets focus solely on "procausal" claims, i.e., statements that support a relationship. "Concausal" claims, i.e., statements that refute a relationship, are entirely ignored or even accidentally annotated as procausal. We address this shortcoming by developing a new dataset that integrates concausality. Based on an extensive literature review, we first show that concausality is an integral part of causal reasoning on incomplete knowledge. We operationalize this theory in the form of a rigorous guideline for annotation and then augment the Causal News Corpus with concausal statements, obtaining a substantial inter-annotator agreement of Cohen's $κ=0.74$. To demonstrate the importance of integrating concausal statements, we show that models trained without concausal relationships tend to misclassify these as procausal instead. Based on our new dataset, this mistake can be mitigated, enabling transformers to effectively distinguish pro- and concausality.

Figures (4)

• Figure 1: Causality extraction can be split into: (1) Detecting if the text contains causal claims, (2) extracting candidate spans that may be part of a causal relation, and (3) identifying the type of causality for each pair.
• Figure 2: Confusion matrices of training on CNCv2 and CCNC and evaluating on the given split. The class labels are noncausal (nc), uncausal (uc), procausal (pc), and concausal (cc). Darker shades denote larger values.
• Figure 3: Example prompt of our preliminary experiments on rewriting procausal statements to be concausal using GPT-4.
• Figure 4: The instructions given to the annotators for annotating our dataset.