Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Rationale-centric Counterfactual Data Augmentation Method for Cross-Document Event Coreference Resolution

Bowen Ding, Qingkai Min, Shengkun Ma, Yingjie Li, Linyi Yang, Yue Zhang

TL;DR

This work reframes cross-document event coreference resolution as a causal reasoning problem by modeling the influence of lexical trigger matching as a spurious correlation. It introduces LLM-RCDA, a rationale-centric counterfactual data augmentation method equipped with Trigger Intervention and Context Intervention to generate plausible CAD that emphasizes event arguments (rationales) over surface trigger similarity. The approach leverages an LLM-in-the-loop to produce minimally edited CAD, achieving state-of-the-art results on ECB+, FCC, and GVC and demonstrating robustness in out-of-domain settings. By combining SCM-based analysis with causal data augmentation, the paper advances both theoretical understanding and practical performance for cross-document ECR, with potential extensions to other pairwise NLP tasks. All results highlight improved causal reasoning and resilience to trigger-based biases in large-scale ECR systems.

Abstract

Based on Pre-trained Language Models (PLMs), event coreference resolution (ECR) systems have demonstrated outstanding performance in clustering coreferential events across documents. However, the existing system exhibits an excessive reliance on the `triggers lexical matching' spurious pattern in the input mention pair text. We formalize the decision-making process of the baseline ECR system using a Structural Causal Model (SCM), aiming to identify spurious and causal associations (i.e., rationales) within the ECR task. Leveraging the debiasing capability of counterfactual data augmentation, we develop a rationale-centric counterfactual data augmentation method with LLM-in-the-loop. This method is specialized for pairwise input in the ECR system, where we conduct direct interventions on triggers and context to mitigate the spurious association while emphasizing the causation. Our approach achieves state-of-the-art performance on three popular cross-document ECR benchmarks and demonstrates robustness in out-of-domain scenarios.

A Rationale-centric Counterfactual Data Augmentation Method for Cross-Document Event Coreference Resolution

TL;DR

This work reframes cross-document event coreference resolution as a causal reasoning problem by modeling the influence of lexical trigger matching as a spurious correlation. It introduces LLM-RCDA, a rationale-centric counterfactual data augmentation method equipped with Trigger Intervention and Context Intervention to generate plausible CAD that emphasizes event arguments (rationales) over surface trigger similarity. The approach leverages an LLM-in-the-loop to produce minimally edited CAD, achieving state-of-the-art results on ECB+, FCC, and GVC and demonstrating robustness in out-of-domain settings. By combining SCM-based analysis with causal data augmentation, the paper advances both theoretical understanding and practical performance for cross-document ECR, with potential extensions to other pairwise NLP tasks. All results highlight improved causal reasoning and resilience to trigger-based biases in large-scale ECR systems.

Abstract

Based on Pre-trained Language Models (PLMs), event coreference resolution (ECR) systems have demonstrated outstanding performance in clustering coreferential events across documents. However, the existing system exhibits an excessive reliance on the `triggers lexical matching' spurious pattern in the input mention pair text. We formalize the decision-making process of the baseline ECR system using a Structural Causal Model (SCM), aiming to identify spurious and causal associations (i.e., rationales) within the ECR task. Leveraging the debiasing capability of counterfactual data augmentation, we develop a rationale-centric counterfactual data augmentation method with LLM-in-the-loop. This method is specialized for pairwise input in the ECR system, where we conduct direct interventions on triggers and context to mitigate the spurious association while emphasizing the causation. Our approach achieves state-of-the-art performance on three popular cross-document ECR benchmarks and demonstrates robustness in out-of-domain scenarios.
Paper Structure (32 sections, 3 equations, 4 figures, 21 tables, 1 algorithm)

This paper contains 32 sections, 3 equations, 4 figures, 21 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Baselines
  4. Method
  5. Causality Analysis and Interventions
  6. Counterfactual Generation
  7. Experimental Settings
  8. Evaluation Metrics and Datasets
  9. Implementation Details
  10. Experimental Results
  11. Analysis
  12. Conclusion
  13. Appendix
  14. Dataset Details
  15. Experimental Details
  16. ...and 17 more sections

Figures (4)

  • Figure 1: The distribution of 'triggers lexical matching' in mention pairs from ECB+ training set, along with a false negative example from held-etal-2021-focus's system which shows that forcing the event trigger in the first mention to lexically match the second one causes a significant change in the predicted coreference score.
  • Figure 2: The procedure of our rationale-centric counterfactual DA with LLM-in-the-loop (LLM-RCDA).
  • Figure 3: SCM illustration. (1) stimulates the decision process of the baseline ECR system; (2) shows the decision process of the causally enhanced system after interventions.
  • Figure 4: Error distribution on resolved baseline errors by the enhanced system.