On the Role of Context for Discourse Relation Classification in Scientific Writing
Stephen Wan, Wei Liu, Michael Strube
TL;DR
This work investigates how discourse context influences Discourse Relation Classification (DRC) in scientific writing. It compares RoBERTa-based PLM fine-tuning and Large Language Model (LLM) in-context inference (GPT-4 and LLaMA-3.1) across CovDTB and SciDTB datasets, using context schemes Default, Add-n, and Oracle-n derived from discourse dependency trees. Results show that structured context (Oracle-n) generally enhances DRC, with statistically significant gains for SciDTB under RoBERTa and notable improvements for LLMs when context is structurally informed; naïve adjacent text can hurt performance. The findings highlight which relation types benefit (e.g., elaboration, attribution, comparison, temporal) and point to longer documents and richer graph representations as fruitful avenues for future work, with implications for evidence justification in AI-assisted scientific workflows.
Abstract
With the increasing use of generative Artificial Intelligence (AI) methods to support science workflows, we are interested in the use of discourse-level information to find supporting evidence for AI generated scientific claims. A first step towards this objective is to examine the task of inferring discourse structure in scientific writing. In this work, we present a preliminary investigation of pretrained language model (PLM) and Large Language Model (LLM) approaches for Discourse Relation Classification (DRC), focusing on scientific publications, an under-studied genre for this task. We examine how context can help with the DRC task, with our experiments showing that context, as defined by discourse structure, is generally helpful. We also present an analysis of which scientific discourse relation types might benefit most from context.