Improving Implicit Discourse Relation Recognition with Natural Language Explanations from LLMs

TL;DR

This work proposes a simple yet effective approach to distill the reasoning capabilities of LLMs into lightweight IDRR models to improve both performance and interpretability, and introduces a novel classification-generation framework that jointly performs relation prediction and explanation generation.

Abstract

Implicit Discourse Relation Recognition (IDRR) remains a challenging task due to the requirement for deep semantic understanding in the absence of explicit discourse markers. A further limitation is that existing methods only predict relations without providing any supporting explanations. Recent advances in large language models (LLMs) have shown strong reasoning capabilities in both deep language understanding and natural language explanation generation. In this work, we propose a simple yet effective approach to distill the reasoning capabilities of LLMs into lightweight IDRR models to improve both performance and interpretability. Specifically, we first prompt an LLM to generate explanations for each training instance conditioned on its gold label. Then, we introduce a novel classification-generation framework that jointly performs relation prediction and explanation generation, and train it with the additional supervision of LLM-generated explanations. Our framework is plug-and-play, enabling easy integration with most existing IDRR models. Experimental results on PDTB demonstrate that our approach significantly improves IDRR performance, while human evaluation further confirms that the generated explanations enhance model interpretability. Furthermore, we validate the generality of our approach on sentiment classification and natural language inference

Figures (5)

• Figure 1: Our lightweight IDRR model distills LLM reasoning to predict discourse relations and generate interpretable explanations. Arg1 and Arg2 are two arguments.
• Figure 2: The prompt template for explanation generation. An explanation consists of two parts: the restatement of two arguments, and a rationale of their discourse relationship. S means two arguments.
• Figure 3: Our proposed classification-generation framework. For ease of understanding, we use the IDRR task as an example to illustrate our framework. It first predicts the discourse relation $\tilde{y}$, which is then used as extra input to generate the explanation $\tilde{e}$.
• Figure 4: Quality of generated explanations. (a) Compared with e-INFERSENT, our method achieves a higher average score of 4.20 out of 5. (b) The distribution of scores assigned through human evaluation. (c) The distribution of scores for each aspect.
• Figure 5: Feature Importance Agreement (Left) and Robustness Equivalence (Right). Con denotes the proportion of label–explanation alignment. Random and Important indicate that randomly selected or top-ranked important features are occluded, respectively.