Leveraging Hierarchical Prototypes as the Verbalizer for Implicit Discourse Relation Recognition

TL;DR

The proposed approach can be extended to enable zero-shot cross-lingual learning, facilitating the recognition of discourse relations in languages with scarce resources and validate the practicality and versatility of the approach in addressing the issues of implicit discourse relation recognition across different languages.

Abstract

Implicit discourse relation recognition involves determining relationships that hold between spans of text that are not linked by an explicit discourse connective. In recent years, the pre-train, prompt, and predict paradigm has emerged as a promising approach for tackling this task. However, previous work solely relied on manual verbalizers for implicit discourse relation recognition, which suffer from issues of ambiguity and even incorrectness. To overcome these limitations, we leverage the prototypes that capture certain class-level semantic features and the hierarchical label structure for different classes as the verbalizer. We show that our method improves on competitive baselines. Besides, our proposed approach can be extended to enable zero-shot cross-lingual learning, facilitating the recognition of discourse relations in languages with scarce resources. These advancement validate the practicality and versatility of our approach in addressing the issues of implicit discourse relation recognition across different languages.

Figures (7)

• Figure 1: The manual verbalizer and the labels.
• Figure 2: The hidden states of [MASK] token represent instances and project them to another embedding space for prototype learning. Three contrastive learning losses adjust the distances among prototypes, the distances among instances, and the distances between prototypes and instances based on the sense hierarchy. Finally, we calculate the similarity scores of query and prototypes during inference.
• Figure 3: Average Cosine Distances between the prototype and all the test examples from the same class with the prototype.
• Figure 4: Label distribution of the top ten nearest neighbors for each second level prototype in PDTB-2.
• Figure 5: Label distribution of the top ten nearest neighbors for each second level prototypes in PDTB-3.