Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Infusing Hierarchical Guidance into Prompt Tuning: A Parameter-Efficient Framework for Multi-level Implicit Discourse Relation Recognition

Haodong Zhao, Ruifang He, Mengnan Xiao, Jing Xu

TL;DR

A prompt-based Parameter-Efficient Multi-level IDRR (PEMI) framework, which leverages parameter-efficient prompt tuning to drive the inputted arguments to match the pre-trained space and realize the approximation with few parameters.

Abstract

Multi-level implicit discourse relation recognition (MIDRR) aims at identifying hierarchical discourse relations among arguments. Previous methods achieve the promotion through fine-tuning PLMs. However, due to the data scarcity and the task gap, the pre-trained feature space cannot be accurately tuned to the task-specific space, which even aggravates the collapse of the vanilla space. Besides, the comprehension of hierarchical semantics for MIDRR makes the conversion much harder. In this paper, we propose a prompt-based Parameter-Efficient Multi-level IDRR (PEMI) framework to solve the above problems. First, we leverage parameter-efficient prompt tuning to drive the inputted arguments to match the pre-trained space and realize the approximation with few parameters. Furthermore, we propose a hierarchical label refining (HLR) method for the prompt verbalizer to deeply integrate hierarchical guidance into the prompt tuning. Finally, our model achieves comparable results on PDTB 2.0 and 3.0 using about 0.1% trainable parameters compared with baselines and the visualization demonstrates the effectiveness of our HLR method.

Infusing Hierarchical Guidance into Prompt Tuning: A Parameter-Efficient Framework for Multi-level Implicit Discourse Relation Recognition

TL;DR

A prompt-based Parameter-Efficient Multi-level IDRR (PEMI) framework, which leverages parameter-efficient prompt tuning to drive the inputted arguments to match the pre-trained space and realize the approximation with few parameters.

Abstract

Multi-level implicit discourse relation recognition (MIDRR) aims at identifying hierarchical discourse relations among arguments. Previous methods achieve the promotion through fine-tuning PLMs. However, due to the data scarcity and the task gap, the pre-trained feature space cannot be accurately tuned to the task-specific space, which even aggravates the collapse of the vanilla space. Besides, the comprehension of hierarchical semantics for MIDRR makes the conversion much harder. In this paper, we propose a prompt-based Parameter-Efficient Multi-level IDRR (PEMI) framework to solve the above problems. First, we leverage parameter-efficient prompt tuning to drive the inputted arguments to match the pre-trained space and realize the approximation with few parameters. Furthermore, we propose a hierarchical label refining (HLR) method for the prompt verbalizer to deeply integrate hierarchical guidance into the prompt tuning. Finally, our model achieves comparable results on PDTB 2.0 and 3.0 using about 0.1% trainable parameters compared with baselines and the visualization demonstrates the effectiveness of our HLR method.
Paper Structure (21 sections, 10 equations, 4 figures, 10 tables)

This paper contains 21 sections, 10 equations, 4 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Implicit discourse relation recognition
  4. Prompt Tuning
  5. Overall Framework
  6. Prompt Tuning for Single-level IDRR
  7. Hierarchical Label Refining
  8. Joint Learning
  9. Experiments
  10. Dataset
  11. Experimental Settings
  12. The Comparison Models
  13. Results and Analysis
  14. Ablation Study and Analysis
  15. Template Selection
  16. ...and 6 more sections

Figures (4)

  • Figure 1: An Instance for multi-level IDRR.
  • Figure 2: The overall architecture of our PEMI framework.
  • Figure 3: The effect of prompt token size for MIDRR on PDTB 2.0. We follow the best template in Table \ref{['tab:all_input_location']} and try to put them uniformly in each location.
  • Figure 4: Visualization of HLR method for connectives. $\Delta$ represents level 2 labels and different colors indicate different classes. We use different markers since some connectives are overlapping due to the many-to-many mapping between level 2 and connectives.