Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MARE: Multi-Aspect Rationale Extractor on Unsupervised Rationale Extraction

Han Jiang, Junwen Duan, Zhe Qu, Jianxin Wang

TL;DR

A Multi-Aspect Rationale Extractor (MARE) to explain and predict multiple aspects simultaneously and multiple special tokens are prepended in front of the text with each corresponding to one certain aspect.

Abstract

Unsupervised rationale extraction aims to extract text snippets to support model predictions without explicit rationale annotation. Researchers have made many efforts to solve this task. Previous works often encode each aspect independently, which may limit their ability to capture meaningful internal correlations between aspects. While there has been significant work on mitigating spurious correlations, our approach focuses on leveraging the beneficial internal correlations to improve multi-aspect rationale extraction. In this paper, we propose a Multi-Aspect Rationale Extractor (MARE) to explain and predict multiple aspects simultaneously. Concretely, we propose a Multi-Aspect Multi-Head Attention (MAMHA) mechanism based on hard deletion to encode multiple text chunks simultaneously. Furthermore, multiple special tokens are prepended in front of the text with each corresponding to one certain aspect. Finally, multi-task training is deployed to reduce the training overhead. Experimental results on two unsupervised rationale extraction benchmarks show that MARE achieves state-of-the-art performance. Ablation studies further demonstrate the effectiveness of our method. Our codes have been available at https://github.com/CSU-NLP-Group/MARE.

MARE: Multi-Aspect Rationale Extractor on Unsupervised Rationale Extraction

TL;DR

A Multi-Aspect Rationale Extractor (MARE) to explain and predict multiple aspects simultaneously and multiple special tokens are prepended in front of the text with each corresponding to one certain aspect.

Abstract

Unsupervised rationale extraction aims to extract text snippets to support model predictions without explicit rationale annotation. Researchers have made many efforts to solve this task. Previous works often encode each aspect independently, which may limit their ability to capture meaningful internal correlations between aspects. While there has been significant work on mitigating spurious correlations, our approach focuses on leveraging the beneficial internal correlations to improve multi-aspect rationale extraction. In this paper, we propose a Multi-Aspect Rationale Extractor (MARE) to explain and predict multiple aspects simultaneously. Concretely, we propose a Multi-Aspect Multi-Head Attention (MAMHA) mechanism based on hard deletion to encode multiple text chunks simultaneously. Furthermore, multiple special tokens are prepended in front of the text with each corresponding to one certain aspect. Finally, multi-task training is deployed to reduce the training overhead. Experimental results on two unsupervised rationale extraction benchmarks show that MARE achieves state-of-the-art performance. Ablation studies further demonstrate the effectiveness of our method. Our codes have been available at https://github.com/CSU-NLP-Group/MARE.
Paper Structure (35 sections, 5 equations, 4 figures, 11 tables)

This paper contains 35 sections, 5 equations, 4 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Gradient Flows
  4. Interlocking
  5. Problem Definition
  6. Method
  7. Hard Deletion for Complete Token Removal
  8. Multi-Aspect Multi-Head Attention
  9. Multi-Aspect Controller (MAC)
  10. Computation Process of MAC
  11. Multi-Task Training
  12. Overall Loss
  13. Experiments
  14. Experimental Setup
  15. Datasets
  16. ...and 20 more sections

Figures (4)

  • Figure 1: Comparison of our methods (MARE) with previous typical uni-aspect encoding models.
  • Figure 2: Attention mask visualization. left: attention mask in Attention Mask Deletion. right: attention mask in Hard Deletion.
  • Figure 3: Overall model architecture. left: the overall model architecture of MARE. right: the computational graph of MAMHA.
  • Figure 4: A example for Multi-Aspect Controller. left: The token mask for each aspect. "Good place" and "bad service" stands for the rationales of location and service aspect, respectively. right: The attention mask is obtained by performing an outer product operation on token masks.