Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Text Summarization With Graph Attention Networks

Mohammadreza Ardestani, Yllias Chali

Abstract

This study aimed to leverage graph information, particularly Rhetorical Structure Theory (RST) and Co-reference (Coref) graphs, to enhance the performance of our baseline summarization models. Specifically, we experimented with a Graph Attention Network architecture to incorporate graph information. However, this architecture did not enhance the performance. Subsequently, we used a simple Multi-layer Perceptron architecture, which improved the results in our proposed model on our primary dataset, CNN/DM. Additionally, we annotated XSum dataset with RST graph information, establishing a benchmark for future graph-based summarization models. This secondary dataset posed multiple challenges, revealing both the merits and limitations of our models.

Text Summarization With Graph Attention Networks

Abstract

This study aimed to leverage graph information, particularly Rhetorical Structure Theory (RST) and Co-reference (Coref) graphs, to enhance the performance of our baseline summarization models. Specifically, we experimented with a Graph Attention Network architecture to incorporate graph information. However, this architecture did not enhance the performance. Subsequently, we used a simple Multi-layer Perceptron architecture, which improved the results in our proposed model on our primary dataset, CNN/DM. Additionally, we annotated XSum dataset with RST graph information, establishing a benchmark for future graph-based summarization models. This secondary dataset posed multiple challenges, revealing both the merits and limitations of our models.

Paper Structure

This paper contains 15 sections, 6 equations, 5 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. Proposed model
  4. Encoder
  5. Decoder
  6. Fine-tuning and experimental setup
  7. Evaluation
  8. Results
  9. Discussions
  10. Conclusion and future directions
  11. GNN main components
  12. Additional evaluation
  13. Dataset generation
  14. Architectural details for reproducibility
  15. Assessment of MLP-GAT performance gap

Figures (5)

  • Figure 1: Proposed model overview and its MLP-based variation.
  • Figure 2: Message aggregation and update in GNNs for a single node from its local neighborhood.
  • Figure 3: XSum data processing pipeline.
  • Figure 4: Details of GNN and classification modules.
  • Figure 5: RST and Coref graph examples. Document A is selected from the test set of our processed XSum dataset. Part B, adapted from DiscoBERT, shows Coref edges for a particular named entity.