Modeling Unified Semantic Discourse Structure for High-quality Headline Generation

TL;DR

This work proposes using a unified semantic discourse structure (S3) to represent document semantics, achieved by combining document-level rhetorical structure theory (RST) trees with sentence-level abstract meaning representation (AMR) graphs to construct S3 graphs.

Abstract

Headline generation aims to summarize a long document with a short, catchy title that reflects the main idea. This requires accurately capturing the core document semantics, which is challenging due to the lengthy and background information-rich na ture of the texts. In this work, We propose using a unified semantic discourse structure (S3) to represent document semantics, achieved by combining document-level rhetorical structure theory (RST) trees with sentence-level abstract meaning representation (AMR) graphs to construct S3 graphs. The hierarchical composition of sentence, clause, and word intrinsically characterizes the semantic meaning of the overall document. We then develop a headline generation framework, in which the S3 graphs are encoded as contextual features. To consolidate the efficacy of S3 graphs, we further devise a hierarchical structure pruning mechanism to dynamically screen the redundant and nonessential nodes within the graph. Experimental results on two headline generation datasets demonstrate that our method outperforms existing state-of-art methods consistently. Our work can be instructive for a broad range of document modeling tasks, more than headline or summarization generation.

Figures (8)

• Figure 1: An example of news and corresponding headline. The news is segmented into sentences. We mark the decisive sentences in red color, where the keywords used in the headline are highlighted in blue color.
• Figure 2: Illustration of our proposed unified semantic discourse structure (S$^3$) of the corresponding document (left), which is composed of the document-level RST structure and sentence-level AMR structure. In the RST structure, the non-leaf text-span nodes are virtual nodes, while the terminal EDU nodes are sentences or clauses. In the AMR structure, dummy nodes represent the virtual concepts, and word nodes connect to the corresponding EDU node via the RST-AMR relation.
• Figure 3: The procedure of the S$^3$ graph construction.
• Figure 4: Overview of our framework. A PLM first encodes the input document into representations. Then, a GAT model encodes the discourse structure S$^3$ of the document for structural feature modeling. Thereafter, the proposed punning mechanism dynamically filers those less-informative structures within S$^3$ for discourse feature refining. Finally, the resulting contextual features are integrated into the PLM decoder for headline generation.
• Figure 5: Illustration of the pruning mechanism.