Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting

TL;DR

This work tackles long document summarization by marrying extractive and abstractive paradigms in a two-stage, coarse-to-fine architecture. An extractor selects salient sentences using a hierarchical representation and a pointer-network, while an abstractor rewrites these sentences with an encoder-aligner-decoder that includes a copy mechanism. The extractor is trained via sentence-level reinforcement learning with an Advantage Actor-Critic critic, bridging non-differentiable selection to fluent abstractive rewriting; a stop action dynamically determines how many sentences to extract. Empirically, the model achieves state-of-the-art ROUGE and METEOR on CNN/Daily Mail, generalizes to DUC-2002, and delivers substantial speedups in training and decoding (10–20x inference, ~4x training), with additional gains from a reranking step that reduces cross-sentence redundancy. The results demonstrate the practicality and effectiveness of a modular, RL-guided extractive-abstractive pipeline for scalable, fluent summarization of long texts.

Abstract

Inspired by how humans summarize long documents, we propose an accurate and fast summarization model that first selects salient sentences and then rewrites them abstractively (i.e., compresses and paraphrases) to generate a concise overall summary. We use a novel sentence-level policy gradient method to bridge the non-differentiable computation between these two neural networks in a hierarchical way, while maintaining language fluency. Empirically, we achieve the new state-of-the-art on all metrics (including human evaluation) on the CNN/Daily Mail dataset, as well as significantly higher abstractiveness scores. Moreover, by first operating at the sentence-level and then the word-level, we enable parallel decoding of our neural generative model that results in substantially faster (10-20x) inference speed as well as 4x faster training convergence than previous long-paragraph encoder-decoder models. We also demonstrate the generalization of our model on the test-only DUC-2002 dataset, where we achieve higher scores than a state-of-the-art model.

Figures (5)

• Figure 1: Our extractor agent: the convolutional encoder computes representation $r_j$ for each sentence. The RNN encoder (blue) computes context-aware representation $h_j$ and then the RNN decoder (green) selects sentence $j_t$ at time step $t$. With $j_t$ selected, $h_{j_t}$ will be fed into the decoder at time $t+1$.
• Figure 2: Reinforced training of the extractor (for one extraction step) and its interaction with the abstractor. For simplicity, the critic network is not shown. Note that all $d$'s and $s_t$ are raw sentences, not vector representations.
• Figure 3: RL learning curve.
• Figure 4: Example from the dataset showing the generated summary of our best models. The colored (marked) sentences correspond to our extractor's sentence selection. The listed ROUGE scores are computed for this specific example.
• Figure 5: Example from the dataset showing the generated summary of our best models. The colored (marked) sentences correspond to our extractor's sentence selection. The listed ROUGE scores are computed for this specific example.