P^3SUM: Preserving Author's Perspective in News Summarization with Diffusion Language Models

TL;DR

This work tackles the challenge of preserving author perspectives in news summarization by showing that existing systems often drift from the original stance. It introduces $\textsc{P}^3\textsc{Sum}$, a diffusion-model-based summarizer that uses an external political-bias classifier to steer the decoding process at inference time, enabling stance preservation without retraining on new classifiers. Evaluations across CNN/DM, XSUM, and POLITICS demonstrate up to $13.7\%$ gains in perspective preservation while maintaining competitive ROUGE and abstractiveness scores. The approach highlights a crucial gap in current methods and offers a practical, modular framework for faithful, perspective-aligned summaries with broad applicability to media analysis and policy-relevant communication.

Abstract

In this work, we take a first step towards designing summarization systems that are faithful to the author's intent, not only the semantic content of the article. Focusing on a case study of preserving political perspectives in news summarization, we find that existing approaches alter the political opinions and stances of news articles in more than 50% of summaries, misrepresenting the intent and perspectives of the news authors. We thus propose P^3SUM, a diffusion model-based summarization approach controlled by political perspective classifiers. In P^3SUM, the political leaning of a generated summary is iteratively evaluated at each decoding step, and any drift from the article's original stance incurs a loss back-propagated to the embedding layers, steering the political stance of the summary at inference time. Extensive experiments on three news summarization datasets demonstrate that P^3SUM outperforms state-of-the-art summarization systems and large language models by up to 13.7% in terms of the success rate of stance preservation, with competitive performance on standard metrics of summarization quality. Our findings present a first analysis of preservation of pragmatic features in summarization, highlight the lacunae in existing summarization models -- that even state-of-the-art models often struggle to preserve author's intents -- and develop new summarization systems that are more faithful to author's perspectives.

Figures (6)

• Figure 1: Changes in political stances between the summary and the article. The political perspective classifier produces left, center, or right labels for each text sequence. Left (or Right) indicates a shift in summary stance towards left (or right) by 2 units while Lean Left (Or Lean Right) indicates a shift by 1 unit. No change indicates that there is no difference in the political leaning of the summary and the context. Our study shows that existing approaches alter the stances of news articles in more than 50% of cases across both datasets.
• Figure 2: During inference time, we iteratively refine the noisy logits and guide the perspective towards the original political stance by modular control. At each time step, we compare the stance between the current version of the summary and the given article. Then a loss will be calculated if there is any inconsistency, and the corresponding gradients will be backpropagated to steer the generation for the following steps. At training time, we add progressive noise to $\mathbf{S}_0$ and learn to predict $\mathbf{S}_0$ from each noisy $\mathbf{S}_t$.
• Figure 3: We measure models' inherent biases by averaging the shift in political stances across all center-leaning articles in POLITICS. $\textsc{P}^3\textsc{Sum}$ with explicit controllable generation has the lowest absolute bias.
• Figure 4: We show how gold summaries as in-context examples alter the perspectives and how model-generated summaries are affected accordingly. We provide chatgpt with both articles and gold summaries as in-context examples. The left-rightward shift of examples can greatly increase the possibility of similar shifts in the model-generated summaries.
• Figure 5: We observe how our model behaves if the total diffusion steps change from $1000$ to $8000$. If the number of total steps is increased beyond $1000$, a drop in the performance would be observed.