Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Polarity Calibration for Opinion Summarization

Yuanyuan Lei, Kaiqiang Song, Sangwoo Cho, Xiaoyang Wang, Ruihong Huang, Dong Yu

TL;DR

This work addresses polarity bias amplification in opinion summarization by proposing polarity calibration, a reinforcement-learning framework that aligns output polarity with input text. It combines three rewards—polarity distance, content preservation, and language naturality—to balance bias mitigation with semantic fidelity and fluent language. A base summarizer is first trained supervisedly, then refined with policy-gradient optimization guided by the rewards, yielding the PoCa model. Empirical results on AmaSum and NeuS show reduced polarity mismatch and maintained Rouge scores, with both automatic and human evaluations supporting improved polarity alignment and quality. The approach offers a principled way to faithfully represent both majority and minority opinions in subjective summaries, with practical impact for consumer reviews and political discourse analysis.

Abstract

Opinion summarization is automatically generating summaries from a variety of subjective information, such as product reviews or political opinions. The challenge of opinions summarization lies in presenting divergent or even conflicting opinions. We conduct an analysis of previous summarization models, which reveals their inclination to amplify the polarity bias, emphasizing the majority opinions while ignoring the minority opinions. To address this issue and make the summarizer express both sides of opinions, we introduce the concept of polarity calibration, which aims to align the polarity of output summary with that of input text. Specifically, we develop a reinforcement training approach for polarity calibration. This approach feeds the polarity distance between output summary and input text as reward into the summarizer, and also balance polarity calibration with content preservation and language naturality. We evaluate our Polarity Calibration model (PoCa) on two types of opinions summarization tasks: summarizing product reviews and political opinions articles. Automatic and human evaluation demonstrate that our approach can mitigate the polarity mismatch between output summary and input text, as well as maintain the content semantic and language quality.

Polarity Calibration for Opinion Summarization

TL;DR

This work addresses polarity bias amplification in opinion summarization by proposing polarity calibration, a reinforcement-learning framework that aligns output polarity with input text. It combines three rewards—polarity distance, content preservation, and language naturality—to balance bias mitigation with semantic fidelity and fluent language. A base summarizer is first trained supervisedly, then refined with policy-gradient optimization guided by the rewards, yielding the PoCa model. Empirical results on AmaSum and NeuS show reduced polarity mismatch and maintained Rouge scores, with both automatic and human evaluations supporting improved polarity alignment and quality. The approach offers a principled way to faithfully represent both majority and minority opinions in subjective summaries, with practical impact for consumer reviews and political discourse analysis.

Abstract

Opinion summarization is automatically generating summaries from a variety of subjective information, such as product reviews or political opinions. The challenge of opinions summarization lies in presenting divergent or even conflicting opinions. We conduct an analysis of previous summarization models, which reveals their inclination to amplify the polarity bias, emphasizing the majority opinions while ignoring the minority opinions. To address this issue and make the summarizer express both sides of opinions, we introduce the concept of polarity calibration, which aims to align the polarity of output summary with that of input text. Specifically, we develop a reinforcement training approach for polarity calibration. This approach feeds the polarity distance between output summary and input text as reward into the summarizer, and also balance polarity calibration with content preservation and language naturality. We evaluate our Polarity Calibration model (PoCa) on two types of opinions summarization tasks: summarizing product reviews and political opinions articles. Automatic and human evaluation demonstrate that our approach can mitigate the polarity mismatch between output summary and input text, as well as maintain the content semantic and language quality.
Paper Structure (24 sections, 7 equations, 5 figures, 4 tables)

This paper contains 24 sections, 7 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Polarity Bias Amplification
  3. Polarity Calibration
  4. Supervised Training
  5. Reinforcement Training
  6. Reward Models
  7. Polarity Distance Reward
  8. Content Preservation Reward
  9. Language Naturality Reward
  10. Experiments
  11. Datasets
  12. Baselines
  13. Automatic Evaluation
  14. Ablation Study
  15. Human Evaluation
  16. ...and 9 more sections

Figures (5)

  • Figure 1: The x-axis represents input text polarity score, and the y-axis represents output summary polarity score. The model can amplify the polarity bias, by presenting the majority opinions while ignoring the minority opinions.
  • Figure 2: An illustration of polarity calibration with reinforcement learning.
  • Figure 3: The qualitative analysis of generated summaries from base summarizer and calibrated summarizer.
  • Figure 4: The visualization analysis of generated summaries from various models. The x-axis is input text polarity score, and the y-axis is output summary polarity score. The ideal polarity calibration lies on the green y=x line.
  • Figure 5: The qualitative analysis of generated summaries from various models.