Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Prompted Aspect Key Point Analysis for Quantitative Review Summarization

An Quang Tang, Xiuzhen Zhang, Minh Ngoc Dinh, Erik Cambria

TL;DR

This paper introduces Prompted Aspect Key Point Analysis (PAKPA), a framework that uses prompted ABSA and in-context learning to generate and quantify aspect-grounded Key Points (KPs) from reviews. By clustering comments by shared aspect and sentiment and prompting LLMs to produce concise KPs, PAKPA achieves faithful KPs with accurate prevalence estimates while reducing the need for supervised data. Evaluations on SPACE and Yelp show state-of-the-art performance in KP quality and quantification, with ablations highlighting the benefits of ABSA integration and multi-LLM configurations. The work demonstrates a practical approach to quantitative review summarization, enabling detailed, aspect-specific insights for business entities and potentially improving decision support and SEO relevance. Code and data are publicly available, promoting reproducibility and further research in this area.

Abstract

Key Point Analysis (KPA) aims for quantitative summarization that provides key points (KPs) as succinct textual summaries and quantities measuring their prevalence. KPA studies for arguments and reviews have been reported in the literature. A majority of KPA studies for reviews adopt supervised learning to extract short sentences as KPs before matching KPs to review comments for quantification of KP prevalence. Recent abstractive approaches still generate KPs based on sentences, often leading to KPs with overlapping and hallucinated opinions, and inaccurate quantification. In this paper, we propose Prompted Aspect Key Point Analysis (PAKPA) for quantitative review summarization. PAKPA employs aspect sentiment analysis and prompted in-context learning with Large Language Models (LLMs) to generate and quantify KPs grounded in aspects for business entities, which achieves faithful KPs with accurate quantification, and removes the need for large amounts of annotated data for supervised training. Experiments on the popular review dataset Yelp and the aspect-oriented review summarization dataset SPACE show that our framework achieves state-of-the-art performance. Source code and data are available at: https://github.com/antangrocket1312/PAKPA

Prompted Aspect Key Point Analysis for Quantitative Review Summarization

TL;DR

This paper introduces Prompted Aspect Key Point Analysis (PAKPA), a framework that uses prompted ABSA and in-context learning to generate and quantify aspect-grounded Key Points (KPs) from reviews. By clustering comments by shared aspect and sentiment and prompting LLMs to produce concise KPs, PAKPA achieves faithful KPs with accurate prevalence estimates while reducing the need for supervised data. Evaluations on SPACE and Yelp show state-of-the-art performance in KP quality and quantification, with ablations highlighting the benefits of ABSA integration and multi-LLM configurations. The work demonstrates a practical approach to quantitative review summarization, enabling detailed, aspect-specific insights for business entities and potentially improving decision support and SEO relevance. Code and data are publicly available, promoting reproducibility and further research in this area.

Abstract

Key Point Analysis (KPA) aims for quantitative summarization that provides key points (KPs) as succinct textual summaries and quantities measuring their prevalence. KPA studies for arguments and reviews have been reported in the literature. A majority of KPA studies for reviews adopt supervised learning to extract short sentences as KPs before matching KPs to review comments for quantification of KP prevalence. Recent abstractive approaches still generate KPs based on sentences, often leading to KPs with overlapping and hallucinated opinions, and inaccurate quantification. In this paper, we propose Prompted Aspect Key Point Analysis (PAKPA) for quantitative review summarization. PAKPA employs aspect sentiment analysis and prompted in-context learning with Large Language Models (LLMs) to generate and quantify KPs grounded in aspects for business entities, which achieves faithful KPs with accurate quantification, and removes the need for large amounts of annotated data for supervised training. Experiments on the popular review dataset Yelp and the aspect-oriented review summarization dataset SPACE show that our framework achieves state-of-the-art performance. Source code and data are available at: https://github.com/antangrocket1312/PAKPA
Paper Structure (37 sections, 2 equations, 2 figures, 11 tables)

This paper contains 37 sections, 2 equations, 2 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Key Point Analysis
  4. Aspect-based Structured Summarization
  5. Text Summarization
  6. Prompted Opinion Summarization
  7. Methodology
  8. Prompted Aspect-based Sentiment Analysis of Comments
  9. Aspect Sentiment-based Comment Clustering
  10. Prompted Aspect-oriented KP Generation
  11. Experiments
  12. Implementation Details and Baselines
  13. Extractive KPA:
  14. Abstractive KPA:
  15. Prompted Opinion Summarization:
  16. ...and 22 more sections

Figures (2)

  • Figure 1: The PAKPA framework
  • Figure 2: Bradley Terry scores of comparative human evaluation of different KPA frameworks on 7 dimensions in assessing how well they summarize the corpus (\ref{['fig:a']}) and provide KPs for reviews (\ref{['fig:b']}).