Unveiling Dual Quality in Product Reviews: An NLP-Based Approach

TL;DR

This study tackles the dual quality problem in product reviews using NLP by building a Polish dataset of 1,957 reviews (including 540 dual-quality cases) and evaluating diverse models from SetFit to large language models. It demonstrates that language-specific transformer encoders can rival or exceed LLMs, and that prompt-based improvements for LLMs are not guaranteed. The work provides a practical deployment path for a Polish consumer-protection agency, with multilingual transfer assessed and guidance on model selection and integration. Overall, it offers a complete workflow from data collection and labeling to model evaluation, robustness checks, and deployment considerations with clear regulatory relevance.

Abstract

Consumers often face inconsistent product quality, particularly when identical products vary between markets, a situation known as the dual quality problem. To identify and address this issue, automated techniques are needed. This paper explores how natural language processing (NLP) can aid in detecting such discrepancies and presents the full process of developing a solution. First, we describe in detail the creation of a new Polish-language dataset with 1,957 reviews, 540 highlighting dual quality issues. We then discuss experiments with various approaches like SetFit with sentence-transformers, transformer-based encoders, and LLMs, including error analysis and robustness verification. Additionally, we evaluate multilingual transfer using a subset of opinions in English, French, and German. The paper concludes with insights on deployment and practical applications.

Figures (8)

• Figure 1: Illustration of the NLP-based workflow for recognizing dual quality consumer reviews. The dual quality detection system flags reviews for potential issues when a consumer explicitly notes a difference between product versions from different markets. This illustration exemplifies the process with a Polish consumer assessing products from Polish and German markets; the reviews shown are English translations of the original Polish texts for clarity and wider accessibility.
• Figure 2: Charts illustrating the distribution of product categories across various types of reviews.
• Figure 3: Confusion matrices aggregated from five experiments for selected models.
• Figure 4: Diagram showing the process of preparing DQ and multilingual datasets.
• Figure 5: Taxonomy of different product or service issues recognized in reviews.