Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RLPO: Residual Listwise Preference Optimization for Long-Context Review Ranking

Hao Jiang, Zhi Yang, Annan Wang, Yichi Zhang, Weisi Lin

TL;DR

This work addresses the challenge of long-context review ranking by bridging efficient pointwise scoring with global list-aware correction. It introduces RLPO, a residual listwise optimization framework that uses a calibrated pointwise LLM scorer followed by a lightweight representation-level residual head to adjust rankings based on the candidate list, avoiding expensive token-level listwise prompting. A large-scale, human-verified Amazon Reviews 2023-based benchmark is constructed to enable robust evaluation of long-context ranking methods. Experiments show RLPO achieves state-of-the-art NDCG across multiple domains, remains stable as the candidate list scales to 50, and transfers effectively across domains, highlighting its practical impact for scalable, context-rich review ranking in real systems.

Abstract

Review ranking is pivotal in e-commerce for prioritizing diagnostic and authentic feedback from the deluge of user-generated content. While large language models have improved semantic assessment, existing ranking paradigms face a persistent trade-off in long-context settings. Pointwise scoring is efficient but often fails to account for list-level interactions, leading to miscalibrated top-$k$ rankings. Listwise approaches can leverage global context, yet they are computationally expensive and become unstable as candidate lists grow. To address this, we propose Residual Listwise Preference Optimization (RLPO), which formulates ranking as listwise representation-level residual correction over a strong pointwise LLM scorer. RLPO first produces calibrated pointwise scores and item representations, then applies a lightweight encoder over the representations to predict listwise score residuals, avoiding full token-level listwise processing. We also introduce a large-scale benchmark for long-context review ranking with human verification. Experiments show RLPO improves NDCG@k over strong pointwise and listwise baselines and remains robust as list length increases.

RLPO: Residual Listwise Preference Optimization for Long-Context Review Ranking

TL;DR

This work addresses the challenge of long-context review ranking by bridging efficient pointwise scoring with global list-aware correction. It introduces RLPO, a residual listwise optimization framework that uses a calibrated pointwise LLM scorer followed by a lightweight representation-level residual head to adjust rankings based on the candidate list, avoiding expensive token-level listwise prompting. A large-scale, human-verified Amazon Reviews 2023-based benchmark is constructed to enable robust evaluation of long-context ranking methods. Experiments show RLPO achieves state-of-the-art NDCG across multiple domains, remains stable as the candidate list scales to 50, and transfers effectively across domains, highlighting its practical impact for scalable, context-rich review ranking in real systems.

Abstract

Review ranking is pivotal in e-commerce for prioritizing diagnostic and authentic feedback from the deluge of user-generated content. While large language models have improved semantic assessment, existing ranking paradigms face a persistent trade-off in long-context settings. Pointwise scoring is efficient but often fails to account for list-level interactions, leading to miscalibrated top- rankings. Listwise approaches can leverage global context, yet they are computationally expensive and become unstable as candidate lists grow. To address this, we propose Residual Listwise Preference Optimization (RLPO), which formulates ranking as listwise representation-level residual correction over a strong pointwise LLM scorer. RLPO first produces calibrated pointwise scores and item representations, then applies a lightweight encoder over the representations to predict listwise score residuals, avoiding full token-level listwise processing. We also introduce a large-scale benchmark for long-context review ranking with human verification. Experiments show RLPO improves NDCG@k over strong pointwise and listwise baselines and remains robust as list length increases.
Paper Structure (28 sections, 11 equations, 3 figures, 5 tables)

This paper contains 28 sections, 11 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Unsupervised and Encoder-Based Ranking
  4. Pointwise LLM Ranking
  5. Pairwise and Setwise LLM Ranking
  6. Listwise LLM Ranking
  7. Review Ranking Benchmark
  8. Data Collection and Annotation
  9. Human Verification
  10. Listwise Ranking Consistency.
  11. Pairwise Accuracy Check.
  12. RLPO Framework
  13. The RLPO Architecture
  14. Phase 1: Semantic Score Generation and Encoding (Pointwise)
  15. Phase 2: Residual Contextualization (Listwise)
  16. ...and 13 more sections

Figures (3)

  • Figure 1: Illustration of review ranking for the product "Retro Lazy Suede Half Slippers." The left panel shows randomly ordered reviews, while the right panel shows the ranked list. The top-ranked review is more informative and detailed than the second, whereas the bottom review is ranked last because it is unrelated to the product.The full review texts are provided in Appendix \ref{['app2']}.
  • Figure 2: The overall framework of RLPO
  • Figure 3: Performance comparison of human annotators and GPT-4o against the dataset ground truth. The radar chart depicts agreement across six metrics (e.g., NDCG, Spearman), highlighting the high quality and consistency of the generated labels.