ReFeed: Multi-dimensional Summarization Refinement with Reflective Reasoning on Feedback

TL;DR

The paper addresses the challenge of refining text summaries across multiple dimensions, notably faithfulness, completeness, and conciseness. It introduces ReFeed, a refinement pipeline that leverages reflective reasoning on feedback to balance trade-offs and resist ordering biases and noisy feedback. To train such capabilities, the authors create SumFeed-CoT, a large Long-CoT dataset with goal-guided reasoning and quality control, enabling a lightweight student model to perform multi-dimensional refinement. Empirical results show that ReFeed achieves superior balance across dimensions, robustness to feedback quality and order, and competitive performance with faster inference, compared to single-dimension baselines and existing pipelines. The work also releases the SumFeed-CoT dataset and provides extensive analysis of different refinement strategies, highlighting the importance of explicit goals and well-structured guidelines for effective reasoning.

Abstract

Summarization refinement faces challenges when extending to multi-dimension. In this paper, we introduce ReFeed, a powerful summarization refinement pipeline that enhances multiple dimensions through reflective reasoning on feedback. To achieve this, we release SumFeed-CoT, a large-scale Long-CoT-based dataset optimized for training a lightweight model with reflective reasoning. Our experiments reveal how the number of dimensions, feedback exposure, and reasoning policy influence refinement performance, highlighting reflective reasoning and simultaneously addressing multiple feedback is crucial to mitigate trade-off between dimensions. Furthermore, ReFeed is robust to noisy feedback and feedback order. Lastly, our finding emphasizes that creating data with a proper goal and guideline constitutes a fundamental pillar of effective reasoning. The dataset and model will be released.

Figures (1)

• Figure 1: Overview of ReFeed and data construction of SumFeed-CoT, a Long-CoT-based dataset for training ReFeed. Faith., Comp., and Conc. denote faithfulness, completeness, and conciseness.