MGFRec: Towards Reinforced Reasoning Recommendation with Multiple Groundings and Feedback

TL;DR

MGFRec bridges the gap between language-space reasoning and actual item-space grounding by introducing multiple grounding rounds and a user-agent feedback loop within a reinforcement-learning framework. By training with Group Relative Policy Optimization and a final ranking head, it achieves improved recommendation performance across three Amazon datasets, and analyses show that iterative grounding reduces the search space and helps uncover less popular items. The work underscores the importance of grounding in the actual item space for faithful, actionable recommendations and demonstrates practical gains through comprehensive ablations and hyperparameter studies. This approach lays groundwork for more interactive, item-aware reasoning in recommender systems and opens avenues for richer grounding attributes and more realistic user-model feedback in conversational settings.

Abstract

The powerful reasoning and generative capabilities of large language models (LLMs) have inspired researchers to apply them to reasoning-based recommendation tasks, which require in-depth reasoning about user interests and the generation of recommended items. However, previous reasoning-based recommendation methods have typically performed inference within the language space alone, without incorporating the actual item space. This has led to over-interpreting user interests and deviating from real items. Towards this research gap, we propose performing multiple rounds of grounding during inference to help the LLM better understand the actual item space, which could ensure that its reasoning remains aligned with real items. Furthermore, we introduce a user agent that provides feedback during each grounding step, enabling the LLM to better recognize and adapt to user interests. Comprehensive experiments conducted on three Amazon review datasets demonstrate the effectiveness of incorporating multiple groundings and feedback. These findings underscore the critical importance of reasoning within the actual item space, rather than being confined to the language space, for recommendation tasks.

Figures (8)

• Figure 1: Comparison of reasoning in the language space versus the actual item space. The language space includes all possible constructs, which may lead to recommendations deviating from the target items. In contrast, repeated grounding in the actual item space helps steer the inference towards the target item.
• Figure 2: Framework of MGFRec. For each input, MGFRec operates in multiple iterative cycles: ① The recommendation agent begins by performing reasoning, followed by grounding operations over the item space. ② The grounding operation returns a list of relevant items to both the recommendation agent and the user agent. ③ The user agent provides feedback to the recommendation agent. This loop continues until the recommendation agent determines that no further information is needed and outputs the final recommendation result.
• Figure 3: Average sample difficulty on the Movies validation set under different grounding frequencies. The difficulty is defined as the inverse of the popularity of the ground truth item for each sample.
• Figure 4: The example of how the recommendation agent finds the correct item through multiple groundings and feedback.
• Figure 5: Average rank of ground truth items under different maximum grounding numbers on the Movies validation set.