LLM-Powered User Simulator for Recommender System

TL;DR

This work tackles the challenge of building high-fidelity, transparent user simulators for RL-based recommender systems by introducing an LLM-powered simulator that explicitly models user preferences through objective and subjective item descriptions. It combines two logical models (keywords matching and similarity-based embeddings) with a pretrained SASRec statistical model into an ensemble, all integrated within an MDP framework for RL training. The methodology emphasizes interpretability, reliability, and efficiency by distilling LLM insights into concise keywords and leveraging a consensus-based reward. Empirical evaluation across five diverse datasets demonstrates improved fidelity, stability, and competitiveness against existing simulators, highlighting the approach's potential to accelerate RL-based recommender development while preserving user privacy.

Abstract

User simulators can rapidly generate a large volume of timely user behavior data, providing a testing platform for reinforcement learning-based recommender systems, thus accelerating their iteration and optimization. However, prevalent user simulators generally suffer from significant limitations, including the opacity of user preference modeling and the incapability of evaluating simulation accuracy. In this paper, we introduce an LLM-powered user simulator to simulate user engagement with items in an explicit manner, thereby enhancing the efficiency and effectiveness of reinforcement learning-based recommender systems training. Specifically, we identify the explicit logic of user preferences, leverage LLMs to analyze item characteristics and distill user sentiments, and design a logical model to imitate real human engagement. By integrating a statistical model, we further enhance the reliability of the simulation, proposing an ensemble model that synergizes logical and statistical insights for user interaction simulations. Capitalizing on the extensive knowledge and semantic generation capabilities of LLMs, our user simulator faithfully emulates user behaviors and preferences, yielding high-fidelity training data that enrich the training of recommendation algorithms. We establish quantifying and qualifying experiments on five datasets to validate the simulator's effectiveness and stability across various recommendation scenarios.

Figures (2)

• Figure 1: The user interaction logic for recommended items includes two problems: understanding what it is and how it is, given the historical interactions.
• Figure 2: User interaction simulation pipeline. We use an LLM to generate preference keywords for items. For a candidate item $i_c$, we separate user historical interactions $h$ into positive and item sets $I_{pos}$ and $I_{neg}$, and apply three base models, i.e., two logical models $f_{mat}$, $f_{sim}$, and one statistical model $f_{sta}$, to evaluate $i_c$'s pros and cons, i.e.,$D_{pos}^{i_c}$ and $D_{neg}^{i_c}$, against the historical item pros and cons. We ensemble the three base models' output as the final interaction inference $y_c$.