Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Self-EvolveRec: Self-Evolving Recommender Systems with LLM-based Directional Feedback

Sein Kim, Sangwu Park, Hongseok Kang, Wonjoong Kim, Jimin Seo, Yeonjun In, Kanghoon Yoon, Chanyoung Park

TL;DR

The paper tackles the limitations of fixed NAS search spaces and scalar evaluation in recommender systems by introducing Self-EvolveRec, an open-ended, LLM-driven code evolution framework. It establishes a directional feedback loop that combines qualitative critiques from a User Simulator with quantitative verification from a Model Diagnosis Tool, supported by a Diagnosis Tool - Model Co-Evolution strategy to adapt diagnostics as the architecture evolves. The framework demonstrates superior recommendation performance, higher user-satisfaction metrics, and richer codebase quality than NAS and prior LLM-based baselines across multiple datasets, while providing case studies showing reliable diagnosis and structured evolutionary trajectories. This approach has practical implications for industrial deployment, enabling more expressive, user-aligned, and robust recommender pipelines with open-ended design capabilities and adaptive diagnostics.

Abstract

Traditional methods for automating recommender system design, such as Neural Architecture Search (NAS), are often constrained by a fixed search space defined by human priors, limiting innovation to pre-defined operators. While recent LLM-driven code evolution frameworks shift fixed search space target to open-ended program spaces, they primarily rely on scalar metrics (e.g., NDCG, Hit Ratio) that fail to provide qualitative insights into model failures or directional guidance for improvement. To address this, we propose Self-EvolveRec, a novel framework that establishes a directional feedback loop by integrating a User Simulator for qualitative critiques and a Model Diagnosis Tool for quantitative internal verification. Furthermore, we introduce a Diagnosis Tool - Model Co-Evolution strategy to ensure that evaluation criteria dynamically adapt as the recommendation architecture evolves. Extensive experiments demonstrate that Self-EvolveRec significantly outperforms state-of-the-art NAS and LLM-driven code evolution baselines in both recommendation performance and user satisfaction. Our code is available at https://github.com/Sein-Kim/self_evolverec.

Self-EvolveRec: Self-Evolving Recommender Systems with LLM-based Directional Feedback

TL;DR

The paper tackles the limitations of fixed NAS search spaces and scalar evaluation in recommender systems by introducing Self-EvolveRec, an open-ended, LLM-driven code evolution framework. It establishes a directional feedback loop that combines qualitative critiques from a User Simulator with quantitative verification from a Model Diagnosis Tool, supported by a Diagnosis Tool - Model Co-Evolution strategy to adapt diagnostics as the architecture evolves. The framework demonstrates superior recommendation performance, higher user-satisfaction metrics, and richer codebase quality than NAS and prior LLM-based baselines across multiple datasets, while providing case studies showing reliable diagnosis and structured evolutionary trajectories. This approach has practical implications for industrial deployment, enabling more expressive, user-aligned, and robust recommender pipelines with open-ended design capabilities and adaptive diagnostics.

Abstract

Traditional methods for automating recommender system design, such as Neural Architecture Search (NAS), are often constrained by a fixed search space defined by human priors, limiting innovation to pre-defined operators. While recent LLM-driven code evolution frameworks shift fixed search space target to open-ended program spaces, they primarily rely on scalar metrics (e.g., NDCG, Hit Ratio) that fail to provide qualitative insights into model failures or directional guidance for improvement. To address this, we propose Self-EvolveRec, a novel framework that establishes a directional feedback loop by integrating a User Simulator for qualitative critiques and a Model Diagnosis Tool for quantitative internal verification. Furthermore, we introduce a Diagnosis Tool - Model Co-Evolution strategy to ensure that evaluation criteria dynamically adapt as the recommendation architecture evolves. Extensive experiments demonstrate that Self-EvolveRec significantly outperforms state-of-the-art NAS and LLM-driven code evolution baselines in both recommendation performance and user satisfaction. Our code is available at https://github.com/Sein-Kim/self_evolverec.
Paper Structure (40 sections, 9 equations, 20 figures, 9 tables)

This paper contains 40 sections, 9 equations, 20 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Definition
  4. Proposed Framework: Self-EvolveRec
  5. Directional Feedback Generation
  6. User Simulator: Qualitative Critique
  7. Model Diagnosis Tool: Quantitative Verification
  8. Evolution Pipeline
  9. Diagnosis Tool - Model Co-Evolution
  10. Experiments
  11. Performance Comparison
  12. Overall performance.
  13. User Satisfaction Analysis.
  14. Codebase Quality Evaluation.
  15. Ablation Studies
  16. ...and 25 more sections

Figures (20)

  • Figure 1: Two core mechanism of Self-EvolveRec. (a) is the overview of the Directional Feedback Generation: (a.1) is the user simulator, (a.2) is the model diagnosis tool. (b) is the Diagnosis Tool - Model Co-evolution in Self-EvolveRec.
  • Figure 2: Overall evolutionary pipeline of Self-EvolveRec.
  • Figure 3: LLM-as-a-Judge evaluation of the evolved models.
  • Figure 4: Case study on Diagnosis Tool - Model Co-Evolution on CDs dataset (Seed Recommender: SASRec).
  • Figure 5: Case study on evolutionary trajectory on CDs dataset (Seed Recommender: SASRec). (a) is comparison of evolutionary paths. Color-coded markers (e.g., Red) illustrate causal alignment between directional feedback and evolved codebase. (b) is performance comparison across iterations.
  • ...and 15 more figures