Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Taming the Long Tail: Efficient Item-wise Sharpness-Aware Minimization for LLM-based Recommender Systems

Jiaming Zhang, Yuyuan Li, Xiaohua Feng, Li Zhang, Longfei Li, Jun Zhou, Chaochao Chen

Abstract

Large Language Model-based Recommender Systems (LRSs) have recently emerged as a new paradigm in sequential recommendation by directly adopting LLMs as backbones. While LRSs demonstrate strong knowledge utilization and instruction-following abilities, they have not been systematically studied under the long-standing long-tail problem. In this paper, we conduct an empirical study and reveal that LRSs face two distinct types of long-tail: i) prior long-tail, inherited implicitly from pretraining corpora, and ii) data long-tail, originating from skewed recommendation datasets. Our analysis shows that both contribute to the performance disparity between head and tail items, with the intersection of the two heads exhibiting an even stronger head effect. Nevertheless, the overall performance distribution in LRSs, especially on the tail, remains dominated by the data long-tail. To address this challenge, we propose Efficient Item-wise Sharpness-Aware Minimization (EISAM), a novel optimization framework that improves tail-item performance by adaptively regularizing the loss landscape at the item level. EISAM introduces an efficient penalty design that captures fine-grained item-specific sharpness while maintaining computational scalability for LLMs. In addition, we derive a generalization bound for EISAM. Our theoretical analysis shows that the bound decreases at a faster rate under our item-wise regularization, offering theoretical support for its effectiveness. Extensive experiments on three real-world datasets demonstrate that EISAM significantly boosts tail-item recommendation performance while preserving overall quality, establishing the first systematic solution to the long-tail problem in LRSs.

Taming the Long Tail: Efficient Item-wise Sharpness-Aware Minimization for LLM-based Recommender Systems

Abstract

Large Language Model-based Recommender Systems (LRSs) have recently emerged as a new paradigm in sequential recommendation by directly adopting LLMs as backbones. While LRSs demonstrate strong knowledge utilization and instruction-following abilities, they have not been systematically studied under the long-standing long-tail problem. In this paper, we conduct an empirical study and reveal that LRSs face two distinct types of long-tail: i) prior long-tail, inherited implicitly from pretraining corpora, and ii) data long-tail, originating from skewed recommendation datasets. Our analysis shows that both contribute to the performance disparity between head and tail items, with the intersection of the two heads exhibiting an even stronger head effect. Nevertheless, the overall performance distribution in LRSs, especially on the tail, remains dominated by the data long-tail. To address this challenge, we propose Efficient Item-wise Sharpness-Aware Minimization (EISAM), a novel optimization framework that improves tail-item performance by adaptively regularizing the loss landscape at the item level. EISAM introduces an efficient penalty design that captures fine-grained item-specific sharpness while maintaining computational scalability for LLMs. In addition, we derive a generalization bound for EISAM. Our theoretical analysis shows that the bound decreases at a faster rate under our item-wise regularization, offering theoretical support for its effectiveness. Extensive experiments on three real-world datasets demonstrate that EISAM significantly boosts tail-item recommendation performance while preserving overall quality, establishing the first systematic solution to the long-tail problem in LRSs.
Paper Structure (27 sections, 4 theorems, 19 equations, 4 figures, 3 tables, 1 algorithm)

This paper contains 27 sections, 4 theorems, 19 equations, 4 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLM-based Recommender Systems
  4. Long-tail in Recommender Systems
  5. Problem Setup
  6. Empirical Study
  7. Methodology
  8. Item-wise Sharpness Regularization
  9. Efficient Optimization Procedure
  10. Theoretical Analysis
  11. Notation.
  12. Experiments
  13. Experimental Settings
  14. Datastes
  15. LRS Models
  16. ...and 12 more sections

Key Result

lemma 1

Following Bernstein’s Inequality bernstein1924modification, with probability $1-\delta$ over the choice of the training set $\mathcal{S} \sim \mathcal{D}$

Figures (4)

  • Figure 1: Performance across data/prior groups.
  • Figure 2: Weight distributions of different weighting functions on the ML-1M dataset, where the x-axis denotes item indices sorted by descending frequency.
  • Figure 3: visualizations of the loss landscapes of LRS on tail items. Darker colors indicate lower loss values.
  • Figure 4: Hyperparameter sensitivity analysis.

Theorems & Definitions (4)

  • lemma 1: Concentration of weighted risks
  • lemma 2: Empirical control of weighted perturbed risks
  • lemma 3: Second-order de-noising via curvature
  • theorem 1: Generalization bound of EISAM