Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SimAug: Enhancing Recommendation with Pretrained Language Models for Dense and Balanced Data Augmentation

Yuying Zhao, Xiaodong Yang, Huiyuan Chen, Xiran Fan, Yu Wang, Yiwei Cai, Tyler Derr

TL;DR

Sparse and biased interaction data hinder recommender performance. SimAug leverages PLM-derived textual embeddings in a lightweight pre-processing step to augment interactions, focusing on inactive users and unpopular items. Across nine datasets, SimAug improves utility and fairness, with item-based augmentation and compact PLMs performing well. The approach offers a plug-and-play solution that densifies and balances data without modifying downstream models, enabling fairer recommendations in real-world systems.

Abstract

Deep Neural Networks (DNNs) are extensively used in collaborative filtering due to their impressive effectiveness. These systems depend on interaction data to learn user and item embeddings that are crucial for recommendations. However, the data often suffers from sparsity and imbalance issues: limited observations of user-item interactions can result in sub-optimal performance, and a predominance of interactions with popular items may introduce recommendation bias. To address these challenges, we employ Pretrained Language Models (PLMs) to enhance the interaction data with textual information, leading to a denser and more balanced dataset. Specifically, we propose a simple yet effective data augmentation method (SimAug) based on the textual similarity from PLMs, which can be seamlessly integrated to any systems as a lightweight, plug-and-play component in the pre-processing stage. Our experiments across nine datasets consistently demonstrate improvements in both utility and fairness when training with the augmented data generated by SimAug. The code is available at https://github.com/YuyingZhao/SimAug.

SimAug: Enhancing Recommendation with Pretrained Language Models for Dense and Balanced Data Augmentation

TL;DR

Sparse and biased interaction data hinder recommender performance. SimAug leverages PLM-derived textual embeddings in a lightweight pre-processing step to augment interactions, focusing on inactive users and unpopular items. Across nine datasets, SimAug improves utility and fairness, with item-based augmentation and compact PLMs performing well. The approach offers a plug-and-play solution that densifies and balances data without modifying downstream models, enabling fairer recommendations in real-world systems.

Abstract

Deep Neural Networks (DNNs) are extensively used in collaborative filtering due to their impressive effectiveness. These systems depend on interaction data to learn user and item embeddings that are crucial for recommendations. However, the data often suffers from sparsity and imbalance issues: limited observations of user-item interactions can result in sub-optimal performance, and a predominance of interactions with popular items may introduce recommendation bias. To address these challenges, we employ Pretrained Language Models (PLMs) to enhance the interaction data with textual information, leading to a denser and more balanced dataset. Specifically, we propose a simple yet effective data augmentation method (SimAug) based on the textual similarity from PLMs, which can be seamlessly integrated to any systems as a lightweight, plug-and-play component in the pre-processing stage. Our experiments across nine datasets consistently demonstrate improvements in both utility and fairness when training with the augmented data generated by SimAug. The code is available at https://github.com/YuyingZhao/SimAug.
Paper Structure (29 sections, 5 figures, 5 tables, 1 algorithm)

This paper contains 29 sections, 5 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Data Sparsity and Imbalance
  3. Data Sparsity
  4. Data Imbalance
  5. SimAug: Interaction Augmentation
  6. Motivation
  7. Method
  8. Experiments
  9. Experimental Settings
  10. Datasets
  11. Recommender Backbones
  12. Pretrained Language Models
  13. Evaluation Measurements
  14. Compared Methods
  15. RQ1: Effectiveness of SimAug
  16. ...and 14 more sections

Figures (5)

  • Figure 1: Two-step SimAug framework for interaction augmentation based on Pretrained Language Models (PLMs). The knowledge from PLM is first extracted in the textual embeddings and then interactions are augmented based on the embeddings.
  • Figure 2: Investigating data sparsity and imbalance issues in real-world datasets: (A) utility and density, (B) average degree of popular and unpopular items, (C) utility performance of popular and unpopular items. Vertical dashed line in (B)(C) means that popular items have a higher score than the unpopular items.
  • Figure 3: Comparison of user and item-based interaction augmentation (A) Baby Products, (B) Office Products.
  • Figure 4: Comparison of other content-based variants.
  • Figure 5: Impact of the number of augmentation.